How Was Your Day? Evaluating a Conversational Companion
Webb, N. (2013). How Was Your Day? Evaluating a Conversational Companion. IEEE Transactions on Affective Computing, , (), .
Abstract—The “How Was Your Day” (HWYD) Companion is an embodied conversational agent that can discuss work-related
issues, entering free-form dialogues that lack any clearly defined tasks and goals. The development of this type of Companion
technology requires new models of evaluation. Here, we describe a paradigm and methodology for evaluating the main aspects of
such functionality in conjunction with overall system behaviour, with respect to three parameters: functional ability (i.e., does it do
the ‘right’ thing), content (i.e., does it respond appropriately to the semantic context), and emotional behaviour (i.e., given the
emotional input from the user, does it respond in an emotionally appropriate way).
We demonstrate the functionality of our evaluation paradigm as a method for both grading current system performance, and
targeting areas for particular performance review. We show correlation between, for example, ASR performance and overall
system performance (as is expected in systems of this type) but beyond this, we show where individual utterances or responses,
which are indicated as positive or negative, show an immediate response from the user, and demonstrate how our combination
evaluation approach highlights issues (both positive and negative) in the Companion system’s interaction behaviour
+44 131 455 2736
Senior Research Fellow
+44 131 455 2761
Areas of Expertise
See all themes
Expressive interaction describes the range of new forms of human-computer interaction that are made available through technologies based on speech, touch and gesture. The opportunities presented by these developments promise to be far-reaching, enabling new forms of interaction with information...