1.1 Expectation-based action selection

In a simplified setting, the interaction process consists of four phases [11]: 1) expectation, in which the user estimates the consequences of the intended action; 2) action; 3) observation, in which the user discovers the actual consequences of the action; 4) learning, in which the user assesses the differences between expectation and observation.

Expectations can be updated by learning. When what is observed is unexpected, the user tends to re-explore the product and to gather information about what caused the unexpected result [11]. Each time the user learns about the product, the expectations are updated, which in turn may bring changes to future action selection strategies. Learning is an on-going process. In reality, in user–product interaction, the user can only interpret observations and create strategy accordingly but can never acquire all the information available in principle [11]. This is because the interface is not transparent, which inhibits the user from knowing what is happening inside the machine. Nevertheless, users can explore a product until they gather sufficient knowledge to establish habits that they retain. For instance, one can use a computer without necessarily knowing how the hardware works, owing to the graphic user interface.

Regarding user motivation in virtual assistant interactions, a crucial aspect is the expectation of the assistant’s capability. The users might ask themselves: “How smart is it, after all?” or “Where is its limit?” Accordingly, when the user has an intention (“I need to book a flight”), expectation of interaction result is formed before asking. If the user holds an expectation (of the assistant’s capability) lower than what the intention would require, they are unlikely to initiate an interaction for booking (“I’d rather book it from my smartphone app!”). In a recent survey-based research, many of the interviewees mentioned that sometimes the use of virtual assistants made reaching their goal slower [7]. The user motivation (to interact) is clearly vulnerable to low-capability expectation with respect to the difficulty of the intended tasks.

While the aforementioned situations involved only extrinsic motivation (pragmatic intentions), we argue that intrinsic motivation would be damaged equally. When the user’s intention is to seek enjoyment or reduce boredom, which by definition is intrinsically motivated, such motivation can also be damaged if the capability expectation in terms of entertainment capability is low. Furthermore, in such a situation, the user just “won’t bother” to interact for fun. For instance, the user would not choose to chat with the assistant to pass time, if they believed that the assistant was not smart enough to adhere to the topic.

Unlike video games, which serve solely entertainment purposes, virtual assistant carries both pragmatic and entertainment functions. It is likely that capability expectations of a virtual assistant are formed throughout daily usage, where expectations of pragmatic and entertainment capabilities are not independent of each other. In other words, both intrinsic and non-intrinsic motivation are affected by the capability expectation. This statement is based on a mapping between motivation type and interaction type. Interactions that are enjoyment-oriented are intrinsically motivated (by the definition of intrinsic motivation [4]). Interactions whose goal is to complete tasks or raise productivity are extrinsically motivated (by the definition of extrinsic motivation [4]), hence non-intrinsically motivated. Interactions whose goal is to gather information or to learn are driven by epistemic motivation, and therefore also non-intrinsically motivated [12].

We have established that motivation and action selection are affected by expectation; thus, the next question is: what factors affect the forming of expectations and the learning process?

1.2 Expectation and uncertainty in virtual assistant interactions

In a previous study addressing expectation-based sensory perception (of physical property such as weight, denoted by θ in Eq 1), a computational model was proposed and validated, explaining how human perception (a posteriori distribution, ) follows certain probability distributions. As shown in Eq 1, perception is formed based on expectation (a priori distribution, P(θ)) and information gained through observations (the likelihood, P(R|θ) where R is the firing rate distribution of the sensory stimulus θ). The bias between the a priori and the a posteriori (the bias is termed “expectation effect”) is a function of uncertainty, prediction error, and noise [13]. According to this model, the estimation follows the Bayesian estimator [13–15]: (1) In Eq 1, E[P(θ)] and V[P(θ)] represent the mean and the variation of the distribution P(θ), respectively. Although this model was proposed to explain sensory perception, the knowledge is transferable to virtual assistant interactions. Here, we borrow Yanagisawa’s model [13] to understand expectation and perception of the virtual assistant’s capability. In Fig 1, the horizontal axis represents the assistant’s capability. The observation mean indicates that the assistant is capable of tasks easier than the difficulty represented by the mean; meanwhile, it indicates that the assistant is incapable of tasks to the right of the mean difficulty. Similarly, the prior expectation mean indicates the expected capability before observation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Expectation and perception of the virtual assistant’s capability.

Unexpected observations update the user’s expectation of the assistant’s capability.

https://doi.org/10.1371/journal.pone.0250326.g001

Following the Bayesian model [13], if the expectation variation and observation variation are the same, the perception of capability should be in the middle of expectation mean and observation mean. When two variations are unequal, the perception should always be closer to the mean with smaller variation. In the case illustrated by Fig 1, the prior expectation has smaller variation; therefore, the perception is closer to expectation.

The prior variation indicates the prediction uncertainty: in the Bayesian estimation, a small variation in prior expectation indicates small uncertainty, and a large variation in prior expectation indicates large uncertainty in the prediction. Visually, the larger the uncertainty, the flatter the distribution curve. Likewise, the observation variation indicates the noise during sensory perception.

The capability expectation is learned through previous interactions. The user compares the assistant’s expected capability against the expected “task difficulty”, i.e., the minimal capability required to cope with the user’s intention. If the assistant fails “rather easy” tasks, which are expected to be within the capability, the observation will be on the left of the expectation mean; hence, the posterior perception of capability will shift to the left. (“The assistant is not as capable as I thought.”) Conversely, if the user observes that the assistant copes with difficult tasks, which are expected to be beyond its capability, the posterior perception of capability will shift to the right. (“The assistant is more capable than I thought.”) Coping with tasks that are expected to be within the capability or failing tasks that are expected to be beyond the capability does not affect the mean of posterior.

In the long run, posterior perceptions update the user’s experience, thus affecting the prior expectation in the future. In virtual assistant interaction, inconsistent performance by the virtual assistant gives rise to uncertainty. For instance, consider the following scenario: a virtual assistant complies with a task, and moments later, it fails to comply with an apparently similar task. Observing such inconsistent performance results in a flattened posterior distribution compared with when uncertainty is small. In other words, the user becomes uncertain about the virtual assistant’s capability.

Thus, we can conclude that the virtual assistant’s capability is perceived in consideration of “task difficulty” and that uncertainty weakens the confidence that the user places in their expectation. The relationship of expectation and uncertainty with intrinsic motivation is discussed in the following paragraphs, and a hypothesis of their effects on intrinsic motivation is proposed.

1.3 Hypothesis on expectation

From here on, we use “high” versus “low” to describe capability expectation: high expectation means the assistant is expected by the user to be able to cope with difficult tasks or satisfy complex intentions. Moreover, high versus low, as well as difficult versus easy, are discussed using relative measurement. Considering the capabilities of virtual assistants at present, a real-world example of an easy task would be “Set a 3-minute timer.” Conversely, a difficult (if not impossible) task would be “Wake me up at 4 a.m. if Team A still has a chance to advance”, when in reality, the chance is dependent on another game that ends at 3:30 a.m.

As discussed in Section 2.1, the user’s motivation to interact with a virtual assistant is limited by its expected capability. The higher the expectation, the more rational it is for the user to believe that their intention will be satisfied through interaction. The lower the expectation, the more likely the user would “not bother” to interact. Therefore, we hypothesized that high expectation has positive effects on both intrinsic and extrinsic motivation to further interact with the assistant.

1.4 Hypothesis on uncertainty

As we discussed in Section 2.2, uncertainty can be perceived through observing inconsistent performances by the assistant. From here on, we use “small” versus “large” to describe uncertainty. The uncertainty is small (if not non-existent) when the assistant’s performance is always consistent in response to the same type of task. Conversely, the uncertainty is perceived as large if the assistant copes with rather difficult tasks, and later fails at easier tasks of the same type. Similar to expectation, large versus small uncertainty is discussed using relative measurement. Another way to interpret uncertainty is to ask: “How capable is the assistant, after all?” The more difficult to judge its capability, the flatter the prior distribution (as a result of larger uncertainty). In our experiment, to introduce large uncertainty we deliberately made the virtual assistant perform in an inconsistent manner.

We hypothesized that the effect of uncertainty on motivation depends on the level of expectation. If the user holds high expectation of an assistant, the intrinsic motivation to further interact will be greater if small uncertainty has been perceived during previous interactions. If the expectation is low, we hypothesized that uncertainty would work in the opposite direction.

Fig 2 illustrates the reasoning behind the hypothesis. Assume that the “task difficulty” of an intention is indicated by the vertical dashed line. When the expectation is low, the prior mean is on the left side of the dashed line. In this situation, large uncertainty (the flatter distribution) portends higher probability than small uncertainty that the assistant can satisfy the intention. Conversely, when the expectation is high, large uncertainty portends higher probability that the assistant is unable to satisfy the intention.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2.

a. Hypothesis on uncertainty. A supposedly difficult task “intersects” with the large uncertainty prior distribution at higher probability than when uncertainty is small. When the expectation is low, it is more rational to expect an assistant exhibiting large uncertainty to perform better than it usually does. b. Hypothesis on uncertainty. A supposedly easy task “intersects” with the large uncertainty prior distribution at higher probability than when uncertainty is small. When the expectation is high, it is more rational to expect that an assistant exhibiting large uncertainty may fail to cope with easy tasks.

https://doi.org/10.1371/journal.pone.0250326.g002

To sum up, we hypothesized a positive effect of expectation on motivation, and an interaction effect of expectation and uncertainty on motivation. To verify these hypotheses, we designed a two-by-two experiment, where expectation was manipulated within group while uncertainty was manipulated between groups. In Section 3, we explain how we created small versus large uncertainty and high versus low expectation in the experimental settings. We also explain why expectation is better manipulated within group than between groups (end of Section 3.2).

2.1 Experimental design

To verify the hypotheses, we conducted an experiment using Echo Dot and Echo Plus, the smart speakers by Amazon (Fig 3). In the experiment, the virtual assistants had different wake words (names): “Echo” and “Alexa”. Participants were told that they were two inherently different assistants; no additional information, such as information regarding capability, was provided throughout the experiment (see Section 3.4 and S1 Appendix). The experiment involved two task sections, followed by a “free-choice period”, during which the participants could interact (could also decide not to interact) with the virtual assistants. While the participants believed they were asked to interact with the virtual assistants during the task sections, it was actually the experimenter covertly simulating autonomous responses using the Wizard of Oz method [16]. However, in the ensuing “free-choice period”, the Wizard of Oz method manipulation was not used, and the participant could interact with the real Amazon Echo assistants (their different wake words, “Alexa” and “Echo”, were retained; the interaction was English-only).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Smart speakers used for experiment.

Amazon Echo Plus (left) and Amazon Echo Dot (right).

https://doi.org/10.1371/journal.pone.0250326.g003

The purpose of the task sections is to shape the participant’s prior expectation and uncertainty regarding the two assistants, which is the basis of their action selections once they move on to the free-choice period. The purpose of the free-choice period is to covertly observe any spontaneous interaction initiated by the participant, which serves as the observed behavior measurement of intrinsic motivation [17].

Since we have two variables, uncertainty and expectation, we used a two-by-two design. Participants were randomly assigned to either the small-uncertainty group or to the large-uncertainty group. For each participant, there were two task sections. During each section, the participant was asked to interact with a different assistant and perform interactive tasks designated in a different task list.

The two task lists differed in the overall task difficulty. As we elaborate later in Section 3.2, we designed the tasks and their responses such that, after the two task sections, the participant should conclude that one assistant is more capable than the other. In other words, by the time the participant gets a chance to interact freely (in the free-choice period), we have constructed a high-expectation assistant and a low-expectation assistant. Lastly, when comparing the two groups, the participants in the large-uncertainty group should place less confidence in their expectations (low or high), than those in the small-uncertainty group. In the following paragraphs, we explain how uncertainty and expectation were manipulated.

2.2 Manipulation of variables

The Wizard of Oz method allowed us to manipulate the response correctness by the virtual assistants, regardless of Amazon Echo’s real capability. Responses during the task sections were audio files generated using the Alexa Developer Console prior to the data collection phase. During the task sections, we streamed the audio files via Bluetooth from the smart speaker.

Task lists and response manipulation details are shown in S1 and S2 Tables. The tasks involved “coin flip”, “dice-roll”, “get a random number”, “judge a number as odd or even”, “smart home light bulb maneuver”, “converting units”, “taking notes”, and combinations of them. These tasks were selected because: 1) they were commonly seen functions of real virtual assistants; 2) the responses are time-invariant (unlike weather inquiries) and; 3) the responses are relatively short.

In order to introduce uncertainty, we made the virtual assistants in the large-uncertainty group respond in an inconsistent manner. For instance, we made the assistant cope successfully with the task “Get a random number between 1 and 6”. Then, for the next task, “Get two random numbers between 1 and 6”, we made the assistant fail (see tasks 4 and 5 in S1A Table and their response manipulation in S2A Table). We assume that these two tasks are of equal difficulty. Therefore, we considered it “inconsistent performance”.

To differentiate high- versus low-capability expectation, we made the high-expectation assistant capable of accomplishing a list of difficult tasks, while the low-expectation assistant was made capable of easy tasks but began to struggle when the task difficulty escalated. Difficulty was added gradually: the easy task list (for the low-expectation assistant) primarily consisted of single tasks, whereas the difficult task consisted of successive tasks (do A and then do B), storing variables, calculations using variables, etc. As a recent survey-based study on voice assistant usage revealed, today’s voice assistants still cannot “remember context” or “answer several questions at once” [7].

The capability limit of the low-expectation assistant was set at “successive tasks.” For instance, the low-expectation assistant was able to “Flip a coin” and “Roll a dice” separately but could not cope with “Flip a coin and then roll a dice”. In contrast, the high-expectation assistant was able to cope with “Roll a dice and flip that many coins”, which involves calculations using variables. In the experiment, the wake words of the low- and high-expectation assistant were “Echo” and “Alexa”, respectively.

In sum, in this experiment, uncertainty was manipulated between groups, while expectation was manipulated within group. Participants from the large-uncertainty group experienced “large uncertainty–low expectation” condition and “large uncertainty–high expectation” condition. Participants from the small-uncertainty group experienced “small uncertainty–low expectation” condition and “small uncertainty–high expectation” condition.

The expectation was manipulated within group because of unwanted learning effects. If we used between-group design for expectation, the two task lists would need to have equally difficult tasks for each group. Meanwhile, the two assistants would still need to show different responses to those virtually identical tasks in order to manipulate uncertainty. If that were the case, participants would expect the same interaction outcomes before the second task section (because they can read task lists before interacting, see Section 3.4 and S1 Appendix), only to be surprised by a different capability profile from the second assistant. Eventually, this would cause participants to have different capability expectations of the two assistants after experiencing both task sections; therefore, we believed expectation must be manipulated within group.

2.3 Measurement

Intrinsic motivation was assessed by self-reported and observed behavior measurements. For self-reported measurement, we used questionnaires taken partially from the Intrinsic Motivation Inventory [18] to appraise five aspects of user attitudes toward the virtual assistant: intrinsic motivation of the user, smartness of the assistant, comprehensibility (the extent to which the virtual assistant’s thought process is understandable), trust, and human-likeness (see S3 Table). Additional questions were borrowed or modified from previous studies on motivation and technology acceptance [19,20]. For observed behavior measurement, we applied the free-choice paradigm, in which participants are left alone in the experiment room and made to believe that they are no longer under observation. During this free-choice period, participants can initiate interactions of their own decision or do nothing at all [17,21]. The number of interactions within the free-choice period (which lasts for five minutes) served as an indicator of intrinsic motivation at the behavioral level.

2.4 Implementation

The experiment was conducted indoor. The smart speakers Echo Dot and Echo Plus were placed on a table, next to a Philips Hue LED bulb that is mounted on a bulb socket (Fig 3, in the lower left corner). The experimental setting from above is shown in Fig 4.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Overhead view of experimental setting.

The participant sits 50 centimeters (19.69 inches) away from the table. Echo Dot is placed on a supporting box so that both speakers are at the same altitude.

https://doi.org/10.1371/journal.pone.0250326.g004

The overview of experimental procedures can be found in Fig 5. Instructions were given on how to interact with the virtual assistant. These instructions also presented a coherent cover story, implying that the experiment was part of a user-experience research project. The experiment consisted of two task sections, the second one was followed by a 5-muinute free-choice paradigm period. To start the free-choice period, the experimenter used the excuse of needing to print extra documents in order to justify leaving the room. Before leaving, the experimenter covertly reactivated the microphones of the smart speakers, which meant that the participants could interact with the real Echo Dot and Echo Plus during the free-choice period. Interactions during the free-choice period were recorded by the experimenter’s computer microphone. The detailed experimental procedures are documented in S1 Appendix.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Overview of experimental procedures.

Uncertainty was controlled between groups, whereas expectation was controlled within each subject.

https://doi.org/10.1371/journal.pone.0250326.g005

Thirteen students from the University of Tokyo participated in the experiment (4 female, 9 male, age range from 21 to 26, experiment was conducted in November and December 2018). One set of data had to be discarded because devices lost connection during the experiment. All participants were novices at virtual assistant usage. (We screened potential participants in this way to avoid the difficulty of quantifying different levels of prior user experience, which would introduce noise in the measurement of expectation and uncertainty.) Participants were rewarded with 1000 Japanese Yen (8.8 US Dollar) for participating. They were informed of the monetary reward when recruited and received the reward after the experiment. The study protocol was approved by the Ethics Committee of the Graduate School of Engineering, the University of Tokyo. All participants provided written informed consent prior to their participation in this study.

5.1 Manipulation of uncertainty

Consider a scenario where the virtual assistant appears unable to cope and produces an ambiguous response such as: “Sorry, I don’t know.” Since the response does not disclose causes of the failed task, it can be difficult for the user to adjust their strategies and utterances in future interactions. In some cases, the virtual assistant is able to recognize the user’s utterances, but the task is beyond its capability. An example is the task of “doing multiple tasks at once”. Alexa does not disclose its limitation to the user (cannot do successive tasks); the user has to determine it through trials. We argue that uncertainty about the (expected) capability can be reduced by disclosing causes of failed tasks, that is, by issuing an informative response instead of an ambiguous one.

In this experiment, the only variable is uncertainty, with two levels, small and large; therefore, we manipulated uncertainty within group. All participants were asked to interact with the small-uncertainty assistant in one of the task sections and interact with the large-uncertainty assistant in the other (order counterbalanced). The wake words were “Alexa” and “Echo” for the small- and large-uncertainty assistant, respectively. The experimental procedures were the same as in the first experiment.

The small- and large-uncertainty task lists are documented in S4 Table. In the large-uncertainty task list, the virtual assistant only issues ambiguous responses when it is unable to perform the task. Conversely, in the small-uncertainty task list, informative responses were issued. For instance, when asked to “calculate 1000 US dollars minus 70,000 Japanese Yen,” we prompted an informative response: “Sorry, I can’t calculate between different currencies.” Because the responses of the two assistants did not differ in terms of their capability profile, we argue that the expectations induced by the two task sections should be identical.

5.2 Implementation

The experimental setup and procedures, including instructions, were identical to those of the first experiment (see Section 3.4 and S1 Appendix). The questionnaires used can be found in S5 Table. To counterbalance the effects of the order, half the participants started in the large-uncertainty condition, and half in the small. The experimental procedures are detailed in S1 Fig.

Ten students from the University of Tokyo (4 female, 6 male, age range from 21 to 27, experiment conducted in July 2019) participated in the experiment. One set of data had to be discarded because devices lost connection to the Internet during task sections. All participants were new to virtual assistants, and none of them had participated in Experiment 1. Participants were rewarded with 1000 Japanese Yen for participating. They were informed of the monetary reward when recruited and received the reward after the experiment. The study protocol was approved by the Ethics Committee of the Graduate School of Engineering, the University of Tokyo. All participants provided written informed consent prior to their participation in this study.

8.1 Mechanism of intrinsically motivated interaction

We constructed a theory to discuss user motivation and spontaneous interactions in the context of conversing with virtual assistants. Our theory withstood experimental examination: we proved that when the user has high expectations of capability, intrinsically motivated interaction can be fostered by reducing uncertainty (more specifically, by making the virtual assistant issue an informative response whenever it is unable to satisfy the user’s intention).

The mechanism behind our findings is best explained by focusing attention on three interlocked aspects: motivation, expectation, and action selection. Motivation depends on the expectation of capability: it will result in spontaneous interaction only when the user anticipates that the virtual assistant will successfully cope with the stated intention.

We emphasize that user motivation should be divided into two categories: intrinsic and non-intrinsic. Intrinsic motivation is derived from the expected enjoyment or pleasure of the action itself. Non-intrinsic motivation is derived from the need to achieve an external goal and should be further divided into epistemic and extrinsic motivation.

In general, if the user is exploring whether the assistant is capable of doing something, the action should be considered epistemically motivated. If the user is certain about the capability and the interaction promises pragmatic value in terms of fulfilling separable goals by the user, it should be seen as extrinsically motivated. Borrowing terminologies from the active inference theory, the former case counts as exploration, which enables future pragmatic actions; the latter counts as exploitation, which fulfills the goals directly [12].

In the context of virtual assistant interaction, intrinsically motivated interactions involve activities such as pastime and playing activities. Epistemically motivated interactions involve testing and making trials (“What can I do with the assistant?”). Extrinsically motivated interactions involve pragmatic usage to gain information or have some “chores” done by the assistants instead of the users themselves, thus saving time [1]. For clarity, using the virtual assistant as a learning tool counts as extrinsic usage because the separable goal is to learn.

Spontaneous interactions can be intrinsic as well as non-intrinsic. To ask when actions are intrinsically motivated is to ask how they are selected. Action selection, according to reversal theory [22], depends on the user’s state of mind. The human mind can operate in two meta-motivational modes: the goal-oriented telic mode and the activity-oriented paratelic mode. Intrinsically motivated interaction is most likely to occur in the paratelic mode. The mind shifts between these two modes situationally; such shifts are termed “reversals”. Our work shows that an informative response can facilitate reversal from the telic to the paratelic mode.

Issuing an informative response is merely one of many methods to foster intrinsically motivated interaction; it works by reducing uncertainty, so that the user is placed in a comfortable condition (paratelic mode), free from pressure. However, one still might ask why reducing uncertainty is the key to intrinsic motivation.

Imagine being trapped in a jungle and facing dire survival concerns. Clearly, actions are not likely to be taken for enjoyment of the activity itself. One must explore the environment, in other words, reduce uncertainty, before deciding which way to go as well as where to find food. Interestingly, the human mind can actually derive pleasure from resolving uncertainty; designers and artists have long used this trick to add novelty to their work. In the theory of aesthetic valence, it is pointed out that optimum novelty requires work to be perceived initially as a chunk of incomprehensible information (which introduces uncertainty) [23]. Artists and designers weave hints and cues that facilitate sense-making into the novel work, so that it does not take observers too long to arrive at an understanding of the work. The pleasure perceived when the work finally makes sense is termed the “second reward” [23], as opposed to the first reward from simply receiving the information. The informative response strategy works the same way.

An informative response would most likely start with “Sorry, I can’t/I don’t know…” The user is notified by the “Sorry,” knowing that their intention is not to be satisfied. Then, the informative part explains “what went wrong” or “where was the limit,” helping the user make sense. It does not only offer consolation but also provides the user with hints for future interaction. For instance, after hearing that the assistant cannot perform multiple tasks at once, the user may adjust interaction strategies to simplify a complex intention into single tasks. For example, recent studies on explainable AI stressed the importance of clarifying the assistant’s limitations, so that users understand what interaction choices they have [7,24,25].

Studies on the relationship between physiological arousal and hedonic states have also stated that reducing uncertainty “makes the world enjoyable” [26,27]. Certainly, this does not mean that, from the hedonic point of view, an informative failure response would be better than a coping response. As we observed in experiments, high performance by the virtual assistant leads to high expectation of capability, resulting in more intrinsically motivated interactions. Nevertheless, if we wish to apply such knowledge to foster intrinsically motivated interaction, it is effective to create a second reward when the user’s intention cannot be satisfied, by informing of the limitations of the assistant after the inevitable “Sorry”.

8.2 Application to virtual assistant design

Our experimental findings are transferable to practical virtual assistant design. Modern virtual assistants recognize users’ intentions by filling slots. Slots are variables that are embedded in applications to serve the user’s requests. They are filled by picking up key information from user utterances. For instance, a flight booking application typically expects to fill slots such as “destination” and “date”. For Amazon Echo series, developers are allowed to define intents and slots in order to capture meaningful information from user utterances. For each intent and under each slot there can be multiple corresponding utterances, which are spoken phrases with a high likelihood of being used to convey intent [28].

If the assistant picks up an utterance that is understandable but not implementable, it should issue an informative response. To achieve this, developers should collect utterances that are recognized but with which the assistant cannot yet cope. Next, they need to categorize the reasons for this failure and try to obtain clusters of such reasons. Each cluster should then be addressed by a set of (at least one) informative responses explaining to the user that the virtual assistant cannot cope with the intention yet. Addressing ambiguous responses cluster by cluster is not an exhaustive method, but it does not take an exhaustive method to reduce uncertainty. For instance, telling users “I’m not able to do two things at once” actually forestalls a very large set of problems that would be impossible to list exhaustively.

8.3 Limitations

A major limitation of this study is that manipulation of expectation and uncertainty was implemented only in task sections, in which all activities were extrinsically motivated. In addition to our current findings, we wish to investigate results of such manipulation when they are performed during intrinsically motivated or intrinsically motivating activities. In real-life interaction scenarios, the learning process is experienced through tasking as well as through playing. Moreover, one study found that, in the context of gaming, the uncertainty of success brings suspense, which in turn leads to enjoyment [29]. Indeed, with the current findings we cannot predict how uncertainty during intrinsically motivated interaction will further affect ensuing action selection because the user’s acceptance of uncertainty is likely different when in paratelic rather than telic mode.