Fig 1.
Participants interact with the cognitive assistant through task-related inputs and outputs—in practice, these correspond to the video feed captured by the assistant and the instructions provided by it. The assistant itself has been instrumented with a data collection layer, which collects and processes experiment-related data such as biometric signals from the participants (these are merely processed here and do not form part of the inputs to the cognitive assistant as such, however), and a delay buffer, which introduces controlled delays in the transit of information from the core processing component.
Fig 2.
Example of a cognitive assistance task and its component steps.
Fig 3.
Hierarchical cognitive structure of a step in the LEGO task.
Fig 4.
Components of the cognitive assistance task.
(a) Structure of a step in a generic cognitive assistance application. The assistant provides an instruction to the user and continuously samples the step state; inputs captured while the step is unfinished are silently “discarded” (i.e. they do not cause the generation of a new instruction). Once the user finishes performing the step, the next sample will cause the generation of a new instruction. (b) In the experimental task, an additional variable segment of time is introduced immediately following the processing of the input frame in order to extend the perceived processing time of the input to a specific target delay. (c) Structure of a block in the experimental task. (d) Visualization of the execution time of a step.
Table 1.
Means and standard deviations of normalized questionnaire scores.
Fig 5.
Sample frames captured during the experiment.
(a) Frame from the face recording of a random participant, clearly showing the locations of the EEG electrodes. (b) Frame from the board recording of the same participant.
Fig 6.
Per-step execution time by block length vs. delay.
Error bars indicate the Standard Error of the Mean (S.E.M.).
Table 2.
Significant effects on per-step execution time from ANOVA on factors delay and block length.
Table 3.
Significant effects on per-step execution time from ANOVA on factors block slice and delay.
Fig 7.
Mean per-step execution time vs. delay, by step slice.
Error bars indicate S.E.M.
Fig 8.
Per-step execution time across the first four steps after a block transition from block Bk−1 to Bk.
Error bars indicate S.E.M.
Table 4.
Significant effects on accelerometer data from ANOVA on factors delay and block length.
Fig 9.
Movement score vs. delay, per block length.
Error bars indicate S.E.M.
Fig 10.
Movement score vs. delay, per block slice.
Error bars indicate S.E.M.
Table 5.
Significant effects on accelerometer data from ANOVA on factors delay and slice number.
Table 6.
Significant effects on log EEG power from ANOVA on factors delay and block slice.
Fig 11.
Log of the average EEG Power for alpha and beta bands per step slice.
Error bars indicate S.E.M.
Table 7.
Principal component analysis.
Fig 12.
Correlation between neuroticism score of participants and their execution time in the longest block at the longest delay.
Pearson correlation coefficient r =.40; 2-tailed p =.01.