Fig 1.
An asymmetric user interface, where there is slow, unreliable input coupled with a high-capacity, error-free feedback channel; e.g. a noisy switch with a visual display.
Fig 2.
Non-invasive BCI accuracy summary.
Summary of per-classification accuracy from many binary non-invasive BCI EEG studies, plotted from the Tables A1-A3 of Lotte et al. [22], including movement intention and mental task imagination BCIs. This figure summarises the accuracy of classifiers used in a large number of brain-computer interfaces, indicating that accuracies vary widely from around 65% to 95% (i.e. there are error rates of 5% to 35%). Where ranges or bounds were given the stated numerical value is used.
Fig 3.
Entropy, channel and line coding.
The nested entropy coding, channel coding and line coding stages in a communication channel [60].
Fig 4.
Entropy, channel and line coding in the user interface.
The input problem in an asymmetric user interface, viewing the interface communication channel. The diagram shows how the feedback loop allows users to drive the internal state of a system towards their intention, via a nested series of entropy, channel and line coding steps. Sophisticated transport of information across an interface can be implemented by pushing the complexity of the encoding process from the user into the system and relying on feedback to mediate the process.
Fig 5.
The noisy button model of an interface.
An intended input is flipped with probability f0 or f1, depending on which button was pressed, resulting in the detected input bi.
Fig 6.
The Shannon bound for the noisy binary channel and classical code performance.
The bound is shown in terms of numbers of input decisions/bits per error-free output bit. The upper single hatched region shows the binary symmetric case f0 = f1 = f, and the lower double hatched region shows the capacity of the fully biased Z-channel
with the same average error rate (one completely reliable input and one noisy input). Curves for the classic Hamming and Hadamard codes for various word lengths are shown for reference.
Fig 7.
Capacity of the binary asymmetric channel.
Different values of error probabilities f0 and f1 are plotted with the capacity given by Eq 2. The white contours show lines of constant channel capacity; the dark lines indicate lines of fixed bias b ∈ [−1, 1], where f0 = f + bf, f1 = f − bf.
Fig 8.
Capacity of the binary symmetric channel with backspace Rb(k, f).
Capacity of backspace for alphabets with k = 2, 3, 4, 6 for N = 10000 simulated entries of a n = 32 symbol sequence. Throughput is shown as the number of input bits per correct bit R; solid lines shows the mean throughput, and the shaded region shows the standard deviation. The hatched region is the Shannon bound for the binary symmetric channel. Even with k = 2 bit symbols, capacity goes to zero as the reliability drops below 75%. Dashed lines show the fit of Eq 3.
Fig 9.
The key step of the Horstein algorithm: Distorting the CDF.
The CDF transition is shown for the case where the initial transition is b0 = 0. The CDF Fi(x) is initially a line segment with gradient 1. It is partitioned at the point mi where Fi(x) = 0.5 and the left and right gradients are scaled by factors p and 1 − p. In the case b1 = 1, the respective factors would be 1 − q and q.
Fig 10.
Uniform and nonuniform targets.
Subdivision of the unit interval into discrete targets for selection can be performed uniformly (top, corresponding to a flat prior over targets), or according to some known prior distribution π(si) (bottom).
Fig 11.
Ridge-plots of the PDFs from the Horstein algorithm.
PDFs are plotted following each input bi applying Horstein algorithm to select the the 5 bit symbol 01010 (mapped to the interval around θ = 0.3125, highlighted in red), for the noise-free case (left, f = 0) and with simulated noise (right, f = 0.15), with k = 5, β = 2. Sharpening of the PDF is gentler in the case with noise.
Fig 12.
Example simulation of the Horstein algorithm with noisy inputs.
k = 8, β = 0 selecting a target θ = 0.71875, with symmetric (left) and biased (right) noise. The pulse traces show the inputs for bi = 0 and bi = 1 respectively; highlighted sections indicate erroneous inputs. The centre plot shows the log PDF logfi(x) at each step.
Fig 13.
The Horstein algorithm being applied to approach θ = 0.71875, with an assumed f = 0.1, shown as the inverse PDF f′(x) distorting a ruler spanning the unit interval. The median mi is shown as a red line.
Fig 14.
Rotating the plane 45 degrees allows partition on both axes with only left-right decisions.
Fig 15.
A 2D Horstein-zooming interface using linear and non-linear zooming displays (every second step shown). Targets are shown as black points, with the intended target shown as a larger red marker. (a) Linear zooming, where the geometry of points is fixed, and the view spans an interval of constant density. (b) Non-linear zooming, where the inverse PDF is directly applied to points in a 2D space, pushing unlikely points to the edges of space, always showing the entire unit interval.
Fig 16.
Screenshots of prototype Horstein decoder interfaces.
Left-to-right: (a) Simple 2D point based non-linear zooming (b) Nonlinear zooming with rectangular area targets (c) Linear zooming with a randomised circle-packing (d) Diagonal-split interface which only requires left-right decisions (e) An interface using Jigsaw space-filling curves [111] to layout blocks of ordered targets.
Fig 17.
Decoder performance for varying confirmation β.
k = 8 in all trials. (Left) input bits per output bit R, (Centre) uncorrected k bit symbol errors ek (Right) backspace corrected rate R′(k; f). As β increases ek → 0.
Fig 18.
Simulated uncorrected error rates ek from N = 10000 repetitions, for k ∈ 4, 6, 8, 10 and various different β. There is a very strong linear relation between the uncorrected error rate and β, which does not depend on k. The linear fit is shown as a dashed line. The gradient and offset of the line reduce exponentially with increasing β. The dashed line shows the approximated error rate ek(f) using Eq 11.
Fig 19.
Decoder performance for varying symbol size k.
β = 2. Larger k has improved capacity.
Fig 20.
Decoder performance for varying relative bias b.
k = 12, β = 2, fδ = bf where f0 = f + bf, f1 = f − bf. Dashed lines show the theoretical bound for the given bias, and the solid lines show the mean simulated results from the Horstein decoder. The decoder approaches the bound for any level of bias.
Fig 21.
Decoder performance for mismatch between decoder and simulated statistics.
f′ = f + fh, k = 8 and β = 0. When fh < 0, the decoder is optimistic and uncorrected error rates rapidly rise. When fh > 0, the decoder is pessimistic and induces a penalty to the rate while decreasing uncorrected errors.
Fig 22.
Decoder performance with mismatched statistics.
Decoder performance is shown for k = 8 and β = 8, as a function of fh and f. There is a complex trade off between capacity and the pessimism/optimism of the decoder. The mean rate (left) uncorrected error rate (centre) and backspace-corrected rate (right) are shown. White spaces indicate regions where throughput is zero due to error cascades. Each contour line indicates one additional input bit per error-free output bit.
Fig 23.
The Gilbert-Elliot Markov chain for a bursty channel.
The Markov chain randomly transitions between a good state G and bad state B.
Fig 24.
Effect of burstiness on decoding performance.
The plots show the effects of varying burstiness t and average error rate f on theHorstein decoder with k = 8, β = 8 from N = 10000 random simulations. Increasing burstiness leads to increased uncorrected error rates but a decrease in the decisions/bit. Consequently, the backspace-corrected entry rate is largely unaffected by burstiness.
Fig 25.
Example entropy time series with a change of heart.
Change of heart occurs at λ = 0.5; xΔ = 0.25(Hλ = −5). The decoder is configured with k = 8, β = 2; f = 0.15; fh = 0.1, N = 250 trials, mean curve shown in blue. The simulation changes target when Hλ ≤ −5 to a target with separation xΔ = 0.25. 99.2% of trials acquired the changed target sb correctly. There is a marked v shape to the curve as the decoder entropy increases after input starts to become incompatible with the original target sa, and then decreases as sb is approached.
Fig 26.
This plot shows the effect of a sudden change of intention during selection. Coloured by β (left) and by xδ (right). Simulations run with k = 8, f ∈ [0.0, 0.3], β ∈ {0, 1, 2, 4, 8}, fh ∈ [0.0, 0.2], f′ = f + fh, xΔ ∈ [0.0, 1.0], λ ∈ [0.0, 1.0]. Each point represents the mean of N = 500 trials. λ represents the proportion of the selection at which the target intention changes (in terms of decoder entropy). xΔ is the distance between the originally intended and final targets in the unit interval. Artificial jitter added to x values to separate points.
Fig 27.
Online adaptation of the channel statistics.
A decoder with k = 8, β = 8 is used for selection. In each panel, the simulated error level f is held fixed, and the configured expected error level f′ is adapted online. Each panel shows 50 replications with random initial starting values for f′ (shown as circles at left), showing the decoder will converge regardless of the initialisation.
Fig 28.
Online adaptation of the channel statistics for a biased channel.
A decoder with k = 8, β = 8 is used for selection. In each panel, the decoder is initially run with and
(both randomly chosen in the range [0.0, 0.35]). At symbol 100, f0 and f1 are changed to random values, and
and
are automatically adapted using Eq 16.
Fig 29.
The visual display used in the experiment.
A single target is shown in red, appearing at an initial random location. The target has sides of length 2−6. A diagonal split is used to elicit user responses and linear zooming is used. A progress bar shows the entropy drop remaining.
Table 1.
The experimental conditions for the human-in-the-loop experiment.
Table 2.
Summary of experimental results, including observed input error rates , decisions/bit
, uncorrected errors
and backspace-corrected equivalent rates
.
Fig 30.
Additional input errors introduced over the simulated noise.
Error bars are 95% CI. Red line indicates the headroom fh. Mean additional input error is 0.95%.
Table 3.
Comparison of experimental results with numerical simulations Rs, es.
Table 4.
Average number of targets selected correctly by participants, and the average number of bits entered correctly, averaged across all targets in each condition.
The average time to select one target is also given. Each condition has six twelve-bit targets.
Table 5.
Backspace-corrected experimental rates against simulated backspace-corrected rate and Shannon bound, and fraction of simulator performance/Shannon bound achieved.
Table 6.
Backspace-corrected rate against theoretical Shannon upper bounds.
Fig 31.
Measured performance against Shannon upper bound.
is plotted against
,
for each condition. A line at α = 0:5 showing 50% of the Shannon bound is shown.
Table 7.
All numbers in seconds. is duration of one decision (from prompt to keypress); Tmin is the 300ms minimum delay enforced;
is the time taken to enter one bit of information, on average.
Fig 32.
Decisions per bit and timing of decisions.
(Left) The number of decisions R required for each bit (Right) The time taken for each decision , which remained approximately constant across conditions. Red line shows the minimum fixed delay.
Fig 33.
PDF evolution form a random run from C-15-15.
(above) Colormap showing PDF after each input; darker indicates greater density (below) Ridge-plot of the same density sequence, where the height of the line is proportional to the log PDF; each line corresponds to a single decision. The maintenance of multiple hypotheses is visible.
Fig 34.
Each plot shows the mean (solid line), one standard deviation (shaded area) and the theoretical prediction (dashed line) of the entropy of the decoder’s PDF against decision number, for each of the experimental conditions.