Figures
Abstract
Selection methods that require only a single-switch input, such as a button click or blink, are potentially useful for individuals with motor impairments, mobile technology users, and individuals wishing to transmit information securely. We present a single-switch selection method, “Nomon,” that is general and efficient. Existing single-switch selection methods require selectable options to be arranged in ways that limit potential applications. By contrast, traditional operating systems, web browsers, and free-form applications (such as drawing) place options at arbitrary points on the screen. Nomon, however, has the flexibility to select any point on a screen. Nomon adapts automatically to an individual's clicking ability; it allows a person who clicks precisely to make a selection quickly and allows a person who clicks imprecisely more time to make a selection without error. Nomon reaps gains in information rate by allowing the specification of beliefs (priors) about option selection probabilities and by avoiding tree-based selection schemes in favor of direct (posterior) inference. We have developed both a Nomon-based writing application and a drawing application. To evaluate Nomon's performance, we compared the writing application with a popular existing method for single-switch writing (row-column scanning). Novice users wrote 35% faster with the Nomon interface than with the scanning interface. An experienced user (author TB, with 10 hours practice) wrote at speeds of 9.3 words per minute with Nomon, using 1.2 clicks per character and making no errors in the final text.
Citation: Broderick T, MacKay DJC (2009) Fast and Flexible Selection with a Single Switch. PLoS ONE 4(10): e7481. https://doi.org/10.1371/journal.pone.0007481
Editor: Nick Monk, University of Nottingham, United Kingdom
Received: July 31, 2009; Accepted: September 18, 2009; Published: October 22, 2009
Copyright: © 2009 Broderick, MacKay. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This research was supported by the Nine Tuna Foundation. (http://www.ninetuna.org/) and Nokia (http://www.nokia.co.uk/). TB's research was supported by a Marshall Scholarship (http://www.marshallscholarship.org/). Sensory Software (http://www.sensorysoftware.com/) generously provided the 60-day free trial of The Grid 2 used in our performance comparison. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: This research was supported in part by an unrestricted donation from Nokia (http://www.nokia.co.uk/). Sensory Software (http://www.sensorysoftware.com/) provided the 60-day free trial of The Grid 2 used in our performance comparison. All donations to this study, financial and of software, were unrestricted. There are no patents related to this study.
Introduction
In single-switch communication, user input consists of repeated clicks, distinguished only by timing information; these clicks might be generated by pressing a button or blinking. For instance, the range of movement of individuals with severe motor impairments may be limited to a single muscle. Alternatively, a crowded or jostled mobile technology user may be able to click precisely while other actions are difficult or sloppy. A single switch may also be useful when information conveyed, such as a PIN, is sensitive and hand location on a normal keyboard might betray this content. Our method, Nomon (Figures 1, 2), expands the application scope of existing methods and facilitates faster writing than the most common single-switch writing interface.
The horizontal and vertical positions of the option points were chosen uniformly at random in the box shown to illustrate the flexibility of the method.
Words that are prefixed by the concatenation of the current context and the letter X appear next to the letter X. Underscore represents a space. Options for period, a character-deletion function, and an undo function are also available.
Existing single-switch communication methods include scanning [1]–[9] and One-Button Dasher [10]–[15]. (Morse Code does not fall under the strict definition of a single switch interface since it requires either click duration information or multiple switches.) Scanning is the most popular single-switch selection method. In a scanning interface, options such as letters are arranged in a grid (Figure 3). For standard row-column scanning, each row of the grid is highlighted in turn, with the highlight moving to the next row at fixed time intervals, a.k.a. scanning delays. When a click is made, the columns of the selected row are then highlighted in turn, typically iterating at the same fixed time intervals. To select a column, and thereby make a final selection, the user clicks when the highlight is on that column. A variety of customizable commercial scanning software exists for writing and computer navigation [16]–[19] although customization is often not single-switch accessible. The Gnome Onscreen Keyboard [19], by contrast, can generate a grid for new applications “on the fly.”
The six long rectangles on the left hold word completions. The remaining options are fixed and include letters, an underscore for space, a period, a character-deletion function, and a word-deletion function.
While the scanning method can be used to select anything that can be arranged in a grid, One-Button Dasher is limited to writing with alphabetic character sets. Dasher works by arranging all possible character strings in alphabetic order and having the user zoom in on the desired string. More likely strings, according to the language model, are given relatively more space and are thus easier to select.
Scanning and One-Button Dasher require options to be arranged in a particular configuration. By contrast, traditional operating systems, web browsers, and free-form applications such as drawing place options at arbitrary points on the screen. Scanning, the most popular single-switch communication method, is limited in further ways by its grid structure. For instance, the grid options may theoretically be reordered after any selection to allow the most likely options to be selected the most quickly. However, in practice this reordering requires that users either learn many grid arrangements or search the grid for their desired option upon each reordering.
Even scanning a grid that maintains a fixed layout at all times has drawbacks. Previous studies suggest that, at least among children, scanning a fixed grid demands a higher cognitive load than direct selection [20]–[23]—though an earlier study found no difference [24]. One implicated factor is the need for a user to divide her attention between the scanning highlight and the desired option [20], [23]. Another issue in scanning is the possibility of distraction, and loss of the target from working memory, while highlighting progresses [23], [25].
Therefore, we seek a single-switch selection method that is not limited to certain forms of option placement. We want our method to work for any number of options; to be able to effectively reorder the set of selections without imposing additional cognitive load; and to allow the user to attend only to the desired target.
Below, we begin by describing such a method, which we call “Nomon.” We also describe how our method can adapt to individuals' clicking abilities and how it can incorporate prior beliefs about option selection frequency. In order to evaluate our method's performance, we note that much single-switch research has focused on optimizing writing speed [1]–[6] and the number of clicks per output symbol [7]–[9] in scanning interfaces. In light of these studies, we developed a writing application, the Nomon Keyboard (Figure 2), using our method and compared its performance with a popular commercial scanning interface, The Grid 2 [16] (Figure 3). We examined the study participants' writing speeds, error rates, and number of clicks made per character as well as the subjective ratings of their experiences.
The full technical report describing Nomon is available online at http://www.inference.phy.cam.ac.uk/nomon/files/nomon_tech_report.pdf. The Nomon Keyboard, as well as a drawing application (Nomon Draw) and instructions for the use of both applications, is available for download at http://www.inference.phy.cam.ac.uk/nomon/ under the GNU General Public License 3.0.
A New Method
Nomon, a new single-switch communication method, does not limit the user to selecting options that can be arranged in a grid or alphabetically. Rather, it can be used to select among any points of interest on a screen. The trademark of a Nomon application is a set of small clocks, one clock associated with each selectable option. Each clock appears alongside its corresponding option on the screen. For instance, Figure 1 illustrates clocks corresponding to 16 arbitrary option locations. Another example might be a drawing application where a clock appears at every “pixel” on the canvas and also next to each menu option. In a writing application (the Nomon Keyboard), a clock appears next to each character, word completion, or text editing function (Figure 2).
Just as menu options and drawing tools in a point-and-click interface are accessed in the same way by the mouse, all Nomon clocks are selected in the same way by a single switch. Each Nomon clock features a moving hand and a fixed line at noon. All moving hands rotate at the same, fixed speed but, at any time, are located at a variety of angles relative to noon. The user tries to click precisely when the moving hand on her desired clock is at noon. She repeats this action until the clock is selected. Selection is signalled by the desired clock being highlighted with a darker color and the entire application flashing a lighter color; there may also be audio feedback. Between clicks (if more than one click is required to select a clock), the clock angular offsets are adjusted by a heuristic to maximize the expected information content of the user's next click.
Row-column scanning can be viewed as a special case of the Nomon selection method where clocks are arranged in a grid, moving-hand angular offsets are aligned alternately across rows and columns of clocks, and each selection is based only on the times of the last two clicks. But this synchrony does not take full advantage of the continuous, periodic representation of the clock and imposes an order on the set of options relative to their positions onscreen. Rather, by allowing more general clock hand positions, we can, effectively, completely reorder the set of selections after each click without demanding any extra cognitive load from the user.
Similarly, the independent movement of the clock hands frees the user to attend only to the desired target, in contrast to the need, in scanning, for the user to attend both to the desired target and the moving highlight. Further, the scanning user may forget her target as highlighting progresses. But in Nomon, once the target is located visually, the user is free (without suffering a performance penalty) to focus on selecting a single, fixed clock. Since the clock periods are usually much shorter than a full scanning rotation, there is also no significant penalty for missing a potential click time.
In Nomon, by contrast with scanning, we assume that the user will not always click perfectly at the desired time. The details of Nomon operation are described more fully in the Nomon Operation section below and outlined here. Nomon can learn a user's probability of clicking at different (typically small) offsets relative to noon. This learning is accomplished via an approximate Parzen window estimator, with contributions from more recent clicks weighted more strongly to allow adaptation to a user whose skill changes over time. We can also specify a prior probability distribution over clocks according to a predictive model of user choices. For instance, in the writing application tested below, our language model assigned prior probabilities to letters and word completions based on the British National Corpus word-frequency list [26]. These prior probabilities could also be adaptive and context dependent.
During a particular selection process, the posterior probability of any clock given the clicks thus far can be calculated from Bayes' theorem. When the probability of a single clock is sufficiently high, we declare it the winner. The probability threshold for winning is an adjustable parameter of the model; it can vary according to context or from clock to clock. A higher threshold can ensure greater safety for critical actions.
Results
We developed a writing program using the Nomon method, the Nomon Keyboard (Figure 2), and conducted a study to compare writing with Nomon to writing with a popular commercial scanning interface, The Grid 2 [16] (Figure 3). To that end, sixteen study participants with no previous experience of either interface wrote with Nomon and The Grid 2. In each of two sessions, a participant used one of the interfaces to write short phrases appearing on screen. A session was divided into four blocks, each lasting approximately minutes. During the first three (of four) writing blocks, each participant was allowed to adjust the rotation-period or scanning-delay parameter, as appropriate to the current interface, at the end of each written phrase. No changes were allowed during the final block. For each interface, cash prizes were won by the faster half of participants in the final block.
In total, we collected 34 hours of data from 16 novice participants and one experienced single-switch user (TB, with hours experience in each interface). We compared three objective measures of the novice participants performance between the two interfaces: text-entry rate, error rate, and click load (clicks per character). We also examined subjective ratings of the two interfaces given by the novice participants.
Text-entry Rate
We calculated text-entry rate in words per minute, where a word is defined as five consecutive characters in the output text. At the beginning of each fourteen-minute block, the participants were asked to write two periods “..” using the interface for that session. This action signalled that they were ready to begin and initiated the display of the first target phrase. Timing started once the two periods were written. After every phrase, participants wrote two periods to signal that they were ready for a new phrase. Timing stopped after the final two periods following the last phrase were written. All periods except the first two in a block were counted as characters in what follows, and the time spent writing them was counted as well.
The left panel of Figure 4 shows the novice participants' mean entry rates across the four blocks for each interface. Also shown, for comparison, is the performance of the experienced user. Participants wrote faster with Nomon than with The Grid 2 during the first block (,
). The total session time was short for both interfaces, but participants' writing speed with each interface improved with practice. Participants became faster at writing using the Nomon Keyboard during the Nomon session (
,
) and became faster at writing using The Grid 2 during the scanning session (
,
). In the final block we see that participants remained faster at writing with Nomon than with The Grid 2 (
,
). In this fourth block, participants wrote
faster with Nomon than with the scanning interface; participants were writing at
words per minute on average with The Grid 2 and
words per minute with the Nomon Keyboard. The experienced user wrote, on average, at
words per minute with the Nomon Keyboard and
words per minute with The Grid 2.
Mean entry rate is measured in words per minute, and click load is measured in clicks per output character. In both panels, error bars represent 95% confidence intervals for the novice user means, and the average experienced user (TB) performance is illustrated by horizontal lines for comparison.
While the alphabetic layout was easy for novices to use, a computer simulating writing from a conversational corpus with no errors has been shown to achieve a faster writing speed with a frequency-ordered layout than with an alphabetic layout [6]. Even if we artificially inflate the novice writing speeds using The Grid 2 by
, novices remain faster at writing with Nomon (
,
).
Error Rate
To find the error rate during a block, we begin by computing the character-level Levenshtein distance [27] between the
target phrase in the block and the text written by the participant;
is also known as the edit distance. We define the error rate for the block to be
, where
is the number of characters in the
target phrase.
The average novice character-level error rate (over all blocks) for the Nomon Keyboard was , and the average novice error rate for The Grid 2 was
. There was no significant difference in novice error rate between the two interfaces (
,
). The experienced user made no errors while using Nomon (
) and made one error while using The Grid 2, for a mean scanning block error rate of
.
We believe that the participants' output errors were mostly caused by poor recall of the target sentence. For instance, one participant pluralized “head” in “head_shoulders_knees_and_toes” and wrote “reading_week_is_almost_here” instead of “reading_week_is_just_about_here”.
Click Load
The click load is the number of clicks per output-text character. Other names for this measure include “keystrokes per character” [28] and “gestures per character” [14]. The click load is calculated as the number of button presses in a block divided by the number of characters in the output. Clicking often can be tiring for any user and especially so for some users with specific motor impairments.
While the inclusion of word-completion options in a scanning grid has been shown to have no positive effect on writing speed with a scanning interface [7], other studies confirm that word completion options yield substantial click-rate savings over the baseline (mistake-free) row-column click load of two clicks per character [8], [9]. Therefore, we included six word-completion options in the leftmost row of our scanning grid (consistent with the default layouts in The Grid 2 [16]). These were ordered from top to bottom and filled in automatically by the software.
Click loads are illustrated in the right panel of Figure 4. The average novice rate (over all blocks) for the Nomon Keyboard was clicks per character, and the average novice rate for The Grid 2 was
clicks per character. There was no significant difference in novice click load between the two interfaces (
,
).
While the experienced user required, on average, clicks per character in The Grid 2, she required only
clicks per character using the Nomon Keyboard. For comparison, writing with the same character set on a normal keyboard requires at least one key press for each character and thus at least 1 click per character (possibly more due to error correction). To compare to Morse code, we find letter, space, and period frequencies directly from our phrase set. We assume the Morse encoding of [17], [29]. In this case, an error-free Morse code click load estimate is
clicks per character. This load is over twice as high as the click load of the experienced user on the Nomon Keyboard.
Subjective Ratings
We assessed novice participants' opinions with a questionnaire immediately after writing with an interface was completed. The questionnaires for each interface were identical (except for the name of the interface). Participants were asked to rate how much they agreed with a series of statements on a scale from 1 (strongly disagree) to 7 (strongly agree). These statements were largely the same as those in [30]. Participants were encouraged to write any thoughts about the interfaces in an “Open Comments” box.
Participants' responses to selected statements are summarized in Table 1. Not only did participants like using the Nomon Keyboard in aggregate, but every participant individually liked using Nomon at least as much as The Grid 2. Contributing factors for why the Nomon Keyboard was preferred became apparent in the remaining responses. Participants found it easier to select word completions and easier to correct errors with the Nomon Keyboard. These responses corroborate our objective findings above.
While many written comments agreed with participants' numerical ratings, unique to the open comments section was the sentiment that Nomon looks unusual at first but is worth getting to know. One participant remarked, “Surprisingly, I found this more user-friendly.” Another noted, “The writing system looks intimidating when it first comes up on screen but is actually very easy to use.”
Discussion
Nomon benefits in this comparison from its nice scaling properties and clock-position flexibility. Our posterior-based selection method implies that the time taken to make a selection in Nomon scales logarithmically with the number of clocks if the prior over clocks is uniform. The entropy of the discrete uniform distribution, which happens to be the highest-entropy (finite) discrete distribution, scales logarithmically with respect to the number of points in the support. Figure 5 shows that, generally, clicks are required by an experienced Nomon user (TB, with
hours experience) to make a selection in a 30-clock application. In a Nomon application with uniform prior and
clocks,
clicks are generally required for this user to make a selection. The difference in entropy between the prior for the
-clock application and the highest-entropy prior for the
-clock application is about
bits, in agreement with
.
Entropy is shown as a function of clicks remaining to selection. Each solid line represents a single selection process. Dotted lines decreasing to zero at respective rates of (lower) and
(upper) bits per click are illustrated for reference. Left: 25 selections on the Nomon Keyboard: 30 clocks, non-trivial prior
, clock period
seconds, switch input from joystick button. Right: 25 selections on another Nomon application: 401 clocks, uniform prior
, clock period
seconds, switch input from space bar. Data was generated by the experienced user (TB).
Not only does the number of clicks to selection in Nomon scale well, but including additional options with small prior probabilities has little effect on clicks-to-selection for more-likely clocks. Therefore, we could place many more word completions on screen than would be feasible for a scanning interface. We limited ourselves to three per character so as to allow fast reading of the three relevant options. Placing word completions next to letters in Nomon was feasible since clock position onscreen does not affect Nomon operation. Interspersing word completions with letters in row-column scanning would increase the number of scanning steps required to reach many options.
While a Nomon writing application allows a straightforward comparison of Nomon with existing single-switch communication methods, the Nomon selection method is not limited to writing. For example, Nomon can be used for internet browsing by placing a Nomon clock next to each link. Or Nomon can be used for drawing by placing a dense grid of, say, hundreds of clocks on a canvas. (The Nomon Draw application works in this way.) A user can draw a line by selecting points directly from the canvas. Options for colors, shape drawing, saving, and printing can likewise be accessed with clocks. A general graphical user interface can be navigated with Nomon by placing clocks at the points where a user might traditionally point and click.
It is worth pointing out that the flexibility of Nomon is not specific to our clock display choice. Other local periodic representations of the global set of options would also allow the arbitrary placement of options onscreen. For instance, the clocks could be replaced by bouncing balls at different points in their trajectories; instead of clicking at noon, the user would click when the desired ball hits the ground. It remains to be studied whether such alternative display choices might facilitate even faster or easier use of this system.
Materials and Methods
We begin by detailing the experimental method used in the study above and follow with a description of how Nomon functions.
Experiment
Participants.
We recruited sixteen participants from the university community across a wide range of academic disciplines. All participants gave written informed consent. In accordance with the University of Cambridge ethical review procedure as defined in the Cambridge Psychology Research Ethics Committee Handbook (http://www.bio.cam.ac.uk/sbs/psyres/), the experimental design received an internal peer review within the department, where it was decided that ethics approval from the committee was not necessary.
The participants' ages ranged from 22 to 39 (mean = 26, sd = 4). Eight were women, and eight were men. Participants were screened for motor or cognitive difficulties; in particular, no participant had dyslexia or RSI. None of the participants had used a scanning or Nomon interface before. No participant had regularly used any single-switch interface before. Twelve of the participants had used word completion (e.g. on cell phones).
In addition to the sixteen novice participants, an experienced user of Nomon and The Grid 2 ( hours writing with each interface) was run through the same experimental procedure for comparison.
Apparatus and Software.
All sessions were run on a Dell Latitude XT Tablet PC with a partitioned hard drive. The 12.1 inch color screen had a physical screen size of 261163 mm. The single-switch hardware device in all cases was the trigger button of a Logic3 Tornado USB joystick. Participants operated the trigger button with the first finger of their left or right hand. None of the other joystick inputs was used. For both writing interfaces, automated spoken feedback was provided as the user wrote.
Nomon Keyboard.
We ran the Nomon Keyboard (Figure 2) on an Ubuntu 8.10 operating system running the Linux kernel. The screen resolution was 1280800 pixels, and the physical size of the keyboard display was 224
85 mm (1125
416 pixels). The interface was docked in the upper part of the screen. A text box and phrase box were located below the keyboard in the same window. The keys of the keyboard were arranged in six rows and five columns. Each key contained a principal character, with letters in alphabetical order (across and then down) first, followed by four special characters: an underscore (representing space), a period, a character-deletion function, and an undo function. Each letter key also contained up to three word completions. The undo function undid the previous selection if it was a character selection, word-completion selection, or deletion.
The clock rotation period could be set to
seconds for
. Higher
corresponded to faster rotation. The initial setting of the period for novices was
(
). The experienced user initially chose
(
).
The Grid 2.
We ran The Grid 2 (Figure 3) on a Windows Vista Service Pack 1 operating system. The screen resolution was again 1280800 pixels. The physical size of The Grid 2 display, using the scanning grid we designed for this experiment, was 261
102 mm (1280
500 pixels). The interface was docked in the upper part of the screen, and the text box and phrase box were docked immediately below. Six word-completion boxes appeared on the left side of the main interface. The remaining space was divided into six rows and five columns of keys. Each key contained a single character. First were letters arranged in alphabetical order (across and then down), followed by an underscore, a period, a character-deletion function, and a word-deletion function.
The Grid 2 allowed scanning delay values at
for
. Higher
corresponded to faster scanning. The initial setting of the delay for novices was
(
). The experienced user initially chose
(
).
Procedure.
The experiment consisted of two identical sessions, one for each interface. The starting interface was balanced across participants, and sessions were spaced at least four hours apart.
Each session proceeded according to the same schedule. The first ten minutes were introductory. First, the supervisor either explained or reviewed the experimental procedure according to the session number. Then the participant was shown how to use one of the interfaces. The demonstration included basic writing, word completion, and error correction.
The next hour was divided into four 14-minute blocks, separated by short breaks. During the blocks, participants were asked to write phrases drawn from a modified version of the phrase set provided by [31], with British spellings and words substituted for their American counterparts. For each participant, a different random ordering of the initial phrase set was generated. Phrases appeared one at a time in the phrase box at the bottom of the screen. Once a participant finished a phrase, writing the period character twice would cause a new target phrase to appear and the text box to empty. Participants were instructed that no changes relevant to a particular phrase could be made after the two periods were written.
During the first three (of four) writing blocks, each participant was allowed to adjust the rotation-period or scanning-delay parameter at the end of each written phrase. In particular, immediately after writing two periods and receiving the new target phrase, the participant could increment or decrement (defined above) by one. The experienced user incremented to
(
) after two blocks using the Nomon Keyboard and incremented to
(
) after two blocks using The Grid 2. No other changes were made by this user.
Novice participants were paid £10 for each of the two sessions; the experienced participant was not paid. Novice participants were informed at the beginning of the study that they could receive a £5 bonus for achieving a writing speed among the top half of novice participants for each interface. They were further informed that, for the purposes of the bonus, writing speed would be measured only during the final writing block. They were told that they would not be allowed to change the rotation-period or scanning-delay parameter during this block and thus would have to calibrate it as they saw fit during the previous blocks. Information about their own writing speeds across full blocks and also phrase-by-phrase was made available to participants during the break after each block.
We performed seven significance tests with a family-level significance of . Observing the Bonferroni correction, we performed each individual test at a significance level of
. Wherever
values are quoted, an analysis of variance (ANOVA) test for repeated measures was performed.
Nomon Operation
We here describe the prior over clocks, click likelihood (given a clock), and the resulting posterior over clocks in turn. While we focus on a prior for a specific application (the Nomon Keyboard), the likelihood and posterior discussions are germane to a general Nomon application.
Prior.
In the absence of information about clock probabilities, we use a uniform prior over clocks
. We can choose a more informative prior for our Nomon writing application, the Nomon Keyboard (Figure 2). This interface features four special characters (underscore representing space; period; Delete; and Undo), 26 letters, and up to three word completions per letter. We assign fixed prior probabilities to the special characters and assign the remaining priors according to Laplace smoothing out of the leftover probability mass
. Let
(
) be the context (all letters from the end of the current output text) before the user begins to make another selection. Let
be the set of word completions appearing on screen, and set
. To form our corpus, we begin with the British National Corpus word list [26], then we remove single-letter words besides “I” and “a” and keep only words appearing with some small minimum frequency (
appearances in the corpus).
When an appropriate word completion is offered, the user may nevertheless choose the next single letter; the following model assumes that the user is equally likely to choose either of these options. If is the number of occurrences of word
in our corpus, we define a context frequency
and a screen word-completion summed frequency
. If
is the clock corresponding to letter
and
the clock corresponding to word
,
(1)
(2)
To model an ideal user, we would subtract the count of words onscreen prefixed by from the numerator of
, and both denominators would equal
. Finally, while the number of letters is fixed at 26,
is variable since, for any letter, we include only those word completions among the three most probable above a certain threshold. It was judged that requiring
yielded a reasonable balance between displaying common words and not cluttering the screen.
Click Distribution.
Any particular clock defines a desired click time at noon. We wish to estimate a user's click time distribution relative to noon
, where we distinguish
only up to the clock period
and set
to zero at noon. To that end, we begin with a broad, and slightly offset, initial setting of our estimate for
:
. The
-dependence ensures the estimate will be nontrivial at any user-chosen period. We update the
distribution with a (modified) Parzen window estimator—with width given below—and a damping factor
that allows learning to continue over time. After any selection is made, we modify the distribution estimate with the data from the
selection before the latest one (here
). This delay allows the user to choose Undo after a selection, in which case we do not use the clicks toward that selection for learning. Once a selection occurred
rounds in the past, we assume that it was correctly chosen. With the clock choice
known for the
selection, we are able to calculate click times around noon
for each click that was made toward this selection. We treat these as data from the distribution
we are estimating. To calculate our estimate
for
after the
selection, we make use of the unnormalized distributions
.
(3)
The update equation specifies that, after each selection, is damped by the factor
. The next term is a sum over clicks
leading to the
selection. Within the summation is a normal density centered at the click time
, as in Parzen window estimation. The width for this Parzen-window term is given by
, which is derived from the normal scale rule estimate [32], [33] for the Parzen window. That is,
(4)where
is the standard (Gaussian maximum likelihood) variance estimator obtained from the last
clicks before the
round. The factor
in the initial
definition is an effective number of samples derived from the damping factor. Using this factor and the unnormalized update, we ensure that the initial estimate
dominates
even after the first few selections. Without the
factor, the Parzen window term for the first click,
, would have nearly equal weight with the initial estimate.
This estimate for allows us to save the estimated distribution and update it quickly and easily during operation of the application. As a result, users can start the Nomon application immediately, without a waiting or calibration period, but they can also enjoy an experience tailored to their abilities. For instance, a user need not click at noon (or any offset) exactly. Their personal offset, reflecting reaction time, is learned by this method rather than hard-coded and, as long as it is not too close to 6 o'clock, will make no difference to program operation. The precision around this personal offset determines the number of clicks necessary to make a selection.
Posterior.
With a prior and likelihood, we may calculate the posterior probability of each clock given the
clicks thus far using Bayes' theorem:
. In practice, we store the unnormalized log probabilities for each
. Checking that the highest clock probability
exceeds some threshold would require exponentiating every stored value and summing over the results. Noting that
is equivalent to
for some
, we instead declare a winner when
. The choice of
represents a desired upper bound on error fraction, per selection, of
. In a sample of 1,714 consecutive selections made by an experienced Nomon user (TB) on the Nomon Keyboard under this setting, the average value of
over all selections after the deciding click was
, and the average value of
was
, suggesting our heuristic stopping criterion is a reasonable approximation to the desired one. In the 1,714 selections, 3 (non-consecutive) selections were Undo, indicating mistakes and giving an empirical error rate of about
, in line with the calculated rate.
Acknowledgments
We are grateful to Per Ola Kristensson, Geoffrey Hinton, Keith Vertanen, Philipp Hennig, Carl Scheffler, and Philip Sterne for helpful discussions.
Author Contributions
Conceived and designed the experiments: TB DJCM. Performed the experiments: TB. Analyzed the data: TB. Wrote the paper: TB DJCM.
References
- 1. Damper R (1984) Text composition by the physically disabled: A rate prediction model for scanning input. Applied Ergonomics 15: 289–296.
- 2. Simpson R, Koester H (1999) Adaptive one-switch row-column scanning. IEEE Transactions on Rehabilitation Engineering 7: 464–473.
- 3.
Evreinov G, Raisamo R (2004) Optimizing menu selection process for single-switch manipulation. Paris, France: Springer-Verlag Berlin/Heidelberg. pp. 836–844.
- 4. Szeto AYJ, Allen EJ, Littrell MC (1993) Comparison of speed and accuracy for selected electronic communication devices and input methods. Augmentative and Alternative Communication 9: 229–242.
- 5.
Baljko M, Tam A (2006) Indirect text entry using one or two keys. New York, NY, USA: ACM. pp. 18–25. https://doi.org/http://doi.acm.org/10.1145/1168987.1168992
- 6. Venkatagiri HS (1999) Efficient keyboard layouts for sequential access in augmentative and alternative communication. Augmentative and Alternative Communication 15: 126–134.
- 7. Koester H, Levine S (1994) Learning and performance of able-bodied individuals using scanning systems with and without word prediction. Assistive Technology 6: 42–53.
- 8. Koester H, Levine S (1996) Effect of a word prediction feature on user performance. Augmentative and Alternative Communication 12: 155–168.
- 9. Lesher G, Moulton B, Higginbotham D (1998) Optimal character arrangements for ambiguous keyboards. IEEE Transactions on Rehabilitation Engineering 6: 415–423.
- 10.
MacKay DJC, Ball CJ, Donegan M (2004) Efficient communication with one or two buttons. In: Fischer R, Preuss R, Toussaint vonU, editors. Maximum Entropy and Bayesian Methods. Melville, NY, USA: American Institute of Physics, volume 735 of AIP Conference Proceedings. pp. 207–218.
- 11.
MacKay DJ, Ball CJ (2006) Dasher's one-button dynamic mode—theory and preliminary results. Technical report, Cavendish Laboratory, University of Cambridge. Available at http://www.inference.phy.cam.ac.uk/mackay/abstracts/OneButton.html.
- 12.
MacKay DJ (2007) Another one-button dynamic mode for Dasher: ‘two-click mode’. Technical report, Cavendish Laboratory, University of Cambridge. Available at http://www.inference.phy.cam.ac.uk/mackay/abstracts/OneButton2.html.
- 13.
Mead JMG, Cowans PJ, MacKay DJC (2009) Efficient communication through the timings of one or two buttons. Technical report, Cavendish Laboratory, University of Cambridge. Available at http://www.inference.phy.cam.ac.uk/mackay/buttondasher2b.pdf.
- 14.
Ward DJ, Blackwell AF, MacKay DJC (2000) Dasher—a data entry interface using continuous gestures and language models. New York, NY, USA: ACM. pp. 129–137. https://doi.org/http://doi.acm.org/10.1145/354401.354427
- 15. Ward DJ, MacKay DJC (2002) Fast hands-free writing by gaze direction. Nature 418: 838.
- 16.
Sensory Software International Ltd (2008) The Grid 2: Version 2.4. Sensory Software International Ltd. URL http://www.sensorysoftware.com/.
- 17.
Words+, Inc (2000) EZ Keys™ User Manual. Words+, Inc. URL http://www.words-plus.com/website/products/manuals/manual.htm.
- 18.
Words+, Inc (2004) EZ Keys™ XP Product Specification Sheet. Words+, Inc. URL http://www.words-plus.com/website/pdf_files/ezkeysxp.pdf.
- 19.
Sun Java Desktop System Documentation Team (2004) GOK Manual V2.3. Sun Microsystems, Inc. URL http://www.gok.ca/usermanual.shtml.
- 20. Horn EM, Jones HA (1996) Comparison of two selection techniques used in augmentative and alternative communication. Augmentative and Alternative Communication 12: 23–31.
- 21. Mizuko M, Reichle J, Ratcliff A, Esser J (1994) Effects of selection techniques and array sizes on short-term visual memory. Augmentative and Alternative Communication 10: 237–244.
- 22. Ratcliff A (1994) Comparison of relative demands implicated in direct selection and scanning: Considerations from normal children. Augmentative and Alternative Communication 10: 67–74.
- 23. Wagner BT, Jackson HM (2006) Developmental memory capacity resources of typical children retrieving picture communication symbols using direct selection and visual linear scanning with fixed communication displays. Journal of Speech, Language, and Hearing Research 49: 113–126.
- 24. Mizuko M, Esser J (1991) The effect of direct selection and circular scanning on visual sequential recall. Journal of Speech and Hearing Research 34: 43–48.
- 25. Petersen K, Reichle J, Johnston SS (2000) Examining preschoolers' performance in linear and row-column scanning techniques. Augmentative and Alternative Communication 16: 27–36.
- 26.
Kilgarriff A (1998) BNC database and word frequency lists. Available at http://www.kilgarriff.co.uk/bnc-readme.html. Accessed on 18 March, 2009.
- 27. Wagner RA, Fischer MJ (1974) The string-to-string correction problem. Journal of the Association for Computing Machinery 21: 168–173.
- 28.
MacKenzie IS (2002) KSPC (keystrokes per character) as a characteristic of text entry techniques. Heidelberg, Germany: Springer-Verlag. pp. 195–210.
- 29.
ITU Radiocommunication Assembly (2004) International Morse code, Recommendation ITU-R M.1677.
- 30.
Kristensson PO, Denby LC (2009) Text entry performance of state of the art unconstrained handwriting recognition: a longitudinal user study. New York, NY, USA: ACM. pp. 567–570. https://doi.org/http://doi.acm.org/10.1145/1518701.1518788
- 31.
MacKenzie IS, Soukoreff RW (2003) Phrase sets for evaluating text entry techniques. New York, NY, USA: ACM. pp. 754–755. https://doi.org/http://doi.acm.org/10.1145/765891.765971
- 32.
Wand MP, Jones MC (1995) Kernel smoothing. London, UK: Chapman & Hall/CRC.
- 33.
Silverman BW (1986) Density Estimation for Statistics and Data Analysis. London, UK: Chapman & Hall/CRC.