Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

A Multi-Area Stochastic Model for a Covert Visual Search Task

  • Michael A. Schwemmer ,

    Contributed equally to this work with: Michael A. Schwemmer, Samuel F. Feng

    schwemmer.2@mbi.osu.edu

    Affiliation Mathematical Biosciences Institute, The Ohio State University, Columbus, OH 43210, United States of America

  • Samuel F. Feng ,

    Contributed equally to this work with: Michael A. Schwemmer, Samuel F. Feng

    Affiliation Department of Applied Mathematics and Sciences, Khalifa University, Abu Dhabi, United Arab Emirates

  • Philip J. Holmes,

    Affiliation Program in Applied and Computational Mathematics, Department of Mechanical and Aerospace Engineering, and Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, United States of America

  • Jacqueline Gottlieb,

    Affiliation Department of Neuroscience, Columbia University, New York, NY 10032, United States of America

  • Jonathan D. Cohen

    Affiliation Department of Psychology and Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544, United States of America

A Multi-Area Stochastic Model for a Covert Visual Search Task

  • Michael A. Schwemmer, 
  • Samuel F. Feng, 
  • Philip J. Holmes, 
  • Jacqueline Gottlieb, 
  • Jonathan D. Cohen
PLOS
x

Abstract

Decisions typically comprise several elements. For example, attention must be directed towards specific objects, their identities recognized, and a choice made among alternatives. Pairs of competing accumulators and drift-diffusion processes provide good models of evidence integration in two-alternative perceptual choices, but more complex tasks requiring the coordination of attention and decision making involve multistage processing and multiple brain areas. Here we consider a task in which a target is located among distractors and its identity reported by lever release. The data comprise reaction times, accuracies, and single unit recordings from two monkeys’ lateral interparietal area (LIP) neurons. LIP firing rates distinguish between targets and distractors, exhibit stimulus set size effects, and show response-hemifield congruence effects. These data motivate our model, which uses coupled sets of leaky competing accumulators to represent processes hypothesized to occur in feature-selective areas and limb motor and pre-motor areas, together with the visual selection process occurring in LIP. Model simulations capture the electrophysiological and behavioral data, and fitted parameters suggest that different connection weights between LIP and the other cortical areas may account for the observed behavioral differences between the animals.

Introduction

Decisions pervade our daily lives. They typically involve the coordination of several steps, including attending to relevant stimuli, extracting the evidence therein, and selecting an appropriate action. Significant advances in understanding the neural mechanisms of decision making have come from studies of the monkey oculomotor system, which have tested the neural correlates of simple perceptual decisions. For instance, when discriminating motion direction in a random dot kinematogram (e.g. [1, 2]). neurons in the lateral intraparietal area (LIP), exhibit spatially-tuned visuo-motor responses that signal the action with which monkeys report their decisions: a saccade inside or opposite their visual receptive field (RF) [36]. Decision-related LIP firing rates (FRs) can be fit remarkably well by pairs of leaky competing accumulators [7], which can in turn be reduced to drift-diffusion processes [8]. Such models are remarkably successful in capturing reaction time distributions and error rates, providing compelling motivation for their continued use [9, 10]

In these investigations the relevant sensory evidence is clear, but in nature animals are surrounded by multiple stimuli and, in addition to deciding which action to take, must decide which stimulus to attend to. Thus, natural decisions entail at least two interdependent processes: selection of a relevant source of information and selection of an appropriate course of action. Most previous modeling studies have considered the neural correlates of attention or action selection, but not their interactions (exceptions include [11, 12]). More abstract connectionist networks have addressed interactions among brain areas supporting different processes (e.g., feature representation vs. location-specific information [13]), but they have generally lacked sufficient detail to make contact with neurophysiological data. Here, we present a model that attempts to bridge this gap, based on a task in which LIP neurons encode interacting effects of visual and motor selection.

In the behavioral paradigm (shown below in Fig 1), monkeys were required to discriminate whether a target (an E-like shape) was oriented to the right or left, and indicate their responses by releasing a bar that was held, respectively, in the right or left paw [14, 15]. Therefore, as in earlier paradigms, the monkeys made a binary perceptual decision and signaled it with a motor action. However, in contrast with those studies, the target was surrounded by irrelevant distractors and, before responding, the monkeys had to use covert attention to find the target in the distractor array. LIP neurons encoded correlates of both the visual and the motor selections. The neurons responded with higher FRs if the target rather than a distractor was in their receptive field (RF), signaling the visual selection, and this visuo-spatial response was sensitive to the motor action, being stronger or weaker when the monkeys released the right or the left paw. This complex coupling of attention and decision-related responses suggests that pairs of competing accumulators that describe only the final decision alternatives (i.e., the two manual actions) may not adequately capture the decision process or the role of LIP within it.

thumbnail
Fig 1. The covert search task.

Displays of set sizes 2, 4 and 6 are shown top to bottom. Monkeys initiated a trial by fixating on a central point and grasping two bars; they were required to maintain fixation throughout the trial. One figure eight placeholder always fell in the recorded neuron’s receptive field (RF, gray disc). After 500 msec, two line segments were removed from each figure, revealing distractors and one target: a left- or right-facing 𝖤 (latter shown here). Correct responses, signalled by respective release of left (L) or right (R) bar were rewarded. Figure from [15] according to Creative Commons Attribution License.

https://doi.org/10.1371/journal.pone.0136097.g001

Here we construct a model containing several accumulators that implement visual and motor selection and interact during decision making. To focus our model, we do not include processes related to early visual representation or target search and discrimination, but focus on the 3-way interactions between a shape selective representation, a visual selection area and a motor selection system. While the properties of the shape selective and motor stages are inferred from the literature, the visual selection stage is modeled based on the LIP data. The model produces acceptable fits not only of LIP neural responses but also reaction time (RT) distributions and accuracy, and captures individual differences between two monkeys through changes in connection strengths among the different modules. Thus we show that a framework built on the competing accumulator mechanism that has been applied to simple perceptual decisions can be extended to account for more complex decisions that involve interaction among distinct processes of visual and skeleto-motor selection.

Analyses and Methods

Before developing the model, we review key observations from the electrophysiological and behavioral data published in [14, 15]. The re-analyses presented in the following sections and Figs 25 motivate and guide our model construction, after which we describe the methods used for fitting the model to the experimental data.

The covert search task: key observations and data analysis

Two macaque monkeys (M11 and M12) performed a covert visual search task in which they discriminated the orientation of a single visual target or cue embedded in an array of stimuli, as shown in Fig 1 [14, 15]. To initiate a trial, a monkey fixated on the central point in a visual display with either 2, 4 or 6 figure eight placeholders, spaced equally around the periphery, the number being the set size. The monkey was also required to hold two bars located below the display, outside its visual field, one in each hand. After 500 msec, at cue onset, two line segments were removed from each placeholder so that one became the target and the others became distractors (having different forms for set sizes 4 and 6). The target was a letter 𝖤, oriented left (∃) or right (𝖤). Maintaining fixation throughout, monkeys had to find the target, identify its orientation and indicate their choice by releasing either the left (L) or right (R) bar with the corresponding limb.

Single unit spike time data was recorded from LIP neurons that had been identified using the memory saccade task [15]. After locating the neuron’s receptive field (RF), the display was oriented so that one placeholder lay entirely inside the recorded neuron’s RF. On each trial, the target location and orientation (𝖤, ∃) were varied uniformly at random across the 2, 4, or 6 locations. Trials were administered in randomly-interleaved blocks with set size fixed throughout each block, and RTs and accuracy were logged. Blocks were typically arranged into sessions in which recordings were made from the same cell, while set size changed across blocks. At most one session of data was collected each day from one of two monkeys (M11 and M12). For M11 there were 46 sessions containing 108 blocks, totaling 11768 trials. For M12 there were 61 sessions containing 124 blocks, totaling 9661 trials. Details of methods and equipment used can be found in [14, 15]. Also, the recorded data is available as S1 Datasets.

The results in this section, including some that previously appeared in [14, 15], were obtained by reanalyzing the original behavioral and spike train data in preparation for building and fitting the model we discuss below. FRs were computed by convolving spike times with a Gaussian distribution of standard deviation 15 msec and averaging over appropriate trials, aligned at cue onset. To reproduce FRs reported in [15], we excluded six cells with maximum FRs exceeding 80 Hz, substantially higher that all others (< 50 Hz): cells 96, 154, and 210 for M11, and 196, 263, and 292 for M12. S1 Text lists the numbers of cells available for analysis after excluding these six. These numbers differ for each set size. In all analyses that follow and the data fits of our model, we include only those cells for which trials were available for all set sizes, and their associated behavioral data. There were 23 such cells for M11 and 21 for M12.

We now describe key features of the electrophysiological and behavioral data that our model incorporates. These include effects produced by target vs. distractor location in RF, set size effects, response-hemifield congruence, and accuracy versus RT patterns.

LIP neurons encode target location.

In [14] it was found that the majority of LIP neurons encoded target location, responding strongly and selectively on trials in which the target fell in the neuron’s RF. Responses with distractors in the RF were much weaker, even though distractors had similar forms and brightnesses. Fig 2 shows these effects, locked to cue onset (i.e. t = 0 denotes the time at which two bars were removed from each figure eight). Shaded regions in the top panels of Fig 2 indicate one standard error from mean (SEM) FRs, averaged over all data; bottom panels show FRs of set size 4 data in each monkey.

thumbnail
Fig 2. LIP neurons show differences in activity for targets and distractors in their receptive fields.

Mean firing rates (FRs) in LIP in M11 (left) and M12 (right) for target (solid) and distractor (dashed) in receptive field (RF) respectively; correct and incorrect trials are included. Top panels show averages over all set sizes. Cue onset occurs at t = 0 and vertical dotted black lines represent overall mean reaction times (RTs) (486.9 ms, M11 and 482.9 ms, M12); shaded regions represent one standard error from the mean (SEM). Bottom panels show averages over set size 4 data; see Table 1 for mean RTs for each set size.

https://doi.org/10.1371/journal.pone.0136097.g002

Fig 2 shows that steady increases in FR ensued after fixation (from −200 ms to shortly after cue onset). Thereafter both target and distractor FRs increased more rapidly for ≈ 100 ms, when distractor FRs peaked and began to decrease, while target FRs continued to increase for a futher ≈ 100 ms. Both FRs then decreased until after the response was made. Such encoding of behaviorally-relevant stimuli exemplify attentional effects that have previously been found in LIP [16]. Fig 2 also reveals differences between the two monkeys: M12 exhibits a higher peak FR than M11 when the target is in the neuron’s RF and M12’s FR decays more rapidly than M11’s with a distractor in the neuron’s RF. Also, differences between FRs with target and distractor in RF are greater for M12 than M11.

Neural and behavioral data exhibit set size effects.

LIP activity and behavior were also affected by set size [15]. As the number of distractors increased, lower FRs were observed throughout the trial, as shown in Fig 3. This suppression occured with both target and distractors in the recorded neuron’s RF. Increases in set size also caused monkeys to respond more slowly and less accurately: an expected behavioral effect shown in Table 1. The set size effect was first demonstrated in [15], where it was noted that it might be due to long-range competitive interactions that limit neural activities in LIP related to spatial attention. Our model implements competition through the use of mutual inhibition among units representing different visual areas, and our model fitting results show that it can produce set size effects similar to those of Fig 3.

thumbnail
Fig 3. LIP neurons display differing levels of activity depending upon the number of visual objects on screen.

Mean firing rates (FRs) in LIP in monkey 11 (M11, left) and monkey 12 (M12, right) for different set sizes, averaged over all neurons for which all three set sizes were tested. As set size increased from 2 (red) to 4 (green) to 6 (blue), firing rates decreased for both target (solid) and distractor (dashed) in receptive field (RF). Standard errors are not shown to avoid clutter.

https://doi.org/10.1371/journal.pone.0136097.g003

Neural and behavioral data show response-hemifield congruence effects.

The covert search task has two stages: to select the informative target (in R or L hemifield) and then release the appropriate bar (with R or L hand). This leads to an interesting congruence effect, first reported in [14], and implicit in RTs shown in the bottom two rows of Table 1. A trial is called response-hemifield congruent if the correct L/R response required a bar release on the same side of the display as the target. Thus, congruent trials had right-facing 𝖤’s to the right of the fixation point and left-facing ∃’s to the left of the fixation point. Incongruent trials were those for which right-facing 𝖤’s were on the left and left-facing ∃’s were on the right, each requiring a lever release on the opposite side to that on which the target appeared (see [14, Fig 7). M11 responded much faster and more accurately on response-hemifield congruent versus incongruent trials. Surprisingly, M12 responded slightly slower on response-hemifield congruent trials, with similar accuracy on both trial types. Furthermore, when the target was in the RF, LIP FRs differed significantly for congruent versus incongruent trials for M11 but not for M12, as shown in Fig 4 (t-test 200ms before bar release, p = 1.723 × 10−49 for M11, p = 0.0993 for M12).

thumbnail
Fig 4. LIP neurons in one monkey display differing levels of activity depending upon the orientation of the target and its position in the left or right visual hemifield.

Response-hemifield congruence effects for set size 4. M11 (left) exhibited significantly greater firing rates (FRs) for congruent (upper, green) versus incongruent (lower, black) trials with target in receptive field (RF) (solid) (p = 1.723 × 10−49); FR differences for distractor in RF (dashed) were not significant (p = 0.05649). M12 (right) showed no significant difference in FRs for congruent versus incongruent trials with either target or distractor in RF (p = 0.09932 and p = 0.2472 respectively). Shaded regions represent one SEM.

https://doi.org/10.1371/journal.pone.0136097.g004

This neural correlate of response-hemifield congruence is interesting given LIP’s role in attention. Below, we will show that a similar effect appears in the model due to interaction between attention and strengths of intrinsic connections among processing areas (potentially reflective of learning effects), and that this may explain M11’s significantly faster responses to congruent than incongruent trials as compared to M12 (Table 1). For more discussion of potential implications of this effect, see [14], where limb preferences are also described and both effects are studied in depth. Cells with limb preferences show significantly greater activity when the L or R limb responds (see S1 Text), but as described there we did not explicitly use this classification in our analyses or model.

Behavioral data show unusual accuracy versus reaction time effects.

In fixed difficulty choice tasks, accuracy often increases as RTs increase, reflecting a speed-accuracy tradeoff [8, 17]. To examine this in the present task, we partitioned RTs into 100 ms bins for each set size and computed accuracies in each bin for response-hemifield congruent and incongruent trials: see Fig 5. M11 showed large decreases in accuracy with RT for congruent trials and a more complex rise and fall for incongruent trials, associated with a substantial set size effect and overall accuracy deficit in that condition (Table 1). M12 also showed decreases in accuracy, stronger for incongruent than congruent trials, but smaller overall than in M11, and with small set size effects on both congruent and incongruent trials.

thumbnail
Fig 5. Response-hemifield congruence also affects accuracy and reaction time.

Accuracy as a function of reaction time (RT) for set sizes 2 (red), 4 (green), and 6 (blue). Downward trends in accuracy at longer RTs are greater in M11 than M12, and more marked for incongruent trials. Set size effects are especially pronounced in M11 for incongruent trials.

https://doi.org/10.1371/journal.pone.0136097.g005

This “reverse speed-accuracy tradeoff” is most evident in both monkeys for set sizes 4 and 6, for which RT distributions are broader. The set size effect is expected: when targets must be found among several distractors, search time increases and accuracy is likely to suffer. The overall decrease in accuracy with RT may reflect differences in experienced difficulty (e.g., due to lapses of attention), with more difficult trials showing longer RTs and lower accuracy.

A multi-area, multi-stage accumulator model

As noted in the Introduction, previous binary choice models have employed pairs of leaky competing accumulators (see [7, 18] for reviews) or scalar drift-diffusion and Ornstein-Uhlenbeck processes [8, 9, 19, 20]. Such models can fit behavioral data and neural recordings during evidence integration (e.g., [6, 21]), but it seems unlikely that two accumulators, much less one representing the difference between activations in two populations, could reproduce the rich variety of LIP responses described above. Moreover, such models have been used to fit neural data from perceptual choice tasks in which a single stimulus was shown on each trial and responses were reported via saccades, in contrast to stimulus arrays in which the target must be located covertly and its identity signalled by lever release, as in the current task. Indeed, the FR patterns of Figs 24 show initial rises but then decreases in the ≈ 300 ms interval prior to response, compared with the monotonic rise to threshold typical of LIP cells before saccades in simpler choice tasks [22]. Furthermore, the normal speed-accuracy tradeoff is reversed.

Components of the model.

The considerations above, and the rationale for its structure and components provided under “Model construction” in the Results section below, led to the model shown in Fig 6. The areas labeled inferior temporal cortex (IT) and anterior intraparietal area (AIP) respectively exemplify a feature-selective area in visual cortex and a limb pre-motor area. We use these labels for brevity, noting that other areas in visual cortex and elsewhere may be involved in place of IT and AIP.

thumbnail
Fig 6. The network model for the covert search task.

There are four main components: LIP, a feature selective area (IT), a limb pre-motor area (AIP) and motor cortex. Boxes to the right of AIP and motor areas indicate that the units in these regions are gated accumulators. The box representing stimulus display illustrates how, following cue onset at t = 0, inputs enter IT and LIP for set size 4 and target a right-facing 𝖤 in the upper right LIP unit’s receptive field. Arrows and filled circles respectively indicate excitatory and inhibitory connections. Connection strengths w, β, time constants τ, leakages k, noise levels c and all other parameters are described in the text, along with further details and the roles of perceptual and target delays Pdelay and Tdelay.

https://doi.org/10.1371/journal.pone.0136097.g006

The LIP area contains six units representing pools of neurons with RFs corresponding to stimulus locations. These pools mutually inhibit one another with connection strengths decaying as RFs grow farther apart as proposed in [12], where analogous data from frontal eye fields were successfully fitted. Net inhibition increases as more units activate, causing overall suppression of FRs and producing the set size effect.

Noisy stimulus inputs enter LIP successively in two stages, following an initial sensory latency Pdelay due to signal transmission from the retina to parietal lobe. First an input of mean strength slip is fed to 2, 4 or 6 LIP units, depending upon set size. This represents a general perceptual signal driven by the placeholders prior to cue onset. After a further delay Tdelay to account for search time to find the target, which we suppose is achieved in another (possibly frontal) brain area, a second input of mean strength star is added to the unit whose RF corresponds to target location. Simultaneous with this input to LIP at t = Pdelay + Tdelay, a separate input of mean strength sori enters one of the two mutually inhibiting units representing target orientation in IT cortex; in congruent trials sori enters on the same side as the LIP input star, and in incongruent trials on the opposite side. No additional delay parameter for the orientation selection process prior to or in IT is included because we assume that this process occurs quickly relative to the target identification process in LIP. In preliminary work [23] we analyzed the set size 4 RT data in a manner similar to that of [24]. We found no evidence of systematic search strategies and so do not explicitly model the target search process.

All input signals are piecewise-constant step functions filtered through Ornstein-Uhlenbeck (OU) processes to represent stimuli passed through synapses, as used previously in reductions of spiking neuron models of perceptual decisions [25, 26]. The filtered signals are combined with mutual inhibition and feedback and passed to their respective LIP and IT units through a firing rate input-output function derived in [26]. Such functions have been shown to be equivalent to other popular activation level models [27].

Area IT feeds forward via excitatory connections to a pair of gated accumulator units representing AIP, and LIP projects to AIP through mutually excitatory connections according to which hemifield the LIP unit belongs. Area AIP also projects back to LIP via cross inhibition, with the left AIP unit inhibiting all right LIP units, and vice-versa. Via this pathway LIP also reflects the activity of AIP. The AIP units are similar to the gated accumulators of [12]; in that paper signals derived from FEF recordings were passed through such units in order to fit RTs. Gated accumulators also play the role of bistable neurons in [28].

The AIP units project forward via excitatory connections to a second pair of mutually inhibitory accumulator units, equipped with response thresholds. As activity grows in this motor layer, all other areas receive inhibition that strengthens substantially as one unit’s FR approaches and crosses threshold. This forms a reset mechanism that drives down IT and LIP FRs, as suggested in previous decision making models [28]. Attention suppression has also been observed in EEG event-related potentials, and has been proposed as a general-purpose mechanism to terminate the allocation of attention [29]. When the first motor unit reaches threshold, the corresponding response is made and the RT is logged, after incorporating a nondecision time T0 representing motor latency.

The model equations

We now present the stochastic differential equations (SDEs) that define the model in explicit detail. The parameters to be fitted are listed in Table 2 and explained below. All other parameters are fixed at constant values, also given below. Throughout ξ(t) denotes an independent (i.i.d.) realization of white noise with mean zero and unit variance in each equation. We use (⋅) to denote normal parentheses and [⋅] to denote arguments in functions; H[⋅] denotes the Heaviside function (1)

Some notation is necessary to define how equations depend upon the trial inputs. We denote the input array by a triplet 𝓐 = (set size, position, 𝖤-orientation) = (𝓐ss, 𝓐p, 𝓐𝖤), indicating set size, position and 𝖤-orientation of the target. set size 𝓐ss is either 2, 4, or 6. The position 𝓐p ∈ {L1, L2, L3, R1, R2, R3} indicates whether the target is on the left or right, and in the top, middle or bottom rows as per the labels on the LIP units of Fig 6. The orientation 𝓐𝖤 indicates if the target is left- (∃) or right-facing (𝖤). E.g., if set size is 4 and the target is a right-facing 𝖤 at upper right, 𝓐 = (4, R1, 𝖤). If set size is 6 and the target is a left-facing 𝖤 in the middle row, left side, 𝓐 = (6, L2, ∃).

The model consists of 20 SDEs. Activities of the six LIP units are denoted by Xlip, L1, Xlip, L2, Xlip, L3, Xlip, R1, Xlip, R2, and Xlip, R3. Subscripts are chosen to indicate the location of each LIP unit’s receptive field in the same manner as for 𝓐, i.e. Xlip, R2 is the state variable for the LIP unit with a receptive field in the middle row on the right side of the hemifield. Each LIP unit has an input OU process paired with it, denoted by ρlip, L1, ρlip, L2, ρlip, L3, ρlip, R1, ρlip, R2, and ρlip, R3, with subscripts indicating the pairings. There are two IT units, one for each possible 𝖤-orientation denoted by Xit, 𝖤 for the right facing 𝖤 and Xit, ∃ for the left facing ∃; each IT unit also has an input OU process, denoted by ρit, 𝖤 and ρit, ∃. AIP contains two units, one for each possible response, denoted by Xaip, L for left and Xaip, R for right responses. The motor area also contains two units, Xm, L for left and Xm, R for the right response.

The LIP and IT units use the input-output function ϕ[I] derived in [26] Eq (24)] with the parameter values specified in [26] Appendix A]: (2)

We now state the equations for the six LIP units: (3) where Imi, defined as (4) is the global motor inhibition signal common to all LIP units.

Next we state the six OU processes which provide stimulus input to each LIP unit: (5) where LOC ∈ {L1, L2, L3, R1, R2, R3} denotes one of the locations for the LIP units, and ξlip, LOC(t) denotes independent additive white noise with zero mean and unit variance input to each unit. Guided by synaptic time constants for fast neurotransmitters (AMPA, GABAA), we fix the time constant for these OU processes at τlip, noise = 5 ms. This is faster than the 2 ms time constants used in [25, 26] but necessary to allow the relatively coarse temporal resolution dt = 0.5 ms used in simulations (see below section on fitting methods). The perceptual input from the stimulus Iper[𝓐ss, LOC] activates the appropriate number of OU processes depending upon set size: This states that the perceptual signal is sent units L1 and R1 for set size 2, L1, R1, L2 and R2 for set size 4, and all 6 for set size 6. All LIP units are active for all set sizes, including those that do not receive perceptual input. The target input Itar[𝓐p, LOC] activates only the unit which corresponds to the target location:

We now state the equations for IT: (6) where Imi is defined in Eq 4. The equations for the two input OU processes to the IT units are (7) Here ORI ∈ {𝖤, ∃} denotes one of the two possible E orientations and we fix the time constant of this OU process at τit, noise = 5 ms, as for τlip, noise above. As above ξit, ORI(t) is additive white noise with zero mean and unit variance. The target input from the stimulus Iori[𝓐𝖤, ORI] is

Before writing the equations for AIP and motor units, we specify the piecewise-linear gating function G[⋅, ⋅] used in both these areas: (8) The equations for AIP may now be written: (9) where Imaip is the same as in Eq 4 but with βmitlip replaced with βmaip. Finally, the equations for the motor units are (10) Again the terms ξaip, R(t) and ξm, L, R(t) denote independent white noise processes with zero mean and unit variance. Motor unit FRs were capped at 150 Hz by setting Xm, L and Xm, R at that value whenever they exceeded it; in practice these FRs rarely exceeded 120 Hz.

In total the system is defined by 20 SDEs: 6 LIP units, 6 LIP OU processes, 2 IT units, 2 IT OU processes, 2 AIP units, and 2 motor units.

Model fitting methods

To fit model parameters, we numerically optimized an objective function that accounts for errors in both FRs and behavioral data: (11) The first term in Eq (11) represents the error between simulated in the model and those in the data for each LIP unit (indexed by RF j), summed over the time interval [0, 800] ms from cue onset, and over the relevant RFs for the set size (SS). Seeking optima for all trial conditions, we summed errors between model results and data in response-hemifield congruent and incongruent cases, with the model’s LIP and IT areas receiving inputs as described in the Results section. Data for each LIP unit fit was computed by averaging FRs over the appropriate trials in which the recorded cell represented the corresponding LIP model unit (e.g., data from a trial simulated with left-facing ∃ in unit L1 was matched to averaged FRs of trials in which the target was in that RF; cf. Fig 6). Other model LIP units were matched to corresponding FRs of trials with distractors in their RFs.

The second and third terms describe the χ2 error for RTs on correct and incorrect trials, computed using the chi-square fitting method of [30] implemented in MATLAB using codes custom written by the authors (MS and SF). This uses the 0.1, 0.3, 0.5, 0.7, and 0.9 quantiles from the observed RT distribution to define six bins, of which the fastest and slowest each contain 10% of the total number of trials, and the central 4 each contain 20%. RTs predicted by model simulations were then collected in the same bins, and each χ2 is computed as: (12) Both congruent and incongruent trials were included in Eq (12), as described above for FRs, but separate histograms were fitted for correct and error RTs to account for accuracy.

The different physical units of neural and behavioral data are accommodated in Eq (11) by nondimensionalization, and weight coefficients αFR, αcorr, and αincorr are used to obtain comparable fit qualities for both data sets. Specifically, the values αcorr = αincorr = 1, and αFR = 0.25 were found to give all terms in Eq (11) similar orders of magnitude at starting points for the optimization algorithm. These values were used to produce all fitting results.

For each call to the objective function, we numerically simulated 6000 trials of the model using a stochastic Runge-Kutta algorithm implemented in custom software written in C and MATLAB by the authors (MS and SF). Temporal resolution was dt = 0.5 ms in all simulations. From these simulations we computed averaged FRs for each LIP unit and RT quantiles for correct and incorrect trials to generate the χ2 error of Eq (11). Good starting points were found by hand, mainly by adjusting parameters until model results were reasonably close to the data (poor starting points caused the optimization to terminate immediately). Given such points, the model was simultaneously fitted to data from M11 and M12 using a global optimization algorithm. As noted in Table 2, we constrained 18 of the 31 free model parameters to be equal for M11 and M12 and allowed 13—those we think most likely responsible for differences in the animals’ behaviors—to vary between them, for a total of 44 to be fit for the two monkeys.

We tried several optimization algorithms with limited success, including MATLAB’s fminsearch, fmincon, PSWARM and simulated annealing. The parallelized routine HOPSPACK [31] improved fine-tuning of parameter sets, although search resolution was coarse and convergence not as tight as we wished. A combination of simulated annealing and the bounded derivative-free optimization routine fminsearchbnd, both implemented in Matlab and available from the Matlab file exchange [32, 33], gave our best results as reported in the main text. We suspect that stochasticity in both model and data makes our objective function bumpy, resulting in poor convergence of the optimization routines. Use of efficiently-parallelized codes and many processors did not alleviate these difficulties.

Results

Before presenting and interpreting fits of the model to the data set described in “Analyses and Methods,” we outline our rationale for selecting the model’s architecture and components, since these are also results of our study.

Model construction

In constructing the model of Fig 6 we sought the simplest architecture consistent with the major findings of [14, 15], and in particular differences between the electrophysiological and behavioral data for M11 and M12 as described in “The covert search task” above. We summarize the main points here; model details appear under “A multi-area, multi-stage, accumulator model.”

A correct choice requires the monkeys to select the target, determine its orientation and release the appropriate paw. Although the FR data of Fig 4 suggests that LIP neurons encode correlates of all these processes, evidence from reversible inactivation experiments suggests that its primary role was in target selection [14]. In [34] LIP was inactivated in one hemisphere by muscimol injection. This impaired the monkeys’ ability to locate the target in the contralateral hemifield, but not their ability to respond with a specific paw, suggesting that LIP activity associated with the manual release may not play a central role in decisions, but reflect computations that are at least partially performed in other areas depending on the requirements of the task. Indeed, there is increasing evidence that other areas, including superior colliculus, prefrontal cortex, frontal eye fields and caudate, are involved in perceptual decisions, e.g. [1, 3537].

We therefore assume that LIP encodes target/distractor identities, but that orientation information is accumulated in a feature-selective area in visual cortex, and that both LIP and this visual area feed forward to a limb pre-motor area that in turn connects with motor cortex to generate the manual response. We make no specific claims about the identity of the visual and premotor areas, but model the former based on published reports about inferior temporal (IT) cortex or area V4, which are known to encode complex shapes (see e.g. [3842] and [43, 44]), and the latter on the anterior intraparietal area (AIP), which is anatomically adjacent to LIP and contains visual and grasp-responsive cells [45, 46]. We also assume that the orientation selection process in the feature-selective area (IT) occurs rapidly relative to target identification in LIP because orientation detection simply requires a choice between two non-noisy options.

In order to produce the target/distractor and congruence effect with target in LIP RF for M11 (Fig 4), we include recurrent excitatory and cross-inhibitory connections in the LIP pathway, but we keep the feature-selective pathway and premotor-motor connections purely feedforward and enforce bilateral (L/R) symmetry (Fig 6). We also exclude direct connections between LIP and the feature-selective area. Even with this (relatively) minimalist architecture the model has 31 free parameters, as noted in Table 2.

Fits to set size 4 data

We first describe the results of model fits to the set size 4 data for M11 and M12, and then discuss predictions made by these two models, with all parameters fixed, for set sizes 4 and 6.

We fit the model simultaneously to the set size 4 data for both monkeys, allowing 13 of the 31 parameters to vary across M11 and M12 as noted in Table 2. The model was run under both congruent and incongruent response-hemifield conditions and fitting errors computed from Eq (11). To simulate these conditions, we respectively input a rightwards facing 𝖤 to the upper right or left LIP unit in Fig 6. This configuration sufficed because data FRs were averaged over multiple RF configurations and model parameters are the same across both hemispheres. Different model LIP unit average FRs were then compared to corresponding FRs from data: e.g., in the congruent condition, the upper right LIP unit was compared with target in RF FRs, upper left and middle LIP units were compared with distractor in RF in hemifield opposite to target FRs, and the middle right LIP unit was compared with distractor in RF in same hemifield as target FRs. The lower two LIP units had no stimulus input and while they remain in the model for all set sizes, their FRs were excluded from the comparisons.

Table 3 shows the resulting parameter values for each monkey. Model accuracies and mean RTs are given in Table 4 in the same format as Table 1, along with the set size 2 and 6 predictions discussed below. Tables 5 and 6 compare model accuracy and mean RTs to each monkey’s data for all set sizes in congruent and incongruent conditions.

thumbnail
Table 5. Comparison of monkey 11 accuracy and mean reaction time data and model predictions.

https://doi.org/10.1371/journal.pone.0136097.t005

thumbnail
Table 6. Comparison of monkey 12 accuracy and mean reaction time data and model predictions.

https://doi.org/10.1371/journal.pone.0136097.t006

Fig 7 shows that the model captures the major features of the data. RT distributions and accuracy versus RT plots are well approximated for M12 in both conditions, excepting the last 4 bins in the incongruent condition (11.7% of responses), and M11’s accuracy is over predicted in the last 3 bins of the congruent condition (19.2% of responses). The model captures the downward trends in accuracy with increasing RT for both monkeys (Fig 7 rows 1 and 3), overestimating accuracy by 10% for M11’s incongruent trials, but only by 0.9—2.5% for the other 3 cases. Mean RTs are underestimated by 1.9—6.2% and overestimated by 4.5% for M11’s congruent trials (see set size 4 data in Table 5).

thumbnail
Fig 7. Model fits to the set size 4 data.

Model parameters are given in Table 3. M11 data and model fits are given in the left panels in rows 1–4, and in the first and second columns of rows 5–7. Similarly, M12 data and model fits are given in the right panels of rows 1–4, and in the third and fourth columns of rows 5–7. Rows 1–2 show accuracy and reaction time (RT) histograms with response-hemifield congruent inputs, while rows 3–4 show histograms for response-hemifield incongruent inputs. In accuracy versus RT plots, solid traces indicate data and circles denote model output; in RT histograms, blue bars indicate data and red bars show model output. Rows 5,6 and 7 respectively show firing rates (FRs) for response-hemifield congruent and incongruent inputs, with distractor in receptive field (RF) opposite to target hemifield, distractor in RF on same side as target, and target in RF. Model FRs are shown dashed and data as solid traces.

https://doi.org/10.1371/journal.pone.0136097.g007

Fig 7 (rows 5–7) shows that LIP FRs rise higher with target in RF than with distractor in RF, and overall FRs are higher in M11 than M12 on all conditions, as in the data, except with distractor opposite target. However, the model’s FRs decay too rapidly for M11 on congruent trials with distractor in RF opposite target and target in RF, and for distractor in RF on the same side as target on incongruent trials. FR decays are also too rapid for M12 with target in RF on both congruent and incongruent trials. Although RT’s are reproduced well, the model systematically overestimates accuracy, while generally underestimating FRs. At the end of this section and in the Discussion we note some modifications and additions to the model that might remedy these discrepancies.

As described above, 18 parameters were held equal across both monkeys, including time constants, several excitatory and inhibitory weights, some input strengths, and internal parameters for the AIP and motor areas (unbolded entries in Table 3). Three parameters that were allowed to vary differ by under 7.3% (star, clip and θ, also unbolded). The remaining 10 parameters significantly differentiate M11 and M12 as we now describe, first considering connection weights, input strength slip, and noise level cit.

Excitatory weights wlipaip (6.495 > 2.520), waiplip (6.276 > 1.166) between LIP and AIP and cross-inhibition βaiplip (8.422 > 4.266) from AIP to LIP are much stronger in M11 than M12, as is inhibition within LIP ηlip (2.936 > 0.1032; see boldface entries in Table 3 and network cartoons of Fig 8). The nonspecific LIP input slip is also stronger for M11 (0.6317 > 0.4088). In contrast, the excitatory weight from IT to AIP witaip is stronger in M12 than M11 (18.75 > 10.98). These differences produce higher LIP FRs for M11 than M12 in all six conditions, as shown in Fig 7 (rows 5 to 7).

thumbnail
Fig 8. Network cartoon illustrating fitted connection weights among IT, AIP and LIP.

Bold lines denote stronger fitted weights, predicting that monkey 11 has stronger connections between AIP and LIP, while monkey 12 has a stronger connections between IT and AIP.

https://doi.org/10.1371/journal.pone.0136097.g008

These parameter values provide insights into mechanisms that cause differences in LIP FR patterns and their relationship to behavioral differences observed between the two animals. Specifically, the relative dominance of the LIP-AIP pathway in M11 and the IT-AIP pathway in M12 account for the response-hemifield congruence effect noted earlier. On incongruent trials both AIP units receive target activation, one from IT and the other from LIP, causing competition, slower RTs and lower accuracy. Moreover, M11’s strong AIP-LIP back-excitation amplifies the influence of target location, and the substantially greater IT noise cit (1.465 > 0.6744) further degrades M11’s ability to decode orientation. In congruent trials, M11’s strong LIP-AIP re-entrant connections produce higher FRs in target RF of LIP; assisted by the IT-AIP pathway, these yield responses that are faster than M12’s and almost as accurate. In M12, the dominant IT-AIP pathway, moderate AIP-LIP cross-inhibition and weak AIP-LIP back-excitation favor target orientation for both congruent and incongruent trials, consistent with M12’s greater accuracy and uniformity over all conditions. See Tables 5 and 6 and compare Tables 1 and 4. Ratios of M11’s and M12’s connection weights witaip, wlipaip, waiplip and βaiplip and mutual inhibition ηlip within LIP are larger than those of slip and cit, suggesting that the former parameters are primarily responsible.

The delay parameters Pdelay, Tdelay and the nondecision time T0 also differ significantly between M11 and M12. These account for sensory and motor latencies and target search times (excluded from the model), and they allow estimation of decision times associated with task-specific processes. Summing the three values gives 250 ms for M11 and 189 ms for M12, and subtracting them from the mean experimental RTs of 444/537 ms (C/I, M11) and 492/488 ms (C/I, M12: see Tables 5 and 6), yields the values 195/288 ms (C/I, M11) and 303/299 ms (C/I, M12) for mean decision times predicted by the model. Thus, M12’s decision times and accuracies (≈ 96%) are similar on reponse-hemifield congruent and incongruent trials. M11’s decision process is faster overall, yielding similar accuracy with a mean decision time more than 100 ms shorter than M12’s on congruent trials. However, M11’s strong cross-inhibition reduces LIP activity on incongruent trials (Fig 7, row 7), lengthening RTs, and the weak IT-AIP pathway downgrades orientation information, reducing accuracy.

Model predictions for set size 2 and 6 data

We next ask if the model correctly predicts the set size effect, by simulating the task for set sizes 2 and 6, using the parameter values fitted to each monkey for set size 4 (Table 3). We simply feed the same input signals used above into only 2 or all 6 LIP units. Figs 910 show the results.

thumbnail
Fig 9. Set size 2 predictions of the model.

Using the model parameters fitted to the set size 4 data (Table 3), the input to model was changed to simulate the set size 2 condition. Figure layout analogous to that of Fig 7, but without firing rates for distractor in RF on same side as target, since distractor and target necessarily lie in opposite hemifields for 2 stimuli.

https://doi.org/10.1371/journal.pone.0136097.g009

thumbnail
Fig 10. Set size 6 predictions of the model.

Using the model parameters fitted to the set size 4 data (Table 3), the input to model was changed to simulate the set size 6 condition. Figure layout analogous to that of Fig 7.

https://doi.org/10.1371/journal.pone.0136097.g010

Set size 2 predictions show some strong qualitative similarities to the data, albeit with notable quantitative deviations. Model accuracies lie within 1.8% of data for both animals and response-hemifield conditions, but the model’s histograms in Fig 9 (rows 1 and 3) show long tails, while the data contain no RTs in these ranges (rows 2 and 4). Unlike set size 4, larger fractions of model trials in these tails (18—28.5%) cause substantial over prediction of mean RTs, which exceed the data by 20% (C and I, M11) and 3.6/11% (C/I, M12). The model underestimates M11’s LIP FRs with distractor in RF on congruent trials and overestimates them with target in RF on incongruent trials, but the other cases for M11 and all cases for M12 are quite well predicted (Fig 9, rows 5 and 6).

Set size 6 predictions of RT distributions and accuracy patterns are generally better than those for set size 2, and much better for congruent trials (Fig 10, rows 1–2). However, the model’s RT distributions are substantially skewed to faster responses for M11’s incongruent trials, accuracy is over predicted by 5.7% and 8.5% for M11 and M12 in this condition (rows 3–4), and M11’s accuracy on congruent trials is over predicted by 5%: see Tables 5 and 6. The model nonetheless captures the decrease in accuracy with RT for M11 although accuracy remains essentially flat for M12. Mean RTs are under predicted by 3.8/15% (C/I, M11) and 13.8/8.4% (C/I, M12). However, LIP FR predictions for M12 are poorer that those for set size 2: peak model FRs are all too high, those for target in RF peak too early and some FRs for M11 decay too slowly. See Fig 10, rows 5–7.

These predictions are encouraging, even given the prediction errors for set sizes 2 and 6. The model produces lower accuracies for larger set sizes and captures the qualitative effects of Fig 3, as shown in Fig 11 (top row). As set size increases, LIP FRs for both target and distractor in RF are successively suppressed due to mutual inhibition, although the relative magnitudes for M11 and M12 are reversed (probably due to M12’s much larger LIP inhibition, ηlip = 2.936 compared to 0.1032 for M11) and target and distractor FRs do not coalesce for M11 as quickly in Fig 11 as in the data of Fig 3. Recalling that back inhibition weights from motor units to LIP were constrained to be equal for both animals (Table 3), we conjecture that increasing βmitlip might improve the collapse of FRs.

thumbnail
Fig 11. Set size effects on FRs predicted by the model.

Model parameters are again given by the fit to the set size 4 data (Table 3). Top row: LIP firing rates (FRs) for monkey 11 (left) and monkey 12 (right) for the different set size conditions: set size 2 (red), set size 4 (green), and set size 6 (blue). Model LIP firing rates (FRs) were averaged over both congruent and incongruent conditions. Also, all LIP units with a distractor in the receptive field (RF) were averaged to produce the distractor FRs (dashed lines). Target in RF is given by the solid lines. Bottom row: FRs in left (solid) and right (dashed) AIP units for a right-facing 𝖤 in the left hemifield, for the incongruent response-hemifield condition only.

https://doi.org/10.1371/journal.pone.0136097.g011

The most striking qualitative discrepancy in the model results is the decrease in model RTs as set size increases, compared with the increase observed in the data. FRs of individual LIP units in the model do decrease with set size (Fig 11 (top row)), but model FRs in AIP modestly increase as more LIP units become activated, as shown in Fig 11 (bottom row; for clarity, we show only the incongruent condition). The resulting stronger AIP outputs drive motor units to cross threshold faster for larger set sizes. AIP FRs for M12 are similar in both conditions, but for M11 they peak at 65–75 Hz in the congruent condition (right-facing 𝖤’s in the right hemifield), significantly higher than in Fig 11 (bottom left) and much like M12’s FRs in the incongruent condition. This provides further evidence suggesting M11’s weak IT-AIP connections relative those of M12. The erroneous speed-up in decision time might be corrected by an additional inhibitory mechanism in either the motor or AIP units that modulates their FRs depending on set size. More simply, nondecision delays Tdelay could be allowed to depend on set size, and either fit or estimated by incorporating a model for visual search.

The model’s consistent over prediction of accuracy is also notable. Increasing noise levels cmaip, cit, clip may improve fits. We note that the current fitted values of IT and LIP noise levels cit, clip are both higher for M11 than M12 (Table 3), consistent with the animals’ overall accuracies.

The fits presented here were preceded by extensive work in which we fit data from all cells (including the six noted above that were excluded in the fits presented here) with the additional constraint that Pdelay and Tdelay should coincide for M11 and M12. A brief account of this appears in S1 Text. Using the optimization method described in Materials and Methods, we found parameters close to those described above and distinctions between connection strengths similar to those in Table 3 and Fig 8. This lends some confidence to the dominant pathway interpretation and other results presented above.

Discussion

We propose a model of a complex decision process based on interactions of brain areas involved in shape discrimination, visual attention and manual action selection (Fig 6), and we quantitatively fit the model parameters to detailed behavioral and electrophysiological data from two macaque monkeys. The model replicates key features of the data, previously described in [14, 15], including the response of neurons in the lateral intraparietal area (LIP) to target location, the effects of the number of visual stimuli (set size) and cue-hemifield congruence on LIP firing rates (FRs) and reaction times (RTs), and a counter-intuitive speed-accuracy trade-off displayed by both animals. In addition, using parameters fitted to set size 4 data, the model fits some but not all of the quantitative effects observed for set sizes 2 and 6 (Figs 710 and Tables 46).

The LIP responses recorded on this task differed in two important ways from those previously reported on simple decision paradigms. First, rather than increasing up to the time of the final decision as is the case when the decision is reported with a saccade (e.g. [3]), firing rates peaked close to the middle of the reaction time period and declined by the time of the manual response. Second, rather than encoding only visuo-spatial selection as was the case in previous paradigms, LIP firing rates showed additional sensitivity to a non-spatial variable: the limb used to report the shape discrimination. Our model suggests that both of these features reflect interactions between area LIP, which is involved in visuo-spatial selection [34], and skeletomotor areas involved in planning the manual release.

The decline of the LIP response was successfully modeled using an inhibitory signal that arises in the motor area and reduces the responses in both LIP and IT as a motor unit’s FR approaches and crosses threshold. Such a suppressive effect has been proposed as a general-purpose reset mechanism in previous decision-making models [28], and has been observed in EEG event-related potentials, where it was proposed to terminate the allocation of attention [29].

Interactions of visuo-spatial and manual responses, including the congruence effects that differed in the two monkeys, were captured by variability in the connections weights among the three areas, while the intrinsic properties of all areas remained similar for both animals (Table 3). Interestingly, the pronounced asymmetry of incongruent vs. congruent trials found in M11 did not require left-right asymmetry in the LIP-AIP pathways as may be a priori assumed, but emerged from competition between pathways connecting AIP to, respectively, LIP and IT. In M11, connection weights were strong for the excitatory AIP-LIP pathways, which are confined to a single hemisphere and can give rise to a bias toward congruent visuo-manual configurations, and weaker for the IT to AIP pathways, which do not encode information about target location and hence produce no congruent effect. In M12, by contrast, the IT to AIP pathway was dominant, resulting in a much weaker congruence effect. Therefore our findings suggest that differences in long-range connection strengths, whether dictated by anatomy or shaped through each monkeys’ learning of distinct strategies, produced the individual differences that are empirically observed.

As noted in the “Model construction” section of Results, our goal was not to exhaustively account for activity on the task, but to build a model with the simplest architecture that can capture the main effects in the data. It is instructive, however, to consider features that we have deliberately excluded but whose introduction may improve the fits. One such missing feature are feedback projections from the attentional to the feature-selective areas (e.g., LIP to IT), which are supported by studies of the visual and oculomotor systems and thought to mediate attentional effects (e.g. [43, 44, 47, 48]). A second feature is the absence of visual receptive fields in area AIP, which runs contrary to empirical observations (e.g. [4952]) and may account for the failure of our model to capture the behavioral set-size effects. In its current form the model makes the counterintuitive (and wrong) prediction that RTs will decrease with set size, which may be explained by the fact that the addition of distractors activates more receptive fields in LIP, which in turn converge on a single population of manual response units in AIP and speed up the process of action selection (Figs 911). This defect might be corrected by smaller receptive fields in AIP, which may have mutual inhibitory interactions and are topographically connected with those in LIP.

Optimization of parameter sets for the present study took several days on a 24 processor machine, and adding new features will further increase the time required by increasing the search space for the optimal parameter combinations. However, improvements in hardware, coupled with additional software optimization, are likely to render this a feasible goal, and allow us to identify parameters that provide better quantitative fits, provide more precise and confident inferences about differences between the monkeys, and permit analyses of model variants that explore other potential connection architectures within and between areas.

In sum, our work offers an example for multi-area analyses of cognitive processes that are partially constrained by behavioral and electrophysiological data, and could be extended for use with extracellular recordings, local field potentials or imaging data. Their construction could extend our understanding of decision making and its neural substrates to more realistic tasks, which engage multiple brain areas with re-entrant connections and can be solved through alternative, individual strategies.

Supporting Information

S1 Text. Notes on preliminary data fits and modeling.

https://doi.org/10.1371/journal.pone.0136097.s001

(PDF)

S1 Datasets. Electrophysiological and behavioral data from the covert search task [14, 15].

https://doi.org/10.1371/journal.pone.0136097.s002

(ZIP)

Acknowledgments

PH thanks Anitha Pasupathy for pointing out that V4 may be responsible for encoding object orientation.

Author Contributions

Conceived and designed the experiments: MAS SFF PJH JG JDC. Performed the experiments: MAS SFF. Analyzed the data: MAS SFF PJH JG JDC. Contributed reagents/materials/analysis tools: MAS SFF. Wrote the paper: MAS SFF PJH JG JDC.

References

  1. 1. Gold JI, Shadlen MN. Representation of a perceptual decision in developing oculomotor commands. Nature. 2000; 404:390–394. Available from: http://dx.doi.org/10.1038/35006062. pmid:10746726
  2. 2. Hanks TD, Ditterich J, Shadlen MN. Microstimulation of macaque area LIP affects decision-making in a motion discrimination task. Nat Neurosci. 2006; 9(5):682–689. Available from: http://dx.doi.org/10.1038/nn1683. pmid:16604069
  3. 3. Shadlen MN, Newsome WT. Neural Basis of a Perceptual Decision in the Parietal Cortex (Area LIP) of the Rhesus Monkey. J Neurophysiol. 2001; 86(4):1916–1936. Available from: http://jn.physiology.org/content/86/4/1916.abstract. pmid:11600651
  4. 4. Gold JI, Shadlen MN. Neural computations that underlie decisions about sensory stimuli. Trends Cogn Sci. 2001; 5(1):10–16. Available from: http://www.sciencedirect.com/science/article/pii/S1364661300015679. pmid:11164731
  5. 5. Mazurek ME, Roitman JD, Ditterich J, Shadlen MN. A Role for Neural Integrators in Perceptual Decision Making. Cereb Cortex. 2003; 13(11):1257–1269. Available from: http://cercor.oxfordjournals.org/content/13/11/1257.abstract. pmid:14576217
  6. 6. Rorie AE, Gao J, McClelland JL, Newsome WT. Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey. PLoS ONE. 2010; 5(2):e9308. Available from: http://dx.doi.org/10.1371/journal.pone.0009308. pmid:20174574
  7. 7. Usher M, McClelland JL. The time course of perceptual choice: The leaky, competing accumulator model. Psychol Rev. 2001; 108(3):550–592. pmid:11488378
  8. 8. Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD. The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. Psychol Rev. 2006; 113(4):700–765. Available from: http://www.ncbi.nlm.nih.gov/pubmed/17014301. pmid:17014301
  9. 9. Simen P, Contreras D, Buck C, Hu P, Holmes P, Cohen JD. Reward rate optimization in two-alternative decision making: Empirical tests of theoretical predictions. J Exp Psychol Human. 2009; 35(6):1865–1897.
  10. 10. Bogacz R, Hu PT, Holmes PJ, Cohen JD. Do humans produce the speed-accuracy trade-off that maximizes reward rate? Q J Exp Psychol. 2010; 63(5):863–891.
  11. 11. Purcell BA, Heitz RP, Cohen JY, Schall JD, Logan GD, Palmeri TJ. Neurally Constrained Modeling of Perceptual Decision Making. Psychological Review. 2010;117(4):1113–1143. pmid:20822291
  12. 12. Purcell BA, Schall JD, Logan GD, Palmeri TJ. From Salience to Saccades: Multiple-Alternative Gated Stochastic Accumulator Model of Visual Search. J Neurosci. 2012; 32(10):3433–3446. Available from: http://www.jneurosci.org/content/32/10/3433.abstract. pmid:22399766
  13. 13. Cohen JD, Romero RD, Servan-Schreiber D, Farah MJ. Mechanisms of spatial attention: The relation of macrostructure to microstructure in parietal neglect. J Cog Neurosci. 1994; 6(4):377–387.
  14. 14. Oristaglio J, Schneider DM, Balan PF, Gottlieb J. Integration of Visuospatial and Effector Information during Symbolically Cued Limb Movements in Monkey Lateral Intraparietal Area. J Neurosci. 2006; 26(32):8310–8319. Available from: http://www.jneurosci.org/content/26/32/8310.abstract. pmid:16899726
  15. 15. Balan PF, Oristaglio J, Schneider DM, Gottlieb J. Neuronal Correlates of the Set-Size Effect in Monkey Lateral Intraparietal Area. PLoS Biol. 2008; 6(7):e158. Available from: http://dx.doi.org/10.1371/journal.pbio.0060158. pmid:18656991
  16. 16. Gottlieb J, Mazzoni P. Action, Illusion, and Perception. Science. 2004; 303(5656):317–318. Available from: http://www.sciencemag.org/content/303/5656/317.short. pmid:14726577
  17. 17. Wickelgren WA. Speed-accuracy tradeoff and information processing dynamics. Acta Psychol. 1977; 41:67–85.
  18. 18. Deco G, Rolls ET, Albantakis L, Romo R. Brain mechanisms for perceptual and reward-related decision-making. Prog Neurobiol. 2013; 103:194–213. pmid:22326926
  19. 19. Ratcliff R. A theory of memory retrieval. Psychol Rev. 1978; 85(2):59–108.
  20. 20. Feng S, Holmes P, Rorie A, Newsome WT. Can Monkeys Choose Optimally When Faced with Noisy Stimuli and Unequal Rewards? PLoS Comput Biol. 2009; 5(2):e1000284. Available from: http://dx.doi.org/10.1371/journal.pcbi.1000284. pmid:19214201
  21. 21. Ratcliff R, Cherian A, Segraves M. A Comparison of Macaque Behavior and Superior Colliculus Neuronal Activity to Predictions From Models of Two-Choice Decisions. J Neurophysiol. 2003; 90(3):1392–1407. Available from: http://jn.physiology.org/content/90/3/1392.abstract. pmid:12761282
  22. 22. Shadlen M, Britten K, Newsome W, Movshon J. A computational analysis of the relationship between neuronal and behavioral responses to visual motion. J Neurosci. 1996;16(4):1486–1510. Available from: http://www.jneurosci.org/content/16/4/1486.abstract. pmid:8778300
  23. 23. Feng S. Extensions and Applications of Stochastic Accumulator Models in Attention and Decision Making. Ph. D. Thesis, Princeton University. 2012. Available: dataspace.princeton.edu/jspui/bitstream/88435/dsp01n870zq86t/1/Feng_princeton_0181D_10433.pdf.
  24. 24. Buschman TJ, Miller EK. Serial, Covert Shifts of Attention during Visual Search Are Reflected by the Frontal Eye Fields and Correlated with Population Oscillations. Neuron. 2009; 63(3):386–396. Available from: http://linkinghub.elsevier.com/retrieve/pii/S0896627309004905. pmid:19679077
  25. 25. Wong KF, Wang XJ. A Recurrent Network Mechanism of Time Integration in Perceptual Decisions. J Neurosci. 2006; 26(4):1314–1328. Available from: http://www.jneurosci.org/content/26/4/1314.abstract. pmid:16436619
  26. 26. Eckhoff P, Wong-Lin K, Holmes P. Dimension Reduction and Dynamics of a Spiking Neural Network Model for Decision Making under Neuromodulation. SIAM J Appl Dyn Syst. 2011; 10(1):148–188. Available from: http://epubs.siam.org/doi/abs/10.1137/090770096.
  27. 27. Miller KD, Fumarola F. Mathematical Equivalence of Two Common Forms of Firing Rate Models of Neural Networks. Neural Comput. 2011; 24(1):25–31. Available from: http://dx.doi.org/10.1162/NECO_a_00221. pmid:22023194
  28. 28. Simen P. Evidence Accumulator or Decision Threshold—Which Cortical Mechanism are We Observing? Front Psychol. 2012; 3:183. pmid:22737136
  29. 29. Sawaki R, Geng JJ, Luck SJ. A Common Neural Mechanism for Preventing and Terminating the Allocation of Attention. J Neurosci. 2012; 32(31):10725–10736. Available from: http://www.jneurosci.org/content/32/31/10725.abstract. pmid:22855820
  30. 30. Ratcliff R, Tuerlinckx F. Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychon B Rev. 2002; 9:438–481. Available from: http://dx.doi.org/10.3758/BF03196302.
  31. 31. Plantenga TD. HOPSPACK 2.0 User Manual (v. 2.0.2) Sandia National Labs; 2009. Available: sandia.gov/hopspack/HopspackUserManual_2_0_2.pdf.
  32. 32. Vandekerckhove J. General simulated annealing algorithm; 2008. Available: mathworks.com/matlabcentral/fileexchange/10548 MATLAB Central File Exchange. Accessed August 2012.
  33. 33. D’Errico J. fminsearchbnd, fminsearchcon; 2012. Available: mathworks.com/matlabcentral/fileexchange/8277 MATLAB Central File Exchange. Accessed August 2012.
  34. 34. Balan PF, Gottlieb J. Functional Significance of Nonspatial Information in Monkey Lateral Intraparietal Area. J Neurosci. 2009; 29(25):8166–8176. Available from: http://www.jneurosci.org/content/29/25/8166.abstract. pmid:19553456
  35. 35. Horwitz GD, Newsome WT. Representation of an abstract perceptual decision in macaque superior colliculus. J Neurophysiol. 2004; 91:2281–2296. pmid:14711971
  36. 36. Ding L, Gold JI. Separate, causal roles of the caudate in saccadic choice and execution in a perceptual decision task. Neuron. 2012; 75:865–874. pmid:22958826
  37. 37. Yates J, Katz L, Park IM, Pillow JW, Huk A. Dissociated functional significance of choice-related activity across the primate dorsal stream. In: COSYNE 2014 main meeting; 2014. Abstract T-35. Available: cosyne.org/cosyne14/Cosyne2014_-program_-book.pdf.
  38. 38. Gross C, Rocha-Miranda C, Bender D. Visual properties of neurons in inferotemporal cortex of the Macaque. J Neurophysiol. 1972; 35:96–111. pmid:4621506
  39. 39. Gross C. Representation of visual stimuli in inferior temporal cortex. Philos T R Soc B. 1992;335:3–10.
  40. 40. Deyns K, Vanduffel W, Frize D, Nielissen K, Peuskens H, Van Essen D, et al. The processing of visual shape in the cerebral cortex of human and nonhuman primate: a functional magnetic resonance imaging study. J Neurosci. 2004; 24:2551–2565.
  41. 41. Gross C. Single neuron studies of inferior temporal cortex. Neuropsychol. 2007; 45:841–852.
  42. 42. Schwarzlose R, Swisher J, Dang S, Kanwisher N. The distribution of category and location information across object-selective regions of visual cortex. PNAS. 2008; 105:4447–4452. pmid:18326624
  43. 43. Bisley JW, Goldberg ME. Attention, intention, and priority in the parietal lobe. Ann Rev Neurosci. 2010; 33(6):1–21. pmid:20192813
  44. 44. Squire RF, Noudoust B, Schafer RJ, Moore T. Prefrontal contributions to visual selective attention. Ann Rev Neurosci. 2013; 36:451–466. pmid:23841841
  45. 45. Fogassi L, Luppino G. Motor functions of the parietal lobe. Curr Opin Neurobiol. 2005; 15:626–631. pmid:16271458
  46. 46. Murata A, Gallese V, Luppino G, Kaseda M, Sakata H. Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J Neurophysiol. 2000;83:2580–2601. pmid:10805659
  47. 47. Reynolds JH, Heeger DJ. The normalization model of attention. Neuron. 2009; 61:168–185. pmid:19186161
  48. 48. Felleman DJ, Van Essen DC. Distributed hierarchical processing in the primate cerebral cortex. Cereb Cortex. 1991; 1:1–47. pmid:1822724
  49. 49. Borra E, Belmalih A, Calzavara R, Gerbella M, Murata A, Rozzi S, et al. Cortical connections of the macaque anterior intraparietal (AIP) area. Cereb Cortex. 2007; 18(5):1094–1111. pmid:17720686
  50. 50. Gamberini M, Passarelli L, Fattori P, Zucchelli M, Bakola S, Luppino G, et al. Cortical connections of the visuomotor parietooccipital area V6Ad of the macaque monkey. J Comp Neurol. 2009; 513(6):622–642. pmid:19235224
  51. 51. Lehmann SJ, Scherberger H. Reach and gaze representations in macaque parietal and premotor grasp areas. J Neurosci. 2013; 33(16):7038–7049. pmid:23595761
  52. 52. Borra E, Gerbella M, Rozzi S, Tonelli S, Luppino G. Projections to the superior colliculus from inferior parietal, ventral premotor, and ventrolateral prefrontal areas involved in controlling goal-directed hand actions in the macaque. Cereb Cortex. 2014; 24(4):1054–1065. pmid:23236207