Fig 1.
Costs and benefits of considering sequential information in learning and decision making.
Costs and benefits of considering sequential information in learning and decision making. a: Parameter description for the model. b: The utility function U(ℓ, T) visualized for sample values of T with n set to 12. c: Optimal decision depth ℓ when T and n vary. In both (a) and (b) r is set to 0.5 τ is set to 10. For visualization of the effect of variation in r and τ, see S1 File.
Fig 2.
Performance of Depth-ℓ representations of stimulus sequences in environments of different sizes.
The x-axis represents the time-steps or learning opportunities and the y-axis represents the performance measured after a given number of time-steps, as described in the methods section. a: Learning in an environment consisting of 20 different stimuli. b: Learning in an environment consisting of 500 different stimuli. In both environments, the rate of increase of information with respect to the increase of ℓ is 0.5 (approximating the parameter setting r = 0.5 in the analytical model). However, the information increase ceases when ℓ > 4, as we are only including Depth-1 − 4 representations in the simulations. The learning rate in the simulations approximates τ = 10 in the analytical model.
Fig 3.
Performance of Depth-ℓ and trace representations of stimulus sequences in environments that vary in the temporal distribution of information.
The number of stimuli (including informative and uninformative stimuli) is 66 in all environments. The trace decay rate θ = 0.5. The x-axis represents the time-steps or learning opportunities and the y-axis represents the performance measured after a given number of time-steps, as described in the methods section. a: Examples of environments in which productive decisions depend on the last stimulus only (top) or on the last two stimuli (bottom). ✲ indicates uninformative stimuli selected at random for each pattern; ● and ◯ indicate stimuli whose identity determines the correct output. 1 and 0 indicate whether a response is productive or not. b: Learning in an environment of 32 sequences in which only the last stimulus is informative. c: Learning in an environment of 32 sequences in which all four temporal positions are equally likely to be informative. d: Learning in an environment of 32 sequences in which only the first of the four temporal positions is informative.
Table 1.
Sequential order and meaning in language.
Fig 4.
Summary of representation strategies.
This illustrates how an input sequence (A, B) is represented differently by four strategies, and thus generates different representations on which each respective decision on response is based. The Trace strategy represents B and also a trace of A that has faded in intensity from 1 to 0.5 according to the decay rate θ = 0.5. The Depth-1 strategy only represents B at the time of decision. The Depth-2 and Flexible Sequence strategies represent A and B with full strengths and their order, at the time of decision. The Depth-2establishes a unique representation of the full sequence (A, B). The Flexible Sequence strategy establishes the same representation of the sequence (A, B) but also represents sub-sequences, here the single stimuli, thus enabling decision making based on any of these representations.
Fig 5.
Performance of Flexible Sequence, Depth-4 and Trace representations, in environments with varying proportions of sequentially structured information.
For the Flexible Sequence and Depth-4 representations ℓ = 4. For the trace representation θ = 1/2. The probability of encountering information in sequences is determined by p in each environment. Sequential information is contained in the two sequences (A, B) and (B, A) that are equally distributed over the three time steps where they fit. All other information is in single stimuli and is equally distributed over the four time steps. The x-axis represents the time-steps or learning opportunities and the y-axis represents the performance measured after a given number of time-steps, as described in the methods section. a: Learning in an environment where information is encountered in sequences with p = 0 and all information thus is in single stimuli. b: Learning in an environment where information is encountered in sequences with p = 0.25. c: Learning in an environment where information is encountered in sequences with p = 0.5. d: Learning in an environment where information is encountered in sequences with p = 0.75. e: Learning in an environment where all information is encountered in sequences.