Fig 1.
Partial reinforcement effect and the neural circuit models.
(A, B) During fear conditioning, a CS, e.g., a tone, was fully (in the full reinforcement schedule (A)) or partially (in the partial reinforcement schedule (B)) paired several times with a US, e.g., electric foot shock (left panels). The fear memory formed during fear conditioning can be diminished by extinction training, during which the CS is repeatedly presented alone, without the US (right panels). (C) Conditioned responses to the CS, which are usually measured as the degree of behavioral freezing responses, are depicted during fear conditioning and extinction. The fear memory (measured as the degree of behavioral freezing responses) that was acquired through the partial reinforcement schedule with P(US|CS)<1 exhibits a PREE (blue line), which is evident as increased resistance to extinction compared to that of the fear memory acquired through the full reinforcement schedule with P(US|CS) = 1 (black line). (D, E) The two neural circuit models are shown as schematics. Black and blue lines describe synaptic connections and the learning signals that regulate plasticity at synapses indicated by black open circles, respectively. (D) The basic model based on fear, persistent and extinction neural units (F, P and E). CS-related input activates all the units, and the extinction neural unit inhibits the fear neural unit, the activity of which represents the strength of the fear memory (black lines) (eqs (1–3)). The efficacy of CS-related input to the fear, persistent and extinction neural units changes based on the learning signals (blue lines) (eqs (4–6)). (E) Extended model including subregions of the amygdala (the LA, CEA and ITC) as well as the vmPFC. In this model, the LA and CEA correspond to persistent and fear neural units, respectively, and there are two extinction neural units: the ITC and vmPFC. CS-related input activates all subregions, and the ITC receives excitatory input from the vmPFC and inhibits the CEA (black lines) (eqs (7–10)). A behavioral fear response was triggered by the CEA. The efficacy of plastic synapses (black open circles) changed based on learning signals (blue lines) (eqs (11–14)). Parameter values are listed in S1 Table.
Fig 2.
Fear memory acquired during the full and partial reinforcement schedules in the basic model.
Basic model simulation results for the full and partial reinforcement schedules are presented in the left (A, C, E and G) and right (B, D, F and H) columns, respectively. (A, B) CS and US schedules during fear conditioning and extinction. (C, D) The black, blue and red lines indicate the activity of the fear, persistent and extinction neural units, respectively. (E, F) The black, blue and red lines indicate the synaptic weights of the CS-related inputs to the fear, persistent and extinction neural units, respectively. (G, H) The black, blue and red lines indicate the learning signals that changed the weights of CS-related synaptic inputs to the fear, persistent and extinction neural units, respectively. Note that overlapping lines were changed to dashed lines to make them visible. (I-L) The uncertainty of the next US observation (I), the time constant of fear memory decline (J), the relative fear-related neural activity at the conclusion of extinction (K) and the learning signal received by the extinction neural unit at the beginning of extinction (L) vary with the probability of the US. The time constant (J) was evaluated by fitting time course of the fear neural unit activity during extinction with F = F1exp(-t/τ)+F0, where F0 and F1 indicate positive constants, and t and τ indicate the number of extinction trials and the time constant, respectively.
Fig 3.
Fear memory in the extended model when the vmPFC was activated and silenced.
(A) Schedules of CS and US; the presentation schedules for the CS and US during fear conditioning and extinction were the same as those used in the basic model (Fig 2A), but the resting phase, during which neither the CS nor the US was presented, and the retrieval phase, during which only the CS was presented to evaluate extinction memory, were set after extinction. (B-F) Extended model simulation results for the control condition (B), vmPFC activation during extinction (C), vmPFC activation during retrieval (D), vmPFC silencing during extinction (E) and vmPFC silencing during retrieval (F). The blue, green, red and black lines indicate the activity of the LA (persistent neurons), vmPFC (extinction neurons), ITC (another group of extinction neurons) and CEA (fear neurons), respectively. Note that overlapping lines were changed to dashed lines to make them visible.
Fig 4.
Shock procedure in the extended model.
(A, B) CS and US schedules; after extinction training of the partially reinforced fear memory, an additional CS-US pairing was applied, with the US being three times stronger (A) or the same intensity (B). (C, D) The blue, green, red and black lines indicate the activities of the LA (persistent neurons), vmPFC (extinction neurons), ITC (another group of extinction neurons) and CEA (fear neurons), respectively. (E, F) The blue, green, red and black lines indicate the learning signals that changed the weights of CS-related synaptic input to the LA, vmPFC, ITC and CEA, respectively. Note that overlapped lines were changed to dashed lines to make them visible. (G) Final activity levels of the CEA (fear neural unit) after re-extinction training were plotted depending on the US intensity.