Fig 1.
fNIR100® headband and associated channels numbering.
Only the four closest detectors to an emitter constituted channels. The emitter-detector distance is 25 mm. Channels are represented in red with their associated number. The original image comes from the fNIRSOFT® manual and has been slightlty modified.
Fig 2.
Pilot’s interaction with the FCU.
The participants controlled the flight simulator from the pilot’s seat. The red rectangle corresponds to the FCU used to set the autopilot with four control knobs, according to ATC clearances (speed, heading, altitude, and vertical speed selection).
Fig 3.
ATC span task trial design.
Fig 4.
Illustration of the fNIRS based inference system.
Pre-recorded ATC messages were sent to the pilot (1). The pilot’s prefrontal activity was measured with a fNIRS device (2). Output measures (3) were MACD-filtered and synchronized with the temporal design of the trial (4). During the entire session, the MACD-based state estimator detected whether the pilot’s state was not-on-task or on-task (5). When all of the required data were available for the trial, a request was sent to the pilot’s classifier to assess the WM load of the trial (6).
Fig 5.
The experiment was split into three successive phases.
Data gathering (phase D) and classifier testing (phase T) consisted of 20 ATC instructions each. The pilot’s classifier was trained between these two phases (phase L). The time scale of the figure is illustrative.
Fig 6.
Example of real-time state estimation (performed on pilot 16).
The upper graph shows MACD-filtered fNIRS signal and the signal line computed from the latter (dashed line). The two lower graphs show the participant’s state estimated from crossovers between MACD and signal lines and the operator’s actual state, respectively.
Fig 7.
Activation maps according to the level of difficulty.
Units are in μmol.l−1. Both high and low load conditions elicit bilateral DLPFC activities. The high load minus low load subtraction map (High—Low) shows significantly greater activation of the right DLPFC. Activations shown 14 s post-stimulus onset. p < 0.001. fNIRSOFT® software (www.biopac.com/fNIR-Software-Professional-Edition) was used to produce this figure.
Fig 8.
Off-line estimated onset and offset latencies compared to the stimuli onset, in low WM load and high WM load conditions.
Average for 20 trials per difficulty, on 19 pilots’ results. ***: p<0.001.
Fig 9.
Machine learning result: WM Load level estimation accuracy for each participant.
Fig 10.
Trial timeline and computing latencies.
The upper timeline shows ATC span task trial events duration (see Fig. 3). Bottom timeline illustrates duration constraints to get pilot’s estimated WM load: classifier’s response is available in the worst case less than 3.3s after pilot’s response window.