Fig 1.
A. Gaze at partner’s eyes: Real Eye condition. Partners viewed each other at an eye-to-eye distance of 140 cm. The eye regions subtended by both the real eyes and the video eyes were 3.3 × 1.5 degrees of visual angle (red boxes). Small green LED indicator lights located to either side of their partner indicated rest and diverted gaze targets. B. Gaze at eyes in video: Video Eye condition. Two 24-inch 16x9 monitors were placed between the participants and a size-calibrated, pre-recorded dynamic video of a face was presented in the same field-of-view as the live interaction. C. Diagram of the Real Eye condition, with participant and lab partner sitting 140 cm apart from each other and LED indicator lights placed 10 degrees to the left and right of the Eye. D. Diagram of the Video Eye condition, with monitors arranged between partners. The face and LED sizes and positions were calibrated to subtend the same visual angles in both conditions.
Fig 2.
A. Time course. The duration of the run was three minutes and each run was repeated twice for both the Real Eye and Video Eye conditions. Each run included six alternating 15-second task and rest periods. In task periods (blue bars), participants alternated their gaze in three-second epochs between the eyes and the left or right lighted LED (See Fig 1C and 1D). During the 15-second rest period, participants looked only at the lighted LED. The task is similar to those used in previous experiments (Hirsch et al., 2017; Noah et al., 2020). B. Eye tracking traces of eye-to-eye contact. Red traces represent eye movements from a participant with ASD; blue traces represent the eye movements of a lab partner. The eye tracking data acquired on the Tobii system provides a frame-by-frame (8 ms) binary value that indicates whether or not eye gaze was directed within the eye-box of the partner. The blue dashed line (top) represents the duration of eye gaze (number of frames) that the lab partner’s gaze was within the eye-box of the participant. Similarly, the red dashed line (bottom) represents the duration of gaze (number of frames) that the participant’s eye gaze was in the eye-box of the lab partner. The green dashed line (middle) represents the length of time (number of frames) that the eyes of both partners were simultaneously focused within each other’s eye-boxes for a minimum of 83 ms. This is taken as a measure of eye-to-eye contact between the participant and the lab partner. C, D. Gaze performance, Real Eye condition. Eye contact epochs (3 s) are indicated as E; Diverted gaze epochs (3 s) are indicated as D; and Rest periods (15 s) are indicated as R. The color bar (top) indicates the percent of time eye gaze was within the eye box of the partner. C. Example eye tracking report for one TD participant and lab partner pair. D. Example eye tracking report for one participant with ASD and the lab partner pair. Similar computations were performed for all participants when eye tracking data were acquired.
Fig 3.
A. Examples of average participant eye gaze positions when viewing the face of the lab partner. The red box illustrates the target “eye box” and the color gradient from red to green indicates percent of “target hits” in the eye box for an entire run. B. fNIRS channel layout. Right and left hemispheres of a single rendered brain illustrate median locations (blue dots) for 58 channels per participant. Montreal Neurological Institute (MNI) coordinates were determined for each channel by digitizing emitter and detector locations in relation to anterior, posterior, dorsal, and lateral fiduciary markers based on the standard 10–20 system.
Fig 4.
A, B. Marginal means plots based on linear mixed effects models for each measure by participant group (Blue: Participants with Autism Spectrum Disorder (ASD); Red: Typically-developed (TD) participants). A. Dwell time duration of eye contact on either the eyes in the video (Video Eye condition) or eyes of the lab partner (Real Eye condition). B. Standard deviation of horizontal gaze trajectory normalized by duration of contact. Error bars show SEM. ***p≤0.001, *p≤0.05. C, D. Pupil diameter variations observed in participants with ASD and TD participants during Real Eye (C) and Video Eye (D) gaze conditions (Blue: Participants with ASD; Red: TD participants). Black lines indicate points at which pupil diameter differences between groups were significant at p≤0.05.
Fig 5.
Contrast comparison [Real Eye > Rest], typically-developed (TD) participants relative to participants with autism spectrum disorder (ASD).
TD participants (red clusters) show comparatively greater activation in dorsal somatosensory cortex (SSC); supramarginal gyrus (SMG); angular gyrus (AG); pre- and supplementary motor cortex (MC); and extrastriate visual cortex (V3), while heightened activity was observed for participants with ASD (blue clusters) in relatively ventral MC; SSC; pars opercularis (Pars O) and pars triangularis (Pars T); posterior middle temporal gyrus (pMTG); superior temporal gyrus (STG); and auditory cortex (AC). See Table 1. Yellow and light blue indicate responses corrected for multiple comparisons using FDR at p≤0.05. GLM analyses are based on the combined OxyHb and deOxyHb signals.
Table 1.
GLM contrast comparison: [Real Eye] > [Rest] (deOxyHb + OxyHb signals), TD group—ASD group.
Fig 6.
Contrast comparison [Real Eye] > [Rest] modulated by the number of frames within each 3-second eye viewing period where the gaze of both partners was simultaneously within the eye box of the other.
A. Typically-developed (TD) participants. Activity observed in the right hemisphere: supramarginal gyrus (SMG); somatosensory association cortex (SSAC); dorsolateral prefrontal cortex (DLPFC); frontal eye fields (FEF); and pre- and supplementary motor cortex (MC). See Table 2A. Note: n = 15 rather than 19 (see S2 Table) because eye tracking data could not be acquired on four participants. B. Participants with Autism Spectrum Disorder (ASD). Activity observed in the right hemisphere include SMG; angular gyrus (AG); extrastriate visual cortex (V3); visual association cortex (V2); and DLPFC. See Table 2B. Note: n = 12 rather than 17 (see S1 Table) because eye tracking data could not be acquired on five participants. Yellow indicates signals corrected for multiple comparisons at p≤0.05 using FDR. GLM analyses are based on the combined OxyHb and deOxyHb signals.
Table 2.
A. GLM Contrast comparison: [Real Eye] > [Rest] with eye contact covariate (deOxyHb + OxyHb signals), TD group.
B. GLM Contrast comparison: [Real Eye] > [Rest] with eye contact covariate (deOxyHb + OxyHb signals), ASD group.
Fig 7.
Main effect neural results for participants with autism spectrum disorder (ASD) during real eye contact with ADOS-2 (Autism Diagnostic Observation Schedule, 2nd edition) scores as a covariate.
OxyHb and deOxyHb signals are combined. Blue colors indicate a negative relationship between neural responses and ADOS scores indicating that as symptom severity increases, neural responsiveness in these regions decreases. Light blue indicates responses corrected for multiple comparisons using FDR at p≤0.01. Pars O: pars opercularis; Pars T: pars triangularis; DLPFC: dorsolateral prefrontal cortex; SSAC: somatosensory association cortex (or SPL: Superior Parietal Lobule); SMG: supramarginal gyrus; AG: angular gyrus; and V3: Extrastriate Visual Cortex (Area V3). n = 12 instead of 17 because the analysis includes only those participants with usable eye-tracking data. See Table 3.
Table 3.
GLM contrast comparison: [Real Eye >Rest] with ADOS-2 covariate (deOxyHb + OxyHb signals).
Fig 8.
Participants with autism spectrum disorder (ASD) (numbers correspond to S1 Table) during eye contact, median beta values vs. ADOS-2 (Autism Diagnostic Observation Schedule, 2nd edition) scores.
The median hemodynamic signals (beta values, y-axis) within the responsive brain regions (Fig 7 and Table 3) and ADOS-2 scores (x-axis) are shown for each participant with ASD. The main effect of eye-to-eye contact is negatively correlated with fNIRS signals in A. right hemisphere visual association cortex (Area V3) and angular gyrus (r = -0.76) as well as B. somatosensory association cortex (also referred to as the superior parietal lobule, SPL) and supramarginal gyrus (r = -0.92); however, due to variations in optode coverage over the SPL (because of differences in head size and the superior-medial location of the target area), the number of participants with sufficient data for this regional analysis is small (n = 6). A negative correlation between individual median hemodynamic signals and ADOS-2 scores for [Real Eye > Rest] was also found in C. right dorsolateral prefrontal cortex, pars triangularis, and pars opercularis (r = -0.77).
Fig 9.
Main effect neural results for participants with autism spectrum disorder (ASD) and typically-developed (TD) participants during eye contact with SRS-2 (Social Responsiveness Scale, second edition) scores as a linear covariate.
OxyHb and deOxyHb signals are combined. Blue colors indicate a negative relationship between neural responses and SRS-2 scores, which suggests that increased symptom severity is associated with reduced regional neural responsiveness (See Table 4). Light blue indicates responses corrected for multiple comparisons using FDR at p≤0.01. SSC: somatosensory cortex, SSAC: somatosensory association cortex, and SMG: supramarginal gyrus.
Table 4.
GLM contrast comparison: [Real Eye>Rest] with SRS-2 covariate (deOxyHb + OxyHb signals).
Fig 10.
Participants with autism spectrum disorder (ASD) (blue numbers) and typically-developed (TD) participants (red numbers) during eye contact vs. Social Responsiveness Scale (second edition, SRS-2) scores.
The median hemodynamic signals (Beta values, y-axis) within the responsive brain region (Fig 9 and Table 4) and SRS-2 scores (x-axis) are shown for each participant. The main effect of eye-to-eye contact is negatively correlated with fNIRS signals in right hemisphere somatosensory cortex, somatosensory association cortex, and dorsal supramarginal gyrus (r = -0.58). Numbers indicate individual participants shown in S1 (participants with ASD) and S2 Tables (TD participants).
Fig 11.
Cross brain neural coherence between angular gyrus during Real Eye and Video Eye conditions for participants with ASD and TD participants.
The x-axis shows the wavelet components (s) plotted against coherence, the correlation of the components across participants. The inset key indicates the four conditions of interest: TD Real Eye (red); TD Video Eye (dark blue); ASD Real Eye (pink); and ASD Video Eye (light blue). Solid lines represent the mean coherence and shading indicates the standard error. The signal is the combined OxyHb and deOxyHb, and the total number of dyads is 36.