Fig 1.
Illustration of the naturalistic vs. scrambled visual stimuli.
Example frames from the intact movie (A) and the scrambled movie (B). The scrambled movie was created by shuffling the phase of the intact movie in the frequency domain (C).
Fig 2.
Brain activations with the intact movie were found at regions that showed significant intra-subject (A) and inter-subject (B) correlations in cortical activity during free movie watching. The mapping results were based on data from nine subjects. From a single subject, the fMRI signals from two voxels within the primary visual cortex (V1) and the lateral occipito-temporal gyrus (V4) are shown as examples to illustrate the intra-subject reproducibility in cortical activity. The color indicates the cross correlation.
Fig 3.
Cortical activations with the scrambled movie were reduced and confined to V1, as revealed by the intra-subject (A) and inter-subject (B) reproducibility of the fMRI signal. The mapping results were based on data from nine subjects. From a single subject, the fMRI signals from two voxels within the primary visual cortex (V1) and the lateral occipito-temporal gyrus (V4) are shown as examples to illustrate the relatively low intra-subject reproducibility in cortical activity. The color indicates the cross correlation.
Fig 4.
Effects of varying gaze behavior on cortical reliability when watching the scrambled movie.
(A) For most subjects (except one), the horizontal and vertical variations of the gaze location were more consistent for the intact (non-scrambled) movie than for the scrambled movie. Error bars represent the standard deviation across subjects. (B) For an example subject, when the subject did not fixate at the screen center, there was a lack of reliable responses (C) The scrambled movie induced reliable responses within V1 when they fixated at the screen center. In both (B) and (C), the color indicates the intra-subject cross correlation in voxel time series (*: p < 0.0005, **: p < 0.05, ***: p < 0.03). These results were based on eye-tracking data from nine subjects who freely watched both the intact and scrambled movies.
Fig 5.
Varying gaze behavior contributed to fMRI activity and its reproducibility during natural movie stimulation.
(A) Cross correlation between the fMRI signal and the time-varying saccade amplitude during free movie viewing (p < 0.011, FDR < 0.03). (B) The upper panel shows the difference (without regression minus with regression) in intra-subject correlation when taking the four gaze behaviors as the nuisance variables (p < 0.03, uncorrected). The lower panel shows the difference in intra-subject correlation when taking free-viewing the intact movie as the experimental group (n = 9) while taking the eyes-fixated movie watching (n = 6) as the control group (p < 0.03, uncorrected). The color shows the difference in cross correlation following the t-test.
Fig 6.
Dependence of cortical fMRI activity and its reproducibility on scene transitions during natural vision.
(A) A scene transition (video shot boundary) occurred as visual input changed abruptly due to film editing. (B) A train of scene-transition events were defined by the timing and degree of the transition evaluated by the total motion change between two adjacent movie frames. (C) Cross correlations were calculated between the fMRI signals and the regressor derived from the scene-transition events (p < 0.0017, DOF = 17, FDR < 0.03). Representative time courses from one subject are shown to demonstrate the similarity between the scene-transition predicted response and the fMRI signal from a voxel in the ventral visual pathway (D) Comparison between the intra-subject reproducibility without (left) vs. with (right) regressing out the scene-transition induced response. Their difference (bottom) was evaluated and shown to depict the effects of scene transitions on the reproducibility of cortical activity during natural vision. The color shows the correlation coefficient or the difference in correlation coefficient, following the t-test (p < 0.001, FDR < 0.03).