Immersive Virtual Reality (IVR)

The core of IVR is not only virtual content itself. It is the tightly linked perception-action loop created by HMDs, spatial tracking, and multi-channel interaction [18–21]. This allows IVR to support high-risk and hard-to-reproduce experiments, as well as scenario-based training. It also creates opportunities for state sensing, performance evaluation, and dynamic adjustment during learning [22–24]. From a sensing perspective, however, these advantages also bring more complex system requirements. Learning tasks, body movements, and virtual scenes happen at the same time. As a result, sensors need to record continuously while still maintaining comfort and stability [18–24].

For this reason, studies on physiological signal-based identification and modeling in IVR learning should report how the sensing system was implemented. They should not focus only on the final algorithmic results [15–17].

Physiological signals

In IVR learning research, common physiological signals can be grouped into three categories. The first is central nervous system signals, which reflect brain activity, such as EEG and fNIRS. The second is peripheral physiological signals and derived indicators, which reflect autonomic activity, such as PPG/rPPG, HR/HRV, and GSR. The third is eye-tracking data, which reflect visual attention and information processing paths [25–31]. These three categories provide different types of information about cognitive processing, arousal and emotion, and visual behavior [25–31]. Compared with self-report, they are better suited to capturing dynamic changes during IVR interaction. However, they are also affected by motion artifacts, contact quality, sampling stability, and the alignment of scene events [32–36].

Therefore, the value of physiological signals in IVR research depends not only on what is measured. It also depends on how signals are measured, how they are synchronized with the task, and how raw signals are turned into reliable analytical results [32–36].

AI for physiological signal analysis in IVR learning: Role and constraints

AI, especially machine learning and deep learning, provides useful tools for extracting information from complex physiological signals in learning and training processes [37–41]. This information may relate to states, attributes, performance, and response patterns. However, under IVR conditions, the meaning of model outputs still depends strongly on the stability of front-end sensing, the quality of artifact management, multimodal alignment, and clear train-test boundaries [42–48]. In other words, even when studies report similar classification accuracy, differences in HMD integration, synchronization strategy, or train-test split strategy can affect how the results should be interpreted and how well they may transfer to other settings.

This suggests that AI-based physiological signal research in IVR learning is fundamentally a systems issue. The reliability of the results depends on sensing system implementation, signal processing, and validation design together. It does not depend only on back-end classifier comparison [42–48].

Existing reviews in related fields mainly follow three lines. One group focuses on the overall picture of IVR or VR educational applications [2,4,15,49]. Another group discusses wearables, physiological sensing, or AI analysis in education [7,12,17,50]. A third group focuses on physiological measurement and wearable biosensing in immersive experiences [8,51]. These reviews provide important background. However, several issues are still not well connected. These issues include which physiological modalities are used in IVR learning and training, how these modalities are integrated with HMDs and task events, how signals are cleaned, represented, fused, and modeled, and how much reported model performance is affected by label construction and train-test split strategies.

To show how this review differs from earlier work, Table 1 summarizes the focuses and boundaries of representative reviews. Compared with recent reviews that are closer to this topic [12,50,51], this review does not focus mainly on wearable physiological data and AI in general educational settings. Instead, it focuses on IVR learning and training. It compares studies in terms of HMD integration, calibration and synchronization, artifact management, feature engineering and representation learning, fusion strategy, label source, train-test split strategy, potential data leakage risk, and deployment readiness.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of representative related reviews and the positioning of this study.

https://doi.org/10.1371/journal.pone.0355596.t001

Based on these differences, this study uses a scoping review to map research on physiological sensing systems and AI-based signal analysis in IVR learning and training since 2016. This period is important because it marks a key stage in the commercialization of VR hardware. The review focuses on IVR settings. It uses sensing system implementation, signal processing, and validation as its main analytical lines. More specifically, it examines how physiological sensing systems were built, how signals were cleaned and represented, how multimodal data were fused, how models were validated, and how these studies were used to analyze and model states, attributes, performance, and related response patterns. It also examines how close these studies are to near-real-time or real-time closed-loop deployment. Based on this scope, the review addresses the following research questions:

1. RQ1: Which physiological signals were collected in IVR learning environments?
2. RQ2: How was AI used to analyze these physiological signals?
   1. RQ2a: Which AI techniques and methods were used?
   2. RQ2b: At which stages of the sensing and analysis pipeline were these AI techniques applied, such as acquisition, calibration and synchronization, preprocessing and artifact management, feature engineering, representation learning, fusion strategy, validation design, and deployment?

By addressing these questions, this review aims to provide two types of guidance for future research. First, it may help researchers choose appropriate physiological modalities, quality-control strategies, and AI methods. Second, it may provide more specific guidance for system design, including HMD integration, calibration and synchronization, label construction, participant-level validation, leakage control, and deployment readiness.

Inclusion and exclusion criteria

The inclusion criteria were as follows: (1) publication date between January 1, 2016, and March 29, 2026. The year 2016 was chosen as the starting point because commercial HMDs such as the Oculus Rift and HTC Vive entered the mainstream around this time, marking a period of rapid growth in immersive virtual reality research [56]. (2) The study should be empirical and should have collected at least one physiological signal in an immersive virtual reality learning or training context. Collection could be through wearable devices or through a clearly described and validated contactless approach. (3) AI techniques were actually used in the physiological signal analysis pipeline, including but not limited to machine learning classification, regression, clustering, deep learning-based representation learning, or fusion modeling. For studies that combined IVR physiological data collection with external-dataset model benchmarking, eligibility was based on whether the AI-based physiological analysis was explicitly developed as part of the IVR learning study; the use of an external benchmark dataset was recorded separately. (4) Eligible document types included peer-reviewed journal articles, conference papers, and full-text theses or studies available in institutional repositories. Because this topic is an emerging intersection of sensing, learning analytics, and VR, some technical implementations first appear in conference papers, theses, or institutional repository outputs. These sources were therefore retained to avoid missing early evidence [52–54]. However, publication type was treated only as a descriptive variable for evidence source and was not used as a direct proxy for methodological quality. No language filter was applied during database searching; however, only reports available in English were eligible at the screening stage. The exclusion criteria were as follows: (1) the study was not set in an IVR learning/training context; (2) it only discussed AR, MR, desktop VR, or other non-immersive environments; (3) it collected behavioral data only and no physiological signals; (4) it used AI but not for physiological signal analysis; or (5) the full text could not be obtained.

Information sources and search strategy

Seven database searches were conducted: Web of Science Core Collection, Scopus, IEEE Xplore Digital Library, ACM Digital Library, SAGE Journals, ProQuest Dissertations & Theses Global, and a combined EBSCOhost search of Education Source and ERIC. Searches were conducted up to March 29, 2026. To balance sensitivity and topical focus, a two-stage search strategy was used. In the first stage, search strings were built around three core concepts—immersive virtual reality, physiological signals, and education/training contexts—and adapted to the fields and syntax of each database. The core search string was as follows: [(‘virtual reality’ OR ‘virtual environment’ OR ‘immersive virtual reality’ OR ‘immersive environment’ OR ‘VR’ OR ‘IVR’) AND (‘physiological signal’ OR ‘EEG’ OR ‘ECG’ OR ‘EDA’ OR ‘GSR’ OR ‘HR’ OR ‘HRV’ OR ‘PPG’ OR ‘EMG’ OR ‘RSP’ OR ‘Eye* track*’ OR ‘eye movement’) AND (‘education’ OR ‘educational’ OR ‘learner*’ OR ‘student*’ OR ‘teacher’ OR ‘instructor’ OR ‘learning’ OR ‘training’)]. The complete database-specific search strategies, including database platforms and collections, fields searched, limits, search dates, records retrieved, and record-management procedures, are provided in S2 Table.

AI-related keywords were not used as mandatory terms in the first-stage search because terminology in this field is highly heterogeneous. Some studies use broad labels such as machine learning or deep learning, some report only specific model names such as SVM, RF, CNN, or LSTM, and others refer only to tasks such as classification, prediction, or recognition in the methods or results. Adding AI terms too early could therefore exclude relevant studies whose terminology is scattered. For this reason, the first stage aimed to retrieve physiological measurement studies in IVR learning/training as completely as possible, and the second stage then determined at the full-text level whether AI was actually used for physiological signal analysis. This helped balance sensitivity and specificity. Study screening was conducted independently by two reviewers. The title, abstract, and full text of each record were assessed independently by at least two researchers. When disagreements occurred, two additional authors reviewed the record and consensus was reached through discussion.

Study selection process

Study selection consisted of three levels: deduplication/initial filtering, title and abstract screening, and full-text assessment. Following the predefined strategy, 3650 records were identified across the seven databases. After database deduplication and an initial screening of document types, 1932 candidate records remained. A progressive title-abstract-full-text screening procedure was then applied. This left 521 records after title screening and 116 records for full-text assessment after abstract screening.

During full-text assessment, 15 studies that were not conducted in an immersive virtual reality learning or training context were excluded, as were 2 studies whose full texts could not be obtained. After this stage, 99 studies directly related to physiological signal measurement in IVR learning/training settings remained.

In the second stage, these 99 full-text studies were further assessed to determine how AI was actually used, and the fit with the IVR learning/training definition was checked again. At this step, 77 studies that did not use AI, 2 studies that used AI but not for physiological signal analysis, and 1 study that did not meet the IVR learning/training definition after full-text review were excluded. This left 19 included studies. Fig 1 shows the screening process. The reports not retrieved and the reports excluded after full-text assessment, together with the primary reason for each decision, are listed in S2 Table.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Study selection process (PRISMA-ScR).

https://doi.org/10.1371/journal.pone.0355596.g001

Data extraction and organization framework

Based on the research questions and the framework shown in Table 2, we extracted information on physiological signal types, front-end acquisition/calibration/synchronization, device models, sampling rates, preprocessing/artifact management, feature engineering, representation learning, fusion strategies, model families, AI task types, target constructs, and sample size. Where a study used an external benchmark dataset, the IVR sample and the external benchmark sample were recorded separately when identifiable. We also recorded evidence source, label source, train-test split strategy, potential data leakage risk, deployment status, and interpretability reporting to compare the interpretability, transferability, and practical feasibility of the reported systems. Processing steps such as SMOTE, oversampling, undersampling, and data augmentation were coded separately as sample-level processing and were not treated as signal quality control.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Data extraction and organization framework.

https://doi.org/10.1371/journal.pone.0355596.t002

In addition, the included studies were organized at two levels. The first was the sensor/system level, including IVR scenario type, sensing implementation, integration with HMDs and tasks, device model, sampling rate, whether calibration or synchronization was reported, and the main artifact management steps. The second was the validation/deployment level, including target constructs, label sources, train-test split strategies, potential data leakage risk, deployment status, and interpretability reporting. In Table 3, “signal configuration” was coded only according to the physiological modalities included in each study. Auxiliary non-physiological features such as head/hand tracking, controller/posture data, text semantics, and clickstream data were not counted in the row labels and were instead described separately in the “System and validation notes” column of Table 4. When a study reported more than one AI task or more than one explicit validation design, these were recorded in parallel. As a result, some counts in Table 3 and S1 Table may exceed the total number of studies. Train-test split strategy and data leakage risk were judged only from information clearly reported or directly identifiable in the original text. When it was not possible to confirm whether train-test boundaries were separated across participants or sessions, the study was coded as “not reported.”

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Study counts and method distributions by signal configuration.

https://doi.org/10.1371/journal.pone.0355596.t003

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Overview of physiological sensing systems, processing pipelines, and validation in the included studies.

https://doi.org/10.1371/journal.pone.0355596.t004

RQ1: Which physiological signals were collected in IVR learning environments?

Among the 19 included studies (Table 3), when each signal type was counted once per study, eye-tracking data (ETD) was the most widely used modality (13 studies, 68.4%), followed by central nervous system signals (CNS; 11 studies, 57.9%). Peripheral physiological signals (PPS) were used much less often (3 studies, 15.8%). In AI-supported immersive virtual reality (IVR) learning environments, no studies were found that mainly focused on common autonomic signals such as electrodermal activity (EDA). It is also notable that 8 of the 19 studies (42.1%) used combinations of signals. CNS + ETD was the most common combination (6 studies, 31.6%), whereas CNS + PPS and ETD + PPS each appeared in only 1 study. Overall, within the included studies, single-modality and dual-modality designs were more common than designs collecting all three physiological signal categories.

RQ2: How was AI used in physiological signal analysis?

Sensor layer: Coverage of wearable/embedded modalities, HMD integration, and scenario fit

In terms of modality coverage, ETD (13/19) and CNS (11/19) were the two most frequently reported sources in the included IVR learning studies. This pattern is closely related to hardware integration. Eye tracking is often built into the HMD or closely linked to head pose. This makes it suitable for capturing task-related information such as gaze allocation, object focus, and scene navigation [70,72,73]. By contrast, EEG has been used to infer cognitive states such as cognitive load, concentration, and distraction, but such inferences require caution in IVR because EEG recordings are sensitive to head movement, oculomotor activity, muscle activity, wearing stability, and electrode contact quality [57,60,63]. Therefore, although both eye tracking and EEG are used in IVR physiological sensing, they differ clearly in hardware burden, comfort, motion tolerance, construct interpretation, and the difficulty of HMD integration. Current studies still do not report these system-level differences in sufficient detail [57,60,63,70,72,73].

PPS appeared in only 3 included studies. These studies mainly used cardiovascular sensing sources and derived indicators such as PPG/rPPG and HR. As a result, emotional arousal, stress responses, and autonomic activity are still only lightly covered. In addition, existing multimodal studies are not the same as multi-physiological sensing systems. Besides dual physiological combinations, many studies combined one physiological signal with interaction data such as head and hand tracking, text semantics, clickstreams, or quiz responses [66,70,72,73]. This pattern may indicate an emerging interest in multi-source modeling within the included studies. However, more complete wearable or embedded physiological combinations are still uncommon. This is especially true for peripheral modalities such as EDA, which could add useful information about emotion and arousal. There are also still few systematic comparisons of comfort, synchronization cost, and robustness across modalities in HMD settings.

Signal-pipeline layer: Calibration and synchronization, artifact management, and multimodal fusion

At the signal-pipeline level, the included studies already cover the main stages of acquisition, preprocessing, feature engineering and representation learning, fusion, and analysis and modeling. However, most studies report algorithmic results more clearly than the sensing pipeline behind them. In preprocessing, EEG studies often used ICA to remove ocular or muscle artifacts. By contrast, eye-tracking studies more often used smoothing, baseline correction, interpolation, invalid-sample removal, and standardization [57,60,63,71,73]. These steps suggest that researchers are aware of the effects of head movement, occlusion, tracking loss, and multi-device interference on signal quality in IVR. However, only a limited number of studies reported calibration and synchronization in a systematic way. A few studies provided clearer front-end details. For example, Abdurrahman et al. [59] reported 50 Hz eye tracking and 500 Hz heart-beat recording. Study [67] reported Meta Quest Pro with 2048 Hz EEG acquisition. Studies [71] and [73] reported Vive Pro Eye at 120 Hz and pre-session calibration. Study [27] described automatic annotation synchronization between VR stimuli and EEG. Even so, most studies gave only brief information on device models, sampling rates, timestamp alignment, pre-session calibration, and synchronization between sensors and HMD or task events. As a result, even when studies are grouped under labels such as “EEG+ETD” or “ETD+interaction features,” their acquisition conditions may still differ in important ways.

An additional issue concerns the construct validity of EEG-based cognitive-load inference in IVR. Frontal or midline theta has often been found to associate with working-memory load and task demand [76,77], but in IVR it should not be interpreted as a direct or stand-alone marker of cognitive load. Head movement, changes in electrode contact, ocular activity, and scalp or neck muscle activity may co-vary with task difficulty and interaction style [78]. Therefore, high classification accuracy alone is insufficient to establish that this EEG feature captures cognitive load. Future IVR studies should strengthen convergent validity through explicit EOG/head-motion/EMG artifact checks, task-load manipulations, behavioral performance and subjective workload measures, and participant- or session-level generalization tests, as well as combining EEG features with other modalities such as eye-tracking indicators. Under such convergent evidence, frontal theta may be treated as supportive evidence of cognitive-load-related processing; without it, EEG-based labels should be interpreted more conservatively as context-specific discriminative patterns.

In the included studies, feature-engineering, representation-learning, and fusion methods were often relatively simple to implement. EEG and EEG + ETD studies more often used deep models such as CNN, CNN-LSTM, or feature imaging [27,57,60,61]. By contrast, ETD single-modality studies more often used sliding-window statistical features, FRESH/tsfresh, or manual feature engineering [69,71,73]. At the fusion level, early fusion was the most frequently reported strategy among the 12 fusion studies. Late fusion appeared in only 1 study, and no study used intermediate fusion or hybrid fusion. Within this review sample, multi-source features were often combined directly at the input level. One reason may be the need to reduce system complexity in small-sample settings. However, this approach does not address cross-modal temporal alignment, representation learning, or uncertainty propagation in a systematic way. In addition, some studies collected two physiological signals at the same time but did not truly integrate them at the feature level [30,64]. This means that multimodal acquisition and multimodal fusion are not the same in the current literature.

In terms of validation design, label sources, and deployment readiness, the included evidence was concentrated mainly in offline classification studies. Studies using only traditional machine learning accounted for 13 of the 19 included studies. Classification appeared in 17 studies. Regression, clustering, and prediction each appeared only once. Anomaly detection did not appear in any study. In addition, label sources in supervised tasks were highly heterogeneous. Some labels were based on experimental task conditions. Others were based on scores or performance. Some came from self-report, expert or existing diagnosis, or participant attributes. A few were based on physiological proxy indicators. If the one unsupervised clustering study is excluded, among the 18 supervised or regression studies, 4 were coded as low risk for data leakage, 5 as high risk, 8 as not reported, and 1 as scheme-dependent. These findings suggest that, in this review sample, AI-based analysis was mainly used for offline discrimination of experimental data. Its use in real-time, continuous, closed-loop IVR adaptation under strict generalization testing is still limited.

Reproducibility and deployment layer: From algorithmic experiments to deployable sensing systems

From the overall reporting pattern in the included studies, one recurring issue was not simply whether AI was used, but whether the sensing system was described clearly enough to support reproduction, comparison, and deployment. Most studies reported the use of EEG, ETD, or PPS. However, reporting was still uneven for device models, sampling rates, sensor placement, mechanical and logical integration with the HMD, calibration procedures, synchronization methods, validation design, and whether the analysis was offline, near-real-time, or real-time. This problem is especially clear in validation design. Some studies clearly reported sample-level, participant-level, or session-level splits [27,57,58,60,63,67,69,71–73]. However, in many other studies, the train-test boundaries were still not reported clearly enough. In this type of research, such information is essential for interpreting model performance. Only when the acquisition chain, processing chain, and validation chain are all described clearly can findings be compared more directly and moved closer to practical deployment.

Future research should treat signal-pipeline reproducibility and model performance as joint priorities. In addition to expanding PPS coverage and combining multiple physiological modalities, the field may benefit from a minimum reporting set. At a minimum, studies should report the HMD and sensor models, signal acquisition positions, sampling rates, calibration and synchronization methods, the main artifact-management steps, whether sample-level processing was applied only after data splitting, label sources, train-test split strategies, especially whether they were participant-level, session-level, or external-test designs, the stage at which fusion occurred, and whether the system supports near-real-time or real-time closed-loop operation.