Fig 1.
Each study has its own VR content, providing participants with the proper environment for cybersickness or locomotion. We collected eye-tracking, head, and physiological data from the cybersickness study and collected eye-tracking, head, waist, and ankle data from the locomotion study. We used the same model architecture and analysis methods for both studies.
Fig 2.
Left: cybersickness study, Right: locomotion study.
Table 1.
Data features used in each study.
Two studies used the same features from the HMD. The physiological sensor was used in the cybersickness study, and the features from the tracker were used in the locomotion study.
Fig 3.
Model architecture used in two studies.
The model mainly consists of the attention-based individual network and the attention-based LSTM subnetwork. The cybersickness study has four classes, and the locomotion study has five classes (N means time t, which is t = 1,2,…,N).
Table 2.
The model performance by architecture (cybersickness).
Our model yielded the highest performance.
Fig 4.
Attention weights of data modalities.
Table 3.
Ablation study for cybersickness (addition of eye-tracking).
Eye-tracking data already shows a strong relationship with cybersickness, and when it is considered with other data modalities, the performance of the model significantly increases. Gain indicates the increased ratio of the model performance after the addition of the eye-tracking data.
Table 4.
The model performance by architecture (locomotion).
Our model yielded the highest performance.
Table 5.
Ablation study for locomotion (addition of eye-tracking).
Using eye-tracking data only shows little influence on locomotion. However, when eye-tracking data are combined with other data modalities, they served as a strong supplement to improve the performance of the model.