Virtual navigation tested on a mobile app (Sea Hero Quest) is predictive of real-world navigation performance: preliminary data

Virtual reality environments presented on smart-phone and tablet devices have potential to aid the early diagnosis of conditions such as Alzheimer’s dementia by quantifying impairments in navigation performance. However, it is unclear whether performance on mobile devices can predict navigation errors in the real-world. In a preliminary study we tested 30 participants (15 female, 18-30 years old) on their wayfinding ability in our mobile app ‘Sea Hero Quest’ and on novel real-world wayfinding task in London (UK). We find a significant correlation between virtual and real-world navigation performance and a male advantage on both tasks, although smaller in the real-world environment. These results are consistent with prior studies which have reported navigation in the virtual environments are predictive of real-world navigation performance and a consistent male advantage. Future research will need to test a larger sample size and older participants


Introduction
Virtual reality (VR) provides a powerful mean to study how humans navigate, because the properties of virtual environment can be completely controlled and repeated across participants. Testing neuropsychological patients with VR and recording brain activity during VR have allowed numerous important insights to be gained in understanding how brain regions support navigation [6]. VR also provides the potential to act as an early stage diagnostic tool for Alzheimer's dementia (AD), because spatial disorientation is one of the earliest symptoms [10,16,18,22]. Currently there is no standardized test for navigation deficits with AD patients. Until recently most VR used in research was presented on a desk-top display and movement controlled via a joystick or keyboard. Such an interface presents difficulties for people who are in the age range where AD may likely onset (50-70 years old) [8]. However, with the advent of tablet and smart-phone touch screen mobile devices, old participants have found engaging in VR tasks much easier [19]. We recently developed a VR navigation task for mobile and tablet devices -Sea Hero Quest [4]-with aim that this may provide an early diagnostic for AD. For this test to be useful it is important that it has real-world validity, with errors on the VR task predicting errors in real-world navigation experience.
Past research comparing navigation in real and VR environments has generally found a good concordance in performance across both tests [1-3, 5, 11, 14, 15, 17, 20, 24]. They have also found a male advantage for navigation in VR tasks [4,7,9,13] and in real-world tasks [12,21]. However, prior studies comparing VR and real-world performance have used desk-top VR or immersive VR to simulate environments. Thus it remains unknown whether virtual navigation performance, as measured on mobile or tablet device, is also correlated with real-world navigation behavior. Moreover, when assessing 'real world navigation behavior', most prior studies actually used paper and pencil tests such as line orientation, road map, or delayed recall. A few authors designed actual navigation tasks but often in a limited spatial range, like the lobby of a hospital [5]. In this study we designed an ecological navigation task in a large environment covering a whole neighborhood of London.

Methods
The experiment was split into 2 main parts, see Figure 1. First, participants were tested on specific wayfinding levels from Sea Hero Quest [4] on a tablet. Second, they were tested on a real world wayfinding task in London. The whole experiment lasted around three hours.

Participants
We tested 30 participants (15 males), aged 18-30 y.o. (M= 21.96, s.d.= 2.55). We discarded data from two participants (one male and one female) due to recording problems. Participants had normal or corrected to normal vision and gave their written consent to participate. The study was approved by UCL ethics committee. Participants received 3 class credits or £20 for their participation.

Virtual task
We devised a mobile video game designed to measure human spatial navigation ability through gameplay -Sea Hero Quest (SHQ) 1 . This video game involves navigating a boat in a virtual environment (lake or river networks) and has been extensively described in [4]. In brief, the main levels of the game require participants to view a map displaying current position and future goal locations to find ( Figure 2). Participants could study the map without time restrictions and had to navigate to the goal locations in the order indicated, e.g. goal 1 must be found first, then goal 2, etc. Goals were buoys with flags marking the goal number. The task is complete when all goals have been located. If the participant takes more than a set time, an arrow indicates the direction along the Euclidean line to the goal to aid navigation. On basis of the data from the online version, we selected a subset of 6 of the total 75 levels in the game that varied in difficulty: instruction level 1, and wayfinding tasks levels 6, 8, 11, 16, 43 (see Figure 2).

Real world task
Akin to the virtual task, our real-world navigational task consisted of 6 wayfinding trials which varied in difficulty in terms of the number of streets to be navigated, the number of goals and the relative location of the goals to each other ( Figure 3). Each trial was located in a different street network in the Covent Garden London neighborhood. We chose less busy streets to avoid traffic and made sure the participants were not familiar with them. Before each trial, participants were shown a map that only indicated the facing direction, the network of the local streets and the location and the order of the goals (Figure 3). The goals were doors and gates with distinct features (e.g. specific colour, size, or material). Participants had up to 1 min to memorize the map (maximum length for Sea Hero Quest) before walking to located the goals. During navigation they were provided colour photographs of the goal location doors. Based on pilot testing we set specific time limits for each route. Route one: 6 minutes, route two: 6 minutes, route three: 6:30 minutes, route four: 6:30 minutes, route five: 12 minutes, route six: 14 minutes. We chose these time limits to allow for a few mistakes at a reasonable walking pace. Pilot testing indicated that if participants required any longer than that these times they were likely guessing and had failed to remember the goal locations or street layout. If participants reached the limits of the defined region set by the experiment they were told by the experiment they had reached the edge of the search area and should turn back.

Navigation performance measures
Virtual task -Performance was quantified with the distance travelled in each level (in pixels). We averaged the distance travelled over levels 6 to 16. We did not include level 1 because it did not require any spatial ability (the goal was visible from the starting point) and was only designed to assess participants' ability to learn to control the boat. We did not include level 43 because it was too difficult with only 40% of participants finishing it (see also Figure 5). The coordinates of participants' trajectories were sampled at Fs = 2 Hz. Real World task -Because we were not able to reliably measure the metric distance walked we  Figure 4. Spatial ability at a wayfinding task in real world vs virtual environment chose to quantify performance using the number of wrong turns. Wrong turns included going into streets that led away from the goal, waling beyond the perimeter set by the experiment and being ask to turn round. Across the trials we summed all of the wrong turns to provide a score for each participant.

Results
Performance data for the two tasks was close to being normally distributed (Kolmogorov-Smirnov test for the real-world performance: p = 0.46, for the virtual environment performance: p = 0.06). We compared participants' performance in the game and in real life and found a positive correlation between the number of wrong turns in our real-world task and the travelled distance in Sea Hero Quest (r = 0.62, p < 0.001), see Figure 4. We broke down this global correlation score for each Sea Hero Quest level, comparing participants' performance in real life with the distance they travelled in levels 1-6-8-11-16-43, see Figure 5. The correlation coefficient is close to 0 in level 1, confirming that this instruction level does not measure spatial ability. It then increases with level difficulty, before dropping in level 43, suggesting that this level is too hard and participants performed homogeneously poorly.
We also found that in both tasks male participants had an advantage. For comparison purposes we scaled the real world and the virtual environment performance between 0 (worst) and 1 (best) and found that the gender difference is stronger in the virtual environment (M ± SE males: 0.54 ± 0.02, females: 0.39 ± 0.05, Cohen's d = 0.98) than in the real world environment (males: 0.67 ± 0.06, females: 0.61 ± 0.07, Cohen's d = 0.22), see Figure 6.  Figure 6. Gender differences in virtual and real environments. Performance have been scaled from 0 (worst) to 1 (best). Dots correspond to individual data points. Error bars correspond to 95% confidence intervals.

7
. CC-BY 4.0 International license available under a not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which was this version posted April 22, 2018. ; https://doi.org/10.1101/305433 doi: bioRxiv preprint wayfinding task. Given that patients with Alzheimer's Disease experience real-world wayfinding problems early in the disease onset [10,16,18,22], our results provide a useful indication that our potential test has real-world ecological validity. Our findings are consistent with a number of studies that have compared real-world and VR navigation [1-3, 5, 11, 14, 15, 17, 20, 24], and extend the findings to tablet device presentation and real world spatial task spanning complex street networks. This is a useful step since it increases the ability to remotely test people. This is particularly valuable when certain categories of the population have difficulties in mobility, like older people. Currently our results focused on young university students. It will be useful to extend to a broader population including older participants. Based on this preliminary data and our online data, we will be able to predict the average number of wrong turns they will make in our real-world task from their Sea Hero Quest test scores (using the levels selected here). An important next step will be to extend this preliminary data sample to a larger sample and use another environment to replicate these findings in. Future analysis using GPS tracking could potentially provide a more fine-grain analysis of errors that goes beyond our current measure of wrong turns. Similarly in future research it will be useful to examine how the performance relates to questionnaires about self-rated navigation ability, as has been done in recent work [23]. In a follow up study it will also be interesting to examine how performance in other levels of Sea Hero Quest (path integration levels and spatial working memory levels [4]) relate to this newly developed real-world equivalent task performance.
. CC-BY 4.0 International license available under a not certified by peer review) is the author/funder, who has granted bioRxiv a license to display the preprint in perpetuity. It is made The copyright holder for this preprint (which was this version posted April 22, 2018. ; https://doi.org/10.1101/305433 doi: bioRxiv preprint