The 3D hype: Evaluating the potential of real 3D visualization in geo-related applications

Vojtěch Juřík; Lukáš Herman; Dajana Snopková; Adrianne John Galang; Zdeněk Stachoň; Jiří Chmelík; Petr Kubíček; Čeněk Šašinka

doi:10.1371/journal.pone.0233353

Abstract

The use of 3D visualization technologies has increased rapidly in many applied fields, including geovisualization, and has been researched from many different perspectives. However, the findings for the benefits of 3D visualization, especially in stereoscopic 3D forms, remain inconclusive and disputed. Stereoscopic “real” 3D visualization was proposed as encouraging the visual perception of shapes and volume of displayed content yet criticised as problematic and limited in a number of ways, particularly in visual discomfort and increased response time in tasks. In order to assess the potential of real 3D visualization for geo-applications, 91 participants were engaged in this study to work with digital terrain models in different 3D settings. The researchers examined the effectivity of stereoscopic real 3D visualization compared to monoscopic 3D (or pseudo 3D) visualization under static and interactive conditions and applied three tasks with experimental stimuli representing different geo-related phenomena, i.e. objects in the terrain, flat areas marked in the terrain and terrain elevation profiles. The authors explored the significant effects of real 3D visualization and interactivity factors in terms of response time and correctness. Researchers observed that the option to interact (t = -10.849, p < 0.001) with a virtual terrain and its depiction with real 3D visualization (t = 4.64, p < 0.001) extended the participants’ response times. Counterintuitively, the data demonstrated that the static condition increased response correctness (z = 5.38, p < 0.001). Regarding detailed analysis of data, an interactivity factor was proposed as a potential substitute for real 3D visualization in 3D geographical tasks.

Citation: Juřík V, Herman L, Snopková D, Galang AJ, Stachoň Z, Chmelík J, et al. (2020) The 3D hype: Evaluating the potential of real 3D visualization in geo-related applications. PLoS ONE 15(5): e0233353. https://doi.org/10.1371/journal.pone.0233353

Editor: Vincent Lecours, University of Florida, UNITED STATES

Received: February 23, 2020; Accepted: May 4, 2020; Published: May 21, 2020

Copyright: © 2020 Juřík et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The dataset and analysis script supporting the conclusions of this article are available in the Zenodo repository [https://doi.org/10.5281/zenodo.3679376].

Funding: Research team (P.K.) was supported by funding from the Masaryk University project “Research on Changes in Geographical Processes and Relations in Space and Time” (Grant No. MUNI/A/1356/2019; https://www.muni.cz/en/research/projects/52458). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

3D visualizations are being increasingly used in a number of applied areas for data visualization. Since these visualizations allow three-dimensional perception of graphical content, they are considered a promising tool for a range of applications in geo-sciences, for example, teaching geography and cartography [1–5], urban planning [6–8], crisis management [9–12], precision agriculture [13], visibility analysis [14], virtual tourism [15], navigation in built-up areas [16, 17], indoor navigation [18, 19] and others. Different forms of 3D visualization may encourage different types of human behavioural and cognitive responses, i.e. they can affect human sensorimotor and interaction strategies, cognitive processing, and ultimately, human performance. The aim of this study was to evaluate important factors contributing a role in 3D geovisualizations, specifically 3D factors and interactivity factors in the context of different geo-related tasks. In this study, 3D geovisualization is understood as a three-dimensional visual representation of the real world, its parts, or as a representation of the spatially referenced data [20]. The 3D geovisualization may be of a dynamic nature, so it allows for changes of depiction based on the user-computer interaction. Regarding this, 3D geovisualization allows the user to focus on its specific parts or aspects from various positions, perspectives, and other functionalities (see [21]). In this study, authors explored participants response times and accuracy of answers in different forms of geovisualization, specifically focusing on the level of interactivity and 3D visualization type, as stated in detail below in the research hypotheses.

3D visualization types

The types of 3D visualizations vary in the principles that visualizations are built on and the technologies used to display them. The most typical types of 3D visualizations are pseudo 3D visualizations (also known as weak or 2.5D visualizations) and the less common real (or strong) 3D visualizations [22, 23]. Pseudo 3D visualizations depict a scene which is displayed perspective-monoscopically on flat media, such as computer screens or widescreen projection. These scenes are composed solely with the use of monocular depth cues [23]. Real 3D visualizations engage both monocular and binocular depth cues (especially binocular disparity cues) in order to achieve stereoscopic vision. Stereoscopy is a technique using stereopsis [24] to separate the visual signals individually perceived by each eye and present them through a peripheral device, for example, 3D shutter glasses [25]. The slight differences in perspective in image between the left and the right eye are registered and combined to create a 3D representation of the observed scene. Real 3D vision was developed to enhance perception of spatial attributes in an observed scene, i.e. the distances and relative positions of objects in a visual field [26, 27], and to increase depth perception in a scene [28]. Real 3D technology provides a different number of visual cues than pseudo 3D, and therefore, it is expected that the displayed content is processed differently with respect to the visualization type. Hence, real 3D technology has also been proposed as a more effective tool for virtual geovisualization since it provides additional spatial information [1, 29, 30], even though no clear standards for the production and use of real 3D visualizations in geo-sciences are available [22, 23, 31]. Despite the added visual cues in real 3D visualization, the effectiveness and efficiency [32, 33] of stereoscopic 3D visualization in applied issues remains disputed, especially regarding increased response time to solve tasks in a real 3D environment, increased cognitive load and user distraction, neglect of important objects in the scene, or significant visual discomfort while wearing peripheral devices, such as 3D glasses or helmets [23, 28, 34–36]. Several empirical studies support the effectiveness of real 3D visualization [37–40], although their practical use, especially in geo-related issues, remains controversial [30, 41].

Interactivity factor

The shift from static maps to interactive versions is a natural step, as technological progress permits and drives it. Interactivity in current geospatial data visualizations is starting to become available for users to navigate or otherwise work with data to obtain required information and acquire optimal situation awareness [42, 43]. In the present paper, the researchers focused on navigational (or viewpoint) interactivity, which was discussed in previous studies [43–46], and which forms the core functionality of applications such as Google Earth and other virtual globes. Navigational interactivity in 3D space usually includes functions such as rotation, panning and zooming for spatial information acquisition further promoting e.g. spatial orientation [46]. Previous studies [19, 45, 47–49] suggested a difference between static and interactive scenes in perception and cognitive processes, and respectively emphasized the importance of the specific task type [45]. Interaction with a geovisualization helps complete the spatial information about the scene as the scene is moved and presented from various points of view. Interaction provides a collection of perspectives, and hence the mental model of the scene can be more rapidly processed by the observer, as previously discussed by [36, 50, 51]. Other authors have suggested that interaction only improves task solving in a limited manner since users need not be able to use it properly [52]. Regarding this, the present study questions the role of 3D geovisualizations in the context of interactive and static tasks and suggests that real 3D visualization can be possibly substituted with the option to control visualized content. The summary of the available depth cues present in various settings is shown in Fig 1.

Download:

Fig 1. Scheme of the additional depth cues present in various settings.

https://doi.org/10.1371/journal.pone.0233353.g001

Geographical task factor

A number of studies explored simple 3D geovisualizations [36, 45, 53, 54]. Other studies explored complex stimuli similar to real-world settings [4, 19, 55]. Previous findings suggested that the role of 3D technology in geovisualizations is questionable, especially regarding the specific features of the user interface (interactivity factor, type of task, etc.). Research in geo-applications emphasizes the role of the specific task since there is very limited option to control the equivalence of stimuli—a real map is a unique combination of spatial aspects in which it is difficult to find equivalent stimuli. For this reason, the authors developed an experiment that uses three types of tasks as experimental stimuli (i.e. point, surface and elevation profile structures in the virtual terrain), which represent different geo-related phenomena, such as buildings, water bodies and roads. Participants were asked to determine the elevation of objects (cubes), water bodies and the presented terrain profiles in the virtual terrain. These three task types possessed different spatial complexity and represented different spatial structures. As spatially simple and complex stimuli, the cubes and terrain profiles, respectively, had already been tested in previous studies [36, 49] and were complemented in the present study with semi-complex “surface” stimuli, i.e. water bodies.

Research aims

The present study explored the effectivity of real 3D geovisualization in static and interactive tasks. Effective form does not refer to the simplest/easiest form but to one that offers users sufficient information necessary to make a proper choice and prevent mistakes while also facilitating quick response, i.e. it balances effectiveness (score) and efficiency (speed). Quesenbery [56] suggested three general aims that every user interface should follow: (a) effectiveness in achieving determined objectives, usually completing the task (i.e. correctness), (b) efficiency in control or use (i.e. speed and ease), and (c) the operator’s satisfaction in working with such a device. Regarding these points, the suitability of a specific user interface setting can be operationalized and tested. The present study examined the first two above-mentioned points, i.e. goal achievement and speed, as these can be operationalized into objective performance scores and response times in the suggested geo-related tasks. As mentioned in more detail above, previous studies have suggested that real 3D visualization is potentially troublesome and may cause visual discomfort, prolong solution time or promote neglect of important objects in a scene. The aim of this study was to assess the effect of 3D visualization on goal achievement and performance speed under different 3D conditions with different tasks. One of the main questions was the role of real 3D technology in the context of interactive geographic tasks, in which navigational interactivity [43] was suggested as being able to potentially compensate for missing binocular cues (as discussed in [36]). Regarding this, the authors of the present study observed user performance in real 3D, and respectively, pseudo 3D visualizations, both in static (i.e. non-interactive) and interactive environments. The present study is specific in several ways. Compared to the previous research of [28, 57], the authors inspected the effect of interactive 3D visualizations in virtual models of real, existing geographical areas. Previous studies also engaged a limited number of participants, generally only up to 20 people [58–60], thereby limiting general conclusions. Furthermore, the present study emphasized the process of interaction with terrains generated by the participants themselves and avoided the use of automated computer-generated movement.

The research hypotheses were:

Tasks in a real 3D environment require a longer time to be solved,
Tasks in an interactive environment require a longer time to be solved,
Tasks in a real 3D environment result in more correct answers,
Tasks in an interactive environment result in more correct answers.

Methods

Participants

The previous study [36] suggested that the 3D visualization type had a significant effect in identifying an object’s elevation in a static (non-interactive) 3D geovisualization (Cohen’s d = 1.06). The authors of the present study considered the experiment’s design (e.g. computer screens instead 3D wide-screen projection) and other observations concerning similar issues [49]. Only a relatively mild effect was expected. Regarding this, 91 participants were engaged in the experiment. Since the study examined the topic of perception, volunteers, specifically humanities students with no or minimal previous training in geo-visualization from Masaryk University (17 males, 75 females), aged 19 to 53 years (m = 23.46 years; med = 23 years; sd = 4.9), were involved. Course-takers of Masaryk University’s Experimental Humanities course, which is held annually at the Faculty of Arts, were invited via email. By participating in the experiment, they were rewarded with 16 points towards the above-mentioned course. Before the experiment, all participants were questioned about visual impairments and any other possible medical limitations and informed that they could remove themselves from the experiment at any time. The study was approved by the Masaryk University Ethics committee for the research, identification number of the project: EKV-2016-059. Participants in the study gave their consent by means of a written consent form.

Procedure and materials

The entire experiment was fully computerized. The participants used conventional desktop PCs with 27-inch 3D monitors and wore active shutter 3D glasses (NVIDIA 3D Vision^® 2 Wireless Glasses, 60 Hz on each eye). Regular optical mouses were used for UI control. The participants were introduced to the experiment and instructed on the control devices, interface and purpose of the experiment. After the introduction, a questionnaire presented on the PC asked for their demographic data and about any visual impairments. After this, participants were instructed how to proceed through the testing and that they should attempt to be fast as well as accurate since both their speed and correctness would be measured. The testing application was created at the Faculty of Informatics (Masaryk University, Brno) specifically for the purpose of the study. The participants were instructed to follow the information on the screen, for example, when to wear or remove the 3D glasses with respect to the specific test section. After instructions, the participants were pseudo-randomly assigned into the specific experimental groups (based on the task type category). The participants then commenced a training session to learn how to navigate/control the interactive geovisualizations and how to put on/take off the 3D glasses. After the training, the participants completed 24 assigned tasks in the experiment. At the end of the experiment, participants were debriefed and rewarded with sweets.

Research design

The study employed a within-between multi-factorial design. The first factor was 3D visualization type (two levels: pseudo 3D visualization and real 3D visualization), the second factor was interactivity type (two levels: interactive type and static, or non-interactive, type) and the third factor was task type (three levels: cubes representing objects, e.g. buildings, surfaces representing water bodies, and terrain profiles representing, e.g. roads). This legend (buildings, water bodies, roads) was described to the participants at the beginning of the experiment to better illustrate the purpose of the experiment and tasks. Taking into account the length of the experiment and potential fatigue in the participants, the research sample was pseudo-randomly divided into three subsamples (regarding the ratio of males and females) with respect to the specific task type (cubes, water bodies, terrain profiles). This meant that each subsample dealt with only one type of task (24 trials). Each task measured (i) response time (in seconds) and (ii) correct elevation estimation (participants were asked to select one of three possible options and scored 1 point for a correct answer and 0 for an incorrect answer). Time, correctness and the number of the mouse clicks participants made was recorded automatically with the software developed for testing.

The research design was balanced, each participant underwent 24 tasks divided into 8 blocks. Each block consisted of 3 trials of the same type (e.g. interactive real 3D). In an effort to reduce the primacy effect, the participants commenced the test’s static real 3D, static pseudo 3D, interactive real 3D and interactive pseudo 3D condition alternately (Fig 2).

Download:

Fig 2. Diagram of the procedure of the experiment.

https://doi.org/10.1371/journal.pone.0233353.g002

Description of the tasks

Comparing the positions of cubes in the terrain.

Participants were asked to identify a cube placed at the highest elevation in the digital terrain model. Participants were shown a virtual scene with three cubes of the same red colour and asked to identify which one was located at the highest elevation. The cubes represented, for example, buildings. As point-symbols, buildings are considered the simplest spatial structure. Participants could identify the cubes with a mouse click. After clicking, the cube turned yellow, and participants could confirm their answer by clicking the “NEXT” button (Fig 3). Participants could also change their answers by clicking on one of the other cubes before pressing the “NEXT” button.

Download:

Fig 3. Example of Task 1 –Cubes in the terrain.

Yellow indicates that this object was selected by the participant. Adapted from [61] under a CC BY license, with permission from [ČÚZK], original copyright [2018/2019].

https://doi.org/10.1371/journal.pone.0233353.g003

Comparing the positions of water bodies in the terrain.

Participants were asked to identify the water body located at the highest elevation in the digital terrain model. Participants were shown a scene with three structures, illustrated as water bodies with the same blue colour, and asked to select which was positioned highest. Water bodies have a surface character, and in terms of their complexity, are considered stimuli between point symbols and areal symbols. As in the cube tasks, responses were made with a mouse click, turning the flat area yellow (Fig 4), and the task was finished by confirming the choice with the “NEXT” button.

Download:

Fig 4. Example of Task 2 –Water bodies in the terrain.

Yellow indicates that this object was selected by the participant. Adapted from [61] under a CC BY license, with permission from [ČÚZK], original copyright [2018/2019].

https://doi.org/10.1371/journal.pone.0233353.g004

Identifying terrain profiles.

Participants were shown a scene with three structures located in the digital terrain model, each one representing a specific terrain profile between two points. A terrain profile presented as a 2D curve was displayed on the right side of the screen. Participants were asked to identify which of the three terrain profiles conformed to the shape of the curve depicted on the right. Since the elevation curve (complicated 3D structure) was not depicted, i.e. needed to be mentally computed, profile tasks are considered the most difficult. Again, as in the previously described tasks, the choice was made with a mouse click, turning the selected profile yellow (Fig 5), and the final answer was confirmed with the “NEXT” button.

Download:

Fig 5. Example of Task 3 –Terrain profiles.

Yellow indicates that the profile was selected by the participant. Adapted from [61] under a CC BY license, with permission from [ČÚZK], original copyright [2018/2019].

https://doi.org/10.1371/journal.pone.0233353.g005

Stimuli

An original testing application based on the Unity^® game engine was developed for the experiment. This application renders large 3D terrain models in real-time and automatically collects data for further analysis. The 3D models of terrain were generated from a fourth-generation Digital Terrain Model of the Czech Republic (DTM 4G), which was originally created from airborne laser scanning and processed at a ground resolution of 5 × 5 m. DTM 4G is distributed by the ČÚZK (Czech Office for Surveying, Mapping and Cadastre). The terrains were selected so that the relief zones were similar in all territories. The study proceeded from a geographical regionalization of the Czech Republic [62], and all terrains depicted were highlands (relative height variation of 200 to 300 m). Terrains were covered with corresponding orthophotography data [61]. The 3D models were vertically scaled with a fixed factor of 3.0. Non-interactive (static) expositions were designed with respect to the position of the virtual camera from which the scene was viewed—screenshots of the terrains were created from two specific angles (45 degrees and 75 degrees), and the virtual camera was maintained at the same height in all static scenes. These static scenes were further used in testing, and the interactive scene was rendered in the real time.

Analysis

As the dependent variables, the researchers analysed response times (RTs), correct elevation estimation (scores) and the number of mouse-clicks (clicks under interactive conditions). Correct answers scored 1, and incorrect answers scored 0. The RTs indicated asymmetric distribution, therefore linear mixed effect (LMER) modelling [63] was applied for analysis (due to the leftward inclination, a box-cox transformation was applied to the response time data). Score data were analysed with the use of generalized linear mixed effect models (GLMER), with logit as a link function and raw correctness as the dependent variable (1 = correct, 0 = incorrect). For the response times and scores, the visualization type (real 3D, pseudo 3D), interactivity type (interactive, static) and task type (cubes, water bodies and terrain profiles) and their interactions were considered fixed factors. The experiment included individual participants (according to their assigned identification numbers) as random intercepts. The models were executed using R software [64], R packages lme4 [65] and lmerTest [66]. The p-values for fixed effects were obtained using Satterthwaite approximation for degrees of freedom. All reported confidence intervals were obtained via bootstrapping (100000 iterations for LMER and 1000 iterations for GLMER) [63]. The mouse clicks were checked with the use of Spearman correlations to search for possible correspondence between RT and correctness.

Results

Response time

Overall, the linear mixed effects model for RTs revealed significant main effects of visualization type (t = 4.64, p < 0.001) and interaction type (t = -10.849, p < 0.001). Specifically, the real 3D environment increased the response time by 0.23 s ± 0.05 (95% CI [0.13309, 0.32925), and the option to interact prolonged response time by 0.54 s (se = 0.05s), 95% CI [-0.63256, -0.43963]. The main effect of the task type was also significant (t = 9.56, p < 0.001). Terrain profiles took 0.88 s ± 0.09 longer to solve than cubes (95% CI [0.69630, 1.05498]) and water bodies took 0.31 s ± 0.09 longer to solve than cubes (95% CI [0.13034, 0.48149]). The interaction effect of task and visualization factors was significant (t = -2.49, p = 0.0130) in water bodies compared to cubes. Real 3D visualization significantly reduced the response time in water bodies by 0.18 s ± 0.07 (95% CI [-0.31702, -0.03610]) and terrain profiles compared to cubes (t = -2.34, p = 0.0195). The 3D visualization significantly reduced the response time in terrain profiles by -0.17 s ± 0.07 (CI [-0.30700, -0.02639]). The interaction factor prolonged the response time in terrain profiles compared to cubes (t = 2.50, p = 0.0126) by 0.18 s ± 0.07 ([0.03815, 0.31652]). No other fixed effects were significant regarding response time (p > 0.5).

Fig 6 highlights that the response times exactly matched the expected trends of interactivity and real 3D visualization prolonging the task solving process. As the most spatially complex tasks, the terrain profiles took longest to solve, while cubes were solved the quickest. The largest difference in speed was observed between cubes solved in static pseudo 3D conditions and terrain profiles in interactive real 3D conditions.

Download:

Fig 6. Participants’ response times according to conditions.

https://doi.org/10.1371/journal.pone.0233353.g006

Correctness

Generalized linear mixed effects modelling (the Hosmer–Lemeshow Test suggested adequate goodness of fit: χ² = 3.3563, df = 8, p = 0.91) on correctness revealed a significant main effect for interaction type (β = 1.842, z = 5.38, p < 0.001, 95% CI [1.30245, 2.40780]), suggesting that the static tasks were overall solved with greater accuracy. Water bodies were solved with greater accuracy than cubes (β = 0.564, z = 2.23, p = 0.0258, 95% CI [0.08624, 1.05926]). The interaction effects were significant for visualization type and interaction type (β = -2.232, z = -5.30, p < 0.001, 95% CI [-2.88130, -1.49627]). Interactive pseudo 3D conditions tended to encourage greater accuracy than static real 3D conditions. The effect of interaction was also observed in interaction type and task type in the case of terrain profiles (β = -1.376, z = -3.16, p = 0.0016). The terrain profiles were solved with less accuracy than cubes (95% CI [-2.09665, -0.65452]), and water bodies were solved with less accuracy than cubes (β = -2.318, z = -5.40, p < 0.001, 95% CI [-3.01544, -1.70323]). A triple interaction effect was observed in terrain profile tasks (β = 2.624, z = 4.57, p < 0.001), 95% CI [1.73593, 3.49377]), with a trend demonstrating that the terrain profiles were solved with greater accuracy in static real 3D conditions than cubes. The same trend was observed for water bodies (β = 1.670, z = 0.557, p < 0.001), where water bodies were solved with greater accuracy than cubes (95% CI [0.76593, 2.52489]) in the static real 3D condition.

Fig 7 (left) depicts the probabilities of accurate responses in different types of tasks with regards to the specific 3D and interactive condition, suggesting that no specific pattern between performance, tasks and conditions existed for correctness. The correctness scores in Fig 7 (right) illustrate that the study’s expectations on participant performance were not met and barely corresponded to the trends indicated by the time responses. The visual trend in both static conditions suggested a greater dispersion between individual types of tasks (cubes, water bodies, terrain profiles). Close values in the interactive conditions demonstrated interactivity as a feature that eliminated extreme values in participant performance.

Download:

Fig 7. Calculated probabilities of accurate scoring according to conditions (left); participants’ average correctness scores (right).

https://doi.org/10.1371/journal.pone.0233353.g007

Mouse clicks

Fig 8 charts the frequency of mouse clicks in the interactive tasks. The graph also contains data from static tasks. The number of mouse clicks participants performed while solving tasks in a specific condition corresponded to the response times, especially with respect to specific task type. A significant close positive correlation between RT and the number of mouse clicks was observed (Spearman’s rho = 0.827, p < 0.001). No correlation or trend was observed between correctness and the number of mouse clicks (Spearman’s rho = -0.056, p = 0.079). Generally, visual inspection suggested that more clicks were done in real 3D tasks, and the number of mouse clicks had an increasing trend in the terrain profiles tasks (Fig 8). The frequency of mouse clicks was lowest in the cube tasks.

Download:

Fig 8. Number of mouse clicks according to the conditions.

https://doi.org/10.1371/journal.pone.0233353.g008

Discussion

The study examined the effectivity of stereoscopic real 3D visualizations and pseudo 3D visualizations under static and interactive conditions. Several effects were observed that contributed to the participants’ correctness and response times in solving the tasks. The number of mouse clicks to answer questions was also inspected.