Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Perceiving Object Shape from Specular Highlight Deformation, Boundary Contour Deformation, and Active Haptic Manipulation

  • J. Farley Norman ,

    Contributed equally to this work with: J. Farley Norman, Flip Phillips

    Farley.Norman@wku.edu

    Affiliation Department of Psychological Sciences, Ogden College of Science and Engineering, Western Kentucky University, Bowling Green, Kentucky, United States of America

  • Flip Phillips ,

    Contributed equally to this work with: J. Farley Norman, Flip Phillips

    Affiliation Department of Psychology & Neuroscience Program, Skidmore College, Saratoga Springs, New York, United States of America

  • Jacob R. Cheeseman,

    Affiliation Department of Psychological Sciences, Ogden College of Science and Engineering, Western Kentucky University, Bowling Green, Kentucky, United States of America

  • Kelsey E. Thomason,

    Affiliation Department of Psychological Sciences, Ogden College of Science and Engineering, Western Kentucky University, Bowling Green, Kentucky, United States of America

  • Cecilia Ronning,

    Affiliation Department of Psychological Sciences, Ogden College of Science and Engineering, Western Kentucky University, Bowling Green, Kentucky, United States of America

  • Kriti Behari,

    Affiliation Department of Psychology & Neuroscience Program, Skidmore College, Saratoga Springs, New York, United States of America

  • Kayla Kleinman,

    Affiliation Department of Psychology & Neuroscience Program, Skidmore College, Saratoga Springs, New York, United States of America

  • Autum B. Calloway,

    Affiliation Department of Psychological Sciences, Ogden College of Science and Engineering, Western Kentucky University, Bowling Green, Kentucky, United States of America

  • Davora Lamirande

    Affiliation Department of Psychological Sciences, Ogden College of Science and Engineering, Western Kentucky University, Bowling Green, Kentucky, United States of America

Perceiving Object Shape from Specular Highlight Deformation, Boundary Contour Deformation, and Active Haptic Manipulation

  • J. Farley Norman, 
  • Flip Phillips, 
  • Jacob R. Cheeseman, 
  • Kelsey E. Thomason, 
  • Cecilia Ronning, 
  • Kriti Behari, 
  • Kayla Kleinman, 
  • Autum B. Calloway, 
  • Davora Lamirande
PLOS
x

Abstract

It is well known that motion facilitates the visual perception of solid object shape, particularly when surface texture or other identifiable features (e.g., corners) are present. Conventional models of structure-from-motion require the presence of texture or identifiable object features in order to recover 3-D structure. Is the facilitation in 3-D shape perception similar in magnitude when surface texture is absent? On any given trial in the current experiments, participants were presented with a single randomly-selected solid object (bell pepper or randomly-shaped “glaven”) for 12 seconds and were required to indicate which of 12 (for bell peppers) or 8 (for glavens) simultaneously visible objects possessed the same shape. The initial single object’s shape was defined either by boundary contours alone (i.e., presented as a silhouette), specular highlights alone, specular highlights combined with boundary contours, or texture. In addition, there was a haptic condition: in this condition, the participants haptically explored with both hands (but could not see) the initial single object for 12 seconds; they then performed the same shape-matching task used in the visual conditions. For both the visual and haptic conditions, motion (rotation in depth or active object manipulation) was present in half of the trials and was not present for the remaining trials. The effect of motion was quantitatively similar for all of the visual and haptic conditions–e.g., the participants’ performance in Experiment 1 was 93.5 percent higher in the motion or active haptic manipulation conditions (when compared to the static conditions). The current results demonstrate that deforming specular highlights or boundary contours facilitate 3-D shape perception as much as the motion of objects that possess texture. The current results also indicate that the improvement with motion that occurs for haptics is similar in magnitude to that which occurs for vision.

Introduction

One hundred and seventy-eight years ago [1] Charles Wheatstone expressed his view that relative motion between environmental objects and human observers produced as good a perception of solid object shape as that that occurs during binocular stereopsis. In particular, Wheatstone (p. 380) said:

“when different projections of the same object are successively presented… the form to which they belong is completely characterized”.

Later in the nineteenth-century, Hermann Helmholtz [2] expressed a similar belief (p. 297), that motion permits observers to form “correct apperceptions of the material shapes of their surroundings”. Perhaps surprisingly, good empirical evidence to support Wheatstone (and von Helmholtz) was not obtained for more than 100 years; that evidence was published in 1953 by Wallach and O'Connell [3] in a now well-known article entitled “The kinetic depth effect”. Wallach and O'Connell found that when human observers viewed deforming 2-dimensional (2-D) cast shadows of rotating 3-dimensional (3-D) objects, those deforming 2-D shadows appeared to be solid. These perceptions of solid shape occurred whenever the projected images contained motions of trackable object features, such as corners and object vertices; when these trackable object features were absent, however, Wallach and O'Connell found that the projected motion sequences appeared not as rigid rotations of a solid object in depth, but as nonrigid deformation. Later researchers [49] extended these investigations of the kinetic depth effect in human observers using computer-generated patterns and motion sequences. In such investigations, the perception of 3-D structure was enabled by the projected motions of trackable features, such as random dots or surface texture.

Over the past 35 years, a parallel series of investigations produced computational models and algorithms that can recover 3-D structure from motion [1015]. Such models require the presence of identifiable object features, such as surface texture elements or sharp corners. The 2-D projected motions of these identifiable features are used (along with certain assumptions, such as object rigidity, fixed-axis motion, etc.) to recover information about 3-D object shape. Over the past twenty years, however, psychophysical research using human observers has shown that the presence of identifiable surface features is not necessary for the effective perception of solid object shape from motion—for example, observers can discriminate 3-D object shape from optical patterns containing only boundary contours, shading, and/or specular highlights [1619]. Similarly, the presence of identifiable surface features or texture is not necessary for human observers to effectively perceive local aspects of 3-D object shape, such as the orientation of local surface patches [2023].

While it has long been known that object motion leads to a significant improvement in the visual perception of 3-D object shape (i.e., the kinetic depth effect), almost no similar research has investigated the haptic (active touch) perception of 3-D object shape. Gibson [24] used a set of six 2-D outline shapes (in actuality, these outline shapes were 1-inch diameter “cookie cutters”, which are normally pressed into rolled dough to create differently-shaped cookies). In one condition (passive touch) a randomly-selected cookie cutter was pressed into the palm of a participant's hand on any particular trial, and the participant was required to identify it. In the active touch condition, the participants were allowed to actively explore the cookie cutters using their hands and fingers—the task was again to identify the single object presented on any given trial. Gibson found that while the performance in the active touch condition was excellent (95 percent correct identification), performance in the passive condition was considerably worse (49 percent correct identification). One can legitimately consider this to be a haptic analog to the visual “kinetic depth effect”: the shape-identification performance obtained with relative motion of the object and hand/fingers was superior to that obtained from the tactile stimulation produced by a stationary object and hand. In a more recent investigation by Pont, Kappers, and Koenderink evaluating tactile curvature discrimination [25], however, performance for dynamic touch was no better than for static touch (see their Figure 17)—in this case, there was no “kinetic depth effect” for the haptic perception of shape.

One purpose of the current set of experiments was to compare the improvement that occurs with motion across vision and haptics—does the introduction of object motion lead to similar quantitative improvements in 3-D shape identification performance for vision and haptics? A direct comparison (using the same set of object shapes) between visual and haptic shape identification and how they are affected by object motion has never been performed. An equally important purpose of the current research was to evaluate whether the magnitude of the kinetic depth effect for the recognition/identification of shape is different when object surfaces contain texture (a situation where traditional computational models can operate and recover 3-D structure) and when they do not (i.e., when moving surfaces are defined only by boundary contours and/or specular highlights; most computational models cannot recover 3-D shape under these conditions).

Experiment 1

Methods

Apparatus.

The experimental stimulus objects in the visual conditions were generated by an Apple MacPro computer (2 quad-core processors) and were displayed on a 22-inch Mitsubishi Diamond Plus 200 color monitor (resolution was 1280 x 1024 pixels). The stimulus rendering was accelerated by an ATI Radeon HD 5770 graphics processor. The viewing distance was 100 cm.

Experimental Stimuli.

The stimulus objects were plastic copies of 12 ordinary bell peppers (Capsicum annuum, see Fig 1) that have been used in previous research [2628]. The solid shapes of the bell pepper replicas were digitally scanned using a 3-D laser scanner (NextEngine, Inc.). The object surfaces in the visual conditions were defined by the positions and orientations of about 74,000 triangular polygons. The objects were optically presented to the participants 1) as silhouettes (i.e., shapes defined only by outer boundary contours), 2) by specular highlights (without boundary contours), 3) by specular highlights combined with boundary contours, and 4) by boundary contours combined with solid/volumetric texture. Fig 2 illustrates these conditions, while Fig 3 demonstrates the results of the solid texturing in more detail.

thumbnail
Fig 1. A photograph of the naturally-shaped solid objects (replicas of bell peppers, Capsicum annuum) used in Experiment 1.

They are arranged in numerical order (1–12) from top-left to bottom right. These objects (and subsets of them) have been used in multiple previous investigations [18, 2628].

https://doi.org/10.1371/journal.pone.0149058.g001

thumbnail
Fig 2. Example stimulus displays used in the visual conditions of Experiment 1.

From bottom-left going clockwise are depicted objects defined by 1) specular highlights without occlusion boundary contours, 2) specular highlights with occlusion boundary contours, 3) occlusion boundary contours only (i.e., silhouettes), and 4) volumetric/solid texture, where the surface markings resemble those of marble.

https://doi.org/10.1371/journal.pone.0149058.g002

thumbnail
Fig 3. Examples of the solid/volumetric texture applied to six of the 12 stimulus objects: from top-left to bottom-right are depicted bell peppers 1, 2, 5, 6, 9, and 10.

To better appreciate the stimulus shapes, the renderings in this figure include image shading and specular highlights as well the results of solid texturing.

https://doi.org/10.1371/journal.pone.0149058.g003

All rendering was done using the standard OpenGL reflectance model [29]. For the boundary contour conditions all shading coefficients for the ambient, Lambertian diffuse, and specular components (e.g. ka, kd & ks) were made equal to 0.0, resulting in an all-black silhouette rendered against a background of {r, g, b} = {0.616, 0.455, 0.455}. In the specular highlight conditions the objects were rendered with a specular coefficient ks = 1.0, a highlight color of {r, g, b} = {1.0, 1.0, 1.0} and a shininess exponent of 20.0 [2930]. For this condition, a single infinite light source was placed such that it illuminated the objects from a direction corresponding to θslant = 30° and ϕtilt = 45°. For the boundary contour inclusive highlight condition the objects were displayed against a background of {r, g, b} = {0.616, 0.455, 0.455}. In the highlights only condition, the objects were rendered onto a black background, thus making the boundary invisible. For the texture condition, a solid (i.e., volumetric) texture was created using 3-D value-noise [3132]. In effect, material was then “carved away” from the solid block of texture so that the solid shape of the stimulus objects was all that remained; this process led to the creation of surface markings that somewhat resemble those of marble. The diffuse and specular (kd, ks) lighting coefficients were set to 0.0 while the ambient coefficient, ka was set equal to 1.0. This resulted in the rendering of only texture markings with no shading or specular highlights. As in the other boundary contour inclusive conditions, the objects were rendered against a background of {r, g, b} = {0.616, 0.455, 0.455}.

Physical replicas of the bell peppers were used in the haptic condition. Rubber molds (Evergreen 30, Smooth-on, Inc.) were created from the original set of bell peppers; solid copies of the original objects were then produced by pouring liquid plastic (C-1506, Smooth-on, Inc.) into the molds.

Procedure.

In all conditions, a single randomly-chosen bell pepper was presented on any given trial—the participants' task was to indicate which of 12 simultaneously visible, physical bell peppers (laid out on a table in front of the participants at a viewing distance of 55 cm) possessed the same 3-D shape. No feedback was provided to the participants regarding the accuracy of their shape-matching responses. Each single stimulus object was presented for 12 seconds in a random orientation. In the four visual conditions (surfaces defined by boundary contours only, boundary contours+specular highlights, boundary contours+volumetric texture, specular highlights only), the objects were either presented statically or in motion (rotated in depth about a Cartesian vertical axis located in the plane of the computer monitor; the angular rotation per frame transition was 2 deg, while the frame update rate was 75 Hz). The viewing transform of the computer generated stimuli was chosen to mimic the viewing angle of the individual physical stimuli—approximately 8° of visual angle. In the haptic condition, the objects were either held statically (static enclosure) [33] or were actively manipulated by the participant using both hands [28]. The presentation type (4 visual conditions + 1 haptic condition) was a between-groups manipulation, while the presence or absence of object motion was a within-groups manipulation. Each participant made a total of 60 shape-matching judgments (30 judgments with object motion & 30 judgments without object motion). Half of the participants judged the objects with motion first, while the remaining half first judged objects presented without motion.

Participants and ethics.

Thirty adults participated in the experiment (mean age was 22.7 years, SD = 3.7; their ages ranged from 20 to 36 years). The participants' vision was assessed with a standard eye chart (Precision Vision catalog number 2195) at a distance of 1 m. All participants possessed normal, or corrected-to-normal visual acuity. The experiment was approved by the Western Kentucky University Institutional Review Board; all participants gave written consent prior to participation in the experiment.

Results and Discussion

The participants' results are shown in Fig 4. As is readily evident, the participants’ shape matching performance was much higher (93.5% higher) in the conditions with motion and/or active haptic manipulation (F(1, 25) = 70.8, p < .000001, η2p = 0.74). This improvement with motion (relative to performance obtained in the static conditions) is the kinetic depth effect. One can also see from an inspection of Fig 4 that the magnitude of the kinetic depth effect was essentially identical for all of the visual and haptic conditions (i.e., there was no motion x presentation type interaction: F(4, 25) = 0.25, p = .91). There was a significant main effect of presentation type, however (F(4, 25) = 4.2, p < .01, η2p = 0.41). As can be seen from Fig 5, the participants' overall shape-matching performance was similar for all of the visual and haptic conditions, except for the visual condition where the objects were defined only by specular highlights (with no accompanying boundary contours). Performance for that condition (specular highlights only) was significantly lower than that obtained for the others according to Fisher LSD (least significant difference) post-hoc tests (all p < .05).

thumbnail
Fig 4. Results of Experiment 1.

The participants’ shape-matching accuracies (in terms of percent correct) are plotted as functions of both 1) the stimulus presentation type and 2) the presence or absence of object motion/active haptic manipulation. BC = boundary contours only, SH = specular highlights only, SH+BC = specular highlights with accompanying boundary contours, VT+BC = volumetric texture with accompanying boundary contours, H = haptic manipulation. The error bars indicate ± 1 SE. The dashed line indicates chance performance.

https://doi.org/10.1371/journal.pone.0149058.g004

thumbnail
Fig 5. Results of Experiment 1, plotting the participants’ overall shape matching performance for the various stimulus presentation types.

BC = boundary contours only, SH = specular highlights only, SH+BC = specular highlights with accompanying boundary contours, VT+BC = volumetric texture with accompanying boundary contours, H = haptic manipulation. The error bars indicate ± 1 SE. The dashed line indicates chance performance.

https://doi.org/10.1371/journal.pone.0149058.g005

Experiment 2

In Experiment 1 we found for a particular class of naturally-shaped objects that the presence of motion (whether visual motion or active haptic exploration) produces a large improvement in the ability to perceive shape. The shapes of these particular bell peppers have been well characterized mathematically by Norman, Phillips, et al. [28]. Despite the fact that the 12 individual peppers possess distinctive global shapes and “look different” (Fig 1), the distributions of the objects’ local surface shapes (i.e., shape indices) and surface curvature magnitudes are quite similar (see Fig 2 of Norman, Phillips, et al.). The purpose of Experiment 2 was to extend the current investigation to evaluate the effects of visual and haptic motion for a set of solid objects that were purposefully designed to possess a wide range of stimulus complexities.

Methods

Apparatus.

The experimental stimulus objects in the visual conditions were generated and displayed on an Apple iMac computer (27-inch, mid 2011). The monitor was set to a resolution of 1280 x 1024 pixels to match the previous experiment. Stimulus rendering was accelerated by an AMD Radeon HD 6770M graphics processor. The viewing distance was 100 cm.

Experimental Stimuli.

The stimulus objects were 8 pseudo-random objects called ‘glavens’ (see Fig 6) that have been used in previous research [3437]. The objects were optically presented in the same fashion as in Experiment 1 using the same rendering conditions. Glavens were chosen for this experiment as they can be produced with continuously varying amounts of geometric complexity [34, 36]. The eight stimuli used in this study were selected from a set previously used [36]. When ordered in terms of complexity from simplest to most complex, each one (visually) differs from the next by approximately one jnd. Physical copies of the glavens were created in ABS-M30 plastic using a uPrint SE 3-D printer (Stratasys, Inc.). The objects were printed in ‘ivory’ color and were left unfinished (e.g., without sanding, painting or other surface treatment).

thumbnail
Fig 6. A photograph of the solid objects (glavens) used as stimuli in Experiment 2.

The glavens are ordered in terms of stimulus complexity; the complexity increases from the bottom-left to the upper right.

https://doi.org/10.1371/journal.pone.0149058.g006

Procedure.

The procedure was identical to that of Experiment 1. Depending on the condition, the objects were presented visually or haptically for 12 seconds, with and without motion (or active exploration in the haptic condition). The 8 target glavens were visible on the table in front of the participant. Each participant made a total of 60 shape-matching judgments (30 judgments with object motion & 30 judgments without object motion). Half of the participants judged the objects with motion first, while the remaining half first judged objects presented without motion.

Participants and ethics.

Twenty-seven adults participated in the experiment (mean age was 19.2 years, SD = 1.2 years). The participants' vision was assessed with a standard Snellen-style eye chart (Precision Vision catalog number 5002). All participants possessed normal, or corrected-to-normal visual acuity. The experiment was approved by the Skidmore College Institutional Review Board; all participants gave written consent prior to participation in the experiment.

Results and Discussion

The results are shown in Figs 79. Fig 7 plots performance as functions of stimulus presentation type and the presence/absence of object motion. As with Experiment 1, the participants’ shape-matching performance was significantly higher (34.6 percent higher overall) in the conditions with motion and/or active haptic manipulation (F(1, 22) = 13.68, p < .001, η2p = 0.38). Again, there was no motion x presentation type interaction—F(4, 22) = 1.28, p = .19) and there was a significant main effect due to presentation type—F(4, 22) = 4.0, p < .01, η2p = 0.42. Note that this is the same pattern of results found in Experiment 1 with the bell peppers, but with smaller effect sizes throughout.

thumbnail
Fig 7. Results of Experiment 2.

The participants’ shape-matching accuracies (in terms of percent correct) are plotted as functions of both 1) the stimulus presentation type and 2) the presence or absence of object motion/active haptic manipulation. BC = boundary contours only, SH = specular highlights only, SH+BC = specular highlights with accompanying boundary contours, VT+BC = volumetric texture with accompanying boundary contours, H = haptic manipulation. The error bars indicate ± 1 SE. The dashed line indicates chance performance.

https://doi.org/10.1371/journal.pone.0149058.g007

thumbnail
Fig 8. Results of Experiment 2.

The participants’ shape-matching accuracies (in terms of percent correct) are plotted as functions of both 1) the stimulus presentation type and 2) stimulus object complexity (the 8 stimulus objects were partitioned into two groups possessing low and high stimulus complexity). BC = boundary contours only, SH = specular highlights only, SH+BC = specular highlights with accompanying boundary contours, VT+BC = volumetric texture with accompanying boundary contours, H = haptic manipulation. The error bars indicate ± 1 SE. The dashed line indicates chance performance.

https://doi.org/10.1371/journal.pone.0149058.g008

thumbnail
Fig 9. A comparison of the results of Experiments 1 (bell pepper replicas) and 2 (glavens).

The error bars indicate ± 1 SE.

https://doi.org/10.1371/journal.pone.0149058.g009

The stimuli (glavens) used in Experiment 2 are parameterized such that they can be ordered in terms of relative complexity [34, 36]. We repeated our analysis after partitioning the stimulus objects into low- and high-complexity groups. The first four stimuli were placed in the low-complexity group and the last four into the high-complexity group. Fig 8 shows the results of this partitioning. First of all, there was a main effect of stimulus complexity, such that the participants’ performance was significantly higher for the objects with lower complexity (F(1, 22) = 22.9, p < .0001, η2p = 0.51). While there was a significant main effect of stimulus presentation type (F(4, 22) = 4.2, p < .02, η2p = 0.44), the variations in presentation type only differentially affected the participants’ performance for objects with lower complexity (i.e., there was a presentation type x complexity interaction, F(4, 22) = 3.2, p < .04, η2p = 0.37). There was no significant variation in performance across the various presentation types for the stimulus objects with higher complexity (note that the error bars for the high complexity conditions in Fig 8 overlap substantially). Interestingly, the effects of motion (evident in the results shown in Fig 7) did not vary by complexity (i.e., there was no motion x complexity interaction, F(1, 22) = 0.04, p = .85): the presence of motion or active haptic exploration was equally beneficial for the low- and high-complexity stimulus objects.

An overall comparison between the results of Experiments 1 and 2 is shown in Fig 9. It is clearly evident that while the presence of motion/active haptic manipulation resulted in significant improvements in performance for both object types (F(1, 47) = 74.4, p < .000001, η2p = 0.61) that the enhancement obtained for the bell pepper stimuli was much higher than that that occurred for the glavens (i.e., there was a motion x object type interaction, F(1, 47) = 13.4, p < .001, η2p = 0.22). There was also a significant main effect of object type, such that overall, the participants’ shape-matching performance was higher for the bell pepper objects (F(1, 47) = 6.6, p < .02, η2p = 0.12).

General Discussion

In both Experiments 1 (replicas of bell peppers) and 2 (glavens), significant improvement in visual object identification occurred for the objects defined by texture (condition VT+BC in Figs 4 and 7) when motion, rotation in depth, was present. In our experiments, the volumetric texture condition is the “control condition” and the obtained improvement in performance (relative to the performance that occurred for the static condition) represents the "conventional" kinetic depth effect [39]. In our experiments, the participants’ performance improved (for the volumetric texture condition) by 30 and 114 percent for the glavens and bell peppers, respectively. It is well known [19, 2223, 3839] that specular highlights contain useful perceptual information about solid object shape. Our current results now demonstrate that the magnitude of the kinetic depth effect is as strong for stimulus objects defined only by specular highlights as for objects that are defined by surface texture: the participants’ object identification performance for the stimulus displays containing specular highlights (condition SH in Figs 4 and 7) improved with motion by 108.7 and 185.4 percent for the glavens and bell peppers, respectively. Little is known at present about how human observers actually extract information about 3-D object shape from optical patterns containing specular highlights. We do know that specular highlights generally cling to surface areas that are highly curved [19, 40] (e.g., see Fig 10). Unfortunately, computational models have not yet been developed that can successfully extract complete 3-D structure from specular highlights, probably because the optical deformations of specular highlights behave quite differently from the projected motions of surface texture [4142] (e.g., see the apparent motion sequence depicted in Fig 10).

thumbnail
Fig 10. A sequence of views of a shiny solid object (bell pepper 11) rotating in depth; the views progress from the upper-left (first view) to the bottom-right (last view).

The object in each view has been rotated by 20 degrees from its orientation in the previous view. It is important to note that the shiny specular highlights deform quite radically over time in response to the object rotation in depth relative to the environmental light source. Note also that the object’s outer boundary contour also deforms in a complicated manner over time. Both types of motion (specular highlight and boundary contour deformation) differ qualitatively from the optical motions of surface texture elements (when they exist).

https://doi.org/10.1371/journal.pone.0149058.g010

A comparison of Figs 4 and 7 show that dynamic (i.e., moving) boundary contours, all by themselves (condition BC), were perceptually quite informative in the current study and enabled accurate object identification for the naturally-shaped bell peppers, but not for the glavens. The optical deformation of boundary contours that accompanies object motion also differs fundamentally from the projected motions of surface texture elements [18, 21, 4142] and successful extraction of 3-D object shape must occur via mechanisms that operate differently from conventional models of structure-from-motion (for reviews of this issue, see references [18, 21] & [43]). As a solid object rotates in depth, local regions of its outer boundary change both quantitatively and qualitatively as the rotation brings new 3-D structure to the projected boundary. Catastrophes in the boundary occur, for example, when a previously visible boundary feature (i.e., a particular convexity or concavity) disappears, or when a new boundary feature (a particular convexity or concavity) suddenly emerges that did not previously exist. Boundary catastrophes like these necessarily specify local 3-D shape [4445]—it is possible to obtain information about an entire object’s 3-D shape from such boundary events if, like our participants, one is allowed to watch the entire rotation of the object. There is, therefore, no more optical ambiguity to specify shape for one class of objects (e.g., bell peppers) over another (e.g., glavens). In our experiments, however, the participants were much better able to utilize boundary contour deformation to identify bell peppers than glavens (our participants’ shape identification performance was approximately 50 percent correct for the bell peppers, but was only 30 percent correct for the glavens). This empirical difference is interesting, because it suggests that some objects produce boundary contour changes that are somehow perceptually more informative than others; further investigation of this issue is definitely warranted.

The results of Experiment 2 demonstrate that variations in object complexity are perceptually significant. When the participants’ performance (Fig 8) was examined separately for the lower and higher complexity glavens, it is clear that except for the specular highlights only condition, the participants’ object identification ability was generally quite good for the lower-complexity glavens. Apparently, too many (i.e., too high a density of) surface undulations makes it difficult for people to identify objects no matter how object shape is defined (vision, haptics, texture, specular highlights, boundary contours, etc). Interestingly, while people consider highly complex solid objects to be visually attractive and aesthetically pleasing [46], our current results demonstrate that increases in complexity can produce deteriorations in the ability to perceive shape (the average visually obtained performance for the high-complexity glavens in Experiment 2 was only 27.1 percent correct).

Finally, our current results demonstrate that the kinetic depth effect for haptics is quantitatively as strong as that obtained by vision (see Figs 4 & 7). While the overall visual object identification performance in the motion conditions was 30 percent higher than that obtained for the static conditions for the glavens (Fig 7), the corresponding increase in performance for active haptic exploration was 56 percent. For the naturally-shaped bell peppers, the increases in performance due to visual object motion and active haptic exploration were 98.9 and 76.7 percent, respectively. While we have known for decades that active haptic manipulation leads to better shape perception than more passive tactile/cutaneous stimulation [24, 28, 33, 4749], it is very interesting that the magnitude of the haptic improvement in performance with motion (i.e., active object manipulation) is about as large as that that occurs for vision. This similarity in outcome for the modalities of vision and touch is striking (e.g., compare conditions H & VT+BC within both Figs 4 and 7) and reinforces recent findings [28, 5052] that these modalities are in many ways functionally equivalent, despite the obvious differences in the information that is actually being detected (spatiotemporal optical patterns of light versus dynamic pressures applied to the skin & skin/tissue deformation) as well as the peripheral neurophysiology itself (e.g., the rod and cone photoreceptors within the retina of the eye versus mechanoreceptors within the skin of our fingers, etc.).

Conclusions

Specular highlights and occlusion boundary contours are effective sources of visual information about solid object shape; deforming (moving) highlights and boundary contours enhance object identification as much as the enhancement that occurs with moving objects that possess surface texture. In addition, active haptic exploration produces as much of an enhancement in performance as visual object motion.

Supporting Information

S1 Dataset. Individual participant results for Experiment 1.

https://doi.org/10.1371/journal.pone.0149058.s001

(XLS)

S2 Dataset. Individual participant results for Experiment 2 as a function of motion and presentation type.

https://doi.org/10.1371/journal.pone.0149058.s002

(XLS)

S3 Dataset. Individual participant results for Experiment 2 as functions of motion, presentation type, and stimulus complexity.

https://doi.org/10.1371/journal.pone.0149058.s003

(XLS)

Author Contributions

Conceived and designed the experiments: JFN FP. Performed the experiments: JFN FP JRC KET CR KB KK ABC DL. Analyzed the data: JFN FP JRC. Contributed reagents/materials/analysis tools: JFN FP. Wrote the paper: JFN FP JRC. Designed software used to display experimental stimuli: JFN FP. Designed software used to analyze results: JFN FP.

References

  1. 1. Wheatstone C. Contributions to the physiology of vision—Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Phil Trans R Soc Lond. 1838;128: 371–394.
  2. 2. Helmholtz H. Treatise on physiological optics (Vol. 3). Rochester, NY: Optical Society of America; 1925.
  3. 3. Wallach H, O’Connell DN. The kinetic depth effect. J Exp Psychol. 1953; 45: 205–217. pmid:13052853
  4. 4. Braunstein ML. Sensitivity of the observer to transformations of the visual field. J Exp Psychol. 1966;72: 683–689. pmid:5972008
  5. 5. Braunstein ML. Depth perception through motion. New York: Academic Press; 1976.
  6. 6. Green BF. Figure coherence in the kinetic depth effect. J Exp Psychol. 1961;62: 272–282. pmid:13901397
  7. 7. Norman JF, Lappin JS. The detection of surface curvatures defined by optical motion. Percept Psychophys. 1992;51: 386–396. pmid:1603652
  8. 8. Todd JT, Akerstrom RA, Reichel FD, Hayes W. Apparent rotation in three-dimensional space: Effects of temporal, spatial, and structural factors. Percept Psychophys. 1988;43: 179–188. pmid:3340517
  9. 9. Todd JT, Norman JF. The visual perception of smoothly curved surfaces from minimal apparent motion sequences. Percept Psychophys. 1991;50: 509–523. pmid:1780199
  10. 10. Bruckstein A, Holt RJ, Katsman I, Rivlin E. Head movements for depth perception: Praying mantis versus pigeon. Auton Robot. 2005;18: 21–42.
  11. 11. Hoffman DD, Bennett BM. The computation of structure from fixed-axis motion: Rigid structures. Biol Cybern. 1986;54: 71–83.
  12. 12. Holt RJ, Netravali AN. Number of solutions for motion and structure from multiple frame correspondence. Int J Comput Vision. 1997;23: 5–15.
  13. 13. Koenderink JJ, van Doorn AJ. Affine structure from motion. J Opt Soc Am A. 1991;8: 377–385. pmid:2007912
  14. 14. Norman JF, Todd JT. The perceptual analysis of structure from motion for rotating objects undergoing affine stretching transformations. Percept Psychophys. 1993;53: 279–291. pmid:8483692
  15. 15. Ullman S. The interpretation of structure from motion. Proc R Soc Lond B. 1979;203: 405–426. pmid:34162
  16. 16. Norman JF, Bartholomew AN, Burton CL. Aging preserves the ability to perceive 3-D object shape from static but not deforming boundary contours. Acta Psychol. 2008;129: 198–207.
  17. 17. Norman JF, Dawson TE, Raines SR. The perception and recognition of natural object shape from deforming and static shadows. Perception. 2000;29: 135–148. pmid:10820597
  18. 18. Norman JF, Lee Y, Phillips F, Norman HF, Jennings LR, McBride TR. The perception of 3-D shape from shadows cast onto curved surfaces. Acta Psychol. 2009;131: 1–11.
  19. 19. Norman JF, Todd JT, Orban GA. Perception of three-dimensional shape from specular highlights, deformations of shading, and other types of visual information. Psychol Sci. 2004;15: 565–570. pmid:15271003
  20. 20. Koenderink JJ, Kappers AML, Todd JT, Norman JF, Phillips F. Surface range and attitude probing in stereoscopically presented dynamic scenes. J Exp Psychol Hum Percept Perform. 1996;22: 869–878. pmid:8830112
  21. 21. Norman JF, Raines SR. The perception and discrimination of local 3-D surface structure from deforming and disparate boundary contours. Percept Psychophys. 2002;64: 1145–1159. pmid:12489668
  22. 22. Norman JF, Todd JT, Phillips F. The perception of surface orientation from multiple sources of optical information. Percept Psychophys. 1995;57: 629–636. pmid:7644323
  23. 23. Norman JF, Wiesemann EY. Aging and the perception of local surface orientation from optical patterns of shading and specular highlights. Percept Psychophys. 2007;69: 23–31. pmid:17515213
  24. 24. Gibson JJ. Observations on active touch. Psychol Rev. 1962;69: 477–491. pmid:13947730
  25. 25. Pont SC, Kappers AML, Koenderink JJ. Similar mechanisms underlie curvature comparison by static and dynamic touch. Percept Psychophys. 1999;61: 874–894. pmid:10499001
  26. 26. Norman JF, Norman HF, Clayton AM, Lianekhammy J, Zielke G. The visual and haptic perception of natural object shape. Percept Psychophys. 2004;66: 342–351. pmid:15129753
  27. 27. Norman JF, Clayton AM, Norman HF, Crabtree CE. Learning to perceive differences in solid shape through vision and touch. Perception. 2008;37: 185–196. pmid:18456923
  28. 28. Norman JF, Phillips F, Holmin JS, Norman HF, Beers AM, Boswell AM, et al. Solid shape discrimination from vision and haptics: Natural objects (Capsicum annuum) and Gibson's "feelies". Exp Brain Res. 2012;222: 321–332. pmid:22918607
  29. 29. Shreiner D. OpenGL Programming Guide (7th ed.). Upper Saddle River, NJ: Addison-Wesley; 2010.
  30. 30. Hughes JF, van Dam A, McGuire M, Sklar DF, Foley JD, Feiner SK, et al. Computer graphics: principles and practice (3rd ed.). Upper Saddle River, New Jersey: Addison-Wesley; 2014.
  31. 31. Peachey DR. Solid texturing of complex surfaces. SIGGRAPH Comput Graph. 1985;19(3): 279–286.
  32. 32. Perlin K. An image synthesizer. SIGGRAPH Comput Graph. 1985;19(3): 287–296.
  33. 33. Lederman SJ, Klatzky RL. Hand movements: A window into haptic object recognition. Cogn Psychol. 1987;19: 342–368. pmid:3608405
  34. 34. Phillips F. Creating noisy stimuli. Perception. 2004;33: 837–854. pmid:15460511
  35. 35. Phillips F, Thompson CH, Voshell MG. Effects of 3-D complexity on the perception of 2-D depictions of objects. Perception. 2004;33: 21–33. pmid:15035326
  36. 36. Phillips F, Egan EJL. Crossmodal information for visual and haptic discrimination. Proc. SPIE. 2009;7240: 72400H.
  37. 37. Phillips F, Egan EJL, Perry B. Perceptual equivalence between vision and touch is complexity dependent. Acta Psychol. 2009;132: 259–266.
  38. 38. Fleming RW, Torralba A, Adelson EH. Specular reflections and the perception of shape. J Vis. 2004;4(9): 798–820. pmid:15493971
  39. 39. Todd JT, Mingolla E. Perception of surface curvature and direction of illumination from patterns of shading. J Exp Psychol Hum Percept Perform. 1983;9: 583–595. pmid:6224894
  40. 40. Koenderink JJ, van Doorn AJ. Photometric invariants related to solid shape. Opt Acta. 1980;27: 981–996.
  41. 41. Norman JF, Todd JT. Perception of rigid motion in depth from the optical deformations of shadows and occlusion boundaries. J Exp Psychol Hum Percept Perform. 1994;20: 343–356. pmid:8189197
  42. 42. Todd JT. The visual perception of 3D shape. Trends Cogn Sci. 2004;8(3): 115–121. pmid:15301751
  43. 43. Norman JF, Dawson TE, Raines SR. The perception and recognition of natural object shape from deforming and static shadows. Perception. 2000;29: 135–148. pmid:10820597
  44. 44. Koenderink JJ. Solid Shape. Cambridge, MA: MIT Press; 1990.
  45. 45. Koenderink JJ, van Doorn AJ. The singularities of the visual mapping. Biol Cybern. 1976;24: 51–59. pmid:963132
  46. 46. Phillips F, Norman JF, Beers AM. Fechner's aesthetics revisited. Seeing Perceiving. 2010;23: 263–271. pmid:20819476
  47. 47. Heller MA, Myers DS. Active and passive tactual recognition of form. J Gen Psychol. 1983;108: 225–229. pmid:6854286
  48. 48. Kappers AML, Koenderink JJ. Human unilateral and bilateral discrimination of curved surfaces. Perception. 1996;25: 739–749.
  49. 49. Dostmohamed H, Hayward V. Trajectory of contact region on the fingerpad gives the illusion of haptic shape. Exp Brain Res. 2005;164: 387–394. pmid:15883809
  50. 50. Norman JF, Crabtree CE, Norman HF, Moncrief BK, Herrmann M, Kapley N. Aging and the visual, haptic, and cross-modal perception of natural object shape. Perception. 2006;35: 1383–1395. pmid:17214383
  51. 51. Jao RJ, James TW, James KH. Crossmodal enhancement in the LOC for visuohaptic object recognition over development. Neuropsychologia. 2015;77: 76–89. pmid:26272239
  52. 52. Stilla R, Sathian K. Selective visuo-haptic processing of shape and texture. Hum Brain Mapp. 2008;29: 1123–1138. pmid:17924535