Table 1.
The selection of referent sounds used in the observational study.
Fig 1.
Spectrograms of the eight referent sounds used in the observational studies.
The frequency range has been limited to 10kHz for the sake of legibility.
Fig 2.
Top panel: most participants used the vertical dimension to represent pitch (“spatial metaphor of pitch”). Middle panel: many participants also rapidly shook their arms, hands, and fingers to express random fluctuations of the sounds. Bottom panel: some participants created a gap between their hands to contrast noisy sounds from tonal sounds.
Fig 3.
Spectrograms of the eight referent textures used in the experimental study.
The left panel represents the four stationary textures (1-4), the right panel represents the four upward textures (5-8). Odd-numbered textures (1, 3, 5, 7) are tonal, even-numbered textures (2, 4, 6, 8) are noisy. Among tonal textures, textures 1 and 5 are (pitched) tonal (series of pure tones), whereas textures 3 and 7 are (tonal) granulated textures (resulting from granular synthesis based on a database of pure tones). Among noisy textures, textures 2 and 4 are based on filtered white noise, whereas textures 6 and 8) are made of granulated noises.
Fig 4.
Example of regression for pitch.
The slope of the regression was used as an estimate of the direction of the pitch.
Fig 5.
Box plots of the distributions of descriptor values for the vocal imitations of textures for the factors tonalness, granularity, and morphological profiles of the referent sounds.
The boxes represent the interquartile range. Horizontal bars within the boxes represent the median of the distributions. Circles represent the mean of the distributions. The dashed vertical lines represent the ± 2.7 standard deviation range. Crosses represent outliers outside this range. Empty boxes correspond to smooth textures. Shaded boxes correspond to granular texures. The top panel represent the aperiodicity of the imitations. The middle panel represents the “spectral direction” of the vocalizations (the pitch or spectral slope). The left bottom panel represents the depth of amplitude modulations for the imitations of noisy referent sounds. The right bottom panel represents the jitter of the imitations of tonal referent sounds.
Fig 6.
Box plots of the distributions of the gestural descriptors for the tonalness, granularity, and profile of the referent sounds.
Empty boxes correspond to smooth textures, gray boxes correspond to granular textures (see the legend of Fig 5 for more detail). The top panel represents the centroid of the wavelet spectrum of the wrist acceleration, corresponding to the speed of the gestures (higher values corresponding to slower speeds). The middle panel represents the spread of the wavelet spectrum of the wrist acceleration, corresponding to the complexity of the gestures. The bottom panel represents the speed of the right hand along the vertical direction.