Participants

Twenty-two HI children (13 males and 9 females, mean age μ = 10.45 years, standard deviation σ = 3.13 years) participated in the experiment. Nine were CI listeners, and thirteen were HA listeners. Children were affected by non-syndromic hearing loss (“GEN NO SDR”) in 8 cases (6 GJB2 gene mutations, 2 other gene mutations), syndromic hearing loss (“SDR”) in 4 (2 Usher syndromes, 1 chromosomal instability, 1 Waardenburg syndrome), and 1 enlarged vestibular aqueduct (inner ear malformation, “IEM”). Other causes of hearing loss (“Other”) were congenital cytomegalovirus infection in 2 cases, chemotherapy with platinum derivatives for neuroblastoma in 2, preterm delivery in 1, and prolonged neonatal intensive care unit stay in 1 case. The cause was not identified (“ND”) in 3 cases. The two populations were similar in age (μ = 10.3 years, σ = 3.0 years for the CI listeners and μ = 10.5 years, σ = 3.2 years for the HA listeners), but they differed substantially in terms of interaural experience of their devices (μ = 2.4 years, σ = 2.9 years for the CI listeners and μ = 0.1 years, σ = 0.3 years for the HA listeners). All participants confirmed verbally that they were right-handed and had no diagnosis of motor impairment. Recruitment started on June 17th, 2022, and ended on January 19th, 2023. Anonymized data became accessible to the authors on January 22nd, 2023. Table 1 displays the collected participants’ data.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Age, cause of hearing impairment (GEN NO SDR: Non-syndromic hearing loss; SDR: Syndromic hearing loss; IEM: Inner ear malformation; ND: Not identified, other: Other causes), ear devices, and years of experience with the device for each child.

https://doi.org/10.1371/journal.pone.0312073.t001

Setup

The acoustic reproduction system consisted of 13 Seeburg i4 loudspeakers (SEEBURG acoustic line GmbH) driven by a Sonible d:24 multi-channel amplifier (Sonible GmbH). These loudspeakers were arranged to form a semi-circular array with a radius equal to 1.4 m in a small enclosure measuring 3×2.6 m, having a 60 dB-reverberation time of 200 ms. With this array, 13 sound sources radiating from equally-spaced horizontal angles of arrival were reproduced across an egocentric scene spanning between −90° and + 90°, with angles equal to 15° between each pair of adjacent loudspeakers.

A three-dimensional (3D) virtual environment (VE) was developed in the Unity3D programming environment. An Oculus Quest 2 HMD, including the Oculus Touch hand controllers (Meta Platforms Technologies Ireland Limited, Dublin, Ireland), was employed to reproduce the visual scene. The HMD tracks the 3D position of the head with submillimeter precision [28]. The four-valued quaternion representing the orientation of the head in the 3D scene had a precision of ±1° at 20 Hz sampling rate [29]. The data from the HMD were received via the mqtt protocol [30] running on a 2.4 GHz Wi-Fi connection by an mqtt broker as a Docker container (Docker, Inc.). From here, data were sent to a custom client app, allowing the experimenter to monitor head tracking and hand pointing and to check whether the connection between the Oculus and the computer was constantly up and running. A Max (Cycling ’74) real-time sound synthesis patch running on the same computer was used to reproduce the auditory scene at runtime. Sounds with 16-bit resolution at 44.1 kHz sampling frequency were sent to the loudspeakers by a MADIface USB 2.0 Audio Interface (RME GmbH).

A seat whose height could be adjusted by the experimenter was placed on the focal point of the semi-circular array so that the head was distant 1.4 m from the speakers at elevation zero. The experimenter aligned each virtual source to the corresponding loudspeaker by aiming at each speaker from the seat and then reading the angle displayed by the client app. This angle defined the target angle. The angular resolution of the pointer was set to 1 degree, based on the Oculus controller accuracy rotation found in the literature [31].

Stimuli

The acoustic stimulus consisted of pulsated pink noise, with each burst lasting 200 ms and completed with a 100 ms linear onset and a 100 ms linear decay. Adjacent noise bursts were separated by 200 ms of silence. This acoustic stimulus has been selected because normal listeners localize it easily; it has a rich broadband spectrum [32]. Furthermore, it provides a periodic temporal envelope enabling listeners to capture the binaural cues relevant to spatial hearing. The pulsated stimulus lasted until a participant produced a response. It was presented at a sound pressure level (SPL) equal to 65 ± 1 dB, measured with a calibrated meter (XL2 Sound Level Meter, NTi Audio). SPL was measured during setup by aligning the meter to the experimenter’s external ear while he was seated in the test focal point. The measurement was repeated for each speaker on both ears.

Procedure

Before each session, the seat’s height was adjusted to align the loudspeakers at the participant’s ear level. Then, participants were instructed to use one controller with their dominant hand to point toward the sound source position. At this point, they were invited to sit and wear the HMD. The interpupillary distance was adjusted, and the real and virtual worlds were aligned in the HMD through a spatial calibration procedure. Calibration was performed by asking each participant to point to specific visual markers occupying a playground until the system recorded every hit to a marker to mismatch with the corresponding loudspeaker by less than 1°. The resulting calibration was loaded through the Oculus Guardian system.

During the task, participants were immersed in a VE consisting of a homogeneous landscape free of any absolute azimuth reference (see Fig 1). They were listening to sounds coming from the loudspeakers and were in a condition to point to a guessed sound source position with the controller in their hand. A beam was displayed to give participants visual feedback about the pointing direction. Participants were not instructed to respond as soon as they heard a sound nor informed that the stimuli came only from the frontal hemifield.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. The 3D scene with the beam pointing to a guessed sound source position.

https://doi.org/10.1371/journal.pone.0312073.g001

Before a test session started, participants completed a brief training session of five trials. They were invited to take the time they needed to produce a response. Each trial began when one loudspeaker started to reproduce a stimulus; concurrently, the system started to track the participant’s head movement. It finished when a participant pulled the Oculus Touch trigger. At this moment, the acoustic stimulus stopped. The system recorded the guessed angular position and the trajectory of the participant’s head during the trial. After pausing for one second in silence, a new trial began.

Before a test session started, participants completed a brief training session of five trials. They were invited to take the time they needed to produce a response. Each trial began when one loudspeaker started to reproduce a stimulus; concurrently, the system started to track the participant’s head movement. It finished when a participant pulled the Oculus Touch trigger. At this moment, the acoustic stimulus stopped; the system recorded the guessed angular position and the trajectory of the participant’s head during the trial. After pausing for one second without any acoustic stimuli, a new trial began.

Conditions

Each session included multiple test conditions: up to four for the HA listeners and three for the CI listeners. In each condition, the stimulus was presented 5 times from each loudspeaker position across a sequence of 13 × 5 = 65 trials. The sequence was constrained to equate the number of target position shifts alternating leftward and rightward. Every participant received a randomly rotated version of this constrained sequence in such a way that our listeners completed a random module of a series of angular arcs in a test session. To this end, the angular position forming the sequence tail was pasted to the sequence head, and later, the first trial was removed from the analysis. We deliberately chose not to reset the participant’s head to a starting position after each trial since the literature reports that interruptions decrease performance in individuals who are cognitively engaged in a demanding task [33]. Considering the age of our population we favored engagement, by letting participants keep focus on the pointing task.

Each participant performed the task first with both devices turned on (“ON”), then with one device turned on and one off (“L” for the left device turned on, or “R” for the right device turned on, respectively) by randomly starting with either the left or right ear, and finally with both devices turned off (“NO”). The ON condition was presented first during each test session because it provided an everyday listening context to which participants were accustomed and confident. The NO condition was omitted if a patient’s pure tone average threshold in the frequency range [0.5-4] kHz was above the stimulus level used for the test. For this reason, only nine HA listeners completed the NO condition. Five HA listeners could not complete the L and R conditions either since their session had to be stopped immediately as they reported annoyance or fatigue to the experimenter. In the end, four HA listeners completed the whole session in all four conditions; four completed only the ON, L, and R conditions, and five completed only the ON and NO conditions. Conversely, CI listeners completed the ON, L, and R conditions. We analyzed only sessions including the complete set of 65 trials, except seven sessions that were completed by CI listeners (two in both the R and L conditions, one in the R condition, one in the L condition, and one in the ON condition), each missing one trial that was not recorded due to a technical problem.

Data analysis

From the 3D array of the positions and the 4D quaternion, we computed the difference between the target angle and head orientation angle when a response was produced (head rotation) and the total distance covered by the head during each trial (head distance). The latter has already been used to measure head dynamism in studies examining spontaneous actions during music listening [34]. The signed error was computed as the difference between the target and the pointed angle [35]. From it, we computed the unsigned error as the absolute value of the signed error. While the signed error indicated angular bias across repeated trials (e.g., the tendency to shift leftward or rightward), the unsigned error quantified overall accuracy. In the following, we will name the signed error as bias and the unsigned error as accuracy. Although the unsigned error is continuous by nature, its measurement to the sexagesimal degree had to be analyzed with data bins 1 degree apart from each other due to the precision of the Oculus.

The experiment had a mixed design, where the factors were the two populations with HAs and CIs and conditions, four for the former population and three for the latter. No participant with CIs had a hearing threshold sufficient for attending the NO condition; hence, seven subsets were analyzed. As mentioned before, not all participants performed the test in every condition. The mean μ and standard deviation σ of the unsigned error were computed for each subset. Trials resulting in unsigned errors larger than three standard deviations above the mean deviation per target angle were considered outliers and removed from the respective subset [36]. Exclusion of the outliers is a common procedure in sound localization studies [23, 35]. In our case, the outliers were finally 79 out of 4218 trials, i.e., 1.87%. A wide variance in the data was noted during the training sessions, especially in terms of localization accuracy by the CI listeners. As in other studies [16], this variance was mapped on a logarithmic scale, in such a way as to de-emphasize larger variations. The median of each participant’s five repeated measurements was computed for each loudspeaker position. It is reasonable to assume that adult listeners with NH responses are normally distributed because they are likely stable and consistent across positions [37]. The localization of a specific loudspeaker position in children who experienced auditory deprivation before being bilaterally implanted is less likely to follow a normal distribution [38]. Since our subsets often violated the assumption of a normal distribution of their residuals, linear mixed-effects models were not chosen for the analysis.

The analysis first compared each experimental variable of the two populations in the ON condition. Then, the other conditions were analyzed separately for each population. The data were hierarchical; they were aggregated via the median of all positions per participant and condition; moreover, they were divided by position and compared by condition. A Shapiro-Wilk test was performed to check if data residuals followed a normal distribution [39]. Given the limited sample size in some cases, the normality of the residuals was also tested with the Anderson-Darling test [40]; we did not report the results of the latter test because they rarely disagreed with the Shapiro-Wilk test results (14 cases over 504); when they did, we did not find statistical significance with any test of the subsequent analysis. If the samples came from the same population and the normality assumption was violated, a Friedman test [41] was performed, followed by a post-hoc Nemenyi test. The test employs the critical difference (CD) statistics and, according to Nemenyi [42], was developed to account for a family-wise error, hence being already a conservative test. For this reason, we did not apply p-adjustments. If the residuals from the same population followed a normal distribution, sphericity was checked with a Mauchly test [43]. If the sphericity hypothesis was met, we checked the equality of the means through an RM-ANOVA and performed a post-hoc analysis with a t-test; if sphericity was violated, the same tests were performed, but with a Greenhouse-Geisser correction of the p-value. If the samples came from two populations in the ON conditions, we checked the normality with a Shapiro-Wilk test and the homoscedasticity with a Levene test [44]. If the normality and homoscedasticity assumptions were met, we compared them with a t-test. If the latter was violated, a Welch t-test was employed. Otherwise, if evidence of a violation of normal distribution was found, a Mann-Whitney U test was used to compare the distribution with residuals with similar variances; diversely, a Yuen test [45] was used. The effect size was reported for every test; the partial η² was computed for the repeated measures ANOVA and the Hedges’ g for the pairwise tests. An effect size (named W) was estimated as in Tomczak [46] for the non-parametric Friedman test. Hedges’ g was computed from Cohen’s d using an average variance if the samples came from the same population. A correlation analysis for each condition’s data was performed using Spearman’s ρ correlation coefficient since we did not assume a normal distribution of the variables. All tests were two-tailed. A Bonferroni correction was applied for every multiple comparison, except for the Nemenyi test. Statistical significance was set at α = 0.05. The analysis was made using the Python packages Pingouin [47] and Scipy [48] and the data visualization libraries Seaborn [49] and Matplotlib [50].

Bilateral hearing: CI head movements and HA binaural skills

The bias and accuracy medians represented by the signed and unsigned error boxplots in Fig 2 ranked as expected among populations [51]. CI listeners’ accuracy with both CIs turned on (median: 18.4°) was consistent with previous research, where absolute azimuth errors in the frontal space did not exceed 39.4° [38, 52]. Nevertheless, our tests uncovered higher variances and numbers of outliers, probably because no visual cues were available to support localization, unlike previous reports that used a touch screen for response validation, and in which loudspeakers were visible [38, 52]. CI listeners’ accuracy was significantly worse than HA listeners’, both medians and across positions; the difference was not statistically significant at the most eccentric positions, where binaural ability was less necessary. Moreover, the CI listeners’ unsigned error peak was reached in correspondence with a central target position, i.e., 15°, where the effect size of the difference with the HA listeners’ unsigned error was the highest (g = 1.78). These results confirmed the poor ITD sensitivity of CI listeners [53, 54], probably due to two main reasons: the absence of auditory experience during an early critical period [55], and the pulse rate at which CIs operate [56]. CI processors generally run at fixed rates between 900 and 3700 pulses per second (pps) [57]; faster temporal sampling of speech envelopes might improve speech recognition in CI users. Nevertheless, even 600 pps could be “too fast for ITD,” given the poor performance of CI listeners at rates of 300 pps or above [56].

An inspection of bias and accuracy revealed that the signed and unsigned errors in HA listeners performing localization with both devices turned off were close to the ones made with the devices turned on: the medians of the accuracy were 11.4° and 6.0°, respectively. The benefit of HAs to spatial hearing was found to be significant in old adults, even if poorer than predicted [15], and was confirmed in children. However, it cannot be generalized as not being statistically significant. The median of the angular bias improved when both devices were turned off (−1.0°) rather than on (2.0°), even if not significantly. This result has already been found in some studies [58]. It is interpretable as a residual binaural sensitivity of HA listeners, who can achieve good localization performance without device support. The importance of a prelingual binaural experience is supported by the strong correlation between age and accuracy in HA listeners when both their devices were turned on or both are turned off. Older listeners consolidated a longer experience, and relied on it for spatial hearing; in the HA ON condition, these individuals did not rely on dynamic cues produced by head movements, as the negative correlation between the head distance and the age suggested.

Concerning motor activity during the task in the ON condition, the results in Table 2 about head distances show that CI listeners were especially active; the head distance covered by HA listeners was significantly smaller. The positive correlation between head distance and unsigned error in the CI ON, HA ON, and HA NO conditions can be interpreted as a sign of uncertainty in localization [59]. Mueller et al. [60] found that head movements disturbed localization, yet they instructed their participants to move or, conversely, keep their heads still before attending to specific listening conditions. Head movements were spontaneous during our task and may have been amplified when a target proved difficult to localize.

Asymmetrical hearing: CI intensity maximization and HA active search compensatory strategies

The accuracy and angular bias of CI listeners were significantly impacted by asymmetric hearing. The former decreased significantly, while the variance of the latter soared when one CI was turned off. Both were indicators of uncertainty and difficulty in localization. Unsigned errors increased at every target location except for ±15°. Accuracy did not significantly deteriorate for a target positioned in the aided ear’s hemifield if the CI was turned off on the other side. These findings suggested that CI children’s localization was mostly based on intensity [16]; their strategies aimed at maximizing intensity in the aided ear. This hypothesis was supported by the significant correlation between head rotation and accuracy. When the assisted ear pointed towards the source, the intensity was at its maximum, and the best performances were obtained. As shown in Fig 2, angular bias leaned toward the aided ear in CI asymmetrical conditions. CI children’s head distance seemed to mitigate the asymmetrical intensity. Moreover, it correlated positively with the head rotation when only the left CI was turned on, indicating a search for the intensity peak at the left ear.

Even if the signed and unsigned error variances increased in both asymmetrical conditions, the effect of asymmetrical hearing on the accuracy of HA listeners was way more limited. The post-hoc analysis found significant differences only at 15° between the bilateral hearing condition and the condition with the left HA turned off, and between the medians of the former condition and the one with the right HA turned off. Again, these two findings suggested that having just one device turned on created confusion and increased difficulty in localization. HA listeners compensated for the adverse condition more efficiently; the difference in accuracy was found at an almost frontal target position. Here, binaural differences are more subtle, requiring greater sensitivity. Unlike CI listeners, HA children’s data in asymmetrical conditions did not present a correlation between head rotation and unsigned error. This indicated that they probably used residual binaurality for spatial hearing.

Fig 3 shows a general tendency for HA children to orient the right ear towards the source in every condition. An advantage of the right ear in auditory processing has been firmly established in decades of behavioral, electrophysiological, and neuroimaging research [61]. Even the correlation between head rotation and unsigned error in the condition with both HAs turned on can be read as an attitude to favor right-ear intensity maximization.

Bias and head rotation were positively correlated when only one device, CI or HA, was turned on, indicating a tendency to couple the pointing gesture with head rotation under asymmetric listening conditions. Our result extends the training-induced observed behavior of the NH [62] to the HI.

Previous research found no relationship between head dynamism and age in children with CIs [5], and our results support this. Children with HAs behaved differently. Head dynamism correlated negatively with age in the ON condition. HA listeners who developed binaural sensitivity behaved like the NH population, as they did not need to rely on dynamic cues elicited by head movement. They did so in asymmetric hearing conditions. Older HA listeners produced more pronounced head dynamism, as the positive correlation between head distance and age showed. They faced localization cue disruptions with dynamical changes in binaural cues, enabling a more reliable response in difficult hearing conditions [63]. The negative correlation between unsigned error and age confirmed that they were also the most successful in localization, at least when the right HA was turned off. Ultimately, the correlation with age suggests that active listening is refined throughout life [5].

These findings could be exploited when planning specific interventions for diverse HI pediatric populations. Children with HA must be assisted in acquiring binaural skills, not necessarily by insisting on head movement or unilateral maximization of intensity, unless they are in adverse hearing situations like speech-in-noise. The bilateral perception of children with CI should be trained, and vice versa, by insisting on active search motor behavior that includes unilateral training of even the weaker hemifield.

Spatial hearing investigations ask clinical research to take everyday listening into deeper consideration. The current study highlighted how crucial it is to support more ecological scenarios so that active listening can be taken into account when assessing hearing ability. An open research question is whether our results would be confirmed after the substitution of noise bursts with the most informative sound messages we are exposed to during the day, that is, speech. Although pink noise is a standard stimulus in the literature [24], Neuman et al. [51] did not find differences in localization accuracy of speech and pink noise sources, suggesting that the salient ILD cues are preserved by vocal messages. However, the processing algorithms that an auditory device applies to pink noise in terms of signal compression, noise reduction, and directional sensitivity remain generally unknown outside the manufacturing company. At any rate, we remain non-committal about the effectiveness of an accurate rendering of ITDs with CI because individuals who have not accumulated enough experience with these cues may not be responsive as expected.