Senescent Decline in Verbal-Emotion Identification by Older Hearing-Impaired Listeners – Do Hearing Aids Help?

Introduction

It has long been known that vocalizations play an important role in conveying emotions.¹ In the case of speech, the acoustic signal carries emotional information in several ways, for example the duration of the speech segments and their rhythm, the voice quality, the levels of intensity and the fundamental frequency and their variations over time.^2–4 There is substantial evidence that the ability to identify vocally transmitted emotions tends to decrease with increasing age.^5,6 However, there is no consensus as to the underlying mechanism(s) responsible for this decline (for a discussion, see Mitchell⁷): Is it the primary consequence of differential aging and degradation in neural mechanisms subtending emotion processing (eg, in the fontal and medial temporal lobes)^8,9 or a secondary effect mediated by the ubiquitous age-related changes in hearing sensitivity^10,11 and cognitive functioning¹² associated with the aging process? Recent clinical interest for this aging phenomenon is based on the assumption that an impairment in the processing of voice emotion is associated with lower quality of life, poorer social relationships, and greater levels of depression.^13,14 This raises the question of effective clinical interventions to mitigate the impact of this dysfunction on social interaction and everyday communication.

Hearing sensitivity, generally assessed by pure-tone audiometry, worsens progressively across the lifespan,^15,16 with one third of the population aged 65 years and above being affected by disabling hearing loss.¹⁷ Hearing aids (HAs) are currently the most common form of treatment for age-related hearing loss (ARHL). They provide frequency-specific amplification that restores, at least partially, audibility of those sounds that are no longer perceived by the hearing-impaired (HI) person. Their benefits in terms of improved speech intelligibility have been widely demonstrated.¹⁸ However, relatively little is known about whether HAs also impact the ability to recognize verbal expressions of emotions. It can be speculated that, as for speech identification, the acoustic amplification provided by HAs makes more audible those auditory cues that are important for emotion perception (eg, pitch height and contour).¹⁹ On the other hand, some of the signal-processing schemes used in HAs (eg, dynamic range compression affecting the temporal envelope of the processed sounds) might distort acoustic cues,^20,21 thereby having a deleterious effect on the ability to process vocal emotions.

So far, only a few studies investigated the auditory processing of voice emotions in speech in older-adult HI (OHI) persons when listening unaided and aided with HAs (for a recent overview, see Picou et al²²). Schmidt et al²³ assessed the emotional responses to speech sounds by asking OHI listeners (N=23), listening unaided or when wearing their own HAs, to rate short sentences in terms of valence (ie, their un/pleasantness) and arousal (ie, the level of activation produced). Those ratings were then compared to affect-related acoustic features (eg, the fundamental frequency, intensity) measured in the vocal stimuli. The results indicated that the OHI listeners, when listening through their HAs (but not when listening unaided), relied on acoustic cues similar to those used by young-adult normal-hearing (YNH) listeners, leading the authors to conclude that HAs help to restore the perception of vocal emotions.

Picou²⁴ also assessed the perception of emotional valence and arousal in OHI listeners (N=13), but this time for nonspeech stimuli such as non-verbal human vocalizations (eg, laughter), animal vocalizations (eg, dog barking), and machine sounds (eg, dentist drill). In contrast to the findings of Schmidt et al,²³ the ratings given by the OHI listeners were significantly different from those of the YNH listeners, even when the OHI listeners were fitted with hearing instruments used for research purposes. The signal processing provided by the HAs neither improved nor impaired the perception of emotions in the OHI listeners.

Finally, Goy et al²⁵ studied the perception of vocal emotions in speech in OHI listeners (N=14) who had to identify speech stimuli portraying one of several possible emotional states (eg, happiness, sadness, fear). The linguistic material was taken from the emotional speech corpus developed by Dupuis and Pichora-Fuller.²⁶ This corpus consists of short sentences each composed of the carrier phrase “Say the word … ” followed by a monosyllabic word (eg, “bean”, “bath”). Goy et al²⁵ observed that emotion identification did not significantly improve when the OHI listeners wore their own HAs, and the OHI listeners’ performance (averaged across the unaided and aided conditions) was significantly worse than that of unaided YNH listeners.

The apparent discrepancy in results across studies might be due to the difference in tasks (valence and arousal ratings vs emotion identification), stimuli (speech vs nonspeech), and/or the OHI listeners’ familiarity with the HAs (personal HAs vs research HAs). Also, due to relatively small sample sizes used in these studies, there might have been a lack of statistical power.

Given the potentially important clinical implications, further studies are warranted to establish the effects of ARHL and HA use on the perception of emotions carried by speech signals. In the present study, the unaided and aided ability to identify emotions portrayed over a longer segment of continuous speech was evaluated in a larger group of OHI listeners. The choice of a multi-sentence passage was motivated by the assumption that it is more representative of everyday speech communication and affords repeated sampling of (potentially more diverse) acoustic cues associated with a given emotion, thus resulting in more robust and better emotion perception in OHI listeners than with shorter linguistic material (such as sentences). All OHI listeners were experienced HA users and were tested with their own HAs to ensure that perceptual acclimatization to HA processing had already occurred.²⁷ To mimic everyday listening preferences and behavior of HA users in terms of volume control,^28,29 listeners were allowed to adjust the presentation level of the stimuli to a comfortable listening level.

Materials and Methods

Listeners

Twenty-nine (13 females, 16 males) older (age range: 58–86 years, mean: 71.7, standard deviation, SD: 8.7) native-French-speaking adults with bilateral sensorineural hearing loss, recruited from the patient pools of several HA dispensing centers located in the South-West of France, took part in the study. All had been fitted in both ears for at least 6 months prior to the study (the average duration was 6 years) with middle- or top-of-the-range behind-the-ear HAs from a variety of manufacturers: Siemens (N=11), Beltone (N=6), Widex (N=5), Phonak (N=4), Rexton (N=2), and Oticon (N=1).

Audiometric thresholds were measured with an Otometrics Aurical Aud audiometer (Natus Medical Denmark ApS, Taastrup, Denmark) and Telephonics TDH-39 headphones (Telephonics, Farmingdale, USA). Full audiograms for the ear with the better pure-tone average between 0.5 and 4 kHz (PTA) are shown for each of the OHI listeners in Figure 1. Despite some variability in the shape of the individual audiograms, on average, audiometric thresholds declined monotonically with increasing frequency, as typically found in ARHL. Better-ear PTAs ranged from 25 to 64 dB HL, corresponding to mild to moderately severe hearing impairment.³⁰

Figure 1 Individual (thin lines) and mean (thick line) audiograms for the better ear of the 29 older-adult hearing-impaired (OHI) listeners. The grey-shaded area represents ±1 standard deviation (SD).

Seventeen (7 females, 10 males) younger (age range: 21–31 years, mean: 23.6, SD: 3.1) native-French-speaking adult listeners with audiometrically normal hearing sensitivity (ie, ≤ 20 dB HL at all audiometric frequencies between 0.25 and 8 kHz) were also tested as controls.

The study complied with the Declaration of Helsinki and was approved by the Toulouse University Hospital review board (CER-072012), and listeners provided informed written consent prior to study participation.

Stimuli

The linguistic material was created from the transcript of an explanation given by a French glaciologist during a radio interview on the topic of climate. The selected 20-sentence passage mainly used narrative and informative discourse modes and the lexicon was characterized by a neutral polarity.

A male native-French-speaking professional actor read the transcript out loud several times, portraying each time one of the five possible emotional states (ie, “happiness”, “pleasant surprise”, “sadness”, “anger”, “fear”), as well as a neutral emotion. Recordings, using a 44.1-kHz sampling rate and 16-bit quantization, were made in a professional sound recording studio with a Schoeps MK5 omnidirectional microphone (Schoeps GmbH, Karlsruhe, Germany), situated at 1 m from the actor, and a Fostex FR-2 digital recorder (Foster Electric Company, Tokyo, Japan). Depending on the emotional state enacted, the duration of the recording varied from 84 to 167 s (mean: 118, SD: 31).

Procedure

Testing was performed in the audiometric booths of the HA dispensing centers from which the listeners were recruited. The booths complied with the French public health code,³¹ requiring the ambient noise level (measured over a 1-h period) not to exceed 40 dB(A) and the reverberation time for a 0.5-kHz tone to be shorter than 500 ms.

Listeners were seated in front of a Yamaha MSP5 amplified speaker (Yamaha Corporation, Hamamatsu, Japan), placed at 0° azimuth and at the same height as the listener’s head. The distance between the loudspeaker and the center of the listener’s head was 0.5 m.

A DELL Inspiron 5570 laptop (DELL Inc., Round Rock, USA) was used to display a graphical user interface (GUI) and to present the auditory stimuli. The GUI consisted of six virtual boxes, each showing the orthographical label corresponding to one of the five emotional states and the neutral voice. For each listening condition, each of the six versions of the passage was displayed as an unlabeled file icon on the GUI. Two additional file icons, each corresponding to a copy of one of the six audio files representing the emotions selected at random, were also presented on the GUI. The duplicates were used to prevent listeners from associating the last file icon still to be classified with the last unused emotion label. The listeners could listen to each file (by double-clicking on it with the computer mouse) as many times as desired before providing a response by dragging the file icon onto one of the virtual boxes.

Prior to testing in each condition, listeners were asked to adjust the presentation level of a short excerpt from the recording using a neutral voice until it was judged comfortable.

All listeners completed the task unaided. The OHI listeners also completed the task when wearing their own bilateral HAs; the order of the listening conditions (unaided and aided) was randomized. No feedback was provided. The completion of the task in each listening condition took approximately 15 min.

Scoring and Statistical Analyses

Due to the use of stimulus duplicates in the emotion-identification task, responses for emotions presented more than once in each listening condition were averaged prior to statistical analysis. All variables were normally distributed (according to Kolmogorov–Smirnov tests), except for the overall emotion identification by YNH listeners and the identification of individual emotions by OHI listeners in both listening conditions. Hence, the difference between age groups and the difference between listening conditions for individual emotions were assessed using the appropriate nonparametric two-tailed tests. The within-subject effect of HA use in the OHI group was assessed using a two-tailed t-test. Bivariate and partial Pearson’s correlation coefficients, followed by multiple linear regressions, were computed to analyze the association between emotion perception and several listener-related factors. For all tests, the criterion used for statistical significance was P<0.05. Statistical analysis was performed using SPSS software (version 24; IBM Corporation, Armonk, NY, USA).

Results

The average self-adjusted presentation level was 58.6 dB SPL (range: 54–63, SD: 2.9) for the YHN listeners, and 65.7 dB SPL (range: 58–74, SD: 3.9) and 61.4 dB SPL (range: 56–66, SD: 2.5) for the OHI listeners in the aided and unaided listening condition, respectively. For the OHI listeners, the PTA correlated significantly with the self-adjusted presentation level in the unaided listening condition (r=0.76; P<0.001) but not in the aided listening condition (r=0.31; P=0.101).

Figure 2 shows the percentage of listeners who correctly identified each of the six emotions for the YNH listeners and the OHI listeners in the two listening conditions.

Figure 2 Percentage of listeners correctly identifying each of the six emotional states, and percentage of correct identification across all emotional states (All) for the YNH listeners, and the OHI listeners when unaided (OHIunaided) and when aided through their own bilateral HAs (OHIaided). The emotional states are presented, from left to right, in order of the size of the impact of the HAs on emotion perception in the OHI listeners.

Emotion Identification by Unaided YNH and OHI Listeners

Most YNH listeners were able to correctly identify verbally transmitted emotions (89.2%), even though ~25% failed to recognize the neutral voice and sadness. In contrast, only just over half of the unaided OHI listeners (58.7%) correctly recognized vocal emotions, struggling particularly with sadness and fear which were identified correctly by less than 45% of the OHI listeners. According to a Mann–Whitney test, the observed group difference was significant (U=48.5; P<0.001).

Emotion Identification by OHI Listeners without and with Their Hearing Aids

The proportion of OHI listeners who were able to recognize verbal emotions when wearing their HAs (54.8%) was marginally lower than the proportion of unaided OHI listeners correctly identifying emotions (58.7%). However, the difference between the two listening conditions was not significant (t[28]=1.083, P=0.288). Analyzing the identification of individual emotions, both small increases and small decreases were observed in the aided listening condition: happiness and anger were identified by more OHI listeners (an increase of 6.9 and 1.7 percentage points, respectively), while fewer recognized the pleasant surprise, neutral emotion, fear, and sadness (a decrease by 3.5, 6.9, 10.3, and 12.6 percentage points, respectively). According to Wilcoxon signed-rank tests, none of these changes in emotion identification was significant (all P≥0.237).

As the duration of the speech passage varied for the different emotions, Pearson correlation coefficients were computed to test for an association between passage length and the proportion of OHI listeners correctly identifying the verbal emotion. No significant correlation was found in either of the two listening conditions (both P≥0.587).

Contribution of Listener-Related Factors to Emotion Identification in OHI Listeners

The relationship between overall emotion identification and each of the individual factors of the OHI listeners of age, PTA, and self-adjusted presentation level was assessed using bivariate Pearson correlations, followed by partial correlations controlling for the effect of the other two factors (eg, PTA and presentation level when studying the association between emotion identification and age). The results for both listening conditions are shown in Table 1.

Table 1 Correlations Between Each of the Listener-Related Factors of Age, PTA, and Self-Adjusted Presentation Level and Verbal-Emotion-Perception Performance in the Unaided and Aided Listening Conditions

The strongest and only significant correlation was that between emotion perception and age. With increasing age, the ability to identify verbal emotions in OHI listeners diminished both in the unaided (r = –0.70; P<0.001) and the aided condition (r = –0.61; P<0.001). This moderately strong association changed only marginally and remained highly significant when the potentially confounding factors (PTA and presentation level) were partialled out. Somewhat surprisingly, hearing loss as measured by the PTA did not correlate with emotion identification even in the unaided condition.

To explore further the relative contribution of these factors to the variance in emotion identification and to determine their combined explanatory power, multiple regression analyses were carried out separately for the two listening conditions, using age, PTA, and presentation level as possible predictor variables.

When entering all three predictor variables, the model was significant (both P≤0.002) and explained 49% and 44% of the variance in emotion identification in the unaided and aided listening condition, respectively. In both cases, age was the only significant contributor (unaided: standardized β = –0.695, t = –4.463, P<0.001; aided: standardized β = –0.705, t = –4.303, P<0.001). Tests to verify that the data met the assumption of collinearity indicated that multicollinearity was not a concern when predicting emotion identification performance in the unaided condition (Age: Tolerance = 0.84, variance inflation factor, VIF = 1.2; PTA: Tolerance = 0.38, VIF = 2.6; Presentation level: Tolerance = 0.43, VIF = 2.3) and in the aided condition (Age: Tolerance = 0.84, VIF = 1.2; PTA: Tolerance = 0.77, VIF = 1.3; Presentation level: Tolerance = 0.90, VIF = 1.1).

Discussion

In line with substantial evidence from previous studies using convenience or clinical samples of older listeners with varying degrees of ARHL (either reported by the authors or assumed based on the age of the listeners),^{6–8,12,32–34} our results showed that, compared to YNH controls, unaided OHI listeners have a lower ability to identify emotional states of a speaker based the on the acoustic information contained in speech. This group difference was observed even though a longer speech passage was used, and the presentation level was higher (on average by 7 dB) for the OHI group.

In contrast to previous findings,^10,32 PTA was not significantly related to the recognition of verbal expressions of emotions, and this, despite the relatively wide range of hearing impairments (varying from mild to moderately severe) included in the study. This observation did not change when focusing on hearing sensitivity in the low-frequency (ie, 250–1000 Hz) range (all P≥0.246), as was done in some earlier work.³²

Most importantly, our study failed to find evidence that, compared to unaided listening, the use of HAs by the OHI listeners improves the perception of voice emotions contained in speech and restores young-adult-like verbal emotion identification.^11,25,35,36

In both (ie, unaided and aided) listening conditions, age, but not audibility-related factors (ie, PTA and presentation level), was related to emotion perception in our OHI listeners aged 58 to 85 years. This confirms previously reported age effects observed across the wider adult lifespan, including young adults (eg, Christensen et al tested listeners as young as 22 years and as old as 74 years),¹¹ and suggests that verbal-emotion identification continues to decrease from young-old to old-old adulthood, as part of an aging process unrelated to ARHL.

The study used a single long test item (ie, the same multi-sentence passage spoken by the same talker) to assess verbal-emotion identification. This contrasts with previous investigations using many shorter test items (eg, words or sentences), sometimes uttered by different speakers (varying in gender and age).^11,26 While our approach is prone to biases linked to the choice of the speech token and the talker, it is compatible with the time constraints of a clinical assessment, and mimics the brevity of other tests of emotion perception (eg, the Emotional Perception Test³⁷).

Several hypotheses – some related to the device itself or its interaction with the listener, others related to the nature of the underlying deficit – can be advanced to explain why HA use did not affect emotion perception.

The most obvious HA-related hypothesis is probably that HA amplification was insufficient or inappropriate in the present study. Indeed, the actual gain delivered by the HAs was not verified at the time of testing (eg, by measuring real-ear insertion gains). However, all HAs had been professionally fitted by the listeners’ HA audiologists according to established gain prescription rules (eg, DSL,³⁸ NAL-NL2³⁹). Hence, gross misfitting or malfunctioning of the HAs is unlikely. However, it cannot be excluded that there was a mismatch between prescribed target gains and actual gains as implemented in the HAs. Indeed, HA audiologists modify the prescribed HA gains to accommodate the HA user’s listening preferences.^25,40 Hence, underfitting of the HAs is a common phenomenon, lending some ecological validity to the current experimental design.⁴¹

Another possible limitation of the present study, which is also a strength, is the inclusion of a range of HAs using the listeners’ personal settings. Opting against the use of the same research HA for all OHI listeners most likely introduced additional variability in the results due to non-identical signal processing implemented in the different HAs. On the other hand, this choice ensured a high degree of perceptual acclimatization to speech processed through the HAs. Importantly, it also meant that the study findings can be generalized to a larger, more heterogenous population of HA users.¹¹ Nevertheless, it would be of interest to study in future investigations which specific HA signal-processing schemes improve or hinder verbal emotion identification.

It is also conceivable that allowing listeners to adjust the presentation level (resulting, on average, in a lower presentation level in the aided than in the unaided listening condition) reduced any potential benefit of the HAs. Against this explanation argues the finding reported by Goy et al that, even when the presentation level was kept the same in the unaided and aided listening condition, no HA benefits were observed.²⁵ Furthermore, a reanalysis of our results, limited to the eight OHI listeners who chose the same presentation level in the unaided and aided listening condition, revealed that only one of them identified more verbal emotions with HAs than without them; six of them showed a reduction in the ability to identify emotions.

Lastly, the lack of improvement in emotion perception might be explained by the deleterious effect of signal distortions produced by HA processing (eg, dynamic range compression), off-setting benefits due to increased audibility.⁴² This hypothesis could be tested in future studies by comparing the effect of amplification with dynamic range compression to that of linear amplification, which does not distort the temporal envelope of acoustic signals.

Alternatively, the inefficacy of HAs in terms of restoring identification of verbally expressed emotions might have little to do with the devices themselves, if impairment observed in OHI listeners were not (or only to a limited degree) due to ARHL. Other audibility-unrelated factors associated with the aging process, such as changes in cognitive functioning¹² (that have already been shown to be associated with speech-identification difficulties),^43–45 and in the neural circuitry involved in emotion processing,^5,8,46 might affect the ability to recognize emotions. Our observation of a moderately strong negative correlation between age and emotion identification in the OHI group for unaided and aided listening even when controlling for PTA and presentation level is consistent with this hypothesis. However, studies of verbal-emotion perception in OHI listeners, in which cognitive functioning was also assessed, generally failed to find evidence in favor of the existence of a (strong) link between emotion perception and different cognitive abilities, such as working memory, verbal intelligence, and sustained attention.^7,8,25,47 The reduction in verbal-emotion identification also does not seem to be due to difficulties with the identification task per se (requiring cognitive processes for the labelling of emotions), as the discrimination of verbal emotions is also affected in this population.⁴⁸

As a final explanation, age-related changes in suprathreshold auditory processing abilities (such as changes in the sensitivity to temporal-envelope and temporal-fine-structure information) that occur in association with^49,50 or independently of audiometric loss^44,51 might also affect the perception of emotions carried in speech. These processing abilities are generally not assessed in investigations of verbal-emotion perception. Consequently, it is difficult to establish an association, let alone a causal relationship, between the sensory and emotional processing abilities. It is noteworthy that some of the suprathreshold auditory processing abilities deteriorate progressively from early mid-life onwards,^51–53 and continue to do so throughout old age,^54,55 thus matching the adult-lifespan trajectory for verbal-emotion perception.^6,33 Crucially, apart from restoring audibility, current HAs provide only limited compensation of the age-related changes in suprathreshold auditory processing (eg, by mimicking the compressive input-output function of the healthy inner ear), leaving many of the listeners perceptual problems unmitigated (eg, loss of frequency resolution, reduction in temporal-fine-structure sensitivity).

Conclusion

The reported results corroborate the existence of age differences in emotion processing across adulthood. Compared to young adults with normal hearing, older listeners with age-typical hearing losses showed a reduced ability to identify verbal emotions. This deficit was not linked to the listeners’ hearing sensitivity as measured by the audiogram. In addition, the use of HAs, the standard clinical intervention for ARHL, did not compensate for the dysfunction in verbal-emotion processing.