The hearing aid dilemma: amplification, compression, and distortion of the neural code

Hearing aids face a seemingly intractable problem: the amplification required to make speech audible also makes it less intelligible. But there is hope.

Like Comment
Read the paper

Hearing loss is a major problem with no adequate solution. Hearing aids can amplify soft sounds to make them easier to hear, but in loud environments they don’t really help. One of my main research goals is to understand why the benefits of hearing aids are so limited and how they can be improved.

“I can hear you, but I can’t understand you”

We often distinguish between the effects of hearing loss on intelligibility and audibility, i.e. whether sounds are heard correctly versus whether they are heard at all. Hearing loss decreases audibility because it decreases the sensitivity of the cochlea and, thus, increases the sound level required to elicit a response from the auditory nerve fibers that carry information about sound to the brain. This is the effect of hearing loss that is measured in a typical hearing test.

Hearing loss also decreases intelligibility, partly because of its effects on audibility; sounds that are harder to hear are harder to hear correctly. But even when sound levels are increased to well above the sensitivity threshold, intelligibility is often still impaired. This impairment is typically blamed on “supra-threshold” effects of hearing loss such as decreased frequency selectivity. Many hearing researchers, including me1, have suggested that the problem with hearing aids is their failure to compensate for these supra-threshold effects.

We set out to study the supra-threshold effects of hearing loss, hoping that it might lead us to some new ideas about how hearing aids might compensate for them. But the results were not at all what we expected. In fact, we found little evidence of any fundamental supra-threshold effects, at least for hearing loss that is only mild-to-moderate. In the end, we arrived at a totally different explanation for the intelligibility problems experienced by hearing aid users.

Examining distortions in the neural code

We approached our study from the perspective of neural coding. If auditory perception is impaired, it is because the neural code -- the activity patterns in the brain that carry information about sound -- have been distorted. By analyzing neural activity with normal hearing, hearing loss, and hearing aids, we were able to determine which features of the neural code were distorted by hearing loss and to measure the success of hearing aids in correcting those distortions.

We started by comparing the neural code for speech with normal hearing and hearing loss. We found, as expected, that hearing loss weakened neural activity in general and made the activity patterns elicited by different speech syllables more difficult to distinguish, reflecting decreased audibility and intelligibility (see Figure 1 for a schematic illustration of our results). We then examined the neural code for speech with hearing aids and found that activity was returned to normal strength overall, but the patterns elicited by different speech syllables remained hard to distinguish.


Figure 1 | The neural code for speech with normal hearing, hearing loss, and hearing aids:
Speech elicits complex patterns of distributed neural activity in the central auditory system. Intelligibility is determined by how easily the neural activity patterns for different speech elements can be distinguished. With normal hearing, the patterns for different consonants are easily distinguishable. With hearing loss (1), the patterns are weaker and, thus, harder to distinguish. Even with normal hearing, amplifying speech to high sound levels (2) makes the activity patterns for different consonants harder to distinguish, an effect known as rollover. Users of current hearing aids (3) appear to suffer from the same problem. An ideal hearing aid (4) would amplify sounds to restore the strength of activity while also transforming it to compensate for the deleterious effects of rollover.

So far, these results are consistent with our original assumption that hearing aids restore audibility but not intelligibility because there are supra-threshold effects of hearing loss that they fail to correct. But when we tried to identify the supra-threshold effects by examining the neural code for simple sounds like pure tones, we found that the neural activity with hearing aids was essentially normal*. Eventually, we questioned our initial assumption regarding supra-threshold effects and considered the alternative that the problem was not the hearing loss but rather the hearing aid itself.

Amplification and compression

Most hearing aids amplify sound using an algorithm known as wide-dynamic range compression (WDRC). The amplification provided by the algorithm decreases as the amplitude of the incoming sound increases, so the distribution of sound amplitudes is compressed. The need for amplification is obvious, but the benefits of compression are debatable2. We found that removing the compression helped for speech in quiet. But the neural code for speech in background noise remained distorted even without compression.

It’s turns out that the amplification in the hearing aid is also a problem. Even for people with normal hearing, amplifying sounds to the levels produced by a typical hearing aid decreases intelligibility substantially. (This effect is called “rollover” and it has been demonstrated many times. If you take a recording of speech in background noise and simply turn up the volume from a comfortable level to a high level, your ability to understand the speech will drop off considerably).

When we compared the neural code for speech with normal hearing and hearing loss at the same high sound levels produced by the hearing aid, we found that the activity patterns elicited by different speech syllables were equally distinguishable with or without hearing loss. We finally concluded that rather than suffering from supra-threshold effects of hearing loss, listeners with hearing aids are simply experiencing the same rollover effect as those with normal hearing**. And when we looked back through the literature on previous studies of speech perception with normal hearing and hearing loss at high sound levels, we found plenty of evidence to support this conclusion.

A neuro-perceptual perspective on hearing

So what now? People with hearing loss have no choice but to amplify sounds to high levels or they may not hear them at all. But when they do, they suffer the effects of rollover (see Figure 2 for a schematic illustration of the dilemma). Is there really any hope of improving hearing aids?


Figure 2 | The hearing aid dilemma:
With normal hearing (blue), the intelligibility of speech is high at typical conversational sound levels but decreases at higher levels, an effect known as rollover. With hearing loss (red, left), intelligibility is decreased at conversational levels because of decreased audibility. With amplification (red, right), the audibility of speech is restored but the effects of rollover keep intelligibility below normal.

I think there is, but seeing it requires a change of perspective. The traditional approach to hearing aid design treats the ear as a simple information channel and attempts to optimize the use of its limited dynamic range. But the critical information about complex sounds like speech are encoded in the fine details of neural activity patterns, not simple variations in overall activity level. Between the ear and perception, there are dozens of stages of processing through which the signal from the auditory nerve is mapped into the complex, distributed neural representation that underlies perception. It is in this neuro-perceptual space that hearing aids need to be optimized. Of course, damage to the cochlea will constrain the downstream possibilities. But, at least with mild-to-moderate hearing loss, there is still plenty of capacity left in the system.

Optimizing hearing aids in neuro-perceptual space is easier said than done. Until recently, adopting the simplified view of the ear as an information channel has been the only way to make the hearing aid design problem tractable; principled approaches to optimization in neuro-perceptual space simply didn’t exist. But they do now. Deep learning provides a new set of tools that are perfectly suited to this problem. We are currently working with these tools to give hearing aids a fundamental overhaul. Stay tuned.

References

  1. Lesica, N. A. Why Do Hearing Aids Fail to Restore Normal Auditory Perception? Trends Neurosci. 41, 174–185 (2018).
  2. Souza, P. E. Effects of Compression on Speech Acoustics, Intelligibility, and Sound Quality. Trends Amplif. 6, 131–165 (2002).

 Footnotes

* This is not to say that the neural code for simple sounds with normal hearing and a hearing aid are exactly the same; we didn’t explore every feature of the neural code. But there don’t seem to be any supra-threshold effects of hearing loss that can easily explain differences in speech intelligibility.

** Don’t forget that our study was focused on mild-to-moderate hearing loss; with severe cochlear damage there are many other problems.

Nicholas Lesica

Professor of Neuroengineering, University College London