Title: Listening effort, also known as cognitive load, is measured with pupil dilation (pupillometry). When speech sounds are degraded, people expend greater amount of ‘cognitive load’ to decipher the meaning of the speech.
Legend: A: Subjects listen to sentences while their pupil size is measured using an eye tracker. Stimuli are varied from highly natural and understandable, to highly degraded. A simulation of a cochlear implant is used to degrade the speech and the smaller the number of channels the greater the amount of degradation. Time (milliseconds) is plotted relative to stimulus offset. The boxed area represents the time window traditionally used for aggregated analysis.
B: Change in pupil dilation as a function of spectral resolution. Data represent mean (bar height) and maximum (point) pupil dilation during the time window beginning 500 msec before and ending 2000 msec after stimulus offset (Fig. 1). Error bars represent ± 1 SE.
Citation: Winn MB, Edwards JR, Litovsky RY. (2015). The Impact of Auditory Spectral Resolution on Listening Effort Revealed by Pupil Dilation. Ear Hear. Jul-Aug;36(4):e153-65. doi: 10.1097/AUD.0000000000000145. PMID: 25654299
Abstract: Objective: Listening effort is a component of auditory perception involving cognitive processing or cognitive load. People with hearing impairment (HI) routinely report that they experience elevated listening effort and demonstrate increased effort associated with speech perception. This study measured the impact of auditory spectral resolution on listening effort. Systematic degradation in spectral resolution was hypothesized to elicit corresponding systematic increases in pupil dilation, consistent with the notion of pupil dilation as a marker of cognitive load. Design: Spectral resolution of sentences was varied with “vocoders” which are designed to simulate front-end processing of a cochlear implant. Pupil dilation was measured after subject-specific luminance adjustment and trial-specific baseline measures. Mixed-effects growth curve analysis was used to model pupillary responses over time. Results: Pupil dilation grew with each successive degradation in spectral resolution. That is, the more the speech was degraded, the greater amount of listening effort, or cognitive load, displayed by the participants. Furthermore, pupillary responses were not related to the speech understanding. That is, even when reaching 100% correct, the effect of spectral resolution on pupil dilation persisted. Conclusions: Speech intelligibility scores alone is not sufficient to quantify the effort required to understand speech with poor resolution. Degraded spectral resolution results in increased effort required to understand speech, even when intelligibility is at 100%. Pupillary responses are a sensitive and highly granular measurement to reveal changes in listening effort. Pupillary responses might potentially reveal the benefits of aural prostheses that are not captured by speech intelligibility performance alone as well as the disadvantages that are overcome by increased listening effort.
About the Lab: The goal of the Litovsky Lab is to investigate the role of the auditory system in sound localization and the ability to understand speech in noise. The lab studies the ability of deaf children and adults, who are fitted with cochlear implants, to function in complex listening environments. We focus on patients who are fitted with bilateral (two) cochlear implants. Our ultimate goal is to use reverse engineering approaches to restore functionality to levels of performance seen in people with normal hearing. We study many older adults who are also faced with everyday challenges of increased listening effort. Finally, we are also interested in the development of auditory perception in young children.