Cite: Fajardo, I., Gómez-Merino, N., Ferrer, A., & Rodríguez-Ortiz, I. R. (2024). Hearing what you can’t see: Influence of face masks on speech perception and eye movement by adults with hearing loss. Journal of Speech, Language, and Hearing Research, 67, 3841–3861. https://doi.org/10.1044/2024_JSLHR-22-00562

Hearing without seeing the mouth? Masks, gaze, and speech understanding in adults with hearing loss

Study context

Speech understanding is multimodal: we listen and we “read” the face—especially the mouth. The pandemic made this obvious: what changes when the mouth is covered? We tested three situations (no mask, transparent-window mask, opaque FFP2 mask) and three presentation modes (audiovisual, video-only, audio-only) in adults with and without hearing loss (HL) .

What we investigated

We asked how masks affect (1) speech recognition accuracy and (2) where people look (eyes vs. mouth) during communication, comparing HL and typical-hearing (TH) adults .

How we did it

We tested 40 participants (20 HL, 20 TH). They repeated Spanish words and sentences across six conditions (mask × mode; incomplete design). We measured accuracy and, with eye tracking, the percentage of time spent on eyes and mouth. Testing took place in quiet to provide a clear baseline .

What we found

In audiovisual mode, HL participants recognized more words without a mask than with an opaque mask; transparent vs. no mask did not differ. In TH, mask type did not change accuracy (ceiling effect) .
Audiovisual input consistently benefited HL; for TH it only outperformed video-only (both groups ≈ ~30% in video-only) .
With opaque masks, viewers looked more at the eyes and less at the mouth; with transparent or no mask, they looked more at the mouth (useful visual cue) .
For illustration (words, HL): ~88% no mask vs. ~73% opaque; transparent ≈ no mask .

Why it matters

In settings where masks persist (e.g., hospitals, clinical sessions), transparent masks are advisable to avoid the hindering effect of opaque masks on speech perception in HL—provided fogging is controlled and front lighting reduces shadows on the window .
In classrooms and therapy, maximize audiovisual access and ensure good frontal lighting so the mouth is visible.
Training visual attention to the mouth and arranging seating/lighting to keep it visible can support comprehension in HL .

Future directions

Quiet-only testing; single talker; incomplete design (not all mask × mode combos); some sentence lists repeated; frontal lighting and window cleaning controlled in-lab; audio level (~65 dB SPL) not individualized (possible interactions with hearing-aid compression). Exploratory acoustics suggested slightly greater low-frequency attenuation with the opaque mask, with no clear high-frequency differences, but results are preliminary due to stimulus confounds. Future work should add noise, more varied materials, individualized levels, and real-world constraints.