Research is being carried out into how smartglasses equipped with a camera, AI, and a data connection could help deaf people better follow real-time conversations. The technology won’t replace hearing aids, but enhance them instead, and you can think of it a bit like equipping them with smart noise cancellation.
The concept is being studied at Heriot-Watt University in Scotland, and project lead Professor Mathini Sellathurai explained how it worked:
“You simply point the camera or look at the person you want to hear. Even if two people are talking at once, the AI uses visual cues to extract the voice of the person you’re looking at. It’s aimed to support people who use hearing aids and who have severe visual impairments, but it could help anyone working in noisy places, from oil rigs to hospital wards.”
Sellathurai said the technology will give hearing aids, “superpowers,” and went into further detail about how they will work, and where AI fits in. The camera on the smartglasses tracks a speaker’s lip movements, and through lip-reading technology and AI cleaning up background noise and other surrounding conversations, a “clean” version of the speaker’s voice is sent to hearing aids or headphones.
Cloud processing
The processing is not performed on device though. The smartglasses will send data to a connected phone, where it’s then sent to cloud servers. There, AI smarts are applied and the final version returned to the wearer. According to Sellathurai, there’s only a slight delay when the data is sent over a 5G connection.
The team is currently using cloud servers in Sweden, and says this approach is necessary to avoid putting too much strain on a wearable device. However, there may be privacy concerns over not only a device wearer recording a conversation in real-time, but also the integrity of that data when it’s transferred and analyzed in the cloud.
Using smartglasses with a camera, hearing aids, and AI technology to enhance hearing may sound like an overly complex way of addressing the problem, but it’s an established approach known as audio-visual speech enhancement. It is already used in hearing devices to enhance the voice of a person speaking in a noisy environment, in video production, and in hearing aids as a type of noise cancellation.
Competition and prototypes
Similar technology has been demonstrated in the past. Speech-to-text technology from TranscribeGlass was incorporated into the Vuzix Z100 smart glasses, where conversations were transcribed onto the Z100’s screen for the wearer to read, like subtitles for the world around them. The Nuance Audio Glasses combine both corrective lenses and hearing aid technology into smart eyewear. The Ray-Ban Meta, one of the best current smartglasses you can buy, include accessibility features for those with hearing and vision impairments.
TranscribeGlass’s eyewear starts at $377 and requires a $20 per month subscription, while Nuance Audio Glasses cost from $1,200. Professor Sellathurai said the intention is to increase the amount of options available for hearing-impaired people, and help, “children and older adults access affordable, AI-driven hearing support.”
The researchers are in talks with hearing aid manufacturers about partnerships and how to reduce costs, and hope to have a working prototype pair of smartglasses in 2026.
