How an app (yes, an app!) will revolutionise education for deaf students

Imagine you’re in class trying to take down notes for an important test next week. How you do in this test and what you’re learning in school are not without far-reaching consequences in how your life and career will pan out. Now imagine trying to do this without being able to hear.

This is the reality for deaf and hard-of-hearing students everywhere. Not everyone is privileged enough to have an interpreter and not every learning environment will meet their needs. Google’s Live Transcribe app may change deaf students’ learning potential forever.

“This app really helps all kinds of information, not just [making] education…accessible to a greater set of people. So if you’re hard of hearing or deaf, you don’t have to go to a special school anymore,” said Sagar Savla, a product leader in Google’s AI Research group, in a video conference at Google Malaysia last Friday.

“You can continue in the same hearing school or university and follow along and get conversations without having to learn or teach somebody else sign language.”

All you need is a smartphone and Internet connection for Live Transcribe to turn speech into large, easy-to-read text on your phone screen in real-time. There’s even punctuation and some context thrown in – for example, it is able to correctly capitalise the state in this sentece, “I am buying a new jersey in New Jersey.”

Available in 70 languages (including dialects such as Indian and South African English), the app also detects other sounds apart from speech, such as running water, a whistle or a baby crying. The app lets you set a primary and secondary language, a convenient feature considering that in many countries, several languages can be spoken in one sentence alone. The app is able to correctly transcribe two languages at the same time.

The proximity of these sound events are also displayed on the screen. If there’s silence, the app will make the phone vibrate once the person starts speaking again so they know to look at the screen to see what’s being said. Transcripts will be saved for up to three days. The app is free for download from the Google Play store.

In terms of storage, Sagar said it will “most likely not be a problem”. A six-year-old Android phone has been using the app for around one month now without crashing. Data usage is kept low through the app’s compression feature.

But there are some limitations; for one, it’ll only work in a one-to-one or small group setting and it’s not available for Apple phones. Accuracy isn’t 100 percent and Google is still a long way from reaching that. Only 70 languages are on the app now, though there are plans to add more in the future.

Despite this, it’s still hard to downplay the big benefits this app could have for deaf or hard of hearing students.

An analysis by the National Deaf Children’s Society (NDCS) found that deaf pupils in England are struggling “at every stage of their education” – almost two-thirds of hearing pupils leave school with two A Levels, but that figure goes down to fewer than half among deaf pupils.

In US universities, only 1.3 percent of enrolled undergraduates are deaf and 30.8 percent of deaf college students have learning difficulties. Much fewer complete college degrees when compared to their hearing peers.

They are more likely to enrol in online courses than hearing students, mostly due to the lack of “quality and availability of real-time accommodations in their immediate geographical vicinity”, according to a report by the National Deaf Center. Only one university, the private Gallaudet University in Washington DC, is designed to be barrier-free for the dead and hard of hearing.

Globally, 56 out of 70 million deaf people receive no education at all, according to the World Federation of the Deaf.

Increasing access is a key factor in improving this.

Sagar explains how speed, down to milliseconds, can impact how a deaf person accesses conversation. His team found that if the captions appear more than one second late, users would just be vegetative, passive listeners. The text now appears within 200 milliseconds, which feels “instant” allowing users to feel “more comfortable being able to chime in and speak up in a conversation”.

“We didn’t just want people to be able to follow…we want them to be able to participate in a full conversation.”

When tested in a classroom setting, the sole deaf student in a class of hearing students reported that he was able to follow what was being taught much faster. While the app wasn’t as accurate as a professional interpreter, it was way “way faster” – and free.

“This means the student could use the app for conversations with the teacher after the class was done as well as to participate in conversations much faster than a professional interpreter would,” said Sagar.

Liked this? Then you’ll love…

Should SEN children be enrolled in mainstream schools?

SEND students in England aren’t getting adequate support in schools – report