My father’s hearing aid batteries died. He was in the emergency room (ER). The doctor was asking questions. My mom, sister and I had to interpret for my Dad but he couldn’t hear us.
I took out my smartphone and turned on the Google Translate app. Since I type very slowly and poorly in Russian, it was faster for me to speak into the Google Translate app in English and show my Dad the Russian translation.
The doctor said, “This app is a miracle!”
We were desperate.
When you’re desperate in the hospital, you don’t care about Google Translate’s “creative” translations, weird word choice or that Google Translate translates your words to your father in the formal tense instead of the informal tense or mistakes your gender. If you do care, then your priorities are in the wrong place!
Since that episode, I had many other experiences with my father’s faulty hearing aids and used Google Translate to communicate, sometimes with comical translations.
Live Transcribe app
In February, Google came out with the Live Transcribe app that does live transcriptions for conversations in 70 languages using speech recognition technology. This app has revolutionized the way I communicate with my father when he can’t hear me. Instead of saying one sentence at a time in Google Translate, I just leave the Live Transcribe app on and my Dad reads our conversation in real-time. The text appears on the screen as we speak. It works very well in Russian! Sometimes the app displays the wrong word. I need to read what my Dad sees and show him when a word is incorrect. Sometimes, I have to repeat the word several times before the correct transcription appears. If the right word doesn’t appear, I tap on the keyboard button and type the word I need. The app is free but only works with an Internet connection.
[embedyt] https://www.youtube.com/watch?v=jLCwjIaPXwA[/embedyt]
Video about Live Transcribe
How do other people in similar situations communicate on a daily basis?
Even before my father’s age-related hearing loss, I was curious about how deaf and hard-of hearing (DHH) people communicate. Anywhere from 37 to 140 out of every 1,000 people in the United States have some kind of hearing loss. Either you or someone you know needs or will need assistive hearing devices. (If you listen to music at a high volume, beware about your potential hearing loss.)
I am relaying what I have learned about how American Sign Language (ASL) and written and spoken English are different. I show some ways the deaf or hard-of-hearing use technology to communicate with each other and with people in the hearing community and the problems or missing features with these technologies. I also provide some information about upcoming technologies.
I am new to this topic and am in no way an expert of DHH communication. I am just offering what I have learned. If you know of other effective technologies, please write a comment.
Gallaudet University and ASL
I visited Gallaudet University in Washington DC. It’s the only four-year college in the US for the deaf and hard-of-hearing (DHH). Half of the staff is deaf, and half is hearing. All staff know American Sign Language.
While walking around the campus and the surrounding area, I saw many people signing. I am used to being in places where I can’t understand the language spoken, but sometimes I can pick up a couple words or make out what people are saying by their expressions. But ASL was so totally foreign to me that I couldn’t interpret any of the signs. I felt like I was a foreigner in my own country! I walked into an Ethiopian restaurant. Instead of hearing the Amharic language spoken, I saw people using sign language.
Gallaudet University, There is no other place like this in the world
Structures of American Sign Language (ASL) and English are different
Imagine this:
You’re deaf. ASL is your first language. You have to learn how to write and read in English as a new language. Since you don’t hear spoken English, your only way of learning English is by reading. You’re at a disadvantage to hearing people who learned English since they were infants by hearing the language.
Example of different sentence structure:
English: The girl throws the dog a bone.
ASL: GIRL — DOG BONE — THROW
Literacy rates among the deaf are low because it’s hard to learn written English when the structure of ASL is different from written English. People who are born without being able to hear, don’t hear English spoken so they don’t learn English like other children do by listening to their parents, friends and family speak in English.
Written English is not a universal option for the deaf.
Not all deaf people will successfully communicate via written English.
Many people with hearing disabilities rely on human ASL interpreters. Apps that allow for text chatting and voice-to-text features may not appeal to all DHH users because they don’t incorporate ASL.
I don’t know if the structures of other languages have vast differences between their written and signed languages. If you have any information about this, please add a comment below.
Why sign language is important in emergencies
Did you ever wonder why emergency announcements on TV usually have an ASL interpreter rather than just closed captioning (text in English at the bottom of the screen)?
It’s to make sure that all deaf people can understand what to do during the emergency.
Do you remember New York City Mayor Michael Bloomberg’s sign language interpreter during Hurricane Sandy in 2012?
Lydia Callis’ dramatic ASL to describe the severity of the hurricane made her famous and took the attention away from the mayor.
[embedyt] https://www.youtube.com/watch?v=gvaEV_5FJRs[/embedyt]
Video of ASL interpreter Lydia Callis and Mayor Michael Bloomberg
Or maybe you saw the Saturday Night Live sketch of Mayor Michael Bloomberg and his (fake) New York sign language interpreter. She isn’t really signing. She’s making funny faces and gestures.
Saturday Night Live fake ASL sketch
Some ways that deaf and hard-of-hearing (DHH) communicate via ASL using technology
Automatic Speech Recognition (ASR) with WhatsApp
In classroom settings, where mixed groups of DHH and hearing students are supposed to work in groups and ASL interpreters may not be available, hearing students speak into an app like WhatsApp with Automatic Speech Recognition (ASR) that converts speech to text.
Problem/What’s missing:
- ASR is not perfect, especially in noisy environments
- Real-time texting (seeing what people type as they type, before pressing “Send”)
- Hearing people can hear someone else formulate their thoughts as they speak. With messaging apps, you only see what the person has written after the message has been sent. This isn’t a natural way of communicating.
Video chats
Video chats via Skype, FaceTime and other video messaging platforms are preferred, rather than text or instant messages because the DHH want to see each other on screen and use sign language. With video calls, you can have a three-way call and add a sign-language interpreter to communicate between hearing and DHH people. This is easier to do on a desktop computer or laptop because you can more easily see the different people on screen (the interpreter and the person you are calling).
Live transcription with video calls
Skype Translator uses speech-to-text technology to convert spoken language into text for video calls between DHH and hearing people.
Image of Skype Translator: text converted to speech, speech converted to text
Scenario:
- DHH person types to hearing person
- Hearing person reads or hears text spoken aloud by Skype Translator
- Hearing person speaks —> spoken word converted to text for DHH person to read
Video on how to use Skype Translator Preview:
Problem: This can be expensive if people are not using Wi-Fi and are using mobile data on their phones to use Skype Translator. Some people may also like asynchronous (not live) video messaging to send video messages to friends/family to watch when they want to rather than doing a live video chat.
A DHH person calling a hearing person
What if you’re DHH and need to call a store, such as Target, to place an order and communicate with a hearing person over a telephone call? You can’t call Target and do a video call via Skype with an ASL interpreter. Unless you have a friend, family member or caregiver nearby who knows sign language and can interpret for you, which technologies can you use to make a phone call?
Option #1
The Telecommunications Relay Service (TRS) is a telephone service that allows persons with hearing or speech disabilities to place and receive telephone calls. When a person with a hearing or speech disability initiates a TRS call, the person uses a teletypewriter (TTY) or other text input device to call the TRS relay center, and gives a communications assistant the number of the party that he or she wants to call. To use this service, the DHH person has to have a TTY, which he or she wil most likely have at home.
Option #2
How do you make a call to Target when you are not at home or if you don’t have a TTY?
While at Gallaudet, I saw a Sorenson VRS (video relay service) videophone booth where the DHH can make a call via a sign language interpreter. The user sits in the booth and sees a sign language interpreter on the screen. The deaf person signs to the interpreter on the screen. The interpreter calls the hearing person and interprets between the hearing person and hard-of-hearing user. People can also make sign language calls via the Sorenson ntouch app on smartphones and tablets. The calls are free and are paid for by the government.
Sorenson VRS Videophone booth
What’s missing?
- I only saw the booths indoors. It’s impractical to have these outside because the rain and snow will ruin the booth unless they are well enclosed and protected against water.
- They require a human interpreter. No privacy.
- People walking by can see what the person is signing.
If you have a delicate or private conversation, you have to be comfortable with the sign language interpreter knowing the contents of your conversation.
What’s next?
- In the upcoming Android Operating System, Android Q, Google will launch Live Caption, an app that will provide captions for any spoken audio content (videos, podcasts, radio programs, live chats, etc).
[embedyt] https://www.youtube.com/watch?v=YL-8Xfx6S5o[/embedyt]
Video about Live Caption
2. Google is also working on Live Relay, a program to translate text into speech for live phone calls. “Live Relay uses on-device speech recognition and text-to-speech conversion to allow the phone to listen and speak on the users’ behalf while they type.” This program is still in the research phase.
3. Article with resources for home additions for the deaf and hard-of-hearing.
4. Tips for driving with hearing loss
As I wrote before, I am not expert and this post is not meant to be an exhaustive list of all the resources available or of all of the communication difficulties that exist.
If you’d like to learn more about deaf culture, I recommend Train Go Sorry, written by a hearing woman who grew up in a boarding school for the deaf and hard of hearing in New York that was run by her father.
I look forward to learn more about DHH communication.
Do you use other technologies that help you communicate? Please add your comments below!
