Aubrey Lee can tell a lot about someone by the look on their face. Pretending to understand her.
The wrinkles between his eyebrows were caught in purgatory concentration. A polite laugh that doesn’t match the funny nature of the punch line.
Apparently I had an obvious point:
Lee was born with a rare form of muscular dystrophy. She uses a wheelchair, has a speech impediment, and can’t laugh much.
Lee, 31, is a brand manager at Google and helped name Project Relate, an artificial intelligence-enabled app that makes it easier to have conversations like:
I may have mistaken “test” for “breast,” but Relate’s computerized voice translated the word “riddle” that I was curious about.
Project Relate is one of a suite of artificial intelligence tools known as AAC (Augmentative and Alternative Communication) among people with nonstandard voices.
Deep learning is at the core of these breakthroughs. In this case, an algorithm built on thousands of recordings of people with speech disorders is used, which is then trained based on the individual user’s speech patterns. This is an older line of AI that is different from the large language models that power ChatGPT. However, over the past few years, we have become very good at decoding non-standard speech.
“So what the machine is doing is becoming a professional listener,” said Jordan Green, a speech-language pathologist at Massachusetts General Hospital’s Health Professions Institute who works with Google. “And you can take a person who is so incomprehensible that you or I can only understand 10% of what he’s saying. But the machine says he can only understand 80% to 90% of what he’s saying. can learn to recognize with an accuracy of
Since its release in 2021, Project Relate’s beta app for Android smartphones has been downloaded by thousands of users. Many small businesses also offer similar products specifically for people with severe speech impairments caused by cerebral palsy or ALS (also known as Lou Gehrig’s disease).
“About a year ago, we were asking individuals to record 1,000 phrases to help our technology understand their unique way of speaking,” says co-founder of AAC AI startup Voiceitt. says Sara Smalley, author. “To date, we have been able to reduce the initial training time for most users to about 200 phrases in a very short period of time, with some users doing almost no training at all.”
The recent explosion of generative AI tools such as ChatGPT and Dall-E has also provided new avenues for people with severe mobility disabilities to express themselves.
Amy Thornburgh was diagnosed with ALS 25 years ago. Her voice is almost taken away and she can’t move her muscles under her head at all.
Over Zoom, she showed me a digital image that looked like an eerie oil painting. In front of her is a bright red poppy with a creepy eyeball in the middle of her pistil.
Thornburg created the images using the AI art tool Fotor, software that tracks eye movements to complete prompts. She calls this image “Always Watching.”
“When you go out in a wheelchair, you tend to attract a lot of attention,” Thornburg says.
Thornburgh prints some of these images on canvas and sells them on Etsy. She said AI is helping her scratch her artistic itch. Before her diagnosis, she was creating large product displays at malls in the Midwest.
Amy’s husband, Pat, said he is impressed by the advances in technology that are helping Amy communicate. However, he developed a highly efficient way to understand Amy without machines, even if she lost her voice permanently.
“When I do something out of sync or forget something, a certain gaze tells me, ‘Hey, what are you doing?'”
After 38 years of marriage, he said he doesn’t need AI to translate everything.
There’s a lot going on in the world. For everything, Marketplace is here for you.
You use the Marketplace to analyze world events and communicate how they affect you in a factual and approachable way. We rely on your financial support to continue making that possible.
Your donation today helps power the independent journalism you depend on. For as little as $5 a month, you can help sustain our marketplace. This allows us to continue reporting on the things that matter to you.