How People with Disabilities Are Using AI to Improve Their Lives
Advances in artificial intelligence have spurred the development of smart devices to help people overcome physical and cognitive challenges. And, this may just be the beginning.
A young man operates an Xbox Adaptive Controller, which makes gaming accessible to people with a broad range of disabilities. Photo credit: Microsoft
Kaden Bowen is a smiley 12-year-old who loves music and riding in fast cars. He also can’t walk or talk, is legally blind, and can only use the pinky finger on his left hand.
Kaden has cerebral palsy, and his father, James Bowen, has been trying out new technology to see if he can create tools that let his son have more independence. One such experiment led James to set up an Amazon Echo Show, a smart assistant device that uses AI to power its voice-control capabilities. Kaden already has a speaking device that has buttons he can press with his working finger to verbalize preselected words or sentences that communicate with the Echo.
James set it up so his son could ask for things like “Echo, tell me a joke” or “Echo, tell me the news." He also programmed it so Kaden could call his cell phone, Kaden’s grandparents, and a few other people.
Kaden Bowen, 12, who has cerebral palsy and is non-verbal, uses a speaking device to communicate with Amazon Echo. His father, James, is "proud of Kaden’s initiative and grateful for the technology that made it possible." Photo credit: the Bowen family
But Kaden did much more than that. While James and his wife were out one evening, Kaden, who was home with a babysitter, took it upon himself to use his speaking device to activate the Echo, tell it to call his grandfather, and then asked to go out on a car ride—which his grandfather was happy to do. Sharing that story in a Facebook post, James said that he was proud of Kaden’s initiative and grateful for the technology that made it possible.
“It was the first time since he was a toddler playing with a rattler that he was able to interact with something all by himself,” James says. “This Echo device goes way beyond ordering groceries or looking up a recipe for us."
Related
Google says it built a ‘superhuman’ game-playing AI. Is it truly intelligent?
AI Technology is Disrupting the Traditional Classroom. Here’s a Progress Report.
Can We Build a Brain?
Necessity is the Mother of Invention
Technology has been opening doors for individuals with disabilities, from motorized scooters to hearing aids, for a long time. And in the coming years, AI will begin to supercharge those efforts with new abilities and expanded access. With one billion-plus people with disabilities around the world, there is plenty of work to be done—and a large market to tap into.
“They are our customers, our friends, everybody,” said Jenny Lay-Flurrie, the Chief Accessibility Officer at Microsoft, who is herself deaf.
Disabilities, whether related to vision, hearing, mental health, learning, cognition, or mobility, can be permanent, temporary, or even situational. Designing new products with different levels of abilities in mind—a concept called inclusive design—has gone a long way in ensuring that technology works for everyone.
Making design widely accessible also ends up being good for those without a disability. One of the first typewriters sprung from a creator’s desire for his blind friend—some say lover—to be able to write more legibly. Alexander Graham Bell’s mother was deaf and his invention of the telephone came out of his work with the deaf community.
More recently, audiobooks were created as a way for the blind to enjoy literature. Video captioning was invented to make content compliant with the Americans with Disabilities Act (ADA) but now is regularly relied on by anyone watching videos when they can’t listen to the audio.
Creating new tools could help integrate a segment of our population that has often been left out of routine daily life activities and job opportunities. The unemployment rate is twice as high for Americans with disabilities, according to the Bureau of Labor Statistics. Worldwide, only one in 10 people who could use assistive products have access to them.
Artificial intelligence hasn’t been around long enough to tackle every opportunity, but Microsoft wants to speed up innovation with a program called “AI for Accessibility” that was announced earlier this year with $25 million in funding. Its goals include developing more AI in the company and giving out grants to others who want to build tools for disabled communities. With AI, Lay-Flurrie says that technology that she thought was five to 10 years out is now more like one to three years away.
“It’s stupidly exciting for disability, honestly,” she says.
Microsoft's Seeing AI app helps the visually impaired by translating text into spoken words. Photo credit: Microsoft
Of course, companies get something out of this, too. Businesses trying to recruit from the limited AI talent pool can use social good projects to convince potential hires that their work will contribute in a meaningful way to the world. Building products that work for previously ignored disabled communities also expands their potential user base. And in a lot of cases, new features can be baked into products already used by non-disabled groups, making them better and more profitable.
It’s not just big companies innovating in this space. Over 100,000 deaf and hard of hearing individuals have used Ava, an app that allows them to take part in group conversations in either English or French (with more limited use for Spanish, Italian, German, and Russian). Everyone engaged in a conversation opens Ava on their phones, then speaks normally as the app listens in. Ava converts spoken words into text in nearly real time, rendering each speaker’s words into a different color for those needing to read along to follow the chat.
Voiceitt is an app for people with speech impediments, including both those who need it temporarily after strokes and brain injuries, and those with more long-term conditions like cerebral palsy, Parkinson’s, and Down’s syndrome. Using machine learning, Voiceitt picks up speakers’ unique speech patterns, recognizes any mispronunciations, and normalizes their speech before creating an output of audio or text.
Products that were designed for a general audience are also being used to increase accessibility. Smart assistants like Amazon’s Alexa and Apple’s Siri have become some of the biggest helpers for blind users, allowing them to get online more easily. Users are discovering new ways to open up voice-based assistants to the deaf, too. One project made an Echo able to understand and respond in sign language using a webcam.
“Accessibility is designed into everything today,” Lay-Flurrie says. “We need to make it part of the air that we breathe.”
Driving Innovation
Paulo Pinheiro, a 34-year-old robotics PhD from Brazil, was waiting for a flight at the airport when he saw a person his age in a wheelchair who didn’t seem like she could move very well.
“But she had a great smile,” he says.
That insight led Pinheiro to launch the startup HOOBOX Robotics. Its first product, called the Wheelie 7, lets wheelchair users initiate moves like going forward, turning, and stopping by making nine distinct facial expressions. While the computer vision behind the technology is cutting edge, the kit itself takes just seven minutes—hence, the name—to add to a wheelchair and is easy enough to install that even non-techie family members can do it. There are currently 55 people in the U.S. using the Wheelie 7—mostly quadriplegics, people with ALS, and senior citizens.
“Five years ago, this would have been impossible,” Pinheiro says.
While Pinheiro’s team is still testing the Wheelie 7, which retails for $3,600, he says they are already working on features that would allow users to use facial expression to activate and use smart assistants.
“Not only can they use it to get around, but they can also use it to connect with the world,” Pinheiro says.
The HOOBOX Wheelie uses machine learning to recognize facial expressions to command a user's wheelchair. Photo credit: hoobox2017, Flickr
An Inclusive Future
All types of AI need diverse data sets to prevent algorithms from learning bias or coming up with results that discriminate against certain groups. While this problem is usually invoked in the context of racial and gender discrimination, people with disabilities are also at risk, according to an American Association of People with Disabilities’ (AAPD) technology consultant, Henry Claypool.
“We need to have a more progressive agenda to make sure people with disabilities are represented in data,” says Claypool.
Including all types of disabilities will be tricky. Data profiles of people with disabilities are sometimes easy to spot, which makes privacy a concern—especially for conditions that have a high chance for stigmatization, like mental health issues. But gathering the necessary data is doable with the right efforts to anonymize any identifying information. Figuring out how to get this right is essential, Claypool adds: Not being reflected in the data from the start will jeopardize people with disabilities’ access to ubiquitous technologies that are becoming an essential part of the fabric of the modern world.
“It could have a huge equalizing effect,” Claypool says. But, he adds, “These data innovations won’t benefit everyone if we aren’t careful.”
James Bowen’s family has felt the effect of the lack of planning for accessibility. An update to the Echo now requires users to confirm that they want to make a call, which makes it harder for Kaden to use. Still, James sees new technology as a way to open up the world for his son and is already researching new tools to try out.
“It’s the future of what we are looking for and how we are trying to motivate him,” James says.
Jackie SnowPosts By This Contributor