Ava: Creating total accessibility for the deaf and hard-of-hearing

Pieter Doevendans of Ava talks about breaking down communication barriers between the deaf & hearing worlds with the power of mobile & speech technologies.
Ava: Creating total accessibility for the deaf and hard-of-hearing
S1 EP1Nov 2017

Ava: Creating total accessibility for the deaf and hard-of-hearing

Ava: Creating total accessibility for the deaf and hard-of-hearing

with Pieter Doevendans, Co-founder & COO

Ava is breaking down communication barriers between the deaf & hearing worlds with the power of mobile and speech technologies.

Based in San Francisco, Ava is on a mission to empower deaf and hard-of-hearing people with total accessibility 24/7.

Ava's mobile app transcribes audio conversations into text messages, making it easier for deaf and hard-of-hearing people to communicate, especially in group conversations.

In our first episode of HealthRedesigned, we speak to Ava’s Co-founder & COO, Pieter Doevendans.

The d/Deaf communities

Ava’s founder and CEO, Thibault Duchemin, comes from a family that is entirely Deaf. Even though Thibault can fully hear and is physically not deaf, he is classed as culturally Deaf because he grew up with sign language as his first language. People who are Deaf have a strong deaf identity and are often quite proud to be deaf.

A person who is deaf (lowercase ‘d’) instead, might have grown up in a hearing family, attended a hearing school and does not know how to sign. In other words, this person identifies more with the hearing world.

A deaf person’s association with either community can vary as they grow up and as circumstances around them change.

It’s often assumed that anyone who is deaf knows sign language, but in reality, only two to three percent of people who are deaf understand how to sign. Hearing loss can also range across many levels including deafness from birth to senior people who progressively lose their hearing as they get older.

This whole adventure has taught us a lot. We started only really understanding the capital D community where Thibault grew up in.

Bridging communication gaps

People with hearing loss often find it incredibly challenging to communicate, especially when there are more than two people involved in a conversation.

While interpreters, captioners, and various technological solutions like hearing aids and cochlear implants already exist on the market, these often go a little way in helping people with hearing loss thrive in group conversations.

Imagine being at a family dinner or a business meeting and not being able to follow the conversation of the multiple people in the room. It’s frustrating and isolating. This is the biggest issue Ava seeks to improve for its users.

Even though captioning has been around for years, Ava is bringing forth a user experience that’s truly innovative to make communication more accessible. Ava is also focused on understanding the complexities of a multi-person conversation and presenting it in a way that makes it easy for users to understand via text.

Innovating and iterating

The real-time component of Ava is a feature that makes it unique as it helps users to better understand and engage in physical conversations. Ava can be used to transcribe a conversation in the same physical space that involves users who are deaf.

Through user research, Ava learnt that a deaf or hard-of-hearing person's number one challenge was to understand rather than being understood. That’s when the focus shifted from building an app that conveys what a deaf or hard-of-hearing person is saying to one that transcribes conversations from all speakers in real-time.

One of the early obstacles of building Ava was having multiple microphones pick up the same person’s voice at a single time and repeatedly display it as text on the interface. This is an issue that the team is fine-tuning today.

Ava also continues to improve the digital onboarding experience, both for users with hearing loss and fully-hearing users. The goal is to create a seamless and lightweight experience that doesn’t discourage anyone from jumping on the app to converse, even if it's just for a quick conversation.

I invite you, you join, and then we can start talking to each other. It’s almost like a phone call but in a physical space together.

The user testing phase

Before any development began, Ava interviewed over 300 people to find out how they could build a great user experience and learn about features that users actually wanted. Ava’s positive mission and story helped them find the testing user base they needed for critical feedback and they found that participants would continuously refer them to more people to test with.

One of the biggest insights they received was for a a user to be able to follow along a conversation without any delays, something that they've managed to incorporate within the app.

They also learnt that deaf and hard-of-hearing people like to communicate in different ways, for example, some prefer to type out their speech instead of vocalising it. So, they designed the app to display text as it’s being typed out while merging it with speech from other users in the conversation. To give users more options, Ava is currently working on a text-to-speech feature that voices out typed-text.

Impacting lives

Ava has already had an impact on a number of lives. One user uses Ava to have conversations with clients—something that she wasn’t able to do before. Another used it to have a dialogue with nurses and doctors when an interpreter wasn’t present. These spontaneous and day-to-day interactions are exactly what Ava is trying to help facilitate.

Ava is just scratching the surface when it comes to its features. A recent release of the app includes the text-to-speech feature and allows users to scan a QR Code to join conversations. A web application is also currently under development.

To support Ava, Pieter asks us to keep accessibility in mind as we go about our daily lives. You can become a part of the Ava Force to learn how your community and country can be more accessible.

Connect with Ava on:

Hosted by
Laïla von Alvensleben

Laïla was a designer at Hanno from 2015-2018, helping us implement ways to work more effectively as a distributed team.