Bradley University graduate and social justice advocate Pierre Paul is leading a team working on a program to help hearing impaired individuals.
Paul is founder and CEO of “We Hear You,” a pilot program developing an app that will translate American Sign Language into spoken English. Made possible with software developed by Bradley’s Computer Science Capstone, the app is on track for a test run later this month at The Spot Coffee in Peoria.
In a conversation with Joe Deacon, Paul discusses the concept and development of the “We Hear You” app, plans to make it available and expanding the program, and how accessibility, diversity and inclusion served as inspiration for the project.
Joe Deacon: What can you tell us about “We Hear You” – what is it, and what will it do?
Pierre Paul: Absolutely. So, “We Hear You” is an idea that came to me about two summers ago. I was just kind of sitting in my room and I couldn’t fall asleep, and I would find myself having deeper thoughts at night trying to conceptualize new ways to help the people around me.
Just to kind of give a little bit of back story, when I first came to the United States, I was bullied and just disrespected a lot for things out of my control: I had an accent, I was new to the school, I was a foreigner. And I just remember as a kid thinking that I didn’t want people to feel like an outsider inside of their own skin for things outside their control.
And the same kind of goes with individuals who utilize American Sign Language, or individuals who are deaf or hard of hearing. They’re speaking in their own language, and our society doesn’t accommodate that. So with that in mind, I had the idea for this application of “We Hear You,” a device that turns American Sign Language into audible speech and then is backward compatible to turn speech and text into American Sign Language.
With that, I came to Bradley’s campus and talked to a lot of professors. A lot of them liked the idea, but didn’t think it was possible. I found some individuals who were able to make it possible, and then we’ve kind of just been progressing ever since. So that’s kind of the basics of the application, and then we’ll be releasing it this December in The Spot Coffee shop to test out, and kind of get some trial runs and see how it works in a live setting.
What else can you tell us about the “We Hear You” app itself? Can you describe a little bit about how it works?
Paul: Yes or no. So we’re still getting some provisional patents on it, so I’m not going to be too detailed. But essentially, we have trained the system on a neural network and we’ve also utilized some open source data that Google has made available. So our system knows certain things we’ve taught it.
So for spacebar, we’ve made that open palm; erase the screen, we’ve made that a thumbs up. And then after a certain amount of time elapses that the words are on the screen, it knows to verbalize that. Then the backwards compatibility is just a simple speech-to-text system, so if you speak into Google or use one of your more prominent, one of your more advanced technologies, you can speak into it and it will type it out as well.
So we just taught our system with training to recognize ASL letters and then turn those into words, and then string them together so they make words. Then in the future, we’re going to be training on Word so full sign language with both hands can be recognized, and then full sign language with body language and facial (expression) will also be recognized.
Who are some of the other people on your team that are working to make this possible?
Paul: So when I first got here, I reached out to Adam Byerly and he was extremely busy. He loved the idea, so he passed me on to one of his colleagues, David Brennan, and David’s kind of been following us a long way through this process. When I contacted David, he had some students – some of them have graduated by now – but they kind of helped give us the base of what we were working on, even though we steered away from what they had developed. It was all of these amazing guys who just were wide-eyed and super excited and they helped give us a lot of the basis of what we were doing.
Then from there, this year I got a notice of a Capstone project, and I have some amazing Capstone students who are working on it and just kind of helping us overall. So many amazing guys; there’s so many names of people who have just been so monumental, and I do give a lot of credit to Bradley and the way that Bradley does facilitate academics here, finding students who are willing to go outside of themselves is to go outside of what you’re already doing with their workload and put time and effort into a project that somebody else sees value in, even if it doesn’t impact them. It’s beautiful.
You mentioned you will be doing some trial runs soon at The Spot Coffee. Beyond that, what are plans for launching the app and making it more available to the broader community?
Paul: Yeah, so we’ve had a really strong and good connections with Target, so the goal is definitely to have Target kind of guide us, seeing our application and helping develop our application. Then we want to test it inside of Target as well, once it’s a little bit more finalized. Then from there, we hope to expand “We Hear You” so the system is all over the place.
And I’ve got a little insider secret for you. So, I decided that I didn’t want “We Hear You” to just be kind of like a one-trick pony or like a one-hit wonder. So there are a couple other inventions – I’m actually working on two other ones right now specifically – that would be through the “We Hear You” brand, but are in a different vein for accessibility. And the goal with all of these is not to charge the people who need to utilize the product, because our society should already be accommodating them.
So there are a lot of big plans for me and “We Hear You” in the future. It’s just executing them and keeping my head up.
On the “We Hear Your” website, you have a motivational quote posted rather prominently. Can you tell us what that quote is and how it serves as your inspiration?
Paul: Absolutely. So: “Diversity and inclusion should never be the afterthought of innovation, it should always be the foundation.” I love that quotation; I actually started up when I was writing a speech for the social impact competition last year.
The reason that quotation came to my mind is because when I was reading about other businesses, and looking at business models and seeing what was happening in the news and seeing what was happening around the world, I realized that everybody was doing a mad scramble to now have diversity and inclusion inside of their institutions, because they realized they didn’t start with that.
But when you start with diversity and inclusion as the base of your institution, you’re ensuring that all of the ideas are culturally rich, all of the ideas are charged with the energy of diverse populations and diverse ideas and being inclusive of people from different backgrounds, regardless of race, sex, etc., etc. So, that quotation definitely hits near and dear to my heart, because I really do believe it. It’s just, once you start with diversity and inclusion at the heart of everything that you do, everything becomes so much easier.
This has been quite a remarkable and, in many ways, tumultuous year, with heightened social awareness coming amid a global pandemic. How do you work through all of that to make sure the focus remains on accessibility and diversity and inclusion?
Paul: I guess all of the strange things happening have almost fueled in my mindset, because I’ve been fortunate enough, as a hall director, and having traveled to Minnesota and helping the cleanup, and protesting etc, etc., I’ve met so many individuals who are all struggling with one common thing, which seems to be like the COVID guidelines about what’s happening with COVID, and the brutalization of certain individuals in America.
They were all from diverse backgrounds and they were all able to converge on the common ground, and we were able to get along and work together and move forward, even if we didn’t fully agree. And it was that inclusivity.
And I think the same thing is happening right now, as kind of some of the unrest around the nation has died down ever so slightly, but COVID is still like running rampant and still ruining certain things with people’s lives. People are still able to come together and care about one another, no matter their diverse backgrounds, attempting to be inclusive.
So it all just kind of ties together for me, but there are a lot of distractions. COVID definitely isn’t making it easier for the development side of “We Hear You” or the implementation side of “We Hear You.” But we’re just working through it, knowing that what we’re trying to do is bigger than us.
You can also listen to the full interview:
There's no subscription fee to listen or read our stories. Everyone can access this essential public service thanks to community support. Donate now, and help fund your public media.
