I Hear Dead People

At this years (by invitation only) Mars conference, Amazon revealed perhaps the creepiest tech I’ve ever heard of: an Alexa skill that can have it mimic other people’s voices.

On the surface, it doesn’t seem that bad right? Think about the possibilities of having your Alexa speak to you using your favorite superhero or celebrity’s voice.

But rather than going down this celebrity route, Amazon’s Alexa senior vice-president Rohit Prasad presented the idea alongside a clip of a child being read to by Alexa using the voice of their recently-deceased grandparent. According to Engadget, an Amazon spokesperson explained this clip could be recreated using “as little as one minute of audio” of the person Alexa is mimicking.

The scene was likely meant to inspire heartwarming feelings as you see how Amazon’s tech helps a child process their grief, but it could just as easily have been a scene from Black Mirror. In fact, the Black Mirror episode “Be Right Back” has a pretty much identical premise to this. Personally, I’d never want Alexa to sound like my departed relatives and when asked, my kids echo my sentiments exactly.

In his explanation of the Alexa tool, Rohit Prasad, senior vice president and head scientist for Amazon Alexa, said that while it wouldn’t eliminate the pain of loss “it can definitely make memories last” and could help ease any heartache.

This is a sentiment that I can understand; grief is a difficult emotion to process, especially at a young age like the child in the clip Amazon shared. But blurring the lines between life and death doesn’t seem like the healthiest way to deal with loss.

Thanks to The Verge for keying up this spot in the video: https://www.theverge.com/2022/6/23/23179748/amazon-alexa-feature-mimic-voice-dead-relative-ai

Based on Prasad’s comment it’s clear Amazon sees this as an evolution of reminiscing about loved ones using old photos and videos recorded before they passed away, but this isn’t the same. A photo or a video was taken with their consent and is of something the person actually did before they died; this AI-led feature isn’t playing a recording of a book reading, it’s using their voice to make a fabricated memory.

More than just digital-undeath

Outside of the potential nightmare-inducing uses above, this copycat feature could also pave the way for new scams. While many of us know to ignore and hang up on robotic voices on the other end of the line, we might be more inclined to listen to the pleas for money from our grandmother or grandson.

Coupled with sophisticated video deepfaking tools, we could eventually find it impossible to believe any video we watch – anything could have been constructed from realistic fake audio and visuals.

Amazon won’t be the only company developing AI voice tools like this. We suspect Apple, Google, and every other voice assistant maker is looking to make them sound more realistic and personal. But these sophisticated mimicking features are very much a pandora’s box, something to be opened with extreme care.

To that end, it does seem like Amazon is aware of the need to be cautious. It hasn’t outlined a timeline of when it will get this feature rolled out to existing Alexa devices, and it hasn’t even confirmed if it will even be publicly available.

But now that its audio immortality has been revealed to the world, it’s likely only a matter of time before Alexa gains the ability to speak using voices of those departed or even those still walking on earth.