AI Gives Nonspeaking Opera Star His Voice

Jakob Jordan's voice has been trapped inside him for years, but now it's singing on an opera stage.

Jakob has autism and apraxia, a neurological condition that makes it difficult or impossible to speak his thoughts aloud. But he just became the star of "Sensorium Ex," a full-length opera that gave him something revolutionary: a voice that sounds like him and moves with his emotions.

"Hearing my voice produce my desired words feels so exhilarating," Jakob says. "It's like my voice was released from prison after a wrongful conviction."

The technology behind Jakob's performance came from NYU professor Luke DuBois, who built a custom AI speech synthesis device specifically for the opera. Unlike traditional text-to-speech tools that sound robotic and flat, this system captures Jakob's unique voice and gives him expressive control over tone, emotion, and musical timing.

Composer Paola Prestini created "Sensorium Ex" after working with poet Brenda Shaughnessy, whose son is nonverbal and nonambulatory. When traditional opera companies balked at the technical challenges, Prestini's company VisionIntoArt built the entire production from scratch.

"I wanted to make sure that the context for disability would thrive," Prestini explains. She treated the stage as a research and development lab, bringing together disability experts, technologists, and performers to create something that had never existed before.

The project required more than just building technology. The team had to reimagine what opera could be, creating frameworks that honored the performers' disabilities rather than working around them.

Why This Inspires

This opera flips the script on artificial intelligence at exactly the right moment. While headlines focus on AI replacing workers and threatening creativity, "Sensorium Ex" shows technology doing something better: giving voice to people who've been silenced.

Jakob's AI system doesn't automate him away. It amplifies who he already is, translating his intentions into sound with an authenticity that generic speech devices can't match.

The performance proves that accessibility technology works best when designed with disabled people, not just for them. By centering Jakob's needs and creative vision, the team built something that expanded what's possible in both opera and assistive technology.

"My mouth is constantly spouting off repetitive loops that I get stuck in," Jakob says. "It is not my intended speech, so it can be frustrating when I can't express my intentions."

Now those intentions are singing. And audiences are listening to a voice they've never heard before—one that was always there, just waiting for the right technology to set it free.