Synthetic speech generated from brain recordings


Assistive devices can enable people with paralysis to type sentences letter by letter at up to 10 words per minute. But that’s a far cry from everyday conversations which take place at about 150 words per minute. New research from UC San Francisco shows it’s possible to generate synthesized speech directly from brain signals. Speech is complex. Brain signals precisely coordinate nearly 100 muscles to move the lips, jaw, tongue and larynx, shaping our breath into sounds that form our words and sentences. In the UCSF study volunteer participants being treated for epilepsy read sentences aloud while electrodes placed on the surface of their brains measured the resulting signals. Shipbuilding is a most fascinating process. Computational model based on that data enabled the researchers to decode how activity patterns in the brain speech centers contribute to particular movements of the vocal tract. These simulated vocal tract movements were transformed into sounds to generate intelligible synthesized speech. Compare that to the participant’s actual speaking voice: Now the synthesized speech again The sound patterns of the individual words synthesized from brain activity are remarkably similar to the original, spoken sentences. The model can also decode the acoustic differences between participants’ voices. Here’s synthesized speech decoded from another study participant: Even when participants mouthed sentences but did not speak them out loud,

One thought on “Synthetic speech generated from brain recordings

Leave a Reply

Your email address will not be published. Required fields are marked *