NEJM: The first "neural prosthesis" capable of "speaking" fluently

This result was developed in cooperation with the first participant in a clinical research trial.

"As far as we know, this is the first successful demonstration of a method of directly decoding complete words from the brain activity of a paralyzed and unable to speak," said Jeanne Robertson, director of neurosurgery at the University of California, San Francisco

Every year, thousands of people lose the ability to speak due to stroke, accident or illness

Translate brain signals into language

Previously, work in the field of neuroprostheses focused on restoring communication by typing letters one by one in the text through spelling-based methods

"For voice, we usually exchange information at a very high speed, up to 150 or 200 words per minute

In the past ten years, patients at the University of California, San Francisco Epilepsy Center are undergoing neurosurgery to determine the origin of their seizures through an array of electrodes placed on the surface of their brains, which has promoted Zhang's progress towards this goal

Previously, Chang and colleagues at the Weir Institute of Neuroscience at the University of California, San Francisco, mapped the cortical activity patterns associated with the vocal tract movement that produces each consonant and vowel

But their success in decoding speech on participants who can speak does not guarantee that the technology will work on a person with paralyzed vocal tract

In addition, the research team does not know whether the brain signals that control the vocal tract remain intact for those who have been unable to move the vocal cord muscles for many years

The participant, code-named BRAVO1, worked with researchers to create a 50-word vocabulary that Chang's team could use advanced computer algorithms to recognize from brain activity

In this study, Chang surgically implanted a high-density electrode array into the speech motor cortex of BRAVO1

Translate speech into text

In order to convert the recorded neural activity patterns into specific expected vocabulary, the other two lead authors of the study, Sean Metzger and Jessie Liu, PhD students in bioengineering in the Chang laboratory, used a customized neural network model, which is a part of artificial intelligence Kind of form

To test their method, the research team first showed BRAVO1 a short sentence consisting of 50 words and asked him to try to say it several times

Then, the research team turned to "How are you today?" and "Do you want to drink some water?" and other questions to remind him, just like before, the BRAVO1 speech appeared on the screen, "I am fine" and "No.

The research team found that the system can decode words in brain activity at a rate of 18 words per minute, with an accuracy rate of 93% (median 75%)

Moses characterized the early trial results as a "proof of principle.
" He said: "We are very excited to see the accurate decoding of various meaningful sentences
.
We have proven that it is actually possible to promote communication in this way, and it is There are also potential uses in conversational environments
.
"

Looking to the future, Chang and Moses said they will expand the scope of the trial to include more participants affected by severe paralysis and communication barriers
.
The team is currently working to increase the number of words in the existing vocabulary and increase speaking speed
.

Both said that although the study only focused on one participant and a limited vocabulary, these limitations did not reduce the sense of accomplishment
.
Moses said: "This is an important technological milestone for a person who cannot communicate naturally, and it demonstrates the potential of this method to give voice to severely paralyzed and aphasia
.
"

David A.
Moses, Sean L.
Metzger, Jessie R.
Liu, Gopala K.
Anumanchipalli, Joseph G.
Makin, Pengfei F.
Sun, Josh Chartier, Maximilian E.
Dougherty, Patricia M.
Liu, Gary M.
Abrams, Adelyn Tu -Chan, Karunesh Ganguly, Edward F.
Chang.
Neuroprosthesis for Decoding Speech in a Paralyzed Person with Anarthria.
New England Journal of Medicine, 2021; 385 (3): 217 DOI: 10.
1056/NEJMoa2027540