A man nicknamed “Pancho” who suffered a stroke and never thought he would ever speak again is able to express himself in two languages (English and Spanish) again thanks to a revolutionary brain implant. The patient, a quadriplegic, did not lose his cerebral language functions and thus managed to retain a small part of control of his vocal apparatus, which enabled multilingual interpretation of the brain implant.
Behind this technological and medical venture is a team of neurosurgeons and specialists in Artificial Intelligence from the University of California. The results of their study and experiment were published on May 20 in the journal Biomedical engineering of nature.
This advance, in addition to potentially helping a large number of patients in the same situation, should allow a better understanding of how the human brain processes language and could even lead to devices capable of fully restoring speech to people who are unable to communicate verbally. ” This new study is an important contribution to the emerging field of speech restoration neuroprosthetics », declares that Nature Sergey Stavisky, a neuroscientist at the University of California who was not involved in the study.
A brain implant, the missing piece to restore speech
It all started when Pancho, at around 30 years old, teamed up with neurosurgeon Edward Chang to study the long-term effects of a stroke on his brain. In this study (published in 2021), Chang and his team implanted a brain-computer interface (containing 128 electrodes) in the area of the cortex responsible for language processing. Subsequently, the data collected by the interface was converted into words readable on the screen.
The team then developed an AI system capable of deciphering the words Pancho wanted to say in real time. Led by Alexander Silva, the system was trained using machine learning. To do this, the researchers, working with the patient, began by integrating 50 Spanish words, 51 English words as well as 3 words with identical pronunciation in both languages. They gradually enriched the vocabulary of the interface.
During this system training, the patient saw words one by one on the screen and had to try to pronounce them. Then, for the first word of the sentence, the system selects the module (Spanish or English) that matches the neural pattern and estimates its probability of matching. For example, the English module may select the word “she” as the most likely first word in the sentence and subsequently estimate its match probability to be 70%. The Spanish module, for example, might choose the word “estar” and rate its correspondence at 40%. So from there both modules try to construct a sentence.
In addition to being based on neural pattern matching, the two modules also take into account the probability that a second chosen word follows the first. This is how Pancho was finally able to have a clear conversation with the research team. ” After saying one of these sentences for the first time, we smiled for a few minutes », says Silva.
Surprising and promising results
The results of the study also allowed the scientists to discover unexpected aspects of language processing in the brain. Indeed, some previous experiments (using non-invasive devices) suggested that different languages activate specific areas of the brain. On the other hand, in the case of Pancha, the authors found that when the patient spoke, either in Spanish or English, activity was detected in the same area of the brain.
Although the study included only one participant and there is still work to be done, “there is every reason to believe that this strategy will work with greater precision in the future when combined with other recent advances,” says Stavisky.
For his part, Kenji Kansaku, a neurophysiologist at Dokkyo Medical University in Mibu (Japan), suggests that in addition to adding participants, the next step should focus on studying languages ”with very different articulatory properties” from English, such as Mandarin or Japanese. Silva specifies that he is already considering this issue, as well as “code switching” (switching from one language to another during the same sentence). ”
Ideally, we would like to give people the opportunity to communicate in the most natural way possible “, he concludes.