Neuroscientists have created brain-reading software which can decode speech signals in real-time into written sentences, potentially transforming the lives of people who have lost the ability to speak.
Doctors at the University of California in San Francisco took on the challenge in the hope of creating a product that allows paralyzed people to communicate more fluidly than using existing devices that pick up eye movements and muscle twitches to control a virtual keyboard.
The work, funded by Facebook, was possible thanks to three epilepsy patients who were about to have neurosurgery for their condition.
Before their operations went ahead, all three had a small patch of tiny electrodes placed directly on the brain for at least a week to map the origins of their seizures.
The electrodes allowed researchers to record their brain activity, while they were asked a set of nine questions and asked to read a list of 24 potential responses.
Using this data, the team then created a computer model which was trained to match patterns of brain activity to the questions the patients heard and the answers they gave.
The software was able to identify from the brain speech signals what questions the patient had heard and the response they gave, with an accuracy of 76% and 61% respectively.
The model is currently limited in scope because it was trained on a small set of words, but researchers are hopeful they can increase its “flexibility” and “accuracy” in the future.