Brain activity decoder can reveal stories in people's minds
Semantic decoder AI turns brain activity into coherent text.
vneo
Summary
Researchers at The University of Texas at Austin have developed an artificial intelligence system, a semantic decoder, that translates brain activity into text, potentially allowing speech-impaired individuals to communicate.
Introduction
The AI-driven semantic decoder represents a significant advancement in brain-computer interface technology. The system functions by training an AI model on a person's brain activity while they listen to stories, later translating this activity into text during live listening or imagination sessions.
Main Points
- The decoding system utilizes transformer models, similar to those powering OpenAI's ChatGPT and Google's Bard.
- The technology is noninvasive and doesn't limit subjects to specific words or phrases.
- Training involves listening to hours of podcasts while undergoing an fMRI scan.
- The decoder produces text that closely matches the intended meanings of the original words.
- The technology can only work with cooperative participants and fails with untrained individuals or if trained individuals resist.
- The technology also successfully described silent video content based on brain activity.
Conclusion
While the current model depends on fMRI scans and is therefore limited to laboratory use, the researchers envision that the work could translate to more portable brain-imaging systems, potentially revolutionizing communication for individuals with speech impairments.
Primary perspectives
For a noninvasive method, this is a real leap forward compared to what's been done before, which is typically single words or short sentences. We're getting the model to decode continuous language for extended periods of time with complicated ideas.
- Alex Huth, assistant professor of neuroscience and computer science at UT Austin
We take very seriously the concerns that it could be used for bad purposes and have worked to avoid that. We want to make sure people only use these types of technologies when they want to and that it helps them.
- Jerry Tang, a doctoral student in computer science at UT Austin.
Materials provided by: University of Texas at Austin
Original author: Marc Airhart
Note: text modified for style and brevity
Date: May 2023
