A New Strategy for Artificial Intelligence: Training Foundation Models Directly on Human Brain Data
q-bio.NC, cs.AI, cs.LG
Submitted: 2026-01-17
Updated: 2026-09-06
Comments: 36 pages, 5 figures. v2: added algorithmic formulations for RLHB and CoTHB; added sections on temporal scale synchronization and scalability
License: http://creativecommons.org/licenses/by/4.0/
The gist: While foundation models have achieved remarkable results across a diversity of domains, they still rely on human-generated data, such as text, as a fundamental source of knowledge.
Terminology
Abstract
While foundation models have achieved remarkable results across a diversity of domains, they still rely on human-generated data, such as text, as a fundamental source of knowledge. However, this data is ultimately the product of human brains, the filtered projection of a deeper neural complexity. In this paper, we explore a new strategy for artificial intelligence: moving beyond surface-level statistical regularities by training foundation models directly on human brain data. We hypothesize that neuroimaging data could open a window into elements of human cognition that are not accessible through observable actions, and argue that this additional knowledge could be used, alongside classical training data, to overcome some of the current limitations of foundation models. While previous research has demonstrated the possibility to train classical machine learning, deep learning, or reinforcement learning models on neural patterns, this path remains largely unexplored for high-level cognitive functions. Here, we classify the current limitations of foundation models, as well as the promising brain regions and cognitive processes that could be leveraged to address them, along four levels: perception, valuation, execution, and integration. Then, we propose two general methods that could be implemented to prioritize the use of limited neuroimaging data for strategically chosen, high-value steps in foundation model training: reinforcement learning from human brain (RLHB) and chain of thought from human brain (CoTHB). We also discuss the potential implications for agents, artificial general intelligence, and artificial superintelligence, as well as the ethical, social, and technical challenges and opportunities.
Sources
- Do As I Can, Not As I Say: Grounding Language in Robotic Affordances
- Constitutional AI: Harmlessness from AI Feedback
- On the Opportunities and Risks of Foundation Models
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
- Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models
- PaLM-E: An Embodied Multimodal Language Model
- Towards Physics-Guided Foundation Models
- The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization
- AugMix: A Simple Data Processing Method to Improve Robustness and Uncertainty
- Neural Brain: A Neuroscience-inspired Framework for Embodied Agents
- Towards Deep Learning Models Resistant to Adversarial Attacks
- NeuroAI for AI Safety
- Improving Semantic Understanding in Speech Language Models via Brain-tuning
- GPT-4 Technical Report
- Generative Agents: Interactive Simulacra of Human Behavior
- Language Models as Knowledge Bases?
- Hierarchical Text-Conditional Image Generation with CLIP Latents
- SocialIQA: Commonsense Reasoning about Social Interactions
- Make-A-Video: Text-to-Video Generation without Text-Video Data
- Chain of Thoughtlessness? An Analysis of CoT in Planning
Related papers
- BrainWave: A Brain Signal Foundation Model for Clinical Applications
- Toward Robust, Reproducible, and Widely Accessible Intracranial Speech Brain-Computer Interfaces: A Comprehensive Narrative Review of Neural Mechanisms, Hardware, Algorithms, Evaluation, Clinical Pathways and Future Directions
- CytoNet: A Foundation Model for the Human Cerebral Cortex at Cellular Resolution
- Emergence of psychopathological computations in large language models
- NeuroAI and Beyond: Bridging Between Advances in Neuroscience and Artificial Intelligence
- Attraction to hierarchical feature memory explains orientation bias