Inverting Foundation Models of Brain Function with Simulation-Based Inference
cs.LG, cs.AI, stat.ML
Submitted: 2026-04-26
Updated: 2026-08-28
Comments: Accepted at the ICML 2026 Workshop on Structured Probabilistic Inference & Generative Modeling
Code: https://github.com/OHFVoice/piper1-gpl
License: http://creativecommons.org/licenses/by/4.0/
The gist: Foundation models of brain activity promise a new frontier for in silico neuroscience by emulating neural responses to complex stimuli across tasks and modalities.
Terminology
Abstract
Foundation models of brain activity promise a new frontier for in silico neuroscience by emulating neural responses to complex stimuli across tasks and modalities. A natural next step is to ask whether these models can also be used in reverse. Can we recover a stimulus or its properties from synthetic brain activity? We study this question in a proof-of-concept setting using TRIBEv2. We pair the brain emulator with large language models (LLMs) that generate news headlines from linguistic parameters such as valence, arousal, and dominance. We then use simulation-based inference to learn a probabilistic mapping from brain maps to latent stimulus parameters. Our results show that these parameters can be recovered from predicted brain maps, demonstrating that the emulator's synthetic neural encodings preserve information about the controlled stimulus dimensions. They also show that LLMs can serve as controllable stimulus generators for simulated experiments. Together, these findings provide a step toward decoding and inverse design with foundation brain models.
Sources
- Diffusion Models in Simulation-Based Inference: A Tutorial Review
- Simulation-Based Inference: A Practical Guide
- Meta-learning In-Context Enables Training-Free Cross Subject Brain Decoding
- BayesFlow 2: Multi-Backend Amortized Bayesian Inference in Python
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: Local-Consistency Optimisation for Bayesian Neural Networks
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks