AI Insight
Researchers demonstrate that multimodal large language models (omni models) that process video, audio, and text develop brain-like representations, and that this relationship can be leveraged bidirectionally. They show that models with more input channels better predict brain activity across participants, and that organizing AI model components according to the seven functional networks of human cortex (Brain-MoE) improves performance across 15 model-benchmark combinations by an average of 6.42 percentage points. The study identifies specific sparse features responsible for brain-model correspondence and proposes that human brain architecture can serve as a design principle for improving AI systems.
Why it matters
This work suggests a new approach to AI development where neuroscience-inspired architectures could make models more efficient and capable. The bidirectional relationship between brain activity and AI representations could advance both artificial intelligence design and our understanding of human cognition.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Multimodal large language models predict brain activity, but brain alignment has been a measurement, not a design tool. We propose the Platonic brain bridge hypothesis: omni models, multimodal large language models that process video, audio and text jointly, converge on brain-like representations usable in both directions. From model to brain, brain-likeness of seven omni models is stable across participants, rises with every input channel in three bases, and our encoders lead the Algonauts 2025 out-of-distribution leaderboard. From brain to model, Brain-MoE fixes the expert partition of a frozen base to the seven networks of human cortex, trains experts on network-labelled Brain-AVQA questions, raises held-out accuracy in all 15 model-benchmark pairs by 6.42 percentage points on average and exceeds capacity-matched random experts in 14. Brain-Scope localizes the correspondence to sparse features whose removal weakens brain prediction. Human brain organization is therefore a usable architectural prior for multimodal large language models.