AI Insight
FlatClip is a new approach for analyzing fMRI brain activity data that represents cortical activity as flattened surface maps and processes them using a pre-trained image recognition model (SigLIP2). The method achieves intermediate performance between coarse region-of-interest models and fine-grained voxel-level models across multiple neuroimaging benchmarks, while requiring less specialized architecture and training than voxel-based approaches. Spatial arrangement that preserves brain anatomy consistently improves performance compared to randomized layouts, suggesting that geometric organization of the cortex helps transfer learning from image models to brain data.
Why it matters
This work offers a computationally efficient middle-ground approach for fMRI analysis that could make brain imaging research more accessible by reducing the need for expensive, domain-specific model training. The finding that preserving anatomical spatial relationships improves performance has implications for how researchers should structure brain data inputs when applying machine learning models.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Recent fMRI foundation models differ substantially in the spatial scale at which they represent brain activity. ROI- and connectivity-based models are efficient but coarse, whereas voxel-level models preserve fine-grained spatial structure but require specialized 3D/4D architectures and costly fMRI-specific pretraining. We ask how effectively an image-pretrained encoder can reuse the spatial organization of cortical activity. Motivated by evidence that macroscale brain activity is strongly constrained by brain geometry, we introduce FlatClip, a frozen-encoder surface-level baseline that renders cortical activity as geometry-aware flatmap sequences and reuses a frozen SigLIP2 image encoder with only a lightweight downstream probe. Across resting-state benchmarks, FlatClip serves as a competitive middle-ground representation, outperforming ROI-level baselines on HCP and ADNI tasks while remaining weaker on PPMI and below the strongest voxel-level models overall. On visual-fMRI decoding, restricting the input to visual or NSD-provided task-active cortex improves performance, highlighting the value of task-relevant cortical coverage. Spatial perturbation controls reduce the predictive performance of flatmap features under both retrained and fixed readouts, and anatomy-linked arrangements consistently outperform vertex permutations across three colormaps. Together, these results position surface-level flatmap sequences as a practical middle-ground baseline between ROI and voxel models, and support the utility of anatomy-linked spatial organization for reusing image-pretrained features. Code is available at https://github.com/OneMore1/FlatClip.
Source: FlatClip: A Geometry-Aware Surface-Level Baseline for fMRI Representation Learning