AI Insight
ScaleBrain is a new framework for pre-training foundation models on EEG data that explicitly represents brain signals at multiple temporal scales simultaneously. The framework introduces multi-scale tokenization using different analysis windows with shared time anchors, paired with scale-specific reconstruction objectives and an axial spatio-temporal attention mechanism. Evaluated on six public Brain-Computer Interface benchmarks against 12 baselines, ScaleBrain achieved the highest accuracy on five datasets while using only 5.0 million parameters, demonstrating superior performance with efficient parameter usage.
Why it matters
This work addresses a key limitation in current EEG foundation models by ensuring that multi-scale temporal information receives adequate supervision during training. The improved performance and parameter efficiency could accelerate the development of more accessible and deployable Brain-Computer Interface applications for clinical diagnostics, assistive technologies, and neuroscience research.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Foundation models pre-trained on large unlabeled electroencephalography (EEG) corpora have emerged as a transformative paradigm for Brain-Computer Interface (BCI) applications. Recent EEG foundation models have shown that multi-scale representations derived from fixed-length patches can improve EEG representation learning. Existing frameworks commonly construct temporal scales through feature-level operations. However, these operations leave the correspondence between learned features and scale-specific characteristics of the original EEG signals implicit. Moreover, when the self-supervised objective is not aligned with the multi-scale input representations, information at some temporal scales may receive insufficient supervision during pre-training. In light of this, we propose ScaleBrain, a scale-consistent pre-training framework that explicitly coordinates multi-scale representations of EEG from both the model input and optimization objective sides. Specifically, ScaleBrain introduces multi-scale EEG tokenization to represent the same EEG segment through analysis windows with different temporal spans and shared temporal anchors, and pairs these representations with scale-specific reconstruction to align with the multi-scale input structure and provide supervision across temporal resolutions during self-supervised learning. We further develop an axial spatio-temporal attention mechanism to contextualize the fused multi-scale representation along spatial and temporal dimensions. Extensive evaluations on six public BCI benchmarks against 12 representative baselines show that ScaleBrain achieves the highest mean Balanced Accuracy on five datasets and ranks third on the remaining one, with only 5.0M parameters, demonstrating strong performance and favorable parameter efficiency.
Source: ScaleBrain: A Scale-Consistent Framework for EEG Foundation Models