Latent Dynamics · 2026-10-26
Controlled Evaluation of EEG Foundation Models: Random Initialisation, Band-Power and Validation-Selected Probes
Mingyu Jeon, Suwan Cho, Jae Young Suh
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors evaluate six EEG foundation models (five public + one in-house JEPA-style) under a single controlled protocol with three controls: random-initialized same encoder, a five-band log-power baseline, and validation-selected probe layer (not test-selected). They report that these controls shrink or remove many downstream gains, with layer selection on test inflating peaks and fine-tuning gains mostly within seed noise. They release per-cell results for re-analysis.
Why it matters
The paper argues that common evaluation protocols for EEG foundation models miss key controls, making published numbers non-comparable. By adding matched random baselines, a band-power anchor, and validation-based layer selection, the authors show how much reported improvements can depend on evaluation choices (e.g., test-argmax layer leakage) and training variability. Their released per-cell audit is intended to enable re-analysis of each claim against the authors’ specific numbers.
Method
- Evaluate six EEG foundation models (LaBraM, CBraMod, EEGPT, LUNA, LuMamba, plus in-house JEPA-style) under one protocol across 5 original benchmarks plus 4 added task families, using several seeds.
- Apply three controls: a random-initialized twin of the same architecture, a five-band log-power baseline computed on the same split, and a probed layer chosen by validation (argmax over stages) rather than test.
- Report results for frozen probing and full fine-tuning, defining fine-tuning gain as pretrained minus random-init test metric at the best-validation epoch, and a “clear” gain heuristic (beyond twice seed spread).
Limitation
The authors note the study is exploratory rather than pre-registered because parts of the protocol (validation rule and added tasks) came after early results were seen, and they scope conclusions to the tested panel and protocols.
Abstract (from arXiv)
EEG foundation models report downstream gains, yet their protocols seldom include three controls. These are the same encoder untrained, spectral features fitted on the same data, and a layer picked without looking at test scores. We therefore evaluate five publicly released EEG foundation models and an in-house JEPA-style model under one protocol with all three controls and several seeds, on five benchmarks and four added task families. The controls change the reading in three ways. First, picking the probed layer by its test score raises the layer-wise peak by an average of +0.034 and reverses the peak-versus-output gap in 9 of 36 cells. Second, with the layer picked on validation, random initialisation matches or beats pretraining for half of the models on sleep and emotion, and no model beats a five-band log-power baseline on emotion or motor imagery. Third, after full fine-tuning only 5 of 30 pairs gain beyond twice their seed spread, and the one clear loss of -0.290 AUROC persists under matched learning rates. From these results we draw four reporting practices, and we release every cell for re-analysis.
Related papers
- What masking geometry works best for EEG foundation models?
- SPERA: Spherical Prior EEG Foundation Model with Geometry- and Frequency-Aware Latent Prediction
- Similar Predictive Fit but Different Latent Dynamics: Characterizing Learned Dynamical Structure in Personalized Models of Brain Disorders
- Latent Prediction Yields Cross-Subject Alignable Representations in Sensory-Evoked fMRI
- NeuroLens: Learning Latent Embeddings of Neural Semantics from Chronic Recordings
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-23 · Indexed by Pub-AI: 2026-10-26 UTC
Controlled Evaluation of EEG Foundation Models: Random Initialisation, Band-Power and Validation-Selected Probes
Preprint posted: 2026-10-23 · Original record ↗
- arxiv
- 2610.32067
Explicit source versions (1)
- v1 · 2026-10-23
- Preprint available
Source date: 2026-10-23 · First detected: 2026-11-02 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
EEG foundation models report downstream gains, yet their protocols seldom include three controls. These are the same encoder untrained, spectral features fitted on the same data, and a layer picked without looking at test scores. We therefore evaluate five publicly released EEG foundation models and an in-house JEPA-style model under one protocol with all three controls and several seeds, on five benchmarks and four added task families. The controls change the reading in three ways. First, picking the probed layer by its test score raises the layer-wise peak by an average of +0.034 and reverses the peak-versus-output gap in 9 of 36 cells. Second, with the layer picked on validation, random initialisation matches or beats pretraining for half of the models on sleep and emotion, and no model beats a five-band log-power baseline on emotion or motor imagery. Third, after full fine-tuning only 5 of 30 pairs gain beyond twice their seed spread, and the one clear loss of -0.290 AUROC persists under matched learning rates. From these results we draw four reporting practices, and we release every cell for re-analysis.