Unassigned · 2026-10-19
Is a Linear Probe Evidence of a Linear Representation?
Nadir Kutluozen, Christian Intern\`o, David Klindt
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors argue that linear-probe success is too permissive evidence for “linear world models.” Asymptotically, rich feature maps—even random ones—can yield high forward linear decoding. They propose three asymmetric diagnostics (reverse predictivity, spectral concentration, and out-of-distribution generalization) and validate them in a fully observed simulation and on pretrained vision transformers using SVG-World and CelebA, finding probe success sits between random features and truly linear world models.
Why it matters
If a factor is linearly decodable, it does not necessarily mean the representation’s dominant geometry aligns with that factor; the orthogonal curved structure can matter for robustness, OOD generalization, and compositional transfer. The paper provides diagnostics that can rule out a linear world model when forward decoding alone is ambiguous.
Method
- Formalize a “linear world model” (Def. 1) requiring variance-dominant linear alignment with whitened latents, with residual level σ².
- Show that high forward linear decoding is asymptotically “free” from rich random features (Prop. 1), motivating three asymmetric tests: reverse predictivity, spectral concentration, and OOD generalization.
- Validate diagnostics in a controlled simulation and then apply them to six pretrained vision transformers on SVG-World and to two models on CelebA.
Limitation
The authors state the diagnostics are necessary rather than sufficient: high scores do not establish a linear world model, and low scores only rule it out; they also note reverse-predictivity comparisons depend on the fully specified factor set and its encoding.
Abstract (from arXiv)
The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space, and comes with a standard methodology: train a linear probe, declare a factor represented if accuracy is high. We show this test is too permissive. Asymptotically, any sufficiently rich feature map, including a random one, admits a linear readout, so high probe accuracy is consistent with two very different geometries: a true linear world model, where the latent-aligned subspace dominates the representation, or a kernel machine, where the latents lie linearly inside a much larger curved manifold. This distinction matters because the curved structure orthogonal to the probe is relevant for out-of-distribution generalization, robustness, and compositional transfer. We propose three asymmetric diagnostics that distinguish the two cases: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known. Applied to six pretrained vision transformers on SVG-World, our deterministic generative benchmark with paired visual styles, and to two of them on CelebA, they show pretraining produces structure beyond a random kernel but well short of a linear world model. On dSprites, 3D Shapes and MPI3D, forward decodability and latent-aligned variance dissociate. Thus, forward decoding alone does not show that the probed factors align with the dominant geometry of the representation. In summary, our work cautions about the success of linear probes as opposed to truly linear and low dimensional representations.
Related papers
- The (Butter)fly Dream: Linear Separability of World-Model Rollouts in a Connectome-Constrained Visual Model
- How Representational Priors Shape Generalization in World Action Models
- Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models
- What Visual Representations Do World Action Models Need?
- DSReg: Provably Recovering Individual World Latents without Reconstruction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-19 UTC
Is a Linear Probe Evidence of a Linear Representation?
Preprint posted: 2026-10-15 · Original record ↗
- arxiv
- 2610.22655
Explicit source versions (1)
- v1 · 2026-10-15
- Preprint available
Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space, and comes with a standard methodology: train a linear probe, declare a factor represented if accuracy is high. We show this test is too permissive. Asymptotically, any sufficiently rich feature map, including a random one, admits a linear readout, so high probe accuracy is consistent with two very different geometries: a true linear world model, where the latent-aligned subspace dominates the representation, or a kernel machine, where the latents lie linearly inside a much larger curved manifold. This distinction matters because the curved structure orthogonal to the probe is relevant for out-of-distribution generalization, robustness, and compositional transfer. We propose three asymmetric diagnostics that distinguish the two cases: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known. Applied to six pretrained vision transformers on SVG-World, our deterministic generative benchmark with paired visual styles, and to two of them on CelebA, they show pretraining produces structure beyond a random kernel but well short of a linear world model. On dSprites, 3D Shapes and MPI3D, forward decodability and latent-aligned variance dissociate. Thus, forward decoding alone does not show that the probed factors align with the dominant geometry of the representation. In summary, our work cautions about the success of linear probes as opposed to truly linear and low dimensional representations.
