Unassigned · 2026-10-19

Is a Linear Probe Evidence of a Linear Representation?

Nadir Kutluozen, Christian Intern\`o, David Klindt

arXiv:2610.22655PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors argue that linear-probe success is too permissive evidence for “linear world models.” Asymptotically, rich feature maps—even random ones—can yield high forward linear decoding. They propose three asymmetric diagnostics (reverse predictivity, spectral concentration, and out-of-distribution generalization) and validate them in a fully observed simulation and on pretrained vision transformers using SVG-World and CelebA, finding probe success sits between random features and truly linear world models.

Why it matters

If a factor is linearly decodable, it does not necessarily mean the representation’s dominant geometry aligns with that factor; the orthogonal curved structure can matter for robustness, OOD generalization, and compositional transfer. The paper provides diagnostics that can rule out a linear world model when forward decoding alone is ambiguous.

Method

  • Formalize a “linear world model” (Def. 1) requiring variance-dominant linear alignment with whitened latents, with residual level σ².
  • Show that high forward linear decoding is asymptotically “free” from rich random features (Prop. 1), motivating three asymmetric tests: reverse predictivity, spectral concentration, and OOD generalization.
  • Validate diagnostics in a controlled simulation and then apply them to six pretrained vision transformers on SVG-World and to two models on CelebA.

Limitation

The authors state the diagnostics are necessary rather than sufficient: high scores do not establish a linear world model, and low scores only rule it out; they also note reverse-predictivity comparisons depend on the fully specified factor set and its encoding.

Abstract (from arXiv)

The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space, and comes with a standard methodology: train a linear probe, declare a factor represented if accuracy is high. We show this test is too permissive. Asymptotically, any sufficiently rich feature map, including a random one, admits a linear readout, so high probe accuracy is consistent with two very different geometries: a true linear world model, where the latent-aligned subspace dominates the representation, or a kernel machine, where the latents lie linearly inside a much larger curved manifold. This distinction matters because the curved structure orthogonal to the probe is relevant for out-of-distribution generalization, robustness, and compositional transfer. We propose three asymmetric diagnostics that distinguish the two cases: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known. Applied to six pretrained vision transformers on SVG-World, our deterministic generative benchmark with paired visual styles, and to two of them on CelebA, they show pretraining produces structure beyond a random kernel but well short of a linear world model. On dSprites, 3D Shapes and MPI3D, forward decodability and latent-aligned variance dissociate. Thus, forward decoding alone does not show that the probed factors align with the dominant geometry of the representation. In summary, our work cautions about the success of linear probes as opposed to truly linear and low dimensional representations.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-19 UTC

preprint

Is a Linear Probe Evidence of a Linear Representation?

Nadir Kutluozen, Christian Internò, David Klindt

Preprint posted: 2026-10-15 · Original record ↗

arxiv
2610.22655
Explicit source versions (1)
  • v1 · 2026-10-15
  1. Preprint available

    Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

The linear representation hypothesis holds that pretrained vision models encode generative factors as linearly decodable directions in feature space, and comes with a standard methodology: train a linear probe, declare a factor represented if accuracy is high. We show this test is too permissive. Asymptotically, any sufficiently rich feature map, including a random one, admits a linear readout, so high probe accuracy is consistent with two very different geometries: a true linear world model, where the latent-aligned subspace dominates the representation, or a kernel machine, where the latents lie linearly inside a much larger curved manifold. This distinction matters because the curved structure orthogonal to the probe is relevant for out-of-distribution generalization, robustness, and compositional transfer. We propose three asymmetric diagnostics that distinguish the two cases: reverse predictivity, spectral concentration, and out-of-distribution generalization. We validate them in a fully observed simulation where ground truth is known. Applied to six pretrained vision transformers on SVG-World, our deterministic generative benchmark with paired visual styles, and to two of them on CelebA, they show pretraining produces structure beyond a random kernel but well short of a linear world model. On dSprites, 3D Shapes and MPI3D, forward decodability and latent-aligned variance dissociate. Thus, forward decoding alone does not show that the probed factors align with the dominant geometry of the representation. In summary, our work cautions about the success of linear probes as opposed to truly linear and low dimensional representations.