JEPA · 2026-11-10

The Delta Gap: Cellular World Models for Perturbation Response Prediction Are Far Harder Than Global Correlation Implies

Junhui Gao

arXiv:2611.10658PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors show that cellular perturbation world models are often evaluated with global Pearson correlation, which is dominated by non-responsive genes and inflates apparent performance. On Replogle K562, GEARS gets global Pearson 0.977 but delta Pearson only 0.076; a trivial mean-training-delta baseline gets 0.267. They propose delta-space evaluation and show delta-aligned training plus post-hoc corrections improve delta Pearson and direction/gene recovery, with zero-shot transfer across cell lines.

Why it matters

This paper identifies a systematic evaluation artifact (“delta gap”) in perturbation response prediction: global correlation can look excellent even when models fail to predict perturbation-induced changes. By introducing delta-space evaluation as the primary benchmark and quantifying a gap between predicted and observed perturbation deltas, the authors provide a framework to diagnose whether models capture action-induced dynamics (what a world model should learn) rather than reconstructing largely unchanged baseline expression.

Method

  • Defines the delta as predicted-minus-control vs observed-minus-control transcriptome transition and evaluates Pearson correlation on this perturbation-induced delta (delta-space evaluation).
  • Introduces two post-hoc corrections without retraining: RWR propagation on a control-derived co-expression graph and kNN matching that retrieves true training deltas using cosine similarity on predicted deltas.
  • Proposes delta-aligned training that augments the loss with a delta Pearson alignment term (with lambda=3.0) and an amplitude-calibration variant (eta=0.3).

Limitation

The paper reports delta-space scores and related improvements for GEARS on the stated Replogle K562 and Norman datasets, but does not provide any additional stated limitations beyond the presented results.

Abstract (from arXiv)

Cellular world models aim to predict how cells respond to genetic perturbations, analogous to world models in reinforcement learning. Perturbation response models are evaluated almost exclusively by global Pearson correlation between predicted and observed transcriptomes. We show this metric is dominated by thousands of non-responsive genes and severely inflates apparent performance. On the Replogle K562 screen (509 unseen perturbations, 5000 genes), GEARS achieves global Pearson 0.977, but its delta Pearson is only 0.076. A trivial baseline predicting mean training delta achieves 0.267, which is 3.5x better than GEARS. Our primary contribution is methodological: we identify and quantify this delta gap as a systematic evaluation artifact, and propose delta-space evaluation as the primary benchmark. We provide two post-hoc corrections (RWR on co-expression graphs and kNN matching) and show that 2.8x of the remaining gap lies in perturbation representation geometry, not decoder capacity. We further show the gap is partially addressable end-to-end: delta-aligned training (lambda = 3.0) raises delta Pearson to 0.179 on Replogle (2.35x over GEARS) and surpasses the trivial baseline on direction accuracy (0.680 vs 0.567) and gene recovery (1.52 vs 1.19); combined with kNN it reaches 0.217. An amplitude-calibration variant (eta = 0.3) further raises gene recovery to 2.16 while reaching 0.218. Critically, these representations transfer zero-shot across cell lines: from Replogle K562 to unseen Norman A549, the delta-aligned model reaches 0.328 on 741 shared genes, 4.7x a trivial baseline and 2.1x standard GEARS. This is the first systematic quantification of the delta gap, and the first diagnostic framework localizing the residual gap to perturbation representation geometry via an oracle upper bound.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

The Delta Gap: Cellular World Models for Perturbation Response Prediction Are Far Harder Than Global Correlation Implies

Junhui Gao

First announced by arXiv: 2026-11-10 · Original record ↗

arxiv
2611.10658
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.10658v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Cellular world models aim to predict how cells respond to genetic perturbations, analogous to world models in reinforcement learning. Perturbation response models are evaluated almost exclusively by global Pearson correlation between predicted and observed transcriptomes. We show this metric is dominated by thousands of non-responsive genes and severely inflates apparent performance. On the Replogle K562 screen (509 unseen perturbations, 5000 genes), GEARS achieves global Pearson 0.977, but its delta Pearson is only 0.076. A trivial baseline predicting mean training delta achieves 0.267, which is 3.5x better than GEARS. Our primary contribution is methodological: we identify and quantify this delta gap as a systematic evaluation artifact, and propose delta-space evaluation as the primary benchmark. We provide two post-hoc corrections (RWR on co-expression graphs and kNN matching) and show that 2.8x of the remaining gap lies in perturbation representation geometry, not decoder capacity. We further show the gap is partially addressable end-to-end: delta-aligned training (lambda = 3.0) raises delta Pearson to 0.179 on Replogle (2.35x over GEARS) and surpasses the trivial baseline on direction accuracy (0.680 vs 0.567) and gene recovery (1.52 vs 1.19); combined with kNN it reaches 0.217. An amplitude-calibration variant (eta = 0.3) further raises gene recovery to 2.16 while reaching 0.218. Critically, these representations transfer zero-shot across cell lines: from Replogle K562 to unseen Norman A549, the delta-aligned model reaches 0.328 on 741 shared genes, 4.7x a trivial baseline and 2.1x standard GEARS. This is the first systematic quantification of the delta gap, and the first diagnostic framework localizing the residual gap to perturbation representation geometry via an oracle upper bound.