Latent Dynamics · 2026-10-22
PhysiCalWorld: In-Context Physics Calibration for World Models
Yunhao Luo, Wentao He, Yinpei Dai, Hyelin Nam, Hongze Fu, Joyce Chai, Nima Fazeli, Jeong Joon Park
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PhysiCalWorld calibrates action-conditioned latent world models to instance-specific physics using in-context prompting. It encodes an exploratory interaction clip from the test scene into a compact physics descriptor token and conditions future rollouts on it, trained end-to-end with only a latent prediction loss (no physics labels, no fine-tuning). At inference, the descriptor is extracted once and reused across rollouts.
Why it matters
World models conditioned only on the current frame or short history may not identify physics properties that determine transition dynamics, causing drift on novel scenes. PhysiCalWorld grounds predictions in the scene’s actual dynamics via a reusable, single-token physics descriptor extracted from in-context interactions, improving rollout prediction accuracy and visual planning success without analytical physics modeling or test-time weight updates.
Method
- Collect a short in-context interaction clip (observation-action pairs) in the test scene and encode it into a single compact physics descriptor token via a context encoder.
- Condition an action-conditioned latent dynamics predictor on the physics descriptor token (in addition to the default history window) to produce future latent predictions.
- Train the context encoder and world model end-to-end using only the latent prediction objective, with no physics labels; at inference, extract the descriptor once and reuse it across rollouts/planning.
Limitation
At test time, the action prompt should produce physical contact with the environment since interactions lacking contact or motion cannot reveal the underlying dynamics.
Abstract (from arXiv)
Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without fine-tuning or analytical physics modeling. Our method encodes an exploratory interaction clip collected within the test scene into a compact physics descriptor token, which grounds subsequent predictions in the scene's actual physical behavior. PhysiCalWorld is trained end-to-end with the latent prediction loss alone, without physics labels. At inference time, the descriptor token is extracted once and can be reused across rollouts with negligible compute overhead. We evaluate PhysiCalWorld across four environments covering diverse physics variations and show that it substantially improves rollout prediction accuracy and visual planning success rate. The learned descriptors also form a physically structured latent space and generalize beyond the physics ranges seen during training.
Related papers
- STRIKE: Learning Visual State Transitions for Physical World Modeling
- World Models' Last Exam in Physics
- World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
- Counterfactual Action Evaluation, Observation Bottlenecks, and Representation Geometry in Joint-Embedding Predictive World Models
- Code2Worlds-2: Physically Plausible Agent Harness for Embodied World Simulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-21 · Indexed by Pub-AI: 2026-10-22 UTC
PhysiCalWorld: In-Context Physics Calibration for World Models
Preprint posted: 2026-10-21 · Original record ↗
- arxiv
- 2610.30125
Explicit source versions (1)
- v1 · 2026-10-21
- Preprint available
Source date: 2026-10-21 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Existing world models typically perceive a scene through current visual frame or a narrow history window. However, these observations may not reveal the physics properties that determine transition dynamics, such as an object's mass distribution, joint damping, or surface friction. Without calibrating these physics factors for world models, predictions tend to regress toward an empirical prior and induce severe drift in novel test scenes. To this end, we propose PhysiCalWorld, which calibrates action-conditioned world models to the instance-specific dynamics via in-context prompting, without fine-tuning or analytical physics modeling. Our method encodes an exploratory interaction clip collected within the test scene into a compact physics descriptor token, which grounds subsequent predictions in the scene's actual physical behavior. PhysiCalWorld is trained end-to-end with the latent prediction loss alone, without physics labels. At inference time, the descriptor token is extracted once and can be reused across rollouts with negligible compute overhead. We evaluate PhysiCalWorld across four environments covering diverse physics variations and show that it substantially improves rollout prediction accuracy and visual planning success rate. The learned descriptors also form a physically structured latent space and generalize beyond the physics ranges seen during training.