Driving · 2026-10-20
Axiomatic World Modeling for Physics Reasoning
Xinye Yang, Zhenyang Liu, Yuxuan Wang, Yuanyuan Lei
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors define “modeling drift” as the divergence between a model’s internal axiomatic world model (AWM) and the true physical world specified in a physics-reasoning problem. They propose Reinforcement Learning with Axiomatic World Modeling (RLAWM), using a two-level training reward: a modeling reward that scores the externalized AWM and a solving reward that verifies reasoning via executable checkers. The framework operates only during training and improves PhysReason/PHYSICS up to +12.2/+5.2 points.
Why it matters
Physics reasoning failures can come from premise-level mistakes: models can corrupt the physical world they reason inside and then remain self-consistent in that corrupted world. RLAWM targets this directly by supervising (1) what axioms/world the policy constructs and (2) whether its executed reasoning stays physically consistent within that constructed AWM, using interactive checkers for training-time feedback.
Method
- Define modeling drift as divergence between the policy’s inferred AWM and the true physical world specified in the problem; propose RLAWM to penalize this divergence.
- Train with two rewards: a modeling reward (PMF over externalized [modeling] blocks with five fields) and a solving reward (execute [solving] pseudocode and run unit-test checkers for invariants, trace consistency, and response to perturbations).
- Optimize a joint reward R_joint=λ_M R_M + λ_S R_S + λ_A R_A, where R_A is final-answer correctness; authors state RLAWM operates exclusively during training with no extra cost during inference.
Abstract (from arXiv)
Recent research-level physics-reasoning benchmarks for LLMs reveal a failure mode that lies not in knowledge or execution, but in modeling the world being reasoned about. Models often silently alter given quantities, fabricate laws, or inject unstated assumptions to force a problem into familiar patterns, and then reason inside this corrupted world. We define this premise-level divergence as modeling drift: the gap between a model's internal axiomatic world model (AWM) and the true physical world specified in the problem. Although RLVR and PRM have yielded substantial gains in mathematics and code, their reward shapes do not directly supervise world modeling in axiomatic domains such as physics. We propose Reinforcement Learning with Axiomatic World Modeling (RLAWM), which optimizes the policy by penalizing divergence between its inferred axiomatic world model and the true physics world. RLAWM probes physics reasoning at two hierarchical levels (modeling and solving), each scored by a physically grounded reward. The modeling reward targets abstract knowledge formulation, evaluating whether the model captures the true axiomatic world before attempting calculation. The solving reward targets instantiated knowledge reasoning, checking whether the model's concrete derivation remains consistent with the axiomatic world. This two-level structure separates what world the model believes it is reasoning in from how it reasons within that world. The framework operates exclusively during training, with no extra cost during inference. RLAWM improves performance by up to +12.2 points on PhysReason and +5.2 on PHYSICS over the baseline. It also generalizes zero-shot across difficulty, source, and domain, with a 2.45x gain and +10 points over the strongest baseline on olympiad-level physics and math.
Related papers
- Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probing
- Stable and Counterfactually Robust Physical World Models from Imposed Structure and Learned Physics
- Algebraic World Model: From Exploration to Proofs
- $\Psi$-0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model
- ALDER: Discovering the Laws of a World by Acting in It
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-17 · Indexed by Pub-AI: 2026-10-20 UTC
Axiomatic World Modeling for Physics Reasoning
Preprint posted: 2026-10-17 · Original record ↗
- arxiv
- 2610.24519
Explicit source versions (1)
- v1 · 2026-10-17
- Preprint available
Source date: 2026-10-17 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Recent research-level physics-reasoning benchmarks for LLMs reveal a failure mode that lies not in knowledge or execution, but in modeling the world being reasoned about. Models often silently alter given quantities, fabricate laws, or inject unstated assumptions to force a problem into familiar patterns, and then reason inside this corrupted world. We define this premise-level divergence as modeling drift: the gap between a model's internal axiomatic world model (AWM) and the true physical world specified in the problem. Although RLVR and PRM have yielded substantial gains in mathematics and code, their reward shapes do not directly supervise world modeling in axiomatic domains such as physics. We propose Reinforcement Learning with Axiomatic World Modeling (RLAWM), which optimizes the policy by penalizing divergence between its inferred axiomatic world model and the true physics world. RLAWM probes physics reasoning at two hierarchical levels (modeling and solving), each scored by a physically grounded reward. The modeling reward targets abstract knowledge formulation, evaluating whether the model captures the true axiomatic world before attempting calculation. The solving reward targets instantiated knowledge reasoning, checking whether the model's concrete derivation remains consistent with the axiomatic world. This two-level structure separates what world the model believes it is reasoning in from how it reasons within that world. The framework operates exclusively during training, with no extra cost during inference. RLAWM improves performance by up to +12.2 points on PhysReason and +5.2 on PHYSICS over the baseline. It also generalizes zero-shot across difficulty, source, and domain, with a 2.45x gain and +10 points over the strongest baseline on olympiad-level physics and math.