Video World Models · 2026-10-23
PhyEvolve: Multi-Agent State-Evolution Planning for Physics-Aware Video Generation
Yang-Hao Zhou, Lixing Tan, Zhiqi Li, Guang Yang, Xiao Yang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PhyEvolve is a multi-agent prompt-enhancement framework for physics-aware text-to-video generation. A Director grounds the scene and separates physical actions from observation changes, a Physical Evolver constructs causally linked state transitions (including out-of-sight evolution), and a Composer rewrites the prompt. It improves existing generators via prompt replacement without retraining or modifying the generator. The authors report state-of-the-art gains on four physics video benchmarks.
Why it matters
The authors target failures where prompts do not explicitly preserve physical mechanisms and state continuity across visible and unobserved intervals. By planning causal state evolution and embedding it into a complete replacement prompt, PhyEvolve aims to improve physical consistency and state continuity without retraining the underlying video model. The reported improvements are on benchmarks designed for state evolution under interrupted observation and broader physical correctness.
Method
- Director grounds the scene and disentangles physical events from observation changes; it records which events may alter physical conditions, object state, visibility, or combinations.
- Physical Evolver analyzes physical mechanisms and performs causal state rollout into dependent state transitions, carrying evolved after-states through subsequent events including occluded intervals.
- Composer integrates the planned records into a self-contained replacement prompt and refines only flagged sentences for omissions or conflicts before passing it to the unchanged video generator.
Limitation
In VideoPhy-2, the authors report that semantic adherence remains low, imposing a tighter ceiling on Joint success than physical commonsense under this evaluator.
Abstract (from arXiv)
Recent advances in text-to-video generation have substantially improved visual fidelity, yet existing models still struggle to understand the physical evolution implicit in user prompts, making it difficult to generate videos with continuous physical state evolution. Here, we propose PhyEvolve, a multi-agent framework that enhances natural-language prompts through structured state-evolution planning. PhyEvolve consists of three agents: a Director, a Physical Evolver, and a Composer. Given a user prompt and an optional first frame, the Director grounds the scene and disentangles physical events from observation changes. The Physical Evolver uses scene evidence to identify physical mechanisms then constructs causally linked state transitions that unfold throughout the video, whether visible or not. The Composer utilizes these transitions to enhance the original prompts. Without retraining or model modification, PhyEvolve works directly with existing generators through prompt replacement. Extensive experiments on four major benchmarks show that PhyEvolve achieves state-of-the-art performance, outperforming the cutting-edge prior agentic baseline by 7.9 percentage points on average. These results demonstrate that explicit causal state-evolution planning improves physical consistency and state continuity in generated videos through natural-language prompt enhancement.
Related papers
- Physis-Lang: Self-Evolving Language as a Physical Representation for Video World Model
- StateCraft: Augmenting Video World Models with Abstract-State Reasoning
- MindWorldBench: Evaluating Mental-State-to-Behavior Reasoning in Image-to-Video Generation
- PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements
- RoboJudge: Multimodal Language Models as Judges for Embodied Video Generation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-22 · Indexed by Pub-AI: 2026-10-23 UTC
PhyEvolve: Multi-Agent State-Evolution Planning for Physics-Aware Video Generation
Preprint posted: 2026-10-22 · Original record ↗
- arxiv
- 2610.30717
Explicit source versions (1)
- v1 · 2026-10-22
- Preprint available
Source date: 2026-10-22 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Recent advances in text-to-video generation have substantially improved visual fidelity, yet existing models still struggle to understand the physical evolution implicit in user prompts, making it difficult to generate videos with continuous physical state evolution. Here, we propose PhyEvolve, a multi-agent framework that enhances natural-language prompts through structured state-evolution planning. PhyEvolve consists of three agents: a Director, a Physical Evolver, and a Composer. Given a user prompt and an optional first frame, the Director grounds the scene and disentangles physical events from observation changes. The Physical Evolver uses scene evidence to identify physical mechanisms then constructs causally linked state transitions that unfold throughout the video, whether visible or not. The Composer utilizes these transitions to enhance the original prompts. Without retraining or model modification, PhyEvolve works directly with existing generators through prompt replacement. Extensive experiments on four major benchmarks show that PhyEvolve achieves state-of-the-art performance, outperforming the cutting-edge prior agentic baseline by 7.9 percentage points on average. These results demonstrate that explicit causal state-evolution planning improves physical consistency and state continuity in generated videos through natural-language prompt enhancement.