Latent Dynamics · 2026-11-09
MotionCore: A Latent Neural Simulator for Video Generation with Rigid-Body Dynamics
Zheng Wang, Zhengwei Gong, Yuanzhi Liang, Haibin Huang, Chi Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
MotionCore learns rigid-body physical dynamics in latent space using a neural operator/PDE-inspired solver, then feeds predicted object trajectories into a controllable video diffusion model via spatially aligned latent controls. The framework uses a diffusion-matching objective plus a low-capacity decodability constraint to preserve trajectory through the visual interface. It evaluates on physical-consistency benchmarks and paired data with known trajectories.
Why it matters
The authors address a gap where realistic video generation can produce physically inconsistent motion. MotionCore separates physical trajectory prediction (learned dynamics) from visual rendering (fixed diffusion generator), aiming for improved physical consistency and trajectory obedience while keeping visual quality comparable. The paper also includes an ablation protocol comparing learned dynamics to an analytic prior and studying latent evolution processors.
Method
- Learn a latent neural simulator (encoder→latent operator→decoder) to predict future rigid-body trajectories from an observed motion history, approximating rigid dynamics via a PDE formulation using compliant elastic contact laws.
- Convert predicted physical states into spatially aligned latent controls for a controllable video diffusion generator (VACE built on Wan2.1-1.3B), keeping the generator fixed as the visual interface.
- Preserve trajectory through the interface using a diffusion-matching objective together with a low-capacity decodability constraint on object identity and position; include analytic-prior and processor-choice ablations.
Abstract (from arXiv)
Video generation models can produce highly realistic videos while generating physically inconsistent object motion. This limitation suggests that visual modeling alone may not provide a transferable representation of physical dynamics. We introduce MotionCore, a framework for learning physical dynamics via a learnable partial differential equation solver and focuses on rigid-body motion. We introduce a latent neural simulator learning to predict the evolution of object-level physical states from observed motion. Its predictions are then converted into spatially aligned latent controls for a controllable video diffusion model. The generator serves as a visual interface: it provides appearance and scene detail, while the learned dynamics model specifies the physical trajectory. To preserve this trajectory through the interface, MotionCore combines a diffusion-matching objective with a low-capacity decodability constraint on object identity and position. We evaluate MotionCore on public physical-consistency benchmarks and on paired data with known trajectories. Results indicate that MotionCore improves physical consistency and trajectory obedience in the evaluated rigid-body settings, while maintaining comparable visual quality. A controlled ablation protocol compares learned dynamics with an analytic prior and studies the choice of latent evolution processor.
Related papers
- Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models
- PVBench: Quantifying Physical Plausibility in Video Generation with Object-Level Physical Measurements
- PhysLDM: Latent Diffusion for High-Fidelity Deformable Simulation
- WorldFlow: Joint Video and 3D Motion Prediction via Large-Scale Pretraining
- Structured Latent Dynamics for Long-Horizon Embodied World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-06 · Indexed by Pub-AI: 2026-11-09 UTC
MotionCore: A Latent Neural Simulator for Video Generation with Rigid-Body Dynamics
Preprint posted: 2026-11-06 · Original record ↗
- arxiv
- 2611.08904
Explicit source versions (1)
- v1 · 2026-11-06
- Preprint available
Source date: 2026-11-06 · First detected: 2026-11-09 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · atom:published (official metadata)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.08904v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Video generation models can produce highly realistic videos while generating physically inconsistent object motion. This limitation suggests that visual modeling alone may not provide a transferable representation of physical dynamics. We introduce MotionCore, a framework for learning physical dynamics via a learnable partial differential equation solver and focuses on rigid-body motion. We introduce a latent neural simulator learning to predict the evolution of object-level physical states from observed motion. Its predictions are then converted into spatially aligned latent controls for a controllable video diffusion model. The generator serves as a visual interface: it provides appearance and scene detail, while the learned dynamics model specifies the physical trajectory. To preserve this trajectory through the interface, MotionCore combines a diffusion-matching objective with a low-capacity decodability constraint on object identity and position. We evaluate MotionCore on public physical-consistency benchmarks and on paired data with known trajectories. Results indicate that MotionCore improves physical consistency and trajectory obedience in the evaluated rigid-body settings, while maintaining comparable visual quality. A controlled ablation protocol compares learned dynamics with an analytic prior and studies the choice of latent evolution processor.
