Video World Models · 2026-11-03

History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

Hangyi Kuang, Chengzhi Wang, Yefei He, Weijie Wang, Yuke Zhao, Zhuokun Chen, Zisu Huang, Yifan Yang, Xiaoxuan He, Feng Chen, Bohan Zhuang

arXiv:2611.02782PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

History Forcing is an on-policy distillation framework for few-step autoregressive video diffusion. It computes a history-dependent residual by comparing an AR teacher’s predictions with vs. without informative historical context, then adds a weighted corrected term to the distillation score/velocity target. This strengthens long-range historical constraints for long rollouts and out-of-sight world modeling.

Why it matters

Long-horizon autoregressive video world models struggle to maintain consistency, especially when subjects or scenes reappear after leaving the field of view. The authors address history–quality coupling by decoupling historical supervision from the teacher’s overall generative supervision, aiming for better long-range consistency and more consistent states upon revisit.

Method

  • History Forcing computes a residual by running an AR teacher on the same noisy student chunk with and without additional historical context, keeping shared conditions fixed.
  • The residual is weighted and added to the distillation target: it reweights/updates the reference score for long-horizon consistency (DMD) and corrects the velocity target for out-of-sight evolution (on-policy self-distillation).
  • History is constructed either from earlier generated student chunks (less drift) for long-video generation, or from a fixed-view teacher rollout (scene visible while student looks away) for out-of-sight world modeling.

Limitation

The fixed-camera teacher rollouts used for historical supervision can themselves contain temporal errors and exhibit limited scene evolution, reducing the accuracy of historical supervision and contributing to inconsistencies upon revisit.

Abstract (from arXiv)

Few-step autoregressive video diffusion enables efficient generation, yet maintaining consistency over long rollouts remains challenging, especially when subjects or scenes reappear after going out of sight. Despite progress in fidelity and rollout robustness, existing distillation methods do not fully exploit richer historical information to constrain the student's current prediction. To address this gap, we introduce \textbf{History Forcing}, a unified on-policy distillation framework that strengthens historical supervision for few-step autoregressive video generators. The framework uses the residual between teacher predictions with and without informative historical context to decouple this supervision. Such a historical context can be obtained using earlier generated chunks with less accumulated drift or a fixed-view teacher rollout that keeps the scene visible while the student looks away. The residual is then weighted and added to the score or velocity prediction used for distillation, forming a corrected target. By aligning the student with this corrected target through distillation, History Forcing incorporates long-range historical constraints into autoregressive generation. Our method achieves state-of-the-art performance among 1.3B models, with a VBench quality score of 85.07 for 15s video generation and a VBench-Long aggregate score of 79.82 for 60s generation. For out-of-sight world modeling, evaluations further demonstrate more plausible scene evolution and more consistent states when the camera returns, with Evolution Quality improving from 2.35 to 2.70 on PlayWorld and Object Consistency from 58.43 to 62.18 on MemoBench.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC

preprint

History Forcing: Distilling History-Dependent Continuations for Long-Horizon Video World Models

Hangyi Kuang, Chengzhi Wang, Yefei He, Weijie Wang, Yuke Zhao, Zhuokun Chen, Zisu Huang, Yifan Yang, Xiaoxuan He, Feng Chen, Bohan Zhuang

First announced by arXiv: 2026-11-03 · Original record ↗

arxiv
2611.02782
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.02782v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Few-step autoregressive video diffusion enables efficient generation, yet maintaining consistency over long rollouts remains challenging, especially when subjects or scenes reappear after going out of sight. Despite progress in fidelity and rollout robustness, existing distillation methods do not fully exploit richer historical information to constrain the student's current prediction. To address this gap, we introduce \textbf{History Forcing}, a unified on-policy distillation framework that strengthens historical supervision for few-step autoregressive video generators. The framework uses the residual between teacher predictions with and without informative historical context to decouple this supervision. Such a historical context can be obtained using earlier generated chunks with less accumulated drift or a fixed-view teacher rollout that keeps the scene visible while the student looks away. The residual is then weighted and added to the score or velocity prediction used for distillation, forming a corrected target. By aligning the student with this corrected target through distillation, History Forcing incorporates long-range historical constraints into autoregressive generation. Our method achieves state-of-the-art performance among 1.3B models, with a VBench quality score of 85.07 for 15s video generation and a VBench-Long aggregate score of 79.82 for 60s generation. For out-of-sight world modeling, evaluations further demonstrate more plausible scene evolution and more consistent states when the camera returns, with Evolution Quality improving from 2.35 to 2.70 on PlayWorld and Object Consistency from 58.43 to 62.18 on MemoBench.