Video World Models · 2026-11-06

MAC-v2: Long-horizon model-based value expansion from pixels

Kwanyoung Park, Sergey Levine

arXiv:2611.07245PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

MAC-v2 is an offline model-based RL method for pixel observations that enables long-horizon model-based value expansion. The authors train a reconstruction-free latent world model with action chunking, learning representations from prediction and control objectives. This reduces recursive compounding of prediction error and allows long-horizon latent imagination over visual benchmarks (visual OGBench, MMBench2) and robotics tasks (Robomimic, LIBERO-10).

Why it matters

Pixel-based offline RL is harder for long-horizon value propagation because the agent must learn both representation and dynamics from fixed data, and errors can accumulate recursively in imagined rollouts. The paper’s approach combines reconstruction-free latent modeling with action chunking to mitigate compounding error, and evaluates across multiple visual and robotic offline benchmarks.

Method

  • Trains a deterministic encoder fξ for a compact latent state, using reconstruction-free dynamics and reward learning objectives plus behavior cloning/control objectives.
  • Learns an action-chunked latent dynamics model pψ(Z, A^n)->Z and a two-hot encoded action-chunked reward model rψ(Z, A^n)->Δ(R), trained from dataset transitions.
  • Performs long-horizon latent value expansion using action-chunked rollout in latent space with rejection sampling from a conditional flow (behavior cloning) policy.
Abstract (from arXiv)

We study whether \emph{long-horizon model-based value expansion} can serve as an effective framework in pixel-based offline reinforcement learning (RL). Long-horizon model-based value expansion from pixels is substantially harder than state-based environments, as the agent must learn both the representation and its dynamics, and errors from either can accumulate recursively over imagined trajectories. We introduce MAC-v2, which enables long-horizon model-based expansion from pixels by training a reconstruction-free latent world model with action chunking. By learning representations from prediction and control objectives, MAC-v2 does not need to encode control-irrelevant visual details that can hinder the dynamics prediction. Action chunking further mitigates the prediction error by reducing the number of recursive model predictions required for value expansion. Across diverse visual benchmarks including visual OGBench, MMBench2, Robomimic, and LIBERO-10, MAC-v2 achieves strong performance against model-free and model-based offline RL baselines. Our analyses show that long-horizon model-based value expansion is critical for the performance of MAC-v2, and that both action chunking and latent learning objectives are essential for reliable long-horizon latent imagination.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-06 · Indexed by Pub-AI: 2026-11-06 UTC

preprint

MAC-v2: Long-horizon model-based value expansion from pixels

Kwanyoung Park, Sergey Levine

First announced by arXiv: 2026-11-06 · Original record ↗

arxiv
2611.07245
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-06 · First detected: 2026-11-06 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.07245v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

We study whether \emph{long-horizon model-based value expansion} can serve as an effective framework in pixel-based offline reinforcement learning (RL). Long-horizon model-based value expansion from pixels is substantially harder than state-based environments, as the agent must learn both the representation and its dynamics, and errors from either can accumulate recursively over imagined trajectories. We introduce MAC-v2, which enables long-horizon model-based expansion from pixels by training a reconstruction-free latent world model with action chunking. By learning representations from prediction and control objectives, MAC-v2 does not need to encode control-irrelevant visual details that can hinder the dynamics prediction. Action chunking further mitigates the prediction error by reducing the number of recursive model predictions required for value expansion. Across diverse visual benchmarks including visual OGBench, MMBench2, Robomimic, and LIBERO-10, MAC-v2 achieves strong performance against model-free and model-based offline RL baselines. Our analyses show that long-horizon model-based value expansion is critical for the performance of MAC-v2, and that both action chunking and latent learning objectives are essential for reliable long-horizon latent imagination.