Robotics · 2026-11-06

Learning Robotic World Models with Semantic and Visual Feedback

Hao Wu, Yuqi Li, Yizhou Zhao, Kun Wang, Xiao Luo, Yu Cheng, Bo An, Yingli Tian, Laszlo A. Jeni, Ming-Hsuan Yang

arXiv:2611.07850PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose RoboImagine, a post-training framework for autoregressive robotic world models that learns by comparing candidate imagined futures under shared observations, actions, and instructions. Grouped diverse beam search generates diverse candidates; a frozen multimodal evaluator provides semantic preferences via pairwise video comparisons in both orders, complemented by pixel- and perceptual-level visual consistency feedback. Group-relative advantages drive policy optimization; bounded, branch-specific rolling histories support long-horizon generation.

Why it matters

Evaluating robot world models is difficult because visual reconstruction alone misses task-relevant state progress, while semantic signals may overlook fine-grained visual errors. RoboImagine aims to improve both visual fidelity and task-relevant semantic correctness by combining comparative semantic feedback with visual consistency constraints, and by using diverse candidate generation plus group-relative policy optimization.

Method

  • RoboImagine first trains an autoregressive robotic world model with supervised fine-tuning, then post-trains by comparing candidate futures generated from grouped diverse beam search under shared observation/action/instruction inputs.
  • A frozen multimodal evaluator (Jev) scores candidate pairs for visible task progress using instruction + initial observation, evaluated in both presentation orders to mitigate order bias; reward fusion combines standardized semantic preferences with pixel- and perceptual-level visual consistency feedback.
  • Policy optimization uses group-relative advantages and replays each branch’s recorded rolling state–action history during evaluation, with fixed initial context and bounded branch-specific history for long-sequence generation.

Limitation

The analysis in Appendix E concerns a fixed selected candidate set with fixed feedback and replayed histories; it does not imply an improvement for newly generated candidates, human judgments, or unseen tasks.

Abstract (from arXiv)

Robot world models predict action-conditioned future visual states to support robotic planning and decision making. However, visual reconstruction does not fully capture task progress, semantic evaluation alone provides limited constraints on fine-grained visual errors, and homogeneous predictions limit comparative feedback. We present RoboImagine, a framework for post-training autoregressive robotic world models through candidate-future comparisons with semantic and visual feedback. Diverse search promotes diversity among candidate trajectories under shared visual observations, action sequences, and task instructions. A frozen multimodal evaluator provides task-relevant semantic preferences through pairwise video comparisons in both presentation orders, complemented by pixel- and perceptual-level visual consistency feedback that constrains local structural and appearance errors. These signals drive policy optimization through group-relative advantages. Fixed initial context and branch-specific rolling histories support long-sequence generation with bounded context and consistent conditioning between candidate generation and policy optimization. Experiments with separately trained models on RT-1, BridgeV2, and RoboCasa cover real-world and simulated manipulation. On RT-1, under matched initialization, update counts, and evaluation protocols, RoboImagine improves PSNR by 1.26 dB and reduces LPIPS by 11.93% relative to continued supervised fine-tuning. Ablations show that diverse search improves prediction quality and that combined semantic and visual feedback yields higher external multimodal evaluation scores than visual feedback alone while maintaining comparable visual fidelity. Rolling histories further improve long-horizon prediction. These results support improving visual fidelity, task relevance, and long-horizon prediction through candidate comparison and complementary feedback.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-06 · Indexed by Pub-AI: 2026-11-06 UTC

preprint

Learning Robotic World Models with Semantic and Visual Feedback

Hao Wu, Yuqi Li, Yizhou Zhao, Kun Wang, Xiao Luo, Yu Cheng, Bo An, Yingli Tian, Laszlo A. Jeni, Ming-Hsuan Yang

First announced by arXiv: 2026-11-06 · Original record ↗

arxiv
2611.07850
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-06 · First detected: 2026-11-06 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.07850v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

Robot world models predict action-conditioned future visual states to support robotic planning and decision making. However, visual reconstruction does not fully capture task progress, semantic evaluation alone provides limited constraints on fine-grained visual errors, and homogeneous predictions limit comparative feedback. We present RoboImagine, a framework for post-training autoregressive robotic world models through candidate-future comparisons with semantic and visual feedback. Diverse search promotes diversity among candidate trajectories under shared visual observations, action sequences, and task instructions. A frozen multimodal evaluator provides task-relevant semantic preferences through pairwise video comparisons in both presentation orders, complemented by pixel- and perceptual-level visual consistency feedback that constrains local structural and appearance errors. These signals drive policy optimization through group-relative advantages. Fixed initial context and branch-specific rolling histories support long-sequence generation with bounded context and consistent conditioning between candidate generation and policy optimization. Experiments with separately trained models on RT-1, BridgeV2, and RoboCasa cover real-world and simulated manipulation. On RT-1, under matched initialization, update counts, and evaluation protocols, RoboImagine improves PSNR by 1.26 dB and reduces LPIPS by 11.93% relative to continued supervised fine-tuning. Ablations show that diverse search improves prediction quality and that combined semantic and visual feedback yields higher external multimodal evaluation scores than visual feedback alone while maintaining comparable visual fidelity. Rolling histories further improve long-horizon prediction. These results support improving visual fidelity, task relevance, and long-horizon prediction through candidate comparison and complementary feedback.