Digest

Reset

1167 papers

2026-10-19 · arXiv:2610.23732

ESC: Escaping 2D Diffusion Ambiguity with Geometric Constraints for Consistent Novel View Synthesis

Ge Ya Luo, Daniel Severo, Brian Karrer et al.

ESC is a geometry-aware diffusion framework for monocular novel view synthesis under drastic camera changes. It jointly predicts target RGB and depth, conditioned on the source image/depth and target camera rays via Plücker ray conditioning. A Projection Consistency Constraint (PCC) enforces bidirectional 3D depth re-projection consistency, and inference uses PCC-based best-of-N selection to return the most geometrically aligned sample.

2026-10-19 · arXiv:2610.23795

Persistent Planar Memory for Video World Models

Yuze He, Bin Tan, Zelin Gao et al.

PPMem is a video world model that maintains a persistent 3D memory using plane primitives. It augments a video diffusion transformer with a planar prediction head and a Depth Token Fusion module: each autoregressive chunk jointly generates RGB and plane geometry, merges planes into a persistent pool, renders depth from the pool, and uses it to condition later chunks. The authors report much better cross-frame coherence and video quality on DL3DV and RealEstate10K, plus metric depth, normals, and exportable planar geometry.

2026-10-16 · arXiv:2610.20614

Thermodynamically Consistent Machine Learning for Fracture Prediction in Heterogeneous Materials

Muhamad Alim, Xiaolong He, Mei-Na Tsao et al.

The authors introduce tLaSDI-pGFINNs to predict fracture patterns and evolution in heterogeneous materials. The model learns a thermodynamic state in a low-dimensional latent space using parametric GENERIC Formalism-Informed Neural Networks (pGFINNs), enforcing total energy conservation and non-decreasing entropy generation via hard constraints. A geometry-conditioned autoencoder improves geometry-guided learning; experiments show better fracture-pattern accuracy than an AE-LSTM baseline and thermodynamic constraint preservation on unseen samples.

RoboticsAI summary
2026-10-16 · arXiv:2610.20693

Orbia: Do Generated Videos Form a Consistent 3D World?

Dongyue Lu, Guian Fang, Hur Lim et al.

Orbia is presented as the first benchmark dedicated to systematic, quantitative evaluation of 3D consistency in interactive video world models. It evaluates three dimensions—input preservation, generated-content persistence, and 3D self-consistency—using controlled exploration-and-revisit protocols with registered RGB-D references and annotated anchors, plus minute-long extensions. The authors build 720 evaluation instances and evaluate 27 models across text-, action-, and camera-conditioned interfaces.

Video World ModelsCode availableAI summary
2026-10-16 · arXiv:2610.20731

WorldCast: Exploring Intelligent Visual Generation in World State Prediction

Wenhao Yan, Haodong Li, Xiaotong Li et al.

WorldCast is a benchmark for multi-step world state prediction from multimodal observations and diverse references, targeting L3–L4 capabilities. It evaluates 12 image/video generation models on 320 manually designed cases (1,352 requested states) using sequence-level keyframes and LMM-as-a-judge scoring (TC/CP/VC). The authors report GPT-Image-2.5 System scoring 72.8, while gaps remain large even for best open-source (20.1).

2026-10-16 · arXiv:2610.20750

Causal Analysis of Robot Foundation Models Enables Efficiency Gains

Aiden Swann, Mathilde Papillon, Mozes Jacobs et al.

The authors analyze, with activation patching, how three vision-language-action (VLA) robot foundation models route image and instruction information into an action expert. They find a narrow middle-to-late “action-conditioning interface” (1–3 cross-attention layers) and use it to sparsify action-expert conditioning. For MolmoBot, restricting to 3/36 layers reduces inference FLOPs by up to 33% while keeping comparable task success; they also report real-robot transfers for π0.5.

RoboticsCode availableAI summary
2026-10-16 · arXiv:2610.20764

When Reality Outpaces Cognition: A New Scaling Principle for Reliable Intelligence

Ahsan Adeel, Talha Bin Riaz

The authors propose the effective Speed of Coherent Cognition (SCeff): the highest tested Speed of Reality SR through which an agent maintains a predefined behavioral reliability criterion continuously from the training baseline. In RL proof-of-concept, agents are trained at SR=1 and evaluated without retraining as SR increases up to 50× with fixed compute. The Co4 architecture achieves substantially higher SCeff than Transformer and gMLP baselines.

DrivingAI summary
2026-10-16 · arXiv:2610.20811

RIGen: Beyond Action Following with Reciprocal Interaction in Generative Gameplay

Mengfei Li, Yi Zhang, Xingtong Ge et al.

RIGen (Reciprocally Interactive Generation) is a framework for reciprocal interaction in generative gameplay. It couples a state/rule engine with an interaction reasoning agent and an action-conditioned audiovisual generator. In an FPS setup, the stateful rules and agent planning determine interaction consequences before generation, then are reconciled using generated visual outcomes. RIGen is trained on a Xonotic gameplay dataset and post-trained for an interactive prototype, RIGen-Arena.

2026-10-16 · arXiv:2610.20866

SHRED: Self-Play for Heterogeneous Road-Users without Expert Demonstrations

Wei-Jer Chang, Zhouchonghao Wu, Akshay Rangesh et al.

SHRED is a demonstration-free heterogeneous self-play framework that jointly trains vehicles, pedestrians, and cyclists from random initialization for closed-loop traffic simulation and ego planning. It uses type-specific dynamics, rewards, and discrete action spaces, plus procedural scenario generation and per-agent randomization (including procedural goal regeneration and goal dropout). On WOSAC, SHRED outperforms a demonstration-free self-play baseline and achieves strong zero-shot realism; on InterPlan, it outperforms compared planners.

DrivingAI summary
2026-10-16 · arXiv:2610.20903

Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation

Yinan Li, Han Han, Kwang-Sung Jun

The authors study bounded regression with misspecification using L1(PX) error to the conditional mean η(x), targeting downstream decision settings. They analyze sign-betting regression (SBR), a min–max estimator from Baraud and Maillard (2025). They prove high-probability L1 oracle inequalities under misspecification with variance-adaptive bounds under realizability, and show the optimal universal approximation factor for proper learning is 3.

AI summary
2026-10-16 · arXiv:2610.20904

EgoJEPA: Beyond Retargeting for Dexterous Manipulation from Egocentric Videos

Meng Cao, Can Zhao, Jinjie Wei et al.

EgoJEPA is a three-stage action-conditioned world modeling framework for dexterous manipulation from egocentric human videos. It avoids rule-based human-to-robot retargeting by learning embodiment-agnostic, effect-aware latent actions via JEPA-style prediction, then trains a latent-action world model, and finally adapts to different robots with lightweight action adapters. On five real-world tasks across three embodiments, it reports 74% average success under out-of-domain evaluation vs 65% for a retargeting baseline.

Latent DynamicsAI summary
2026-10-16 · arXiv:2610.21095

Can World and Action Modeling Benefit Each Other under Unified Causal Training?

Yue Su, Bokai Lin, Xinkai Wang et al.

The authors introduce OneWM, a unified causal training framework that uses a shared video–action architecture where actions are both conditions for forward dynamics prediction and targets for action generation. On LIBERO, LIBERO-Plus, and RoboCasa365 (plus real-world tasks), they report that learning action-conditioned forward dynamics improves policy performance and generalization, and that learning to generate actions improves action-conditioned multi-view world prediction.

RoboticsAI summary
2026-10-16 · arXiv:2610.21283

Mallea: Building, Steering, and Memorizing Worlds with Coding Agent

Xuanyi Liu, Ruiqi Li, Siwei Ma

Mallea is a coding-agent system that builds, steers, and memorizes explicit “code world” representations from text. A coding agent jointly plans object structure, typed properties (fields), and executable rules for interaction dynamics, and preserves unrelated state when edits extend or modify behavior. It stores and reuses appearance only for whitebox-defined objects via object texture memory, while a generative renderer combines current geometry with retrieved textures.

2026-10-16 · arXiv:2610.21319

Constrained Multi-Agent Reinforcement Learning in Imagination

Raz Shmueli, Shahaf S. Shperberg

CA-MAMBA is a constrained multi-agent RL method that extends a multi-agent world-model learner (MAMBA) with per-agent cost prediction. By training policies with reward and cost in imagination and setting the cost Lagrange multiplier from measured real episode cost, the authors report that model-free constrained baselines need at least an order of magnitude more interactions to match CA-MAMBA’s cost and task performance on most SMAC tasks.

DrivingAI summary
2026-10-16 · arXiv:2610.21370

Same Target, Different Origins: Aligning Latent Rollout for Consistent PDE Emulation

Xinying Li, Jing Xu, Wenjun Wang et al.

ALR (Align Latent Rollout) is a latent-space regularization for autoregressive neural PDE solvers. It enforces cross-origin consistency: latent states and their local evolution responses should agree at a shared target time even when reached from different rollout origins/depths. Motivated by second-order Taylor analysis, ALR adds zeroth-order latent state consistency and first-order latent dynamics consistency via Jacobian–vector products, improving accuracy and reducing shared-target discrepancies.

Latent DynamicsAI summary
2026-10-16 · arXiv:2610.21374

From Optical Flow to Safety: Visual Safety Certification via Scale-Free Relative Motion

Bazeela Banday, Jishnu Keshavan

The authors present a monocular visual safety-certification approach that uses scale-free, depth-normalized relative motion from optical flow to certify collision risk. A Concurrent Learning observer estimates the depth-normalized relative velocity with a computable estimation-error bound, which is propagated into a Robust Vision-Based Collision Cone Control Barrier Function (RV-C3BF). An optimal-decay CBF-QP minimally filters a nominal command to enforce robust safety, validated on a TurtleBot3 with only monocular vision and proprioception.

DrivingAI summary
2026-10-16 · arXiv:2610.21389

Fast-VLA: You Only Think Once for Robot Manipulation

Kehong Gong, Guanli Hou, Xiaoxu Zheng et al.

Fast-VLA invokes a frozen 4B VLM once per episode to encode the instruction and initial views into fixed, multi-layer KV task memory. A lightweight online actor then uses learned time- and arm-indexed queries with current visual/proprioceptive feedback to query this fixed memory and predict action chunks without re-running the VLM. The authors report strong success on RoboTwin 2.0 and LIBERO/LIBERO-Plus and lower RoboTwin amortized inference time versus continually conditioned baselines.

RoboticsAI summary
2026-10-16 · arXiv:2610.21539

Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond

Zhiyuan Ma, Jiabao Wei, Tianyu Wang et al.

The authors review and analyze “RL-driven diffusion models” where reinforcement learning post-trains diffusion models by reward-driven trajectory optimization. They discuss preference- and policy-based RL formulations for diffusion, survey RL for diffusion-based language, multimodal, robotic, and scientific models, and identify three bottlenecks for real-world deployment: reward design fragility, exploration–exploitation trade-off, and sparse-reward training instability. They also provide a public repository of related work.

JEPAAI summary
2026-10-16 · arXiv:2610.21612

Learning Intrinsic Dynamics in Latent World Models

Jiamin Shi, Yuanhong Zhang, Zhiyang Liu et al.

The authors study how to preserve control-relevant “intrinsic dynamics” (action→consequence) under visual diversity. They propose that action predictability—reconstructing the executed action from consecutive latent states—discourages encoders from merging outcomes of different actions. They instantiate this with Concat IDM, which adds an inverse-dynamics loss during training for latent planning. On Push-T with 24 appearances, Concat IDM raises planning success from 9.1% to 77.2%.

Latent DynamicsAI summary
2026-10-16 · arXiv:2610.21686

MAIA: Anticipating Interaction Anomalies in Contact-Rich Manipulation via Multimodal World Models

Yi Qiao, Maolin Zheng, Haoran Sun et al.

MAIA anticipates contact-rich manipulation interaction anomalies by predicting future visual and force consequences of planned actions using a multimodal vision–action–force world model, then running a lightweight anomaly head for early warnings. The paper introduces CRIA, a real-world dataset with frame-level annotations of anomaly onset/evolution, enabling evaluation on seen and novel tasks and anomaly-aware action selection.

DrivingAI summary