Digest

Reset

1167 papers

2026-10-07 · arXiv:2610.07031

Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update

Sitian Shen, Jiuming Liu, Mengmeng Liu et al.

Artemis is a geometry-grounded multi-agent driving world model that reconstructs an explicit shared 3D world map from multi-agent observations, injects action-guided geometric control maps into a diffusion transformer via a GeoAdapter, and progressively updates the shared 3D state during rollouts. It also maintains history for uncontrolled non-agent dynamics and pose memory for controlled agents.

DrivingAI summary
2026-10-07 · arXiv:2610.07127

PlaySuite: A Large-Scale Benchmark for Interactive Visual Intelligence

Dheeraj Varghese, Anna Vettoruzzo, Walter Simoncini et al.

PlaySuite is an open, large-scale benchmark for interactive visual intelligence across 5,734 open-source video games curated from PyWeek and itch.io. It uses a unified keyboard/mouse interaction layer for HPC closed-loop evaluation and a Video-LLM-as-a-judge that maps milestone evidence to 5 progress levels. Testing 14 open multimodal interactive models shows a perception-action gap, with recurring failures in spatial grounding, action execution, and self-correction.

2026-10-07 · arXiv:2610.07355

Tracking Is Not Permanence: What Video World Models Keep of a Hidden Object

Peng Xie, Amr Alanwar

The authors probe permanence in frozen video world models by hiding an object from a V-JEPA 2 predictor and comparing its hidden-region prediction to the frozen encoder’s representations of two futures that differ only in the hidden region. V-JEPA 2 keeps a stationary hidden object partly, keeps a carried one not at all, and loses a moving one within 0.3s (0.5s with its tube mask). Training can install permanence via a prior from synthetic containers.

2026-10-07 · arXiv:2610.07381

GeoWM: Efficient Direct World Modeling in Explicit Geometry

Mehrdad Noori, Guile Wu, Sam Hosseini et al.

GeoWM is a geometry world model that directly forecasts future 3D scene geometry at specified horizons, without recursive rollout. It turns observed RGB frames into a geometric history using a frozen geometry foundation model, predicts future camera motion, projects the last observed depth into the predicted viewpoint as an anchor, and uses a flow-matching diffusion transformer to predict future depth directly for each horizon.

DrivingAI summary
2026-10-07 · arXiv:2610.07382

Toward Trustworthy Physical AI for Human Interaction

Niccol\`o Pagliarani, Maximilian St\"olzle, Cosima du Pasquier et al.

The authors propose a framework for “trustworthy physical AI for human interaction” that integrates three system-side dimensions: Safety, Behavioral Intelligibility, and Perceptual Alignment. Trustworthiness is treated as a context-dependent system property, emerging from alignment between physical capabilities, observable behavior, and what people expect during interaction. The framework connects these dimensions to three technical pillars: Embodied, Motor-Cognitive, and Communicative Intelligence.

RoboticsAI summary
2026-10-07 · arXiv:2610.07383

HyperNSDE: Personalized Neural SDEs for Joint Static-Longitudinal Clinical Data Generation

Perrine Chassat, Agathe Guilloux

HyperNSDE is a continuous-time generative model for clinical cohorts that jointly generates heterogeneous static covariates, irregularly sampled longitudinal trajectories, and informative observation times. It uses a HI-VAE for static latents, a hypernetwork-personalized Neural SDE with stochastic latent dynamics (without a trajectory encoder), and a latent-state-dependent intensity process for observation times, trained via deterministic–stochastic path decomposition and a non-adversarial signature-kernel objective.

Latent DynamicsAI summary
2026-10-07 · arXiv:2610.07540

Preserving Unstable Modes Through Inverse Dynamics in JEPA World Models

Leonardo F. Toso, Yann LeCun, James Anderson et al.

The authors show that standard JEPA training (next-step prediction + anti-collapse like SIGReg) can minimize loss while collapsing controllable unstable modes, preventing stabilization from the learned latent representation. They add an endpoint inverse-dynamics (EP-IDM) action reconstruction objective and prove that exact reconstruction makes the encoder injective on the H-step reachable subspace, preserving unstable modes; they further show convergence to the controllable unstable subspace as H grows.

JEPAAI summary
2026-10-07 · arXiv:2610.07599

Modeling Latent Disturbances for Robust Decision-Making in World Models

Junwon Seo, Andrea Bajcsy

The authors study robust decision-making in world-model (WM) latent space by introducing a latent-space disturbance that yields pessimistic but plausible WM imaginations. They build an uncertainty set of plausible latent dynamics using a dynamics-aware KL similarity metric plus out-of-distribution detection, calibrate it with conformal prediction, and then jointly optimize robust actions and the worst-case latent disturbances via game-theoretic optimization for robust safety filtering and sample-and-verify steering.

Latent DynamicsAI summary
2026-10-07 · arXiv:2610.07609

PhysLDM: Latent Diffusion for High-Fidelity Deformable Simulation

Yu Zhang, Xudong Xu, Xingang Pan

PhysLDM is a latent-diffusion paradigm for one-shot prediction of high-fidelity volumetric deformable-body trajectories. It uses a holistic spatiotemporal VAE (ST-VAE) that avoids “staircase” artifacts and decouples rigid motion from local deformation, achieving ~2.48 mm reconstruction precision at up to 78× token compression. In chaotic dynamics, diffusion models physically plausible multi-modal futures better than deterministic regression. A single kinematics-only model generalizes zero-shot to unseen OOD datasets and supports differentiable inverse problems.

2026-10-07 · arXiv:2610.07652

SMART: Zero-Shot Sim-to-Real Articulated Object Manipulation via Large-Scale Synthetic Pretraining

Jicong Ao, Shuhan Jiang, Yuling Zhong et al.

SMART introduces SMART-Sim, an articulation-aware simulation platform, and an agentic/distributed synthesis pipeline to generate SMART-Data: over 1M synthetic demonstrations across 44 atomic task types, 5 robot setups, and 2,507 articulated objects. A VLA model pretrained on SMART-Data achieves competitive simulation benchmark performance and reports zero-shot sim-to-real transfer plus scalable performance for real-world articulated-object manipulation tasks.

DrivingAI summary
2026-10-07 · arXiv:2610.07704

Independent Multi-Agent Reinforcement Learning with Counterfactual Semantic-Social World Models

Fernando Martinez, Tao Li, Yingdong Lu et al.

CASTLE is an offline-training, online-in-context guidance method for fully decentralized (independent) multi-agent reinforcement learning. Each agent uses two frozen world models: a Local Dynamics World Model (LDWM) for predictive local dynamics/partial observability and a Semantic-Social World Model (SSWM) trained on counterfactual simulator rollouts for action-specific short-horizon task/social consequences. Logits from these predictions guide an independent PPO policy without communication or a centralized critic.

Latent DynamicsAI summary
2026-10-07 · arXiv:2610.07922

OpenWAM: An Open Framework for Composable World-Action Models

Heng Yu, David D. Yuan, Juze Zhang et al.

OpenWAM is an open world-action modeling framework that couples a common causal robot-video backbone (from Wan2.2-5B) with configurable video–action interaction programs (joint, VTA, ATV, decoupled) using a shared Mixture-of-Transformers architecture. Starting from causal robot-video pretraining on 10,000+ hours, it achieves higher VTA success on LIBERO-Long via causal adaptation and enables local-context inverse/forward dynamics learning with counterfactual transitions.

DrivingCode availableAI summary
2026-10-07 · arXiv:2610.07949

Commit While Futures Agree: Consequence-Aware Adaptive Action Chunking for Robot Manipulation

Yuyan Li, Yujia Wang, Yusong Huang et al.

CA$^3$C is an inference-time action-chunk execution framework for robot manipulation. It samples multiple candidate action chunks, uses an action-conditioned world model to imagine their consequences under shared sampling noise, and computes cross-candidate future disagreement. It then uses Bayesian change-point inference to choose a trustworthy execution horizon (“commit while imagined futures agree”) and selects the candidate closest to future consensus.

RoboticsAI summary
2026-10-07 · arXiv:2610.07961

IronMan: Information-Constrained Video-Action Learning for Robot Manipulation

Yuanshuo Zhang, Wenzhe Zhao, Zixing Lei et al.

IronMan is an information-constrained video-action learning framework for robot manipulation. It distills noisy one-step video features into compact “world tokens” using a dynamics-aware bottleneck based on the information bottleneck principle. Video and action are trained jointly with flow-matching and a variational IB objective. The authors report strong in-distribution performance and out-of-distribution robustness while keeping efficient inference.

2026-10-07 · arXiv:2610.07969

EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation

Yikai Qin, Yifei Deng, Mingjian Liang et al.

EmbodiedSmith is a simulation pipeline for scalable embodied data generation via recursive self-improvement. It unifies asset, scene, and task generation, using a loop where scene generation anticipates task requirements and task generation drives targeted scene edits. The authors report higher scene quality metrics and higher demonstration success rates than prior scene-generation methods.

RoboticsAI summary
2026-10-07 · arXiv:2610.08119

AutodidactWAM: Cross-Modal Self-Distillation from Generated Video to Robot Actions

Sergei Kurchev, Iaroslav Kolomiets, Miguel Altamirano Cabrera et al.

The authors adapt a Cosmos 3 world-action model (video+robot actions) to a new embodiment (Unitree G1 with BrainCo five-fingered hands) and find a video–action asymmetry: generated videos are often plausible, while native co-generated actions are systematically mis-targeted. AutodidactWAM extracts action targets from the model’s own generated video via a frozen hand-pose estimator + IK, then fine-tunes only action-related layers using self-distillation objectives.

RoboticsAI summary
2026-10-07 · arXiv:2610.08150

ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models

Yuan Xu, Yixiang Chen, Qisen Ma et al.

ViDAL introduces a Visual Dynamics-grounded Action Latent Space for Vision-Language-Action (VLA) models. It trains an Action VAE to reconstruct action chunks while aligning action latents with future visual dynamics extracted by a frozen video VAE. The frozen Action VAE is then a plug-in action interface for downstream VLA policies and enables optional future-video prediction. The authors report improved success on LIBERO, RoboTwin 2.0, and two real-robot platforms.

RoboticsAI summary
2026-10-07 · arXiv:2610.08192

MacJEPA: Missingness-Robust Audio-Visual Recognition from Untrimmed Egocentric Videos

Souptik Sen, Zahra Ahmadi

MacJEPA is a missing-modality-robust audio-visual recognition model for untrimmed egocentric videos with temporally localized sensor outages. It uses interval-conditioned queries over multimodal untrimmed context, trains with window-local modality dropout using typed, timed masks, and repurposes JEPA’s masked-to-clean latent alignment as a supervised robustness objective jointly with recognition—without test-time adaptation.

2026-10-07 · arXiv:2610.08350

How Much Planning Is Enough? Reducing Search and Computation in World-Model Planning

Changbai Li, Sirui Li, Yichen Yang et al.

The authors analyze why visual world-model planning can waste computation: fixed “Full” budgets can evaluate excess candidates after outcome sufficiency, and iterative planners repeatedly encode solve-static observation/goal representations inside the refinement loop. They propose SufficientPlan with PSBC (certify a reduced budget using paired closed-loop outcomes) and SCR (reuse solve-static encodings across CEM iterations) without retraining or planner changes.

2026-10-07 · arXiv:2610.08400

Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems

Kasper Helverskov Petersen, Rasmus Hannibal Tirsgaard, Fran\c{c}ois R J Cornet et al.

Atom-JEPA is a self-supervised pretraining framework for 3D atomistic systems that learns latent representations from unlabeled molecular and crystalline structures using complementary atom-level and substructure-level latent-space prediction objectives inspired by JEPA. The authors pretrain on large molecular (Uni-Mol) and crystal (Alexandria) datasets and fine-tune the pretrained context encoder for downstream property prediction.

JEPACode availableAI summary