Digest

Reset

1167 papers

2026-11-06 · arXiv:2611.08392

Learning a patient world model for health system simulation in intensive care

Yingtai Li, Ruoyan Wang, Yu Li et al.

The authors propose a time-conditioned patient world model (pWM) trained on longitudinal EHRs (MIMIC-IV). The frozen pWM is coupled to a discrete-event virtual ICU via scenario-specific mappings to generate admission, care tasks, and discharge proposals. Experiments show it matches Markov/semi-Markov comparators on aggregate occupancy and directional policy responses, and it recovers a 24-h recovery difference that compact state models miss.

DrivingAI summary
2026-11-06 · arXiv:2611.08449

VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding

Yu-Cheng Chou, Shihao Wang, Jin Huang et al.

VFlash uses Flash Primitive Decoding to generate complete visual primitives (e.g., boxes and variable-length trajectories) jointly in one forward pass instead of one coordinate at a time. It trains on annotated primitives and accepts structurally valid predictions without requiring target-token agreement for spatial plans, while still using verified speculative decoding for text. Across 2D/3D robot trajectory tasks it reduces prediction errors vs AR decoding, and a VFlash-conditioned world action model improves MolmoSpaces success.

DrivingAI summary
2026-11-05 · arXiv:2611.05443

SurgTransfer: Multi-Step Latent Action Modeling for Action Transfer in Surgical World Model

Saurabh Koju, Love Panta, Deepak Thapa et al.

SurgTransfer learns transferable latent surgical actions from unlabeled videos. It introduces a multi-step latent action model (LAM) that learns a single composite action spanning longer horizons, and an action-conditioned multi-step dynamics model based on finetuning Stable Video Diffusion. The authors test on SurgToolLoc-2022 and PitVis-2023 and evaluate with quantitative metrics and qualitative assessments by surgical experts.

Latent DynamicsCode availableAI summary
2026-11-05 · arXiv:2611.05448

Jev with Foresight: Safe Language-Guided Autonomous Driving via a Language World Model

Yansong Qu, Jiancong Chen, Samuel Labi

Jev-AD combines Jev (a typed decision model) with a language world model that predicts, for each candidate action, how nearby vehicles move over the next 2 s. Code converts predictions into foreseen consequences (collision/Danger, TTC, final lane/speed), which Jev uses to choose safe actions that match passenger intent. In KoMA highway merging, foresight lowers collisions and achieves 93% intent; on replayed NGSIM traffic it also lowers collisions and foresees 93% dangerous actions.

DrivingAI summary
2026-11-05 · arXiv:2611.05570

ChangeJEPA: Predict Before You Compare for Remote Sensing Change Detection

Buddhi Wijenayake, Dhananjaya Jayasundara, Vishal M. Patel

ChangeJEPA reformulates remote sensing change detection as latent post-state prediction. Given a pre-change image and partial post-change evidence, it predicts a full post-change latent representation under two complementary conditions (prior vs updated). Change is inferred from disagreement among pre representation, observed post target, and predicted latent states, aiming to suppress nuisance variation while keeping semantic consistency.

JEPACode availableAI summary
2026-11-05 · arXiv:2611.05571

ELA: End-to-End Latent Action Policy Learning from Video Demonstrations

Cheng-Chun Hsu, Peter Stone

ELA is an end-to-end framework for learning robot imitation from video demonstrations using latent actions. It jointly trains a Latent Action Model (inverse + forward dynamics), a latent action policy, and a continuous action decoder, optimizing latent actions for both policy learning and action decoding. The authors report consistent improvements over latent-action baselines on simulated and real-world manipulation tasks.

RoboticsAI summary
2026-11-05 · arXiv:2611.05577

Readable Is Not Steerable: Pre-Registered Tests of Physical Structure in Learned Fluid Representations

Manvendra Modgil

The authors test whether self-supervised world models’ latents are organized around physical quantities and whether such quantities are steerable, using three pre-registered studies on 2D decaying Navier–Stokes flow. They report that what latents contain and what can be controlled come apart: a JEPA prediction objective fails prominence criteria, autoencoders can make enstrophy prominent but steering differs, and label-free “dials” yield loudness but only sometimes spectral-shape control.

Latent DynamicsAI summary
2026-11-05 · arXiv:2611.05847

Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models

Ruilong Yu, Fei Ye

The authors analyze why a flat JEPA planner’s Euclidean goal cost gives weak temporal guidance over long horizons. They introduce Time-to-Goal (T2G), a lightweight learned temporal metric trained from offline latent trajectories (no rewards/policy labels). With the JEPA encoder/predictor frozen, T2G replaces the Euclidean goal cost and improves long-horizon planning success on multiple benchmarks, approaching or surpassing hierarchical H-JEPA in their evaluations.

Latent DynamicsCode availableAI summary
2026-11-05 · arXiv:2611.05890

CellCanon: An Agentic Cellular World Model for Prediction, Explanation, and Observation-Driven Updating

Zhaokang Liang, Shuyang Zhuang, Xiaoran Jiao et al.

CellCanon is an agentic cellular world model that unifies single-cell expression prediction, evidence-grounded explanation, and observation-driven updating. It reuses condition-level biological evidence across gene-specific numerical readouts. On 339 held-out SciPlex3 conditions, the authors report improved differential-expression AUPRC and Macro-F1 over the best baseline, and reduced language-model token use for Predict/Explain. The Explain module is source-grounded, and Update revises predictions from newly measured responses without changing model weights.

Latent DynamicsAI summary
2026-11-05 · arXiv:2611.05907

Does Your Trial Simulator Know What the Drug Does? An Intervention-Blind Test of Interventional Validity on 7,207 Registered Trials

Alicem Koyun

The authors evaluate “interventional validity” for trial simulators using an intervention-blind reference. On a temporal split (878 test contrasts / 655 trials), a registry-trained gradient-boosting simulator removes about a fifth of trivial error but has interventional skill −0.12 (95% CI −0.21 to −0.05). A frontier protocol-reading LLM has interventional skill 0.20 [0.08, 0.31].

DrivingAI summary
2026-11-05 · arXiv:2611.06007

MotoVerse: A Benchmark for Predictive Perception of Motorized Two-Wheelers in Dense Unstructured Traffic

Varun A. Paturkar, Shankar Gangisetty, C. V. Jawahar

The authors introduce MotoVerse, an ego-centric benchmark for predictive perception of motorized two-wheelers (MTWs) in dense, unstructured traffic. It contains 949 maneuver clips with trajectory annotations for 31,000+ surrounding road agents (16,000+ MTWs), for two tasks: ERMA (ego-rider maneuver anticipation) and TATP (traffic agent trajectory prediction).

DrivingAI summary
2026-11-05 · arXiv:2611.06061

Touch\'{e}World tech report : Towards Online Unified Camera--Hand Exploration in Generative Worlds

Jiaao Yu, Yanjian Zhang, Haoyang Jiang et al.

TouchéWorld integrates continuous hand interaction into a camera-controlled video world model for online exploration. Given an initial image and a scene-description prompt, it generates video conditioned on temporally aligned camera and hand controls, using joint camera–hand data construction, parameter-efficient hand conditioning, causal adaptation, and few-step distillation for chunk-level online generation. The authors introduce TouchéBench to evaluate hand following, camera–hand coordination, online interaction, and object response.

2026-11-05 · arXiv:2611.06093

Drift-adaptive Representation Preservation for Domain-Incremental Learning

Hyunmin Kong, Gyutae Oh, Jitae Shin

The authors address domain-incremental learning forgetting by tracking per domain–class representation drift from a fixed class anchor. Drift is measured using a repurposed JEPA context-to-target predictor without storing past images (store compact context summaries in backbone feature space). Drift-adaptive consolidation and drift-weighted robust prototype construction allocate preservation by measured drift, improving accuracy–forgetting balance across DomainNet, Office-Home, CORe50, and CDDB-Hard with low training memory and smallest storage among compared methods.

JEPAAI summary
2026-11-05 · arXiv:2611.06120

ReactWorld: Structuring Interaction Responses for Embodied Video Generation

Hanwei Liu, Huijia Zhang, Hanzhang Wang et al.

ReactWorld is an embodied video generation model that anticipates structured interaction responses in the generation backbone’s feature space, predicting participant roles, future motion response intensity, and interval motion. It uses a response feedback adapter that forms feature increments via differences from a zero-dynamics reference and converts predicted displacements into positional biases for temporal attention. The authors report improved task completion and physical plausibility on RBench and EZS-Bench.

2026-11-05 · arXiv:2611.06247

CAPE-JEPA - Cross-modal Adversarial Probabilistic Embedding JEPA

Aniket Junghare, Biplab Banerjee

CAPE-JEPA is a probabilistic cross-modal JEPA for remote sensing cross-modal image retrieval. It replaces deterministic latent prediction with a cross-attentional Gaussian predictor (per-patch mean/variance) trained by negative log-likelihood, adds adversarial cross-modal distributional alignment via worst-case Epps–Pulley tests without explicit sample pairing, and uses in-training InfoNCE with attention-pooled heads for scene-level discrimination.

JEPAAI summary
2026-11-05 · arXiv:2611.06544

MKD2World: A Typed Spatial MarkDown World Model For Web Agents

Zhihao Nan, Sifan Zhou, Yiming Cheng et al.

The authors introduce Typed Spatial Markdown (TSM), a structured UI state for desktop web interfaces that encodes UI element types, visual semantics, explicit bounding boxes, optional control states, and hierarchical membership. They build AtomBlock-WebUI to train a screenshot-to-TSM renderer. Building on TSM, MKD2World is a two-stage, action-conditioned world model that predicts next TSM states and supports digest-based action selection for web agents.

Video World ModelsCode availableAI summary
2026-11-05 · arXiv:2611.06627

Interactive World Simulation with Cosmos for Physical AI

NVIDIA, :, Anis Ahmad et al.

The authors present a shared framework that adapts Cosmos world foundation models for real-time interactive world simulation in Physical AI. It streams visual observations in response to actions and scene controls, supporting both action-conditioned visual prediction and observation generation from a maintained physical state via an external simulator. They demonstrate it for camera navigation, robot manipulation, controllable video transfer, and multi-view autonomous-vehicle simulation.

2026-11-05 · arXiv:2611.06653

Detecting Silent Action-Interface Mismatch with Calibrated Relational World Models

Rui Zhu, Xurui Li, Haixu Tang

The authors study offline detection and closed-set localization of silent action-interface mismatches (e.g., axis permutations, sign inversions, rotation unit scaling, gripper polarity). Clean held-out transitions calibrate alarms and a fixed inverse vocabulary enables localization. On ten LIBERO Spatial tasks (432 source episodes), at 2.5% matched clean FPR, recall is 46.1% (ridge), 61.0% (four-component joint GMM), and 65.7% (exact conditional). Joint scoring localizes higher (95.8% vs 94.6%).

Latent DynamicsAI summary
2026-11-05 · arXiv:2611.06768

PyRUA-REAL: Robot Exploration and Adaptive Learning through RSI in the Real World

Qiang Wang, Jianxin Bi, Shulong Jiang et al.

PyRUA-REAL is an embodied recursive self-improvement framework for real-world robot manipulation. Coding agents iteratively explore, execute, diagnose, and revise robot programs, storing interaction experience as persistent memory and reusable executable skills. The resulting programs can be frozen and executed without runtime language-model calls, while still using live observations during execution. Across eight real-world tasks, PyRUA-REAL reports 91.1% average success.

RoboticsAI summary
2026-11-05 · arXiv:2611.06817

Open-HumanWAM: An Open Recipe for Scaling Human and Robot Experience for Embodied Foundation Models

Juncheng Ma, Jiaxin Li, Jiankai Tu et al.

Open-HumanWAM proposes a fully open recipe for scaling egocentric human video pretraining up to 50,000 hours using Open-HumanWAM-50K. The authors report an empirical log-linear scaling law between pretraining hours and downstream robot success rates, then introduce HumanWAM-Vanilla (50k hours human video) and HumanWAM-Goal (joint human–robot pretraining with visual goals), reporting improved RoboTwin 2.0 and physical-robot results and open-sourcing data, pipelines, and checkpoints.

DrivingCode availableAI summary