Digest

Reset

122 papers

2026-11-10 · arXiv:2611.11242

DeepSoMo: Deformable Object Motion Estimation under Proprioceptive Partial Observability via Predictive Latent Dynamics

Hanwen Zhao, Weicheng Dai, Luoyao Chen et al.

DeepSoMo estimates deformable object motion under proprioceptive partial observability using latent temporal consistency. It encodes internally rendered deformation images, aggregates them with an LSTM latent dynamics model, and enforces a self-supervised latent prediction constraint so latent states remain predictive of future latent features. Evaluated in high-fidelity FEM simulation, it reports consistent improvements over architecture-matched recurrent baselines for long-horizon trajectory estimation and related tasks.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.11500

ForeAct: Geometry-Guided Action Learning from World Representations

Wei Lu, Jiaxi Zhang, Yunheng Wang et al.

ForeAct repurposes a pretrained video world model as a single-pass, low-latency backbone for robotic manipulation. It processes the clean observation latent in one forward pass (no future-frame synthesis or iterative denoising) and introduces learnable geometry tokens supervised by next-frame relative depth via bidirectional self-attention. A lightweight geometry readout is used only during training and removed at deployment.

2026-11-10 · arXiv:2611.11509

Review the Episodes, Refine the Lessons: Simple Memory Evolution for Embodied Agents

Yifei Yao, Yude Zou, Xing Gao et al.

SimplEvo improves a frontier MLLM under direct control using two self-written, fixed-size text memories: a coordinate-free task memory from one demonstration and a bounded working memory refined after each training episode via single-turn trajectory reflection. No weight updates, post-trained policies, or privileged information are used, and both memories are fixed at test time. On EBench and RoboDojo, the authors report large success-rate gains for GPT-6-Astra.

DrivingAI summary
2026-11-10 · arXiv:2611.11516

Breaking survivorship bias in driving world models

Zihao Sheng, Zilin Huang, Yuhao Luo et al.

The authors identify a survivorship bias in driving world models: prevailing driving datasets mostly record futures that end safely, so existing models often imagine benign outcomes even when an imminent collision would occur. They introduce a Neutral World Model (NWM) trained on routine driving plus real accident videos with a bounding-box-guided objective, and use it for reward-based action selection. The NWM more faithfully anticipates collisions and improves trajectory selection on NAVSIM and zero-shot pedestrian dummy crossing data.

DrivingAI summary
2026-11-10 · arXiv:2611.11521

K-PWM: Control-Oriented Structured Bilinear World Models under Partial Observation

Santosh Mohan Rajkumar, Sriram Narayanan, Samuel E. Otto et al.

K-PWM is a probabilistic, control-oriented structured bilinear world model for partially observed systems with vector-valued measurements. It couples Koopman-bilinear latent dynamics with a nonlinear decoder and performs probabilistic latent-state inference via extended Kalman filtering and smoothing, without a separately trained inference network. It trains with a generalized expectation conditional maximization (GECM) procedure and uses the model for uncertainty-aware MPPI control.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.11532

DriveFoundry: Co-Evolving Data and Model for Efficient Closed-Loop Iteration in Autonomous Driving

Ziyang Yao, Yuncheng Jiang, Fang Li et al.

The authors introduce DriveFoundry, a closed-loop driving framework that couples structured causal data refinement with adaptive inference to improve iteration efficiency under data and computation budgets. It uses a Data Engine for candidate scene refinement and verified failure-guided VLM-assisted annotation, and a unified vision-language model (Xiaomi-Drive-o1) with block-wise latent refill and a value-of-computation head to allocate computation per decision.

DrivingAI summary
2026-11-10 · arXiv:2611.11554

Kimodo-Policy: Learning Kinematic Action Models for Humanoid Loco-Manipulation

Yunheng Wang, Jiaxi Zhang, Wei Lu et al.

Kimodo-Policy introduces Kinematic Action Models (KAMs), treating whole-body motion as a direct pretraining target using a pretrained motion generator as the action-generation backbone. The policy freezes a Kimodo text-to-motion backbone and adds lightweight ControlNet-based visual conditioning, with a root–body–hand architecture and motion-history conditioning for closed-loop loco-manipulation.

DrivingAI summary
2026-11-10 · arXiv:2611.11605

ScanDreamer: A Map-Grounded World Model for Quadrotor Navigation

Yunyi Gao, Yixing Shi, Yize Liu et al.

ScanDreamer is a map-grounded world model for quadrotor navigation from forward depth images. It builds an online 2D scan-memory occupancy map from depth and pose, conditions the world model/actor/critic on map crops, and grounds imagination by propagating pose over a frozen map to compute geodesic progress and wall-contact signals. It also adds a forward-cone penalty to discourage horizontal motion outside the camera viewing direction.

DrivingCode availableAI summary
2026-11-10 · arXiv:2611.11697

SurgTAP-3D: A Large-Scale Cross-Surgery Video Dataset with Contact-Aware 3D Motion Tracking

Jiaxin Guo, Wenzhen Dong, Taoyu Wu et al.

SurgTAP-3D is a large-scale in-vivo pseudo-3D surgical motion tracking dataset and benchmark for contact-aware 3D point tracking in minimally invasive surgery. It includes 520K frames, 4,549 action clips, and 1.4M 2D/3D point trajectories from diverse robotic and manual procedures under unified action categories. The authors propose a unified framework to convert uncalibrated monocular surgical videos into structured 4D annotations via contact-region prompting, active-tool segmentation, dynamic bundle adjustment, and global 4D refinement.

2026-11-10 · arXiv:2611.11780

Planners Exploit Deterministic Error: Dithered Low-Bit World Models

Yong Hwi Song, Keon Myung Lee

The authors show that round-to-nearest low-bit activation quantization can cause sampling-based planners (MPPI/CEM) to collapse even when latent one-step prediction error stays comparable to a well-planning stochastic rounding scheme. They attribute the failure to persistent, candidate-shared deterministic quantization error exploited across planner iterations. A training-free fix, shared dither redrawn every forward call, restores performance and adds small latency.

DrivingAI summary
2026-11-10 · arXiv:2611.11869

Action Representation for Spatial Generalization in Robot World Models

Yuhang Sun

The authors study how action coordinate representations affect spatial generalization in a controlled robot peeling world-model benchmark. Using matched action–state inputs across world-, object-, and surface-relative frames, they find costs from limited data when equivalent interactions use different coordinate descriptions, irreducible ambiguity when peel thickness is removed, and improved consequence prediction from surface-relative coordinates under accurate local observations—yet vulnerability remains to coordinate-frame uncertainty.

RoboticsAI summary
2026-11-10 · arXiv:2611.11887

How Many Candidates Does a One-Step World Model Need? A Random-Codebook Account of Nearest-of-m Training on Frozen DINO Tokens

Yong Hwi Song, Keon Myung Lee

The authors study one-step stochastic world models trained with nearest-of-m matching (conditional IMLE) on frozen DINOv2/v3 patch tokens, comparing candidate budget m across objectives. They report that under mode-like uncertainty, nearest-of-m with m=16 covers enumerated futures better than 50-step flow matching and other one-step generators at about 30× lower latency, and that rollouts stay on the token manifold where flow rollouts drift.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.11912

LO-World: Unified BEV Latent Modeling for Joint 4D Occupancy and LiDAR Forecasting

Zhimin Liao, Ping Wei, Kai Mao et al.

LO-World is a unified 4D world model that forecasts 3D semantic occupancy and raw LiDAR point clouds in a shared BEV latent space. It combines LO-VAE (hybrid range-voxel autoencoder) with a flow-matching generator (LO-Former) that caches historical context to reduce iterative sampling latency. The authors report occupancy IoU improvements and LiDAR Chamfer Distance reductions over prior baselines, plus real-time throughput on an RTX 4090, and release code.

DrivingCode availableAI summary
2026-11-10 · arXiv:2611.11921

An Evaluation of Tabular Foundation Models as Training-Free World Models

Xian Yeow Lee, Ryoichi Takase, Ahmed Farahat

The authors evaluate TabFM, a pretrained tabular foundation model used training-free for action-conditioned next-state prediction via in-context regression over transition pairs. On limited transitions, they report TabFM is more accurate than LSTM/MLP trained from scratch and often beats regressors like ridge/GP, especially with long context. However, short-context rollouts degrade quickly, training on full data remains substantially better, and TabFM inference is about 3 orders of magnitude slower than an LSTM.

Latent DynamicsAI summary
2026-11-10 · arXiv:2611.12014

Benchmarking language models and agents for scientific reasoning in evolving materials laboratories

Xuyu Dong, Yixuan Li, Jianguo Huang et al.

The authors introduce MatScope to benchmark scientific reasoning for evolving materials laboratories, evaluating (1) unaided condition-specific materials judgement (M1, 200 tasks), (2) agent-assisted transfer to unresolved cases (M2), and (3) sequential prediction and revisions along multistage trajectories (MatEvoBench-20). Across 11 model deployments, the authors report strong handling of given-input calculations and many physical explanations, but weaker fine-grained quantitative inference and robustness as evidence accumulates.

AI summary
2026-11-10 · arXiv:2611.12102

SPARK: Simple Post-training for Adapting pRetrained Knowledge to Robot Control

Taekyung Kim, Jeongeun Park, Byeongho Heo et al.

SPARK is a post-training method that adapts pretrained vision/foundation models for robot control. It uses temporal abstraction to encourage compact features for temporally offset scene prediction and knowledge preservation to align patch-level representations with the original foundation model, aiming to improve success rates and generalization on robot manipulation tasks and transfer to real-world robot manipulation.

DrivingAI summary
2026-11-10 · arXiv:2611.12137

Good Views Are Not Enough: Global Counterexamples for Gaussian-Regularized JEPAs

Anran Xu

The authors show that Gaussian-regularized JEPA objectives can have global optima that do not identify a scene up to rotation. They give globally certified counterexamples where Gaussian marginal matching rewards retaining independently resampled observation variation, and this can outweigh its prediction cost, forcing nuisance retention and allowing equal-value minimizers outside a single rotation orbit.

JEPAAI summary
2026-11-10 · arXiv:2611.12200

Learning to Act with World Models by Valuing What Comes Next

Chenghao Gao, Zhi Zhang, Dian Jin et al.

WM-PPO combines PPO’s value-based success estimation with a world model’s predicted next states. For each candidate action (executed or not), the world model predicts the next observation and the PPO critic values the predicted successor. Comparisons yield auxiliary policy advantages; the mean transition value is a critic target. A shared gate calibrates both channels using real branch outcomes.

DrivingAI summary
2026-11-10 · arXiv:2611.12319

Mitigating SM-Level Load Imbalance in 3D Gaussian Splatting on GPUs

Junha Ryu, Hoi-Jun Yoo

The authors identify SM-level load imbalance in GPU-based 3D Gaussian Splatting (3DGS) alpha blending caused by uneven per-tile workloads, which creates a long execution tail. They propose software scheduling: persistent-kernel LPT dispatch of heavy tiles and selective depth-wise chunking with saturation signaling. A lightweight per-frame policy decides chunking. They report alpha blending speedups up to 2.93× averaged on a large-scale urban scene and up to 5.6× on the most imbalanced frames.

2026-11-10 · arXiv:2611.12324

How Many Exposures Does a World Model Need? Learning Unseen Game Mechanics at Test Time

Ku Onoda, Haruto Kawano, Shota Takashiro et al.

The authors study how many “exposures” (occurrences) a pretrained world model needs at test time to learn unseen game mechanics. They introduce MechEval to label 69 recurring mechanics across 20 ARC-AGI-3 games and score per-mechanic predictions by exposure count. Their testbed WARC updates all parameters after each transition; WARC-Agent then improves closed-loop efficiency by using WARC for action ranking and test-time training.

Latent DynamicsAI summary