Digest

Reset

1180 papers

2026-10-05 · arXiv:2610.02840

PointWAM: 3D World Action Modeling for Dexterous Robotic Manipulation

Chunghyun Park, Beomjun Kim, Seungcheol Park et al.

PointWAM is a 3D world action model for dexterous manipulation. It decomposes the world into scene points and hand keypoints, forecasts both as 3D point trajectories in a shared space-time coordinate frame, and retargets the forecast hand motion to robot actions. Pre-training on 1.15M human demo episodes improves DexJoCo success by 56.9 points; adding scene-trajectory supervision adds 10.9 points.

Latent DynamicsAI summary
2026-10-05 · arXiv:2610.02860

Counterfactual Action Evaluation, Observation Bottlenecks, and Representation Geometry in Joint-Embedding Predictive World Models

Arjun Subramanian

The authors introduce a counterfactual action evaluation protocol for joint-embedding predictive world models that traces the same intervention through simulator state, raster observations, target embeddings, and predictor outputs. In a 2D deformable-physics testbed, actions produce real particle changes but many one-step rasters are identical, and the predictor shows weak response to visible counterfactuals, even though matched state perturbations strongly change predictions.

JEPAAI summary
2026-10-05 · arXiv:2610.02864

NeuroLens: Learning Latent Embeddings of Neural Semantics from Chronic Recordings

Hanrui Lyu, Baiyuan Chen, Tianshu Tan et al.

NeuroLens is a self-supervised JEPA-style model for chronic neural recordings that learns denoised, semantically informative latent embeddings. A day-adaptive cross-attention encoder maps changing neuron populations into a common latent space using calibration-based neuron identification, and a shared causal transformer predicts future latent states. Predicting in latent space reduces sensitivity to recording instability and improves decoding over time.

JEPAAI summary
2026-10-05 · arXiv:2610.02874

SceneFactory-3D: Lifting 2D Traffic Scenes into 3D Physical Counterfactuals for Scalable Physically Grounded Safety Evaluation

Yicheng Zhu, Linfeng Tian, Tianmu Zhao et al.

SceneFactory-3D is a GPU-batched, physics-grounded multi-agent driving simulator that executes acceleration/steering via per-wheel suspension- and friction-limited forces on 3D heightfields with rigid contact. It enables matched physical counterfactuals by keeping scenario and controllers fixed while varying road conditions across parallel worlds. The authors evaluate controller sensitivity to 21 friction/grade conditions, finding reduced friction strongly degrades task outcomes and increases near-collisions.

DrivingCode availableAI summary
2026-10-05 · arXiv:2610.02957

Understanding Trajectory Heterogeneity in Federated World Model Learning

Yipan Wei, Zhaokun Yan, Ziming Hong et al.

The authors benchmark cross-time, action-conditioned clinical world-model learning under federated learning where clients own different time segments of the same patient trajectory. Using eight MIMIC-IV disease cohorts and severity-based client ownership with patient-separated construction, they analyze how ownership and participation limit long-horizon training context and how federated algorithms respond across horizons and partitions.

DrivingAI summary
2026-10-05 · arXiv:2610.03065

Learning Transferable Policies from Action-free Time Series Through Dynamical Embeddings

Niklas Emonds, Georgia Koppe

The authors propose a hierarchical model-based RL framework for learning transferable control policies from action-free time series. A hierarchical dynamical system reconstruction (ALRNN/DSR) learns shared dynamics and per-system embeddings from uncontrolled recordings. These embeddings condition shared SAC policy/value networks, while interventions are modeled as additive latent perturbations with decoder-based constraints for interpretability and modality-protected control.

Latent DynamicsAI summary
2026-10-05 · arXiv:2610.03079

RIFAR: Reliability and Forgetting-Aware Replay for Continual Robot Learning

Zirong Song, Zheng Lu, Haoran Liao et al.

RIFAR is a continual robot learning method for world–action models that reduces forgetting without storing full demonstrations. It keeps compact initial-state anchors (K=20, Hs=16) and generates candidate rollouts, then uses a frozen inverse-dynamics model (IDM) to screen action–visual consistency. It trains in two phases: initial adaptation with screened replay, then drift-aware reselection using action prediction drift before vs. after adaptation. The authors report improved retention vs. REGEN and strong memory efficiency vs. ER.

RoboticsAI summary
2026-10-05 · arXiv:2610.03092

ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe

Weihan Li, Tianshi Zheng, Yangqiu Song et al.

Scientific discovery often begins when scattered clues call for a new way of describing the world. Such abductive exploration can require constructing the representation in which an explanation is stated, when the world is epistemically open, and composing evidence scattered acro…

Source abstract
2026-10-05 · arXiv:2610.03106

S2S-JEPA: Predicting the Predictable at Subseasonal-to-Seasonal Timescales

Chenyu Dong, Gianmarco Mengaldo

S2S-JEPA reformulates subseasonal-to-seasonal forecasting (about 2 weeks to 2 months ahead) as latent-space prediction using a JEPA-style joint-embedding objective to focus on slowly varying, predictable structure. The authors report competitive deterministic skill with ECMWF at weeks 5–6 and probabilistic skill improvements for temperature (and precipitation at longer lead times), while training and running on a single GPU.

Latent DynamicsAI summary
2026-10-05 · arXiv:2610.03137

Keeping JEPA World Models Plannable When Little of the Frame Moves

Florian Strohm, Patrick Wagner, Jannik Schwab et al.

The authors build SLIM, a pushing benchmark with paired visual and language goals on identical scenes where little of the frame responds to actions. They find a vanilla JEPA latent world model (LeWM) largely fails (0.003 success) because its encoder latent is nearly action-insensitive. An inverse-dynamics auxiliary loss (LeWM+I) repairs action-relevant latent dynamics and enables language-goal planning via a small head.

JEPAAI summary
2026-10-05 · arXiv:2610.03154

Does Physics Live in the Activations? Localizing Physical Quantities in Video Diffusion Models

Jonas Kneifl, Jakub Skalski, Bart{\l}omiej Twardowski et al.

The authors probe internal representations of video Diffusion Transformers (DiTs) using simulator-derived physical quantities (kinematics and rigid-body dynamics). They find that physical quantities are linearly decodable from DiT activations with high accuracy early in denoising, outperforming probes on the model’s own noised latents. The information is sharply localized to tokens near objects (ROI) yet aggregates globally over time, and probing directions can steer generation.

2026-10-05 · arXiv:2610.03356

ReFract: Benchmarking Perspective Awareness in Language Model Agents with Text World Models

Hainiu Xu, V\'{i}tor N. Louren\c{c}o, Mohnish Dubey et al.

ReFract is a benchmark (150 expert-validated entries) for “perspective awareness” in LLM language model agents: agents must take role-calibrated actions and tool-calling trajectories that respect both capability and knowledge boundaries. Using text world models that gate tools by role, the authors report that state-of-the-art LLMs solve at most 69% of tasks and more than 50% of trajectories contain perspective-violating actions.

DrivingAI summary
2026-10-05 · arXiv:2610.03374

EVEWorld: Physical Evolution Supervision for Embodied World Models

Kaiqi Wang, Songxin Zhang, Zejian Xie et al.

EVEWorld mitigates “Model Laziness” in embodied world models by adding physical evolution supervision. It combines Instance-Guided Restoration (IGR) for instance consistency via count-edited restoration training and Temporal Instance Alignment (TIA) for cross-frame consistency by aligning target instances across adjacent frames. The authors report large MLR reductions vs GigaWorld-0 and strong WorldArena 2.0 Track 1 leaderboard placement.

2026-10-05 · arXiv:2610.03391

Native Action-Prior Learning from Videos for World Action Models

Zhaochong An, Fei Zhang, Menglin Jia et al.

NAVA-WAM pretrains an action policy directly from observation-only videos via “native action-prior learning,” avoiding intermediate representation-to-control transfer or a separate latent-action model. The Action-DiT is optimized during pre-training only with future-video flow matching using transition-structured joint attention, then post-trained on action-labeled demonstrations with video–action flow matching and asymmetric attention for action-only inference. The authors report consistent gains in simulation and strong real-robot transfer on a Franka FR3.

DrivingAI summary
2026-10-05 · arXiv:2610.03516

XGenAct: Geometry-Enhanced World Action Models through Cross-Task Generation

Tingting Du, Ziyao Wang, Guoheng Sun et al.

XGenAct is a world action model that generates RGB, robot actions, metric depth, surface normals, and functional role segmentation as RGB video streams using deterministic codecs. Training samples pair one perception modality with action, while a single video diffusion transformer and objective handle all tasks without modality-specific learned heads. On held-out RLBench, it achieves 52% average closed-loop success vs 26% for the strongest evaluated baselines.

DrivingAI summary
2026-10-05 · arXiv:2610.03587

AVL-JEPA: Preventing Causal Dynamics Information Collapse In Joint Embedding Predictive Architecture World Models

Yikang Qiao, Ling Zhang, Ziying Song et al.

The authors report AVL, an action-grounded vision-invariance latent method for JEPA world models that targets “causal dynamics information collapse.” They use executed actions as an auxiliary anchor and align clean vs. perturbed latent transitions while freezing the action head. On four robotic control tasks (TwoRoom, PushT, OGBench Cube, Reacher), AVL improves success under visual perturbations while preserving clean performance.

RoboticsAI summary
2026-10-05 · arXiv:2610.03604

Mastering Atari 2600 Games with Discovered Options

Erik M. Lintunen, Marlos C. Machado

Wayfarer is a domain-agnostic, online deep RL agent that discovers options from high-dimensional observations using Laplacian representation learning. It couples representation learning, option discovery, and task learning in a single off-policy loop from a single stream of experience. The authors report substantially faster learning and state-of-the-art performance among single-stream agents on the most challenging Atari 2600 games.

Latent DynamicsAI summary
2026-10-05 · arXiv:2610.03607

World Action Learning via Interaction-Centric Spectral Latent Guidance

Zhiming Liu, Yikun Miao, Ying Chen et al.

WING transfers interaction knowledge from egocentric videos to robot policies by learning interaction-centric latent actions while suppressing observer motion, then extracting slowly varying low-frequency components in the spectral domain as guidance for action generation. The authors report average success rates of 99.20% on LIBERO, 93.80% on RoboTwin 2.0, and 57.7% on RoboCasa–GR1, plus results on four real-world manipulation tasks under diverse generalization settings.

RoboticsAI summary
2026-10-05 · arXiv:2610.03632

World Embedding Benchmark

Yiqi Liu, Ruifeng Yuan, Yang Wang et al.

Physical fidelity has received increasing attention in world models and video generation, yet how video representations encode physical information remains less understood. We introduce the World Embedding Benchmark, comprising 8,000 controlled simulation cases from 80 families s…

Source abstract
2026-10-05 · arXiv:2610.03713

What Should World Models Forget? Stratified Retention for Continual Adaptation

Nishit Anand, Ramani Duraiswami, Dinesh Manocha

The authors argue that continual world models must not treat any performance drop on old data as forgetting, because environments change and correct revision can look like forgetting. They propose retention stratified by invariance timescale: invariants should be refined but never degraded, while instance-level facts should be revised after changes. They introduce differential retention to measure invariant retention and revision latency separately.

Latent DynamicsAI summary