Digest

Reset

1180 papers

2026-10-02 · arXiv:2610.01942

Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models

Efstathios Karypidis, Spyros Gidaris, Nikos Komodakis

Latent-Foresight is an end-to-end framework that jointly learns a latent tokenizer and a flow-based generative dynamics model for forecasting in VFM feature space. It avoids two-stage pipelines by shaping the latent space for temporal predictability, using design choices to prevent latent collapse (stop-gradient targets/noisy inputs, latent normalization, auxiliary reconstruction on denoised latents, and noise scheduling). The authors report consistent gains over prior two-stage baselines across multiple tasks and horizons.

Latent DynamicsCode availableAI summary
2026-10-02 · arXiv:2610.01947

Latent JEPA: Abstract Future Prediction for Latent Reasoning in Chemistry

Xinjian Zhao, Yaoyao Xu, Xuemin Chen et al.

Latent-JEPA trains continuous latent reasoning in chemistry to anticipate abstract future solution aspects in embedding space, without verbalizing intermediate steps. The authors combine autoregressive learning with joint-embedding prediction of one or more future views (textual continuation and molecular outcomes). On ChemCoTBench they report gains in molecular optimization and several editing and reaction metrics, with representation analyses showing improved information about molecular outcomes and stronger correspondence with chemical structure.

JEPAAI summary
2026-10-02 · arXiv:2610.01959

Training-Free Diffusion Planning with Analytical Local Scores

Michael Y. Fatemi, Jinhao Liang, Ferdinando Fioretto

The authors propose TFDP, a training-free diffusion-based motion planner. Instead of learning a global neural score from large feasible trajectory datasets, TFDP builds an analytical surrogate score from local smoothness, obstacle avoidance, inter-agent separation, and velocity-feasibility terms. It refines a coarse trajectory using an iterative denoising procedure with a decreasing noise schedule, scaling to large multi-agent scenes.

DrivingAI summary
2026-10-02 · arXiv:2610.02054

UniWAM: Unified World-Action Model

Jiayi Chen, Wenxuan Song, Jingbo Wang et al.

UniWAM is a unified world-action model that integrates a VLM-based physical reasoner, a VGM-based world generator, and an action predictor in a Mixture-of-Transformers (MoT) architecture. It uses robot, human egocentric, and VQA data with a data cleaning/annotation pipeline, plus pre-training/post-training recipes (natural-language action supervision, future visual noise augmentation, history-conditioned flow matching) to reduce denoising steps while maintaining performance and claims SOTA across multiple evaluations.

RoboticsAI summary
2026-10-02 · arXiv:2610.02120

SkeleWAM: Skeleton World-Action Modeling for Efficient Robotic Manipulation

Juyi Sheng, Hua Wang, Mengyuan Liu

SkeleWAM is a compact world action model for robotic manipulation that represents each scene as a sparse 3D skeleton of robot joints, object centers, and interaction points built online from RGB-D and proprioception. It jointly learns action generation and future skeleton prediction with flow matching; the future branch is omitted at inference. On LIBERO-Plus it reports 85.9% overall success with 57.1M parameters, outperforming Cosmos-Policy by 3.7 points, and reports robustness under camera perturbations.

DrivingAI summary
2026-10-02 · arXiv:2610.02160

4Director: Controlling Video World Models with Rigid 3D Geometry

Wei Cao, Hao Zhang, Vikram Voleti et al.

4Director is a video world model for controlling camera and objects using an explicit 4D scene representation. Each object is reconstructed once as a complete canonical mesh and moved by one prescribed rigid transformation per frame in a shared coordinate frame with the camera. A Motion Adapter injects a depth-rendered rigid scaffold into a pretrained video generator to synthesize view-consistent appearance, illumination, and non-rigid dynamics. The paper introduces RealCOD-Rigid (20,774 clips) and Identity-Gated IoU (IG-IoU).

2026-10-02 · arXiv:2610.02162

World Observer: Joint Actor-Observer Generation for Persistent World Modeling

Hyunwook Choi, Dahyun Chung, Hyunsung Kim et al.

World Observer decouples observing from acting in video world modeling by jointly generating an agent-centric perspective actor plus one or more panoramic observers that continuously maintain out-of-view regions. Shared world state plus geometric warping from a common panorama and an Observer Sink (high-res perspective references) preserve both evolving dynamics and fine appearance when objects re-enter the actor view.

2026-10-02 · arXiv:2610.02205

ROWBench: Do Video Models Render What the Program Specifies?

Zheng-Hui Huang, Guixu Lin, Yu-Ju Tsai et al.

The authors introduce PROWBench, a benchmark for evaluating whether programmable video world models visually realize fine-grained, program-specified world events. PROWBench builds 170 programmatically constructed episodes (600 proxy videos) with logged entity states and timestamped engine events (including off-camera), and uses replayable world records to render synchronized views/proxies. It evaluates entity control, long-horizon memory, and introduces Logic-Render Alignment (timeline) and Interaction Success Rate (event execution).

Video World ModelsCode availableAI summary
2026-10-01 · arXiv:2609.38193

EHR2Trace: Auditable EHR Data Infrastructure for Patient World Models and Clinical Agents

Xinye Yang, Yuli Wang, Cheng Ting Lin et al.

EHR2Trace is an auditable EHR conversion pipeline that turns heterogeneous EHR sources into source-linked patient events for OMOP and MEDS exports. It separates event time from information availability, distinguishes medication orders vs dispensing vs administration, and runs automated validation (55 checks). Across three clinical datasets it converted 846.4M events, detected all 28 injected faults, and produced reproducible byte-identical artifacts across build configurations.

DrivingAI summary
2026-10-01 · arXiv:2609.38278

Masked Swingers: Harnessing Data Augmentation to Advance Autoencoders for Self-Supervised Learning

Anthony Fuller, Scott C. Lowe, Daniel G. Kyrollos et al.

Masked Swingers is a modified MAE that makes two independently augmented views, encodes each view separately, then exchanges the decoder’s CLS token between views before reconstructing masked patches. The authors report consistent gains over MAE on ImageNet-1K kNN and large gains on fine-grained tasks, plus a −64% relative error reduction on three state-probing datasets for preliminary world modeling.

Latent DynamicsAI summary
2026-10-01 · arXiv:2609.38383

Learning to Plan from Random Exploration

Deqian Kong, Guangyan Sun, Sheng Cheng et al.

The authors learn multi-horizon temporal relations from random exploration without action or reward labels, using a conditional energy-based model trained with noise-contrastive estimation on observation pairs. A separate one-step local dynamics model predicts candidate action outcomes, and a planner scores progress toward a goal across horizons using learned temporal log-density ratios. They demonstrate long-range planning in mazes from both states and images, and further egocentric navigation and manipulation planning.

DrivingAI summary
2026-10-01 · arXiv:2609.38444

Audible World Models: Spatially Aware Sound Generation for 3D Worlds

Duowen Chen, Jinjin He, Gouthaman KV et al.

Audible World Models is a training-free pipeline that turns a text prompt into an “audible world” state (semantic sound labels, dry audio assets, 3D source placements, source attributes, scene geometry, acoustic parameters). It then renders listener-dependent binaural audio via geometric acoustic propagation so sound sources remain persistent while audio changes with viewpoint and motion.

2026-10-01 · arXiv:2609.38504

An Empirical Study of Architectural Shift from Traditional to AI-Enabled Simulink Controllers

Hadiza Umar Yusuf, Khouloud Gaaloul

The authors report an empirical study comparing traditional vs AI-enabled Simulink controllers using a literature-derived taxonomy of 10 structural categories and 9 functional roles, analyzing 62 real-world models and surveying 13 practitioners. They identify three architectural tensions: subsystem organization dominates (68–72% footprint), AI-enabled controllers use discrete dynamics and user-defined abstraction not present in AI literature, and constraint enforcement blocks largely disappear in AI-enabled models, reducing traceability.

AI summary
2026-10-01 · arXiv:2609.38562

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation

Byoungwoo Park, Jaemoo Choi, Juho Lee et al.

LongTake trains an autoregressive video diffusion model with Long-Horizon Teacher Forcing (TF) on curated real long videos. Long-Horizon TF supervises later frames using long ground-truth prefixes, strengthening direct initialization for self-rollout distribution matching distillation (DMD) under the same 5s DMD setup. A second-stage Hybrid DMD reuses the Long-Horizon AR teacher to conditionally supervise later frames, achieving the highest dynamic degree at 30s and 60s while maintaining strong quality.

2026-10-01 · arXiv:2609.38598

Reinforcement Learning with Complex (valued) Memories

Sathya Kamesh Bhethanabhotla, Efstratios Gavves, Andr\'e Biedenkapp

The authors revisit unitary recurrent networks (uRNNs), using complex-valued, norm-preserving unitary dynamics as a drop-in recurrent cell for recurrent PPO. They propose three uRNN variants and show significant gains over GRU-PPO and PPO-LD on partially observable memory-improvable tasks, with reported up to 2–3× reward on RockSample and Craftax. Code is released at github.com/Sathya98/qurl.

DrivingCode availableAI summary
2026-10-01 · arXiv:2609.38673

In-Distribution Imagination for Model-Based Offline Reinforcement Learning

Mintae Kim, Koushil Sreenath

The authors propose in-distribution imagination (IDI) for model-based offline RL. IDI measures trajectory support in a learned latent representation space using KNN over offline trajectory windows, and adaptively truncates imagined rollouts when they leave the offline trajectory manifold. Combined with trajectory-regularized actor-critic (TRAC), IDI consistently improves limited-data performance and shows trajectory support predicts rollout failure better than transition-level uncertainty.

DrivingAI summary
2026-10-01 · arXiv:2609.38733

Code to Control: Synthesizing Parameterized Reactive Controllers

Zergham Ahmed, Joshua B. Tenenbaum, Chris Bates et al.

Code to Control synthesizes Python reactive controllers by separating program structure from numerical parameters. An LLM proposes the controller structure; derivative-free search fits parameters from environment returns. After learning, controllers need no LLM inference or planning at decision time, enabling real-time action selection. The authors report better results than planning-based program synthesis on Atari, competitive results with deep RL with fewer interactions, robust transfer in Flappy Bird, and strong MuJoCo returns versus planning baselines.

DrivingCode availableAI summary
2026-10-01 · arXiv:2609.38839

FrameMorrow: Future-guided Frame Selection with Prospective Tokens for Long-Horizon Video Generation

Bo Yin, Xiaobin Hu, Jiaqi Zhao et al.

FrameMorrow selects historical frames for long-horizon video generation by predicting prospective tokens that represent future information needs. The selector outputs explicit historical frames (not internal states), enabling plug-and-play use across diverse generators, including closed-source models, with little additional inference cost. Across five benchmarks and 11 generative models, the authors report improved long-range consistency, visual quality, and action alignment.

2026-10-01 · arXiv:2609.38926

PrecipJEPA: JEPA-Regularized Future-State Prediction with Motion-Source Rendering for Precipitation Nowcasting

Yufeng Zhu, Dan Niu, Qiliang Wu et al.

PrecipJEPA couples a structured forecasting path with a history-only JEPA auxiliary path for long-term precipitation nowcasting. An online encoder produces spatiotemporal tokens; TFP constructs lead- and location-specific future states; PMSR renders future frames via motion and source–sink fields. During joint training, H-JEPA masks history and predicts latent targets to directly supervise the shared encoder.

JEPAAI summary
2026-10-01 · arXiv:2609.38927

World-as-Graph: Relational World Modeling Through Latent Space Graphs

Yaqi Yang, Shuo Huang, Yujin Huang et al.

World-as-Graph (WaG) is an object-centric latent world model that adds relational inductive bias to JEPA-style predictive representation learning. It builds time-varying latent graphs from object slots, uses relation-aware masking policies, and maintains object-level dynamic memory via a temporal GNN transition encoder plus a memory-driven autoregressive future predictor for future latent state prediction.

JEPACode availableAI summary