Latent Dynamics · 2026-10-14
Grid or Nodes? Probing Object Memory in World Models Under Partial Observability
Justin McMillen, Yasin Yilmaz
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors compare FloWM’s grid memory with a node-based Kalman tracker for object memory under partial observability in a synthetic block world. With an oracle-fed tracker, node and grid can match on FloWM’s standard motion prior (89% vs 91% out of view). But when motion is outside the prior, both degrade, and the grid’s built-in design requires many grid copies (25) to cover varied velocities.
Why it matters
The paper tests whether object memory in world models should be stored on a spatial grid with built-in motion priors or as per-object nodes. It also shows that probe choice and model “write-back” can affect measured in-view scores, and it quantifies what happens when the motion prior is wrong (object motion outside the allowed speeds).
Method
- Compare FloWM’s top-down grid memory (5 copies tied to built-in velocities) vs a node memory: an oracle-fed per-block Kalman tracker.
- Use FloWM’s own Dynamic Block World setup (70 context frames, 70 prediction actions with no frames) and an out-of-view object-accuracy probe based on reading memory states.
- Test standard vs varied motion (diagonal and speed-2) and analyze effects of probe type, speed snapping, long gaps, and bounce handling.
Limitation
The authors state limitations: “We used one synthetic environment and one released FloWM model. The tracker’s inputs are oracles.” They also note “We froze the tracker’s settings for the standard setting after seeing some test-set results,” and that probe differences make it hard to isolate effects.
Abstract (from arXiv)
A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities. A node memory instead keeps each object's position and velocity as numbers. We compare them in FloWM's own block world, with an oracle-fed Kalman tracker as the node memory. How the grid is read changes its score: a single-cell probe finds 66% of out-of-view blocks, while FloWM's own probe 89%. FloWM also scores 8 points higher in view although no frame is shown, which we attribute to its write-back of its own predictions. Out of view, the tracker reaches FloWM's range in 14KB against 5.6MB, with no probe. On diagonal motion, outside its five velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged tracker keeps 93%. Faster blocks defeat both until one more allowed speed takes the tracker to 94%, where FloWM's built-in shifts would need 25 copies of its grid. A learned node memory must still replace the oracles and handle velocity under noise, long gaps, and the bounce.
Related papers
- How Many Candidates Does a One-Step World Model Need? A Random-Codebook Account of Nearest-of-m Training on Frozen DINO Tokens
- Roz: A Taste Test for World Model Latents
- UniFlowWAM: A Unified World-Action Model with Visual Semantics and 4D Geometry
- RealtimeWAM: How Fast Can I Run My World Action Model?
- $\Delta$ Flow Matching for Robot Foundation Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC
Grid or Nodes? Probing Object Memory in World Models Under Partial Observability
Preprint posted: 2026-10-13 · Original record ↗
- arxiv
- 2610.18637
Explicit source versions (1)
- v1 · 2026-10-13
- Preprint available
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities. A node memory instead keeps each object's position and velocity as numbers. We compare them in FloWM's own block world, with an oracle-fed Kalman tracker as the node memory. How the grid is read changes its score: a single-cell probe finds 66% of out-of-view blocks, while FloWM's own probe 89%. FloWM also scores 8 points higher in view although no frame is shown, which we attribute to its write-back of its own predictions. Out of view, the tracker reaches FloWM's range in 14KB against 5.6MB, with no probe. On diagonal motion, outside its five velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged tracker keeps 93%. Faster blocks defeat both until one more allowed speed takes the tracker to 94%, where FloWM's built-in shifts would need 25 copies of its grid. A learned node memory must still replace the oracles and handle velocity under noise, long gaps, and the bounce.
