Latent Dynamics · 2026-10-14

Grid or Nodes? Probing Object Memory in World Models Under Partial Observability

Justin McMillen, Yasin Yilmaz

arXiv:2610.18637PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors compare FloWM’s grid memory with a node-based Kalman tracker for object memory under partial observability in a synthetic block world. With an oracle-fed tracker, node and grid can match on FloWM’s standard motion prior (89% vs 91% out of view). But when motion is outside the prior, both degrade, and the grid’s built-in design requires many grid copies (25) to cover varied velocities.

Why it matters

The paper tests whether object memory in world models should be stored on a spatial grid with built-in motion priors or as per-object nodes. It also shows that probe choice and model “write-back” can affect measured in-view scores, and it quantifies what happens when the motion prior is wrong (object motion outside the allowed speeds).

Method

  • Compare FloWM’s top-down grid memory (5 copies tied to built-in velocities) vs a node memory: an oracle-fed per-block Kalman tracker.
  • Use FloWM’s own Dynamic Block World setup (70 context frames, 70 prediction actions with no frames) and an out-of-view object-accuracy probe based on reading memory states.
  • Test standard vs varied motion (diagonal and speed-2) and analyze effects of probe type, speed snapping, long gaps, and bounce handling.

Limitation

The authors state limitations: “We used one synthetic environment and one released FloWM model. The tracker’s inputs are oracles.” They also note “We froze the tracker’s settings for the standard setting after seeing some test-set results,” and that probe differences make it hard to isolate effects.

Abstract (from arXiv)

A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities. A node memory instead keeps each object's position and velocity as numbers. We compare them in FloWM's own block world, with an oracle-fed Kalman tracker as the node memory. How the grid is read changes its score: a single-cell probe finds 66% of out-of-view blocks, while FloWM's own probe 89%. FloWM also scores 8 points higher in view although no frame is shown, which we attribute to its write-back of its own predictions. Out of view, the tracker reaches FloWM's range in 14KB against 5.6MB, with no probe. On diagonal motion, outside its five velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged tracker keeps 93%. Faster blocks defeat both until one more allowed speed takes the tracker to 94%, where FloWM's built-in shifts would need 25 copies of its grid. A learned node memory must still replace the oracles and handle velocity under noise, long gaps, and the bounce.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC

preprint

Grid or Nodes? Probing Object Memory in World Models Under Partial Observability

Justin McMillen, Yasin Yilmaz

Preprint posted: 2026-10-13 · Original record ↗

arxiv
2610.18637
Explicit source versions (1)
  • v1 · 2026-10-13
  1. Preprint available

    Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities. A node memory instead keeps each object's position and velocity as numbers. We compare them in FloWM's own block world, with an oracle-fed Kalman tracker as the node memory. How the grid is read changes its score: a single-cell probe finds 66% of out-of-view blocks, while FloWM's own probe 89%. FloWM also scores 8 points higher in view although no frame is shown, which we attribute to its write-back of its own predictions. Out of view, the tracker reaches FloWM's range in 14KB against 5.6MB, with no probe. On diagonal motion, outside its five velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged tracker keeps 93%. Faster blocks defeat both until one more allowed speed takes the tracker to 94%, where FloWM's built-in shifts would need 25 copies of its grid. A learned node memory must still replace the oracles and handle velocity under noise, long gaps, and the bounce.