Latent Dynamics · 2026-11-10
Action Transfer Across Character Body Plans in Latent-Action World Models: An Empirical Study of Animated Characters
Amarnath R
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study whether latent-action world models can transfer an action learned from one 2D animated character body plan to a different body plan so that the target performs the action in its own way. They introduce an action-specific transfer score and evaluate five training objectives on a synthetic 2D benchmark (six body plans), Sprites, and 431 DeformingThings4D animal/human animations. They find that fixed-camera “transfer” can be high even when the action is ignored; body-centered transfer reaches only 0.46 for unseen animals and 0.20 for humans.
Why it matters
Latent-action models aim to learn and reuse action without labels, but action transfer can fail by encoding camera/screen motion or other shortcuts. By separating “where” vs “how” and adding a transfer score that compares the correct action to wrong ones from the same source, the paper provides an evaluation approach for cross-body generalization across very different 2D body structures (including limbless and disjointed).
Method
- Learn latent actions via an inverse model (infers action from two frames) and a forward model (predicts a later frame from earlier frame + inferred action), following LAPO/AdaWorld setup.
- Measure three aspects separately: use a fixed camera + a baseline that copies only screen movement to test “where,” and a body-centered camera to test “how.”
- Replace the usual transfer gain with an action-specific transfer score that compares prediction error using the correct transferred action vs wrong actions from the same source.
Limitation
The authors state their best model and results use small models at 64×64 pixels with the same training settings; they note that larger models trained on more varied video may transfer better and this needs to be tested before applying results to large world models.
Abstract (from arXiv)
Latent-action world models learn actions from video without labels, and recent methods aim to share these actions across bodies. Transfer between 2D characters with different body structures remains underexplored. In this paper, we study whether an action learned from one character can make a character with a different body perform the same action. A cat, a blob and a human can all jump, but each moves its body in its own way. The usual transfer gain mixes body motion with screen movement and can assign a perfect score to a model that ignores its action. To address this, we introduce a transfer score that compares the correct action with wrong ones from the same source. We test five training objectives on a new synthetic benchmark of 2D cartoon characters with six body plans, the standard Sprites benchmark, and 431 selected animal and human animations from DeformingThings4D. With a fixed camera, a baseline that only copies the source's screen movement scores 0.71, as high as most learned models. In contrast, with a camera that follows the body, the best model reaches only 0.46 for an unseen animal family and 0.20 for humans. With this camera, a linear probe identifies the action class of unseen animal families with 66% to 86% balanced accuracy (chance 17%). The learned actions therefore retain information about which action is performed, but the correct action closes only part of the prediction error gap for a new character.
Related papers
- DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training
- Learning Transferable World-Action Models from Task-Paired Human Videos
- DreamX: Video Diffusion Model is a Universal Latent Action Learner
- RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions
- Dream4ACT: A Shared Visual Action Interface for Multi-Embodiment Video-Action Modeling
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC
Action Transfer Across Character Body Plans in Latent-Action World Models: An Empirical Study of Animated Characters
First announced by arXiv: 2026-11-10 · Original record ↗
- arxiv
- 2611.10523
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.10523v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Latent-action world models learn actions from video without labels, and recent methods aim to share these actions across bodies. Transfer between 2D characters with different body structures remains underexplored. In this paper, we study whether an action learned from one character can make a character with a different body perform the same action. A cat, a blob and a human can all jump, but each moves its body in its own way. The usual transfer gain mixes body motion with screen movement and can assign a perfect score to a model that ignores its action. To address this, we introduce a transfer score that compares the correct action with wrong ones from the same source. We test five training objectives on a new synthetic benchmark of 2D cartoon characters with six body plans, the standard Sprites benchmark, and 431 selected animal and human animations from DeformingThings4D. With a fixed camera, a baseline that only copies the source's screen movement scores 0.71, as high as most learned models. In contrast, with a camera that follows the body, the best model reaches only 0.46 for an unseen animal family and 0.20 for humans. With this camera, a linear probe identifies the action class of unseen animal families with 66% to 86% balanced accuracy (chance 17%). The learned actions therefore retain information about which action is performed, but the correct action closes only part of the prediction error gap for a new character.
