Robotics · 2026-11-03
ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation
Peiyuan Tang, Yao Li, Xiaodong Zhang, Wuyang Zhang, Qin Xia, Yanyong Zhang, Zijiang Yang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ForceLDM is a force-aware latent dynamics model for contact-rich manipulation. It uses a privileged teacher with future force/torque and optical-flow signals to extract control-relevant future dynamics, then distills these latent representations into a student trained only on current observations. At inference, only the student is deployed. The method also uses curriculum-based progressive noise injection for robustness and generalization.
Why it matters
Contact-rich manipulation depends on how contacts and forces evolve, but vision-only policies struggle to infer contact states and interaction forces. ForceLDM aims to model future interaction dynamics without relying on future sensory inputs at deployment, using force/torque and optical flow as dynamics representations and distillation to avoid entangling control with irrelevant future observation details.
Method
- ForceLDM distills privileged future dynamics: a teacher consumes future force/torque and optical-flow tokens to produce compact future interaction representations, aligned to a student via multi-layer representation distillation.
- A future-dynamics encoder converts future 6D force/torque into force tokens and converts optical flow into compressed motion tokens for distillation and query-based student reasoning.
- Progressive noise injection perturbs the student’s future queries with a scheduled Gaussian noise scale to mitigate overfitting; inference keeps only the student policy (no future tokens).
Abstract (from arXiv)
Contact-rich manipulation requires fine-grained control and complex physical interactions with the environment, posing significant challenges for robot policy learning. Vision-only policies struggle with such tasks, as contact states and interaction forces are difficult to infer from visual observations alone. Existing methods mitigate this issue by incorporating force or tactile sensing. However, they often treat these signals as auxiliary observations without fully exploiting their role in modeling future interaction dynamics. To address this, we propose ForceLDM, a force-aware latent dynamics model that learns future contact and motion dynamics to guide action generation. Rather than directly predicting future observations, which can entangle task-relevant dynamics with irrelevant sensory details, ForceLDM learns a compact representation of future dynamics through knowledge distillation. Specifically, a teacher network with privileged future force/torque and optical flow signals extracts future dynamic features, which are then distilled into a student network trained only on current observations. This enables the student model to reason about upcoming contacts and physical dynamics without relying on future sensory inputs at deployment. To improve robustness and generalization, we further introduce a curriculum-based progressive noise injection strategy to mitigate over-reliance on future features. Experiments on five real-world contact-rich manipulation tasks demonstrate that ForceLDM outperforms state-of-the-art baselines, generalizes effectively to novel objects, and remains robust to environmental perturbations. Project page and code are available at https://tpy001.github.io/ForceLDM/.
Related papers
- ForceWAM: Decoupling Future and Response Learning from Demo and Play Data for Contact-Rich Manipulation
- Interaction Latents as Actions for Contact-Rich Manipulation
- ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction
- LaST0.5: A Generalist World Action Model with Interaction-Centric Latent Reasoning
- TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-02 · Indexed by Pub-AI: 2026-11-03 UTC
ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation
Preprint posted: 2026-11-02 · Original record ↗
- arxiv
- 2611.03294
Explicit source versions (1)
- v1 · 2026-11-02
- Preprint available
Source date: 2026-11-02 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.03294v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Contact-rich manipulation requires fine-grained control and complex physical interactions with the environment, posing significant challenges for robot policy learning. Vision-only policies struggle with such tasks, as contact states and interaction forces are difficult to infer from visual observations alone. Existing methods mitigate this issue by incorporating force or tactile sensing. However, they often treat these signals as auxiliary observations without fully exploiting their role in modeling future interaction dynamics. To address this, we propose ForceLDM, a force-aware latent dynamics model that learns future contact and motion dynamics to guide action generation. Rather than directly predicting future observations, which can entangle task-relevant dynamics with irrelevant sensory details, ForceLDM learns a compact representation of future dynamics through knowledge distillation. Specifically, a teacher network with privileged future force/torque and optical flow signals extracts future dynamic features, which are then distilled into a student network trained only on current observations. This enables the student model to reason about upcoming contacts and physical dynamics without relying on future sensory inputs at deployment. To improve robustness and generalization, we further introduce a curriculum-based progressive noise injection strategy to mitigate over-reliance on future features. Experiments on five real-world contact-rich manipulation tasks demonstrate that ForceLDM outperforms state-of-the-art baselines, generalizes effectively to novel objects, and remains robust to environmental perturbations. Project page and code are available at https://tpy001.github.io/ForceLDM/.
