Robotics · 2026-11-03

ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation

Peiyuan Tang, Yao Li, Xiaodong Zhang, Wuyang Zhang, Qin Xia, Yanyong Zhang, Zijiang Yang

arXiv:2611.03294PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

ForceLDM is a force-aware latent dynamics model for contact-rich manipulation. It uses a privileged teacher with future force/torque and optical-flow signals to extract control-relevant future dynamics, then distills these latent representations into a student trained only on current observations. At inference, only the student is deployed. The method also uses curriculum-based progressive noise injection for robustness and generalization.

Why it matters

Contact-rich manipulation depends on how contacts and forces evolve, but vision-only policies struggle to infer contact states and interaction forces. ForceLDM aims to model future interaction dynamics without relying on future sensory inputs at deployment, using force/torque and optical flow as dynamics representations and distillation to avoid entangling control with irrelevant future observation details.

Method

  • ForceLDM distills privileged future dynamics: a teacher consumes future force/torque and optical-flow tokens to produce compact future interaction representations, aligned to a student via multi-layer representation distillation.
  • A future-dynamics encoder converts future 6D force/torque into force tokens and converts optical flow into compressed motion tokens for distillation and query-based student reasoning.
  • Progressive noise injection perturbs the student’s future queries with a scheduled Gaussian noise scale to mitigate overfitting; inference keeps only the student policy (no future tokens).
Abstract (from arXiv)

Contact-rich manipulation requires fine-grained control and complex physical interactions with the environment, posing significant challenges for robot policy learning. Vision-only policies struggle with such tasks, as contact states and interaction forces are difficult to infer from visual observations alone. Existing methods mitigate this issue by incorporating force or tactile sensing. However, they often treat these signals as auxiliary observations without fully exploiting their role in modeling future interaction dynamics. To address this, we propose ForceLDM, a force-aware latent dynamics model that learns future contact and motion dynamics to guide action generation. Rather than directly predicting future observations, which can entangle task-relevant dynamics with irrelevant sensory details, ForceLDM learns a compact representation of future dynamics through knowledge distillation. Specifically, a teacher network with privileged future force/torque and optical flow signals extracts future dynamic features, which are then distilled into a student network trained only on current observations. This enables the student model to reason about upcoming contacts and physical dynamics without relying on future sensory inputs at deployment. To improve robustness and generalization, we further introduce a curriculum-based progressive noise injection strategy to mitigate over-reliance on future features. Experiments on five real-world contact-rich manipulation tasks demonstrate that ForceLDM outperforms state-of-the-art baselines, generalizes effectively to novel objects, and remains robust to environmental perturbations. Project page and code are available at https://tpy001.github.io/ForceLDM/.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-02 · Indexed by Pub-AI: 2026-11-03 UTC

preprint

ForceLDM: A Force-aware Latent Dynamics Model for Contact-Rich Manipulation

Peiyuan Tang, Yao Li, Xiaodong Zhang, Wuyang Zhang, Qin Xia, Yanyong Zhang, Zijiang Yang

Preprint posted: 2026-11-02 · Original record ↗

arxiv
2611.03294
Explicit source versions (1)
  • v1 · 2026-11-02
  1. Preprint available

    Source date: 2026-11-02 · First detected: 2026-11-03 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Contact-rich manipulation requires fine-grained control and complex physical interactions with the environment, posing significant challenges for robot policy learning. Vision-only policies struggle with such tasks, as contact states and interaction forces are difficult to infer from visual observations alone. Existing methods mitigate this issue by incorporating force or tactile sensing. However, they often treat these signals as auxiliary observations without fully exploiting their role in modeling future interaction dynamics. To address this, we propose ForceLDM, a force-aware latent dynamics model that learns future contact and motion dynamics to guide action generation. Rather than directly predicting future observations, which can entangle task-relevant dynamics with irrelevant sensory details, ForceLDM learns a compact representation of future dynamics through knowledge distillation. Specifically, a teacher network with privileged future force/torque and optical flow signals extracts future dynamic features, which are then distilled into a student network trained only on current observations. This enables the student model to reason about upcoming contacts and physical dynamics without relying on future sensory inputs at deployment. To improve robustness and generalization, we further introduce a curriculum-based progressive noise injection strategy to mitigate over-reliance on future features. Experiments on five real-world contact-rich manipulation tasks demonstrate that ForceLDM outperforms state-of-the-art baselines, generalizes effectively to novel objects, and remains robust to environmental perturbations. Project page and code are available at https://tpy001.github.io/ForceLDM/.