Latent Dynamics · 2026-11-10
DeepSoMo: Deformable Object Motion Estimation under Proprioceptive Partial Observability via Predictive Latent Dynamics
Hanwen Zhao, Weicheng Dai, Luoyao Chen, Chen Feng
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
DeepSoMo estimates deformable object motion under proprioceptive partial observability using latent temporal consistency. It encodes internally rendered deformation images, aggregates them with an LSTM latent dynamics model, and enforces a self-supervised latent prediction constraint so latent states remain predictive of future latent features. Evaluated in high-fidelity FEM simulation, it reports consistent improvements over architecture-matched recurrent baselines for long-horizon trajectory estimation and related tasks.
Why it matters
For soft robotics with proprioceptive sensing, instantaneous internal observations can’t directly reveal global pose, contacts, or shape evolution, leading to long-horizon error accumulation and latent drift. The authors propose that enforcing latent temporal consistency via predictive latent dynamics improves long-horizon deformable motion estimation, with reported gains across trajectory, orientation, shape, and contact inference in a controlled FEM benchmark.
Method
- Use proprioceptive visual encoder (shared ResNet-18) to map internal deformation images to 512-dim features from two internal cameras; concatenate features.
- Aggregate temporally with an LSTM latent dynamics model under partial observability; treat LSTM output as a predictive latent state (not an explicit physical state).
- Train with self-supervised latent prediction constraint: align latent state at time t with next-step encoded feature (and optionally cascaded predictive latent alignment), plus supervised losses for translation/orientation/shape/contact.
Limitation
This work is limited by the inherent constraints of proprioceptive visual sensing, which encodes motion only through deformation and contact and therefore cannot observe deformation-sparse motions such as free fall or frictionless sliding. In addition, experiments focus on simulated single-object interactions with static terrain and do not yet include real-world validation, multi-object interactions, or deformable–deformable collisions.
Abstract (from arXiv)
Estimating the motion of deformable bodies in soft robotics from proprioceptive sensing is challenging due to partial observability, as internal observations provide only indirect access to global motion, contact, and shape evolution. In such settings, instantaneous observations are insufficient, and accurate estimation requires temporally consistent latent representations. This paper studies deformable motion estimation under proprioceptive partial observability from a latent dynamics perspective. We argue that a key requirement for long-horizon estimation is latent temporal consistency, where latent states remain predictive of future observations. To this end, we present DeepSoMo, a learning-based framework that enforces predictive structure in latent dynamics through a self-supervised latent prediction constraint. We evaluate the approach in a canonical deformable dynamics setting using high-fidelity finite-element simulation as a controlled source of ground truth. Results demonstrate consistent improvements in trajectory estimation, orientation recovery, shape reconstruction, and contact inference over architecture-matched recurrent baselines.
Related papers
- PhysLDM: Latent Diffusion for High-Fidelity Deformable Simulation
- MotionCore: A Latent Neural Simulator for Video Generation with Rigid-Body Dynamics
- SplitJEPA: Learning Invariant and Variant Latent Worlds without Reconstruction
- Being-M0.7: A Latent World-Action Model for Humanoid Robots
- Few-Shot Residual Adaptation of Pretrained Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC
DeepSoMo: Deformable Object Motion Estimation under Proprioceptive Partial Observability via Predictive Latent Dynamics
First announced by arXiv: 2026-11-10 · Original record ↗
- arxiv
- 2611.11242
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.11242v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Estimating the motion of deformable bodies in soft robotics from proprioceptive sensing is challenging due to partial observability, as internal observations provide only indirect access to global motion, contact, and shape evolution. In such settings, instantaneous observations are insufficient, and accurate estimation requires temporally consistent latent representations. This paper studies deformable motion estimation under proprioceptive partial observability from a latent dynamics perspective. We argue that a key requirement for long-horizon estimation is latent temporal consistency, where latent states remain predictive of future observations. To this end, we present DeepSoMo, a learning-based framework that enforces predictive structure in latent dynamics through a self-supervised latent prediction constraint. We evaluate the approach in a canonical deformable dynamics setting using high-fidelity finite-element simulation as a controlled source of ground truth. Results demonstrate consistent improvements in trajectory estimation, orientation recovery, shape reconstruction, and contact inference over architecture-matched recurrent baselines.
