Driving · 2026-10-14

Physics-Aware Object-Centric SE(3) Forecasting for Real-Time Proactive Dynamic Grasping

Yirui Zhou, Hongli Xu, Ying Zheng, Yi Wang, Lap-Pui Chau

arXiv:2610.17843PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose PAF-Grasp, an object-centric forecast-then-select framework for real-time proactive dynamic grasping. Instead of dense video prediction, it forecasts contact-relevant rigid-body trajectories in SE(3) from short pose histories, then selects reachable contact states under kinematic and timing constraints. Trained in randomized physics simulation, it transfers to real objects after recovering geometry and inertia. They report higher success in both simulation and real-world tasks with 15 ms forecasting.

Why it matters

Proactive dynamic grasping needs a decision about when/where contact becomes feasible under robot reachability and system latency. The paper argues that predicting compact contact-relevant SE(3) rigid-body state is sufficient for this, enabling faster, physically meaningful predictions than dense visual future generation and supporting real-time contact-state selection.

Method

  • Predict future rigid-body trajectories as SE(3) state using a physics-aware object-centric forecaster trained on randomized Genesis primitive-object trajectories and integrated via a Newton–Euler-structured neural ODE.
  • Recover object-specific metric geometry and an approximate inertia prior from an anchor RGB-D observation, then update SE(3) pose history online for closed-loop forecasting.
  • Use a reachability-aware interception selector that filters predicted task contact frames by workspace, IK, collision, and latency/timing constraints, and chooses a feasible contact-time target.
Abstract (from arXiv)

Proactive dynamic grasping requires robots to predict the future state at which physical contact will occur, rather than reacting to the currently observed object pose. But what future must the robot predict, and at what level of abstraction? We argue that contact-relevant rigid-body state in SE(3) is a compact task-sufficient abstraction for this problem: it directly determines where and when the robot should establish contact, without requiring visually complete future scenes. This insight leads to a lightweight object-centric forecast-then-select framework that predicts future rigid-body trajectories from short pose histories and uses a reachability-aware interception selector to choose feasible contact states under kinematic and timing constraints. Trained with randomized physics simulation over primitive shapes, the forecaster transfers to real objects once their metric geometry and SE(3) state histories are recovered. Experiments across simulation and real-world dynamic grasping tasks show that our framework improves the average success rate from \(24.5\%\) to \(62.0\%\) in simulation, and from \(15.3\%\) to \(67.3\%\) in real-world catching, conveyor, and turntable interception. The SE(3) forecaster runs in \(15\,\mathrm{ms}\), orders of magnitude faster than dense visual future prediction, enabling real-time contact decision making. Our results suggest that effective proactive grasping does not require imagining the entire future scene. Instead, predicting the compact physical future that determines contact provides an effective and practical abstraction for real-time dynamic grasping.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC

preprint

Physics-Aware Object-Centric SE(3) Forecasting for Real-Time Proactive Dynamic Grasping

Yirui Zhou, Hongli Xu, Ying Zheng, Yi Wang, Lap-Pui Chau

Preprint posted: 2026-10-13 · Original record ↗

arxiv
2610.17843
Explicit source versions (1)
  • v1 · 2026-10-13
  1. Preprint available

    Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Proactive dynamic grasping requires robots to predict the future state at which physical contact will occur, rather than reacting to the currently observed object pose. But what future must the robot predict, and at what level of abstraction? We argue that contact-relevant rigid-body state in SE(3) is a compact task-sufficient abstraction for this problem: it directly determines where and when the robot should establish contact, without requiring visually complete future scenes. This insight leads to a lightweight object-centric forecast-then-select framework that predicts future rigid-body trajectories from short pose histories and uses a reachability-aware interception selector to choose feasible contact states under kinematic and timing constraints. Trained with randomized physics simulation over primitive shapes, the forecaster transfers to real objects once their metric geometry and SE(3) state histories are recovered. Experiments across simulation and real-world dynamic grasping tasks show that our framework improves the average success rate from \(24.5\%\) to \(62.0\%\) in simulation, and from \(15.3\%\) to \(67.3\%\) in real-world catching, conveyor, and turntable interception. The SE(3) forecaster runs in \(15\,\mathrm{ms}\), orders of magnitude faster than dense visual future prediction, enabling real-time contact decision making. Our results suggest that effective proactive grasping does not require imagining the entire future scene. Instead, predicting the compact physical future that determines contact provides an effective and practical abstraction for real-time dynamic grasping.