Robotics · 2026-11-03
TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation
Jiaxin Shi, Zibin Wu, Yichi Zhang, Jianghua Duan, Siyu Zhu, Weihao Yuan
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation proposes TacMind-Bench (128 simulation tasks) to evaluate how tactile information helps precise contact, recognition under visual ambiguity, and dynamic interaction under 12 perturbations. The authors introduce TacMind-0: an action-conditioned predictive latent world model (Tac-LeWM) that learns streaming tactile states and refines actions using predictive tactile reflection.
Why it matters
The authors argue that tactile sensing should be studied as a temporally evolving interaction representation, not just instantaneous feedback, and they introduce a benchmark organized around the role of touch across manipulation regimes. Their method integrates tactile history into an action-conditioned latent predictive model and uses predicted future tactile latents to refine candidate actions, aiming for consistent gains across regimes.
Method
- Build TacMind-Bench (28 Standard + 100 Extra tasks) spanning Precise, Recognition (under visual ambiguity), and Dynamic interaction, with controlled static/dynamic variants and 12 perturbation settings.
- Train Tac-LeWM: an action-conditioned predictive latent model that integrates tactile history and learns a tactile state by predicting future tactile latents (not reconstructing future tactile images).
- Use TacMind-0 for control: tactile state conditions action generation, and “predictive tactile reflection” refines candidate actions by predicting the future tactile latent state of proposals.
Abstract (from arXiv)
Tactile sensing provides robots with direct observations of contact physics, local geometry, and interaction dynamics, making it a vital source of feedback for contact-rich manipulation. However, existing tactile manipulation studies are often developed and evaluated within specific task settings, making it difficult to systematically compare how tactile information contributes under different interaction conditions. In this work, we develop a benchmark in simulation comprising 128 tasks organized around three complementary evaluation regimes: Precise, Recognition, and Dynamic. The benchmark further includes controlled static and dynamic task variants and 12 perturbation settings for evaluating robustness under changes in visual and physical conditions. These settings motivate studying tactile information as a temporally evolving interaction representation rather than only as an instantaneous observation. We therefore propose an action-conditioned latent predictive world model that integrates tactile history and learns a tactile state through prediction of future tactile latents. Building on this representation, we design a model that conditions action generation on the learned tactile state and uses predicted future tactile information to refine candidate actions through predictive tactile reflection. Experiments on the proposed benchmark show consistent gains from incorporating tactile information across the evaluated regimes. Project Page: https://tacmind-bench.github.io/
Related papers
- TacDyn-WAM: Learning Implicit Tactile Dynamics in a Heterogeneous Visuo-Tactile World Action Model
- Tactile-Guided Robot Learning: From Representation Learning to Foundation Models
- ACTS: Learning Visuo-Tactile World Models from Human Physical Interaction
- Interaction Latents as Actions for Contact-Rich Manipulation
- Dexterous Tactile World Model
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC
TacMind: Benchmarking and Modeling Tactile Interaction States for Robotic Manipulation
First announced by arXiv: 2026-11-03 · Original record ↗
- arxiv
- 2611.00455
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.00455v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Tactile sensing provides robots with direct observations of contact physics, local geometry, and interaction dynamics, making it a vital source of feedback for contact-rich manipulation. However, existing tactile manipulation studies are often developed and evaluated within specific task settings, making it difficult to systematically compare how tactile information contributes under different interaction conditions. In this work, we develop a benchmark in simulation comprising 128 tasks organized around three complementary evaluation regimes: Precise, Recognition, and Dynamic. The benchmark further includes controlled static and dynamic task variants and 12 perturbation settings for evaluating robustness under changes in visual and physical conditions. These settings motivate studying tactile information as a temporally evolving interaction representation rather than only as an instantaneous observation. We therefore propose an action-conditioned latent predictive world model that integrates tactile history and learns a tactile state through prediction of future tactile latents. Building on this representation, we design a model that conditions action generation on the learned tactile state and uses predicted future tactile information to refine candidate actions through predictive tactile reflection. Experiments on the proposed benchmark show consistent gains from incorporating tactile information across the evaluated regimes. Project Page: https://tacmind-bench.github.io/
