Driving · 2026-10-07
WiSPER: Pose-Supervised Predictive and Residual Flow Refinement For Multi-Person 3D Pose Estimation With WiFi CSI
Gabriel Lee Jun Rong, Shanhong Liu, Pai Chet Ng, Konstantinos N. Plataniotis, Jamal Seyedmohammadi, S. Mohammad Sheikholeslami
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WiSPER is a two-stage framework for multi-person 3D pose estimation from WiFi CSI. Stage 1 (PAMEL) performs pose-aware masked latent prediction with auxiliary pose-set supervision on visible CSI context. Stage 2 (ReFT) generates multiple pose candidates with a hierarchical pose Transformer and refines each via conditional residual flow guided by coarse coordinates and per-joint features. Inference uses only CSI.
Why it matters
The authors address multi-person ambiguity in CSI (overlapping reflections) by adding explicit pose supervision during predictive pretraining and by modeling remaining coordinate errors with conditional residual flow refinement. They report lower MPJPE on PiW3D for WiSPER vs WiFi-JEPA, and further MPJPE reductions from enabling the residual refiner during inference.
Method
- PAMEL (pose-aware masked embedding learning): masked latent prediction with auxiliary pose-set supervision on the same visible CSI context.
- ReFT (residual flow refinement with Transformer): hierarchical pose Transformer produces Q pose candidates and presence logits; a conditional residual flow refines each candidate using coarse coordinates and per-joint decoder features.
- Training uses paired CSI and pose annotations in both stages; inference requires only CSI.
Limitation
The authors note that results are based on individual training runs, do not assess person-count accuracy, and evaluation uses temporal portions of shared recordings, leaving generalization to unseen participants and environments, as well as inference latency, for future investigation.
Abstract (from arXiv)
Multi-person 3D pose estimation with WiFi channel state information (CSI) is challenging because reflections from different people overlap without directly identifying individual joints. Existing masked embedding objectives capture wireless relationships without explicit pose supervision, while structured decoders can retain coordinate errors. We propose WiSPER, a two-stage framework combining pose-aware predictive pretraining with conditional residual flow refinement. Pose-Aware Masked Embedding Learning (PAMEL) couples masked latent prediction with auxiliary pose-set supervision on the same CSI context, guiding the encoder toward joint localization from partial observations. Residual Flow refinement with Transformer (ReFT) generates a set of pose candidates to accommodate a variable number of people and refines each candidate through a conditional flow guided by its coarse coordinates and per-joint decoder features. Both stages use paired CSI and pose annotations during training, while inference requires only CSI. Experiments on the PiW3D dataset show that WiSPER achieves an overall mean per-joint position error of 63.72 mm, a 40.0% reduction relative to WiFi-JEPA. For experiments with two and three people, WiSPER reduces MPJPE by 42.1% and 38.1%, respectively. Pose-supervised pretraining configurations obtain lower errors than CSI-only JEPA, and enabling the trained residual refiner reduces overall MPJPE by 13.8-15.6% across the evaluated configurations.
Related papers
- Multi-View 3D Reconstruction Using V-JEPA Features
- World2Motion: Turning Video World Models into 3D Human Motion Generators
- UNITAS: A 3D-Native World Action Model for Embodied Manipulation
- MVG-WAM: Multiple View Geometry-Aware World-Action Modeling for Robotic Manipulation
- Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-04 · Indexed by Pub-AI: 2026-10-07 UTC
WiSPER: Pose-Supervised Predictive and Residual Flow Refinement For Multi-Person 3D Pose Estimation With WiFi CSI
Preprint posted: 2026-10-04 · Original record ↗
- arxiv
- 2610.07025
Explicit source versions (1)
- v1 · 2026-10-04
- Preprint available
Source date: 2026-10-04 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Multi-person 3D pose estimation with WiFi channel state information (CSI) is challenging because reflections from different people overlap without directly identifying individual joints. Existing masked embedding objectives capture wireless relationships without explicit pose supervision, while structured decoders can retain coordinate errors. We propose WiSPER, a two-stage framework combining pose-aware predictive pretraining with conditional residual flow refinement. Pose-Aware Masked Embedding Learning (PAMEL) couples masked latent prediction with auxiliary pose-set supervision on the same CSI context, guiding the encoder toward joint localization from partial observations. Residual Flow refinement with Transformer (ReFT) generates a set of pose candidates to accommodate a variable number of people and refines each candidate through a conditional flow guided by its coarse coordinates and per-joint decoder features. Both stages use paired CSI and pose annotations during training, while inference requires only CSI. Experiments on the PiW3D dataset show that WiSPER achieves an overall mean per-joint position error of 63.72 mm, a 40.0% reduction relative to WiFi-JEPA. For experiments with two and three people, WiSPER reduces MPJPE by 42.1% and 38.1%, respectively. Pose-supervised pretraining configurations obtain lower errors than CSI-only JEPA, and enabling the trained residual refiner reduces overall MPJPE by 13.8-15.6% across the evaluated configurations.
