Driving · 2026-10-01
SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models
Yunhan Wang, Haodong Wang, Zhiming Liu, Zicong Hong, Qianli Liu, Xiaoyi Pang, Yangjia Hu, Quanxin Shou, Yikun Miao, Song Guo
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SteerQuant is a 4-bit post-training quantization method for world-action models. It steers quantization errors using an Action-Impact Stream Map so that errors are prioritized toward computations with less effect on final actions. It coordinates shared channel scaling and stream-specific activation modulation across denoising steps, and uses Rudder to fuse scaling into low-bit kernels for efficient inference.
Why it matters
The authors report that similar local quantization errors across semantic streams can lead to markedly different final-action deviations and task success, and that action impact varies across layers and denoising steps. SteerQuant targets this by guiding fixed 4-bit quantization with downstream action impact, maintaining control performance while improving denoising speed and reducing GPU memory.
Method
- Construct an Action-Impact Stream Map estimating how layer–step–stream quantization errors affect final actions, then use it to weight calibration choices.
- Calibrate shared channel scaling (Shared-Weight Error Routing) using the map to minimize action-weighted reconstruction error while retaining one static 4-bit weight matrix per layer.
- Calibrate stream-specific activation modulation across denoising steps (Stream-Specific Activation Modulation) and execute with Rudder, a fused 4-bit inference engine.
Limitation
PTQ4DiT and Q-DiT results are omitted from Table 2 because their official implementations do not provide low-bit inference kernels for the evaluated settings.
Abstract (from arXiv)
World-action models (WAMs) jointly generate future world states and actions through iterative denoising, using shared weights to process heterogeneous semantic streams of video, proprioceptive, and action tokens. Quantization reduces inference cost, but comparable numerical errors in different streams can have markedly different effects on final actions, making numerical accuracy alone insufficient for reliable control. We introduce SteerQuant, a 4-bit quantization framework for WAMs that steers errors toward computations with less influence on final actions. It maps how each stream's quantization errors affect final actions and uses this map to guide shared channel scaling. Activation scaling is further calibrated for each stream and denoising step to accommodate changes in activation ranges and action impact. This adapts quantization to different stream requirements without duplicating weights or increasing bit-widths for selected streams. To reduce the extra kernel launches and memory traffic introduced by scaling, we develop Rudder, a 4-bit inference engine for WAMs that fuses scaling and output compensation into low-bit kernels. Under W4A8 and W4A4, SteerQuant maintains mean LIBERO success within 0.8 percentage points of full precision, while delivering up to $2.23\times$ denoising speedup over BF16 across three WAMs with reduced peak GPU memory usage. On a real dual-arm robot, W4A8 deployment achieves a $1.35\times$ end-to-end inference speedup while maintaining average task success relative to BF16.
Related papers
- MARS: Systematic Multi-Level Acceleration for World Action Models
- Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination
- Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks
- 1Step-WAM: Your World Action Model but Faster using 1-Step Action Generation
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
SteerQuant: Steering Quantization Error with Action-Guided Scaling in World-Action Models
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.39056
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models (WAMs) jointly generate future world states and actions through iterative denoising, using shared weights to process heterogeneous semantic streams of video, proprioceptive, and action tokens. Quantization reduces inference cost, but comparable numerical errors in different streams can have markedly different effects on final actions, making numerical accuracy alone insufficient for reliable control. We introduce SteerQuant, a 4-bit quantization framework for WAMs that steers errors toward computations with less influence on final actions. It maps how each stream's quantization errors affect final actions and uses this map to guide shared channel scaling. Activation scaling is further calibrated for each stream and denoising step to accommodate changes in activation ranges and action impact. This adapts quantization to different stream requirements without duplicating weights or increasing bit-widths for selected streams. To reduce the extra kernel launches and memory traffic introduced by scaling, we develop Rudder, a 4-bit inference engine for WAMs that fuses scaling and output compensation into low-bit kernels. Under W4A8 and W4A4, SteerQuant maintains mean LIBERO success within 0.8 percentage points of full precision, while delivering up to $2.23\times$ denoising speedup over BF16 across three WAMs with reduced peak GPU memory usage. On a real dual-arm robot, W4A8 deployment achieves a $1.35\times$ end-to-end inference speedup while maintaining average task success relative to BF16.