Scene-Aware Temporal Attention Decoupling for Audio-Driven Multi-Scene Control
A training-free method that improves multi-scene temporal control in audio-driven human animation by steering a diffusion transformer’s spatio-temporal self-attention: it suppresses cross-scene attention for background tokens while preserving cross-scene attention for foreground (the human subject). For robustness under large motion, it derives dynamic foreground masks from self-attention layers/heads that best follow the subject trajectory, and uses those masks to localize which tokens should keep cross-scene links.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
Audio-driven human animation can align motion to sound yet still fail at coordinating background scene transitions at precise timestamps without breaking subject consistency. Decoupling background and foreground attention over time directly targets that failure mode.
What changed
Instead of modifying training, the technique applies inference-time guidance to spatio-temporal self-attention: background token cross-scene attention is suppressed, while foreground token cross-scene attention is retained, with dynamic foreground localization obtained from attention-derived masks to handle large motion.
Where it helps
- Audio-driven human animation with precise multi-scene background timing
- Maintaining subject identity/consistency across scene transitions
- Handling large subject motion when controlling temporal scene changes
- Multi-subject scene timing control where background must change independently
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The provided description does not specify the exact implementation details of how masks are selected/applied, the range of motion complexity it supports, or any conditions under which the foreground localization might fail.
Sources
- SceneShifter: Training-free Multi-Scene Temporal Control for Audio-driven Human Animation ↗
arXiv · primary · Checked 2026-11-10
