Latent World Action Model for GUI Agents
The authors propose a latent GUI world-action model that shifts GUI world modeling from full pixel reconstruction to action-induced future information in latent space. Using a LoRA-adapted LMM with separate action and transition heads plus an action-aware contrastive objective, the model supports action selection with a single forward pass (transition head removed at inference). They report up to 31.9% (offline) and 40.6% (online) relative success-rate gains over pixel-space world models, and up to 98.9% lower inference latency.
