Driving · 2026-09-30
CyberWorld: World Models for Sample-Efficient Autonomous Cyber Defense
Ryozo Masukawa, Sanggeon Yun, Raheeb Hassan, Hyunwoo Oh, SungHeon Jeong, Mohsen Imani
Auto-summarized: this summary was generated by a language model from the paper’s text and has not been reviewed by an editor. Check the paper before relying on it. Benchmark numbers appear only after a human has verified them.
TL;DR
CyberWorld is a Dreamer-style world model for autonomous cyber defense that learns latent cyber dynamics from vector, graph, textual, and multimodal representations of a defended network. Using CyberWheel, the graph-based variant reaches the deploy_then_stop control after 3.6k–15.8k environment steps (vs model-free PPO needing 2.3M–3.1M steps or failing within a 3.2M budget).
Why it matters
The authors address sample inefficiency in autonomous cyber defense by learning predictive dynamics and optimizing policies through imagined trajectories. They study what should constitute the “world” in a cyber world model by comparing multiple observation representations and their scaling across network sizes, reporting robustness differences (graph structure helps under topology-dependent attacks) and that world-model-based defense can use orders-of-magnitude fewer environment interactions than model-free PPO.
Method
- Introduce CyberWorld: a Dreamer-style latent world model that learns predictive cyber dynamics and trains defensive policies inside imagined rollouts (DreamerV3-based RSSM).
- Use an environment-agnostic intermediate representation (IR) and modality encoders (per-host vectors, host graph with message passing, text telemetry, optional visual) fused via cross-attention pooling.
- Train with modality-specific reconstruction/prediction heads and cyber-specific loss weighting; optimize actor-critic entirely from trajectories imagined by the learned world model.
Limitation
The authors state that open-loop imagination errors are concentrated in (i) misplacing deployed decoys among reserved slots and (ii) failing to anticipate new alert activity, and that reward misalignment can occur for sparse decoy-touch events that depend on unobserved adversarial progress and precise reward timing.
Abstract (from arXiv)
Deep reinforcement learning has become a prominent approach to autonomous cyber defense. Existing methods are predominantly model-free and consequently require extensive environment interaction. World models provide an alternative by learning predictive dynamics and optimizing policies through imagined trajectories, yielding substantial gains in sample efficiency in robotics and embodied control. Extending this paradigm to cybersecurity raises a fundamental question: what should constitute the "world" in a cyber world model? We introduce CyberWorld, a Dreamer-style world modeling framework that learns latent cyber dynamics from vector, graph, textual, and multimodal representations of the defended network. Across all four scoreable CyberWheel attack strategies, the graph-based CyberWorld variant exceeds a strategy-agnostic control after 3.6k-15.8k environment steps, compared with millions of steps required by model-free PPO. Across representation choices, graph structure provides greater robustness under topology-dependent attacks, while simpler representations remain competitive in overall performance. Among successful runs, the number of episodes required to reach the control remains approximately constant as network size increases from 15 to 100 hosts. These results establish learned cyber dynamics as a sample-efficient and scalable basis for autonomous defense, and identify world representation as a central design axis for robustness and scalability.