Driving · 2026-10-06

Software World Models: From Consequence Prediction to Decision Value

Tongli Su, Yuntong Hu, Liang Zhao, Bowen Zhu, JayaSai Somasundaram, Hasibul Haque

arXiv:2610.04940PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors introduce the Software World Model (SWM) for coding agents: it predicts a code change’s “blast set” (components the change will break) rather than only the agent’s own observations or static reachability. SWM uses Explore–Learn–Act to collect counterfactual execution outcomes, fine-tune a language-model predictor, and convert predicted blast sets into per-consumer break probabilities for change ranking and proactive migration.

Why it matters

For coding agents, local repository tests can pass while downstream consumers break. Static change-impact analysis can miss failures due to hidden couplings, while existing software world models learn only what the agent itself observes. SWM targets system-level consequence prediction and ties the learned forecasts directly to decision-making (which change to land, when to pay for checks, and which consumers to migrate), improving both prediction quality and downstream decision value on synthetic and real libraries.

Method

  • Explore: execute counterfactual candidate changes from restored states, prioritizing “reachability blind spots” where execution outcomes disagree with typed reachability.
  • Learn: fine-tune a language model on executed consequences to predict the blast set and check feedback, conditioned on the change and typed software relations.
  • Act: sample predicted blast sets per candidate to produce per-consumer break probabilities, then use these beliefs for change ranking, preventive migration, and deciding when additional checks pay off.
Abstract (from arXiv)

A coding agent may safely modify one repository while silently breaking downstream services, libraries, or datastores that depend on it. Exhaustively running integration tests after every agent action is impractical, so the agent must predict these failures before executing them. Existing software world models predict the agent's own observations, while static change-impact analysis only identifies where a change may propagate. We instead introduce the Software World Model (SWM), which models the broader system affected by a code change and predicts its blast set: the components that the change will break. SWM follows three stages: explore, learn, and act. Explore executes candidate changes from restored system states, prioritizing regions where observed failures contradict the dependency graph. Learn fine-tunes a language model on these execution outcomes to predict downstream breakage. Act converts sampled predictions into per-consumer break probabilities for change ranking, proactive migration, and deciding when another execution is worth its cost. On held-out synthetic systems, SWM improves blast-set F1 from 0.431 for static reachability to $0.571\pm0.037$, more than halves ranking regret, and improves migration return at all nine evaluation checkpoints. The two methods are complementary: reachability is stronger on dependencies represented in the graph, while SWM recovers failures caused by couplings the graph misses. Experiments on held-out real libraries further show that predicting structured failure outcomes, rather than only scalar risk, is important for downstream decision quality.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-04 · Indexed by Pub-AI: 2026-10-06 UTC

preprint

Software World Models: From Consequence Prediction to Decision Value

Tongli Su, Yuntong Hu, Liang Zhao, Bowen Zhu, JayaSai Somasundaram, Hasibul Haque

Preprint posted: 2026-10-04 · Original record ↗

arxiv
2610.04940
Explicit source versions (1)
  • v1 · 2026-10-04
  1. Preprint available

    Source date: 2026-10-04 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

A coding agent may safely modify one repository while silently breaking downstream services, libraries, or datastores that depend on it. Exhaustively running integration tests after every agent action is impractical, so the agent must predict these failures before executing them. Existing software world models predict the agent's own observations, while static change-impact analysis only identifies where a change may propagate. We instead introduce the Software World Model (SWM), which models the broader system affected by a code change and predicts its blast set: the components that the change will break. SWM follows three stages: explore, learn, and act. Explore executes candidate changes from restored system states, prioritizing regions where observed failures contradict the dependency graph. Learn fine-tunes a language model on these execution outcomes to predict downstream breakage. Act converts sampled predictions into per-consumer break probabilities for change ranking, proactive migration, and deciding when another execution is worth its cost. On held-out synthetic systems, SWM improves blast-set F1 from 0.431 for static reachability to $0.571\pm0.037$, more than halves ranking regret, and improves migration return at all nine evaluation checkpoints. The two methods are complementary: reachability is stronger on dependencies represented in the graph, while SWM recovers failures caused by couplings the graph misses. Experiments on held-out real libraries further show that predicting structured failure outcomes, rather than only scalar risk, is important for downstream decision quality.