Latent Dynamics · 2026-10-09

LLM-IDEA: Identifiability-Driven Experimental Agent for Autonomous Discovery of Mechanistic World Models

Surya Shetty, Ulisses Braga-Neto

arXiv:2610.11253PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

LLM-IDEA is a closed-loop autonomous discovery agent for mechanistic world models that computes identifiability from the agent’s own proposed model and data. It outputs a three-way plateau verdict—capability limit, resolvable within the design class, or certified exhausted—and uses this to decide whether to search more, run better same-kind experiments, or change the experiment kind. Benchmarks include ODEBench, DiscoverPhysics, and an Alien Universe testbed.

Why it matters

The authors argue that autonomous discovery can plateau for two distinct reasons: limited capability or non-identifiability from the available observation protocol. LLM-IDEA aims to make the agent compute which case applies, rather than guessing, by using a deterministic identifiability engine with a design-class certificate. This is positioned as a concrete stopping and experimental-design rule for closed-loop mechanistic model discovery.

Method

  • Uses an Agentic Council proposer plus a deterministic identifiability engine (diagnostic, design ranker, certificate of exhaustion) to triage plateaus and select next experiments.
  • Local diagnostic uses sensitivity matrix rank and condition number for locally identifiable / practically non-identifiable / locally non-identifiable regimes; symbolic route also checks exact structural symmetries.
  • Certificate checks whether confounded directions are invisible to every design in the current design class; if so, it certifies exhaustion and terminates change_class immediately.

Limitation

The authors note the identifiability analysis is local at the value of theta, and for global identifiability questions probing several operating points is a remedy (they cite a published case where a local check missed a solution).

Abstract (from arXiv)

Large language model agents are being increasingly deployed as autonomous scientists, designing experiments and inferring mechanistic world models with minimal human oversight. Yet identifiability is often overlooked: when a plateau is reached, the agent needs to know whether it is not yet capable enough or the model simply is not identifiable from the data, in which case no amount of further experimentation of the same kind can help. We propose the Identifiability-Driven Experimental Agent (LLM-IDEA) for closed-loop discovery with an identifiability engine that returns a three-way plateau verdict: capability limit, resolvable within the design class, or certified exhausted. On ODEBench, 60 of the 62 systems with free constants are identifiable at round 0; the RC circuit is certified exhausted for every experiment that protocol can run, and a harvesting model is resolvable by one added initial condition. The identifiability engine reproduces known verdicts on Lotka-Volterra, Van der Pol, Lorenz, and a pharmacokinetic model, where it recommends the intravenous arm pharmacologists use, and it ranks the depth scorer of our own benchmark last among four observation designs. On the DiscoverPhysics benchmark, it finds two public worlds whose explanation rubric rewards a distinction no legal experiment can make, and every model there with accurate trajectories failed the explanation grade (15 of 15, against 5 of 9 in identifiable worlds, p = 0.012). On the Alien Universe, a two-body testbed we propose in which a force law switches between a provably non-identifiable and an identifiable protocol, LLM-IDEA on the identifiable protocol reaches discovery depth at least three on 8/8 seeds versus 1/8 without it. An autonomous discovery agent can thus compute, rather than guess, whether a plateau calls for more search, a better experiment of the same kind, or a different kind of experiment.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC

preprint

LLM-IDEA: Identifiability-Driven Experimental Agent for Autonomous Discovery of Mechanistic World Models

Surya Shetty, Ulisses Braga-Neto

Preprint posted: 2026-10-08 · Original record ↗

arxiv
2610.11253
Explicit source versions (1)
  • v1 · 2026-10-08
  1. Preprint available

    Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Large language model agents are being increasingly deployed as autonomous scientists, designing experiments and inferring mechanistic world models with minimal human oversight. Yet identifiability is often overlooked: when a plateau is reached, the agent needs to know whether it is not yet capable enough or the model simply is not identifiable from the data, in which case no amount of further experimentation of the same kind can help. We propose the Identifiability-Driven Experimental Agent (LLM-IDEA) for closed-loop discovery with an identifiability engine that returns a three-way plateau verdict: capability limit, resolvable within the design class, or certified exhausted. On ODEBench, 60 of the 62 systems with free constants are identifiable at round 0; the RC circuit is certified exhausted for every experiment that protocol can run, and a harvesting model is resolvable by one added initial condition. The identifiability engine reproduces known verdicts on Lotka-Volterra, Van der Pol, Lorenz, and a pharmacokinetic model, where it recommends the intravenous arm pharmacologists use, and it ranks the depth scorer of our own benchmark last among four observation designs. On the DiscoverPhysics benchmark, it finds two public worlds whose explanation rubric rewards a distinction no legal experiment can make, and every model there with accurate trajectories failed the explanation grade (15 of 15, against 5 of 9 in identifiable worlds, p = 0.012). On the Alien Universe, a two-body testbed we propose in which a force law switches between a provably non-identifiable and an identifiable protocol, LLM-IDEA on the identifiable protocol reaches discovery depth at least three on 8/8 seeds versus 1/8 without it. An autonomous discovery agent can thus compute, rather than guess, whether a plateau calls for more search, a better experiment of the same kind, or a different kind of experiment.