Driving · 2026-10-15

Active Generative Simulation: Meta-Learning What to Simulate for Sim-and-Real Policy Co-Training

Liqian Ma, Vaibhav Saxena, Utkarsh Aashu Mishra, Shuo Cheng, Yongxin Chen, Danfei Xu

arXiv:2610.19479 PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose Active Generative Simulation and MetaGenSim, which meta-learns a rater network to score synthetic data by utility for real-world validation performance, then constructs a targeted data-generation distribution over simulation parameters. Across real and sim-and-sim experiments with limited real demos, MetaGenSim improves sim-and-real co-training over uniform generation and retrieval, yields a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment.

Why it matters

Instead of treating a simulator as a fixed or uniformly random data source, the authors optimize what simulation to generate by learning a utility landscape tied to downstream real-world performance. This targets the sim-and-real gap by allocating simulation budget to parameter regions that are most helpful for co-training, potentially reducing the need for manual simulator parameter alignment.

Method

MetaGenSim meta-learns an auxiliary rater network via meta-gradients: it assigns utility scores to simulation observation-action pairs based on how they reduce validation loss on held-out real demonstrations after unrolling inner-loop co-training steps.
The rater scores are mapped back to simulation parameters using aggregated trajectory scores, then task-conditioned density estimation with softmax over task modes and weighted KDE over continuous parameters constructs a targeted generation distribution.
Final policies are trained by sim-and-real co-training on simulations sampled from this induced distribution plus few-shot real demonstrations; the authors also explore cross-architecture transfer and partial alignment settings.

Abstract (from arXiv)

Sim-and-real co-training is an effective way to scale robot learning by augmenting limited real-world demonstrations with simulation data. However, existing approaches typically treat simulators as fixed data sources, relying on uniform generation, offline retrieval, or manual parameter alignment by domain experts. We formulate \textit{Active Generative Simulation}: optimizing a simulator's data-generation distribution to maximize downstream real-world performance. To solve this problem, we propose MetaGenSim, a meta-learning framework that trains an auxiliary rater network via meta-gradients to automatically discover high-utility data-generation distributions within a user-specified simulation parameter space, without manual parameter alignment. Across simulation and real-world manipulation tasks, MetaGenSim outperforms uniform generation and retrieval baselines, achieves a 150% relative improvement in real-world full-task success over uniform generation, and matches expert manual alignment. We further show that distributions curated with a lightweight proxy policy transfer to VLA fine-tuning, and that MetaGenSim can exploit partial alignment through atomic skill stitching and object-receptacle composition when exact sim-and-real task alignment is unavailable. Project website: https://meta-gensim.github.io

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-15 UTC

preprint