Latent Dynamics · 2026-10-21
Learning Computational Structure for Sample-Efficient Prediction
Hangyu Zhou, Constantine Dovrolis, Stephen Mussmann
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study whether gradient-based prediction can exploit hidden compositional (graph) structure. They introduce CSFRBench, which hides the target computation during training but keeps it for diagnosis, and propose Learned Wiring (LeWi): learn low-dimensional neuron embeddings that determine which connections are active (within a radius) while jointly training connection weights. LeWi improves sample efficiency on hierarchical, reuse, or modular targets and performs better than a dense baseline on 17 world-model prediction tasks.
Why it matters
High-dimensional learning can suffer from the curse of dimensionality, but compositionally sparse functions can be learnable with sample complexity governed by local arity. The paper addresses whether practical gradient-based training can realize this benefit when compositional structure is hidden. By combining a controlled benchmark (CSFRBench) with a learned-connectivity architecture (LeWi), the authors evaluate both predictive sample efficiency and whether learned internal structure aligns with the hidden computation.
Method
- CSFRBench: a controlled benchmark for compositionally sparse function regression where the target computation graph is hidden from the learner but retained for diagnosis.
- Learned Wiring (LeWi): each neuron has a learned low-dimensional embedding; connections between adjacent layers are active only when embedding distances are within a fixed radius; embeddings and connection weights are trained jointly from the prediction loss.
- World-model experiments: apply the same idea to state-prediction under continuous control, reporting open-loop NMSE at horizon 16 and using AULC over training-set sizes.
Limitation
The authors report that Dense networks perform better on targets with a single high-arity function, while LeWi’s advantage is more pronounced on hierarchical structure, intermediate reuse, or independent modules.
Abstract (from arXiv)
Statistical learning in high dimensions suffers from the curse of dimensionality. Fortunately, many functions of practical interest are compositionally sparse, meaning they admit a composition of local functions, each depending on only a few variables, allowing sample complexity to depend on local arity rather than ambient dimension. Yet realizing this benefit requires exploiting hidden compositional structure, which theory does not establish for practical gradient-based training. We study this gap with CSFRBench, a controlled benchmark that hides the target's compositional computation during training but retains it for diagnosis, and find that dense networks and sparsity-based baselines leave much of that benefit unrealized. Inspired by the spatial organization of biological neural circuits, we introduce Learned Wiring (LeWi): each neuron has a learned low-dimensional embedding, only nearby neurons connect, and embeddings and weights are trained jointly. LeWi improves sample efficiency on targets with hierarchical structure, intermediate reuse, or independent modules. Across 32 additional targets generated by independently sampled computation graphs, LeWi is the most sample-efficient method not given the target graph, with 27% lower mean learning-curve area than dense networks. Dense networks perform better on targets with a single high-arity function. LeWi's representations and connectivity align with the target computation. On 17 world-model prediction tasks spanning control, navigation, and manipulation, LeWi beats the dense baseline on 16 and wins 99/119 baseline comparisons. These results show that jointly learning connectivity and weights can exploit computational structure to improve sample efficiency.
Related papers
- NeuronDiscover: Agent-in-Twin for Mechanistic Discovery in Neuronal Microenvironments with World Action Models
- Adaptive Latent Capacity for World Models
- A Stochastic Latent Variable Generalization of Joint-Embedding Predictive Architectures
- NeuroLens: Learning Latent Embeddings of Neural Semantics from Chronic Recordings
- Identifiable Signed Causal Learning via Latent Stochastic Differential Equations for Inferring Brain Dynamic Effective Connectivity
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-20 · Indexed by Pub-AI: 2026-10-21 UTC
Learning Computational Structure for Sample-Efficient Prediction
Preprint posted: 2026-10-20 · Original record ↗
- arxiv
- 2610.28555
Explicit source versions (1)
- v1 · 2026-10-20
- Preprint available
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Statistical learning in high dimensions suffers from the curse of dimensionality. Fortunately, many functions of practical interest are compositionally sparse, meaning they admit a composition of local functions, each depending on only a few variables, allowing sample complexity to depend on local arity rather than ambient dimension. Yet realizing this benefit requires exploiting hidden compositional structure, which theory does not establish for practical gradient-based training. We study this gap with CSFRBench, a controlled benchmark that hides the target's compositional computation during training but retains it for diagnosis, and find that dense networks and sparsity-based baselines leave much of that benefit unrealized. Inspired by the spatial organization of biological neural circuits, we introduce Learned Wiring (LeWi): each neuron has a learned low-dimensional embedding, only nearby neurons connect, and embeddings and weights are trained jointly. LeWi improves sample efficiency on targets with hierarchical structure, intermediate reuse, or independent modules. Across 32 additional targets generated by independently sampled computation graphs, LeWi is the most sample-efficient method not given the target graph, with 27% lower mean learning-curve area than dense networks. Dense networks perform better on targets with a single high-arity function. LeWi's representations and connectivity align with the target computation. On 17 world-model prediction tasks spanning control, navigation, and manipulation, LeWi beats the dense baseline on 16 and wins 99/119 baseline comparisons. These results show that jointly learning connectivity and weights can exploit computational structure to improve sample efficiency.
