Unassigned · 2026-10-16
Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation
Yinan Li, Han Han, Kwang-Sung Jun
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study bounded regression with misspecification using L1(PX) error to the conditional mean η(x), targeting downstream decision settings. They analyze sign-betting regression (SBR), a min–max estimator from Baraud and Maillard (2025). They prove high-probability L1 oracle inequalities under misspecification with variance-adaptive bounds under realizability, and show the optimal universal approximation factor for proper learning is 3.
Why it matters
Most standard regression objectives align with the conditional mean under realizability but can fail under misspecification for both approximation (linear-in-ε1,* in L1) and noise/variance adaptation. SBR is designed to match the L1 oracle criterion and achieves a provably optimal universal approximation factor (3) for proper learning, with variance-adaptive rates depending on average conditional variance.
Method
- Measure regression quality by L1(PX) distance to the conditional mean η(x)=E[Y|X=x] under misspecification.
- Use sign-betting regression (SBR): a proper min–max estimator over f∈F minimizing a worst-case empirical sign-betting score between candidate predictors.
- Prove high-probability L1 oracle inequalities for SBR and analyze approximation-factor optimality for proper learning.
Limitation
The paper does not assume realizability; however, for log-loss ERM their stated L1 guarantee requires a restricted boundary condition: f(x)∈[γ,1−γ]^{X} for some γ∈(0,1/2).
Abstract (from arXiv)
We study bounded regression under model misspecification, evaluating a function $f$ by its $L_1(P_X)$ distance to the conditional mean $\eta(x)=\mathbb{E}[Y\mid X=x]$, rather than the standard $L_2(P_X)$. This $L_1$ metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call \emph{sign-betting regression}. Our main results show that sign-betting regression achieves high-probability $L_1$ oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability $L_1$ oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.
Related papers
- Block Optimism for Nonstationary Bandits with Latent Linear Dynamics
- Statistical Foundations of Reinforcement Learning: A Non-Asymptotic Perspective
- Reachability-Aware Diffusion Policy Optimization
- FewStepRL: Fine-tuning Few-step Diffusion Models with Reinforcement Learning
- World Models Need Not Be Right to Rank Policies Right: Relative Bounds and Audit Limits
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC
Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation
Preprint posted: 2026-10-14 · Original record ↗
- arxiv
- 2610.20903
Explicit source versions (1)
- v1 · 2026-10-14
- Preprint available
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
We study bounded regression under model misspecification, evaluating a function $f$ by its $L_1(P_X)$ distance to the conditional mean $η(x)=\mathbb{E}[Y\mid X=x]$, rather than the standard $L_2(P_X)$. This $L_1$ metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call \emph{sign-betting regression}. Our main results show that sign-betting regression achieves high-probability $L_1$ oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability $L_1$ oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.
