Unassigned · 2026-10-16

Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation

Yinan Li, Han Han, Kwang-Sung Jun

arXiv:2610.20903PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors study bounded regression with misspecification using L1(PX) error to the conditional mean η(x), targeting downstream decision settings. They analyze sign-betting regression (SBR), a min–max estimator from Baraud and Maillard (2025). They prove high-probability L1 oracle inequalities under misspecification with variance-adaptive bounds under realizability, and show the optimal universal approximation factor for proper learning is 3.

Why it matters

Most standard regression objectives align with the conditional mean under realizability but can fail under misspecification for both approximation (linear-in-ε1,* in L1) and noise/variance adaptation. SBR is designed to match the L1 oracle criterion and achieves a provably optimal universal approximation factor (3) for proper learning, with variance-adaptive rates depending on average conditional variance.

Method

  • Measure regression quality by L1(PX) distance to the conditional mean η(x)=E[Y|X=x] under misspecification.
  • Use sign-betting regression (SBR): a proper min–max estimator over f∈F minimizing a worst-case empirical sign-betting score between candidate predictors.
  • Prove high-probability L1 oracle inequalities for SBR and analyze approximation-factor optimality for proper learning.

Limitation

The paper does not assume realizability; however, for log-loss ERM their stated L1 guarantee requires a restricted boundary condition: f(x)∈[γ,1−γ]^{X} for some γ∈(0,1/2).

Abstract (from arXiv)

We study bounded regression under model misspecification, evaluating a function $f$ by its $L_1(P_X)$ distance to the conditional mean $\eta(x)=\mathbb{E}[Y\mid X=x]$, rather than the standard $L_2(P_X)$. This $L_1$ metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call \emph{sign-betting regression}. Our main results show that sign-betting regression achieves high-probability $L_1$ oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability $L_1$ oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC

preprint

Sign Betting for Misspecified Regression: Variance Adaptivity and Optimal Approximation

Yinan Li, Han Han, Kwang-Sung Jun

Preprint posted: 2026-10-14 · Original record ↗

arxiv
2610.20903
Explicit source versions (1)
  • v1 · 2026-10-14
  1. Preprint available

    Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

We study bounded regression under model misspecification, evaluating a function $f$ by its $L_1(P_X)$ distance to the conditional mean $η(x)=\mathbb{E}[Y\mid X=x]$, rather than the standard $L_2(P_X)$. This $L_1$ metric is natural for downstream decision problems such as bandits and reinforcement learning. We analyze the min--max estimator of Baraud & Maillard (2025), which we call \emph{sign-betting regression}. Our main results show that sign-betting regression achieves high-probability $L_1$ oracle inequalities under misspecification. Under realizability, our bound adapts to the average conditional variance and recovers the variance-adaptive rate of Li et al. (2026). We further characterize the optimal universal approximation factor for proper learning by showing that sign betting achieves factor 3, while no proper learner can satisfy a distribution-free, high-probability $L_1$ oracle inequality with a universal approximation factor strictly below 3. We extend the upper-bound analysis beyond finite classes and complement the theory with experiments in synthetic and real-world model-selection tasks.