Video World Models · 2026-10-23

V-JEPA 2 Meets AQA: Phase-Based Scoring and Reference-Based Feedback for Mountain Biking

Santhiya Rajan, Savio Cardoz

arXiv:2610.30160PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors study mountain-bike action quality assessment (AQA) with weak binary technique labels and candidate corrective feedback. They use frozen V-JEPA 2 video features plus image-relative 2D pose/motion features aggregated over four heuristic phases. A small transformer predicts a binary technique score; a reference comparison produces phase/body/direction cues. In five-fold pooled out-of-fold evaluation, AUROC=0.768 and AP=0.812.

Why it matters

The paper is an exploratory case study on whether frozen video features and simple measurements can separate weak binary technique labels and support reference-based corrective cues when richer supervision is unavailable. It also audits protocol sensitivity (test-fold epoch selection) and feedback validity limits (camera/view dependence, heuristic phase boundaries, reference derived from technique labels, coach disagreement).

Method

  • Four heuristic phase windows (approach, take-off, flight, landing) are formed from image-plane hip motion; features are pooled per phase from frozen V-JEPA 2 video features plus 2D pose/motion descriptors.
  • A two-layer transformer with eight aspect queries outputs eight aspect sub-scores that are mixed into a clip-level binary technique score.
  • A reference layer fits per-parameter/per-phase mean/std from clips labeled good technique and emits candidate corrective cues by comparing observed standardized deviation (optionally rewording direction off by InternVL3 in a separate test).

Limitation

The authors report that the external scoring probe uses an earlier checkpoint and is based on a small 17-clip sample; they also state that the feedback probe does not establish corrective validity due to heuristic phase boundaries, reference distribution derived from technique labels (not adjudicated expert ratings), view/camera dependence, frequent cue triggering, and the deficiency-output head lacking cue-level supervision. They further note that pooled in-corpus results are exploratory because the reporting fold selected the neural training epoch and riders were not separated between folds.

Abstract (from arXiv)

We study mountain-bike action quality assessment (AQA) with weak labels and candidate feedback. The training corpus comprises 95 field-recorded jump clips with binary labels denoting good or poor technique. These are coarse categorical labels, not continuous expert quality scores; phase-boundary and aspect-level annotations are unavailable. Our pipeline summarizes four heuristic phases using frozen V-JEPA 2 video features and two-dimensional pose and motion measurements. A small transformer predicts a binary-label score. A separate reference layer compares phase-specific measurements with those of clips labelled as good technique to propose phase, body-parameter, and direction cues. We detail the measurements, network, and evaluation so that the limits of each output are explicit. In five-fold clip-level cross-validation, the full scorer obtained pooled out-of-fold AUROC 0.768 and average precision 0.812. These are exploratory figures because the reporting fold also selected the neural training epoch and riders were not separated between folds. Removing biomechanical scoring inputs raised AUROC to 0.809. A separate inner-selection run obtained AUROC 0.678, but used about 57 rather than 76 training clips per outer fold, so its difference from the test-selected run does not quantify selection bias. The unsupervised aspect head did not learn distinct factors. On a 17-clip external probe, an earlier checkpoint obtained binary accuracy 0.412 and Spearman association -0.135 with expert scores. Candidate cues had precision 0.118 and recall 1.0 against each of two coaches, with substantial disagreement about direction. These findings do not establish rider-independent scoring or reliable corrective feedback; they identify the empirical work needed to evaluate either use.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-09-27 · Indexed by Pub-AI: 2026-10-23 UTC

preprint

V-JEPA 2 Meets AQA: Phase-Based Scoring and Reference-Based Feedback for Mountain Biking

Santhiya Rajan, Savio Cardoz

Preprint posted: 2026-09-27 · Original record ↗

arxiv
2610.30160
Explicit source versions (1)
  • v1 · 2026-09-27
  1. Preprint available

    Source date: 2026-09-27 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

We study mountain-bike action quality assessment (AQA) with weak labels and candidate feedback. The training corpus comprises 95 field-recorded jump clips with binary labels denoting good or poor technique. These are coarse categorical labels, not continuous expert quality scores; phase-boundary and aspect-level annotations are unavailable. Our pipeline summarizes four heuristic phases using frozen V-JEPA 2 video features and two-dimensional pose and motion measurements. A small transformer predicts a binary-label score. A separate reference layer compares phase-specific measurements with those of clips labelled as good technique to propose phase, body-parameter, and direction cues. We detail the measurements, network, and evaluation so that the limits of each output are explicit. In five-fold clip-level cross-validation, the full scorer obtained pooled out-of-fold AUROC 0.768 and average precision 0.812. These are exploratory figures because the reporting fold also selected the neural training epoch and riders were not separated between folds. Removing biomechanical scoring inputs raised AUROC to 0.809. A separate inner-selection run obtained AUROC 0.678, but used about 57 rather than 76 training clips per outer fold, so its difference from the test-selected run does not quantify selection bias. The unsupervised aspect head did not learn distinct factors. On a 17-clip external probe, an earlier checkpoint obtained binary accuracy 0.412 and Spearman association -0.135 with expert scores. Candidate cues had precision 0.118 and recall 1.0 against each of two coaches, with substantial disagreement about direction. These findings do not establish rider-independent scoring or reliable corrective feedback; they identify the empirical work needed to evaluate either use.