JEPA · 2026-11-03

Cross-Domain Joint-Embedding Pretraining For Heterogeneous Industrial Time Series

Jonathan Adam Rico, Nagarajan Raghavan, Senthilnath Jayavelu

arXiv:2611.02489PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors propose JEPA-HITS, a channel-independent joint-embedding predictive pretraining method for heterogeneous industrial time series. They learn one shared JEPA encoder across battery, CMP, CNC, and bearing data while preserving dataset-specific temporal scales, using leave-one-domain-out (LODO) pretraining to test transfer to unseen domains. Compared to the same architecture with random initialization, LODO pretraining reduces RMSE on NASA/CMP/CNC and improves bearing fault macro-F1.

Why it matters

It targets cross-domain self-supervised pretraining for industrial time series where labels are scarce and datasets differ in sensor characteristics and sequence scales. The LODO evaluation tests whether representations transfer across physically distinct industrial domains without using target-domain data during pretraining.

Method

  • JEPA-HITS builds on TS-JEPA/JEPA with a shared online encoder, EMA target encoder, and latent-space predictor, using Huber loss plus anti-collapse (variance/covariance) and a spectral consistency term on predicted vs target latent trajectories.
  • Pretraining is channel-independent: each sensor channel is treated as an individual univariate sequence processed by the shared encoder, while domains keep dataset-specific temporal patching and windowing via domain-specific training buckets.
  • LODO pretraining excludes the target domain entirely from self-supervised pretraining; downstream heads use linear or MLP with frozen or rank-4 LoRA adaptation depending on validation results.

Limitation

The authors state: “Our work is not evaluated on extreme cases with signal outliers, missing data, and distribution shifts.”

Abstract (from arXiv)

Industrial predictive maintenance and manufacturing tasks frequently contain abundant sensor measurements, but with limited labeled data. We investigate cross-domain self-supervised pretraining using joint-embedding predictive architecture for heterogeneous industrial time series (JEPA-HITS), which learns one encoder from various domains such as battery, semiconductor chemical mechanical planarization (CMP), computer numerical control (CNC) machining, and bearing while preserving dataset-specific temporal scales. We evaluate representation reuse on regression and classification problems under reduced-label downstream training, and leave-one-domain-out (LODO) pretraining. Compared with the same architecture initialized randomly, LODO pretraining reduces RMSE from 10.72 to 7.95 on battery remaining useful life (RUL), 4.34 to 3.89 on CMP, and 13.53 to 11.72 on tool wear regression, while improving bearing fault diagnosis macro-F1 from 0.547 to 0.737. Overall results suggest that unlabeled time series measurements from heterogeneous industrial processes can provide reusable representations for downstream learning with limited labeled data.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-02 · Indexed by Pub-AI: 2026-11-03 UTC

preprint

Cross-Domain Joint-Embedding Pretraining For Heterogeneous Industrial Time Series

Jonathan Adam Rico, Nagarajan Raghavan, Senthilnath Jayavelu

Preprint posted: 2026-11-02 · Original record ↗

arxiv
2611.02489
Explicit source versions (1)
  • v1 · 2026-11-02
  1. Preprint available

    Source date: 2026-11-02 · First detected: 2026-11-03 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Industrial predictive maintenance and manufacturing tasks frequently contain abundant sensor measurements, but with limited labeled data. We investigate cross-domain self-supervised pretraining using joint-embedding predictive architecture for heterogeneous industrial time series (JEPA-HITS), which learns one encoder from various domains such as battery, semiconductor chemical mechanical planarization (CMP), computer numerical control (CNC) machining, and bearing while preserving dataset-specific temporal scales. We evaluate representation reuse on regression and classification problems under reduced-label downstream training, and leave-one-domain-out (LODO) pretraining. Compared with the same architecture initialized randomly, LODO pretraining reduces RMSE from 10.72 to 7.95 on battery remaining useful life (RUL), 4.34 to 3.89 on CMP, and 13.53 to 11.72 on tool wear regression, while improving bearing fault diagnosis macro-F1 from 0.547 to 0.737. Overall results suggest that unlabeled time series measurements from heterogeneous industrial processes can provide reusable representations for downstream learning with limited labeled data.