JEPA · 2026-11-03
Cross-Domain Joint-Embedding Pretraining For Heterogeneous Industrial Time Series
Jonathan Adam Rico, Nagarajan Raghavan, Senthilnath Jayavelu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose JEPA-HITS, a channel-independent joint-embedding predictive pretraining method for heterogeneous industrial time series. They learn one shared JEPA encoder across battery, CMP, CNC, and bearing data while preserving dataset-specific temporal scales, using leave-one-domain-out (LODO) pretraining to test transfer to unseen domains. Compared to the same architecture with random initialization, LODO pretraining reduces RMSE on NASA/CMP/CNC and improves bearing fault macro-F1.
Why it matters
It targets cross-domain self-supervised pretraining for industrial time series where labels are scarce and datasets differ in sensor characteristics and sequence scales. The LODO evaluation tests whether representations transfer across physically distinct industrial domains without using target-domain data during pretraining.
Method
- JEPA-HITS builds on TS-JEPA/JEPA with a shared online encoder, EMA target encoder, and latent-space predictor, using Huber loss plus anti-collapse (variance/covariance) and a spectral consistency term on predicted vs target latent trajectories.
- Pretraining is channel-independent: each sensor channel is treated as an individual univariate sequence processed by the shared encoder, while domains keep dataset-specific temporal patching and windowing via domain-specific training buckets.
- LODO pretraining excludes the target domain entirely from self-supervised pretraining; downstream heads use linear or MLP with frozen or rank-4 LoRA adaptation depending on validation results.
Limitation
The authors state: “Our work is not evaluated on extreme cases with signal outliers, missing data, and distribution shifts.”
Abstract (from arXiv)
Industrial predictive maintenance and manufacturing tasks frequently contain abundant sensor measurements, but with limited labeled data. We investigate cross-domain self-supervised pretraining using joint-embedding predictive architecture for heterogeneous industrial time series (JEPA-HITS), which learns one encoder from various domains such as battery, semiconductor chemical mechanical planarization (CMP), computer numerical control (CNC) machining, and bearing while preserving dataset-specific temporal scales. We evaluate representation reuse on regression and classification problems under reduced-label downstream training, and leave-one-domain-out (LODO) pretraining. Compared with the same architecture initialized randomly, LODO pretraining reduces RMSE from 10.72 to 7.95 on battery remaining useful life (RUL), 4.34 to 3.89 on CMP, and 13.53 to 11.72 on tool wear regression, while improving bearing fault diagnosis macro-F1 from 0.547 to 0.737. Overall results suggest that unlabeled time series measurements from heterogeneous industrial processes can provide reusable representations for downstream learning with limited labeled data.
Related papers
- Does Joint-Embedding Predictive Architecture Pretraining Help Time Series Forecasting?
- Motor compound fault diagnosis using an attention-based cross-talk architecture
- Self-Supervised Latent Action-Conditioned World Modeling for Time Series
- KITE: A Knowledge-Injected Temporal Encoder for Time-Series Event Forecasting
- BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-02 · Indexed by Pub-AI: 2026-11-03 UTC
Cross-Domain Joint-Embedding Pretraining For Heterogeneous Industrial Time Series
Preprint posted: 2026-11-02 · Original record ↗
- arxiv
- 2611.02489
Explicit source versions (1)
- v1 · 2026-11-02
- Preprint available
Source date: 2026-11-02 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.02489v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Industrial predictive maintenance and manufacturing tasks frequently contain abundant sensor measurements, but with limited labeled data. We investigate cross-domain self-supervised pretraining using joint-embedding predictive architecture for heterogeneous industrial time series (JEPA-HITS), which learns one encoder from various domains such as battery, semiconductor chemical mechanical planarization (CMP), computer numerical control (CNC) machining, and bearing while preserving dataset-specific temporal scales. We evaluate representation reuse on regression and classification problems under reduced-label downstream training, and leave-one-domain-out (LODO) pretraining. Compared with the same architecture initialized randomly, LODO pretraining reduces RMSE from 10.72 to 7.95 on battery remaining useful life (RUL), 4.34 to 3.89 on CMP, and 13.53 to 11.72 on tool wear regression, while improving bearing fault diagnosis macro-F1 from 0.547 to 0.737. Overall results suggest that unlabeled time series measurements from heterogeneous industrial processes can provide reusable representations for downstream learning with limited labeled data.
