JEPA · 2026-10-06
T-JEPA: A Temporal Joint-Embedding Predictive Architecture for Learning Better Remote Sensing Representations
Bowen Peng, Li Liu, Yongxiang Liu, Weijie Li, Jie Zhou, Zhen Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
T-JEPA learns remote-sensing representations from sparsely and nonuniformly sampled Earth observation sequences by predicting time-gap-conditioned latent transitions. A shared single-frame encoder produces masked source latents, and a temporal predictor estimates the complete target latent field conditioned on elapsed time, supervised across all temporally ordered pairs at multiple temporal scales. Asymmetric metadata injection plus masked pixel reconstruction improve learning and reduce shortcut learning.
Why it matters
Remote-sensing foundation models often treat temporal observations via predefined relations or holistic reconstruction context. T-JEPA instead uses the actual elapsed time as supervisory signal, encouraging latent trajectories with time-gap-conditioned transition predictability while remaining compatible with single-frame downstream transfer.
Method
- Shared single-frame encoder with optional spatiotemporal metadata on the online pathway; target pathway uses a metadata-free EMA encoder to provide latent prediction targets.
- Temporal predictor takes a masked source latent plus a time-gap embedding (elapsed time) and outputs the complete target latent field; direct supervision across all forward source–target pairs avoids recursive rollout and error propagation.
- Complementary masked pixel reconstruction via a decoder branch; asymmetric metadata injection uses metadata dropout (p_meta=0.5) to mitigate shortcut learning.
Abstract (from arXiv)
Earth observation (EO) data provide rich temporal supervision, yet existing remote sensing foundation models mainly exploit sequential observations through imposing predefined pairwise relations or aggregating holistic reconstruction context. We seek to further exploit the sparse and nonuniform temporal sampling inherent in EO sequences as supervisory signals. To this end, we propose T-JEPA, a temporal joint-embedding predictive architecture that learns time-gap-conditioned latent transitions. A shared single-frame encoder processes each observation, while a temporal predictor estimates the complete target latent field from a masked source latent representation and the actual elapsed time. Across multiple temporal intervals, these predictive constraints organize observed states into structured latent trajectories. Asymmetric metadata injection mitigates shortcut learning, and direct supervision across multiple temporal scales proves more effective than recursively rolling out intermediate states. In parallel, masked pixel reconstruction provides complementary supervision for preserving spatial details. Under matched pre-training data and throughput, T-JEPA achieves leading transfer performance on both static and temporal tasks. Analyses further reveal that T-JEPA learns representations with time-gap-dependent transition predictability and coherent latent dynamics, while maintaining strong cross-period consistency, representation diversity, and semantic discriminability.
Related papers
- RS-JEPA: Remote Sensing Joint-Embedding Predictive Architecture with Scan-Path Views for Cross-Sensor Image Retrieval
- A Stochastic Latent Variable Generalization of Joint-Embedding Predictive Architectures
- Does Joint-Embedding Predictive Architecture Pretraining Help Time Series Forecasting?
- LEF-JEPA: Denoising-Trained Latent Energy Fields for Spatial Representation Learning
- Latent-Foresight: End-to-End Learning Predictable Representations for Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-05 · Indexed by Pub-AI: 2026-10-06 UTC
T-JEPA: A Temporal Joint-Embedding Predictive Architecture for Learning Better Remote Sensing Representations
Preprint posted: 2026-10-05 · Original record ↗
- arxiv
- 2610.05731
Explicit source versions (1)
- v1 · 2026-10-05
- Preprint available
Source date: 2026-10-05 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Earth observation (EO) data provide rich temporal supervision, yet existing remote sensing foundation models mainly exploit sequential observations through imposing predefined pairwise relations or aggregating holistic reconstruction context. We seek to further exploit the sparse and nonuniform temporal sampling inherent in EO sequences as supervisory signals. To this end, we propose T-JEPA, a temporal joint-embedding predictive architecture that learns time-gap-conditioned latent transitions. A shared single-frame encoder processes each observation, while a temporal predictor estimates the complete target latent field from a masked source latent representation and the actual elapsed time. Across multiple temporal intervals, these predictive constraints organize observed states into structured latent trajectories. Asymmetric metadata injection mitigates shortcut learning, and direct supervision across multiple temporal scales proves more effective than recursively rolling out intermediate states. In parallel, masked pixel reconstruction provides complementary supervision for preserving spatial details. Under matched pre-training data and throughput, T-JEPA achieves leading transfer performance on both static and temporal tasks. Analyses further reveal that T-JEPA learns representations with time-gap-dependent transition predictability and coherent latent dynamics, while maintaining strong cross-period consistency, representation diversity, and semantic discriminability.
