JEPA · 2026-10-06

T-JEPA: A Temporal Joint-Embedding Predictive Architecture for Learning Better Remote Sensing Representations

Bowen Peng, Li Liu, Yongxiang Liu, Weijie Li, Jie Zhou, Zhen Liu

arXiv:2610.05731PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

T-JEPA learns remote-sensing representations from sparsely and nonuniformly sampled Earth observation sequences by predicting time-gap-conditioned latent transitions. A shared single-frame encoder produces masked source latents, and a temporal predictor estimates the complete target latent field conditioned on elapsed time, supervised across all temporally ordered pairs at multiple temporal scales. Asymmetric metadata injection plus masked pixel reconstruction improve learning and reduce shortcut learning.

Why it matters

Remote-sensing foundation models often treat temporal observations via predefined relations or holistic reconstruction context. T-JEPA instead uses the actual elapsed time as supervisory signal, encouraging latent trajectories with time-gap-conditioned transition predictability while remaining compatible with single-frame downstream transfer.

Method

  • Shared single-frame encoder with optional spatiotemporal metadata on the online pathway; target pathway uses a metadata-free EMA encoder to provide latent prediction targets.
  • Temporal predictor takes a masked source latent plus a time-gap embedding (elapsed time) and outputs the complete target latent field; direct supervision across all forward source–target pairs avoids recursive rollout and error propagation.
  • Complementary masked pixel reconstruction via a decoder branch; asymmetric metadata injection uses metadata dropout (p_meta=0.5) to mitigate shortcut learning.
Abstract (from arXiv)

Earth observation (EO) data provide rich temporal supervision, yet existing remote sensing foundation models mainly exploit sequential observations through imposing predefined pairwise relations or aggregating holistic reconstruction context. We seek to further exploit the sparse and nonuniform temporal sampling inherent in EO sequences as supervisory signals. To this end, we propose T-JEPA, a temporal joint-embedding predictive architecture that learns time-gap-conditioned latent transitions. A shared single-frame encoder processes each observation, while a temporal predictor estimates the complete target latent field from a masked source latent representation and the actual elapsed time. Across multiple temporal intervals, these predictive constraints organize observed states into structured latent trajectories. Asymmetric metadata injection mitigates shortcut learning, and direct supervision across multiple temporal scales proves more effective than recursively rolling out intermediate states. In parallel, masked pixel reconstruction provides complementary supervision for preserving spatial details. Under matched pre-training data and throughput, T-JEPA achieves leading transfer performance on both static and temporal tasks. Analyses further reveal that T-JEPA learns representations with time-gap-dependent transition predictability and coherent latent dynamics, while maintaining strong cross-period consistency, representation diversity, and semantic discriminability.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-05 · Indexed by Pub-AI: 2026-10-06 UTC

preprint

T-JEPA: A Temporal Joint-Embedding Predictive Architecture for Learning Better Remote Sensing Representations

Bowen Peng, Li Liu, Yongxiang Liu, Weijie Li, Jie Zhou, Zhen Liu

Preprint posted: 2026-10-05 · Original record ↗

arxiv
2610.05731
Explicit source versions (1)
  • v1 · 2026-10-05
  1. Preprint available

    Source date: 2026-10-05 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (1)
Abstract · arxiv

Earth observation (EO) data provide rich temporal supervision, yet existing remote sensing foundation models mainly exploit sequential observations through imposing predefined pairwise relations or aggregating holistic reconstruction context. We seek to further exploit the sparse and nonuniform temporal sampling inherent in EO sequences as supervisory signals. To this end, we propose T-JEPA, a temporal joint-embedding predictive architecture that learns time-gap-conditioned latent transitions. A shared single-frame encoder processes each observation, while a temporal predictor estimates the complete target latent field from a masked source latent representation and the actual elapsed time. Across multiple temporal intervals, these predictive constraints organize observed states into structured latent trajectories. Asymmetric metadata injection mitigates shortcut learning, and direct supervision across multiple temporal scales proves more effective than recursively rolling out intermediate states. In parallel, masked pixel reconstruction provides complementary supervision for preserving spatial details. Under matched pre-training data and throughput, T-JEPA achieves leading transfer performance on both static and temporal tasks. Analyses further reveal that T-JEPA learns representations with time-gap-dependent transition predictability and coherent latent dynamics, while maintaining strong cross-period consistency, representation diversity, and semantic discriminability.