JEPA · 2026-10-02
Latent JEPA: Abstract Future Prediction for Latent Reasoning in Chemistry
Xinjian Zhao, Yaoyao Xu, Xuemin Chen, Xiaozhuang Song, Tianshu Yu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Latent-JEPA trains continuous latent reasoning in chemistry to anticipate abstract future solution aspects in embedding space, without verbalizing intermediate steps. The authors combine autoregressive learning with joint-embedding prediction of one or more future views (textual continuation and molecular outcomes). On ChemCoTBench they report gains in molecular optimization and several editing and reaction metrics, with representation analyses showing improved information about molecular outcomes and stronger correspondence with chemical structure.
Why it matters
The paper proposes a learning principle for connecting continuous latent reasoning with scientific outcomes by supervising internal “future” representations. For chemical reasoning, this provides a way to improve latent thoughts used for generation while also making latent representations more informative about molecular outcomes and their structure.
Method
- Latent-JEPA adds predictive branches to a continuous latent reasoner, supervising latent thoughts h_t to predict encoded representations of withheld future views at training time (using stop-gradient).
- For chemistry, it instantiates two complementary JEPA objectives: a decoder-native textual view predicting remaining reasoning/answer hidden representations, and a structure-native molecular view predicting pretrained molecular encoder embeddings of the future outcome.
- The authors retain Coconut-Chem’s autoregressive and counterfactual objectives and add the two predictive losses; at inference, prediction heads and target encoders are training-only.
Limitation
The paper does not provide a stated limitation in the provided text.
Abstract (from arXiv)
Large language models offer a promising foundation for chemical reasoning, bringing together chemical knowledge and multistep problem solving. Chemical intuition can provide an initial sense of plausible outcomes before the details of a solution are fully worked out. Inspired by how such expectations complement explicit analysis, we study how continuous latent thoughts can be trained to anticipate informative aspects of future solutions without verbalizing every intermediate step. We introduce Latent JEPA, a framework that combines autoregressive learning with joint-embedding prediction of one or more future views. For chemical reasoning, we develop textual and molecular prediction objectives that connect latent thoughts to both subsequent reasoning and molecular outcomes. Experiments on ChemCoTBench show gains in molecular optimization and on several editing and reaction metrics. Representation analyses show that future prediction makes latent thoughts more informative about molecular outcomes and strengthens their correspondence with chemical structure. These findings support abstract future prediction as a learning principle for connecting continuous latent reasoning with scientific outcomes.
Related papers
- Atom-JEPA: Joint-Embedding Predictive Architecture for 3D Atomistic Systems
- M-JEPA: Predictive Self-Supervised Learning for Molecular Graphs with Scaffold-Shift Evaluation on Tox21
- Parallelizable Probabilistic Latent Reasoning for Diverse Reasoning Trajectories
- Inside-JEPA: From Prediction to Relational Grounding in Latent World Models
- NeuroLens: Learning Latent Embeddings of Neural Semantics from Chronic Recordings
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-01 · Indexed by Pub-AI: 2026-10-02 UTC
Latent JEPA: Abstract Future Prediction for Latent Reasoning in Chemistry
Preprint posted: 2026-10-01 · Original record ↗
- arxiv
- 2610.01947
Explicit source versions (1)
- v1 · 2026-10-01
- Preprint available
Source date: 2026-10-01 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Large language models offer a promising foundation for chemical reasoning, bringing together chemical knowledge and multistep problem solving. Chemical intuition can provide an initial sense of plausible outcomes before the details of a solution are fully worked out. Inspired by how such expectations complement explicit analysis, we study how continuous latent thoughts can be trained to anticipate informative aspects of future solutions without verbalizing every intermediate step. We introduce Latent JEPA, a framework that combines autoregressive learning with joint-embedding prediction of one or more future views. For chemical reasoning, we develop textual and molecular prediction objectives that connect latent thoughts to both subsequent reasoning and molecular outcomes. Experiments on ChemCoTBench show gains in molecular optimization and on several editing and reaction metrics. Representation analyses show that future prediction makes latent thoughts more informative about molecular outcomes and strengthens their correspondence with chemical structure. These findings support abstract future prediction as a learning principle for connecting continuous latent reasoning with scientific outcomes.