Latent Dynamics · 2026-10-22
Parallelizable Probabilistic Latent Reasoning for Diverse Reasoning Trajectories
Seokhyun Lee, Seongjun Lee, Changhee Lee
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PLRM (Probabilistic Latent Reasoning Models) moves reasoning from verbal chain-of-thought into a compact continuous latent space using a variational, JEPA-style objective. A parallelizable posterior infers latent distributions from observed reasoning traces, while a problem-conditioned prior predicts the next latent distribution via teacher forcing. A discrete strategy variable with a mixture-based prior models multimodal reasoning trajectories without collapsing them.
Why it matters
The authors argue that existing latent reasoning is often deterministic and cannot represent a distribution over multiple valid reasoning trajectories, and that stochastic approaches can be hard to optimize and may require autoregressive latent rollouts that prevent parallelization. PLRM targets these issues with a variational objective that supports parallel target latent construction and with mixture/strategy modeling intended to preserve Gaussian-mixture structure induced by multiple trajectories, improving multi-trajectory distributional modeling.
Method
- Introduce a variational latent reasoning formulation where a factorized posterior qϕ(z|x,τ) is inferred in parallel from observed CoT traces and trained with an ELBO against a problem-conditioned prior pθ(z|x) via teacher-forced next-latent distribution prediction.
- Adopt a JEPA-style learning setup: the posterior provides target latent distributions and the prior predicts them; latent representations are shaped to retain answer-relevant reasoning information while abstracting trace-specific verbalization.
- Model heterogeneous multi-trajectory reasoning using a discrete strategy variable and a mixture-based prior to preserve multimodality (avoid collapsing distinct modes into a single Gaussian).
Abstract (from arXiv)
Language models relying on verbalized Chain-of-Thought (CoT) are constrained to token-by-token generation in the language space, where many tokens serve textual coherence rather than core reasoning. This entangles genuine differences in logical trajectories with superficial variations in verbalization. Moving reasoning into a compact, continuous latent space offers a promising alternative for efficiently representing and exploring diverse reasoning trajectories. However, current latent models remain largely deterministic, capturing only a single reasoning trajectory rather than modeling a distribution over multiple valid alternatives. While stochastic approaches parameterize distributions over latent reasoning steps, they frequently suffer from unstable optimization and limited empirical performance. Furthermore, their reliance on autoregressive latent prediction during training precludes effective parallelization, leading to prohibitively long training times. To overcome these bottlenecks, we propose Probabilistic Latent Reasoning Models (PLRM), a parallelizable variational framework for probabilistic latent reasoning. PLRM adopts a JEPA-style objective in which the posterior infers latent distributions in parallel from observed reasoning trajectories, while the prior learns to predict the subsequent latent distribution via teacher forcing. To model heterogeneous reasoning processes, we introduce a discrete strategy variable together with a mixture-based prior, allowing the model to preserve the Gaussian-mixture structure induced by multiple distinct reasoning trajectories rather than collapsing them into a single Gaussian. Across diverse benchmarks, PLRM achieves performance competitive with deterministic latent reasoning models on single-trajectory datasets while enabling efficient training and consistently outperforming competing stochastic baselines on multi-trajectory datasets.
Related papers
- Unrolled Flow Models for Reasoning
- MDN-VJEPA: Multimodal Probabilistic Joint-Embedding World Models
- TrajEvolve: Trajectory Evolution for Reinforcement Learning with Hindsight Credit Assignment
- Looped Reasoning Finishes Earlier Than You Think
- Recompose and Refine Latent Reasoning Flows for Vision-Language-Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-21 · Indexed by Pub-AI: 2026-10-22 UTC
Parallelizable Probabilistic Latent Reasoning for Diverse Reasoning Trajectories
Preprint posted: 2026-10-21 · Original record ↗
- arxiv
- 2610.29385
Explicit source versions (1)
- v2 · 2026-10-22
- Preprint available
Source date: 2026-10-21 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-22 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Language models relying on verbalized Chain-of-Thought (CoT) are constrained to token-by-token generation in the language space, where many tokens serve textual coherence rather than core reasoning. This entangles genuine differences in logical trajectories with superficial variations in verbalization. Moving reasoning into a compact, continuous latent space offers a promising alternative for efficiently representing and exploring diverse reasoning trajectories. However, current latent models remain largely deterministic, capturing only a single reasoning trajectory rather than modeling a distribution over multiple valid alternatives. While stochastic approaches parameterize distributions over latent reasoning steps, they frequently suffer from unstable optimization and limited empirical performance. Furthermore, their reliance on autoregressive latent prediction during training precludes effective parallelization, leading to prohibitively long training times. To overcome these bottlenecks, we propose Probabilistic Latent Reasoning Models (PLRM), a parallelizable variational framework for probabilistic latent reasoning. PLRM adopts a JEPA-style objective in which the posterior infers latent distributions in parallel from observed reasoning trajectories, while the prior learns to predict the subsequent latent distribution via teacher forcing. To model heterogeneous reasoning processes, we introduce a discrete strategy variable together with a mixture-based prior, allowing the model to preserve the Gaussian-mixture structure induced by multiple distinct reasoning trajectories rather than collapsing them into a single Gaussian. Across diverse benchmarks, PLRM achieves performance competitive with deterministic latent reasoning models on single-trajectory datasets while enabling efficient training and consistently outperforming competing stochastic baselines on multi-trajectory datasets.