Driving · 2026-11-04
Decision Transformer with Task-Aware Representations for Offline Meta-Reinforcement Learning
Xinbo Zhao, Yuhang Liu, Xin Zhang, Lei Le, Zhaohan Xi, Yingxue Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CoTA-DT is a DT-based offline meta-RL method that learns task latents from four joint signals: contrastive trajectory encoding, alignment to natural-language task descriptions, one-step supervision from a task-conditioned world model, and long-term supervision from a task-conditioned Q function. A language-grounded self-adaptive strategy enables zero-shot and few-shot adaptation to unseen tasks. The pretrained LLM backbone is shared across components with a LoRA adapter.
Why it matters
Offline meta-RL needs reliable task identification from limited offline data, which can be noisy or ambiguous. The authors propose multi-perspective task representations (trajectory, language, transitions, and returns) to improve discrimination and generalization, and integrate task inference directly into a DT-based policy for zero-shot and few-shot adaptation.
Method
- Learns task latents z using multi-perspective training: contrastive InfoNCE on offline trajectory segments plus CLIP-style alignment between trajectory-derived representations and LLM-encoded natural-language descriptions.
- Refines task representations using a task-conditioned world model for one-step reward/next-state prediction and a task-conditioned Q function for long-horizon return-to-go regression, with losses backpropagated through the latent encoder.
- Trains a DT-based policy conditioned on the task latent; zero-shot uses language-grounded latent inversion and self-adaptive rollout, while few-shot initializes from support trajectories and can optionally fine-tune the policy.
Limitation
The authors do not state any explicit limitation in the provided text.
Abstract (from arXiv)
To enable effective generalization in offline meta-RL, it is essential to accurately infer the underlying task through latent task representations, which provide a compact and transferable summary of task-specific characteristics for guiding policy generation. In this paper, we propose Contrastive Task-Aware Decision Transformer (CoTA-DT), a DT-based offline meta-RL framework that learns task representations by jointly integrating four complementary components: (i) contrastive learning from offline trajectories, (ii) alignment with natural-language task descriptions, (iii) one-step supervision via a task-conditioned world model, and (iv) long-term supervision via a task-conditioned Q function. The learned task representations are used as conditioning signals for a DT-based policy, enabling task-aware meta-policy learning. To further support generalization in both zero-shot and few-shot settings, we design a language-grounded, self-adaptive strategy that enables effective and reliable adaptation to unseen tasks. Notably, in CoTA-DT, the pretrained LLM serves as a shared backbone for language encoding, world modeling, Q-function estimation, and policy learning. This unified design facilitates parameter sharing, improving both efficiency and generalization. Extensive experiments across diverse offline meta-RL benchmarks demonstrate that CoTA-DT consistently outperforms prior methods, particularly in challenging zero-shot generalization scenarios.
Related papers
- DT-JEPA: Learning to Predict Future Representations for Decision-Making
- From Suboptimal to Optimal: One-Step Latent Flow Policies for Offline Reinforcement Learning
- Reinforcement Learning-driven Diffusion Models: Language, Vision, and Beyond
- Model What Matters: Decision-Guided Selective World Modeling for Language Agents
- DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-04 · Indexed by Pub-AI: 2026-11-04 UTC
Decision Transformer with Task-Aware Representations for Offline Meta-Reinforcement Learning
First announced by arXiv: 2026-11-04 · Original record ↗
- arxiv
- 2611.04010
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-04 · First detected: 2026-11-04 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.04010v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
To enable effective generalization in offline meta-RL, it is essential to accurately infer the underlying task through latent task representations, which provide a compact and transferable summary of task-specific characteristics for guiding policy generation. In this paper, we propose Contrastive Task-Aware Decision Transformer (CoTA-DT), a DT-based offline meta-RL framework that learns task representations by jointly integrating four complementary components: (i) contrastive learning from offline trajectories, (ii) alignment with natural-language task descriptions, (iii) one-step supervision via a task-conditioned world model, and (iv) long-term supervision via a task-conditioned Q function. The learned task representations are used as conditioning signals for a DT-based policy, enabling task-aware meta-policy learning. To further support generalization in both zero-shot and few-shot settings, we design a language-grounded, self-adaptive strategy that enables effective and reliable adaptation to unseen tasks. Notably, in CoTA-DT, the pretrained LLM serves as a shared backbone for language encoding, world modeling, Q-function estimation, and policy learning. This unified design facilitates parameter sharing, improving both efficiency and generalization. Extensive experiments across diverse offline meta-RL benchmarks demonstrate that CoTA-DT consistently outperforms prior methods, particularly in challenging zero-shot generalization scenarios.
