Driving · 2026-11-04

Decision Transformer with Task-Aware Representations for Offline Meta-Reinforcement Learning

Xinbo Zhao, Yuhang Liu, Xin Zhang, Lei Le, Zhaohan Xi, Yingxue Zhang

arXiv:2611.04010PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

CoTA-DT is a DT-based offline meta-RL method that learns task latents from four joint signals: contrastive trajectory encoding, alignment to natural-language task descriptions, one-step supervision from a task-conditioned world model, and long-term supervision from a task-conditioned Q function. A language-grounded self-adaptive strategy enables zero-shot and few-shot adaptation to unseen tasks. The pretrained LLM backbone is shared across components with a LoRA adapter.

Why it matters

Offline meta-RL needs reliable task identification from limited offline data, which can be noisy or ambiguous. The authors propose multi-perspective task representations (trajectory, language, transitions, and returns) to improve discrimination and generalization, and integrate task inference directly into a DT-based policy for zero-shot and few-shot adaptation.

Method

  • Learns task latents z using multi-perspective training: contrastive InfoNCE on offline trajectory segments plus CLIP-style alignment between trajectory-derived representations and LLM-encoded natural-language descriptions.
  • Refines task representations using a task-conditioned world model for one-step reward/next-state prediction and a task-conditioned Q function for long-horizon return-to-go regression, with losses backpropagated through the latent encoder.
  • Trains a DT-based policy conditioned on the task latent; zero-shot uses language-grounded latent inversion and self-adaptive rollout, while few-shot initializes from support trajectories and can optionally fine-tune the policy.

Limitation

The authors do not state any explicit limitation in the provided text.

Abstract (from arXiv)

To enable effective generalization in offline meta-RL, it is essential to accurately infer the underlying task through latent task representations, which provide a compact and transferable summary of task-specific characteristics for guiding policy generation. In this paper, we propose Contrastive Task-Aware Decision Transformer (CoTA-DT), a DT-based offline meta-RL framework that learns task representations by jointly integrating four complementary components: (i) contrastive learning from offline trajectories, (ii) alignment with natural-language task descriptions, (iii) one-step supervision via a task-conditioned world model, and (iv) long-term supervision via a task-conditioned Q function. The learned task representations are used as conditioning signals for a DT-based policy, enabling task-aware meta-policy learning. To further support generalization in both zero-shot and few-shot settings, we design a language-grounded, self-adaptive strategy that enables effective and reliable adaptation to unseen tasks. Notably, in CoTA-DT, the pretrained LLM serves as a shared backbone for language encoding, world modeling, Q-function estimation, and policy learning. This unified design facilitates parameter sharing, improving both efficiency and generalization. Extensive experiments across diverse offline meta-RL benchmarks demonstrate that CoTA-DT consistently outperforms prior methods, particularly in challenging zero-shot generalization scenarios.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-04 · Indexed by Pub-AI: 2026-11-04 UTC

preprint

Decision Transformer with Task-Aware Representations for Offline Meta-Reinforcement Learning

Xinbo Zhao, Yuhang Liu, Xin Zhang, Lei Le, Zhaohan Xi, Yingxue Zhang

First announced by arXiv: 2026-11-04 · Original record ↗

arxiv
2611.04010
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-04 · First detected: 2026-11-04 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.04010v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

To enable effective generalization in offline meta-RL, it is essential to accurately infer the underlying task through latent task representations, which provide a compact and transferable summary of task-specific characteristics for guiding policy generation. In this paper, we propose Contrastive Task-Aware Decision Transformer (CoTA-DT), a DT-based offline meta-RL framework that learns task representations by jointly integrating four complementary components: (i) contrastive learning from offline trajectories, (ii) alignment with natural-language task descriptions, (iii) one-step supervision via a task-conditioned world model, and (iv) long-term supervision via a task-conditioned Q function. The learned task representations are used as conditioning signals for a DT-based policy, enabling task-aware meta-policy learning. To further support generalization in both zero-shot and few-shot settings, we design a language-grounded, self-adaptive strategy that enables effective and reliable adaptation to unseen tasks. Notably, in CoTA-DT, the pretrained LLM serves as a shared backbone for language encoding, world modeling, Q-function estimation, and policy learning. This unified design facilitates parameter sharing, improving both efficiency and generalization. Extensive experiments across diverse offline meta-RL benchmarks demonstrate that CoTA-DT consistently outperforms prior methods, particularly in challenging zero-shot generalization scenarios.