Beyond Policy Alignment: Closing the Planning-Learning Loop for Robot Control with Learned World Models
The authors propose PL-MPC, an extension of policy-constrained TD-M(PC)2 that modifies (1) critic supervision via hybrid multi-step TD targets, (2) MPC terminal-value evaluation via disagreement-aware terminal estimates, and (3) actor learning via return-weighted actor distillation from planner-executed actions. They report largest gains on HumanoidBench balance-hard and hurdle, and higher zero-shot sim-to-real success on wrench–nut alignment.
