GPU-parallel differentiable MPC via log-barrier-smoothed ADMM inner solves combined with first-order actor-critic training using cached rollout solutions
Use a differentiable MPC layer that blends SQP with a log-barrier-smoothed ADMM inner solver to obtain smoother gradients and exact sensitivities when the inner solve converges. Then train the resulting MPC policy with a first-order actor-critic method (Short-Horizon Actor-Critic) that reuses rollout MPC solutions to avoid repeatedly solving/evaluating MPC during training.
Discovered 2026-11-10 · Sources checked 2026-11-10
Why it matters
Learning MPC policies is often hindered by nonsmooth gradient behavior at active-set transitions and by the cost of repeatedly evaluating an optimization solve. Smoothing the ADMM updates with a barrier improves gradient smoothness, while reusing rollout MPC solutions reduces redundant policy evaluation work during RL training.
What changed
The approach (1) introduces a GPU-parallel differentiable MPC solver that replaces an unsmoothed inner method with a log-barrier-smoothed ADMM procedure inside an SQP-based outer loop, and (2) couples that solver with a short-horizon, first-order actor-critic RL algorithm that uses cached rollout MPC solutions instead of re-evaluating MPC from scratch at each training step.
Where it helps
- End-to-end learning of constraint-aware control policies with MPC structure
- GPU-accelerated differentiable control optimization in robotics or racing-like dynamics
- First-order RL training pipelines that can leverage cached MPC rollout solutions
Evidence & reproduction
Reproduction not evaluated. A linked paper or repository does not establish independent reproduction.
Limitations & open questions
The method’s benefits depend on using a differentiable MPC formulation with an inner ADMM solver that can be barrier-smoothed and on having an RL training setup where rollout MPC solutions can be reused effectively.
Sources
- Efficient Learning of Differentiable MPC Policies on the GPU via Smoothed ADMM and First-Order Reinforcement Learning ↗
arXiv · primary · Checked 2026-11-10
