Lifted Bellman Linear Programming for Offline Reinforcement Learning
The paper proposes replacing bootstrapped critic regression with in-sample Bellman inequality constraints for offline RL.
Its Lifted Bellman Linear Program works in joint `(Q,V)` space so each constraint uses only state-action pairs from the dataset. The authors say the formulation keeps the same in-sample optimum even when constraints span `K`-step trajectory segments, without off-policy correction. Their neural relaxation, ALBUM, trains without target networks or EMA updates. On OGBench, it matches FQL on average and uses fewer parameters and less peak GPU memory than the compared methods. ArXiv · AI/CL/LG's note
Its Lifted Bellman Linear Program works in joint `(Q,V)` space so each constraint uses only state-action pairs from the dataset. The authors say the formulation keeps the same in-sample optimum even when constraints span `K`-step trajectory segments, without off-policy correction. Their neural relaxation, ALBUM, trains without target networks or EMA updates. On OGBench, it matches FQL on average and uses fewer parameters and less peak GPU memory than the compared methods. ArXiv · AI/CL/LG's note
score 4