Near-Optimal Reinforcement Learning with Multi-Step Transition Lookahead
Exact planning stays hard, but fixed-depth lookahead can still be learned efficiently.
The paper shows NP-hardness for exact planning with multi-step transition lookahead at every fixed rational discount factor between 0 and 1. It then gives a randomized polynomial-time approximation scheme when the lookahead depth is fixed. For unknown transitions and stochastic rewards, the authors add optimism and variance-adaptive confidence bounds. Their regret bound matches classical tabular discounted RL in the leading term up to logarithmic factors. Source: ArXiv · AI/CL/LG's note.
The paper shows NP-hardness for exact planning with multi-step transition lookahead at every fixed rational discount factor between 0 and 1. It then gives a randomized polynomial-time approximation scheme when the lookahead depth is fixed. For unknown transitions and stochastic rewards, the authors add optimism and variance-adaptive confidence bounds. Their regret bound matches classical tabular discounted RL in the leading term up to logarithmic factors. Source: ArXiv · AI/CL/LG's note.
score 5