The Surprising Effectiveness of Approximate Value Iteration in Self-Play
A minimal AVI self-play setup beat AlphaZero on value accuracy in the paper’s tested games, while staying cheaper to train and run.
The authors tested Connect Four, Hex(7x7), synthetic games, and preliminary runs on Othello and Go(9x9). They report that AVI learned more accurate value functions than AlphaZero under exact oracle evaluation. Its one-step-lookahead greedy policies stayed competitive with MCTS-based policies at substantially lower training and inference cost. The paper argues that MCTS-heavy methods may have drawn attention away from simpler approaches now helped by modern deep-learning tooling. ArXiv · AI/CL/LG's note
The authors tested Connect Four, Hex(7x7), synthetic games, and preliminary runs on Othello and Go(9x9). They report that AVI learned more accurate value functions than AlphaZero under exact oracle evaluation. Its one-step-lookahead greedy policies stayed competitive with MCTS-based policies at substantially lower training and inference cost. The paper argues that MCTS-heavy methods may have drawn attention away from simpler approaches now helped by modern deep-learning tooling. ArXiv · AI/CL/LG's note
score 5