Adaptive Quantization Preserves Fast Rates for TD Learning over Rate-Limited Channels
The authors prove that O(d log T) transmitted bits suffice to reach an error rate of roughly 1/T, up to logarithmic factors.
The result covers policy evaluation with linear function approximation, where an agent sends quantized temporal-difference updates to a server. Their BAQ-TD algorithm batches updates to reduce variance and adaptively quantizes innovations. Above a finite channel-capacity threshold, its high-probability error bound matches the convergence rate of unquantized TD up to logarithmic factors. The authors report an exponential improvement in communication complexity over prior work.
ArXiv · AI/CL/LG's note
The result covers policy evaluation with linear function approximation, where an agent sends quantized temporal-difference updates to a server. Their BAQ-TD algorithm batches updates to reduce variance and adaptively quantizes innovations. Above a finite channel-capacity threshold, its high-probability error bound matches the convergence rate of unquantized TD up to logarithmic factors. The authors report an exponential improvement in communication complexity over prior work.
ArXiv · AI/CL/LG's note
score 4