TRACE-ROUTER: Task-Consistent and Adaptive Online Routing for Agentic AI
TRACE-Router routes an entire agent task to one model, then learns from the task’s final outcome instead of each individual LLM call.
The paper argues that per-call routing is a poor fit for long agent workflows, where success is only visible at the end. TRACE-Router uses a contextual bandit at task admission, pins later calls to the chosen backend, and updates on terminal reward combining accuracy and latency. In reported benchmarks, it improves the accuracy-latency trade-off, including 7-8 accuracy points over latency-matched interpolation on tau2-Bench and 7.1 points over the strongest single-model baseline on Terminal-Bench with 36% lower latency. ArXiv · AI/CL/LG's note
The paper argues that per-call routing is a poor fit for long agent workflows, where success is only visible at the end. TRACE-Router uses a contextual bandit at task admission, pins later calls to the chosen backend, and updates on terminal reward combining accuracy and latency. In reported benchmarks, it improves the accuracy-latency trade-off, including 7-8 accuracy points over latency-matched interpolation on tau2-Bench and 7.1 points over the strongest single-model baseline on Terminal-Bench with 36% lower latency. ArXiv · AI/CL/LG's note
score 5