Scheduling Mixed RL Rollouts Beyond Prefix Locality
MISA-T targets KV-cache contention when different RL rollout types share the same inference service.
The paper says prefix-aware routing helps reuse cache, but does not decide how RLVR, RLHF, and agentic sessions should compete for KV capacity. MISA-T adds adaptive admission, workload-aware KV allocation, and accounting for how long sessions keep KV resident. In ablations on Step3.7 and Qwen3.6-35B-A3B, it reports rollout throughput gains of 53.3% and 43.6% over a tuned cache-aware vLLM Router. In a 50-iteration Step3.7 run, it raised throughput 35.6% and cut mean iteration time 22.8% while keeping the workload mix close to the trainer target. ArXiv · AI/CL/LG's note
The paper says prefix-aware routing helps reuse cache, but does not decide how RLVR, RLHF, and agentic sessions should compete for KV capacity. MISA-T adds adaptive admission, workload-aware KV allocation, and accounting for how long sessions keep KV resident. In ablations on Step3.7 and Qwen3.6-35B-A3B, it reports rollout throughput gains of 53.3% and 43.6% over a tuned cache-aware vLLM Router. In a 50-iteration Step3.7 run, it raised throughput 35.6% and cut mean iteration time 22.8% while keeping the workload mix close to the trainer target. ArXiv · AI/CL/LG's note
score 5