Ensemble Routing for Prompt-Adaptive Cost-Aware LLM Inference
PISA-E beat the strongest ensemble competitors at selected operating points while cutting average inference cost by about 36% and 25% in two evaluation settings.
The system chooses which models to combine and how many to use for each prompt. Its cost–quality preference can be adjusted at inference time without retraining. Across four routing benchmarks, prioritizing quality produced macro-average task performance gains of 2.09 percentage points in-distribution and 2.87 points out-of-distribution over the strongest evaluated ensemble competitors. The lower-cost results came at selected operating points, rather than across every configuration.
HF Daily Papers' note
The system chooses which models to combine and how many to use for each prompt. Its cost–quality preference can be adjusted at inference time without retraining. Across four routing benchmarks, prioritizing quality produced macro-average task performance gains of 2.09 percentage points in-distribution and 2.87 points out-of-distribution over the strongest evaluated ensemble competitors. The lower-cost results came at selected operating points, rather than across every configuration.
HF Daily Papers' note
score 4