PreSTO: Predictive Subtree Prefetching for Fast LLM Power Sampling
PreSTO keeps PowerMH’s sampling rule intact while batching likely future accept/reject paths to cut model-call overhead.
The paper frames PowerMH’s bottleneck as sequential Metropolis-Hastings updates, where each decision determines the next proposal. PreSTO prefetches a subtree of possible future outcomes in one batch, then follows the realized path using the original acceptance rule. The authors report median speedups of 1.71x for PowerMH, 1.91x for EntropyCut, and 2.33x for MultiTryMH across seven datasets and five base models. HF Daily Papers' note
The paper frames PowerMH’s bottleneck as sequential Metropolis-Hastings updates, where each decision determines the next proposal. PreSTO prefetches a subtree of possible future outcomes in one batch, then follows the realized path using the original acceptance rule. The authors report median speedups of 1.71x for PowerMH, 1.91x for EntropyCut, and 2.33x for MultiTryMH across seven datasets and five base models. HF Daily Papers' note
score 4