Flide: Accelerating Decision Inference with Exact State Forking
Flide delivers up to 9.5× faster decision queries than vLLM and SGLang in the authors’ evaluations.
The engine targets categorical reasoning workloads, where existing inference engines can repeatedly compute shared prefixes or distort output probabilities. Its ForkAttention mechanism executes branching requests in a flattened token stream while preserving branch dependencies through GPU-resident indices. In exact arithmetic, each branch is guaranteed to match an isolated forward pass. Tests on public hybrid and decision models also placed Flide closest to a float32 reference in total variation distance.
HF Daily Papers' note
The engine targets categorical reasoning workloads, where existing inference engines can repeatedly compute shared prefixes or distort output probabilities. Its ForkAttention mechanism executes branching requests in a flattened token stream while preserving branch dependencies through GPU-resident indices. In exact arithmetic, each branch is guaranteed to match an isolated forward pass. Tests on public hybrid and decision models also placed Flide closest to a float32 reference in total variation distance.
HF Daily Papers' note
score 5