SPECTRA: Adaptive Execution of Speculative Decoding on a Runtime-Reconfigurable Tiled Architecture
SPECTRA targets the awkward middle of speculative decoding, where verification is neither cleanly memory-bound nor compute-bound.
The paper proposes a tiled FPGA architecture that can switch execution modes per kernel: systolic for GEMM-heavy work and vector lanes for GEMV-heavy decoding. It also adjusts tile count, partitioning, and communication patterns at runtime across the speculative decoding pipeline. On a 20-tile FPGA prototype tested with Pythia, SmolLM2, and GPT-2 model families, the authors report up to 2.09x speedup from tile-level reconfiguration and another 1.25x from system-level adaptability over fixed designs. The work is listed as accepted at ICCAD 2026. ArXiv · AI/CL/LG's note
The paper proposes a tiled FPGA architecture that can switch execution modes per kernel: systolic for GEMM-heavy work and vector lanes for GEMV-heavy decoding. It also adjusts tile count, partitioning, and communication patterns at runtime across the speculative decoding pipeline. On a 20-tile FPGA prototype tested with Pythia, SmolLM2, and GPT-2 model families, the authors report up to 2.09x speedup from tile-level reconfiguration and another 1.25x from system-level adaptability over fixed designs. The work is listed as accepted at ICCAD 2026. ArXiv · AI/CL/LG's note
score 5