LuthorSQL: A One-Pass Text-to-SQL Model as System 1 for a Slower LLM Agent
LuthorSQL handled nearly half of database questions locally, cutting API costs by 33.3% without reducing accuracy in the authors’ test.
The confidence-based cascade paired the local 7B model with Claude Opus 5.5 on 227 questions from databases published after Opus’s training cutoff. LuthorSQL generates query-plan scores in one pass, runs up to 32 candidates and uses a trained validator to select an answer, escalating uncertain questions to Opus. On its own, it achieved 68.03% execution accuracy on BIRD’s hidden test set at 0.84 seconds per question on a single GPU. With 8-bit weights, it retained its accuracy while fitting in 9 GB of memory.
ArXiv · AI/CL/LG's note
The confidence-based cascade paired the local 7B model with Claude Opus 5.5 on 227 questions from databases published after Opus’s training cutoff. LuthorSQL generates query-plan scores in one pass, runs up to 32 candidates and uses a trained validator to select an answer, escalating uncertain questions to Opus. On its own, it achieved 68.03% execution accuracy on BIRD’s hidden test set at 0.84 seconds per question on a single GPU. With 8-bit weights, it retained its accuracy while fitting in 9 GB of memory.
ArXiv · AI/CL/LG's note
score 5