SparseDitto: Customizing GPU Kernels for Different Sparsity Patterns with LLM-Based Agentic System
SparseDitto uses an LLM-driven agent system to generate GPU sparse-kernel designs tailored to each matrix, operator, and target GPU.
The paper argues that sparse GPU performance varies sharply with sparsity pattern and format, citing a 350x cuSPARSE gap between CSR and Blocked-ELL for the same SpMM case. SparseDitto ranks known strategies from matrix structure, plans architecture-aware candidates, then has coding and verification agents implement and refine them with target-GPU measurements. Reported geometric-mean speedups are 2.68x over cuSPARSE on an NVIDIA RTX PRO 6000 and 2.79x on an NVIDIA H200, with larger best-case gains. Its generated SpMM kernels also speed full-batch GCN training by up to 3.39x. ArXiv · AI/CL/LG's note
The paper argues that sparse GPU performance varies sharply with sparsity pattern and format, citing a 350x cuSPARSE gap between CSR and Blocked-ELL for the same SpMM case. SparseDitto ranks known strategies from matrix structure, plans architecture-aware candidates, then has coding and verification agents implement and refine them with target-GPU measurements. Reported geometric-mean speedups are 2.68x over cuSPARSE on an NVIDIA RTX PRO 6000 and 2.79x on an NVIDIA H200, with larger best-case gains. Its generated SpMM kernels also speed full-batch GCN training by up to 3.39x. ArXiv · AI/CL/LG's note
score 5