SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs
SelectInfer cuts edge-device LLM cost by loading and computing only selected neurons.
The paper proposes an offline profiler that identifies task-specific and general-purpose neurons before deployment. At inference time, the system selectively loads important neuron subsets to reduce memory use, then dynamically computes only the most relevant neurons. The authors say evaluations across multiple datasets showed lower memory footprint and computation while preserving task performance. ArXiv · AI/CL/LG's note
The paper proposes an offline profiler that identifies task-specific and general-purpose neurons before deployment. At inference time, the system selectively loads important neuron subsets to reduce memory use, then dynamically computes only the most relevant neurons. The authors say evaluations across multiple datasets showed lower memory footprint and computation while preserving task performance. ArXiv · AI/CL/LG's note
score 5