SiliconBench: Speed, Memory, and Fidelity for LLM Serving on Unified-Memory Desktops
Only three of the tested Apple Silicon serving stacks cleared the paper’s completion, fidelity, and model-coverage bar.
SiliconBench measures nine Apple Silicon LLM serving engines on speed, memory use, and fidelity, rather than throughput alone. The authors test chat and agent workloads on Qwen3, Qwen3.5, and Gemma 4, with quality checked against an NVIDIA reference. vllm-metal more than doubles throughput from concurrency 1 to 16 on Qwen3-0.6B, but CUDA vLLM and SGLang scale better on the same prompts. The paper also finds that explicit memory budgets can still leave systems near physical-capacity limits, and that Thunderbolt RDMA tensor parallelism scales in the tested two-machine setup while TCP pipeline parallelism regresses. HF Daily Papers' note
SiliconBench measures nine Apple Silicon LLM serving engines on speed, memory use, and fidelity, rather than throughput alone. The authors test chat and agent workloads on Qwen3, Qwen3.5, and Gemma 4, with quality checked against an NVIDIA reference. vllm-metal more than doubles throughput from concurrency 1 to 16 on Qwen3-0.6B, but CUDA vLLM and SGLang scale better on the same prompts. The paper also finds that explicit memory budgets can still leave systems near physical-capacity limits, and that Thunderbolt RDMA tensor parallelism scales in the tested two-machine setup while TCP pipeline parallelism regresses. HF Daily Papers' note
score 4