Kimi K3 (2.8T) at 1 token/s on a MacBook Pro, streamed from four SSDs
Deltafin reports full Kimi K3 running from one 128GB M5 Max MacBook Pro, but the first token is still minutes away.
The ARGODRIVE fork streamed 1.45TB of K3 expert weights from four SSDs and measured 1.00 token/s steady decode over a 512-token answer. A 512-token prompt took about 6.3 minutes to first token because prefill re-reads each layer’s experts eight times; the fix is planned, not built. The authors say the drive scaling is not simply total bandwidth: one drive reached about 52% of four-drive speed, two about 73%, and three about 90%. Each benchmark is described as one cold run with exact prompts, logs, and placement manifests published in the repo. HN · GitHub AI's note
The ARGODRIVE fork streamed 1.45TB of K3 expert weights from four SSDs and measured 1.00 token/s steady decode over a 512-token answer. A 512-token prompt took about 6.3 minutes to first token because prefill re-reads each layer’s experts eight times; the fix is planned, not built. The authors say the drive scaling is not simply total bandwidth: one drive reached about 52% of four-drive speed, two about 73%, and three about 90%. Each benchmark is described as one cold run with exact prompts, logs, and placement manifests published in the repo. HN · GitHub AI's note
score 4
Discussions
- hn · 279 points · 156 comments