FareedKhan-dev/kimi-k3-in-c
The repo claims Kimi K3 can run byte-identical CPU inference from a 1.56 TB checkpoint with as little as 8.24 GB peak RAM.
The implementation is portable C99, with no BLAS, framework, GPU path, or AVX-512 requirement. It streams most weights from disk, keeping only selected dense trunk layers and cache data resident, so more RAM mainly improves tokens per second. The author reports 26.5 seconds per token at 8 GB and 5.6 seconds per token when the model fits in memory. The README also details validation gates against reference outputs and measurement data under `docs/data/`. GitHub · LLM repos' note.
The implementation is portable C99, with no BLAS, framework, GPU path, or AVX-512 requirement. It streams most weights from disk, keeping only selected dense trunk layers and cache data resident, so more RAM mainly improves tokens per second. The author reports 26.5 seconds per token at 8 GB and 5.6 seconds per token when the model fits in memory. The README also details validation gates against reference outputs and measurement data under `docs/data/`. GitHub · LLM repos' note.
score 4