Cerebras Deploys Trillion-Parameter AI Models
Cerebras’s worked example fits a roughly 3-trillion-parameter model across 59 wafers, keeping its weights in on-chip memory.
The Kimi K3 example assumes 4-bit weights, an 8-bit KV cache and an average request size of 256,000 tokens. Each wafer holds one dense-attention layer or two linear-attention layers, leaving room for request state. Only activations pass between wafers, with Cerebras putting Ethernet transfer latency at roughly two microseconds per hop. Multiple requests occupy different pipeline stages simultaneously; available KV-cache memory on dense-layer wafers limits concurrency.
HN · Frontpage AI's note
The Kimi K3 example assumes 4-bit weights, an 8-bit KV cache and an average request size of 256,000 tokens. Each wafer holds one dense-attention layer or two linear-attention layers, leaving room for request state. Only activations pass between wafers, with Cerebras putting Ethernet transfer latency at roughly two microseconds per hop. Multiple requests occupy different pipeline stages simultaneously; available KV-cache memory on dense-layer wafers limits concurrency.
HN · Frontpage AI's note
score 6
Discussions
- hn · 260 points · 85 comments
- hn · 271 points · 90 comments
- hn · 277 points · 95 comments
- hn · 291 points · 99 comments
- hn · 303 points · 107 comments
- hn · 311 points · 110 comments
- hn · 318 points · 113 comments
- hn · 322 points · 114 comments
- hn · 333 points · 118 comments
- hn · 335 points · 119 comments
- hn · 338 points · 121 comments
- hn · 340 points · 123 comments
- hn · 343 points · 124 comments
- hn · 344 points · 127 comments
- hn · 346 points · 127 comments
- hn · 348 points · 127 comments