Megadose The AI curve, compressed.

K2-Horizon: A Frontier Fleet of Open Source Language Models

· HF Daily Papers ·
K2-Horizon’s release includes training data, intermediate checkpoints, logs, code and the pipelines that produced roughly 10 trillion synthetic pre-training tokens.

The six models range from 0.9 billion to 375 billion total parameters, with the largest activating 23 billion per token. The team reports competitive or leading agentic and coding benchmark results against strong open models of similar size. Its 36-billion-parameter model nearly matches a 32-billion-parameter dense model trained on the same data while using one-seventh the active parameters. The paper also introduces Uno adapters, reporting lossless decoding speedups of up to 2.2× without a separate draft model.

HF Daily Papers' note

score 8

Categories: Model Releases, OSS & Tools, Research