Extending the Context Length to 1M Tokens!
Alibaba's Qwen2.5-Turbo extends context window from 128k to 1M tokens, enabling processing of ~1 million words in a single prompt.
Excerpt
API Documentation (Chinese) HuggingFace Demo ModelScope Demo
Introduction After the release of Qwen2.5, we heard the community’s demand for processing longer contexts. In recent months, we have made many optimizations for the model capabilities and inference performance of extremely long context. Today, we are proud to introduce the new Qwen2.5-Turbo version, which features:
Longer Context Support: We have extended the model’s context length from 128k to 1M, which is approximately 1 million English words or 1.
Read at source: https://qwenlm.github.io/blog/qwen2.5-turbo/