Inkling: Our open-weights model

<p><strong><a href="https://thinkingmachines.ai/news/introducing-inkling/">Inkling: Our open-weights model</a></strong></p> Mira Murati's Thinking Machines Lab just released their first open-weights model. Inkling is "a Mixture-of-Experts transformer with 975B total parameters, 41B active" - an Apache-2.0 licensed multimodal model trained on 45 trillion tokens of text, images, audio and video.</p> <p>They're also promising Inkling-Small, a 276B (12B active) model, but that's still being tested and the weights will be released "once that work is complete".</p> <p>The <a href="https://thinkingmachines.ai/model-card/inkling/">model card</a> is much shorter than I've come to expect from US AI labs. It links to even shorter <a href="https://thinkingmachines.ai/training-data-documentation/">Training Data Documentation</a> with almost nothing of interest in it - it's best summarized by these two paragraphs:</p> <blockquote> <p>The datasets Thinking Machines Lab uses to develop its AI services includes content that is in the public domain as well as content that may be subject to intellectual property protection.</p> <p>Thinking Machines Lab’s services were developed using publicly available content obtained from the open internet and publicly accessible data repositories. Certain datasets were also obtained from third parties.</p> </blockquote> <p>By Thinking Machines' own admission, this is not a frontier model. It's instead intended as a strong base model for fine-tuning using their own <a href="https://thinkingmachines.ai/tinker/">Tinker training platform</a>:</p> <blockquote> <p>Inkling is not the strongest overall model available today, open or closed. Instead, a combination of qualities makes it a good open-weights base for customization: multimodal capabilities, efficient thinking, and availability on Tinker for fine-tuning.</p> </blockquote> <p>There's a lot to like about this release. It's Apache-2.0 licensed, and looks competitive with the open weight models comi