OpenAI’s Jalapeño chip is built for fast inference at scale, benchmarks show

· TechCrunch AI ·

OpenAI’s Jalapeño inference chip beat current state-of-the-art systems on throughput per watt and tokens per user benchmarks.

Categories: Money & Moves

Excerpt

Tested on Semianalysis’s InferenceX benchmark, Jalapeño registered both more tokens per user and more throughput per kilowatt than the currently available state-of-the art.