Megadose The AI curve, compressed.

sybil-solutions/inference-optimisation-skill

· GitHub · LLM repos ·
Speed gains count only if they hold under a memory cap and pass full-context stability checks.

The skill guides agents through tuning a local LLM server, starting with theoretical hardware ceilings and changing one variable at a time. Candidates must beat both neighbouring control runs without regressing any measured metric by more than 3%. On an Intel Arc B390 laptop, the repository reports single-user decode rising from 43.6 to 53 tokens per second and 256K-context prefill from 297 to 628 tokens per second. Its benchmark harness records failures as well as wins, with long-context retrieval and concurrent full-cache checks before publication.

GitHub · LLM repos' note

score 4

Categories: OSS & Tools