sybil-solutions/inference-optimisation-skill
Speed gains count only if they hold under a memory cap and pass full-context stability checks.
The skill guides agents through tuning a local LLM server, starting with theoretical hardware ceilings and changing one variable at a time. Candidates must beat both neighbouring control runs without regressing any measured metric by more than 3%. On an Intel Arc B390 laptop, the repository reports single-user decode rising from 43.6 to 53 tokens per second and 256K-context prefill from 297 to 628 tokens per second. Its benchmark harness records failures as well as wins, with long-context retrieval and concurrent full-cache checks before publication.
GitHub · LLM repos' note
The skill guides agents through tuning a local LLM server, starting with theoretical hardware ceilings and changing one variable at a time. Candidates must beat both neighbouring control runs without regressing any measured metric by more than 3%. On an Intel Arc B390 laptop, the repository reports single-user decode rising from 43.6 to 53 tokens per second and 256K-context prefill from 297 to 628 tokens per second. Its benchmark harness records failures as well as wins, with long-context retrieval and concurrent full-cache checks before publication.
GitHub · LLM repos' note
score 4