RunanywhereAI/trnrt
TrnRT serves GLM-5.3-Flash with a 1-million-token context on a single AWS Trainium2 instance.
The open-source engine provides an OpenAI-compatible API, fused decode kernels and a prefix cache spanning device and host memory. GLM-5.3-Flash is its first supported model, with more models planned. In AgentX tests with 32 concurrent coding agents, the default configuration reached 28.9 requests per minute at a reported spot cost of $0.040 per million tokens, counting cached input and output. Those benchmarks used an earlier build; the repository reports matching graphs and byte-identical outputs across 384 verification requests.
GitHub · LLM repos' note
The open-source engine provides an OpenAI-compatible API, fused decode kernels and a prefix cache spanning device and host memory. GLM-5.3-Flash is its first supported model, with more models planned. In AgentX tests with 32 concurrent coding agents, the default configuration reached 28.9 requests per minute at a reported spot cost of $0.040 per million tokens, counting cached input and output. Those benchmarks used an earlier build; the repository reports matching graphs and byte-identical outputs across 384 verification requests.
GitHub · LLM repos' note
score 5