Replacing Large Language Models with Jev Decision Models for Low-Latency Edge Service Orchestration
Jev cut decision latency by 22.7% to 64.5% versus the fastest LLM across the paper’s tests.
The paper evaluates Jev as a replacement for LLM decision calls in edge service orchestration, using bounded intent fields, validation, admission policy, and scheduling. In 8,280 verified requests and a live admission path, Jev’s latency stayed nearly flat as input size, contract width, and catalog size changed. The authors report lower API cost per correct decision on four-field contracts, with some exact-match loss and weaker results on wider contracts. On the live path, Jev kept 0.91-0.95 of requests exact and on time under loads where the tested LLMs fell below 0.1. HF Daily Papers' note
The paper evaluates Jev as a replacement for LLM decision calls in edge service orchestration, using bounded intent fields, validation, admission policy, and scheduling. In 8,280 verified requests and a live admission path, Jev’s latency stayed nearly flat as input size, contract width, and catalog size changed. The authors report lower API cost per correct decision on four-field contracts, with some exact-match loss and weaker results on wider contracts. On the live path, Jev kept 0.91-0.95 of requests exact and on time under loads where the tested LLMs fell below 0.1. HF Daily Papers' note
score 4