EngineAgent: Agentizing On-Device LLM Inference Engines with Inference Stack Hierarchy Awareness and Hardware Abilities
EngineAgent completed 75% of on-device model-enablement tasks, versus 33.3% for the strongest coding-agent baseline.
The authors tested 12 tasks covering three mobile LLMs and four engine–backend pairs across CPUs, GPUs, and NPUs. Success required compilation, execution on the device, and output-logit agreement with reference implementations. EngineAgent maps dependencies across engine layers and validates hardware workflows before a builder and auditor implement and debug each new model. It reached the reported success rate with either of two underlying LLMs, while reducing average task time and model-API cost against baselines using the same LLMs.
ArXiv · AI/CL/LG's note
The authors tested 12 tasks covering three mobile LLMs and four engine–backend pairs across CPUs, GPUs, and NPUs. Success required compilation, execution on the device, and output-logit agreement with reference implementations. EngineAgent maps dependencies across engine layers and validates hardware workflows before a builder and auditor implement and debug each new model. It reached the reported success rate with either of two underlying LLMs, while reducing average task time and model-API cost against baselines using the same LLMs.
ArXiv · AI/CL/LG's note
score 4