OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
The authors report average gains of 9.7–12.6 points over matched models without tools across 22 benchmarks.
OpenVLHarness converts outputs from different tools into evidence a multimodal model can inspect. Persistent memory keeps images, structured data and web references available throughout a query, while a coding interface combines stored results in calculations. The study covers eight open and proprietary models and finds only modest gains from giving base models more inference budget alone. In one comparison, GPT-6 Luna with the harness at medium reasoning effort beat maximum-effort alternatives by 7.7–8.5 points at no more than 37% of their model-token cost.
ArXiv · AI/CL/LG's note
OpenVLHarness converts outputs from different tools into evidence a multimodal model can inspect. Persistent memory keeps images, structured data and web references available throughout a query, while a coding interface combines stored results in calculations. The study covers eight open and proprietary models and finds only modest gains from giving base models more inference budget alone. In one comparison, GPT-6 Luna with the harness at medium reasoning effort beat maximum-effort alternatives by 7.7–8.5 points at no more than 37% of their model-token cost.
ArXiv · AI/CL/LG's note
score 5