WebFovea: When the Model Is Right but the Click Is Wrong -- Reliable Round Trips for Vision-Based Web Agents on Live Websites
The reported score jump came from fixing the browser harness, not changing the model.
WebFovea placed 2nd in the WebRetriever Challenge 2026 with a 57.0/100 final score. The paper says many failures happened between the model and the live page: misparsed actions, clicks landing at the wrong coordinates, silent failures in dropdowns, iframes, and text boxes, and bad feedback to the model. One coordinate mismatch put clicks at three-quarters of their intended position, and chat-template tokens contaminated 4.9% of task episodes. The author says the same model was used across submissions, so the hidden-set rise from 31.0 to 57.0 is attributed to harness changes, within live-site variance. HF Daily Papers' note
WebFovea placed 2nd in the WebRetriever Challenge 2026 with a 57.0/100 final score. The paper says many failures happened between the model and the live page: misparsed actions, clicks landing at the wrong coordinates, silent failures in dropdowns, iframes, and text boxes, and bad feedback to the model. One coordinate mismatch put clicks at three-quarters of their intended position, and chat-template tokens contaminated 4.9% of task episodes. The author says the same model was used across submissions, so the hidden-set rise from 31.0 to 57.0 is attributed to harness changes, within live-site variance. HF Daily Papers' note
score 5