InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models
Top MLLMs still fail most industrial gauge-reading tests when value and unit both have to be right.
InSituMeasure tests 24 models on 2,922 real industrial monitoring scenes across eight categories of engineering instruments. The best model reaches 25.7% joint value-unit accuracy and 51.8% confidence-diagnosis F1. The paper ties failures to text shortcuts, overconfident answers, and real scene noise such as occlusion, viewpoint deviation, mixed disturbances, and environmental interference. ArXiv · AI/CL/LG's note
InSituMeasure tests 24 models on 2,922 real industrial monitoring scenes across eight categories of engineering instruments. The best model reaches 25.7% joint value-unit accuracy and 51.8% confidence-diagnosis F1. The paper ties failures to text shortcuts, overconfident answers, and real scene noise such as occlusion, viewpoint deviation, mixed disturbances, and environmental interference. ArXiv · AI/CL/LG's note
score 5