BrailleBench: Investigating Multi-Criteria Braille Comprehension in Large Language Models
The benchmark finds LLMs still lose capability when Braille becomes the interface, especially with Grade 2 input and fully Braille exchanges.
BrailleBench tests 5,570 instances across math, commonsense, and multi-hop QA in English plus Braille Grades 1 and 2. The authors built it with a deterministic, expert-reviewed Braille Toolkit, without LLM-generated data instances. Six representative LLMs were evaluated on reading Braille-authored content, producing Braille answers, and handling end-to-end Braille interaction. The paper reports an uneven gap: models can differ sharply between understanding Braille and expressing it, with Grade 2 input particularly fragile. Source: ArXiv · AI/CL/LG's note
BrailleBench tests 5,570 instances across math, commonsense, and multi-hop QA in English plus Braille Grades 1 and 2. The authors built it with a deterministic, expert-reviewed Braille Toolkit, without LLM-generated data instances. Six representative LLMs were evaluated on reading Braille-authored content, producing Braille answers, and handling end-to-end Braille interaction. The paper reports an uneven gap: models can differ sharply between understanding Braille and expressing it, with Grade 2 input particularly fragile. Source: ArXiv · AI/CL/LG's note
score 4