EDRAC: Benchmarking Arabic Dialect Reading Comprehension
EDRAC adds 4,977 verified QA pairs built from spoken Arabic across five dialects.
The benchmark covers Egyptian, Moroccan, Emirati, Syrian, and Saudi Arabic, using 499 passages from naturally occurring spoken interactions. Its authors say existing Arabic QA sets lean toward Modern Standard Arabic or multiple-choice formats, leaving dialectal reading comprehension under-covered. They test Arabic-centric and multilingual LLMs and report a split between semantic answer quality and dialectal fidelity. ArXiv · AI/CL/LG's note
The benchmark covers Egyptian, Moroccan, Emirati, Syrian, and Saudi Arabic, using 499 passages from naturally occurring spoken interactions. Its authors say existing Arabic QA sets lean toward Modern Standard Arabic or multiple-choice formats, leaving dialectal reading comprehension under-covered. They test Arabic-centric and multilingual LLMs and report a split between semantic answer quality and dialectal fidelity. ArXiv · AI/CL/LG's note
score 4