CulturalRiddles: A Multicultural Benchmark of Riddles
Even frontier models reach at most 22% accuracy on riddles from some communities.
CulturalRiddles contains human-created, culturally grounded riddles spanning 62 cultures, 45 countries and 51 languages. Solving them requires reasoning alongside factual and social knowledge, language skills and culturally specific associations. The researchers tested 28 LLM configurations using automatic scoring and large-scale human evaluation. They publicly release all artifacts and invite contributions to expand coverage.
HF Daily Papers' note
CulturalRiddles contains human-created, culturally grounded riddles spanning 62 cultures, 45 countries and 51 languages. Solving them requires reasoning alongside factual and social knowledge, language skills and culturally specific associations. The researchers tested 28 LLM configurations using automatic scoring and large-scale human evaluation. They publicly release all artifacts and invite contributions to expand coverage.
HF Daily Papers' note
score 4