Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcripts
Transcript-first systems beat direct audio detection in this benchmark.
The paper introduces a spoken hallucination benchmark with 12,013 news samples in English, Russian, and Kazakh. It pairs original articles with controlled hallucinated versions across text and audio, plus 290 fact-checked fake news items in Russian and Kazakh. Transcript-based detection generally performed better than direct audio processing, and degradation tracked ASR error by language. Synthetic-trained detectors transferred strongly to real-world fakes, but the authors flag machine-style signals as a confound in synthetic benchmarks. ArXiv · AI/CL/LG's note
The paper introduces a spoken hallucination benchmark with 12,013 news samples in English, Russian, and Kazakh. It pairs original articles with controlled hallucinated versions across text and audio, plus 290 fact-checked fake news items in Russian and Kazakh. Transcript-based detection generally performed better than direct audio processing, and degradation tracked ASR error by language. Synthetic-trained detectors transferred strongly to real-world fakes, but the authors flag machine-style signals as a confound in synthetic benchmarks. ArXiv · AI/CL/LG's note
score 5