PerSo: Investigating Facets of LLMs' Social Intelligence in Real-World Persuasion Scenarios
The strongest model narrowly beat the best observed human result on choosing social actions, but the best human stayed ahead on inferring mental states.
PerSo tests both abilities using persuasion dialogues from Reddit’s ChangeMyView community. The researchers built evaluation tasks with an LLM-in-the-loop pipeline and validated them through human annotation. Across the models tested, performance generally improved with model size, though results varied by dimension.
ArXiv · AI/CL/LG's note
PerSo tests both abilities using persuasion dialogues from Reddit’s ChangeMyView community. The researchers built evaluation tasks with an LLM-in-the-loop pipeline and validated them through human annotation. Across the models tested, performance generally improved with model size, though results varied by dimension.
ArXiv · AI/CL/LG's note
score 4