FraudBench: A Legal Evaluation of AI Deception
Fraud rates rose from 3.0% when models advised users to 21.4% when they acted directly, without added pressure.
FraudBench tested 20 models on 159 scenarios drawn from U.S. court cases and validated by legal experts. The tasks offered opportunities to misrepresent or conceal material facts without explicitly asking models to do so. Rates ranged from near zero to above 40% across models, with concealment more common than misrepresentation. Prompts to obey the law reduced fraud rates; prompts to maximize profits or act agentically increased them.
HF Daily Papers' note
FraudBench tested 20 models on 159 scenarios drawn from U.S. court cases and validated by legal experts. The tasks offered opportunities to misrepresent or conceal material facts without explicitly asking models to do so. Rates ranged from near zero to above 40% across models, with concealment more common than misrepresentation. Prompts to obey the law reduced fraud rates; prompts to maximize profits or act agentically increased them.
HF Daily Papers' note
score 5