Evaluating chain-of-thought monitorability
OpenAI's new chain-of-thought monitorability framework across 13 evaluations shows internal reasoning monitoring outperforms output monitoring for scalable AI control.
Excerpt
OpenAI introduces a new framework and evaluation suite for chain-of-thought monitorability, covering 13 evaluations across 24 environments. Our findings show that monitoring a model’s internal reasoning is far more effective than monitoring outputs alone, offering a promising path toward scalable control as AI systems grow more capable.
Read at source: https://openai.com/index/evaluating-chain-of-thought-monitorability