IdeaLens: Detecting AI Ideas in Long-form Writing
IdeaLens tries to detect whether a document’s ideas, not just its wording, came from AI.
The paper represents long-form writing as outlines that strip away much of the surface prose, then trains a detector on those idea-level structures. In controlled tests, its AI flag rate fell from 95% to 7% when models wrote from increasingly detailed human plans, while it stayed above 96% for AI-derived plans. On 50 stories written by humans from AI-generated plans, IdeaLens flagged 68% as AI, compared with 8% for Pangram 4. The authors also report strong results across 19 detection benchmarks and release models and labeled datasets. ArXiv · AI/CL/LG's note
The paper represents long-form writing as outlines that strip away much of the surface prose, then trains a detector on those idea-level structures. In controlled tests, its AI flag rate fell from 95% to 7% when models wrote from increasingly detailed human plans, while it stayed above 96% for AI-derived plans. On 50 stories written by humans from AI-generated plans, IdeaLens flagged 68% as AI, compared with 8% for Pangram 4. The authors also report strong results across 19 detection benchmarks and release models and labeled datasets. ArXiv · AI/CL/LG's note
score 4