Do LLMs Exhibit Coherent Knowledge Structures in Mathematical Reasoning? A Perspective from Knowledge Space Theory
Eight tested LLMs broke the prerequisite patterns that human learners tend to follow in math reasoning.
The paper uses Knowledge Space Theory as a yardstick for whether model answers respect dependencies between mathematical concepts. It reports that LLMs often answer dependent questions without showing the expected prerequisite mastery, and that extra related context does not reliably improve those dependent answers. The authors also find little agreement among models’ inferred knowledge structures. They argue these failures are mostly hidden by plain accuracy scores and LLM-as-judge evaluations. ArXiv · AI/CL/LG's note
The paper uses Knowledge Space Theory as a yardstick for whether model answers respect dependencies between mathematical concepts. It reports that LLMs often answer dependent questions without showing the expected prerequisite mastery, and that extra related context does not reliably improve those dependent answers. The authors also find little agreement among models’ inferred knowledge structures. They argue these failures are mostly hidden by plain accuracy scores and LLM-as-judge evaluations. ArXiv · AI/CL/LG's note
score 4