Do LLMs Understand Sequential Structure? A Controlled Study of Inference and Generation
The study finds LLMs can match action distributions while missing the rules that generated them.
The paper tests models on controlled Rock-Paper-Scissors interactions and a stochastic n-gram continuation task. Longer context did not improve strategy identification, and recognizing a rule did not reliably translate into simulating it. Performance degraded when the task required higher-order conditional dependencies. The authors warn that behavior that looks faithful can hide the wrong generative mechanism. HF Daily Papers' note
The paper tests models on controlled Rock-Paper-Scissors interactions and a stochastic n-gram continuation task. Longer context did not improve strategy identification, and recognizing a rule did not reliably translate into simulating it. Performance degraded when the task required higher-order conditional dependencies. The authors warn that behavior that looks faithful can hide the wrong generative mechanism. HF Daily Papers' note
score 3