Exploring Collaboration between a language and a non-language agent
The paper tests whether translating a chess engine’s internal state into text makes an LLM-agent team worse.
The authors introduce LLAMIA-Bench, six collaborative chess tasks where neither the LLM nor the chess engine solves the task alone. They compare verbal summaries of the non-language agent’s state with learned “state tokens” inserted directly into the LLM’s token stream. Their experiments report a persistent “verbalization debt,” with the gap remaining as models scale from 4B to 14B parameters. The 14B LLAMIA model using latent state internalization matches or beats task specialists and frontier models including GPT-5.1 with tool access across the benchmark. HF Daily Papers' note
The authors introduce LLAMIA-Bench, six collaborative chess tasks where neither the LLM nor the chess engine solves the task alone. They compare verbal summaries of the non-language agent’s state with learned “state tokens” inserted directly into the LLM’s token stream. Their experiments report a persistent “verbalization debt,” with the gap remaining as models scale from 4B to 14B parameters. The 14B LLAMIA model using latent state internalization matches or beats task specialists and frontier models including GPT-5.1 with tool access across the benchmark. HF Daily Papers' note
score 4