LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data
LongZero builds long training contexts from evidence the model writes itself, without external data or a stronger teacher.
The model first generates a question, answer and supporting evidence in a compact structure, then expands that evidence into a long, multi-document context. Training combines these multi-hop reasoning tasks with self-written, multi-round retrieval tasks using GRPO reinforcement learning. Across two LLMs and five long-context reasoning benchmarks, the authors report that LongZero outperformed both baselines and generalized to real tasks. The abstract gives no numerical results.
ArXiv · AI/CL/LG's note
The model first generates a question, answer and supporting evidence in a compact structure, then expands that evidence into a long, multi-document context. Training combines these multi-hop reasoning tasks with self-written, multi-round retrieval tasks using GRPO reinforcement learning. Across two LLMs and five long-context reasoning benchmarks, the authors report that LongZero outperformed both baselines and generalized to real tasks. The abstract gives no numerical results.
ArXiv · AI/CL/LG's note
score 5