AI Insight
Researchers identified a critical problem in large language models performing long-context reasoning: the models excessively copy text from input prompts rather than engaging in productive problem-solving, a behavior that worsens as context length increases. The study found this "repetitive copying" stems from insufficient grounding in relevant evidence, with models copying indiscriminately from both relevant and irrelevant portions of the prompt. To address this, they developed GEAR (Grounding Evidence-Aware Reward), a reinforcement learning method that rewards overlap with key evidence while penalizing copying of irrelevant context, achieving improvements of up to 4.6 points over standard accuracy-based training while reducing repetitive copying.
Why it matters
This work addresses a fundamental limitation in deploying language models for complex reasoning tasks with long inputs, such as legal document analysis, scientific literature review, or technical troubleshooting. By improving models' ability to focus on relevant information rather than copying extraneous text, this approach could make AI assistants more reliable and efficient in real-world applications requiring careful reasoning over extensive context.
Understand the Science
arXiv:2607.19345v1 Announce Type: cross
Abstract: Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical failure mode in this regime: emph{repetitive copying}, where models extensively copy text from the input into their reasoning traces rather than productively solving the problem. We show that this behavior is pervasive across frontier long-context LLMs and intensifies with context length. By separating each prompt into task-relevant key evidence and irrelevant distractor context, we further show that the root cause is insufficient grounding: models copy from the prompt indiscriminately, and those that fail to focus on key evidence are far more likely to answer incorrectly. Motivated by this diagnosis, we propose GEAR (Grounding Evidence-Aware Reward), a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context. To enable GEAR on natural-language data, we develop an automated pipeline that constructs evidence-annotated training data from arbitrary documents. We validate GEAR across multiple model scales and benchmarks, showing consistent improvements of up to +4.6 average points over standard RL with accuracy-based rewards, with larger gains at longer contexts, while also reducing repetitive copying and thinking length. Our findings suggest that, even as long-context evaluation shifts from simple retrieval toward complex reasoning, accurate grounding in relevant evidence remains an indispensable capability with substantial room for improvement.