AI Insight
This study investigates why advanced reasoning language models show improved single-answer accuracy but reduced diversity in generating multiple solutions compared to base models. Researchers found that this "coverage shrinkage" phenomenon is primarily caused by training data containing decision points where multiple valid reasoning paths exist, and that the model learns to favor specific paths over maintaining diverse problem-solving approaches. Through controlled experiments involving graph branching and reasoning tasks, they demonstrate that targeted data design and diversity-encouraging decoding methods can partially mitigate this shrinkage effect.
Why it matters
These findings reveal a fundamental limitation in how reasoning models are currently trained and suggest practical methods to improve their reliability. The ability to generate diverse valid solutions is crucial for applications requiring robust problem-solving, such as mathematical reasoning, code generation, and scientific analysis, where exploring multiple approaches increases the likelihood of finding correct solutions.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: Recent progress in large language models has led to the emergence of reasoning models, which have shown strong performance on complex tasks through specialized fine-tuning procedures. While these methods reliably improve pass@1 accuracy, prior works have observed that they show a coverage shrinkage behavior, where pass@k degrades relative to the base model. In this paper, we investigate the cause of reasoning shrinkage under SFT-based post-training. We hypothesize that this behavior is driven by properties of the fine-tuning data, specifically related to decision points or “forks in the road” scenarios where model encounters indecipherable patterns with multiple valid reasoning paths. To test this hypothesis, we design controlled case studies that simulate such decision-point settings, spanning indecipherable nodes in graph branching, and reasoning modes. By tracking post-training dynamics in these settings, we find that the shrinkage phenomenon is tightly correlated with the prevalence of decision-point scenarios in the training data. We also demonstrate that this shrinkage behavior can be partially mitigated through targeted data synthesis design of decision-points and a more systematic diversity-encouraging decoding mechanism. Our findings identify data-centric factors as a key driver of shrinkage in reasoning models and highlight diversity-aware designs as an effective lever for controlling it. (Data and code for reproducing our experiments are available at https://github.com/psunlpgroup/reasoning_forks )
Source: Why Do Reasoning Models Lose Coverage? The Role of Data and Forks in the Road