AI Insight
Researchers tested a new peer-review model called Discovery Stack that separates the assessment of scientific quality (research rigor) from impact (perceived significance) and found that reviewers could effectively evaluate these as distinct dimensions. In a pilot study involving 162 reviews completed in parallel with traditional journal review, quality scores showed more consistency across reviewers than impact scores, indicating that separating these assessments may reduce bias. Survey data from 86 participants showed strong support for this journal-independent model, which also incorporates in-line comments to improve feedback quality.
Why it matters
This approach could address major problems in scientific publishing including long review times, high costs, inconsistent quality, and systemic biases tied to journal prestige. By decoupling research quality from subjective impact assessments, the model may enable more transparent and efficient evaluation of scientific work independent of journal branding.
Understand the Science
by Maureen A. McGargill, Beiyun C. Liu, Michael S. Kuhns, Daniel Mucida, Isabella Rauch, Lauren B. Rodda, Meghan A. Koch, Hugo Gonzalez Velozo, Ken Cadwell, Tanya S. Freedman, Tiffany C. Scharschmidt, Richard Sever, Jose Ordovas-Montanes, Sara Suliman, Andrew Oberst, Brooke Runnette, Matthew F. Krummel
Peer review serves as the cornerstone of scientific quality control. Yet, the current journal-centric system is hindered by long timelines, high publication costs, inconsistent review quality, systemic biases, and editorial gatekeeping. Notably, the system relies on misaligned measures of impact that are tethered to journal branding and conflate scientific rigor (Quality) with perceived significance (Impact). Here, we report findings from the Discovery Stack Pilot Study, which tested a scientist-designed, journal-independent peer review model. The Discovery Stack model integrates in-line reviewer comments to promote constructive feedback and separately evaluates scientific Quality and Impact using defined criteria. To examine feasibility and effectiveness, manuscripts were reviewed in parallel with traditional journal review. A total of 162 reviews were completed, and survey data from 86 participants were analyzed. The results showed that reviewers effectively evaluated Quality and Impact as separate dimensions, with Quality scores being more consistent across reviewers than Impact scores. Participants strongly supported the core elements of the Discovery Stack model and expressed enthusiasm for its broader adoption to enhance transparency, efficiency, and value in peer review. Future studies will explore integrating this model into a digital platform for reviewing and curating scientific discoveries to improve the production and dissemination of high-quality research.