AI Insight
Researchers have developed GitScholar, a dataset linking GitHub activity from 444,000 repositories to over 558,000 AI research papers from arXiv, to predict which papers will have significant academic impact. Their analysis demonstrates that GitHub engagement metrics improve early prediction accuracy of impactful papers by up to 12% compared to traditional academic indicators alone. The study found that GitHub signals provide near-complete coverage of high-impact AI papers and consistently correlate with future citations and academic success.
Why it matters
This approach could help researchers and institutions efficiently identify promising AI research amid the overwhelming volume of daily publications, potentially accelerating knowledge dissemination and collaboration. The publicly available dataset may enable better resource allocation decisions for research prioritization and funding.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: With the rapid pace of AI research and the hundreds of daily new publications, staying up-to-date with the latest developments has become increasingly difficult. For researchers, quickly identifying impactful work is essential, yet manually reviewing each new publication is impractical. Automated impact prediction methods help address this challenge, usually by combining various information sources available, such as a paper’s content or citation history. In this work, we propose using GitHub engagement as an additional source and demonstrate that it provides both a timely and accurate signal. To this end, we introduce GitScholar, a novel dataset that links GitHub activity from 444,000 repositories to over 558,000 AI arXiv papers. Our experiments show that GitHub reactions improve early prediction precision by up to 12% over a strong academic baseline. Additionally, we find that GitHub signal offers near-complete coverage of high-impact AI papers, and consistently correlates with future academic success. GitScholar is publicly available at https://huggingface.co/datasets/huawei-csl/GitScholar.
Source: GitScholar: A Dataset for Predicting AI Research Impact from GitHub Engagement