An ablation study on the use of publication venue quality to rank computer science departments.

Autor/inn/en	Maiti, Aniruddha; Shi, Sai; Vucetic, Slobodan
Titel	An ablation study on the use of publication venue quality to rank computer science departments.
Quelle	In: Scientometrics, (2023) 8, S.4197-4218 PDF als Volltext Verfügbarkeit
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0138-9130
DOI	10.1007/s11192-023-04733-2
Schlagwörter	Ranking of doctoral programs; Ranking of computer science departments; Peer assessment; Clustering, ablation study
Abstract	Abstract This paper focuses on ranking computer science departments based on the quality of publications by the faculty in those departments. There are multiple strategies to convert publication lists into ranking scores for the departments. Important open questions include handling multi-author publications, inclusion criteria for publications and publication venues, accounting for the quality of publication venues, and accounting for the sub-areas of computer science. An ablation study is performed to evaluate the importance of different decisions for department ranking. The correlation between the resulting rankings and the peer assessment of computer science departments provided by the U.S. News was measured to evaluate the importance of different decisions. The results show that the selection of publication venues has the highest impact on the ranking. In contrast, decisions related to publication recency, multi-author publications, and clustering publications into subareas have less impact. Overall, Pearson’s correlation coefficient between the publication-based scores and the U.S. News ranking is above 0.90 for a large range of decisions, indicating a strong agreement between the objective measure and the subjective opinion of peers.
Erfasst von	OLC
Update	2024/1/01