A new reader trial approach to peer review in funding research grants: An Australian experiment.

Autor/inn/en	Jayasinghe, Upali W.; Marsh, Herbert W.; Bond, Nigel
Titel	A new reader trial approach to peer review in funding research grants: An Australian experiment.
Quelle	In: Scientometrics, (2006) 3, S.591-606 PDF als Volltext Verfügbarkeit
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0138-9130
DOI	10.1007/s11192-006-0171-4
Schlagwörter	Markov Chain Monte Carlo; Australian Research Council; Peer Review Process; Grant Proposal; Reader System
Abstract	Summary Peer reviews are highly valued in academic life, but are notoriously unreliable. A major problem is the substantial measurement error due to the idiosyncratic responses when large numbers of different assessors each evaluate only a single or a few submissions (e.g., journal articles, grants, etc.). To address this problem, the main funding body of academic research in Australia trialed a “reader system” in which each of a small number of senior academics read all proposals within their subdiscipline. The traditional peer review process for 1996 (2,989 proposals, 6,233 assessors) resulted in unacceptably low reliabilities comparable with those found in other research (0.475 for research project, 0.572 for researcher). For proposals from psychology and education in 1997, the new reader system resulted in substantially higher reliabilities: 0.643 and 0.881, respectively. In comparison to the traditional peer review approach, the new reader system is substantially more reliable, timely, and cost efficient - and applicable to many peer review situations.
Erfasst von	OLC
Update	2023/2/05