And the Winner Is … : Inter-Rater Reliability among Scholarship Assessors

Autor/inn/en	Johnston, Lucy; Schluter, Philip J.
Titel	And the Winner Is … : Inter-Rater Reliability among Scholarship Assessors
Quelle	In: Studies in Higher Education, 42 (2017) 9, S.1736-1749 (14 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0307-5079
DOI	10.1080/03075079.2015.1124849
Schlagwörter	Interrater Reliability; Scholarships; Academic Achievement; Program Proposals; Educational Quality; Achievement Rating; Leadership; Personality Measures; Evaluation Criteria; Advisory Committees; Statistical Analysis; Correlation; Scores; Group Structure; Higher Education; Foreign Countries; New Zealand + Suchen Sie Ihr Suchwort? Interrater-Reliabilität; Scholarship; Stipendium; Schulleistung; Quality of education; Bildungsqualität; Achievement; Rating; Leistung; Beurteilung; Leistungsbeurteilung; Führung; Führungsposition; Beratungsstelle; Statistische Analyse; Korrelation; Gruppenstruktur; Hochschulbildung; Hochschulsystem; Hochschulwesen; Ausland; Neuseeland
Abstract	With increasing competition for postgraduate research scholarships, awarding processes demand attention and scrutiny. We examine inter-rater reliability for two prestigious New Zealand scholarships, the Shirtcliffe Fellowship and the Gordon Watson Scholarship. For each scholarship, five assessors (three academic; two non-academic) independently evaluate all applicants over three domains: Academic Merit, Quality of Study Plans and Character/Leadership. Data from years 2009 to 2014 were extracted, comprising 12 separate assessment rounds. Good to excellent agreement was observed for each scholarship in each year. Agreement was significantly higher for the Academic Merit domain compared to the other domains. Moreover, agreement among academics was higher and less variable than non-academics for this Academic Merit domain. No such differences were noted in the other domains. While resource efficiencies could be made, reductions in committee size resulted in poorer applicant selection performance. Applicants and donors alike can be confident that the awardee for these scholarships is a top applicant. (As Provided).
Anmerkungen	Routledge. Available from: Taylor & Francis, Ltd. 530 Walnut Street Suite 850, Philadelphia, PA 19106. Tel: 800-354-1420; Tel: 215-625-8900; Fax: 215-207-0050; Web site: http://www.tandf.co.uk/journals
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2020/1/01