Exploring the Reliability and Its Influencing Factors of Peer Assessment in Massive Open Online Courses

Autor/inn/en	Li, Hongxia; Zhao, ChengLing; Long, Taotao; Huang, Yan; Shu, Fengfang
Titel	Exploring the Reliability and Its Influencing Factors of Peer Assessment in Massive Open Online Courses
Quelle	In: British Journal of Educational Technology, 52 (2021) 6, S.2263-2277 (15 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0007-1013
DOI	10.1111/bjet.13143
Schlagwörter	Peer Evaluation; Online Courses; Open Education; Feedback (Response); Scores; Assignments; Portfolio Assessment; Correlation; College Students; Interrater Reliability; Foreign Countries; China + Suchen Sie Ihr Suchwort? Online course; Online-Kurs; Offene Erziehung; Offener Unterricht; Assignment; Auftrag; Zuweisung; Portfoliobeurteilung; Korrelation; Collegestudent; Interrater-Reliabilität; Ausland
Abstract	As an innovative evaluation tool, peer assessment is essential in Massive Open Online Courses (MOOCs). In both formative and summative peer assessments in MOOCs, providing reliable feedback is crucial in enhancing learning outcomes. Peer assessment has been highlighted as a reliable tool in both traditional classrooms and small-scale online learning contexts by a wealth of studies. Factors impacting the result of peer assessment have also been identified. However, such exploration in MOOCs is very limited. Through an examination of 5,722 assignment submissions and 56,794 review scores collected from 18 peer assessments in three courses on Chinese University MOOC platform, this study investigated the inter-reliability of peer assessment in MOOCs utilizing Krippendorff's alpha and ICC. It also revealed factors affecting peer assessment reliability in MOOCs. The results of this study show that students tended to mark extreme scores, and peer assessment in MOOCs seemed unreliable. In addition, the reliability of the peer assessment in these MOOCs was inversely correlated to the number of reviewers per assignment and reviews completed per reviewer. The assignment type also matters; peer assessment using e-portfolios format was found more reliable than those using papers and proposals. Suggestions on how to improve the reliability of peer assessments in MOOCs are also provided. (As Provided).
Anmerkungen	Wiley. Available from: John Wiley & Sons, Inc. 111 River Street, Hoboken, NJ 07030. Tel: 800-835-6770; e-mail: cs-journals@wiley.com; Web site: https://www.wiley.com/en-us
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01