A Systematic Literature Review of Empiricism and Norms of Reporting in Computing Education Research Literature

Literaturnachweis - Detailanzeige

Trefferliste

Autor/inn/en	Heckman, Sarah; Carver, Jeffrey C.; Sherriff, Mark; Al-zubidy, Ahmed
Titel	A Systematic Literature Review of Empiricism and Norms of Reporting in Computing Education Research Literature
Quelle	In: ACM Transactions on Computing Education, 22 (2022) 1, Artikel 3 (46 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Heckman, Sarah) ORCID (Carver, Jeffrey C.) ORCID (Sherriff, Mark) ORCID (Al-zubidy, Ahmed)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	1946-6226
Schlagwörter	Computer Science Education; Educational Research; Periodicals; Replication (Evaluation); Meta Analysis; Educational Theories; Journal Articles; Norms; Evaluation Methods; Statistical Analysis; Research Methodology + Suchen Sie Ihr Suchwort? Computer science lessons; Informatikunterricht; Bildungsforschung; Pädagogische Forschung; Periodical; Journal; Zeitschrift; Fachzeitschrift; Periodikum; Meta-analysis; Metaanalyse; Educational theory; Theory of education; Bildungstheorie; Journal article; Zeitschriftenaufsatz; Normwert; Statistische Analyse; Research method; Forschungsmethode
Abstract	Context: Computing Education Research (CER) is critical to help the computing education community and policy makers support the increasing population of students who need to learn computing skills for future careers. For a community to systematically advance knowledge about a topic, the members must be able to understand published work thoroughly enough to perform replications, conduct meta-analyses, and build theories. There is a need to understand whether published research allows the CER community to systematically advance knowledge and build theories. Objectives: The goal of this study is "to characterize the reporting of empiricism in Computing Education Research literature by identifying whether publications include content necessary for researchers to perform replications, meta-analyses, and theory building." We answer three research questions related to this goal: (RQ1) What percentage of papers in CER venues have some form of empirical evaluation? (RQ2) Of the papers that have empirical evaluation, what are the characteristics of the empirical evaluation? (RQ3) Of the papers that have empirical evaluation, do they follow norms (both for inclusion and for labeling of information needed for replication, meta-analysis, and, eventually, theory-building) for reporting empirical work? Methods: We conducted a systematic literature review of the 2014 and 2015 proceedings or issues of five CER venues: "Technical Symposium on Computer Science Education" (SIGCSE TS), "International Symposium on Computing Education Research" (ICER), "Conference on Innovation and Technology in Computer Science Education" (ITiCSE), "ACM Transactions on Computing Education" (TOCE), and "Computer Science Education" (CSE). We developed and applied the "CER Empiricism Assessment Rubric" to the 427 papers accepted and published at these venues over 2014 and 2015. Two people evaluated each paper using the "Base Rubric" for characterizing the paper. An individual person applied the other rubrics to characterize the norms of reporting, as appropriate for the paper type. Any discrepancies or questions were discussed between multiple reviewers to resolve. Results: We found that over 80% of papers accepted across all five venues had some form of empirical evaluation. Quantitative evaluation methods were the most frequently reported. Papers most frequently reported results on interventions around pedagogical techniques, curriculum, community, or tools. There was a split in papers that had some type of comparison between an intervention and some other dataset or baseline. Most papers reported related work, following the expectations for doing so in the SIGCSE and CER community. However, many papers were lacking properly reported research objectives, goals, research questions, or hypotheses; description of participants; study design; data collection; and threats to validity. These results align with prior surveys of the CER literature. Conclusions: CER authors are contributing empirical results to the literature; however, not all norms for reporting are met. We encourage authors to provide clear, labeled details about their work so readers can use the study methodologies and results for replications and meta-analyses. As our community grows, our reporting of CER should mature to help establish computing education theory to support the next generation of computing learners. (As Provided).
Anmerkungen	Association for Computing Machinery. 2 Penn Plaza Suite 701, New York, NY 10121. Tel: 800-342-6626; Tel: 212-626-0500; Fax: 212-944-1318; e-mail: acmhelp@acm.org; Web site: http://toce.acm.org/
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01