Toward Human-AI Collaboration: A Recommender System to Support CS1 Instructors to Select Problems for Assignments and Exams

Autor/inn/en	Pereira, Filipe Dwan; Rodrigues, Luiz; Henklain, Marcelo Henrique Oliveira; Freitas, Hermino; Oliveira, David Fernandes; Cristea, Alexandra I.; Carvalho, Leandro; Isotani, Seiji; Benedict, Aileen; Dorodchi, Mohsen; de Oliveira, Elaine Harada Teixeira
Titel	Toward Human-AI Collaboration: A Recommender System to Support CS1 Instructors to Select Problems for Assignments and Exams
Quelle	In: IEEE Transactions on Learning Technologies, 16 (2023) 3, S.457-472 (16 Seiten)Infoseite zur Zeitschrift PDF als Volltext Verfügbarkeit
Zusatzinformation	ORCID (Pereira, Filipe Dwan) ORCID (Cristea, Alexandra I.) ORCID (Carvalho, Leandro) ORCID (Isotani, Seiji) ORCID (de Oliveira, Elaine Harada Teixeira)
Sprache	englisch
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
DOI	10.1109/TLT.2022.3224121
Schlagwörter	Artificial Intelligence; Man Machine Systems; Educational Technology; Technology Uses in Education; Problem Solving; Accuracy; Assignments; Tests + Suchen Sie Ihr Suchwort? Künstliche Intelligenz; Mensch-Maschine-System; Unterrichtsmedien; Technology enhanced learning; Technology aided learning; Technologieunterstütztes Lernen; Problemlösen; Assignment; Auftrag; Zuweisung; Examination; Prüfung; Examen
Abstract	Programming online judges (POJs) have been increasingly used in CS1 classes, as they allow students to practice and get quick feedback. For instructors, it is a useful tool for creating assignments and exams. However, selecting problems in POJs is time consuming. First, problems are generally not organized based on topics covered in the CS1 syllabus. Second, assessing whether problems require similar effort to be completed and map onto the same topic is a subjective and expert-dependent task. The difficulty increases if the instructor must create variations of these assessments, e.g., to avoid plagiarism. Thus, here, we research how to support CS1 instructors in the task of selecting problems, to compose one-size-fits-all or personalized assignments/exams. Our solution is to propose a novel intelligent recommender system, based on a fine-grained data-driven analysis of the students' effort on solving problems in the integrated development environment of a POJ system, and automatic detection of topics for CS1 problems, based on problem descriptions. Data collected from 2714 students are processed to support, via our artificial intelligence (AI) method recommendations, the instructors' decision-making process. We evaluated our method against the state of the art in a simple blind experiment with CS1 instructors (N = 35). Results show that our recommendations are 88% accurate, surpassing our baseline (p = 0.05). Finally, our work paves the way for novel POJ smart learning environments, wherein instructors define learning tasks (assignments/exams) supported by AI. (As Provided).
Anmerkungen	Institute of Electrical and Electronics Engineers, Inc. 445 Hoes Lane, Piscataway, NJ 08854. Tel: 732-981-0060; Web site: http://ieeexplore.ieee.org/xpl/RecentIssue.jsp?punumber=4620076
Erfasst von	ERIC (Education Resources Information Center), Washington, DC
Update	2024/1/01