Supporting Skill Assessment in Learning Experiences Based on Serious Games Through Process Mining Techniques

• Juan Antonio Caballero-Hernández ^[1] ; Manuel Palomo-Duarte ^[1] ; Juan Manuel Dodero ^[1] ; Dragan Gaševic ^[2]
  1. [1] Universidad de Cádiz

     Universidad de Cádiz

     Cádiz, España

     
  2. [2] Monash University

     Monash University

     Australia

     
• Localización: IJIMAI, ISSN-e 1989-1660, Vol. 8, Nº. 6, 2024, págs. 146-159
• Idioma: inglés
• DOI: 10.9781/ijimai.2023.05.002
• Enlaces
  • Texto completo
• Dialnet Métricas: 1 Cita
• Resumen

  • Learning experiences based on serious games are employed in multiple contexts. Players carry out multiple interactions during the gameplay to solve the different challenges faced. Those interactions can be registered in logs as large data sets providing the assessment process with objective information about the skills employed. Most assessment methods in learning experiences based on serious games rely on manual approaches, which do not scalewell when the amount of data increases. We propose an automated method to analyse students’ interactions and assess their skills in learning experiences based on serious games. The method takes into account not only the final model obtained by the student, but also the process followed to obtain it, extracted from game logs. The assessment method groups students according to their in-game errors and ingame outcomes. Then, the models for the most and the least successful students are discovered using process mining techniques. Similarities in their behaviour are analysed through conformance checking techniques to compare all the students with the most successful ones. Finally, the similarities found are quantified to build a classification of the students’ assessments. We have employed this method with Computer Science students playing a serious game to solve design problems in a course on databases. The findings show that process mining techniques can palliate the limitations of skill assessment methods in game-based learning experiences.

• Referencias bibliográficas
  • D. Djaouti, J. Alvarez, J.-P. Jessel, O. Rampnoux, “Origins of Serious Games,” in Serious Games and Edutainment Applications, M. Ma, A. Oikonomou,...
  • T. M. Connolly, E. A. Boyle, E. Macarthur, T. Hainey, J. M. Boyle, “A systematic literature review of empirical evidence on computer games...
  • J. A. Caballero-Hernández, M. Palomo-Duarte, J. M. Dodero, “Skill assessment in learning experiences based on serious games: A Systematic Mapping...
  • G. Siemens, S. Dawson, G. Lynch, “Improving the Quality and Productivity of the Higher Education Sector Policy and Strategy for Systems-Level...
  • W. van der Aalst, A. Adriansyah, A. K. A. De Medeiros, F. Arcieri, T. Baier, T. Blickle, J. C. Bose, P. Van Den Brand, R. Brandtjen, J. Buijs,...
  • T. Hainey, T. M. Connolly, Y. Chaudy, E. Boyle, R. Beeby, M. Soflano, “Assessment integration in serious games,” in Psychology, Pedagogy,...
  • M. Allen, Assessing academic programs in higher education. Bolton, MA: Anker, 2004.
  • V. J. Shute, “Stealth Assessment in Computer-Based Games To Support Learning,” in Computer Games and Instruction, S. Tobias, J. D. Fletcher Eds.,...
  • R. J. Mislevy, G. D. Haertel, “Implications of evidence- centered design for educational testing,” Educational Measurement: Issues and Practice, vol....
  • R. J. Mislevy, R. G. Almond, J. F. Lukas, “A brief introduction to evidencecentered design,” ETS Research Report Series, vol. 2003, no. 1,...
  • V. J. Shute, M. Ventura, M. Bauer, D. Zapata-Rivera, “Melding the power of serious games and embedded assessment to monitor and foster learning,”...
  • V. J. Shute, Y. J. Kim, “Formative and Stealth Assessment,” in Handbook of Research on Educational Communications and Technology, J. M. Spector, M....
  • A. Mitrovic, M. Mayo, P. Suraweera, B. Martin, “Constraint-Based Tutors: A Success Story,” in Engineering of Intelligent Systems, Berlin,...
  • P. Suraweera, A. Mitrovic, “An Intelligent Tutoring System for Entity Relationship Modelling,” International Journal of Artificial Intelligence...
  • L. Zhuhadar, S. Marklin, E. Thrasher, M. D. Lytras, “Is there a gender difference in interacting with intelligent tutoring system? Can Bayesian Knowledge...
  • M. V. Yudelson, K. R. Koedinger, G. J. Gordon, “Individualized Bayesian Knowledge Tracing Models,” in Artificial Intelligence in Education,...
  • W. M. P. van der Aalst, Process Mining Data Science in Action. Berlin Heidelberg: Springer, 2nd ed. ed., 2016.
  • K. Engelmann, M. Bannert, “Analyzing temporal data for understanding the learning process induced by metacognitive prompts,” Learning and...
  • H. A. V. D. Berg, “Occam’s razor: From ockham’s via moderna to modern data science,” Science Progress, vol. 101, no. 3, pp. 261–272, 2018,...
  • M. de Leoni, W. M. van der Aalst, M. Dees, “A general process mining framework for correlating, predicting and clustering dynamic behavior...
  • A. Bogarín, R. Cerezo, C. Romero, “A survey on educational process mining,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery,...
  • A. Bolt, M. de Leoni, W. M. P. van der Aalst, P. Gorissen, “Exploiting Process Cubes, Analytic Workflows and Process Mining for Business Process...
  • J. A. Caballero-Hernández, A. Balderas, M. Palomo- Duarte, P. Delatorre, A. J. Reinoso, J. M. Dodero, “Teamwork assessment in collaborative projects...
  • P. Reimann, “Time is precious: Variable- and event- centred approaches to process analysis in CSCL research,” International Journal of ComputerSupported...
  • M. Bannert, P. Reimann, C. Sonnenberg, “Process mining techniques for analysing patterns and strategies in students’ self-regulated learning,” Metacognition...
  • P. Reimann, L. Markauskaite, M. Bannert, “e-Research and learning theory: What do sequence and process mining methods contribute?,” British...
  • H. M. W. Verbeek, J. C. A. M. Buijs, B. F. Van Dongen, W. M. van der Aalst, “ProM: The Process Mining Toolkit,” in International Conference on...
  • V. U. Kumar, A. Krishna, P. Neelakanteswara, C. Z. Basha, “Advanced prediction of performance of a student in an university using machine learning...
  • S. A. Alasadi, W. S. Bhaya, “Review of data preprocessing techniques in data mining,” Journal of Engineering and Applied Sciences, vol. 12,...
  • S. J. J. Leemans, D. Fahland, W. M. P. van der Aalst, “Discovering BlockStructured Process Models from Event Logs Containing Infrequent Behaviour,”...
  • A. Rozinat, W. M. P. van der Aalst, “Conformance checking of processes based on monitoring real behavior,” Information Systems, vol. 33, no....
  • W. van der Aalst, A. Adriansyah, B. van Dongen, “Replaying history on process models for conformance checking and performance analysis,” WIREs...
  • C. Argyris, D. A. Schön, “Participatory action research and action science compared: A commentary,” American Behavioral Scientist, vol. 32,...
  • G. E. Mills, Action research: A guide for the teacher researcher. Boston: Pearson, 4th ed., 2011.
  • ACM, IEEE, “Computer Engineering Curricula 2016,” ACM, IEEE, 2016.
  • J. A. Caballero-Hernández, “Supporting skill assessment in learning experiences based on serious games through process mining techniques,” 2020....
  • A. Silberschatz, H. F. Korth, S. Sudarshan, Database system concepts. New York: McGraw-Hill, 6th ed. ed., 2011.