Automatic feedback and assessment of team‑coding assignments in a DevOps context

• Autores: Borja Fernández Gauna , Naiara Rojo, Manuel Graña Romay
• Localización: International Journal of Educational Technology in Higher Education, ISSN 2365-9440, Nº. 20, 2023
• Idioma: inglés
• DOI: 10.1186/s41239-023-00386-6
• Enlaces
  • Texto completo (pdf)
• Resumen

  • We describe an automated assessment process for team-coding assignments based on DevOps best practices. This system and methodology includes the defnition of Team Performance Metrics measuring properties of the software developed by each team, and their correct use of DevOps techniques. It tracks the progress on each of metric by each group. The methodology also defnes Individual Performance Metrics to measure the impact of individual student contributions to increase in Team Performance Met‑ rics. Periodically scheduled reports using these metrics provide students valuable feed‑ back. This process also facilitates the process of assessing the assignments. Although this method is not intended to produce the fnal grade of each student, it provides very valuable information to the lecturers. We have used it as the main source of informa‑ tion for student and team assessment in one programming course. Additionally, we use other assessment methods to calculate the fnal grade: written conceptual tests to check their understanding of the development processes, and cross-evaluations.

    Qualitative evaluation of the students flling relevant questionnaires are very positive and encouraging.

• Referencias bibliográficas
  • Almeida, F., Simoes, J., & Lopes, S. (2022). Exploring the benefits of combining devops and agile. Future Internet, 14(2), 63. https://doi.org/10.3390/fi14020063
  • Assyne, N., Ghanbari, H., & Pulkkinen, M. (2022). The state of research on software engineering competencies: a systematic mapping study....
  • Britton, E., Simper, N., Leger, A., & Stephenson, J. (2017). Assessing teamwork in undergraduate education: a measurement tool to evaluate...
  • Cai, Y. & Tsai, M. (2019). Improving programming education quality with automatic grading system. Lecture Notes in Computer Science (including...
  • Clifton, C., Kaczmarczyk, L., & Mrozek, M. (2007). Subverting the fundamentals sequence: Using version control to enhance course management....
  • Clune, J., Ramamurthy, V., Martins, R., & Acar, U. (2020). Program equivalence for assisted grading of functional programs. Proceedings...
  • Coleman, D., Ash, D., Lowther, B., & Oman, P. (1994). Using metrics to evaluate software system maintainability. IEEE Computer, 27(8),...
  • Cortes Rios, J., Embury, S., & Eraslan, S. (2022). A unifying framework for the systematic analysis of git workflows. Information and...
  • De Prada, E., Mareque, M., & Pino‑Juste, M. (2022). Teamwork skills in higher education: is university training contributing to their...
  • Gaona, E., Perez, C., Castro, W., Morales Castro, J. C., Sanchez Rodriguez, A., & Avila‑Garcia, M. (2021). Automatic grading of programming...
  • Gonzalez‑Carrillo, C., Calle‑Restrepo, F., Ramirez‑Echeverry, J., & Gonzalez, F. (2021). Automatic grading tool for jupyter notebooks...
  • Gordillo, A. (2019). Effect of an instructor‑centered tool for automatic assessment of programming assignments on students’ perceptions and...
  • Hamer, S., Lopez‑Quesada, C., Martinez, A., & Jenkins, M. (2021). Using git metrics to measure students’ and teams’ code contributions...
  • Hegarty‑E, K. & Mooney, D. (2021). Analysis of an automatic grading system within first year computer science programming modules. pp....
  • Heitlager, I., Kuipers, T., & Visser, J. (2007). A practical model for measuring maintainability. In 6th International Conference on the...
  • Holck, J. & Jorgensen, N. (2012). Continuous integration and quality assurance: A case study of two open source projects. Australasian...
  • Insa, D., Perez, S., Silva, J., & Tamarit, S. (2021). Semiautomatic generation and assessment of java exercises in engineering education....
  • ISO, IEC,. (2001). ISO/IEC 9126. Software engineering—Product quality: ISO/IEC. https://doi.org/10.1016/j.ijme.2021.100538
  • Jurado, F. (2021). Teacher assistance with static code analysis in programming practicals and project assignments.https://doi. org/10.1109/SIIE53363.2021.9583635
  • Khan, M., Khan, A., Khan, F., Khan, M., & Whangbo, T. (2022). Critical challenges to adopt devops culture in software organizations: A...
  • Le Minh, D. (2021). Model‑based automatic grading of object‑oriented programming assignments. Computer Applications in Engineering Education....
  • Liu, X., Wang, S., Wang, P., & Wu, D. (2019). Automatic grading of programming assignments: An approach based on formal semantics. pp....
  • Macak, M., Kruzelova, D., Chren, S., & Buhnova, B. (2021). Using process mining for git log analysis of projects in a software development...
  • Oman, R., & Hagemeister, J. R. (1994). Construction and testing of polynomials predicting software maintainability. Journals of Systems...
  • Parihar, S., Das, R., Dadachanji, Z., Karkare, A., Singh, P., & Bhattacharya, A. (2017). Automatic grading and feedback using program...
  • Perez‑Verdejo, J., Sanchez‑Garcia, A., Ocharan‑Hernandez, J., Mezura‑E, M., & Cortes‑Verdin, K. (2021). Requirements and github issues:...
  • Petkova, A. P., Domingo, M. A., & Lamm, E. (2021). Let’s be frank: Individual and team‑level predictors of improvement in student teamwork...
  • Planas‑Llado, A., Feliu, L., Arbat, G., Pujol, J., Sunol, J. J., Castro, F., & Marti, C. (2021). An analysis of teamwork based on self...
  • Rubinstein, A., Parzanchevski, N., & Tamarov, Y. (2019). In‑depth feedback on programming assignments using pattern recognition and real‑time...
  • Saidani, I., Ouni, A., & Mkaouer, M. (2022). Improving the prediction of continuous integration build failures using deep learning. Automated...
  • Strandberg, P., Afzal, W., & Sundmark, D. (2022). Software test results exploration and visualization with continuous integration and...
  • Theunissen, T., van Heesch, U., & Avgeriou, P. (2022). A mapping study on documentation in continuous software development. Information...
  • von Wangenheim, C.G., Hauck, J.C.G., Demetrio, M.F., Pelle, R., da Cruz Alvez, N., Barbosa, H., Azevedo, L.F. (2018). Codemaster‑automatic...
  • Wang, Y., Mantyla, M., Liu, Z., & Markkula, J. (2022). Test automation maturity improves product quality‑quantitative study of open source...
  • Wunsche, B., Suselo, T., Van Der W, M., Chen, Z., Leung, K., Reilly, L., Shaw, L., Dimalen, D., & Lobb, R. (2018). Automatic assessment...
  • Youngtaek, K., Jaeyoung, K., Hyeon, J., Young‑Ho, K., Hyunjoo, S., Bohyoung, K., & Jinwook, S. (2021). Githru: Visual analytics for understanding...