Inclusive Learning through Immersive Virtual Reality and Semantic Embodied Conversational Agent: A case study in children with autism

• Yoselie Alvarado ^[1] ; Roberto Guerrero ^[1] ; Francisco Serón ^[2]
  1. [1] Universidad Nacional de San Luis

     Universidad Nacional de San Luis

     Argentina

     
  2. [2] Universidad de Zaragoza

     Universidad de Zaragoza

     Zaragoza, España

     
• Localización: Journal of Computer Science and Technology, ISSN-e 1666-6038, Vol. 23, Nº. 2, 2023
• Idioma: inglés
• DOI: 10.24215/16666038.23.e09
• Títulos paralelos:
  • Aprendizaje Inclusivo mediante Realidad Virtual Inmersiva y Agente Conversacional Semántico Encarnado: Un caso de estudio en niños con autismo
• Enlaces
  • Texto completo
• Resumen
  • español

    El Objetivo de Desarrollo Sostenible 4 de la Agenda 2030 de las Naciones Unidas se centra en garantizar una educación inclusiva, equitativa y de calidad y promover oportunidades de aprendizaje permanente para todos. La educación inclusiva es un componente clave del Objetivo de Desarrollo Sostenible 4, y la tecnología de apoyo es un factor crítico para lograrlo. Este trabajo presenta el desarrollo de un sistema de realidad virtual inmersiva con habilidades conversacionales para procesos de aprendizaje inclusivo. La idea de este trabajo es proporcionar una tecnología de apoyo para motivar, educar y formar a estudiantes discapacitados de una forma más inclusiva. Para mejorar la experiencia del usuario, el sistema fue desarrollado para trabajar en un entorno inmersivo tipo CAVE utilizando Procesamiento del Lenguaje Natural con un Agente Conversacional Semántico Encarnado. Este trabajo pone de relieve que la tecnología de apoyo puede promover beneficios educativos, psicológicos y sociales para los estudiantes con discapacidad. El uso de tecnología de apoyo puede permitir el compromiso académico y la participación social y ser transformador desde una perspectiva psicológica. Se realizó un caso de estudio en niños con autismo, que mostró resultados alentadores del sistema como tecnología de asistencia para la enseñanza y el aprendizaje.

  • English

    Sustainable Development Goal 4 from the United Nations 2030 Agenda, focus on ensuring inclusive and equitable quality education and promoting lifelong learning opportunities for all. Inclusive education is a key component of sustainable development goal 4, and assistive technology is a critical factor in achieving it. In this context, this paper introduces the development of an immersive virtual reality system with conversational skills for inclusive learning processes. The idea of this work is to provide an assistive technology to motivate, educate, and train students with disabilities in a more inclusive way. In order to improve the user experience, the system was developed to work in a CAVE-like immersive environment using Natural Language Processing through a Semantic Embodied Conversational Agent. This work highlights that assistive technology can promote educational, psychological, and social benefits for students with disabilities. The use of assistive technology can enable academic engagement and social participation and be transformative from a psychological perspective. A case study was conducted on children with autism, which showed encouraging results of the system as an assistive technology for teaching and learning.

• Referencias bibliográficas
  • L. Maftei and C. Harty, “Surprise: challenging design perceptions in immersive virtual reality environments? The case of designing a hospital...
  • X. Chen, Z. Chen, Y. Li, T. He, J. Hou, S. Liu, and Y. He, “ImmerTai: Immersive motion learning in VR environments,” Journal of Visual Communication...
  • L. Wu, B. Sekelsky, M. Peterson, T. Gampp, C. Delgado, and K. B. Chen, “Immersive virtual environment for scale cognition and learning: Expert-based...
  • T. T. de Back, A. M. Tinga, and M. M. Louwerse, “CAVE-based immersive learning in undergraduate courses: examining the effect of group size...
  • G. C. Guerino and N. M. C. Valentim, “Usability and user experience evaluation of natural user interfaces: a systematic mapping study,” IET...
  • J. Cassell, “Embodied Conversational Agents: Representation and Intelligence in User Interfaces,” AI Magazine, vol. 22, pp. 67–84, 2001.
  • J. Eisenstein, Introduction to natural language processing. MIT press, 2019.
  • S. Ruder, M. E. Peters, S. Swayamdipta, and T. Wolf, “Transfer Learning in Natural Language Processing,” in Proceedings of the 2019 Conference...
  • T. Sun et al., “Mitigating Gender Bias in Natural Language Processing: Literature Review,” in Proceedings of the 57th Annual Meeting of the...
  • M. Bahja, ‘Natural Language Processing Applications in Business’, E-Business - Higher Education and Intelligence Applications. IntechOpen,...
  • “Laboratorio de Computación Gráfica.” Available at: http://www.lcg.unsl.edu.ar. Accessed on 2023/07/15.
  • “Grupo de Informática Gráfica Avanzada.” Available at: http://giga.cps.unizar.es. Accessed on 2023/07/15.
  • Y. Alvarado, N. Moyano, D. Quiroga, J. Fernández, and R. Guerrero, “Augmented Virtual Realities for Social Developments,” in Experiences between...
  • N. Jofré Pasinetti, G. Rodríguez, Y. Alvarado, J. Fernandez, and R. A. Guerrero, “Towards defining a CAVE like system performance evaluation,”...
  • S. Baldassarri, E. Cerezo, and F. J. Seron, “Maxine: A platform for embodied animated agents,” Computers & Graphics, vol. 32, no. 4, pp....
  • F. J. Serón and C. Bobed, “VOX system: a semantic embodied conversational agent exploiting linked data,” Multimedia Tools and Applications,...
  • D. Chambers, “Assistive Technology Supporting Inclusive Education: Existing and Emerging Trends,” in Assistive Technology to Support Inclusive...
  • P. Encarnação et al., “Using assistive robots to promote inclusive education.,” Disability and Rehabilitation: Assistive Technology, vol....
  • W.-T. Tsai, I.-J. Lee, and C.-H. Chen, “Inclusion of third-person perspective in CAVE-like immersive 3D virtual reality role-playing games...
  • K. Yousaf et al., “A comprehensive study of mobile-health based assistive technology for the healthcare of dementia and Alzheimer’s disease...
  • L. Daniela and M. D. Lytras, “Educational Robotics for Inclusive Education,” Technology, Knowledge and Learning, vol. 24, no. 2, pp. 219–225,...
  • B. Meskhi, S. Ponomareva, and E. Ugnich, “E-learning in higher inclusive education: needs, opportunities and limitations,” International Journal...
  • S. Stenman and F. Pettersson, “Remote teaching for equal and inclusive education in rural areas? An analysis of teachers’ perspectives on...
  • M. Shahab et al., “Utilizing social virtual reality robot (V2R) for music education to children with high-functioning autism,” Education and...
  • A. Boza-Chua, K. Gabriel-Gonzales, and L. Andrade-Arenas, “Inclusive Education: Implementation of a Mobile Application for Blind Students,”...
  • S. Yuko, W. Hiroki, and A. Naoki, “System Development of Transformable Tactile Tools on VR for Blind Children,” in Proceedings of the Fifteenth...
  • J. Kwon and Y. Lee, “Serious games for the job training of persons with developmental disabilities,” Computers & Education, vol. 95, pp....
  • V. Krasniqi, N. Ackovska, and K. Zdravkova, “Emerging Role of Robot-Assisted Occupational Therapy for Children with Down Syndrome,” in Proceedings...
  • L. Cuascota, L. Guevara, R. Cueva, F. Tapia, and G. Guerrero, “Assistance application of people with cognitive disabilities in tasks for their...
  • I. Svensson et al., “Effects of assistive technology for students with reading and writing disabilities,” Disability and Rehabilitation: Assistive...
  • Y. Alvarado, Agentes Semánticos Conversacionales y Entornos CAVE. Computer Science Master Thesis, Universidad Nacional de San Luis, 2022.
  • D. Khurana, A. Koli, K. Khatter, and S. Singh, “Natural language processing: state of the art, current trends and challenges,” Multimedia...
  • “DBpedia Association.” Available at: http://www.dbpedia.org. Accessed on 2023/07/15.
  • F. Seron et al., “A semantic memory bank assisted by an embodied conversational agent for mobile devices,” Engineering and Applied Sciences,...
  • “Alphabet Lore” Available at: http:// www.imdb.com/title/tt21923830/. Accessed on 2023/07/15.
  • M. M. Bradley and P. J. Lang, “Measuring emotion: the self-assessment manikin and the semantic differential,” Journal of behavior therapy...