Virtual Reality and Language Models, a New Frontier in Learning
-
-
[1] Universidad Politécnica de Valencia
Universidad Politécnica de Valencia
Valencia, España
-
- Localización: IJIMAI, ISSN-e 1989-1660, Vol. 8, Nº. 5, 2024, págs. 46-54
- Idioma: inglés
- DOI: 10.9781/ijimai.2024.02.007
- Enlaces
-
Resumen
The proposed research introduces an innovative Virtual Reality (VR) and Large Language Model (LLM) architecture to enhance the learning process across diverse educational contexts, ranging from school to industrial settings. everaging the capabilities of LLMs and Retrieval-Augmented Generation (RAG), the architecture centers around an immersive VR application. This application empowers students of all backgrounds to interactively engage with their environment by posing questions and receiving informative responses in text format and with visual hints in VR, thereby fostering a dynamic learning experience. LLMs with RAG act as the backbones of this architecture, facilitating the integration of private or domain-specific data into the learning process. By seamlessly connecting various data sources through data connectors, RAG overcomes the challenge of disparate and siloed information repositories, including APIs, PDFs, SQL databases, and more. The data indexes provided by RAG solutions further streamline this process by structuring the ingested data into formats optimized for consumption by LLMs. An empirical study was conducted to evaluate the effectiveness of this VR and LLM architecture. Twenty participants, divided into Experimental and Control groups, were selected to assess the impact on their learning process. The Experimental group utilized the immersive VR application, which allowed interactive engagement with the educational environment, while the Control group followed traditional learning methods. The study revealed significant improvements in learning outcomes for the Experimental group, demonstrating the potential of integrating VR and LLMs in enhancing comprehension and engagement in learning contexts. This study presents an innovative approach that capitalizes on the synergy between LLMs and immersive VR technology, opening avenues for a transformative learning experience that transcends traditional boundaries and empowers learners across a spectrum of educational landscapes.
-
Referencias bibliográficas
- A. Cordero, C. J. Lluch, E. Sanabria Codesal, J. Torregrosa, “Towards a Better Learning Models Through OCWs and MOOCs,” International Journal...
- B. Christian, C. Salvador, G. Christian, “Virtual Reality (VR) in Superior Education Distance Learning: A Systematic Literature Review,” JOIV: International...
- M. Figueiredo, R. Mafalda, A. Kamensky, “Virtual reality as an educational tool for elementary school,” in Smart Innovation, Systems and Technologies,...
- C. P. Fabris, J. A. Rathner, A. Y. Fong, C. P. Sevigny, “Virtual Reality in Higher Education,” International Journal of Innovation in Science...
- L. J. Ausburn, F. B. Ausburn, “Desktop virtual reality: A powerful new technology for teaching and research in industrial teacher education,” Journal...
- S. H. Alshammari, “The Role of Virtual Reality in Enhancing Students’ Learning,” International Journal of Educational Technology and Learning, vol....
- E. Ai-Lim Lee, K. Wai Wong, “A Review of Using Virtual Reality for Learning,” Transactions on edutainment I, pp. 231–241, 2008.
- M. M. Asad, A. Naz, P. Churi, M. M. Tahanzadeh, “Virtual Reality as Pedagogical Tool to Enhance Experiential Learning: A Systematic Literature...
- G. Zakaria, S. Wilkie, “Applications for virtual reality experiences in tertiary education,” ASCILITE Publications, pp. 186–193, 2020.
- D. W. Carruth, “Virtual reality for education and workforce training,” in 2017 15th International Conference on Emerging eLearning Technologies and...
- K. Oiwake, K. Komiya, H. Akasaki, T. Nakajima, “VR classroom: enhancing learning experience with virtual class rooms,” in 2018 Eleventh International...
- S. R. B. Hunvik, F. Lindseth, “Making Use of Virtual Reality for Artificial Intelligence Education,” in Bridges and Mediation in Higher Distance Education:...
- P. Smutny, M. Babiuch, P. Foltynek, “A review of the virtual reality applications in education and training,” in 2019 20th International Carpathian...
- M. A. Lopez, S. Terron, J. M. Lombardo, R. Gonzalez- Crespo, “Towards a solution to create, test and publish mixed reality experiences for occupational...
- J. M. Flores-Vivar, F. J. García-Peñalvo, “Reflections on the ethics, potential, and challenges of artificial intelligence in the framework...
- L. Chen, P. Chen, Z. Lin, “Artificial Intelligence in Education: A Review,” IEEE Access, vol. 8, pp. 75264– 75278, 2020, doi: 10.1109/ACCESS.2020.2988510.
- M. Fahimirad, S. S. Kotamjani, “A Review on Application of Artificial Intelligence in Teaching and Learning in Educational Contexts,” International...
- J. Huang, S. Saleh, Y. Liu, “A review on artificial intelligence in education,” Academic Journal of Interdisciplinary Studies, vol. 10, pp....
- D. Leiker, A. R. Gyllen, I. Eldesouky, M. Cukurova, “Generative AI for learning: Investigating the potential of synthetic learning videos,”...
- E. R. Bekeš, V. Galzina, “Utilizing smart digital technology and artificial intelligence in education for transforming the way content is delivered,”...
- K. Mondal, “A Synergy of Artificial Intelligence and Education in the 21 st Century Classrooms,” in 2019 International Conference on Digitization (ICD),...
- B. Du Boulay, “Artificial intelligence as an effective classroom assistant,” IEEE Intelligent Systems, vol. 31, no. 6, pp. 76–81, 2016.
- H. Yang, “Explore How Artificial Intelligence and VR Technology will Change the Development of Future Education,” Journal of Physics: Conference...
- S. Sarsa, P. Denny, A. Hellas, J. Leinonen, “Automatic Generation of Programming Exercises and Code Explanations Using Large Language Models,”...
- D. Leiker, S. Finnigan, A. Ricker Gyllen, M. Cukurova, “Prototyping the use of Large Language Models (LLMs) for adult learning content creation at...
- A. Katz, U. Shakir, B. Chambers, “The Utility of Large Language Models and Generative AI for Education Research,” arXiv preprint arXiv:2305.18125,...
- L. Yan, L. Sha, L. Zhao, Y. Li, R. Martinez-Maldonado, G. Chen, X. Li, Y. Jin, D. Gašević, “Practical and Ethical Challenges of Large Language Models...
- P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-T. Yih, T. Rocktäschel, S. Riedel, D. Kiela, “RetrievalAugmented...
- M. Ranjit, G. Ganapathy, R. Manuel, T. Ganu, “Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models,” arXiv preprint arXiv:2305.03660,...
- W. Yu, “Retrieval-augmented generation across heterogeneous knowledge,” in Proceedings of the 2022 Conference of the North American Chapter...
- W. Chen, H. Hu, X. Chen, P. Verga, W. W. Cohen, “MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and...
- R. Zhao, H. Chen, W. Wang, F. Jiao, X. L. Do, C. Qin, B. Ding, X. Guo, M. Li, X. Li, S. Joty, “Retrieving Multimodal Information for Augmented Generation:...
- S. Siriwardhana, R. Weerasekera, E. Wen, T. Kaluarachchi, R. Rajib, S. Nanayakkara, “Improving the Domain Adaptation of Retrieval Augmented...
- Y. Mao, P. He, X. Liu, Y. Shen, J. Gao, J. Han, W. Chen, “GenerationAugmented Retrieval for Open-domain Question Answering,” arXiv preprint...
- J. Liu, “LlamaIndex,” 11 2022. [Online]. Available: https://github.com/jerryjliu/llama_index.
- S. Es, J. James, L. Espinosa-Anke, S. Schockaert, “RAGAS: Automated Evaluation of Retrieval Augmented Generation,” arXiv preprint arXiv:2309.15217,...
- F. J. García Peñalvo, F. Llorens-Largo, J. Vidal, “The new reality of education in the face of advances in generative artificial intelligence,” RIED-Revista...
- F. García-Peñalvo, A. Vázquez-Ingelmo,“What Do We Mean by GenAI? A Systematic Mapping of The Evolution, Trends, and Techniques Involved in...
