Resumen de Quality evaluation in education: two measurement instruments for virtual learning environments
Pilar Gómez Rey
There has been growing interest in online learning in the past decades as a mode of education. Despite this, online educational institutions are still struggling to gain the acceptance and trust of students, instructors and society in general. Therefore it is essential to define and create quality measurement mechanisms in order to increase people’s trust in online education. This dissertation focuses on the area of quality in virtual education environments and specifically, quality assurance. Several research publications aimed at assessing quality of online programs are reviewed, taking into account their diverse domains (different variables of online programs) at different levels of abstraction.
After performing a detailed analysis of the state-of-the-art, the following two gaps were identified in the literature: (i) many research studies focus on a specific aspect of the teaching- learning process (course technology, student satisfaction, quality of instruction, etc.) that somewhat ignores the holistic nature of educational quality; and (ii) most of the research in the literature measures the quality of a program by only taking into account a single group of actors in the educational process (administrators, instructors or students). Motivated by these gaps, this dissertation presents an educational model that considers the variables involved in the entire teaching-learning process (input-process-outcome model). Specifically, the quality of the program is evaluated jointly by instructors and students through a quality index which includes an additional parameter that can be configured by administrators. This parameter allows administrators to weight the importance of instructors’ and/or students’ perceptions. We use quantitative approaches methodologically. We have implemented mainly factor analysis, clustering and non-parametric analysis testing in the first part of the dissertation. These issues are presented in the first part of the dissertation: “Critical success factors in online education”. We published three research studies in high-impact journals.
The perception of quality in educational environments depends, to a great extent, on the actors who evaluate the quality. In fact, as part of the core contribution of the first part of the dissertation, we stress that, while instructors perceive collaborative learning variables as crucial, learners are more concerned with their own learning benefits. In addition, our study reveals instruction itself to be the main critical factor in student achievement rather than the technological factor. This finding is important considering the nature of the learning environment under study. Motivated by this issue, the we have been researching into he topic of online instructor roles. This research problem is highlighted in the second part of the dissertation: “Roles of online instructors”. We published two research works in high-impact journals and two more papers are pending publication in top-tier journals.
The second part of the dissertation presents an educational model based on online instructors’ six different roles. The proposed model takes into account values that are promoted by organizations such as UNESCO for the 21st century and, therefore, the model comprises new teaching functions with respect to the literature. Furthermore, the quality effectiveness of the online instructor in asynchronous and text-based learning environments is analysed using the Universitat Oberta de Catalunya (Spain) as the case study. The second part of the dissertation ends with the presentation of a quality index that relates the instructor’s ove- rall performance rating (according to students’ perceptions) with the course characteristics, student demographics and the effectiveness of the instructor in his/her teaching roles. Both quantitative (exploratory and confirmatory factor analysis and non-parametric analysis) and qualitative analysis (content analysis) were used in the second part of the dissertation. The main results are summarized as follows: (i) a new role, the life skills promoter, has emerged; (ii) instructors should consider using more audio-visual resources and (iii) the instructor’s overall performance rating is mainly influenced by the effectiveness of the instructor in his/her teaching roles.
