Automating repetitive tasks has long been a priority for many organizations and has been extensively studied within the field of process science. Over the last decade, Robotic Process Automation (RPA) has emerged as a highly effective method to achieve this goal. RPA enables experts to automate and integrate information systems using graphical user interfaces, offering a fast and efficient solution for repetitive task automation. Rather than constructing software robots from scratch, Robotic Process Mining (RPM) and Task Mining (TM) approaches can be used to monitor user behavior through timestamped events—such as mouse clicks and keystrokes—which are recorded in a User Interface log (UI Log) to automatically discover the underlying process model.
A significant challenge in outsourcing environments, where remote virtualized systems are commonly used, is the limited information available from traditional UI logs. These logs do not capture visual context, making it difficult to identify user activities and understand decision-making processes, especially when multiple process variants exist. Existing approaches analyze the UI Log to identify underlying rules but often neglect what is displayed on the screen, resulting in an incomplete understanding of the process.
To overcome these limitations, this dissertation proposes a screen-based task mining framework that enriches UI logs by incorporating visual information through screenshots and eye-tracking data captured during each interaction. This enriched log not only improves the identification of process activities but also enables the discovery of decision models, offering a more comprehensive understanding of human behavior —particularly in outsourcing contexts. By using image-processing techniques to extract relevant visual details from the screenshots, this approach extends the current capabilities of task mining, allowing for the construction of decision models that explain user choices in greater depth.
These decision models are represented as decision trees, which explicitly highlight the visual elements that influence decision-making. The proposed framework has been validated through multiple case studies involving both synthetic mockups and real-life screenshots, demonstrating a high level of accuracy in capturing user decisions. The results indicate that the overall approach significantly enhances the effectiveness of task mining, revealing information previously hidden in traditional log analysis, and has the potential to revamp the outsourcing industry by improving automation applications in this type of environments.
© 2008-2025 Fundación Dialnet · Todos los derechos reservados