Sistema de Rehabilitación de Muñeca con Realidad Aumentada mediante Eliminación Visual del Robot utilizando inpainting

• Benavides, Diego ^[1] ; Cisnal, Ana ^[1] ; de la Fuente, Eusebio ^[1] ; Fraile, Juan Carlos ^[1] ; Pérez Turiel, Javier ^[1]
  1. [1] Universidad de Valladolid

     Universidad de Valladolid

     Valladolid, España

     
• Localización: Jornadas de Automática, ISSN-e 3045-4093, Nº. 46, 2025
• Idioma: español
• DOI: 10.17979/ja-cea.2025.46.12152
• Títulos paralelos:
  • Immersive AR-Based Wrist Rehabilitation with Visual Robotic Removal via Image Inpainting
• Enlaces
  • Texto completo
• Resumen
  • español

    Este trabajo presenta un sistema innovador para la rehabilitación de la muñeca que combina un dispositivo robótico de asistencia con una plataforma de realidad aumentada inmersiva. Para mejorar la experiencia del paciente y favorecer los mecanismos de neuroplasticidad, se presenta un algoritmo de eliminación visual del robot mediante técnicas de segmentación y relleno de imagen (inpainting). El sistema integra un modelo de segmentación basado en UNet y un modelo de inpainting entrenado específicamente, logrando reconstruir la escena de forma realista tras eliminar el dispositivo robótico. Se ha evaluado el desempeño de distintos modelos de inpainting mediante métricas tradicionales y perceptuales, demostrando mejoras significativas en la calidad de las imágenes reconstruidas y en la inmersión del entorno de terapia. Estos resultados abren la puerta a una rehabilitación más eficaz y motivadora, combinando los beneficios funcionales de la robótica con el realismo de la realidad aumentada.

  • English

    This work presents an innovative system for wrist rehabilitation that combines an assistive robotic device with an immersive augmented reality (AR) platform. To enhance the patient's experience and support neuroplasticity mechanisms, a visual robot removal algorithm is proposed using image segmentation and inpainting techniques. The system integrates a UNet-based segmentation model and a specifically trained inpainting model, effectively reconstructing the scene realistically after removing the robotic device. The performance of different inpainting models has been evaluated using both traditional and perceptual metrics, showing significant improvements in the quality of the reconstructed images and the immersive experience of the therapy environment. These results pave the way for more effective and engaging rehabilitation, combining the functional benefits of robotics with the realism of augmented reality.

• Referencias bibliográficas
  • Alonso-Linaje, G., Cisnal, A., Fraile, J. C., Turiel, J. P., 2024. Design and analysis of the m3rob: A robotic platform for wrist and hand...
  • Brizuela, G. S., Rodriguez, D. S., de la Fuente Lopez, E., Fraile, J. C., Turiel, J. P., 2022. Segmentación semántica de elementos mecánicos...
  • Chen, T., Zhang, J., Hong, Y., Zhang, Y., Zhang, L., 2024. Assessing image inpainting via re-inpainting self-consistency evaluation. In: Proceedings...
  • Cisnal, A., Alonso-Linaje, G., Veganzones, M., amd Juan Carlos Fraile, J. P. T., 2023. M3display: Sistema de realidad aumentada para la rehabilitación...
  • Corneanu, C., Gadde, R., Martinez, A. M., January 2024. Latentpaint: Image inpainting in latent space with diffusion models. In: Proceedings...
  • Fernández-Sánchez, M., Aza-Hernández, A., Verdugo-Alonso, M., 4 2022. Models of public care for the population with acquired brain injury...
  • Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y., 6 2014. Generative adversarial...
  • Hao, J., Xie, H., Harp, K., Chen, Z., Siu, K.-C., 2022. Effects of virtual reality intervention on neural plasticity in stroke rehabilitation:...
  • Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., Hochreiter, S., 6 2017. Gans trained by a two time-scale update rule converge to a...
  • Ho, J., Jain, A., Abbeel, P., 6 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 2020-December....
  • Horé, A., Ziou, D., 2010. Image quality metrics: PSNR vs. SSIM. In: 2010 20th International Conference on Pattern Recognition. pp. 2366–2369....
  • INE, 2020. Encuesta de discapacidad, autonomía personal y situaciones de dependencia. 2020. https://www.ine.es/jaxi/Datos.htm?tpx=51614.
  • Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., Van Gool, L., June 2022. Repaint: Inpainting using denoising diffusion probabilistic...
  • Masmoudi, M., Zenati, N., Izountar, Y., Benbelkacem, S., Haicheur,W., Guerroudji, M. A., Oulefki, A., Hamitouche, C., 2 2024. Assessing the...
  • Polygerinos, P., Wang, Z., Galloway, K. C., Wood, R. J., Walsh, C. J., 2015. Soft robotic glove for combined assistance and at-home rehabilitation....
  • Rathore, S. S., Hinn, A. R., Cooper, L. S., Tyroler, H. A., Rosamond, W. D., 2002. Characterization of incident stroke signs and symptoms....
  • Rego, P. A., Rocha, R., Faria, B. M., Reis, L. P., Moreira, P. M., 1 2017. A serious games platform for cognitive rehabilitation with preliminary...
  • Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B., Jun. 2022. High-Resolution Image Synthesis with Latent Diffusion Models. In:...
  • Ronneberger, O., Fischer, P., Brox, T., 2015. U-net: Convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J.,...
  • Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., Chen, X., 6 2016. Improved techniques for training gans. Advances in Neural...
  • Suvorov, R., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., Lempitsky, V., 2022. Resolution-robust...
  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan D., Vanhoucke, V., Rabinovich, A., 9 2014. Going deeper with convolutions....
  • Van Rijsbergen, C. J., 1979. Information retrieval. 2nd. newton, ma. Wang, Z., Bovik, A., Sheikh, H., Simoncelli, E., 2004. Image quality...
  • World Health Organization, World Bank, 2011. World Report on Disability 2011. World Health Organization, Geneva, nLM classification: HV 1553....
  • Xiao, Z., Kreis, K., Vahdat, A., 12 2021. Tackling the generative learning trilemma with denoising diffusion gans. ICLR 2022 - 10th International...
  • Zhang, R., Isola, P., Efros, A. A., Shechtman, E., Wang, O., 1 2018. The unreasonable effectiveness of deep features as a perceptual metric....