Ir al contenido

Documat

Decoding Latent Spaces: Assessing the Interpretability of Time Series Foundation Models for Visual Analytics

  • Inmaculada Santamaría-Valenzuela [1] ; Víctor Rodríguez Fernández [1] ; Javier Huertas Tato [1] Árbol académico ; Jong Hyuk Park [2] ; David Camacho [1]
    1. [1] Universidad Politécnica de Madrid

      Universidad Politécnica de Madrid

      Madrid, España

    2. [2] Seoul National University of Science and Technology

      Seoul National University of Science and Technology

      Corea del Sur

  • Localización: IJIMAI, ISSN-e 1989-1660, Vol. 9, Nº. 7, 2026, págs. 47-66
  • Idioma: inglés
  • Enlaces
  • Resumen

    • The present study explores the interpretability of latent spaces produced by time series foundation models, focusing on their potential for visual analysis tasks. Specifically, we evaluate the MOMENT family of models, a set of transformer-based, pre-trained architectures for multivariate time series tasks such as: imputation, prediction, classification, and anomaly detection. We evaluate the capacity of these models on five datasets to capture the underlying structures in time series data within their latent space projection and validate whether fine tuning improves the clarity of the resulting embedding spaces. Notable performance improvements in terms of loss reduction were observed after fine tuning. Visual analysis shows limited improvement in the interpretability of the embeddings, requiring further work. Results suggest that, although time series foundation models such as those in MOMENT are robust, their latent spaces may require additional methodological refinements to be adequately interpreted, such as alternative projection techniques, loss functions, or data preprocessing strategies. Despite the limitations of MOMENT, foundation models supose a big reduction in execution time and so a great advance for interactive visual analytics

  • Referencias bibliográficas
    • [1] Y. Wu, S. Gao, S. Zhang, X. Dou, X. Wang, Q. Li, “From requirement to solution: Unveiling problem-driven design patterns in visual analytics,”IEEE...
    • [2] X. Zhang, X. Yang, H. Hu, H. Qin, “Visual causal analysis of multivariate time series,” Journal of Visualization, vol. 28, no. 2, pp....
    • [3] TensorFlow, “Embedding projector - visualization of high-dimensional data,” 2023. [Online]. Available: https://projector.tensorflow.org/,...
    • [4] L. McInnes, “Interactive visualizations,” 2018. [Online]. Available: https://umap-learn.readthedocs.io/en/latest/interactive_viz.html,...
    • [5] V. Rodriguez-Fernandez, D. Montalvo-Garcia, F. Piccialli, G. J. Nalepa, D. Camacho, “Deepvats: Deep visual analytics for time series,”Knowledge-Based...
    • [6] G. Woo, C. Liu, A. Kumar, D. Sahoo, “Pushing the limits of pre-training for time series forecasting in the cloudops domain,” arXiv preprint...
    • [7] Z. Zuo, Y. Huang, Z. Li, Y. Jiang, C. Liu, “Mixed contrastive transfer learning for few-shot workload prediction in the cloud,”Computing,...
    • [8] A. G. Meyer, F. Lu, L. Clemente, M. Santillana, “A prospective realtime transfer learning approach to estimate influenza hospitalizations...
    • [9] H. I. Fawaz, G. Forestier, J. Weber, L. Idoumghar, P.-A. Muller, “Transfer learning for time series classification,” in 2018 IEEE international...
    • [10] M. Goswami, K. Szafer, A. Choudhry, Y. Cai, S. Li, A. Dubrawski, “Moment: A family of open time-series foundation models,” 2024.
    • [11] A. F. Ansari, L. Stella, C. Turkmen, X. Zhang, P. Mercado, H. Shen, O. Shchur, S. Rangapuram, S. P. Arango, S. Kapoor, J. Zschiegner,...
    • [12] G. Woo, C. Liu, A. Kumar, C. Xiong, S. Savarese, D. Sahoo, “Unified training of universal time series forecasting transformers,” in 41...
    • [13] I. Santamaria-Valenzuela, V. Rodriguez-Fernandez, D. Camacho, “On the integration of large-scale time series distance matrices into deep...
    • [14] I. Santamaria-Valenzuela, V. Rodriguez-Fernandez, D. Camacho, “Exploring scalability in large-scale time series in deepvats framework,”...
    • [15] M. Shahcheraghi, R. Mercer, J. M. d. A. Rodrigues, A. Der, H. F. S. Gamboa, Z. Zimmerman, K. Mauck, E. Keogh, “Introducing mplots: scaling...
    • [16] L. Su, X. Zuo, R. Li, X. Wang, H. Zhao, B. Huang, “A systematic review for transformer-based long-term series forecasting,” Artificial...
    • [17] A. Zeng, M. Chen, L. Zhang, Q. Xu, “Are transformers effective for time series forecasting?,” in Proceedings of the AAAI conference on...
    • [18] H. Wu, J. Xu, J. Wang, M. Long, “Autoformer: Decomposition transformers with auto-correlation for long-term series forecasting,” Advances...
    • [19] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, I. Polosukhin, “Attention is all you need,” Advances...
    • [20] Y. Wang, H. Wu, J. Dong, G. Qin, H. Zhang, Y. Liu, Y. Qiu, J. Wang, M. Long, “Timexer: Empowering transformers for time series forecasting...
    • [21] Y. Wang, N. Huang, T. Li, Y. Yan, X. Zhang, “Medformer: A multigranularity patching transformer for medical time-series classification,”...
    • [22] S. G. Dhekane, T. Ploetz, “Transfer learning in sensor-based human activity recognition: A survey,” ACM Computing Surveys, vol. 57, Mar....
    • [23] M. Awais, M. Naseer, S. Khan, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, F. S. Khan, “Foundation models defining a new era in
    • vision: a survey and outlook,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 4, pp. 2245–2264, 2025, doi: https://doi.org/10.1109/TPAMI.2024.3506283.
    • [24] A. Rupa, C. Swapna, “Advancing domain-specific adaptations of large language models through transfer learning and fine-tuning techniques:...
    • [25] Y. Yang, J. Zhou, X. Ding, T. Huai, S. Liu, Q. Chen, Y. Xie, L. He, “Recent advances of foundation language models-based continual learning:...
    • [26] W. Dai, P. Chen, M. Lu, D. Li, H. Wei, H. Cui, P. P. Liang, “Climb: Data foundations for large scale multimodal clinical foundation models,”...
    • [27] A. Garza, C. Challu, M. Mergenthaler-Canseco, “Timegpt-1,” arXiv preprint arXiv:2310.03589, 2023.
    • [28] K. Rasul, A. Ashok, A. R. Williams, A. Khorasani, Adamopoulos, R. Bhagwatkar, M. Biloš, Ghonia, N. Hassen, A. Schneider, et al., “Lag-llama:...
    • [29] L. Van der Maaten, G. Hinton, “Visualizing data using t-sne.,” Journal of machine learning research, vol. 9, no. 11, 2008.
    • [30] L. McInnes, J. Healy, J. Melville, “Umap: Uniform manifold approximation and projection for dimension reduction,” 2018. [Online]. Available:...
    • [31] Y. Wang, H. Huang, C. Rudin, Y. Shaposhnik, “Understanding how dimension reduction tools work: An empirical approach to deciphering t-sne,...
    • [32] S. Bhatnagar, A. Singh, V. Awasthi, S. SK, “Combining data mining and visualization: Umap on rope,” in 2024 International Conference...
    • [33] J. Stym-Popper, N. Painchaud, P.-Y. COURAND, C. Rambour, N. Thome, O. Bernard, “Dafted: Decoupled asymmetric fusion of tabular
    • and echocardiographic data for cardiac hypertension diagnosis,” in Proceedings of the Medical Imaging with Deep Learning Conference (MIDL...
    • [34] M. Ali, M. W. Jones, X. Xie, M. Williams, “Timecluster: dimension reduction applied to temporal data for visual analytics,” The Visual
    • Computer, vol. 35, no. 6-8, pp. 1013–1026, 2019.
    • [35] N. Andrienko, G. Andrienko, A. Artikis, P. Mantenoglou, S. Rinzivillo, “Human-in-the-loop: visual analytics for building models recognising...
    • [36] I. Oguiza, “tsai: A state-of-the-art deep learning library for time series and sequential data,” 2023. [Online]. Available: https://github.com/timeseriesAI/tsai,...
    • [37] H. Ismail Fawaz, B. Lucas, G. Forestier, C. Pelletier, D. F. Schmidt, J. Weber, G. I. Webb, L. Idoumghar, P.-A. Muller, F. Petitjean,...
    • [38] S. M. Law, “STUMPY: A Powerful and Scalable Python Library for Time Series Data Mining,” The Journal of Open Source Software, vol. 4,...
    • [39] Y. Matsubara, Y. Sakurai, C. Faloutsos, “The web as a jungle: Non-linear dynamical systems for co-evolving online activities,” in Proceedings...
    • [40] M. Middlehurst, A. Ismail-Fawaz, A. Guillaume, C. Holder, D. Guijo-Rubio, G. Bulatova, L. Tsaprounis, L. Mentel, M. Walter, P. Schäfer,...
    • [41] M. Cuturi, M. Blondel, “Soft-dtw: a differentiable loss function for timeseries,” in Proceedings of the 34th International Conference...
Fundación Dialnet

Mi Documat

Opciones de artículo

Opciones de compartir

Opciones de entorno