Asymmetric hidden Markov models and extensions applied to industry

• Autores: Carlos Puerto Santana
• Directores de la Tesis: Concha Bielza Lozoya (dir. tes.) , Pedro Larrañaga Múgica (codir. tes.)
• Lectura: En la Universidad Politécnica de Madrid ( España ) en 2023
• Idioma: inglés
• Tribunal Calificador de la Tesis: Antonio Jiménez Martín (presid.) , Bojan Mihaljevic (secret.) , Joao Gama (voc.) , Abraham Otero Quintana (voc.) , Javier Diaz Rozo (voc.)
• Programa de doctorado: Programa de Doctorado en Inteligencia Artificial por la Universidad Politécnica de Madrid
• Menciones:
  • Internacional
• Calificación: Sobresaliente Cum Laude
• Enlaces
  • Tesis en acceso abierto en: Archivo Digital UPM
• Resumen

  • The Internet of things is a paradigm with the goal of creating customizable goods and services based on user experience. This paradigm has been applied in industrial environments generating what is called the industrial Internet of things. This new paradigm measures industrial assets continuously. The collected information is processed to extract data insights regarding the assets status or health. Depending on the asset status, maintenance policies can be planned to prevent the assets failure or degradation. To elaborate such insights, artificial intelligence models are usually applied to learn and predict industrial data patterns and behaviors. Nonetheless, in many cases, computational and reliability estrictions are present,and fast and explainable models are required to satisfy the industry needs. Hidden Markov models (HMMs) are statistical models that are capable of learning data patterns and detect non-stationary behavior in data. When HMMs are compared to other models, HMMs are economic, explainable and reliable. They are economic because their learning and inference algorithms can run in a reasonable time without the need for graphic cards or other power-intensive computing devices. They are explainable since all the learned parameters are interpretable from a probabilistic and area-of-knowledge points of view. They are reliable, because, if a mistake is committed by the model, it is possible to detect and infer its causes from the models parameters and structure. Due to the previous discussion, the motivation and results of this thesis aim to extend theoretically current HMMs, to make them more relevant, general and useful for industrial applications. For all the proposed models, the expectation-maximization algorithm was used for the learning phase. The first contribution appears in Chapter 4, where context-specific Bayesian networks were used to model the emission probabilities of continuous variables. That model is referred as AsLG-HMM since linear Gaussian Bayesian networks were used. The model was compared to a mixture of Gaussian HMM, where improvements in log-likelihoods by the proposed model were observed in both synthetic data and real data from ball-bearings. Nonetheless, such model was further developed in Chapter 5, where autoregressive values of the observable variables were considered in the context-specific Bayesian networks. This model is referred to AR-AsLG-HMM. In this case, the model was studied with further mathematical rigor. Also, a forward greedy algorithm was proposed to discover structures of Bayesian networks for the emission probabilities. The model was tested with synthetic and real data incoming from air quality and ball-bearing data. For this model, several types of HMMs were used for comparison. The learning times were also considered for evaluation. The proposed model showed improvements in log-likelihood with fair learning times, and additional data insights were provided due to the learned Bayesian networks. As comment, AR-AsLG-HMM served as a cornerstone model for other contributions in this thesis. In Chapter 6, AR-AsLG-HMM was endowed with feature saliencies to enable the model to perform an embedded feature selection procedure. This implies that the model during its learning procedure determined the relevant features. This model is referred to FS-AsHMM. In this case, the model was compared to other HMMs with feature saliencies. Synthetic data and real data from ball-bearings and cameras with face- expressions data were used for validation. The model obtained better results regarding expression recognition and detection of non-relevant features. The previous contributions were focused to ofline analysis. Nevertheless, this thesis is focused on working in industrial environments, where data-streams are generated and the models are expected to adapt to changes in data. To address such issue, in Chapter 7 the AR-AsLG-HMM was adapted to be used in data-stream and perform continuous learning. Novel-concept detection techniques were used to determine when new unobserved patterns appeared. Based on the data-insights of the AR-AsLG-HMM from the data-stream, a health index and a regression model were proposed to determine the health status and remaining useful life of ball-bearings. Two datasets were used to validate the proposed methodology: open access datasets with ball-bearings which are run to failure, and a ball-bearing testbed from a company promoting the thesis, Aingura IIoT. Additionally, in collaboration with the Barcelona Supercomputer Center, the methodology code was optimized to be embedded into edge devices and use it in real life applications. The methodology was compared to others in the state of the art. It obtained better results in terms of health estimation, and fair results regarding the remaining useful life prediction. Next, in Chapter 8 a feature saliency model for HMMs was adapted to determine relevant harmonics of ball-bearings data in online environments. However, this study was a preliminary work for what was done in Chapter 9, where local feature saliencies were applied on AR-AsLG-HMMs. This model is referred to LFS-AsHMM. This model was adapted to be used in data-streams with novel-concept detection techniques to keep track of the evolution of relevant features. This model updated the relevant features only when the data needed it. Synthetic and real open access data from ball-bearings was used for validation. The model was compared to other strategies and methodologies that perform feature selection in data-streams. However, these strategies did the feature selection whenever a new instance arrived and not when needed. Unfortunately, this model did not get to be implemented into edge devices during the writing of this thesis. Finally, the proposed models assume linear Gaussian data and if such assumption fails, the models are no longer valid. To address such problem, in Chapter 10, the ideas used on AR-AsLG-HMM were imposed over HMMs with non-parametric emission probabilities, more precisely, kernel density estimations were used to approximate the emission probabilities, and the estimations depended on context-specific Bayesian networks. The proposed modelis referred to KDE-AsHMM. The proposed model is validated using synthetic non-linear Gaussian data and open access real data from sound recognition problems and drill milling processes. The model showed improvements in likelihood and sound recognition accuracy when compared to other HMMs. Nonetheless, the learning times and computational resources were high demanding. At the end of the thesis, in Chapter 11, the corresponding conclusions, final remarks and future research lines were proposed.