OBOE: an Explainable Text Classification Framework

• Raúl A. del Águila Escobar ^[1] ; Mari Carmen Suárez-Figueroa ^[1] ; Mariano Fernández-López ^[2]
  1. [1] Universidad Politécnica de Madrid

     Universidad Politécnica de Madrid

     Madrid, España

     
  2. [2] Universidad CEU-San Pablo
• Localización: IJIMAI, ISSN-e 1989-1660, Vol. 8, Nº. 6, 2024, págs. 24-37
• Idioma: inglés
• DOI: 10.9781/ijimai.2022.11.001
• Enlaces
  • Texto completo
• Resumen

  • Explainable Artificial Intelligence (XAI) has recently gained visibility as one of the main topics of Artificial Intelligence research due to, among others, the need to provide a meaningful justification of the reasons behind the decision of black-box algorithms. Current approaches are based on model agnostic or ad-hoc solutions and, although there are frameworks that define workflows to generate meaningful explanations, a text classification framework that provides such explanations considering the different ingredients involved in the classification process (data, model, explanations, and users) is still missing. With the intention of covering this research gap, in this paper we present a text classification framework called OBOE (explanatiOns Based On concEpts), in which such ingredients play an active role to open the black-box. OBOE defines different components whose implementation can be customized and, thus, explanations are adapted to specific contexts. We also provide a tailored implementation to show the customization capability of OBOE. Additionally, we performed (a) a validation of the implemented framework to evaluate the performance using different corpora and (b) a user-based evaluation of the explanations provided by OBOE. The latter evaluation shows that the explanations generated in natural language express the reason for the classification results in a way that is comprehensible to non-technical users.

• Referencias bibliográficas
  • F. Lecue, “On The Role of Knowledge Graphs in Explainable AI | www.semantic-web-journal.net,” Semantic Web Journal, p. 9, 2018.
  • F. Hohman, M. Kahng, R. Pienta, and D. H. Chau, “Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers,” arXiv:1801.06889...
  • L. M. Brasil, F. M. de Azevedo, and R. Moraes, “FUZZYRULEXT: extraction technique of if/then rules for fuzzy neural nets,” in Proceedings of...
  • G. Montavon, S. Lapuschkin, A. Binder, W. Samek, and K.-R. Müller, “Explaining nonlinear classification decisions with deep Taylor decomposition,”...
  • H. Zhang, S. Nakadai, and K. Fukumizu, “From Black-Box to White-Box: Interpretable Learning with Kernel Machines,” in Machine Learning and Data...
  • K. Cho et al., “Learning Phrase Representations using RNN EncoderDecoder for Statistical Machine Translation,” arXiv:1406.1078 [cs, stat], Sep....
  • S. Hochreiter and J. Schmidhuber, “Long Short-term Memory,” Neural computation, vol. 9, pp. 1735–80, Dec. 1997, doi: 10.1162/neco.1997.9.8.1735.
  • Y. Ling, W. Guan, Q. Ruan, H. Song, and Y. Lai Y. “Variational Learning for the Inverted Beta-Liouville Mixture Model and Its Application...
  • A. B. Arrieta et al., “Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI,”...
  • A. Adadi and M. Berrada, “Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI),” IEEE Access, vol. 6, pp. 52138– 52160,...
  • D. Gunning, “Explainable Artificial Intelligence (XAI),” Tech. rep. Defense Advanced Research Projects Agency (DARPA), 2017.
  • Z. C. Lipton, “The Mythos of Model Interpretability,” arXiv:1606.03490 [cs, stat], Mar. 2017, Accessed: Feb. 11, 2020. [Online]. Available:...
  • L. S. Prasanthi and R. K. Kumar, “ID3 and Its Applications in Generation of Decision Trees across Various Domains- Survey”. In (IJCSIT) International...
  • L. Rosenbaum, G. Hinselmann, A. Jahn, and A. Zell, “Interpreting linear support vector machine models with heat map molecule coloring,” Journal of...
  • M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why Should I Trust You?’: Explaining the Predictions of Any Classifier,” in Proceedings of the...
  • S. Lundberg and S.-I. Lee, “A Unified Approach to Interpreting Model Predictions,” presented at the NIPS, Dec. 2017. [Online]. Available:...
  • P. Domingos, “Knowledge discovery via multiple models,” Intelligent Data Analysis, vol. 2, no. 1, pp. 187–202, Jan. 1998, doi: 10.1016/S1088- 467X(98)00023-7.
  • R. Blanco-Vega, J. Hernández-Orallo, and M. J. Ramírez-Quintana, “El Método Mimtico, una Alternativa para la Comprensibilidad de Modelos de...
  • V. Estruch, C. Ferri, J. Hernández-Orallo, and M. J. Ramírez-Quintana, “Simple Mimetic Classifiers,” in Machine Learning and Data Mining in Pattern...
  • C. J. Mahoney, J. Zhang, N. Huber-Fliflet, P. Gronvall, and H. Zhao, “A Framework for Explainable Text Classification in Legal Document Review,”...
  • T. Spinner, U. Schlegel, H. Schäfer, and M. El-Assady, “explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning,” IEEE...
  • F. T. Liu, K. M. Ting, and Z.-H. Zhou, “Isolation Forest,” in 2008 Eighth IEEE International Conference on Data Mining, Pisa, Italy, Dec....
  • S. Wang, W. Zhou, and C. Jiang, “A survey of word embeddings based on deep learning,” Computing, vol. 102, no. 3, pp. 717–740, Mar. 2020,...
  • C. Raffel et al., “Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer,” arXiv:1910.10683 [cs, stat], Jul. 2020,...
  • K. Jaskie and A. Spanias, “Positive And Unlabeled Learning Algorithms And Applications: A Survey,” in 2019 10th International Conference on Information,...
  • J. Bekker and J. Davis, “Learning From Positive and Unlabeled Data: A Survey,” arXiv:1811.04820 [cs, stat], Nov. 2018, Accessed: Feb. 17,...
  • L. M. de Campos, J. M. Fernández-Luna, J. F. Huete, and L. RedondoExpsito, “Positive unlabeled learning for building recommender systems in...
  • D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,” Journal of Machine Learning Research, vol. 3, pp. 993–1022, Mar. 2003.
  • R. Alghamdi and K. Alfalqi, “A Survey of Topic Modeling in Text Mining,” International Journal of Advanced Computer Science and Applications...
  • J. Sahoo, “Modified TF-IDF Term Weighting Strategies for Text Categorization,” Oct. 2018. doi: 10.1109/INDICON.2017.8487593.
  • P. Geurts, D. Ernst, and L. Wehenkel, “Extremely randomized trees,” Machine Learning, vol. 63, no. 1, pp. 3–42, Apr. 2006, doi: 10.1007/s10994-006-6226-1.
  • T. Chen and C. Guestrin, “XGBoost: A Scalable Tree Boosting System,” Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge...
  • W.Cohen, “Fast Efective Rule Induction”. In Proceedings of the Twelfth International Conference on Machine Learning (ICML’95). pp. 115-123. Morgan...
  • J. McAuley, C. Targett, Q. Shi, and A. van den Hengel, “Image-based Recommendations on Styles and Substitutes,” arXiv:1506.04757 [cs], Jun....
  • C. E. Blacke, E. Keogh, and C. J. Merz, “UCI repository of machine learning databases.” University of California, School of Information and...
  • K. Lang, “Newsweeder: Learning to filter netnews,” in Proceedings of the Twelfth International Conference on Machine Learning, 1995, pp. 331–339
  • J. Mockus, V. Tiesis, and A. Zilinskas, “The Application of Bayesian Methods for Seeking theExtremum,” in Toward Global Optimization, vol....
  • V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Computing Surveys, vol. 41, no. 3, pp. 1–58, Jul. 2009, doi: 10.1145/1541880.1541882.
  • W. L. Taylor, “‘Cloze procedure’: A new tool for measuring readability,” Journalism quarterly, vol. 30, no. 4, pp. 415–433, 1953.
  • F. Doshi-Velez and B. Kim, “A Roadmap for a Rigorous Science of Interpretability,” ArXiv, vol. abs/1702.08608, 2017.