Overview of FLARES at IberLEF 2024: Fine-grained Language-based Reliability Detection in Spanish News

• Autores: Robiert Sepúlveda Torres, Alba Bonet Jover, Isam Diab Lozano, Ibai Guillén Pacho, Isabel Cabrera de Castro, Carlos Badenes Olmedo, Estela Saquete Boró , María Teresa Martín Valdivia , Patricio Martínez Barco , Luis Alfonso Ureña López
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 73, 2024, págs. 369-379
• Idioma: inglés
• Títulos paralelos:
  • Resumen de FLARES en IberLEF 2024: Detección detallada de la confiabilidad en el lenguaje de noticias en español
• Enlaces
  • Texto completo
• Resumen
  • español

    Este articulo presenta FLARES, una tarea compartida organizada en el marco de la campaña de evaluación de sistemas de Procesamiento del Lenguaje Natural en español y otras lenguas ibéricas, IberLEF 2024. FLARES tiene como objetivo detectar patrones de confiabilidad en el lenguaje utilizado en las noticias que permita desarrollar técnicas eficaces para la futura detección de información engañosa. Para ello, se propone como base la técnica periodística de las 5W1H para detectar el contenido relevante de una noticia, así como una guía de anotación diseñada para detectar la confiabilidad lingüística. Se proponen dos subtareas: la primera centrada en la identificación de los elementos 5W1H y la segunda en la detección de la confiabilidad. Un total de 7 participantes se registraron en la tarea compartida, de los cuales 3 participaron en la primera subtarea y 4 en la segunda. Los equipos propusieron diversos enfoques, especialmente basado en el ajuste de modelos de codificación y en el ajuste de instrucciones en modelos de decodificación.

  • English

    This paper presents FLARES, a shared task organised in the framework of the evaluation campaign of Natural Language Processing systems in Spanish and other Iberian languages, IberLEF 2024. FLARES aims to detect patterns of reliability in the language used in news that will allow the development of effective techniques for the future detection of misleading information. To this end, the 5W1H journalistic technique for detecting the relevant content of a news item is proposed as a basis, as well as an annotation guideline designed to detect linguistic reliability. Two subtasks are proposed: the first focusing on the identification of the 5W1H elements and the second focusing on the detection of reliability. A total of 7 participants registered in the shared task, of which 3 participated in the first subtask and 4 in the second. The teams proposed various approaches, especially based on fine-tuning of encoding models and adjustment of instructions in decoding models.

• Referencias bibliográficas
  • Abas, A. R., I. El-Henawy, H. Mohamed, and A. Abdellatif. 2020. Deep learning model for fine-grained aspect-based opinion mining. IEEE Access,...
  • AI@Meta. 2024. Llama 3 model card. Bani-Hani, A., O. Adedugbe, E. Benkhelifa, M. Majdalawieh, and F. Al-Obeidat. 2020. A semantic model...
  • Bonet-Jover, A., R. Sepúlveda-Torres, E. Saquete, P. Martínez-Barco, and M. Nieto-Pérez. 2024. Run-as: A novel approach to annotate news reliability...
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Pérez. 2023. Spanish pretrained bert model and evaluation data. arXiv preprint...
  • Chakma, K. and A. Das. 2018. A 5w1h based annotation scheme for semantic role labelling of english tweets. Computación y Sistemas, 22(3):747–755.
  • Chakma, K., S. D. Swamy, A. Das, and S. Debbarma. 2020. 5w1h-based semantic segmentation of tweets for event detection using bert. In International...
  • Chung, H. W., L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. Dehghani, S. Brahma, A. Webson, S. S. Gu, Z. Dai, M. Suzgun,...
  • Grande, E. and A. Begga. 2024. Syntax Savants-UA at IberLEF 2024: Leveraging FLAN-T5-XXL for Automatic 5W1H Identification in Texts. In In...
  • Grieve, J. and H. Woodfield. 2023. The language of fake news. Cambridge University Press.
  • Gutiérrez-Fandiño, A., J. Armengol-Estapé, M. Pàmies, J. Llop-Palao, J. Silveira-Ocampo, C. P. Carrino, A. Gonzalez-Agirre, C. Armentano-Oller,...
  • Horne, B. and S. Adali. 2017. This just in: Fake news packs a lot in title, uses simpler, repetitive content in text body, more similar to...
  • Ibrahim, M. 2024. Fine-Grained Language-based Reliability Detection in Spanish New with Fine-Tuned Llama-3 Model. In In Proceedings of the...
  • Keith, B., M. Horning, and T. Mitra. 2020. Evaluating the inverted pyramid structure through automatic 5w1h extraction and summarization....
  • Khodra, M. L. 2015. Event extraction on Indonesian news article using multiclass categorization. In 2015 2nd International Conference on Advanced...
  • Lugea, J. 2021. Linguistic approaches to fake news detection. Data science for fake news: Surveys and perspectives, pages 287–302.
  • Mangrulkar, S., S. Gugger, L. Debut, Y. Belkada, S. Paul, and B. Bossan. 2022. Peft: State-of-the-art parameter-efficient fine-tuning methods....
  • Pan, R., J. A. García-Díaz, F. García-Sánchez, and R. Valencia-García. 2024. UMUTeam at FLARES@IberLEF 2024: Enhancing Disinformation...
  • Pardo, J., J. Liu, V. Ramón-Ferrer, E. Amador-Domínguez, and P. Calleja. 2024. K-Flares: A K-Adapter Based Approach for the FLARES Challenge....
  • Piad-Morffis, A., Y. Gutiérrez, Y. Almeida-Cruz, and R. Muñoz. 2020. A computational ecosystem to support ehealth knowledge discovery technologies...
  • Saquete, E., D. Tomás, P. Moreda, P. Martínez-Barco, and M. Palomar. 2020. Fighting post-truth using natural language processing: A review...
  • Seddari, N., A. Derhab, M. Belaoued, W. Halboob, J. Al-Muhtadi, and A. Bouras. 2022. A hybrid linguistic and knowledge-based analysis approach...
  • Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin. 2017. Attention is all you need. In...
  • Vlachos, A. and S. Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 workshop on language...
  • Vosoughi, S., D. Roy, and S. Aral. 2018. The spread of true and false news online. science, 359(6380):1146–1151.
  • Wang, R., D. Tang, N. Duan, Z. Wei, X. Huang, G. Cao, D. Jiang, M. Zhou, et al. 2020. K-adapter: Infusing knowledge into pretrained models...
  • Zhang, H., X. Chen, and S. Ma. 2019. Dynamic news recommendation with hierarchical attention network. In 2019 IEEE International Conference...
  • Zhao, S., F. You, and Z. Y. Liu. 2020. Leveraging pre-trained language model for summary generation on short text. IEEE Access, 8:228798–228803.
  • Zhou, L. and D. Zhang. 2008. Following linguistic footprints: Automatic deception detection in online communication. Communications of the...
  • Zhou, X. and R. Zafarani. 2020. A survey of fake news: Fundamental theories, detection methods, and opportunities. ACM Computing Surveys (CSUR),...