Overview of HOPE at IberLEF 2024: Approaching Hope Speech Detection in Social Media from Two Perspectives, for Equality, Diversity and Inclusion and as Expectations

• Autores: Daniel García Baena, Fazlourrahman Balouchzahi, Sabur Butt, Miguel Ángel García Cumbreras , Atnafu Lambebo Tonja, José Antonio García Díaz, Selen Bozkurt, Bharathi Raja Chakravarthi, Hector G. Ceballos, Rafael Valencia García , Grigori Sidorov, Luis Alfonso Ureña López , Alexander F. Gelbukh, Salud M. Jiménez Zafra
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 73, 2024, págs. 407-419
• Idioma: inglés
• Títulos paralelos:
  • Resumen de la tarea HOPE en IberLEF 2024: Abordando la Detección del Discurso de Esperanza en los Medios Sociales desde Dos Perspectivas, para la Igualdad, la Diversidad y la Inclusión y como Expectativas
• Enlaces
  • Texto completo
• Resumen
  • español

    Este articulo presenta la segunda edición de la tarea compartida internacional sobre detección multilingüe del discurso de la esperanza, HOPE 2024, desarrollada como parte del taller IberLEF durante el congreso SEPLN 2024. Esta tarea estuvo compuesta por dos subtareas distintas: la primera de ellas se centró en la detección del discurso esperanzador en el ámbito de la Igualdad, Diversidad e Inclusión, mientras que la segunda se enfocó en las expectativas del discurso esperanzador atendiendo a dos niveles, binario y multiclase. Diecinueve equipos participaron en la competición y dieciséis enviaron sus notas de trabajo. En la primera subtarea, el equipo mejor clasificado logró una puntuación Macro F1 media de 71,61. En la segunda subtarea, los equipos líderes demostraron un rendimiento sólido con puntuaciones F1 superiores a 80,00 para la clasificación binaria y 78,00 para los entornos de clasificación multiclase.

  • English

    This paper presents the second edition of the international shared task on multilingual hope speech detection, HOPE 2024, conducted as part of the IberLEF workshop during the SEPLN 2024 conference. This shared task encompassed two distinct subtasks: the detection of hope speech within Equality, Diversity, and Inclusion texts and the identification of hope speech focusing on expectations in two-level, binary, and multiclass classification settings. Nineteen teams participated in the competition, and sixteen submitted their working notes. In the first subtask, the top-ranking team achieved an average Macro F1-score of 71.61. In the second subtask, leading teams demonstrated robust performance with F1 scores exceeding 80.00 for binary classification and 78.00 for multiclass classification settings.

• Referencias bibliográficas
  • Ahani, Z., M. S. Tash, M. Tash, A. Gelbukh, and I. Gelbukh. 2024. Multiclass hope speech detection through transformer methods. In Proceedings...
  • Armenta-Segura, J. and G. Sidorov. 2024. Ometeotl at hope2024@iberlef: Custom bert models for hope speech detection. In Proceedings of...
  • ArunaDevi, S. and B. Bharathi. 2024. Machine learning based approach for hope speech detection. In Proceedings of the Iberian Languages Evaluation...
  • Bade, G. Y., O. Koleniskova, J. L. Oropeza, G. Sidorov, and K. F. Bergene. 2024. Hope speech in social media texts using transformer. In Proceedings...
  • Balouchzahi, F., G. Sidorov, and A. Gelbukh. 2023. Polyhope: Two-level hope speech detection from tweets. Expert Systems with Applications,...
  • Bruininks, P. and B. F. Malle. 2005. Distinguishing hope from optimism and related affective states. Motivation and Emotion, 29:324–352.
  • Bui Hong, S., Q. Le Minh, and T. Dang Van. 2024. ABCD Team at HOPE 2024: Hope Detection with BERTology Models and Data Augmentation. In Proceedings...
  • Burnap, P., G. Colombo, R. Amery, A. Hodorog, and J. Scourfield. 2017. Multi-class machine classification of suicide-related communication...
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Pérez. 2023. Spanish pre-trained bert model and evaluation data.
  • Chakravarthi, B. R. 2020. HopeEDI: A multilingual hope speech detection dataset for equality, diversity, and inclusion. In Proceedings of...
  • Chakravarthi, B. R., V. Muralidaran, R. Priyadharshini, S. Cn, J. Mc-Crae, M. A. García, S. M. Jiménez-Zafra, R. Valencia-García, P. Kumaresan,...
  • Chau Pham Quoc, H. and T. Dang Van. 2024. Choosing the Right Language Model for the Right Task. In Proceedings of the Iberian Languages Evaluation...
  • Chiruzzo, L., S. M. Jiménez-Zafra, and F. Rangel. 2024. Overview of IberLEF 2024: Natural Language Processing Challenges for Spanish and other...
  • Divakaran, S., K. Girish, and H. L. Shashirekha. 2024. Hope on the horizon: Experiments with learning models for hope speech detection in...
  • Eid, M. and R. Larsen. 2008. The Science of Subjective Well-Being. Guilford Publications.
  • Ibrahim, M. 2024. Parameter-free Spanish hope speech detection. In Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2024), co-located...
  • Jiménez-Zafra, S. M., M. ´A. García, D. García-Baena, J. García-Díaz, B. R. Chakravarthi, R. Valencia-García, and L. A. Ureña-López. 2023....
  • Kitzie, V. 2018. ”i pretended to be a boy on the internet”: Navigating affordances and constraints of social networking sites and search engines...
  • Krasitskii, M., O. Kolesnikova, L. C. Hernandez, G. Sidorov, and A. Gelbukh. 2023. Hope2023@iberlef: A cross-linguistic exploration of...
  • Kumaresan, P. K., B. R. Chakravarthi, S. Cn, M. ´A. García-Cumbreras, S. M. Jiménez Zafra, J. A. García-Díaz, R. Valencia-García, M. Hardalov,...
  • Lemlem, E., Y. Tsadkan, N. Amna, S. Grigori, and B. Ildar. 2024. Enhancing hope speech detection on twitter using machine learning and transformer...
  • Milne, D., G. Pink, B. Hachey, and R. Calvo. 2016. Clpsych 2016 shared task: Triaging content in online peer-support forums. pages 118–127,...
  • Naseeb, A., K. L. Eyob, G. Sidorov, and O. Kolesnikova. 2024. Hope@iberlef 2024: Beyond binary bounds—classifying hope in online discourse....
  • Ngo, A. and H. T. H. Tran. 2023. Zootopi at hope2023iberlef: Is zero-shot chatgpt the future of hope speech detection? In Proceedings of the...
  • Nguyen Thi, T. and T. Dang Van. 2024. An Empirical Study of Prompt Engineering with Large Language Models for Hope Detection in English and...
  • Palakodety, S., A. R. KhudaBukhsh, and J. G. Carbonell. 2019. Hope speech detection: A computational analysis of the voice of peace. arXiv...
  • Pan, R., Ángela Almela, and G. Alcaraz-Mármol. 2024. UMUTeam at hope@iberlef 2024: Fine-tuning approach with sentiment and emotion features...
  • Pérez, J. M., D. A. Furman, L. Alonso Alemany, and F. M. Luque. 2022. RoBERTuito: a pre-trained language model for social media text in Spanish....
  • Pérez, J. M., M. Rajngewerc, J. C. Giudici, D. A. Furman, F. Luque, L. A. Alemany, and M. Vanina Mart´ınez. 2023. pysentimiento: A python...
  • Ponnusamy, K. K., M. Vegupatti, P. K. Kumaresan, R. Priyadharshini, P. Buitelaar, and B. R. Chakravarthi. 2024. VEL@iberlef 2024: Hope...
  • Snyder, C., B. Hoza, W. Pelham, M. Rapoff, L. Ware, M. Danovsky, L. Highberger, H. Ribinstein, and K. Stahl. 1997. The development and validation...
  • Snyder, C. 2002. Hope theory: Rainbows in the mind. Psychological Inquiry, 13:249–275, 10.
  • Snyder, C. 1994. The Psychology of Hope: You Can Get There from Here. Free Press.
  • Subburaj, A., A. Kathiresan, R. Ponnusamy, P. Butilaar, and B. R. Chakravarthi. 2024. UniOfGalway@iberlef 2024: Hope speech recognition...
  • Ullah, F., M. T. Zamir, M. Ahmad, G. Sidorov, and A. Gelbukh. 2024. Hope: A multilingual approach to identifying positive communication in...