Overview of DETESTS at IberLEF 2022: DETEction and classification of racial STereotypes in Spanish

• Autores: Alejandra Ariza, Wolfgang S. Schmeisser Nieto, Montserrat Nofre, Mariona Taulé Delor , Enrique Amigó , Berta Chulvi, Paolo Rosso
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 69, 2022, págs. 217-228
• Idioma: inglés
• Títulos paralelos:
  • Resumen de la tarea de DETESTS en IberLEF 2022: DETEcción y clasificación de eSTereotipos raciales en eSpañol
• Enlaces
  • Texto completo
• Resumen
  • español

    Este artículo presenta un resumen de la tarea DETESTS como parte del workshop IberLEF 2022, dentro de la conferencia SEPLN 2022. Proponemos dos subtareas jerárquicas: En la subtarea 1, los participantes tuvieron que determinar la presencia de estereotipos raciales en oraciones. En la subtarea 2, de las oraciones etiquetadas con estereotipo, los participantes tuvieron que clasificarlas en una o más de diez categorías. El dataset DETESTS contiene 5.629 oraciones de comentarios que responden a artículos de periódicos sobre inmigración en español. 51 equipos se registraron para participar, de los cuales 39 enviaron predicciones de sistemas y 5 de ellos enviaron artículos. En este artículo presentamos información sobre los datasets de entrenamiento y de prueba, los sistemas utilizados por los participantes, las métricas de evaluación y sus resultados.

  • English

    This paper presents an overview of the DETESTS shared task as part of the IberLEF 2022 Workshop on Iberian Languages Evaluation Forum, within the framework of the SEPLN 2022 conference. We proposed two hierarchical subtasks: For subtask 1, participants had to determine the presence of stereotypes in sentences. For subtask 2, participants had to classify the sentences labeled with stereotypes into ten categoriesEste artículo presenta un resumen de la tarea DETESTS como parte del workshop IberLEF 2022, dentro de la conferencia SEPLN 2022. The DETESTS dataset contains 5,629 sentences in comments in response to newspaper articles related to immigration in Spanish. 51 teams signed up to participate, of which 39 sent runs, and 5 of them sent their working notes. In this paper, we provide information about the training and test datasets, the systems used by the participants, the evaluation metrics of the systems and their results

• Referencias bibliográficas
  • Allport, G. W., K. Clark, and T. Pettigrew. 1954. The nature of prejudice. Addison wesley Reading, M
  • Amigo, E. and A. D. Delgado. 2022. Evaluating extreme hierarchical multi label classification. In Proceedings of the 60th Annual Meeting of...
  • Amigo, E., F. Giner, J. Gonzalo, and F. Verdejo. 2020. On the foundations of similarity in information access. Inf. Retr. J., 23(3):216–254.
  • Basile, V., M. Fell, T. Fornaciari, D. Hovy, S. Paun, B. Plank, M. Poesio, and A. Uma. 2021. We need to consider dis agreement in evaluation....
  • Cabestany, D., C. Adsuar, and M. Lopez. 2022. DaMinCi at IberLEF2022 DE TESTS task: Detection and Classification of Racial Stereotypes in...
  • Chiril, P., F. Benamara, and V. Moriceau. 2021. “Be Nice to your wife! The Restaurants are Closed”: Can Gender Stereotype Detection Improve...
  • Cryan, J., S. Tang, X. Zhang, M. Metzger, H. Zheng, and B. Y. Zhao, 2020. Detecting Gender Stereotypes: Lexicon vs. Super vised Learning Methods,...
  • Fersini, E., P. Rosso, and M. E. Anzovino. 2018. Overview of the task on automatic misogyny identification at ibereval 2018. In IberEval@SEPLN.
  • Fokkens, A., N. Ruigrok, C. Beukeboom, S. Gagestein, and W. Van Atteveldt. 2019. Studying muslim stereotyping through microportrait extraction....
  • Garcıa-Dıaz, J. A., S. M. Jimenez-Zafra, and R. Valencia-Garcıa. 2022. UMUTeam at IberLEF2022 DETESTS task: Feature Engineering for the Identification...
  • Jain, H., Y. Prabhu, and M. Varma. 2016. Extreme multilabel loss functions for recommendation, tagging, ranking & other missing label...
  • Kiritchenko, S., S. Matwin, and F. Famili. 2004. Hierarchical text categorization as a tool of associating genes with gene ontology codes....
  • Laknani, F. and M. Garcıa-Martinez. 2022. Lak NLP at IberLEF2022 DETESTS task: Automatic Classification of Stereo types in Text. In Proceedings...
  • Ramirez Ortal, J., M. V. Sabando, M. Maisonnave1, and E. Milios. 2022. MALNIS at IberLEF2022 DETESTS Task: A MultiTask Learning Approach for...
  • Rodrıguez Sanchez, F., J. C. de Albornoz, L. Plaza, J. Gonzalo, P. Rosso, M. Comet, and T. Donoso. 2021. Overview of exist 2021: sexism identification...
  • Sanguinetti, M., G. Comandini, E. di Nuovo, S. Frenda, M. Stranisci, C. Bosco, T. Caselli, V. Patti, and I. Russo. 2020. Haspeede 2 @...
  • Sap, M., S. Gabriel, L. Qin, D. Jurafsky, N. A. Smith, and Y. Choi. 2020. Social bias frames: Reasoning about Social and power implications...
  • Schmeisser Nieto, W., M. Nofre, and M. Taule. 2022. Criteria for the annotation of implicit stereotypes. In Proceedings of the Language Resources...
  • Sanchez-Junquera, J., B. Chulvi, P. Rosso, and S. P. Ponzetto. 2021. How do you speak about immigrants? taxonomy and stereoimmigrants dataset...
  • Tajfel, H. 1984. Grupos humanos y categorias sociales. Herder.
  • Tajfel, H., A. A. Sheikh, and R. C. Gardner. 1964. Content of stereotypes and the inference of similarity between members of stereotyped groups....
  • Taule, M., A. Ariza, M. Nofre, E. Amigo, and P. Rosso. 2021. Overview of DETOXIS at IberLEF 2021: DEtection of TOXicity in comments In Spanish....
  • Uma, A., T. Fornaciari, A. Dumitrache, T. Miller, J. Chamberlain, B. Plank, E. Simpson, and M. Poesio. 2021. SemEval2021 Task 12: Learning...
  • Vazquez, J. M., V. P. Alvarez, C. T. Taybi, and P. P. Sanchez. 2022. I2C at IberLEF 2022 DETESTS task: Detection of Racist Stereotypes in...