Overview of EXIST 2021:: sEXism Identification in Social neTworks
-
Autores: Francisco J. Rodríguez Sánchez
, Jorge Carrillo de Albornoz, Laura Plaza Morales , Julio Gonzalo Arroyo , Paolo Rosso , Miriam Comet, Trinidad Donoso Vázquez
- Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 67, 2021, págs. 195-207
- Idioma: inglés
- Títulos paralelos:
- Overview de EXIST 2021:: Identificación de Sexismo en Redes Sociales
- Enlaces
- Dialnet Métricas: 3 Citas
-
Resumen
- español
El presente artículo describe la organización, objetivos y resultados de la competición sEXism Identification in Social neTworks (EXIST), una tarea propuesta por primera vez en IberLEF 2021. EXIST 2021 propone dos tareas: la identificación y la categorización de sexismo en inglés y español. Se han recibido un total de 70 runs para la tarea de identificación de sexismo y 61 para la categorización de sexismo, enviadas por 31 equipos de 11 países. En este trabajo, se presentan el dataset, la metodología de evaluación, un análisis de los sistemas propuestos por los participantes y los resultados obtenidos. El dataset final está compuesto por más de 11,000 textos anotados procedentes de dos redes sociales (Twitter y Gab) y su elaboración ha sido supervisada por expertas en temas de género.
- English
The paper describes the organization, goals, and results of the sEXism Identification in Social neTworks (EXIST) challenge, a shared task proposed for the first time at IberLEF 2021. EXIST 2021 proposes two challenges: sexism identification and sexism categorization of tweets and gabs, both in Spanish and English. We have received a total of 70 runs for the sexism identification task and 61 for the sexism categorization challenge, submitted by 31 different teams from 11 countries. We present the dataset, the evaluation methodology, an overview of the proposed systems, and the results obtained. The final dataset consists of more than 11,000 annotated texts from two social networks (Twitter and Gab) and its development has been supervised and monitored by experts in gender issues.
- español
-
Referencias bibliográficas
- Amigó, E., J. Carrillo-de Albornoz, M. Almagro-Cádiz, J. Gonzalo, J. Rodríguez-Vidal, and F. Verdejo. 2017. Evall: Open access evaluation for...
- Amigó, E., J. Gonzalo, S. Mizzaro, and J. Carrillo-de Albornoz. 2020. An effectiveness metric for ordinal classification: Formal properties...
- Amigó, E., D. Spina, and J. Carrillo-de Albornoz. 2018. An axiomatic analysis of diversity evaluation metrics: Introducing the rank-biased...
- Barbieri, F., L. E. Anke, and J. CamachoCollados. 2021. Xlm-t: A multilingual language model toolkit for twitter. arXiv preprint arXiv:2104.12250.
- Barbieri, F., J. Camacho-Collados, L. Espinosa Anke, and L. Neves. 2020. TweetEval: Unified benchmark and comparative evaluation for tweet...
- Basile, V., C. Bosco, E. Fersini, N. Debora, V. Patti, F. M. R. Pardo, P. Rosso, M. Sanguinetti, et al. 2019. Semeval2019 task 5: Multilingual...
- Bassignana, E., V. Basile, and V. Patti. 2018. Hurtlex: A multilingual lexicon of words to hurt. In 5th Italian Conference on Computational...
- Baumgartner, J., S. Zannettou, B. Keegan, M. Squire, and J. Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the international...
- Berg, S. H. 2006. Everyday sexism and posttraumatic stress disorder in women: A correlational study. Violence Against Women, 12(10):970–988. Canete,...
- Chiril, P., V. Moriceau, F. Benamara, A. Mari, G. Origgi, and M. CoulombGully. 2020. He said “who’s gonna take care of your children when...
- Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzmán, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2019. Unsupervised...
- Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding....
- Donoso-Vázquez, T. and Rebollo-Catalán. 2018. Violencias de género en entornos virtuales. Ediciones Octaedro.
- Fast, E., B. Chen, and M. S. Bernstein. 2016. Empath: Understanding topic signals in large-scale text. In Proceedings of the 2016 CHI conference...
- Fersini, E., P. Rosso, and M. Anzovino. 2018. Overview of the task on automatic misogyny identification at ibereval 2018. IberEval@ SEPLN,...
- Fox, J., C. Cruz, and J. Y. Lee. 2015. Perpetuating online sexism offline: Anonymity, interactivity, and the effects of sexist hashtags on...
- Frenda, S., B. Ghanem, M. Montes-y Gómez, and P. Rosso. 2019. Online hate speech against women: Automatic identification of misogyny and sexism...
- He, P., X. Liu, J. Gao, and W. Chen. 2020. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654.
- Joulin, A., E. Grave, P. Bojanowski, and T. Mikolov. 2017. Bag of tricks for efficient text classification. In Proceedings of the 15th Conference...
- Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. Roberta: A robustly optimized...
- Martínez-Cámara, E., M. Díaz-Galiano, M. García-Cumbreras, M. García-Vega, and J. Villena-Román. 2017. Overview of tass 2017. IberEval@...
- Mills, S. 2008. Language and sexism. Cambridge University Press.
- Montes, M., P. Rosso, J. Gonzalo, E. Aragón, R. Agerri, M. Angel Alvarez Carmona, E. Alvarez Mellado, J. C. de Albornoz, L. Chiruzzo, L. Freitas,...
- Parikh, P., H. Abburi, P. Badjatiya, R. Krishnan, N. Chhaya, M. Gupta, and V. Varma. 2019. Multi-label categorization of accounts of sexism...
- Rodríguez-Sánchez, F., J. Carrillo-de Albornoz, and L. Plaza. 2020. Automatic classification of sexism in social networks: An empirical study...
- Swim, J., L. Hyers, L. Cohen, and M. Ferguson. 2001. Everyday sexism: Evidence for its incidence, nature, and psychological impact from three...
- Waseem, Z. 2016. Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter. In Proceedings of the First...
- Waseem, Z. and D. Hovy. 2016. Hateful symbols or hateful people? predictive features for hate speech detection on Twitter. In Proceedings...
