Overview of DIPROMATS 2023: automatic detection and characterization of propaganda techniques in messages from diplomats and authorities of world powers

• Autores: Jorge Carrillo de Albornoz, Iván Gonzalo Verdugo, Pablo Moral Martín, Guillermo Marco Remón, Julio Gonzalo Arroyo
• Localización: Procesamiento del lenguaje natural, ISSN 1135-5948, Nº. 71, 2023, págs. 397-407
• Idioma: varios idiomas
• Títulos paralelos:
  • Overview de DIPROMATS 2023: detección y caracterización automáticas de técnicas de propaganda en mensajes de diplomáticos y autoridades de potencias mundiales
• Enlaces
  • Texto completo
• Resumen
  • Multiple

    Este artículo presenta los resultados de DIPROMATS 2023, una tarea compartida incluida en el Iberian Languages Evaluation Forum (IberLEF). DIPROMATS 2023 proporciona un conjunto de datos con 12.012 tweets anotados en inglés y 9.501 tweets en español, publicados por autoridades de China, Rusia, Estados Unidos y la Unión Europea. Se proponen tres tareas para cada idioma. La primera tiene como objetivo distinguir si un tweet tiene técnicas de propaganda o no. La segunda tarea busca clasificar el tweet en cuatro grupos de técnicas de propaganda, mientras que la tercera ofrece una categorización detallada de 15 técnicas. Para las tres tareas, hemos recibido un total de 34 ejecuciones de 9 equipos diferentes.

  • English

    This paper presents the results of the DIPROMATS 2023 challenge, a shared task included at the Iberian Languages Evaluation Forum (IberLEF). DIPROMATS 2023 provides a dataset with 12012 annotated tweets in English and 9501 tweets in Spanish, posted by authorities of China, Russia, United States and the European Union. Three tasks are proposed for each language. The first one aims to distinguish if a tweet has propaganda techniques or not. The second task seeks to classify the tweet into four clusters of propaganda techniques, whereas the third one offers a fine-grained categorization of 15 techniques. For the three tasks we have received a total of 34 runs from 9 different teams.

• Referencias bibliográficas
  • Amigo, E. and A. Delgado. 2022. Evaluating extreme hierarchical multi-label classification. In Proceedings of the 60th Annual Meeting of the...
  • Cañete, J., G. Chaperon, R. Fuentes, J.-H. Ho, H. Kang, and J. Pérez. 2020. Spanish pre-trained bert model and evaluation data. Pml4dc at...
  • Conneau, A., K. Khandelwal, N. Goyal, V. Chaudhary, G. Wenzek, F. Guzman, E. Grave, M. Ott, L. Zettlemoyer, and V. Stoyanov. 2020. Unsupervised...
  • Da San Martino, G., A. Barrón-Cedeño, H. Wachsmuth, R. Petrov, and P. Nakov. 2020. SemEval-2020 Task 11: Detection of Propaganda Techniques...
  • Da San Martino, G., P. Nakov, J. Pirskoski, and N. Stefanovitch. 2022. SemEval2023 shared task on ”Detecting the Genre, the Framing, and the...
  • Da San Martino, G., S. Yu, A. Barrón Cedeño, R. Petrov, and P. Nakov. 2019. Fine-Grained Analysis of Propaganda in News Article. In Proceedings...
  • De la Rosa, J., E. Ponferrada, M. Romero, P. Villegas, P. González de Prado Salas, and M. Grandury. 2022. BERTIN: Efficient Pre-Training of...
  • Devlin, J., M.-W. Chang, K. Lee, and K. Toutanova. 2018. BERT: Pretraining of Deep Bidirectional Transformers for Language Understanding....
  • Hobbs, R. and S. McGee. 2014. Teaching about Propaganda: An Examination of the Historical Roots of Media Literacy. Journal of Media Literacy...
  • Institute for Propaganda Analysis. 1938. Propaganda Analysis: Volume I of the Publications of the Institute for Propaganda Analysis. Institute...
  • Johnson-Cartee, K. S. and G. Copeland. 2004. Strategic political communication: rethinking social influence, persuasion, and propaganda. Communication,...
  • Jowett, G. and V. O’Donnell. 2015. Propaganda & persuasion. SAGE, Thousand Oaks, Calif, sixth edition edition.
  • Liu, Y., M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT...
  • Loureiro, D., F. Barbieri, L. Neves, L. Espinosa Anke, and J. Camacho-collados. 2022. TimeLMs: Diachronic Language Models from Twitter. In...
  • Nguyen, D. Q., T. Vu, and A. Tuan Nguyen. 2020. BERTweet: A pre-trained language model for English Tweets. In Proceedings of the 2020 Conference...
  • Piskorski, J., Stefanovitch, N., Da San Martino, G., and Nakov, P. Semeval-2023 task 3: Detecting the category, the framing, and the persuasion...
  • Pérez, J. M., D. A. Furman, L. Alonso Alemany, and F. M. Luque. 2022. RoBERTuito: a pre-trained language model for social media text in Spanish....
  • Sparkes-Vian, C. 2019. Digital Propaganda: The Tyranny of Ignorance. Critical Sociology, 45(3):393–409, May.
  • Spinde, T., L. Rudnitckaia, K. Sinha, F. Hamborg, B. Gipp, and K. Donnay. 2021a. MBIC – A Media Bias Annotation Dataset Including Annotator...
  • Spinde, T., M. Plank, J.-D. Krieger, T. Ruas, B. Gipp, and A. Aizawa. 2021b. Neu ral Media Bias Detection Using Distant Supervision With BABE...
  • Teninbaum, G. H. 2009. Reductio ad Hitlerum: Trumping the judicial Nazi card. Mich. St. L. Rev., page 541. Publisher: HeinOnline.
  • Weston, A. 2017. A rulebook for arguments. Hackett Publishing Company, Inc, Indianapolis ; Cambridge, fifth edition edition.
  • Wolf, T., L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von...