Does corpus quality really matter for low-resource languages?

• Mikel Artetxe ; Itziar Aldabe ^[1] ; Rodrigo Agerri ^[1] ; Olatz Perez-de-Viñaspre ^[1] ; Aitor Soroa ^[1]
  1. [1] Universidad del País Vasco/Euskal Herriko Unibertsitatea

     Universidad del País Vasco/Euskal Herriko Unibertsitatea

     Leioa, España

     
• Localización: InpAkta: Inpaktu sozialerako ikerlanen akta - Journal of Research for Social Impact, ISSN 2990-2479, ISSN-e 3020-8769, Nº. 0, 2023, págs. 298-307
• Idioma: inglés
• DOI: 10.18653/v1/2022.emnlp-main.499
• Enlaces
  • Texto Completo Ejemplar
• Dialnet Métricas: 3 Citas
• Resumen

  • The vast majority of non-English corpora are derived from automatically filtered versions of CommonCrawl. While prior work has identified major issues on the quality of these datasets (Kreutzer et al., 2021), it is not clear how this impacts downstream performance. Taking representation learning in Basque as a case study, we explore tailored crawling —manually identifying and scraping websites with high-quality content— as an alternative to filtering CommonCrawl. Our new corpus, called EusCrawl, is similar in size to the Basque portion of popular multilingual corpora like CC100 and mC4, yet it has a much higher quality according to native annotators. For instance, 66% of documents are rated as high-quality for EusCrawl, in contrast with < 33% for both mC4 and CC100. Nevertheless, we obtain similar results on down-stream NLU tasks regardless of the corpus used for pre-training. Our work suggests that NLU performance in low-resource languages is not primarily constrained by the quality of the data, and other factors like corpus size and domain coverage can play a more important role.

• Referencias bibliográficas
  • “Coronavirus disease (covid-19) pandemic,” https://www. who.int/emergencies/diseases/novel-coronavirus-2019, accessed: 2021-05-10.
  • [2] E. Dong, H. Du, and L. Gardner, “An interactive web-based dashboard to track covid-19 in real time,” The Lancet Infectious Diseases, vol....
  • [3] C. Cheng, J. Barceló, A. Hartnett, R. Kubinec, and L. Messerschmidt, “Covid-19 government response event dataset (coronanet v.1.0),”Nature...
  • [4] N. M. F. et al., “Report 9: Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand,”Imperial...
  • [5] D. L.Chao, M. E.Halloran,V.J.Obenchain, and I. M. Longini,Jr, “FluTE,apublicly available stochastic influenza epidemic simulation model,”...
  • [6] J. K. et al., “Interventions to mitigate early spread of SARSCoV-2 in Singapore: a modelling study,” The Lancet Infectious Diseases, vol....
  • [7] T. Hale, N. Angrist, E. Cameron-Blake, L. Hallas, B. Kira, S. Majumdar, A. Petherick, T. Phillips, H. Tatlow, and S. Webster, “Variation...
  • [8] J. A. Lewnard and N. C. Lo, “Scientific and ethical basis for social-distancing interventions against covid-19,” The Lancet Infectious...
  • [9] N. Fernandes,“Economic effects of coronavirus outbreak (covid-19) on the world economy,” IESE Business School Working Paper, no. WP-1240-E,...
  • [10] S. Maharaj and A. Kleczkowski, “Controlling epidemic spread by social distancing: Do it well or not at all,” BMC Public Health, vol....
  • [11] “Covid-19 digital rights tracker,” https://www.top10vpn.com/ research/ investigations/covid- 19-digital-rights-tracker/, accessed: 2021-05-10.
  • [12] U. Gasser, M. Ienca, J. Scheibner, J. Sleigh, and E. Vayena, “Digital tools against covid-19: taxonomy, ethical challenges, and navigation...
  • [13] M. E. Kretzschmar, G. Rozhnova, M. C. J. Bootsma, M. van Boven, J. H. H. M. van de Wijgert, and M. J. M. Bonten, “Impact of delays on...
  • [14] “Tracetogether,” https://www.tracetogether.gov. sg/index. html, accessed: 2021-05-10.
  • [15] H.Cho, D.Ippolito, and Y.W. Yu,“Contacttracing mobile apps for covid-19: Privacy considerations and related trade-offs,” arXiv, 2020.
  • [16] K. Michael and R. Abbas, “Behind covid-19 contact trace apps: The google-apple partner-ship,” IEEE Consumer Electronics Magazine, vol....
  • [17] S.Jha, L.Kruger, and P. McDaniel,“Privacy preserving clustering,” in Computer Security – ESORICS 2005. Berlin, Heidelberg: Springer Berlin...
  • [18] Z. Erkin, T. Veugen, T. Toft, and R. Lagendijk, “Privacy-preserving distributed clustering,” EURASIP Journal on Information Security,...
  • [19] S. Oliveira and O. Zaiane, “Privacy preserving clustering by data transformation,” Journal of Information and Data Management, vol. 1,...
  • [20] S. Merugu and J. Ghosh, “Privacy-preserving distributed clustering using generative models,” in Third IEEE International Conference on...
  • [21] G.Jagannathan and R. N.Wright,“Privacy-preserving distributed k-means clustering over arbitrarily partitioned data,” in Proceedings of...
  • [22] S. Basagni, “Distributed clustering for ad hoc networks,” in Proceedings Fourth International Symposium on Parallel Architectures, Algorithms,...
  • [23] H. Kargupta, W. Huang, K. Sivakumar, and E. Johnson, “Distributed clustering using collective principal component analysis,” Knowledge...
  • [24] P. A. Forero, A. Cano, and G. B. Giannakis, “Distributed clustering using wireless sensor networks,” IEEE Journal of Selected Topicsin...
  • [25] E. Januzaj, H. Kriegel, and M. Pfeifle, “Dbdc: Density based distributed clustering,” in Advances in Database Technology - EDBT 2004....
  • [26] M.Jia, Y. Wang, C. Shen, and G. Hug,“Privacy-preserving distributed clustering for electricalload profiling,” IEEE Transactions on Smart...
  • G. Muniraju, S. Zhang, C. T. M. Banavar, A. Spanias, C. Vargas-Rosales, and R. Villalpando-Hernandez, “Location based distributed spectral...
  • [28] A. Bertrand and M. Moonen, “Distributed computation of the Fiedler vector with application to topology inference in ad hoc networks,”...
  • [29] X. Shijie, F. Jiayan, and L. X. Li, “Weighted laplacian method and its theoretical applications,”IOP Conference Series: Materials Science...
  • [30] W. Chen, Y. Song, H. Bai, C. Lin, and E. Chang,“Parallel spectral clustering in distributed systems,” IEEE Transactions on Pattern Analysis...
  • [31] L. Xiao and S. Boyd, “Fast linear iterations for distributed averaging,” Systems & Control Letters, vol. 53, pp. 65-78, 2004.
  • [32] A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in Proceedings of the 14th InternationalConference...
  • [33] J. Shi and J. Malik, “Normalized cuts and image segmentation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22,...
  • [34] D. J. Higham, G. Kalna, and M. Kibble, “Spectral clustering and its use in bioinformatics,” Journal of Computational and Applied Mathematics,...
  • [35] F. Chung and F. Graham, Spectral Graph Theory, ser. CBMS Regional Conference Series. Conference Board of the mathematical sciences, 1997.
  • [36] M. Fiedler, “A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory,” Czechoslovak Mathematical...
  • [37] B. Hendrickson and R. Leland,“An improved spectral graph partitioning algorithm for mapping parallel computations,” SIAM Journal on ScientificComputing,...
  • [38] G. Dahlquist and A. Bjorck, Numerical Methods. Dover, 2003.
  • [39] J. H. Wilkinson, The algebraic eigenvalue problem. Oxford University Press, 1965.
  • [40] C.Boutsidis, P.Kambadur, and A.A.Gittens,“Spectral clustering via the power method - provably,” in Proceedings of the 32nd International...
  • [41] L. Page, S. Brin, R. Motwani, and T. Winograd, “The pagerank citation ranking: Bringing orderto the web,”in Proceedings of the 7th International...
  • [42] “The comix study,” https://www.uhasselt.be/UH/71795-start/ The-CoMix-study, accessed: 2021-09-22.
  • [43] A. Gimma, K. L. Wong, P. Coletti, and C. I. Jarvis, “Comix social contact data (spain),” Jun. 2021. [Online]. Available: https://doi.org/10.5281/zenodo.5040840
  • [44] S. Fortunato, “Community detection in graphs,” Physics Reports, vol. 486, no. 3, pp. 75-174, 2010.