Altmetrics can capture research evidence: an analysis across types of studies in COVID-19 literature

• Pilar Valderrama-Baca ^[1] ; Wenceslao Arroyo-Machado ^[1] ; Daniel Torres-Salinas ^[1]
  1. [1] Universidad de Granada

     Universidad de Granada

     Granada, España

     
• Localización: El profesional de la información, ISSN-e 1699-2407, ISSN 1386-6710, Vol. 32, Nº 2, 2023 (Ejemplar dedicado a: Digital native media ecosystem)
• Idioma: inglés
• DOI: 10.3145/epi.2023.mar.13
• Títulos paralelos:
  • Las altmétricas pueden capturar la evidencia científica: un estudio a través de tipos de estudios en la bibliografía de COVID-19
• Enlaces
  • Texto completo
• Resumen
  • español

    El COVID-19 ha tenido un gran impacto en la ciencia. Se ha convertido en un frente de investigación mundial que constituye un fenómeno único de interés para la comunidad cienciométrica. En consecuencia, han proliferado los trabajos descriptivos de COVID-19 que utilizan las altmétricas. Las métricas de medios sociales sirven para entender cómo se comparte y discute la investigación y uno de los puntos clave es determinar qué factores condicionan las altmétricas. El objetivo principal de este estudio es analizar si las menciones altmétricas de los estudios médicos de COVID-19 están asociadas al tipo de estudio y a su nivel de evidencia. Los datos se recogieron de las bases de datos PubMed y Altmetric.com. Se recuperó un total de 16.672 publicaciones clasificadas por tipo de estudio (por ejemplo, informes de casos, ensayos clínicos o metaanálisis) publicadas en el año 2021 y con al menos una mención altmétrica. Los indicadores altmétricos considerados fueron el Altmetric Attention Score (AAS), las menciones en noticias, las menciones en Twitter y los lectores de Mendeley. Una vez creado el conjunto de datos de COVID-19, el primer paso fue realizar un estudio descriptivo. A continuación, se contrastó la hipótesis de normalidad mediante la prueba de Kolmogorov-Smirnov, y dado que resultó significativa en todos los casos, se realizó la comparación global de grupos mediante la prueba no paramétrica de Kruskal-Wallis. Cuando esta prueba rechazó la hipótesis nula, las comparaciones por pares se realizaron con la prueba U de Mann-Whitney, y la intensidad de la posible asociación se midió mediante el coeficiente V de Cramer. Los resultados sugieren que los datos no se ajustan a una distribución normal. La prueba U de Mann-Whitney reveló coincidencias en cinco grupos de tipos de estudio, siendo el indicador altmétrico con más coincidencias las menciones de noticias y los tipos de estudio con más coincidencias las revisiones sistemáticas junto con los metaanálisis, que coincidieron con cuatro indicadores altmétricos. Asimismo, entre los tipos de estudio y los indicadores altmétricos se observó una asociación débil pero significativa a través de la chi-cuadrado y la V de Cramer. Se concluye que la asociación positiva entre altmétricas y tipos de estudio en medicina podría reflejar el nivel de la "pirámide" de la evidencia científica.

  • English

    COVID-19 has greatly impacted science. It has become a global research front that constitutes a unique phenomenon of interest for the scientometric community. Accordingly, there has been a proliferation of descriptive studies on COVID-19 papers using altmetrics. Social media metrics serve to elucidate how research is shared and discussed, and one of the key points is to determine which factors are well-conditioned altmetric values. The main objective of this study is to analyze whether the altmetric mentions of COVID-19 medical studies are associated with the type of study and its level of evidence. Data were collected from the PubMed and Altmetric.com databases. A total of 16,672 study types (e.g., case reports, clinical trials, or meta-analyses) that were published in the year 2021 and that had at least one altmetric mention were retrieved. The altmetric indicators considered were Altmetric Attention Score (AAS), news mentions, Twitter mentions, and Mendeley readers. Once the dataset of COVID-19 had been created, the first step was to carry out a descriptive study. Then, a normality hypothesis was evaluated by means of the Kolmogorov–Smirnov test, and since this was significant in all cases, the overall comparison of groups was performed using the nonparametric Kruskal–Wallis test. When this test rejected the null hypothesis, pairwise comparisons were performed with the Mann–Whitney Utest, and the intensity of the possible association was measured using Cramer’s V coefficient. The results suggest that the data do not fit a normal distribution. The Mann–Whitney U test revealed coincidences in five groups of study types: The altmetric indicator with most coincidences was news mentions, and the study types with the most coincidences were the systematic reviews together with the meta-analyses, which coincided with four altmetric indicators. Likewise, between the study types and the altmetric indicators, a weak but significant association was observed through the chi-square and Cramer’s V. It can thus be concluded that the positive association between altmetrics and study types in medicine could reflect the level of the "pyramid" of scientific evidence.

• Referencias bibliográficas
  • Arieta-Miranda, Jessica M.; Ruiz-Yasuda, Catherine C.; Pérez-Vargas, Luis-Fernando; Torres-Ricse, Dayhanne A.; Díaz, Solange-Pérez; Arieta,...
  • Aristovnik, Aleksander; Ravšelj, Dejan; Umek, Lan (2020). “A bibliometric analysis of COVID-19 across science and social science research...
  • Arroyo-Machado, Wenceslao; Torres-Salinas, Daniel; Robinson-García, Nicolás (2021). “Identifying and characterizing social media communities:...
  • Arsenault, Benoit J. (2022). “From the garden to the clinic: How Mendelian randomization is shaping up atherosclerotic cardiovascular disease...
  • Bhandari, Mohit; Montori, Victor M.; Devereaux, Philip J.; Wilczynski, Nancy L.; Morgan, Douglas; Haynes, R. Brian (2004). “Doubling the impact:...
  • Brainard, Jeffrey (2021). “No revolution: COVID-19 boosted open access, but preprints are only a fraction of pandemic papers”. Science, 8...
  • Chriscaden, Kimberly (2020). Impact of COVID-19 on people’s livelihoods, their health and our food systems. World Health Organization. https://www.who.int/news/item/13-10-2020-impact-of-COVID-19-on-people’s-livelihoods-their-health-and-our-food-systems
  • Colavizza, Giovanni (2020). “COVID-19 research in Wikipedia”. Quantitative science studies, v. 1, n. 4, pp. 1349-1380. https://doi.org/10.1162/qss_a_00080
  • Colavizza, Giovanni; Costas, Rodrigo; Traag, Vincent A.; Van-Eck, Nees-Jan; Van-Leeuwen, Thed; Waltman, Ludo (2021). “A scientometric overview...
  • Fassin, Yves (2021). “Research on COVID-19: A disruptive phenomenon for bibliometrics”. Scientometrics, v. 126, n. 6, pp. 5305-5319. https://doi.org/10.1007/s11192-021-03989-w
  • Fraumann, Grisha; Colavizza, Giovanni (2022). “The role of blogs and news sites in science communication during the COVID-19 pandemic”. Frontiers...
  • Harris, Joshua D.; Quatman, Carmen E.; Manring, Maurice M.; Siston, Robert A.; Flanigan, David C. (2014). “How to write a systematic review”....
  • Haunschild, Robin; Bornmann, Lutz (2021). “Can tweets be used to detect problems early with scientific papers? A case study of three retracted...
  • Haustein, Stefanie; Costas, Rodrigo; Larivière, Vincent (2015). “Characterizing social media metrics of scholarly papers: The effect of document...
  • Hayawi, K.; Shahriar, S.; Serhani, M. A.; Taleb, I.; Mathew, S. S. (2022). “ANTi-Vax: A novel Twitter dataset for COVID-19 vaccine misinformation...
  • Jung, Richard G.; Di-Santo, Pietro; Clifford, Cole; Prosperi-Porta, Graeme; Skanes, Stephanie; Hung, Annie; Parlow, Simon; Visintini, Sarah;...
  • Kousha, Kayvan; Thelwall, Mike (2020). “COVID-19 publications: Database coverage, citations, readers, tweets, news, Facebook walls, Reddit...
  • Kowalczyk, Nina; Truluck, Christina (2013). “Literature reviews and systematic reviews: What is the difference?”. Radiologic technology, v....
  • Lazcano-Ponce, Eduardo; Salazar-Martínez, Eduardo; Gutiérrez-Castrellón, Pedro; Ángeles-Llerenas, Angélica; Hernández-Garduño, Adolfo; Viramontes,...
  • Majumder, Maimuna S.; Mandl, Kenneth D. (2020). “Early in the epidemic: Impact of preprints on global discourse about COVID-19 transmissibility”....
  • Marcec, Robert; Likic, Robert (2022). “Using Twitter for sentiment analysis towards AstraZeneca/Oxford, Pfizer/BioNTech and Moderna COVID-19...
  • Montori, Victor M.; Wilczynski, Nancy L.; Morgan, Douglas; Haynes, R. Brian; The Hedges Team (2003). “Systematic reviews: A cross-sectional...
  • Murad, M. Hassan; Asi, Noor; Alsawas, Mouaz; Alahdab, Fares (2016). “New evidence pyramid”. Evidence based medicine, v. 21, pp. 125-127. https://doi.org/10.1136/ebmed-2016-110401
  • Nane, Gabriela F.; Robinson-García, Nicolás; Van-Schalkwyk, François; Torres-Salinas, Daniel (2022). “COVID-19 and the scientific publishing...
  • Odone, Anna; Salvati, Stefano; Bellini, Lorenzo; Bucci, Daria; Capraro, Michele; Gaetti, Giovanni; Amerio, Andrea; Signorelli, Carlo (2020)....
  • Okike, Kanu; Kocher, Mininder S.; Torpey, Jennifer L.; Nwachukwu, Benedict U.; Mehlman, Charles T.; Bhandari, Mohit (2011). “Level of evidence...
  • Patsopoulos, Nikolaos A.; Analatos, Apostolos A.; Ioannidis, John P. (2005). “Relative citation impact of various study designs in the health...
  • Pinho-Gomes, Ana-Catarina; Peters, Sanne; Thompson, Kelly; Hockham, Carinna; Ripullone, Katherine; Woodward, Mark; Carcel, Cheryl (2020)....
  • Priem, Jason (2014). “Beyond bibliometrics: Harnessing multidimensional indicators of performance”. In: Cronin, Blaise; Sugimoto, Cassidy...
  • Röhrig, Bernd; Du-Prel, Jean-Baptist; Wachtlin, Daniel; Blettner, Maria (2009). “Types of study in medical research: Part 3 of a series on...
  • Torres-Salinas, Daniel (2020). “Ritmo de crecimiento diario de la producción científica sobre COVID-19. Análisis en bases de datos y repositorios...
  • Torres-Salinas, Daniel; Robinson-García, Nicolás; Castillo-Valdivieso, Pedro A. (2020). Open access and altmetrics in the pandemic age: Forecast...
  • Valderrama, Pilar; Baca, Pilar; Solana, Carmen; Ferrer-Luque, Carmen-María (2021). “Root canal disinfection articles with the highest relative...
  • Valderrama, Pilar; Torres-Salinas, Daniel (2022). “Does the type of study on COVID-19 influence the value of altmetrics?”. In: Robinson-García,...
  • Van-Schalkwyk, François; Dudek, Jonathan (2022). “Reporting preprints in the media during the COVID-19 pandemic”. Public understanding of...
  • Van-Schalkwyk, François; Dudek, Jonathan; Costas, Rodrigo (2020). “Communities of shared interests and cognitive bridges: The case of the...
  • Zahedi, Zohreh; Costas, Rodrigo; Wouters, Paul (2014). “How well developed are altmetrics? A cross-disciplinary analysis of the presence of...
  • Zhang, Lin; Zhao, Wenjing; Sun, Beibei; Huang, Ying; Glänzel, Wolfgang (2020). “How scientific research reacts to international public health...
  • Zhang, Yi; Cai, Xiaojing; Fry, Caroline V.; Wu, Mengjia; Wagner, Caroline S. (2021). “Topic evolution, disruption and resilience in early...