Introduction

SARS-CoV-2 is a highly pathogenic β-coronavirus that infects humans and caused a pandemic, with the first case reported in China on December 31, 2019 [1]. The clinical presentation of the infection caused by coronavirus 2019 (COVID-19) can vary from an asymptomatic condition [2] to a critical disease [3]. Its early diagnosis is fundamental to prevent the emergence of new cases; thus, several diagnostic methods have been used, mostly microbiological [4] and radiological. A large body of literature has been published on the latter methods, mostly aimed at clarifying the evolving role of imaging techniques such as chest radiography, ultrasound, or computed tomography (CT), in the diagnosis and management of patients with suspected or confirmed COVID-19 infection [5, 6].

A scientific document cited in a radiology article, as in any other subject, becomes part of a defined body of literature on that specific topic and shares stronger or weaker relationships with the other cited documents. Citation analysis is precisely the area of bibliometrics that studies those relationships [7], thereby constituting a very common method for evaluating the impact of different information entities in scientific processes (such as authors, institutions, publishers, countries, and publications) [8].

Another bibliographic method that can provide additional information on the study of citations is co-citation analysis, which is most valuable for its ability to identify research clusters and intellectual structures in disciplines within a given field [9, 10].

The objective of this study is to examine the intellectual structure of radiology research on COVID-19 based on citation and co-citation analysis using bibliographic data retrieved from the Web of Science (WoS) Core Collection and mapped with VOSViewer.

Materials and methods

To map systems thinking about COVID-19 from the perspective of radiology, bibliographic data were collected to construct co-citation networks. A co-citation between two documents was established when both were listed in the references of a third document.

Study methodology

Phase 1: Identification of the study subject and data acquisition

A literature search was conducted in the WoS Core Collection (Thomson Reuters) on January 3, 2022. We retrieved all document types in any language published in journals in the category of “Radiology, Nuclear Medicine, and Medical Imaging” of the Journal Citation Reports™ with a title including the words “COVID-19” or “SARS-CoV-2.” The results were restricted to the period of 2020–2021. This time interval was selected since, although cases of the disease were already reported at the end of 2019, COVID-19 was considered a global phenomenon only in 2020 when the pandemic was declared by the World Health Organization (WHO) [11]. “Early Access” content (electronically published in a nearly final state before their assignment to a specific volume and issue) was not excluded.

All study data was publicly available and did not contain public health information. Therefore, approval by the Bioethics Committee was not applicable and informed consent was waived.

Phase 2: Data processing and construction of a co-citation matrix

The publications were classified according to document type descriptions used in WoS Core Collection, which includes articles (research papers, brief communications, technical notes, full papers, and case reports), letters (readers write, questions and answers, letters to the editor, and comments) and editorial material (editorials, interviews, post-paper discussions, round table symposia, conference summary, research highlights, etc.).

During this phase, the necessary data were extracted from the selected sources. From these, a descriptive analysis of multiple variables (such as document type, journals, authors, and citations in WoS) was conducted. Citation density was calculated by dividing the number of citations for an article by the number of years since it was published. For articles with international collaborations, the countries of all co-authors were considered. In contrast, only the affiliation of the corresponding author was considered when analyzing funding.

Second, for the construction of a co-citation matrix, pairs of co-cited references were identified based on the references of the full set of analyzed documents. This facilitated the identification of pairs of documents with a high degree of co-citation.

Phase 3: Creation of co-citation network analysis and data interpretation

Once the matrix with absolute frequencies of co-citation was obtained, the data were processed with VOSViewer version 1.6.18 and Gephi. The combined use of both programs was chosen to minimize their individual limitations. A co-citation network was constructed with lines connecting the co-cited documents. Therefore, the nodes represented cited documents and the edges represented instances of co-citation. The edge weights represented the number of times two documents were co-cited by third articles (Fig. 1). To obtain the final sample, only documents with a co-citation threshold equal to or greater than 12 were considered. Another important parameter was the degree of a co-cited document, which was equal to the number of edges and represented the number of neighboring documents. This allows for the classification of the co-cited documents, thereby detecting close connections that allow inferring communities.

Fig. 1
figure 1

Schematic summary of the relationships between nodes in a co-citation network. A Co-citation network. B Relationships between nodes and the value of their edges. C Co-citation network after excluding nodes with less than three co-citations

Full size image

Network visualization

A graphic representation of the most significant associations between documents forms a network. It is thus possible to illustrate the strongest interrelationships between documents. Moreover, it is also possible to assess the structures formed by the components and subcomponents of the research, the links between the several subcomponents, and the documents that play an important role as intermediaries between nodes (Fig. 2).

Fig. 2
figure 2figure 2

Co-citation networks divided according to co-citation levels in a ≥ 12 edges, b ≥ 24, c ≥ 36, and d ≥ 48

Full size image

Results

Descriptive analysis

Of the 3418 documents that met the inclusion criteria, 2561 (74.93%) were final publications and 857 (25.07%) were initially “Early Access” articles. During 2020, the final version of 1529 documents (44.74%) was published, which increased to 1759 (51.46%) in 2021. Ninety-three “Early Access” articles (2.72%) were assigned to an issue of a journal in 2022, while 37 (1.08%) were still in press. Most articles (79.29%) were indexed in the Science Citation Index Expanded (SCIE), and 19.95% were indexed in the Emerging Sources Citation Index (ESCI).

The different types of documents are presented in Fig. 3. Those published as articles stand out, followed by letters and editorial material. The category “other types” includes corrections (0.97%), proceeding papers (0.76%), news items (0.29%), and book reviews (0.03%). With the exception of 26 documents categorized as proceeding papers presented at a symposium or conference, the remaining 3392 (99.24%) were published in 175 scientific journals. Table 1 shows the 10 journals that published the highest number of articles.

Fig. 3
figure 3

Types of documents included in Web of Science that were published in the period of 2020–2021 and met the search criteria (absolute frequencies and percentages)

Full size image
Table 1 Top 10 radiology journals according to the number of articles published
Full size table

Regarding citations in WoS, 2223 documents had citations (65.04%), 393 had more than 20 citations (11.48%), and 83 had more than 100 (2.43%). In total, they accumulated 47,475 citations, although 1195 documents did not receive any (34.96%). The distribution of the documents according to the degree of citation is shown in Fig. 4. The second issue of Radiology (volume 296) published on August 2020 stood out, receiving 9333 citations. The 10 most cited articles (Table 2) were published in open-access format in 2020 in journals indexed in SCIE; only three of them were funded.

Fig. 4
figure 4

Documents published in the period of 2020–2021 grouped according to the number of citations received in Web of Science (accessed January 3, 2022)

Full size image
Table 2 Top 10 radiology articles according to the number of citations received
Full size table

The five authors with the highest number of publications were Ali Gholamrezanezhad (University of Sothern California, Los Angeles, USA) (n = 42), Riccardo Inchingolo and Andrea Smargiassi (Fondazione Policlinico Universitario Agostino Gemelli, Rome, Italy) (n = 18), Liming Xia (Huazhong University of Science and Technology, Wuhan, China) (n = 17), and Libertario Demi (University of Trento, Trento, Italy) (n = 17). In contrast, the five most cited authors were from China: Liming Xia (3995), Qian Tao (2684), Chenao Zhan (2684), Tao Ai (2643), and Zhenlu Yang (2591).

Table 3 lists the countries and organizations that published the highest number of documents. The first place is occupied by the USA (32.62%), followed by Italy (13.02%), China (12.64%), England (7.93%), and France (5.53%). Furthermore, six of the most prolific organizations were from the USA. China generated the first publications and, in contrast to the other top five countries, published nearly two-thirds of its research in 2020 (62.53%). Most documents were published in English (96.84%), except for a small percentage in other languages, namely, German (1.87%), Spanish (0.94%), and French (0.35%).

Table 3 Top 10 countries and organizations according to the number of documents published
Full size table

In total, 611 studies (17.88%) reported having received funding, either public or private, while 2807 did not (82.12%). The countries with the highest number of funded studies were China (45.38%), the USA (16.29%), Italy (16.8%), England (20%), and Germany (24.09%). Figure 5 shows the relationship between documents with and without funding in the 10 countries with the highest number of publications.

Fig. 5
figure 5

Studies with and without funding in the 10 countries with the highest number of documents published in the period of 2020–2021 based on the affiliation of the corresponding author

Full size image

Co-citation analysis

A total of 33,496 co-citations were identified, with 7187 repeated in two or more documents. The co-citation network supported the identification of seven thematic research clusters that together comprised 647 publications co-cited at least 12 times (Fig. 2a). Of all the publications collected in these clusters, most belonged to articles published after 2010 (97.06%, n = 628) and, essentially, from 2020 (85.78%, n = 555). Only three articles published before 2000 (1946, 1977, and 1988) were found.

The 647 documents were published in 206 different journals. Table 4 shows the main journals in decreasing order of importance based on the number of published documents. Regarding the research areas of the published documents and considering that a given article can belong to several areas, 319 (49%) were assigned to “Radiology, Nuclear Medicine, and Medical Imaging,” while 120 (18.55%) belonged to “General Internal Medicine.” Among the other 56 categories, “Cardiovascular System Cardiology” and “Oncology” stood out, with 48 (7.41%) and 38 (5.87%) documents, respectively. As for the place of origin of the authors, China and the USA were the countries with the highest number of researchers (Fig. 6). The most represented institutions were the Huazhong University of Science and Technology, with at least one author in 60 articles (9.27%), followed by Harvard University with 52 (8.04%), and Wuhan University with 34 (5.25%).

Table 4 Ranking of the 10 journals with the highest number of co-cited documents in the main thematic clusters
Full size table
Fig. 6
figure 6

Map chart showing the countries with the highest number of authors in the articles of the co-citation network

Full size image

Concerning the type of published document, the most frequent were original articles (65.38%, n = 423), considerably ahead of other formats, such as editorial material (11.13%, n = 72), letters (10.66%, n = 69), and reviews (10.20%, n = 66). The other types of documents are a heterogeneous group of publications in which conference presentations or guidelines published by entities such as the WHO are particularly important.

Table 5 shows the most co-cited articles according to their community cluster. The first three communities are composed of documents primarily from high-impact factor radiology journals and mostly address the diagnosis of COVID-19 by chest radiography and CT. However, despite the similarities, nuances differentiate these three blocks. While the profile of the documents of the first community is more oriented towards interdisciplinary consensus or radiological severity indices, the other two communities provide documents focused on the correlation of radiology and laboratory parameters. Group two also includes some key articles on the evolution of COVID-19 in patients in intensive care units.

Table 5 Most cited documents by communities. The three most cited documents from each of the seven communities shown in Figure 2a are presented
Full size table

The documents of community four mostly focus on issues related to coagulation abnormalities observed in coronavirus-infected patients.

The fifth community is where most nuclear medicine articles are found. A few consensus documents and hospital protocols that include recommendations for radiologists are also found in this group.

The sixth group features documents on the ultrasonographic diagnosis of pleuropulmonary manifestations of COVID-19.

Finally, the seventh group is composed of documents from clinical journals on topics such as artificial respiration in critical patients and corticosteroid treatment.

Of the 647 most co-cited articles, 32 were specific articles on artificial intelligence, obtaining a much higher citation average compared to the citation average of the rest of the articles (1864.59 and 1002.23, respectively). The documents with the most specific weight came from the Conference on computer vision and pattern recognition, and medical imaging computing journals, all of them published years before the pandemic.

Discussion

There are currently several articles that have conducted a co-citation analysis on COVID-19 [12,13,14] and some are even focused on a field of research [15] or a specific specialty [16]. However, this is the first study to focus specifically on publications in the specialty of radiology. This new approach provided a more detailed analysis of the most relevant topics from an imaging perspective, which might have been overlooked in a more generic study.

As shown in Table 4, more than 10% of the 647 most co-cited documents were published in the journal Radiology. Far behind is European Radiology. Noteworthy is the presence of Radiology: Cardiothoracic Imaging in fourth place, a journal that was created just a few months before the detection of the first cases of COVID-19 [17]. The relationship between the impact factor and the relative position of the publication in this classification is partially fulfilled in radiology journals, but not in clinical journals, such as NEJM or The Lancet, which have an impact factor that is much higher than the rest. This is expected, as radiology journals tend to frequently co-cite other radiology journals. However, as previously proposed by other authors, the impact factor is a poor measure of relative importance and is not always clearly associated with other bibliometric parameters [18]. Some of the journals in the category “Radiology, Nuclear Medicine, and Medical Imaging” that have received the most citations in the last decade [19] have had a lower relative weight in our co-citation clusters, as they address topics that are not strongly affected by the pandemic (cardiac imaging, neuroradiology, magnetic resonance imaging, and nuclear medicine).

As for the place of origin of the authors, China is the country with the highest number of researchers in our selection of co-citations, which is consistent with a previous article [19]. This contrasts with the low presence of Chinese journals in the group of 647 documents, which could indicate a potential loss of citations by foreign authors, since they are mostly written in Chinese.

The seven obtained communities are largely composed of articles investigating the radiological diagnosis of pulmonary manifestations of COVID-19. This is not surprising, since in 2020, the year in which most of these articles were published, the radiological diagnosis of pulmonary disease was attracting even more interest from researchers than the development of vaccines or treatments, according to the conclusions of another bibliometric study [20].

Community four is surely the best defined of all, since it focuses on hematology articles on the post-Covid hypercoagulable state. This has significant clinical and epidemiological relevance since an incidence of venous thromboembolism of up to 69% has been described in these patients [21].

It is not surprising that ultrasound has its own community since it is an exam that attracted much interest at the beginning of the pandemic for its safety and accessibility [22, 23]. However, it has limitations. First, the doctor is very close to the patient when performing the ultrasound, with the risk of infection that this entails. Second, it is an operator-dependent technique.

Our results suggest that artificial intelligence has been underrepresented in terms of the total number of papers, despite the fact that this topic has received a higher number of citations on average. This is in line with other published work, in which articles on COVID-19 that include artificial intelligence have a higher number of citations and are published in journals with a higher impact factor [24].

Despite their recent publication, almost two-thirds of the analyzed articles had been cited and had a high total citation (47,475). Although the percentage exceeding 100 citations (2.43%) was similar to that described in a systematic review and meta-analysis of high-impact radiology journals (2.88%), the study used a much longer time window [25]. Furthermore, the most cited articles were published in open-access format, which results in faster dissemination and is associated with a higher number of citations and downloads compared to subscription journals [26,27,28].

COVID-19 disease gave rise to an unprecedented number of radiology publications starting in the second quarter of 2020, the number of which was even higher in 2021 (1529 as opposed to 1759). This is consistent with the trend observed in other disciplines, which peaked in March 2021, following an increase in overall incidence in late 2020 and early 2021 [29]. Simultaneously to this phenomenon, which can cause a saturation effect that hinders identifying and citing novel papers with relevant contributions [30, 31], scientific production in other fields of medicine decreased [32].

The speed at which such a volume of publications was generated can be read in two ways. On the one hand, it responds to the effort to produce and share new knowledge on the role of imaging in an emerging disease. On the other hand, many journals prioritized publications on COVID-19 [32] and, to this end, modified the editorial process, which is key to ensuring rigorous and quality studies. Thus, reduced acceptance and publication times, flawed peer-review processes, higher withdrawal and retraction rates [33], increased preprints [34, 35] and corrections, and frequent omission of reference to ethics committee's approval and informed consent [36] have been shown. Furthermore, there were fewer randomized controlled trials and more editorial, opinion, and single- or multi-case-based articles [36], which contrasts with the usual practice in many radiological journals, where original articles and reviews had been predominant [32, 37, 38].

As for funding which ranged from 21.6 to 47.7% in radiology research according to previous studies [27, 39], a decrease in its mean frequency (17.88%) was observed, only surpassed by China (45.38%). Lastly, international collaborations between authors, which result in a positive impact on citation [40], have been described as heterogeneous and insufficient to provide a coordinated response in a context of a global emergency [40].

As limitations of the study, many journals and documents are not indexed in the WoS Core Collection, possibly because this database is likely biased in its inclusion of high-impact literature, neglecting lower-impact documents. Another limitation is that the co-citation analysis focuses only on highly cited publications, ignoring recent publications or those belonging to a niche far from the most common thematic clusters. Regarding possible lines of research emerging from our study, and since co-citation analysis allows for the assessment of the past in a specific field of research, a bibliographic coupling and co-word analysis would be necessary to prepare a review more focused on the present and future, respectively.

In conclusion, the rapid growth of radiology publications related to COVID-19 in the toughest years of the pandemic clearly shows the relevance of the radiologist in the early diagnosis of the disease. This topic has greatly attracted the attention of researchers to radiology publications. Furthermore, the diverse origin of the most co-cited documents indicates that the research required global cooperation between various authors, organizations, and countries to progress.