Relationships among cost, citation, and access in journal publishing by an ecology and evolutionary biology department at a U.S. university

Data compilation

The analyses presented in this paper are built upon a detailed compilation of data about publication, reviewing, and editing activities of department members, as well as data about the journals themselves. Our focus was on peer-reviewed, journal-published scholarly publications, not out of disregard or lack of appreciation for other forms of scholarly publication (e.g., monographs, books, book chapters, blog posts), but rather in view of the fact that, in ecology and evolutionary biology, peer-reviewed journal papers are the principal currency by which faculty are evaluated for promotion, tenure, and general achievement and advancement. We focused on journals in which University of Kansas Ecology and Evolutionary Biology (henceforth “EEB”) faculty (all faculty members combined) had published at least twice (i.e., we eliminated journals in which EEB faculty had published only a single paper).

Data harmonization, improvement, and analysis

We next invested considerable time and effort in standardizing journal names, to ensure that information from individual journals was not duplicated across name variations. Although Google Scholar records presented some journal naming issues (e.g., spelling, duplications), these complications were much more challenging when harmonizing data among sources (e.g., Google Scholar, Web of Science, DOAJ, SHERPA/RoMEO, EEB annual reports, etc.).

The final dataset was then subjected to a series of exploratory analyses, mainly bivariate plots and regression analyses, designed to detect patterns. All statistical analyses were carried out in R. The full, final dataset is available via the University of Kansas’s digital repository, KU Scholarworks, at https://doi.org/10.17161/1808.32708.

Basic characteristics

In all, the 35 EEB faculty members published a total of 5,916 items that were listed in their Google Scholar profiles. Those profiles, however, included many items that were not peer-reviewed publications (e.g., abstracts), not journal publications (e.g., book chapters), or that duplicated other entries in profiles. As such, when we quality-controlled and cleaned the dataset, the total number reduced to 3540 journal publications.

Summarizing the data by journal instead of by faculty member, the number of EEB publications in journals that we analyzed ranged from 2 (see above) to 81 (Molecular Phylogenetics and Evolution). Other journals seeing frequent EEB publication included Zootaxa (76), Journal of the Kansas Entomological Society (76), PLOS One and ZooKeys (74), and Evolution (61) (Fig. 1). The journals with the fewest EEB publications were either journals in other fields, in which EEB researchers were often co-authors as part of early research experiences or out-of-field collaborations, or small and regional journals in which EEB faculty have published only occasionally.

KU EEB investment in and contribution to journals

Over a recent 4-year period, EEB faculty contributed over 110 reviews to single journals (Evolution, Trends in Ecology and Evolution) (Fig. 1), but far fewer in other journals where they nonetheless publish their research frequently (e.g., PLOS ONE, Proceedings of the Royal Society B). EEB faculty editorships were focused in journals of modest publication activity (e.g., Systematic Biology, Zootaxa, Oecologia)—that is, some EEB faculty dedicate significant editorial time to some journals in which they do not often publish. Although both reviewing activity and editorships related positively and significantly to number of EEB papers published (Table 1), researchers’ investment of time as reviewers or editors was not related directly to annual subscription price either (Fig. 2).

Costs and benefits

A first consideration is the relationship between various measures of journal quality and journal price—i.e., are academic libraries and (less directly) university-based scholars getting their money’s worth when they pay for expensive journals? The relationship between JIF™ and annual subscription price showed a relatively strong, positive relationship (R² = 0.220, P < 0.05). However, the positive slope of this relationship was a consequence only of Science and Nature, statistical outliers for JIF™ and both relatively high as far as price. Removing these two journals reduced the R² by 83.1% to 0.0372 (Fig. 3). The relationship between JIF™ and numbers of EEB publications was positive, but not strong: the overall regression had R² = 0.0236; removing two outlier journals (Science and Nature) reduced the strength of the relationship, to R² = 0.0036; in only the former case was the relationship statistically significant (Fig. 3).

Journal usage at the University of Kansas showed a positive relationship to annual subscription price (R² = 0.1054, P < 0.05). Again, however, the relationship was a consequence of the inclusion of Science and Nature and not a more general trend: removing those two journals eliminated the positive slope of the relationship and reduced the R² dramatically, effectively to 0, and the relationship was no longer statistically significant (Fig. 3, Table 1). Benefits to researchers in terms of citation rate, however, showed no relationship to annual subscription price. Overall, the relationship was weak (R² = 0.0429, P > 0.05; Fig. 3). Articles published in Science and Nature did not show any marked elevation in citation rate compared to other journals, such that the relationship did not change with their inclusion (R² = 0.0167, P > 0.05; Fig. 3, Table 1).

For open access journals, EEB faculty published as many as 81 articles in single journals (PLOS ONE). The APCs for publishing in open access journals ranged from $0 (i.e., “platinum” open access journals) to $5,300 (PLoS Biology). EEB faculty did not tend to publish more in free journals or in expensive journals (Fig. 4), but rather these publication choices appeared to be unrelated to APCs (R² = 0.003, P > 0.05).

The relationship between JIF™ and APCs was decidedly positive, with few or no platinum open access journals being accorded JIF™ ratings (= indexing in Web of Science). Indeed, JIF™ increased by ∼2 for every $1,000 of increase in APCs (R² = 0.679, P <0.05; Fig. 4, Table 1). Numbers of citations that individual papers received showed a positive relationship to APCs (R² = 0.1083, P < 0.05; Fig. 4, Table 1).

Finally, the cost of openness in journals in which EEB faculty published showed some rather impressive sums. That is, for two journals (Nature Ecology & Evolution, Nature Plants), the cost of making a published paper open access was above $9,000. In another seven journals, costs of making a published paper open access were above $5,000. Simply for the purposes of illustration, the total cost of publication, the sum of numbers of papers multiplied by the cost of openness for each journal where EEB faculty published their work is above $900,000.

Getting your money’s worth

To what degree are institutional investments of scarce financial resources “paying off” in the currencies of importance in academia, such as research impact, prestige, and citation rates? The extent to which impact factors are (or should be) of importance in judging academic excellence is hotly contested (Anonymous, 2016; Saha, Saint & Christakis, 2003; Seglen, 1997). Our analyses revealed that, without the outliers Nature and Science, three metrics that may be seen as assessing payoff or benefit to researchers (i.e., JIF™, citation rate, journal usage by readers), show no relationship to subscription costs—that is, expensive journals do not “pay off” in greater return to the researcher. Nature and Science rank among the most expensive of all KU EEB-related journals and represent outliers in several of our analyses. Similarly, those two journals (plus Scientific Reports and Proceedings of the National Academy of Science USA) are used at KU much more than the usage-price relationship among the remaining journals (Fig. 3). Undoubtedly, this effect might be attributed to the multi-disciplinary nature of these journals, whereas most of the rest are limited to ecology and evolutionary research dimensions. With or without those few, highest-profile journals, the relationship between number of publications by EEB faculty and impact factor was positive, but quite weak and not statistically significant.

Curiously, though, when one focuses on numbers of citations accrued by individual publications (“citation rates”) instead of journal-level impact factors, relationships change dramatically (Neylon & Wu, 2009), both for subscription-based and open access journals. Overall, citation rates per year were only slightly positively related to subscription costs. Among subscription-based journals, the standout journals were Ecography, Ecological Modelling, and Nature, each with modest subscription prices, or sufficiently high numbers of citations that a higher subscription price was possible. Similarly, among open access journals, citation winners were Ecography (in transition from subscription-based to open access), Science Advances, Royal Society: Open Science, and Nature Communications. Note that this calculation omits the so-called platinum open access journals, which have no APCs—journals with highest citation rates in this category include Emerging Infectious Diseases, Revista Mexicana de la Biodiversidad, Journal of Librarianship and Scholarly Communication, Revista da Sociedade Brasileira de Medicina Tropical, and Biodiversity Informatics. These contrasts between citation rates and impact factors as a basis for valuing journals have been pointed out in other contexts (Galligan & Dyas-Correia, 2013; Vanclay, 2009), but suggest that the current shift toward article-based valuation metrics may lead to consequent shifts in emphasis on different sets of journals as “winners”.

Journal Impact Factor (IF™) is a proprietary, journal-level metric calculated and published annually by Clarivate Analytics (Clarivate, 2021). In brief, this index is a proprietary and not-reproducible function of the mean number of citations in a given year of citable articles over the previous two years. Although impact factors were devised by a librarian and bibliometrician, who intended it primarily as a tool for use by librarians in collection development and management (Garfield, 1972), they have since been taken up by a commercial enterprise. JIF™ has been the subject of numerous critiques of both its inherent nature and its (mis)use (Baldwin, 2017; Juyal et al., 2019; McVeigh & Mann, 2009; PLOS Medicine Editors, 2006). Garfield himself stated “that you should not use JIF to evaluate a person or department” (Kim, 2000), a sentiment that was underlined in 2012 with the signing of the San Francisco Declaration on Research Assessment (DORA) (DORA, 2012). DORA was subsequently signed by >24,000 individuals and organizations in 164 countries (as of Dec. 2023).

Article impact is more appropriately assessed through article-level metrics, such as numbers of citations or citation rates, as well as by a number of so-called “alternative” metrics (“alt-metrics”), such as views and downloads, social media attention, and news reporting on an article (Altmetric, 2021). Of course, the best assessment of the quality of an article is to read the article, but this can quickly become unwieldy for evaluators such as those on hiring, promotion, and funding committees. Article-level metrics also have limitations and must be used appropriately, but they do address the article rather than the venue of publication, and therefore represent a more suitable set of tools than JIF™ or other journal-level metrics of any sort for assessing the quality, impact, and merit of individual publications. These ideas are underlined by the results of this study, in which the “best-value” journals for researchers and academic institutions are nuanced and subtle in the qualities that distinguish them.

Investing time and energy in the right journals

One can imagine diverse motivations for scholars to invest their time in particular journals: keeping current with the literature, supporting a particular field, allegiance to a scientific society, and the prestige of reviewing or holding an editorial position at a high-profile journal. KU EEB faculty reviewed manuscripts with particular frequency for journals such as Evolution and Trends in Ecology and Evolution, in which they published frequently, but also for prestigious journals like Current Biology, where KU EEB publication is not as frequent. Similarly, KU EEB editing effort was focused in journals such as Systematic Biology, Zootaxa, and Oecologia, which were not particularly high in frequency as publishing venues for KU EEB faculty.

KU EEB “investment” in journals was also not related to price. One might expect that such a relationship might be positive, if price were a good indicator of value or prestige, or that it would be negative, if KU EEB faculty were investing their time and effort in journals that are good bargains for them and the university. In fact, the relationship was not at all clear, such that we suspect that numerous other priorities and motivations direct how faculty dedicate their time and effort. This set of motivations would seem to represent an interesting and fruitful area for future research.

Perspectives for the future

The landscape of scholarly publishing is continuing its rapid shift. Older, subscription-based models are clearly decreasing in number, with journal after journal “going open” (e.g., Araújo, Svenning & Tuomisto, 2019), notwithstanding the mixture of good and bad consequences from those changes (Peterson et al., 2019a). This shift mirrors the earlier, massive-scale commercial investment in academic publishing and its effects. Once again, commercial interests (i.e., profit, market share, prestige) frequently outweigh the academic motivations of communication and access, leading to discords and mismatches between the publishing and academic communities and their respective interests (Buranyi, 2017).

Journal-level metrics are heavily used by committees conducting job searches or evaluation junior faculty for promotion and tenure. Our data show that, at least for the case of KU EEB, the most expensive journals are not necessarily those most cited or even, excluding a few outliers, those with highest impact. Our results clearly highlight that, for individual researchers, investment in service to certain prestigious and expensive journals makes dubious sense. What to do then? The idea that impact factors should be used to assess individual productivity has been so strongly criticized (Schimanski & Alperin, 2018; Vanclay, 2012) that we wonder why it continues being used. Better ways to evaluate academic productivity have already been suggested, particularly in the form of article-level quality metrics (Schimanski & Alperin, 2018).

A consequence of these evolving publishing modes and outcomes has been the broad exploration of new models of subscription and payment—in effect, business models for academic publishing. Significant experiments include the “Public Library of Science”, and its exploration of not-for-profit open access journals (which nonetheless have high APCs) (Bernstein et al., 2003); another novel business model is the author membership-based PeerJ (which also ends up charging the equivalent of a significant APC in the absence of an institutional membership) (Binfield, 2013). Another force in this academic publishing world has been that of open access mandates, particularly by funding agencies (e.g., Suber, 2008), and so-called “green” open access, in which individual researchers make versions of their research open and available. This opening of access absent financial return to the journals, regardless of annual subscription prices or APCs, was originally feared to bankrupt academic journals; however, in the case of the largest funder mandate to date in the United States, it did not result in noticeable changes in viability and survival of journals in the fields that were most affected (Peterson et al., 2019b). Indeed, in some cases, changes forced on journals by commercial publishing enterprises have led to mass resignations of editorial boards, exemplifying the priority contrasts between the different stakeholders in the academic publishing world (Peterson et al., 2019a).