Introduction

Readability is a measure of how easy a piece of text is to read and is determined by sentence structure complexity and the reader’s familiarity with the vocabulary (Bailin and Grafstein 2016). Evidence from studies of multiple scientific disciplines highlights that the readability of scientific literature is decreasing over time such that even educated members of the public are finding the research difficult to understand (Bauerly et al. 2006; Graf-Vlachy 2021; Plaven-Sigray et al. 2017). Furthermore, internet search algorithms use text readability to rank results and voice-search assistants preferentially return results that are easy to read emphasising that the ease of reading of research material is increasingly important for wider outreach and impact (McCulloch 2019). Furthermore, in several fields a positive association has been found between the readability of a scientific article and its impact as measured by its subsequent citation in the literature (Dowling et al. 2018; Snizek et al. 1991). Therefore, whether the goal is to disseminate information more widely among non-specialists or ensure research articles have greater academic impact, authors and journal editors should aim to improve the readability of scientific articles. Given the importance of public engagement to manage biological invasions worldwide (Hulme 2014; Matzek et al. 2014; Morelli et al. 2021), a decline in the readability of the underpinning science would limit the flow of knowledge between researchers and stakeholders as well as impede outreach to both the media and general public.

Scientific articles are often complex, assume prior knowledge of specialist vocabularies and use difficult technical terms. But even within the same discipline, some scientific articles are easier to read than others. There is evidence that abstracts penned by many authors are less readable since it can become increasingly difficult to accommodate editing suggestions of many authors without making the text harder to read (Graf-Vlachy 2021; Plaven-Sigray et al. 2017). Readability has also been found to be a function of the first language of the authors (Graf-Vlachy 2021; Hayden 2008). Linguistic studies indicate that authors proficient in English tend to write longer clauses and use more complex nominals per clause which might tend to reduce the readability of text (Lu and Ai 2015). Consistent with this view is that abstracts of articles where English was the first language of the principal author have been found to be less readable than those where this was not the case since they tended to use longer, more complex sentences (Hayden 2008). Declining readability of scientific articles over time might therefore reflect an increasing trend in the size of authorship teams and/or progressive over-representation of authors from English speaking countries. Given the importance of communicating research on biological invasions it is surprising that there has been no assessment of the readability of peer-reviewed research articles addressing this important topic, nor how readability might be changing over time or what factors determine variation in readability.

To address these issues, the readability of abstracts in the leading international journal Biological Invasions was examined from when it was first published in 2001 up to 2021. Although the target audience of Biological Invasions is primarily invasion scientists, published articles are becoming increasingly applied and over the last two decades a growing proportion of articles consider the implications of the research for the management of non-native species (Fig. 1). Yet managers still struggle to apply information published in scientific journals reflecting a significant knowing-doing gap (Funk et al. 2020; Hulme 2014). The advantages of targeting this journal rather than a wider search across the entire research corpus in the discipline is that Biological Invasions has a clear and specific remit to publish leading science in biological invasions, it is the primary resource for much of the discipline and also provides a standard, short abstract for every research article published. Abstracts were chosen since they are available outside of journal paywalls and thus the main source of information for journalists, stakeholders and the general public. Yet even when full-texts are accessible, abstracts are often the only section of an article that is read (Saint et al. 2000), and are the primary resource for filtering relevant articles in systematic reviews (James et al. 2016). Furthermore, the readability of abstracts is often correlated with that of the associated full text (Plaven-Sigray et al. 2017). Thus, the objectives of this study were to:

  1. 1.

    Identify any trend in decreasing readability of abstracts in the journal Biological Invasions over the last 20 years.

  2. 2.

    Assess whether authors based in countries where English is a de facto official language produce more complex abstracts than those countries where English is not an official language.

  3. 3.

    Examine whether the number of authors influences the readability of abstracts.

  4. 4.

    Determine whether the readability of an abstract influences the rate at which the work is subsequently cited.

Fig. 1
figure 1

Significant increasing trend over time in the percentage of articles published in the journal Biological Invasions that refer to management (r = 0.872, df 17, P < 0.001). Data are derived from a Topic search undertaken on 15/11/21 using the keyword “management” for articles published in Biological Invasions between 2001 and 2020 and listed in the Web of Science

Full size image

Methods

Selecting abstracts

A “Publication Name” search using the journal title “Biological Invasions” was undertaken using Web of Science on 24th November 2020 (which included articles published in December of that year) for all articles published between 2001 and 2020 in the journal Biological Invasions. Search results were filtered to only include research articles rather than reviews, corrections, conference proceedings or editorial material. The following data were extracted from each article: title, abstract, the number and institutional addresses of all authors, and the number of times the article had been cited across all databases archived in Web of Science.

To assess if the readability of abstracts was dependent on whether the authors were based in a country where English was a de facto official language (e.g., Australia, South Africa, United States etc.) abstracts were further filtered into two subsets based on the institutional address of the corresponding author. The institutional affiliation of the corresponding author was chosen so that results would be consistent with previous studies that have found this estimate of proficiency in English to be associated with subsequent abstract readability (Graf-Vlachy 2021; Hayden 2008). However, this measure cannot distinguish the occasions where the corresponding author is fluent in English but based in a country where English is not a de facto language or alternatively where an author who is not fluent in English is based in an English-speaking country. It might also be expected that where there are multiple authors the likelihood of one or more being fluent in English increases. However, in most cases, the entire set of authors belonged to the same country as the corresponding author (often the same institution). Any inaccuracies in correctly assigning the proficiency in English of the author team, if they are sufficiently common, should act to mask any signal of English language proficiency on abstract readability. Allocation of countries in terms of English being a de facto official language was based on Crystal (2012).

From each language subset, ten abstracts were selected at random for each year in the time series. Where there were fewer than ten papers published that year (primarily in the early years of publication) all abstracts were selected. A secondary check was made on the institutional addresses of the authors of all abstracts selected to ensure that each abstract could be classified in terms of whether the primary institution was based in a country where English was or was not a de facto language. This resulted in several abstracts being rejected from both subsets and replaced with the required number of additional abstracts chosen at random that met the language criterion.

Processing abstracts

All abstracts were read before processing to ensure any errors that might have arisen through the download process (e.g., deleting of spaces between words, insertion of non-print symbols) were corrected. Previous studies have processed abstracts by removing all abbreviations, adding spaces after periods when missing, adding a final period at the end of the abstract when missing, removing numbers that ended sentences, identifying sentences that end with’etc.’ and keeping the period, removing all single letter words except’a’,’A’ and’I’, replacing hyphens with a space, removing periods arising from the use of binomial nomenclature, and removing copyright and funding information (Graf-Vlachy 2021; Plaven-Sigray et al. 2017). Such pre-processing is important when comparing different fields that use a different syntax, abbreviations or abstract styles. Given this analysis compared a similar topic within the same journal it was assumed that there would be a common syntax and further processing of the abstracts was not undertaken.

Readability assessment

The importance of text readability in information sharing has led to the development of over 200 readability formulas that attempt to capture the complexity of writing (Gazni 2011). Nevertheless, reading comprehension is a multifaceted process involving not only lexical and syntactic complexity but also cohesion and sentiment that is unlikely to be captured by a single metric (Jin et al. 2021). Therefore two indices that best capture different components of text complexity (Bailin and Grafstein 2016) were used in the analysis. The Coleman-Liau Index (CLI) uses the number of letters and number of sentences per hundred words to generate a score that approximates the United States grade level required to understand the text (Huang et al. 2015). The index is calculated as 0.0588*LPW−0.296*ASW−15.8, where LPW is the number of letters per 100 words and ASW is the average number of sentences per hundred words (Yeung et al. 2018). The New Dale-Chall index (NDC) considers sentence length and frequency of unfamiliar words and has been found to be the only reliable measure of readability that captures Oral Reading Fluency across all ability groups (Begeny and Greene 2014). The NDC also generates a score that can be converted to a US grade level and words are considered difficult if they do not appear on a predetermined list of 3000 common words recognized by the average 9- to 10-year-old in the United States. The NDC is calculated 0.1579*PDW + 0.0496*ASL, where PDW is the percentage of difficult words and ASL is average sentence length in words and if PDW is > 5 then 3.6365 is added to derive the score (Graf-Vlachy 2021). An NDC of 10 or above indicates very difficult text that could be understood by a 16th grader or a college graduate, between 9 and 10, the text is difficult enough to be appropriate only for a college level of literacy (13th to 15th grade), between 8 and 9 the text is fairly difficult requiring a reading level of an 11th or 12th grader, only when scores are below 8 is text deemed sufficiently simple to be appropriate for conversational English. As in previous studies (Jayaratne et al. 2014; Mcinnes and Haglund 2011; Yeung et al. 2018), the online Readability Calculator was used to compute the CLI readability scores (www.online-utility.org/english/readability_test_and_improve.jsp) while the NDC was calculated using the online Readability Formulas tool (readabilityformulas.com/free-dale-chall-test.php).

Analysis

One-way ANOVA was used to examine whether there were significant differences between abstracts from authors who were or were not from de facto English-speaking countries in relation to: the number of characters, words and sentences in the abstract, the mean number of words per sentence, and the mean number of characters and syllables per word. Two different multiple linear regression analyses were undertaken. The first assessed the degree to which the two readability indices were related to the year of publication and the number of authors of each article. The second examined the degree to which the number of times an article was cited was a function of the year of publication, the number of authors, and either one of the readability measures. These multiple regressions were undertaken across the entire dataset as well as separately in relation to whether articles originated from countries where English was or was not a de facto official language. Prior to analysis, the number of times an article was cited was log10 transformed. All multiple regression models showed no significant collinearity (Variance Inflation Factor < 2) and had error terms that were normally distributed (assessed by a normal probability plot of the residuals). All analyses were undertaken in SPSS v 26 (IBM Corp. 2019).

Results

As might be expected, the two readability indices were significantly correlated with each other (r = 0.709, df 360, P < 0.001) but presented different interpretations regarding the reading difficultly of abstracts published in Biological Invasions. Both the CLI and NDC estimated the mean readability of abstracts to be at a level of difficulty appropriate for university undergraduates (US grade level 15), though many abstracts were more appropriate for the higher level of comprehension expected of doctoral researchers (Fig. 2). The similarity between the CLI and NDC scores suggests that it is the structure of the text (sentence length, number of characters per word) rather than the percentage of difficult words that most limits the readability of the abstracts. Authors based in countries where English was a de facto language tended to produce less readable abstracts, but only significantly so for readability assessed using the CLI (one-way ANOVA F(1.360) = 5.658, P = 0.018). This coincided with abstracts from authors in countries were English was a de facto language having a significantly higher number of characters (one-way ANOVA F(1.360) = 7.841, P = 0.005) and syllables (F(1.360) = 7.666, P = 0.006) per word.

Fig. 2
figure 2

Boxplots of readability scores using the Coleman-Liau and New Dale-Chall indices for abstracts stemming from countries where English was or was not a de facto language. Boxplots display the median, first and third quartiles while the whiskers show the maximum and minimum values, with the exceptions of outliers (circles)

Full size image

Irrespective of the index used, the readability of abstracts declined significantly over time. Between 2001 and 2020, the reading level of abstracts as measured by CLI increased by almost a whole grade level from 15.20 to 16.16. While this coincided with a general increase in the mean number of authors associated with each article (r = 0.257, df 360, P < 0.001), the number of authors was not a significant predictor of readability (CLI: F(2359) = 6.202, P = 0.002, adjusted R2 = 0.028; NDC: F(2359) = 3.905, P = 0.021, adjusted R2 = 0.016). Over the two decades examined, the number of characters (r = 0.190, df 360, P < 0.001) and syllables (r = 0.149, df 360, P < 0.001) per word increased significantly. However, these overall trends masked significant differences that depended on whether abstract were from countries where English was or was not a de facto language (Table 1). A significant decline in abstract readability was only found in abstracts from authors based in countries where English was not a de facto language (Fig. 3). Between 2001 and 2020, the reading level of abstracts as measured by CLI increased by almost two grade levels from 13.92 to 15.91 for articles arising from countries where English is not a de facto language (Fig. 3a), more than twice the rate of countries where English is a de facto language (15.56 to 16.40). A similar, though less marked, trend was found for NDC (Fig. 3b). For abstracts from countries where English was not a de facto language, there was a significant increase over time in the number of words (r = 0.181, df 173, P < 0.017) as well as the number of characters (r = 0.277, df 173, P < 0.001) and syllables (r = 0.227, df 173, P < 0.001) per word in the abstracts whereas no such trends were evident for countries where English was a de facto language.

Table 1 Standardised regression coefficients (Beta) and their statistical significance for the role of year of publication and number of authors per article on the readability of abstracts as estimated by the Coleman–Liau Index (CLI) and New Dale–Chall Index (NDC) for articles published in countries where English was or was not one of the de facto languages. Statistically significant values at P < 0.01 are highlighted in bold
Full size table
Fig. 3
figure 3

Declining readability over two decades of abstracts stemming from countries where English was or was not a de facto language a Coleman-Liau Index, b New Dale-Chall Index

Full size image

Not surprisingly, across all abstracts the number of citations an article attracted increased with the length of time since publication but also when there were more authors (CLI: F(2358) = 122.481, P < 0.001, adjusted R2 = 0.504; NDC: F(2358) = 118.974, P < 0.001, adjusted R2 = 0.496). For the CLI but not the NDC, the article citations increased with lower readability of abstracts. Once again, these overall trends were contingent on whether the article was from a country where English was or was not a de facto language (Table 2). Whereas the year of publication was an important determinant of the number of citations irrespective of whether English was a de facto language or not, the significant positive effect of number of authors was only found for articles arising from countries where English was not a de facto language. A further contrast is that while both readability indices were negatively (though not significantly) related to the number of citations for articles from countries where English was a de facto language, this trend was reversed where English was not a de facto language, particularly in the case of CLI. Thus, whereas for countries where English was the de facto language there was a weak trend for more readable articles to be cited more often, it was the less readable abstracts that were cited more frequently for countries where English was not the de facto language.

Table 2 Standardised regression coefficients (Beta) and their statistical significance for the role of year of publication, number of authors per article and readability of abstracts (as measured by the Coleman–Liau Index and New Dale–Chall Index on the total number of citations attributable to articles published in countries where English was or was not one of the de facto languages. Statistically significant values at P < 0.01 are highlighted in bold and at P < 0.05 in italics
Full size table

Discussion

Abstracts published in Biological Invasions, the leading journal in its discipline, are difficult to read. The two different readability measures both indicated that the average readability of the abstracts was equivalent to the United States 15th grade. The low readability effectively restricts the interpretability of the abstracts to those individuals with a university education. Although many commonly used words in the field of biological invasions (e.g., alien, biological, control, exotic, impact, invader, invasion, invasive, management, native, risk, vector) are not in the 3000 word vocabulary used by the New Dale–Chall index it appears that low readability is more a function of the structure of the abstracts. In particular, the tendency for authors to use long sentences as well as employ words with many characters and syllables, appears to be at the root of the low readability. Although the readability of abstracts is often lower than that of the full text to which they refer, readability is usually highly correlated and reflects a similar United States grade level of difficulty (Bauerly et al. 2006; Dronberger and Kowitz 1975; Plaven-Sigray et al. 2017; Yeung et al. 2018). Low readability is not unique to Biological Invasions, and a similar or greater level of reading complexity has been found for the abstracts of articles published in other disciplines (Barbic et al. 2015; Graf-Vlachy 2021; Kitchenham et al. 2008) Nevertheless, since the average American adult reads at an eighth-grade level and nearly one fifth of adults in the United States cannot comprehend fourth-grade level text (Eltorai et al. 2014), most abstracts published in Biological Invasions are unlikely to inform many individuals outside of a specialist community of university and government researchers.

It might be argued that abstracts are designed primarily to communicate with similarly educated research peers and thus a high level of reading complexity is expected but also unlikely to be a problem. However, many researchers working to address the problems of biological invasions around the world are not native English speakers and thus abstract complexity will pose a challenge for communicating research findings clearly. This is supported by the finding that authors from countries where English was not a de facto language produced less complex abstracts, as has been shown previously (Hayden 2008). Furthermore, as an applied discipline, it is crucial that the messages regarding biological invasions are clearly communicated to a much broader audience than simply the research community. For example, raising public awareness and influencing public attitudes are at the core of any attempts to resolve difficulties arising from society and its relationship with biological invasions (Heger et al. 2013). Much of scientific research is locked away behind paywalls such that abstracts are often the only freely available source of information to most professionals outside of academia (Schiltz 2018). There has been considerable discussion regarding how best to communicate the problems of biological invasions to the land managers, policymakers and community groups (Davis et al. 2018; Li et al. 2021; Verbrugge et al. 2021) but the important obstacle that complex abstracts pose to getting clear messages out to non-specialists remains under-appreciated.

The finding of declining readability of abstracts over time is consistent with evidence from other disciplines (Bauerly et al. 2006; Graf-Vlachy 2021; Plaven-Sigray et al. 2017) but was detected over a much shorter time-span for biological invasions. Will the readability of abstracts continue to decline in the future? There is no indication of a decelerating temporal trend in the mean value of either readability index suggesting there is scope for declining readability to continue into the future. The overall mean CLI was 9.93 but it ranged from 7.50 to 12.00, and similarly the overall mean NDC was 15.45 but it ranged from 8.72 to 21.05. Thus, average readability is still some way off from the least readable abstracts currently observed and could decrease in the future. Furthermore, there is already proof that scientific abstracts can become even less readable, as seen in the case of neuroimaging research, where the mean CLI for abstracts was 15.89, much higher than found for Biological Invasions (Yeung et al. 2018).

For the first time, a declining temporal trend in readability has been shown to be dependent on whether English was or was not a de facto language in the country in which an author's institution was located. Readability has declined most dramatically in those countries where English is not a de facto language. Over only two decades, the readability of abstracts from countries where English was not a de facto language progressively declined to reach a similar level of difficulty as those produced from countries where English was a de facto language by 2020. There was a clear change in the structure of abstracts from countries where English was not a de facto language, which may reflect that these authors are writing longer abstracts with a greater number of characters and syllables per word to match the dominant style of published abstracts. Most articles published in Biological Invasions stem from institutions based in the United States (46%), Australia (11%), Canada (8%), and the United Kingdom (7%) and thus countries where English was a de facto language will likely determine the dominant style of abstract. As the readership and authorship of Biological Invasions becomes increasingly more globalised, actions to prevent a further decline in abstract readability, especially those written by authors resident in countries where English is not a de facto language, should be prioritised.

Whereas the number of authors associated with an abstract did not influence readability significantly, it did increase the citation rate. However, even when controlling for the number of authors, citation rates were found to increase with reading difficulty, but only for abstracts from countries where English was not a de facto language. Such a counterintuitive finding has been found before and has been interpreted as illustrating a situation where expositional clarity and readability may not always be considered positive by peers (Gazni 2011; Stremersch et al. 2007). This effectively means that scientific peers may be more persuaded by a technically challenging abstract pitched towards a readership with high literacy than simpler abstracts. The reason this is only seen for countries where English is not a de facto language is probably that these have generally been the most readable abstracts published in Biological Invasions and thus there was more scope for them to be made more complex. Of course, there might be other reasons for this counterintuitive finding such as abstracts that are difficult to read may reflect multifaceted and challenging research that is subsequently more frequently cited.

The foregoing points to a major dilemma for authors and editors of Biological Invasions. Should abstracts be pitched to inform (and impress) scientific peers and thus garner more citations or should they be written in such a style as to increase the accessibility of the information to less formally trained stakeholders and members of the public? Clearly these two goals should not be traded-off against each other, but the current status quo favours a stronger emphasis of technical complexity rather than readability of abstracts. At least three potential solutions exist. The first could be for the journal to ask authors to report readability scores of their abstract when submitting a manuscript. As an example, the abstract of this article has a readability close to the average for authors from countries where English is a de facto language (CLI = 15.71, NDC = 9.1) but could be revised to be more readable (Box 1), increasing the readability by more than three grade levels (CLI = 12.38, NDC = 8.1). This was largely achieved by reducing the number of words per sentence by 40% from 23.64 to 14.00. Copyeditors could also focus on improving abstract readability in addition to checking spelling and formal style before publication. However, authors and copyeditors may find simplifying the text to be challenging since designing abstracts to fit with a readability index can lead to perverse outcomes such as over simplified text or fragmented sentences that might lead to misinterpretation of the research findings. In addition, there is no escaping the use of certain technical terms that will be unfamiliar to non-specialists and thus indices such as the New Dale-Chall may not be responsive to further edits (as seen in the example above).

The second is to require authors to provide a prescriptively structured abstract with distinct labelled sections (e.g., Aim, Methods, Results, Conclusions) which would help guide authors in summarising the content of their article. A growing body of evidence suggests that structured abstracts are more readable (both in terms of readability indices and reader ratings) than unstructured abstracts (Bertout and Schneider 2005; Hartley 2003; Hartley and Benjamin 1998; Kitchenham et al. 2008). Structured abstracts are required by several other journals in the related disciplines of ecology and biogeography and thus such a change could be rapidly accommodated by the author community of Biological Invasions. Third, authors could be asked to provide plain-language abstracts (or impact statements) once their articles have been accepted for publication. Plain-language abstracts are brief summaries of the research that have been pitched at a high school reading level for a more general audience, including land managers, policymakers and contractors, and have been found to be an effective means of communicating research to non-expert audiences (Kuehne and Olden 2015; Stricker et al. 2020). A growing number of journals across many disciplines are adopting plain-language abstracts and there is evidence that such summaries increase the profile of the research (Shailes 2017). Such an initiative would also require clear instructions regarding length and format to ensure plain-language abstracts are effective (Dube and Lapane 2014; Kuehne and Olden 2015). Many researchers are required to report on research impact and also provide simplified press releases associated with their publications, thus asking for an additional plain-language abstract may not be a significant burden.

Given the pressing problem of biological invasions worldwide, it is essential that research addressing non-native species has the widest possible readership and if this cannot easily be achieved through improved readability of abstracts and full papers, then establishing a requirement for an additional plain-language abstract may be a solution. As the leading journal in this field, Biological Invasions is in a strong position to improve accessibility of the information it publishes that could include considering one or more of these options: increasing the readability of abstracts, implementing structured abstracts or requiring plain-language abstracts. These additional aspects of publishing will likely pose only a minor inconvenience to researchers but promise great gains to the wider understanding of biological invasions and the urgent need to not only document but also mitigate the problems posed by non-native species.