1 Introduction

The COVID-19 pandemic has had a significant negative impact on mental health worldwide [1]. Lockdowns, public health restrictions, and associated disruptions to daily living increased stress, mental distress, and vulnerability in the population. Between 2020 and 2021, 3.5% of all presentations made to Australian emergency departments (EDs) related to mental health [2]. The ED remained open during COVID-19 despite public health restrictions and limited access to in-person primary health care services, providing a service for those who required emergency mental health care.

Data on mental health presentations is vital for health service planning as it can assist in retrospectively understanding trends and preparing for future pandemics [3]. In their analysis of mental health presentations to EDs in Western Australia from 2019 to 2020, Dragovic et al. [4] identified a decrease in cases at the onset of COVID-19 compared to 2019. Likewise, Jessup and Bramston [5] found a decrease in weekly presentations to the ED overall, not only for mental health, in Victoria as a result of the COVID-19 pandemic. Studies conducted internationally also found decreases in ED presentations for mental health crisis care during the COVID-19 pandemic, suggesting that these trends are not unique to Australia [6, 7].

Understanding variations in trends based among diagnostic groups is valuable as it informs which populations are more vulnerable and in need of more targeted interventions and specialised support. In Germany, Seifert et al. [8] found that during COVID-19, patients with affective disorders were less likely to present to an ED, whilst patients with personality and behavioural disorders were more likely to present to ED [8]. Additionally, both diagnostic groups were more likely to re-present to ED within a month of previous psychiatric care [8]. No differences in presentation rates for other diagnostic groups (e.g., substance use disorders, schizophrenia) were found [8].

A patient’s encounter with a health service generates substantial amounts of unstructured data, including triage data, clinical observations, and admission and discharge notes [9]. Large health service datasets provide insights into those accessing a health care service, as well as determining patterns of presentation [10]. Mental health presentations to public hospital emergency department settings also produces a large dataset, where significant shifts in mental health presentations have been observed since the onset of COVID-19 [11, 12]. Despite the opportunities that such large datasets present, there are several limitations. At present, there is no standardised method for data collection, input, or structuring across healthcare services within Australia, which can cause difficulty in extracting meaning and making comparisons across services [9]. Large datasets can also be costly to mine manually in terms of time and skilled human resources [13,14,15,16,17].

Machine learning (ML) refers to a wide range of techniques, which can automatically detect trends or patterns in a given dataset [18]. ML can be particularly useful when applied to problems which are otherwise too time or cost prohibitive to address. ML methodologies have been used to mine large volumes of unstructured data to extract and create structured data in relation to specific mental health presentation [19, 20]. Rozova et al. [21] used ML to detect self-harm presentations from 477,627 ED triage notes (2012–2018) at The Royal Melbourne Hospital, Australia. Data mining can automate the data extraction process, thereby reducing the cost of time and skill required to manually extract the data [10].

This research aimed to present a novel approach to classifying mental health presentations to EDs in Australia, using a suite of ML algorithms. The goal of these ML algorithms was to explore the trends in five mental health-related diagnostic groups in people presenting to EDs before and during, the COVID-19 pandemic in Australia:

  1. 1.

    Suicidality—presentations including suicide attempt, suicidal ideation, and non-suicidal self-injury.

  2. 2.

    Eating disorder—presentations relating to new or ongoing eating disorder diagnosis, such as anorexia nervosa, bulimia nervosa, binge eating disorder, and other specified eating disorders.

  3. 3.

    Mania—presentations including or relating to manic depression, bipolar disorder, or manic episodes.

  4. 4.

    Psychosis—presentations including or related to psychotic episodes or symptoms, and psychotic disorders such as schizophrenia or schizoaffective disorder.

  5. 5.

    Substance Use—presentations including accidental or intentional drug or alcohol intoxication, overdose, or other related problems.

2 Methods

2.1 Context and emergency department data analysed

This work was conducted under ethics exemption (EX/2022/QGC/85883). This study examined ED data within the Gold Coast Hospital and Health Service (GCHHS), a public health catchment within the metropolitan Gold Coast region, in Queensland, Australia, which provides specialised mental health services, in addition to general public health care to a population of approximately 640,000 people [22]. The Gold Coast is a transient population with a significant number of interstate and international visitors. This region has two public EDs, at Gold Coast University Hospital and Robina Hospital, which are both public hospitals providing tertiary level care. The former is recognised as the ED with the highest throughput in Australia [23]. All ED presentations between April 2019 to February 2022 were assessed (N = 690,514).

2.2 Identification of mental health presentations with specific characteristics

An ML framework, called Mining Emergency Department Records, Evolutionary Algorithm Data Search (MEDREADS), was used to identify five mental health diagnostic groups in people presenting to the ED: (1) suicidality, (2) eating disorder, (3) mania, (4) psychosis, and (5) substance use. MEDREADS was developed using a range of variables extracted from Cerner FirstNet®, the ED patient records database used within GCHHS [24].

For each of the five diagnostic groups, an evolutionary algorithm was used to weight categorical variables as shown in Appendix A. Defined values for each of the included categorical variables have been provided in Appendix B. The presenting problem description, an unstructured text field consisting of the notes written by the triage intake nurse (e.g., “BIB QPS causing disturbance outside high school. Mhx previous inpatient admission”), was also included. These were chosen by a senior psychiatrist (NS) and selected if they were deemed to be of clinical significance.

The evolutionary based approach used in this study is a standard genetic algorithm which was tailored for a psychiatric classification task, to minimise deviation between observed and predicted values. A separate genetic algorithm implementation is developed for each of the five diagnostic groups. Each model assigns weights or scores to each of the variables in Appendix A. For a given presentation, the sum of the scores is compared to a model-specific threshold value to determine whether the presentation involves the diagnostic group or not. The threshold value was selected to achieve a specificity of 95% while maximising sensitivity.

The genetic algorithm was trained using 24,996 ED presentations obtained between 7th January 2020 and 31st August 2020 and validated on a dataset of 39,885 ED presentations obtained between 1st September 2020 and 28th November 2020. For the training dataset, each presentation was manually rated either 0 (no) or 1 (yes) for each of the diagnostic categories, by trained and supervised raters (CH and GB, supervised by NS). The presentations that were coded for were not mutually exclusive (i.e., one presentation could be coded for multiple diagnostic categories). The trained algorithm was then applied to the whole dataset.

2.3 Statistical analysis

Statistical analyses were undertaken using Stata 17 and WinBUGS 1.4.3 [25, 26]. The MEDREADS diagnostic algorithm for each of suicidality, eating disorder, mania, psychosis, substance use was applied to a test data set consisting of 39,885 ED presentations for which a definitive diagnosis was available. The overall diagnostic test accuracy for each MEDREADS score was estimated by the area under the receiver operating characteristic (ROC) curve (AUC-ROC) and the optimal cut-point identified by Youden’s method. Sensitivity (Se) and specificity (Sp) were calculated with binomial exact 95% confidence intervals (CI). MEDREADS was then applied to each month of ED presentations from April 2019 to February 2022. Crude prevalence estimates for each month were calculated as the proportion (with logit 95% CI) of MEDREADS positive diagnoses. Unless a diagnostic test is perfect, false positives and false negatives will result in an inaccurate true prevalence estimate. If the true prevalence is small, the crude prevalence will likely be an overestimate.

To address this problem, we used a Bayesian approach with Gibbs sampling described by Joseph et al. [27] and Messam et al. [28] and undertaken in WinBUGS. We assumed that the number of positive test results (correct identification of mental health diagnostic groupings), TP, is binomially distributed. \({T}_{P}\sim B\left({P}_{C}, n\right)\), where PC is the crude prevalence and \(n\) is the total number in the population (ED presentations in a month). Crude prevalence is related to true prevalence, PT, Se, and Sp through the equation\({P}_{C}={P}_{T}Se+\left(1-{P}_{T}\right)\left(1-Sp\right)\). For example, for a rare true prevalence such as 0.04 if Se and Sp are perfect, 1.0, the crude prevalence will be accurately estimated as 0.04. However, if Se and Sp are 0.95, still good for a diagnostic test, the crude prevalence estimate is 0.09, more than double the true prevalence. Prior estimates of PT, Se, and Sp are used in the analyses and were obtained from MEDREADS results from the test data set. These prior estimates are distributions around point estimates ranging from 0 to 1 and can be represented as beta distributions with hyperparameters a and b. Appropriate a and b hyperparameters were obtained using the beta parameters utility from Epitools [29] based on the mode (mean) and the 95th (PT prior) or 5th (Se and Sp priors) percentile of the PT, Se, and Sp estimates from the test data set. For example, the PT prior estimate for suicide prevalence was based on a mean of 0.029 and its 95th percentile, 0.031, from the test set data. Entering these values into the Epitools beta distribution utility, the hyperparameters a = 828.7 and b = 27,485.7 were obtained. Prior estimates of PT, Se, and Sp were entered into WinBUGS code along with their associated beta distribution hyperparameters, the observed number of positive diagnoses, and total ED presentations for each month for each diagnosis type. The posterior estimate of PT (true prevalence estimate) was obtained as a median and 95% probability interval following the generation of 50,000 random samples. WinBUGS code for each diagnosis type is provided in the Supplementary file 1.

3 Results

Between April 2019 and February 2022, a total of 46,849 presentations across the five diagnostic groups were identified. There was an increase in the overall number of presentations across the five diagnostic groups, from April 2019 (before the COVID-19 pandemic) (N = 599, 7.40%) to February 2022 (during the COVID-19 pandemic) (N = 1202, 7.00%) (refer to Fig. 1). The overall number of cases across the five diagnostic groups peaked in December 2021 (N = 1846, 5.51%). Total numbers of mental health presentations across the five diagnostic groups are provided Table 1. Table 2 gives sensitivity (Se) and specificity (Sp) for these estimates.

Fig. 1
figure 1

Bayesian Estimate (with 95% probability interval) of Presentations Made to Gold Coast EDs Across the Five Diagnostic Groups Between May 2019 and January 2022: Total Number Across All Diagnostic Groups (Note. Data was incomplete for April 2019 and February 2022, and is not displayed in the graph)

Full size image
Table 1 Bayesian Estimate (with 95% probability interval) of Overall Presentations to the Gold Coast Emergency Departments, April 2019—February 2022
Full size table
Table 2 Se and Sp for Mental Health Presentation Estimates
Full size table

3.1 Trends in presentation rates over the COVID-19 pandemic

Whilst there is a general upward trend in the overall number of presentations across all five diagnostic groups, there is evidence of peaks and troughs throughout the pandemic period (Fig. 1). Notable troughs in the overall number of presentations occur in April 2020 (N = 997), April 2021 (N = 1382) and August 2021 (N = 1474). There was a clear peak in the presentation numbers occurring in August 2020 (N = 1710). In the time periods before and during the pandemic, there were recurring peaks at the end and beginning of each year: November 2019 (N = 1217), December 2020 (N = 1542), January 2021 (N = 1603), December 2021 (N = 1846), and January 2022 (N = 1742).

3.2 Presentation rates per diagnosis

Across the study period, suicidality (N = 18,746), substance use (N = 17,809) and psychosis (N = 8,994) were the most prevalent mental health presentations to the ED. Figure 2 displays presentations per diagnostic group (2a—suicidality, 2b—eating disorder, 2c—mania, 2d—psychosis, 2e—substance use). Further tables displaying the numbers of presentation per month for each diagnostic group are provided in Table 3 (suicidality), Table 4 (eating disorder), Table 5 (mania), Table 6 (psychosis), and Table 7 (substance use).

Fig. 2
figure 2

Bayesian Estimate (with 95% probability interval) of Presentations Made to Gold Coast EDs Across the Five Diagnostic Groups Between April 2019 and February 2022 Per Diagnostic Group

Full size image
Table 3 Suicidal Presentations to the Gold Coast Emergency Departments, April 2019—February 2022
Full size table
Table 4 Eating Disorder Presentations to the Gold Coast Emergency Departments, April 2019—February 2022
Full size table
Table 5 Mania Presentations to the Gold Coast Emergency Departments, April 2019—February 2022
Full size table
Table 6 Psychosis Presentations to the Gold Coast Emergency Departments, April 2019—February 2022
Full size table
Table 7 Substance Use Presentations to the Gold Coast Emergency Departments, April 2019—February 2022
Full size table

3.3 Rates of presentations for mental health relative to other emergency department presentations

Figure 3 shows the number of mental health presentations relative to the total number of ED presentations. Relative rates of mental health presentations can be seen for all five diagnostic groups in Table 1, and per diagnostic group in Table 3 (suicidality), Table 4 (eating disorder), Table 5 (mania), Table 6 (psychosis), and Table 7 (substance use). Although the overall number of mental health presentations increased across the study period, the proportion of these presentations, relative to the total number of ED presentations (prevalence), decreased. At the beginning of the study period (April 2019), presentations across the diagnostic groups examined accounted for 7.40%, compared to 7.00% of all ED presentations at the end of the study period (February 2022). This decrease was evident for suicidality (2.96% to 2.85%), mania (0.10% to 0.09%), psychosis (1.41% to 1.32%), and substance use (2.83% to 2.66%). Prevalence of eating disorders remained relatively stable (0.10% to 0.11%).

Fig. 3
figure 3

Percentage of Mental Health Presentations Relative to Total ED Presentations (Note. Percentages for each presentation can be viewed in the Tables 3–7. Data was incomplete for April 2019 and February 2022, and is not displayed in the graph.)

Full size image

4 Discussion

This study analysed the trends that occurred before and during the COVID-19 pandemic in ED presentations across five mental health diagnostic groups. This revealed an increasing trend in the number of mental health presentations across the COVID-19 pandemic, which was consistent across all diagnostic groups examined. Decreases were found earlier in the pandemic, when lockdown restrictions were in place, with increases consistent with the times at which public health restrictions were lifted (refer to timeline in Appendix C). However, the proportion of these presentations, relative to the total number of ED presentations, decreased (7.40% in April 2019 vs. 7.00% Feb 2022).

The COVID-19 pandemic significantly impacted mental health, with disruptions to social interaction and other life events, changes in public health directions, and a high level of psychological distress [1, 30]. Whilst not as restricted as other regions in Australia, the densely population Southeast Queensland, including the Gold Coast was highly affected by lockdowns and other restrictions which were imposed on Queensland. Public health restrictions during this time reduced accessibility to in-person mental health care services, whilst increasing the prevalence of mental health symptoms [31]. It is possible that due to this, more patients were accessing the ED, which remained available during the pandemic, to access mental health support that they would have otherwise received within the community [32].

Both increases and decreases were observed at certain times throughout the pandemic (i.e., decreases in April 2020, April 2021, and August 2021 and an increase in August 2020), which coincided with key COVID-19 pandemic events (timeline provided in Appendix C), with the decreases in cases consistent with more restrictive COVID-19 events (i.e., state lockdowns and restrictions), and increases at times where lockdown and other public health restrictions were lifted. Possibly in months where COVID-19 presentation rates were high, there was an increased perceived risk in attending the ED for mental health care. This risk may be reduced when restrictions were eased, with a perceived increase in safety relating to risk of infection, resulting in an increase in mental health presentations.

Mental health presentations consistently increased in November, December, and January. This was not only during the COVID-19 pandemic, but in 2019 (pre-pandemic), which suggests that the holiday period corresponds to heightened psychological distress. Christmas-related distress has been examined previously, such as by Velamoor et al. [33] who found that the majority expressed negative feelings, citing financial burden, loneliness, and expectations from others as reasons for distress during the Christmas period [33]. The Gold Coast hosts a post-graduation social event for school leavers in in November and December each year (‘Schoolies’) and sees a significant rise in alcohol and substance consumption, incidences of related harm, and is likely to be related to increases in presentations during this time [34].

It was also found that there was a decrease in ED presentations, relative to the total number of ED presentations, for suicidality, mania, psychosis, and substance use in 2022 compared to 2019. Dragovic et al. [4] similarly found a 26% decrease in suicidal presentations to EDs in Western Australia compared to the previous year. Likewise, the relative decrease in mania-related presentations is similar to Seifert et al. [8], who found a decrease in ED presentations related to affective disorders in Germany compared to the previous year.

In contrast, the trends in presentations for the other diagnostic groups (psychosis, substance use, and eating disorders) remains unclear. In Germany, Seifert et al. [8] did not find any differences in the number of presentations for substance use and psychosis in 2019 compared to 2020. In Western Australia, Haripersad et al. [35] found admissions for anorexia nervosa increased during the COVID-19 pandemic, consistent with similar findings in other countries [36]. This contrasts with the findings of our study, which found that there were fewer presentations of eating disorders, whilst the number of relative presentations remained relatively stable, however, this does not necessarily suggest that there is a lower prevalence rate; rather that these presentations are more likely to seek support from services besides the ED.

This work’s secondary contribution is in the MEDREADS algorithm’s description. The MEDREADS framework was used to collect data, including triage notes, to identify mental health presentations. The need for and value of such approaches in Australian EDs has been highlighted in prior research conducted in Queensland and Victoria [20, 21]. Their improvement supports measurement for both research and more direct healthcare strategies. In the case of MEDREADS, three elements may allow for further improvement of the underlying models. First, the models do not allow for interactions where the presence of multiple variables simultaneously may affect risk beyond their individual contributions. Second, the risk scores produced by the algorithms cannot be interpreted other than by comparison to a specific threshold value. As produced by alternative techniques, probabilistic predictions allow for more flexible use of models. Third, several of the variables used in the model are semantically equivalent but can be associated with different risk scores. For example, different risk scores are assigned depending on whether the triage notes contain the string “drown himself” or “drown himself”. Variation in risk scores for semantically equivalent information is undesirable as it decreases model reliability and undermines the confidence of those using the model. Finally, the impact of COVID-19 on mental health presentations to ED is likely to be a result of a number of factors that differ across health services and countries, such as the severity of COVID-19 cases, existing health infrastructure, governance structures, and population characteristics [3].

5 Clinical implications

The main implications of our findings are twofold. Firstly, our study contributes to the understanding of the trends in mental health presentations to the Gold Coast EDs during the COVID-19 pandemic. By using a novel machine learning (ML) approach, we were able to analyse a large volume of mental health data and identify patterns and trends that would have been challenging to uncover using traditional methods (manually coding). This provides important insights into the impact of the pandemic on mental health and helps healthcare professionals and policymakers make informed decisions regarding resource allocation, intervention strategies, and support services, particularly with regards to specific health conditions which may require more specialised care. This work can also be used to inform planning and resource allocation in the event of future major health events, such as future health pandemics.

Secondly, our ML algorithms have demonstrated their effectiveness in analysing mental health data derived from hospital EDs. While other populations may have shown different patterns of mental health presentations, our approach has proven to be a valuable tool in understanding the unique dynamics of the Gold Coast population during the pandemic. The use of ML techniques offers the potential for scalability and transferability to other healthcare settings, enabling similar analyses to be conducted in different regions or during different time periods.

We acknowledge that the findings are specific to the Gold Coast population and the unique circumstances of the COVID-19 pandemic. The Gold Coast is a unique, transient population, with many interstate and international visitors each year. Factors such as demographic characteristics, cultural context, healthcare infrastructure, and pandemic-related restrictions may influence the generalisability of our results to other regions. Therefore, caution should be exercised when extrapolating our findings to different populations. Whilst the findings themselves are unique to the Gold Coast population, the MEDREADS algorithm has potential to be used across other settings, such as other EDs, to examine patterns in mental health presentations across the five diagnostic groupings assessed in this research. Future research could apply the MEDREADS algorithm to other EDs across Australia, to compare such presentations across the same time period. It is also important to consider that the presentations made in this study may not be standalone, rather, it is likely that the same individual may have made multiple presentations across the study period. Assessing re-presentations to the ED for mental health is important, and will be examined in a broader follow-up study.

6 Conclusions

This study examines the trends in mental health presentations made to the Gold Coast EDs during the COVID-19 pandemic, using a novel ML approach. Whilst other populations were in consistent with the rates of presentations within the Gold Coast, the ML algorithms that we have presented are an effective method of analysing mental health data derived from the hospital ED. Future research should consider the potential improvement of the models used to identify relevant presentations. Such improvement could be driven by considering variable interactions, enforcing consistency in risk scores assigned to semantically equivalent text, and assessing alternative, probabilistic models.