Introduction

Early-onset scoliosis (EOS) is a complex and heterogenous condition, with varying etiologies and genetic associations. Certain cases of EOS can lead to substantial spine and chest deformity, pain, and disability if left untreated [1]. Historical treatment of EOS consisted of early spinal fusion or external bracing. Currently, growth-friendly techniques, including distraction-based, compression-based, and growth-guidance, have become prevalent and popular [2]. Both spinal fusion and growth-friendly instrumentation have been associated with a significant risk of perioperative and postoperative complications [3, 4].

Reporting complications following surgical procedures is essential to properly assess the risk of a poor outcome. However, reporting this information needs to be done in an objective, accurate, and reliable manner, which requires the creation of a standardized complication classification system. Classification systems are widely utilized as they assist in guiding management, augmenting communication, and facilitating future research studies by providing a common language that can be used [5]. For instance, classification systems provide a common language for providers to describe various pathologies and outcomes. They also allow for the comparison of management strategies and outcomes in an objective and simple manner. Research gaps exist with regards to a granular classification system for use in EOS surgical complications. In 2014, Williams et al. developed a classification system for EOS, which has been utilized to standardize communication in the clinical setting by simplifying the complexities of the pathology based on etiology, major curve angle, and kyphosis [6]. In the following year, Smith et al. established a classification system to report complications following growth-friendly spinal instrumentation in EOS patients [7]. While the Smith classification system has proven useful, there is a need for a more detailed complication classification system to use in research studies and to compare complications of similar magnitude. The Clavien–Dindo system is a widely used, treatment-based classification of surgical complications initially developed for general surgery [8]. Sink et al. modified the Clavien–Dindo system, now known as the Clavien–Dindo-Sink (CDS) system, and validated its use in orthopedic surgery, specifically in surgical hip patients [9]. This system classifies complications based on its potential for long-term morbidity and the magnitude of the treatment required to manage the complication. The CDS system is also currently being used for adolescent idiopathic scoliosis surgery [10]. However, the use of the CDS system has not been studied for EOS.

The absence of a reliable complication classification system for EOS has contributed to difficulty in establishing consistent management strategies and also in interpreting outcome studies. Given the importance of appropriately measuring and classifying surgical complications in children with EOS, the purpose of this study was twofold: (1) to evaluate the CDS system of complication classification in the EOS population, and (2) to assess the inter-rater and intra-rater reliability of the modified CDS system among a group of pediatric spinal deformity surgeons. It was hypothesized that the modified CDS system is a reliable method for classifying surgical complications in EOS patients.

Materials and methods

Study participants

A panel of surgeons experienced in the treatment of pediatric patients with spinal deformity was chosen for this study. Surgeons were chosen based on participation in an international spine study group. Twelve surgeons from twelve institutions were invited to participate in the study and no compensation was provided.

Study design

Thirty case scenarios were developed based on actual complication data captured in the Pediatric Spine Study Group (PSSG) registry. Complication categories assessed included gastrointestinal, pulmonary, wound, implant-related, neurologic, and intraoperative complications. Pain, revision, mortality, and other medical complications were also assessed. The CDS classification system was initially evaluated and then modified based on expert consensus for use in surgically treated EOS patients. A group of six surgeons initially classified 100 actual cases with the CDS system to evaluate its utility in this patient population. A review of these cases led to the recognition that many of the complications were medical in nature and thus should be captured as different from surgical complications. As such, in the modified CDS system, Grade III classification on the CDS system was divided into Grade IIIa and Grade IIIb to highlight the difference in unplanned hospital readmission (Grade IIIa) and unplanned surgical, endoscopic, or interventional radiology procedure (Grade IIIb), respectively (Fig. 1). For each case, the surgeon participants were asked to select the level of complication severity based on the modified CDS system. The survey was administered at two separate time points to assess for inter-observer and intra-observer reliability. The surveys were performed independently and were blinded using a web-based surveying platform. The surgeons were also provided with a summary sheet of the modified CDS classification system, which could be viewed during each survey. This study was approved by the primary institution’s Institutional Review Board (IRB).

Fig. 1
figure 1

Modified Clavien–Dindo-Sink system developed for EOS patients

Full size image

Study procedures

Utilizing the 30 case scenarios and a web-based surveying platform (Qualtrics, Provo, UT), the coordinating center administered the survey twice, two weeks apart, to be completed independently by the participating surgeons. In order to calculate intra-observer reliability, the same survey was administered at both time points. Each case scenario included the patient’s age and gender, type of surgery, complication presentation and course, and treatment. Examples of the user interface typical of each case scenario are shown in Figs. 2, 3, 4 and 5. At no point during the study were any of the participating surgeons given personal feedback or made aware of one another’s responses.

Fig. 2
figure 2

User interface of surveys 1 and 2 showing example 1 of a case scenario presented to participating surgeons, as created by Qualtrics

Full size image
Fig. 3
figure 3

User interface of surveys 1 and 2 showing example 2 of a case scenario presented to participating surgeons, as created by Qualtrics

Full size image
Fig. 4
figure 4

User interface of surveys 1 and 2 showing example 3 of a case scenario presented to participating surgeons, as created by Qualtrics

Full size image
Fig. 5
figure 5

User interface of surveys 1 and 2 showing example 4 of a case scenario presented to participating surgeons, as created by Qualtrics

Full size image

Statistical analysis

Descriptive data analyses were performed to evaluate the participating surgeons’ experience with EOS cases. Weighted kappa values, along with 95% confidence intervals (CIs), were calculated between surgeons for each survey to assess the degree of agreement between the observers (inter-observer reliability). Weighted kappa values were also calculated within surgeons between the first to the second survey to assess for intra-observer reliability. Weighted kappa values were utilized for this study as it takes into consideration that the differences between the severity levels of the CDS system (e.g., between I and II versus between I and IVa) may not be of equal importance [11]. It is accepted that a kappa value of less than 0.20 indicates poor agreement, 0.21 to 0.40 indicates fair agreement, 0.41 to 0.60 indicates moderate agreement, 0.61 to 0.80 indicates substantial agreement, and 0.81 to 1.00 indicates nearly perfect agreement [12]. IBM SPSS Statistics, Version 28 (Armonk, NY) was used for all analyses in this study.

Source of funding

There were no external sources of funding for this study.

Results

Participants

Of the 12 surgeons invited to participate in the study, 11 (91.7%) surgeons completed the first survey and 8 (66.7%) surgeons completed the second. The surgeons had an average of 12.9 ± 8.3 years (range, 3 to 32 years) in practice. The annual EOS surgical case volume for the majority of surgeons (46%) was between 20 and 29 cases. 27% of the surgeons had an annual case volume between 10 and 19 cases, and 27% had an annual case volume greater than 30 cases. All participants were fellowship-trained, board-certified pediatric orthopedic surgeons with active membership in the PSSG. A summary of the frequency of each severity level among the observers’ cumulative responses is shown in Table 1.

Table 1 Frequency of each grade among cumulative surgeon responses (N = 570)
Full size table

Inter-observer reliability of the modified CDS classification system

The weighted kappa value for the first survey was 0.75, indicating substantial agreement among observers. For the second survey, the weighted kappa value was 0.84, indicating nearly perfect agreement. Table 2 shows the inter-observer weighted kappa values and confidence intervals for both surveys.

Table 2 Inter-Observer reliability of the modified CDS classification system
Full size table

Intra-observer reliability of the modified CDS classification system

The average weighted kappa between the first and second surveys was 0.86 (range, 0.74 to 0.95), indicating nearly perfect agreement within participants. Table 3 shows the intra-observer weighted kappa values, along with confidence intervals, for all participants.

Table 3 Intra-observer reliability of the modified CDS classification system between surveys 1 and 2
Full size table

Discussion

Classification systems are an essential medical tool to provide a common language for providers to describe various pathologies and outcomes. Being able to simplify communication among surgeons can allow for the comparison of management strategies and outcomes in an objective manner [5]. This study aimed to assess the reliability of the modified CDS system for use in EOS patients. The modified CDS system demonstrated substantial to nearly perfect agreement between and within observers for the evaluation of complications following EOS surgery. This study used weighted kappa values for the assessment of agreement among observers, as it takes into consideration the relative distance between two observations. This is important when utilizing the modified CDS system, as distinguishing between very severe and mild complications will be beneficial to the treating surgeons.

The detrimental impact EOS has on health-related quality of life and pulmonary development has been well-documented in the literature [13, 14]. Compared to other pediatric spine deformities such as adolescent idiopathic scoliosis (AIS), EOS treatment has a higher complication rate. Given the frequency of complications, it is important to establish a standardized assessment of the complications and the severity of impact to the patients [3, 15]. This led to the classification scheme developed by Smith et al., which provides information on how complications can alter a patient’s overall course of care [7]. In an investigation by Michael et al., the Smith classification was shown to have strong inter-rater and intra-rater agreement [16]. However, this classification system has its limitations. Although it divides complications into device- or disease-related, it does not go into the specifics of the complication. For instance, it stratifies disease-related complications into requiring outpatient or inpatient medical management, but does not convey any information as to whether the complication has clinical relevance, and whether it is a treatable complication. The modified CDS classification system addresses these limitations as it provides further details as to the comprehensive impact of the complication on the patient. The modified CDS allows providers to specify the severity of the complication and the magnitude of treatment required, such as if the patient will require admission to the intensive care unit. Also, the CDS classification accounts for a complication’s long-term morbidity. The detailed CDS classification system can allow for clear dialog among surgeons, while also conveying the specifics of the complication.

The Clavien system has become popular in the surgical literature as one of the preferred methods to grade the severity level of complications, due to its consistent and precise boundaries [8]. In a systematic review of 11,481 surgical articles, Yoon et al. reported the Clavien system to be the most frequently used instrument for the standardized reporting of complications [17]. The Clavien–Dindo classification was initially adapted for use in general surgery [18]. In recent years, it has been further utilized in other surgical subspecialties, including urology, nephrology, hepatology, and gastroenterology [19,20,21,22]. Sink et al. modified this classification, currently known as the Clavien–Dindo-Sink system, for use in hip preservation surgery, and demonstrated high inter- and intra-observer reliabilities for grading of complications [9]. This system has also been applied to total knee arthroplasty [23]. A study by Dodwell et al. tested the CDS system both internally and externally for use in pediatric orthopedic case scenarios, specifically surgeries involving the upper and lower extremity, and they found good to excellent agreement among the surgeons [24]. Similarly, Zhou et al. demonstrated strong reliability when they used the modified Clavien–Dindo system to evaluate complications following lower limb surgery in patients with cerebral palsy [25]. All studies discouraged the use of the terms minor and major when describing complications as these terms are inherently subjective and provide no information regarding how the complication affects a patient’s management. As shown in our study’s findings, the reliability values for the modified CDS classification system we developed for EOS are comparable with these other studies. Furthermore, having a CDS system for EOS surgery allows clinicians to place the risks of these surgeries in greater context with other pediatric surgeries using this classification system, such as AIS or upper or lower extremity surgery, or even non-pediatric orthopedic surgeries.

Guissé et al. recently demonstrated that the modified CDS system has very good inter-rater (kappa value of 0.8) and intra-rater (kappa value between 0.6 and 0.9) reliability following surgery for AIS [10]. The Clavien–Dindo system has also been used in adult spine surgery, specifically in patients undergoing cervical fusion [26]. In 2016, Rampersaud et al. illustrated strong agreement when utilizing the Spinal Adverse Events Severity System, version 2, to assess complications in spinal surgery [27]. In 2020, Klineberg et al. developed a new comprehensive classification system for capturing complications in adult spinal deformity and demonstrated good accuracy and repeatability among surgeons and research coordinators [28]. These studies all demonstrate the importance of creating a granular classification system to report surgical complications.

In the present study, we propose a modified CDS classification system to report complications in EOS patients that retains both high inter- and intra-rater reliability among surgeons with a wide range in years of experience and number of EOS cases performed per year. Although the CDS system has been modified previously, our modification to Grade III complications by separating “unplanned hospital readmission” (Grade IIIa) and “unplanned surgical, endoscopic, or interventional radiology procedure” (Grade IIIb) has not been explicitly described previously. While many recent studies describe “unplanned return to the operating room” as a significant complication endpoint, unplanned hospital admission can be as significant and is now able to be captured with this system. The strong level of agreement supports the use of the modified CDS system to distinguish between varying severity levels for treatment decision-making and for elevating the level of research in the field by allowing for easier interpretation of the literature. Given the current focus on value-based care and increasing incidence of growth-friendly surgery for EOS patients, this classification system will serve an important role when evaluating the impact of complications and interventions on costs and outcomes [2, 28]. This classification system can help to clarify the discussion of complication rates by bundling the complications. For instance, complications that are classified as CDS Grade 1 requires minimal or no treatment and can therefore be bundled together, which would indicate to other providers that these complications have minimal clinical relevance.

There are several limitations to this study. The classification system was developed based on expert opinion and was not subjected to a formal consensus-building method such as the Delphi technique. Also, inexperience using this modified classification system in the EOS population likely accounts for agreement being less than 100%. However, we were able to find near perfect reliability and we expect that continued use of this classification scheme would be associated with improved agreement over time, perhaps highlighted by an increase in inter-rater reliability during our second-round survey. Another limitation is that certain complications may change with time, therefore their grade would change to reflect the appropriate level of treatment. The modified CDS does not take into account a potential change in grade over time. In addition, institutions may vary with regards to how they manage complications, which can result in discrepancies in grading a specific complication. Further, the case scenarios in this study described only one complication per scenario. In reality, EOS patients may experience many complications following their procedure, which would result in the potential of different grades for each complication. Also, EOS patients have many medical comorbidities at baseline, therefore it may be difficult to denote which complications are due to the underlying disease state compared to those which are specifically related to the surgery. Despite these limitations, our study demonstrated that the modified CDS system is a reliable tool, and it can be applicable to future outcomes analyses of EOS surgery as it provides long-term morbidity information and the magnitude of the treatment required for the complication, which is important to both the hospital and the patient.

In conclusion, the modified Clavien–Dindo-Sink system is a straightforward tool that can be used to classify complications in surgically treated EOS patients, and it demonstrates substantial to nearly perfect agreement between and within observers. With the current emphasis on increasing quality, safety and value in patient care, adoption of this reliable classification system as a standard for reporting complications in EOS patients can help to interpret outcome studies and structure future clinical research to ultimately improve patient outcomes and satisfaction.