External validation of the breast reconstruction risk assessment calculator

https://doi.org/10.1016/j.bjps.2017.04.010Get rights and content

Summary

Introduction

The Breast reconstruction Risk Assessment (BRA) Score estimates patient-specific risk for postsurgical complications using an individual's unique combination of preoperative variables. In this report, we externally validate the BRA Score models for surgical site infection, seroma, and explantation in a large sample of intra-institutional patients who underwent prosthetic breast reconstruction.

Methods

We reviewed all initiated tissue expander/implant reconstructions by the senior authors from January 2004 to December 2015. BRA Score risk estimates were computed for each patient and compared against observed rates of complications. Hosmer–Lemeshow goodness-of-fit test, concordance statistic, and Brier score were used to assess the calibration, discrimination, and accuracy of the models, respectively.

Results

Of the 1152 patients (1743 breasts) reviewed, 855 patients (1333 breasts) had complete data for BRA-score calculations and were included for analysis. Hosmer–Lemeshow tests for calibration demonstrated a good agreement between observed and predicted outcomes for surgical site infection (SSI) and seroma models (P-values of 0.33 and 0.16, respectively). In contrast, predicted rates of explantation deviated from observed rates (Hosmer–Lemeshow P-value of 0.04). C statistics demonstrated good discrimination for SSI, seroma, and explantation (0.73, 0.69, and 0.78, respectively).

Conclusions

In this external validation study, the BRA Score tissue expander/implant reconstruction models performed with generally good calibration, discrimination, and accuracy. Some weaknesses in certain models were identified as targets for future improvement. Taken together, these analyses validate the clinical utility of the BRA score risk models in predicting 30-day outcomes.

Introduction

The number of breast reconstructions performed annually in the United States continues to grow.1, 2, 3, 4, 5, 6 Reasons for this trend include the expanding use of bilateral mastectomies and increasing comfort among plastic surgeons with high-risk cases.1, 4 With this widening pool of patients, selection of operative technique among the myriad of options is becoming ever more challenging.7, 8, 9, 10 Given an individual patient's unique history and physical exam, a number of considerations must be weighed and effectively communicated throughout the shared decision-making process, including aesthetic goals, patient and physician preference, and, importantly, the risk of complications.

Among these complications, seroma, surgical site infection (SSI), and reconstructive failure can be particularly devastating for patients and thus play an important role in patient education.11 Although traditionally the surgical community has relied on coarse, population-derived measures of average risk to inform these conversations,12, 13, 14, 15 there is increasing evidence for the inaccuracy of extrapolating such figures to the individual patients within a heterogeneous population.16, 17, 18, 19

The Breast reconstruction Risk Assessment (BRA) Score risk calculator was developed to improve the understanding of an individual's surgical risk by considering her unique combination of preoperative variables.18, 19 Initially developed using cases from the American College of Surgeons National Surgical Quality Improvement Project (ACS NSQIP) dataset, the calculator has since then expanded to include additional plastic surgery specific data sources.18, 20 Available online, the BRA Score provides measures of individual risk for eight complications and four reconstructive modalities. As its user base grows internationally, the BRA Score must be held to the same rigorous standards of validity as other high-profile prognostic tools, such as the CHA2DS2-VASc Score.21, 22, 23

The models underlying the BRA Score were internally validated using bootstrapping methodologies at the time of their conception. However, external validation is critical to their clinical implementation in a generalized population.24, 25, 26 This is the first report to assess the reproducibility of BRA Score validation in an external cohort of patients. Specifically, we endeavored to externally validate the BRA Score models for SSI, seroma, and explantation in patients undergoing tissue expander/implant-based breast reconstruction by assessing their performance using a large sample of intra-institutional patients. For each model, we assess its discrimination, calibration, and accuracy of predictions and determine both areas of strength and potential weaknesses as we continue to develop and improve the risk calculator.

Section snippets

BRA score risk calculator

The BRA Score Risk Calculator was developed by the senior authors (NK, JYSK), and is available online at www.brascore.org.18, 19 Its back-end computations rely on a set of multivariate logistic regression models that translate preoperative patient traits into probabilities for developing various postoperative complications. The models for tissue expander/implant reconstruction were derived from two national databases: ACS NSQIP and the American Society of Plastic Surgeons Tracking Operations

Primary outcomes

The dataset consisted of 1743 tissue expander placements among 1152 patients in the 12-year period covered by this study. Exclusion of patients with any missing data for the necessary input variables resulted in 855 remaining patients (1333 reconstructed breasts) who were ultimately included for analysis. Median follow-up time was 14.0 months from the placement of the tissue expander.

Demographic data for these patients are displayed in Table 1. Patient age ranged from 22 to 82 years with a mean

Discussion

The advent of nationwide, multicenter patient registries has enabled a paradigm shift in the way that the surgical community approaches patient risk. With demographics, risk factors, and surgical outcomes available for tens of thousands of patients, researchers can develop high-powered, multivariable models predicting the risk of complications with great accuracy. These models provide the foundation for the concept of surgical risk calculators such as the BRA Score. Using data from three

Conclusion

In this external validation study, the BRA Score tissue expander/implant reconstruction models performed with generally good calibration, discrimination, and accuracy for 30-day outcomes. Some weaknesses in certain models were identified as targets for future improvement. Taken together, these analyses validate the clinical utility of the BRA score risk models in predicting acute events following two-stage prosthetic breast reconstruction.

Author contributions

JYSK, NK, and CSQ contributed substantially to project design.

NK, CSQ, and ASM contributed substantially to data analysis.

NK, CSQ, ASM, and MMV contributed substantially to data interpretation.

JYSK, NK, CSQ, ASM, MMV, RGD, and NAF contributed substantially to drafting of the manuscript and final approval of the version to be submitted.

Conflict of interest

The research presented here received no external funding.

Financial disclosures

JYSK receives research funding from the Musculoskeletal Transplant Foundation.

Ethical approval

This study also utilizes data from the National Surgical Quality Improvement Program (NSQIP) and the Tracking Operations and Outcomes for Plastic Surgeons (TOPS) program, which are HIPAA-compliant, de-identified databases available to members of NSQIP and the American Society of Plastic Surgeons, respectively. Intra-institutional data were obtained following the approval by the Northwestern University Institutional Review Board (IRB). A statement from the IRB is available upon request.

References (39)

  • N. Khavanin et al.

    Underlying reasons and timing associated with readmission following plastic surgery: analysis of a national surgical database

    J Plast Reconstr Aesthet Surg

    (2016)
  • S.E. Bleeker et al.

    External validation is necessary in prediction research: a clinical example

    J Clin Epidemiol

    (2003)
  • C.R. Albornoz et al.

    Diminishing relative contraindications for immediate breast reconstruction

    Plast Reconstr Surg

    (2014)
  • T. Hernandez-Boussard et al.

    Breast reconstruction national trends and healthcare implications

    Breast J

    (2013)
  • R. Jagsi et al.

    Trends and variation in use of breast reconstruction in patients with breast cancer undergoing mastectomy in the United States

    J Clin Oncol

    (2014)
  • C.R. Albornoz et al.

    A paradigm shift in U.S. breast reconstruction: increasing implant rates

    Plast Reconstr Surg

    (2013)
  • N. Ilonzo et al.

    Breast reconstruction after mastectomy: a ten-year analysis of trends and immediate postoperative outcomes

    Breast

    (2016)
  • 2015 Reconstructive plastic surgery statistics

    (2015)
  • G.G. Caputo et al.

    Skin-reduction breast reconstructions with prepectoral implant

    Plast Reconstr Surg

    (2016)
  • P. Ciudad et al.

    The combined transverse upper gracilis and profunda artery perforator (TUGPAP) flap for breast reconstruction

    Microsurgery

    (2015)
  • S.W. Jordan et al.

    An algorithmic approach for selective acellular dermal matrix use in immediate two-stage breast reconstruction: indications and outcomes

    Plast Reconstr Surg

    (2014)
  • N. Khavanin et al.

    Tabbed tissue expanders improve breast symmetry scores in breast reconstruction

    Arch Plast Surg

    (2014)
  • S.W. Jordan et al.

    Seroma in prosthetic breast reconstruction

    Plast Reconstr Surg

    (2016)
  • A.S. Mlodinow et al.

    Risk factors for mastectomy flap necrosis following immediate tissue expander breast reconstruction

    J Plast Surg Hand Surg

    (2014)
  • N. Khavanin et al.

    Synergistic interactions with a high intraoperative expander fill volume increase the risk for mastectomy flap necrosis

    J Breast Cancer

    (2013)
  • A.S. Mlodinow et al.

    Increased anaesthesia duration increases venous thromboembolism risk in plastic surgery: a 6-year analysis of over 19,000 cases using the NSQIP dataset

    J Plast Surg Hand Surg

    (2015)
  • I. Chow et al.

    Is there a safe lipoaspirate volume? A risk assessment model of liposuction volume as a function of body mass index

    Plast Reconstr Surg

    (2015)
  • N. Khavanin et al.

    Combining abdominal and cosmetic breast surgery does not increase short-term complication rates: a comparison of each individual procedure and pretreatment risk stratification tool

    Aesthet Surg J

    (2015)
  • J.Y. Kim et al.

    Individualized risk of surgical complications: an application of the breast reconstruction risk assessment score

    Plast Reconstr Surg Glob Open

    (2015)
  • Cited by (0)

    View full text