Skip to main content

Advertisement

SpringerLink
Log in

Development and validation of machine learning models to predict gastrointestinal leak and venous thromboembolism after weight loss surgery: an analysis of the MBSAQIP database

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Postoperative gastrointestinal leak and venous thromboembolism (VTE) are devastating complications of bariatric surgery. The performance of currently available predictive models for these complications remains wanting, while machine learning has shown promise to improve on traditional modeling approaches. The purpose of this study was to compare the ability of two machine learning strategies, artificial neural networks (ANNs), and gradient boosting machines (XGBs) to conventional models using logistic regression (LR) in predicting leak and VTE after bariatric surgery.

Methods

ANN, XGB, and LR prediction models for leak and VTE among adults undergoing initial elective weight loss surgery were trained and validated using preoperative data from 2015 to 2017 from Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program database. Data were randomly split into training, validation, and testing populations. Model performance was measured by the area under the receiver operating characteristic curve (AUC) on the testing data for each model.

Results

The study cohort contained 436,807 patients. The incidences of leak and VTE were 0.70% and 0.46%. ANN (AUC 0.75, 95% CI 0.73–0.78) was the best-performing model for predicting leak, followed by XGB (AUC 0.70, 95% CI 0.68–0.72) and then LR (AUC 0.63, 95% CI 0.61–0.65, p < 0.001 for all comparisons). In detecting VTE, ANN, and XGB, LR achieved similar AUCs of 0.65 (95% CI 0.63–0.68), 0.67 (95% CI 0.64–0.70), and 0.64 (95% CI 0.61–0.66), respectively; the performance difference between XGB and LR was statistically significant (p = 0.001).

Conclusions

ANN and XGB outperformed traditional LR in predicting leak. These results suggest that ML has the potential to improve risk stratification for bariatric surgery, especially as techniques to extract more granular data from medical records improve. Further studies investigating the merits of machine learning to improve patient selection and risk management in bariatric surgery are warranted.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Nguyen NT, Varela JE (2017) Bariatric surgery for obesity and metabolic disorders: state of the art. Nat Rev Gastroenterol Hepatol 14:160

    Article  Google Scholar 

  2. Böckelman C, Hahl T, Victorzon M (2017) Mortality following bariatric surgery compared to other common operations in Finland during a 5-year period (2009–2013). A nationwide registry study. Obes Surg 27:2444–2451. https://doi.org/10.1007/s11695-017-2664-z

    Article  PubMed  Google Scholar 

  3. Fry BT, Scally CP, Thumma JR, Dimick JB (2018) Quality improvement in bariatric surgery: the impact of reducing postoperative complications on medicare payments. Ann Surg 268:22–27

    Article  Google Scholar 

  4. Funk LM, Jolles S, Fischer LE, Voils CI (2015) Patient and referring practitioner characteristics associated with the likelihood of undergoing bariatric surgery: a systematic review. JAMA Surg 150:999–1005

    Article  Google Scholar 

  5. Funk LM, Jolles SA, Greenberg CC, Schwarze ML, Safdar N, McVay MA, Whittle JC, Maciejewski ML, Voils CI (2016) Primary care physician decision making regarding severe obesity treatment and bariatric surgery: a qualitative study. Surg Obes Relat Dis 12:893–901

    Article  Google Scholar 

  6. Vidal J, Corcelles R, Jiménez A, Flores L, Lacy AM (2017) Metabolic and bariatric surgery for obesity. Gastroenterology 152:1780–1790

    Article  Google Scholar 

  7. Alizadeh RF, Li S, Inaba C, Penalosa P, Hinojosa MW, Smith BR, Stamos MJ, Nguyen NT (2018) Risk factors for gastrointestinal leak after bariatric surgery: MBASQIP analysis. J Am Coll Surg 227:135–141. https://doi.org/10.1016/j.jamcollsurg.2018.03.030

    Article  PubMed  Google Scholar 

  8. Mocanu V, Dang J, Ladak F, Switzer N, Birch DW, Karmali S (2019) Predictors and outcomes of leak after Roux-en-Y Gastric Bypass: an analysis of the MBSAQIP data registry. Surg Obes Relat Dis 15:396–403

    Article  Google Scholar 

  9. Turrentine FE, Denlinger CE, Simpson VB, Garwood RA, Guerlain S, Agrawal A, Friel CM, LaPar DJ, Stukenborg GJ, Jones RS (2015) Morbidity, mortality, cost, and survival estimates of gastrointestinal anastomotic leaks. J Am Coll Surg 220:195–206

    Article  Google Scholar 

  10. Ward-Smith P (2012) Body mass index, surgery, and risk of venous thromboembolism in middle-aged women. Urol Nurs 32:220–223

    Article  Google Scholar 

  11. Klovaite J, Benn M, Nordestgaard BG (2015) Obesity as a causal risk factor for deep venous thrombosis: a M endelian randomization study. J Intern Med 277:573–584

    Article  CAS  Google Scholar 

  12. ASMBS Clinical Issues Committee (2013) ASMBS updated position statement on prophylactic measures to reduce the risk of venous thromboembolism in bariatric surgery patients. Surg Obes Relat Dis 9(4):493–497. https://doi.org/10.1016/j.soard.2013.03.006

    Article  Google Scholar 

  13. Aminian A, Andalib A, Khorgami Z, Cetin D, Burguera B, Bartholomew J, Brethauer SA, Schauer PR (2017) Who should get extended thromboprophylaxis after bariatric surgery. Ann Surg 265:143–150

    Article  Google Scholar 

  14. Dang JT, Switzer N, Delisle M, Laffin M, Gill R, Birch DW, Karmali S (2018) Predicting venous thromboembolism following laparoscopic bariatric surgery: development of the BariClot tool using the MBSAQIP database. Surg Endosc. https://doi.org/10.1007/s00464-018-6348-0

    Article  PubMed  Google Scholar 

  15. Gaborit B, Aron-Wisnewsky J, Salem J-E, Bege T, Frere C (2018) Pharmacologic venous thromboprophylaxis after bariatric surgery. Ann Surg 268:e51–e52

    Article  Google Scholar 

  16. Thereaux J, Lesuffleur T, Czernichow S, Basdevant A, Msika S, Nocca D, Millat B, Fagot-Campagna A (2018) To what extent does posthospital discharge chemoprophylaxis prevent venous thromboembolism after bariatric Surgery? Results from a nationwide cohort of more than 110,000 patients. Ann Surg 267:727–733

    Article  Google Scholar 

  17. Kumar SB, Hamilton BC, Wood SG, Rogers SJ, Carter JT, Lin MY (2018) Is laparoscopic sleeve gastrectomy safer than laparoscopic gastric bypass? A comparison of 30-day complications using the MBSAQIP data registry. Surg Obes Relat Dis 14:264–269. https://doi.org/10.1016/j.soard.2017.12.011

    Article  PubMed  Google Scholar 

  18. Bahl V, Hu HM, Henke PK, Wakefield TW, Campbell DA, Caprini JA (2010) A validation study of a retrospective venous thromboembolism risk scoring method. Ann Surg 251:344–350. https://doi.org/10.1097/SLA.0b013e3181b7fca6

    Article  PubMed  Google Scholar 

  19. Finks JF, English WJ, Carlin AM, Krause KR, Share DA, Banerjee M, Birkmeyer JD, Birkmeyer NJ, Collaborative MBS (2012) Predicting risk for venous thromboembolism with bariatric surgery: results from the Michigan Bariatric Surgery Collaborative. Ann Surg 255:1100–1104

    Article  Google Scholar 

  20. Hashimoto DA, Rosman G, Rus D, Meireles OR (2018) Artificial intelligence in surgery: promises and perils. Ann Surg 268:70–76

    Article  Google Scholar 

  21. Kim JS, Merrill RK, Arvind V, Kaji D, Pasik SD, Nwachukwu CC, Vargas L, Osman NS, Oermann EK, Caridi JM (2018) Examining the ability of artificial neural networks machine learning models to accurately predict complications following posterior lumbar spine fusion. Spine 43:853–860

    Article  Google Scholar 

  22. Bertsimas D, Dunn J, Velmahos GC, Kaafarani HMA (2018) Surgical risk is not linear: derivation and validation of a novel, user-friendly, and machine-learning-based predictive optimal trees in emergency surgery risk (POTTER) Calculator. Ann Surg 268:574–583. https://doi.org/10.1097/SLA.0000000000002956

    Article  PubMed  Google Scholar 

  23. Goto T, Camargo CA, Faridi MK, Freishtat RJ, Hasegawa K (2019) Machine learning-based prediction of clinical outcomes for children during emergency department triage. JAMA Netw Open 2:e186937. https://doi.org/10.1001/jamanetworkopen.2018.6937

    Article  PubMed  PubMed Central  Google Scholar 

  24. Wong A, Young AT, Liang AS, Gonzales R, Douglas VC, Hadley D (2018) Development and validation of an electronic health record–based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment. JAMA Netw Open 1:e181018. https://doi.org/10.1001/jamanetworkopen.2018.1018

    Article  PubMed  PubMed Central  Google Scholar 

  25. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539

    Article  CAS  Google Scholar 

  26. Chen T, Guestrin C (2016) Xgboost: a scalable tree boosting system. In: Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. pp 785–794;

  27. MBSAQIP. MBSAQIP participant use data file. https://www.facs.org/quality-programs/mbsaqip/participant-use. Accessed 14 Jan 2019

  28. Pavlou M, Ambler G, Seaman SR, Guttmann O, Elliott P, King M, Omar RZ (2015) How to develop a more accurate risk prediction model when there are few events. BMJ 351:h3868

    Article  Google Scholar 

  29. Lemaître G, Nogueira F, Aridas CK (2017) Imbalanced-learn: a python toolbox to tackle the curse of imbalanced datasets in machine learning. J Mach Learn Res 18:559–563

    Google Scholar 

  30. Millman KJ, Aivazis M (2011) Python for scientists and engineers. Comput Sci Eng 13:9–12

    Article  Google Scholar 

  31. Oliphant TE (2007) Python for scientific computing. Comput Sci Eng 9:10–20

    Article  CAS  Google Scholar 

  32. Anaconda Software Distribution. Computer software. Vers. 2-2.4.0. Anaconda, Nov. 2016. Web. https://anaconda.com.

  33. McKinney W (2010) Data structures for statistical computing in python. In: Proceedings of the 9th Python in Science Conference, vol 445. pp 51–56

  34. Oliphant TE (2006) A guide to NumPy. Trelgol Publishing USA

  35. Collins GS, Reitsma JB, Altman DG, Moons KGM (2015) Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 162:55. https://doi.org/10.7326/m14-0697

    Article  PubMed  Google Scholar 

  36. Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in Pytorch.

  37. Howard JAO (2018) Fastai. GitHub. https://github.com/fastai/fastai.

  38. Seth Y (2018) A neural network in PyTorch for tabular data with categorical embeddings. Let the machines learn. https://yashuseth.blog/2018/07/22/pytorch-neural-network-for-tabular-data-with-categorical-embeddings/.

  39. Ng A, Katanforoosh K. CS230 Deep learning course notes and code examples

  40. Guo C, Berkhahn F (2016) Entity embeddings of categorical variables. arXiv preprint. arXiv: 160406737

  41. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv

  42. Caruana R, Lawrence S, Giles CL (2001) Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. Advances in neural information processing systems. MIT Press, Cambridge, pp 402–408

    Google Scholar 

  43. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15:1929–1958

    Google Scholar 

  44. Seabold S, Perktold J (2010) Statsmodels: Econometric and statistical modeling with python. In: Proceedings of the 9th Python in Science Conference, vol 57. p 61

  45. DeLong ER, DeLong DM, Clarke-Pearson DL (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 44:837–845

    Article  CAS  Google Scholar 

  46. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Müller M (2011) pROC: an open-source package for R and S+ to analyze and compare ROC curves. BMC Bioinf 12:77

    Article  Google Scholar 

  47. Team RS (2015) RStudio: integrated development for R. RStudio Inc., Boston, p 42

    Google Scholar 

  48. Team RC (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna

    Google Scholar 

  49. Wickham H (2016) ggplot2: elegant graphics for data analysis. Springer, New York

    Book  Google Scholar 

  50. Pollard TJ, Johnson AEW, Raffa JD, Mark RG (2018) Tableone: an open source Python package for producing summary statistics for research papers. JAMIA Open 1:26–31. https://doi.org/10.1093/jamiaopen/ooy012

    Article  PubMed  PubMed Central  Google Scholar 

  51. Frizzell JD, Liang L, Schulte PJ, Yancy CW, Heidenreich PA, Hernandez AF, Bhatt DL, Fonarow GC, Laskey WK (2017) Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: comparison of machine learning and other statistical approaches. JAMA Cardiol 2:204–209. https://doi.org/10.1001/jamacardio.2016.3956

    Article  PubMed  Google Scholar 

  52. Johnson KW, Soto JT, Glicksberg BS, Shameer K, Miotto R, Ali M, Ashley E, Dudley JT (2018) Artificial intelligence in cardiology. J Am Coll Cardiol 71:2668–2679

    Article  Google Scholar 

  53. Golas SB, Shibahara T, Agboola S, Otaki H, Sato J, Nakae T, Hisamitsu T, Kojima G, Felsted J, Kakarmath S (2018) A machine learning model to predict the risk of 30-day readmissions in patients with heart failure: a retrospective analysis of electronic medical records data. BMC Med Inform Decis Mak 18:44

    Article  Google Scholar 

  54. Telem DA, Yang J, Altieri M, Patterson W, Peoples B, Chen H, Talamini M, Pryor AD (2016) Rates and risk factors for unplanned emergency department utilization and hospital readmission following bariatric surgery. Ann Surg. 263:956–960. https://doi.org/10.1097/SLA.0000000000001536

    Article  PubMed  Google Scholar 

  55. Maier-Hein L, Vedula SS, Speidel S, Navab N, Kikinis R, Park A, Eisenmann M, Feussner H, Forestier G, Giannarou S (2017) Surgical data science for next-generation interventions. Nat Biomed Eng 1:691

    Article  Google Scholar 

  56. Lee G, Rubinfeld I, Syed Z (2012) Adapting surgical models to individual hospitals using transfer learning. In: proceedings of the 2012 IEEE 12th International Conference on Data Mining Workshops, pp 57–63

  57. Bihorac A, Ozrazgat-Baslanti T, Ebadi A, Motaei A, Madkour M, Pardalos PM, Lipori G, Hogan WR, Efron PA, Moore F (2019) MySurgeryRisk: development and validation of a machine-learning risk algorithm for major complications and death after surgery. Ann Surg 269:652–662

    Article  Google Scholar 

  58. Natekin A, Knoll A (2013) Gradient boosting machines, a tutorial. Front Neurorobot 7:21

    Article  Google Scholar 

  59. Masoomi H, Kim H, Reavis KM, Mills S, Stamos MJ, Nguyen NT (2011) Analysis of factors predictive of gastrointestinal tract leak in laparoscopic and open gastric bypass. Arch Surg 146:1048–1051. https://doi.org/10.1001/archsurg.2011.203

    Article  PubMed  Google Scholar 

  60. Qu Y, Cai H, Ren K, Zhang W, Yu Y, Wen Y, Wang J (2016) Product-based neural networks for user response prediction. In: Proceedings of the 2016 IEEE 16th International Conference on Data Mining. pp 1149–1154

  61. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2013) Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems. MIT Press, Cambridge, pp 3111–3119

    Google Scholar 

  62. Covington P, Adams J, Sargin E (2016) Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM conference on recommender systems. pp 191–198

Download references

Acknowledgements

The Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program (MBSAQIP) the hospitals participating in the MBSAQIP are the source of the data used herein; they have not verified and are not responsible for the statistical validity of the data analysis or the conclusions derived by the authors.

Funding

Supported by the Boston University Institute for Health System Innovation and Policy and the Boston Medical Center Department of Surgery. On a separate project, Dr. Bishara was supported by a National Institute of General Medical Sciences training grant (T32 GM008440, PI: Dexter Hadley).

Author information

Authors and Affiliations

  1. Department of Surgery, Boston University School of Medicine, Boston, MA, USA

    Jacob Nudel, Susanna W. L. de Geus & Donald T. Hess

  2. Institute for Health System Innovation and Policy, Boston University, 601, 656 Beacon Street, Boston, MA, 02215, USA

    Jacob Nudel, Jayakanth Srinivasan & Jonathan Woodson

  3. Department of Anesthesia, University of California, San Francisco, San Francisco, CA, USA

    Andrew M. Bishara

  4. Bakar Computational Health Sciences Institute, University of California San Francisco, San Francisco, CA, USA

    Andrew M. Bishara

  5. Department of Biostatistics, Boston University School of Public Health, Boston, MA, USA

    Prasad Patil

Authors
  1. Jacob Nudel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrew M. Bishara
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Susanna W. L. de Geus
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Prasad Patil
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jayakanth Srinivasan
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Donald T. Hess
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jonathan Woodson
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jonathan Woodson.

Ethics declarations

Disclosures

Drs. Nudel and Bishara are co-founders of Bezel Health, a company building software to measure and improve healthcare quality interventions. Drs. Woodson, De Geus, Srinivasan, Patil, and Hess have no conflicts of interest or financial ties to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 7569 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nudel, J., Bishara, A.M., de Geus, S.W.L. et al. Development and validation of machine learning models to predict gastrointestinal leak and venous thromboembolism after weight loss surgery: an analysis of the MBSAQIP database. Surg Endosc 35, 182–191 (2021). https://doi.org/10.1007/s00464-020-07378-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-020-07378-x

Keywords

Search

Navigation