https://orcid.org/0000-0002-7078-7780
Figures
Abstract
Radiofrequency ablation (RFA) using the CARTO 3D mapping system is a common approach for pulmonary vein isolation to treat atrial fibrillation (AF). Linkage between CARTO procedural data and patients’ electronical health records (EHR) provides an opportunity to identify the ablation-related parameters that would predict AF recurrence. The objective of this study is to assess the incremental accuracy of RFA procedural data to predict post-ablation AF recurrence using machine learning model. Procedural data generated during RFA procedure were downloaded from CARTONET and linked to deidentified Mercy Health EHR data. Data were divided into train (70%) and test (30%) data for model development and validation. Automate machine learning (AutoML) was used to predict 1 year AF recurrence, defined as a composite of repeat ablation, electrical cardioversion, and AF hospitalization. At first, AutoML model only included Patients’ demographic and clinical characteristics. Second, an AutoML model with procedural variables and demographical/clinical variables was developed. Area under receiver operating characteristic curve (AUROC) and net reclassification improvement (NRI) were used to compare model performances using test data. Among 306 patients, 67 (21.9%) patients experienced 1-year AF recurrence. AUROC increased from 0.66 to 0.78 after adding procedural data in the AutoML model based on test data. For patients with AF recurrence, NRI was 32% for model with procedural data. Nine of 10 important predictive features were CARTO procedural data. From CARTO procedural data, patients with lower contact force in right inferior site, long ablation duration, and low number of left inferior and right roof lesions had a higher risk of AF recurrence. Patients with persistent AF were more likely to have AF recurrence. The machine learning model with procedural data better predicted 1-year AF recurrence than the model without procedural data. The model could be used for identification of patients with high risk of AF recurrence post ablation.
Citation: Peng M, Doshi A, Amos Y, Tsoref L, Amit M, Yungher D, et al. (2024) Does radiofrequency ablation procedural data improve the accuracy of identifying atrial fibrillation recurrence? PLoS ONE 19(4): e0300309. https://doi.org/10.1371/journal.pone.0300309
Editor: Ibrahim Marai, Baruch Padeh Medical Center Poriya, ISRAEL
Received: September 28, 2023; Accepted: February 27, 2024; Published: April 5, 2024
Copyright: © 2024 Peng et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data used for this study comes from Mercy Health data. Mercy retains all right and interest of the data. Our data use agreement does not allow us to share the data without the permission from Mercy Health, Kim Collison Farr (Kimberly.CollisonFarr@Mercy.Net).
Funding: This study is supported by Johnson and Johnson, New Brunswick, NJ, USA without specific grant number.
Competing interests: MP, YA, LT, MA, DY, RK, and PC are Johnson & Johnson employees. AD has no conflicts of interest to declare.
Introduction
Atrial fibrillation is the most common arrhythmia diagnosed in the clinical practice and is associated with increased risk of stroke, heart failure, dementia, and overall mortality [1]. Catheter ablation has been established as an effective treatment for paroxysmal and persistent AF. However, AF recurrence still occurs for around 20% to 40% of patients post ablation [2].
During ablation procedures, circumferential lesion sets around right and left pulmonary veins and additional ablation lesion lines are created to achieve the isolation of AF triggering PV foci, elimination of non-PV triggering foci, and/or as the result of modification of the arrhythmogenic substrate. Ablation factors such as catheter tip-tissue contact, power, time of ablation, catheter stability, inter-lesion distance affect lesion quality, leading to varying rates of AF recurrence [3]. Durable pulmonary vein isolation (PVI) is associated with a lower risk of AF recurrence based on meta-analysis [4]. Ablation index is a composite index of contact force, ablation time, and power. Ablation index guided ablation strategy has been shown to achieve high rate of durable PVI and freedom from arrhythmia in persistent AF patients [5]. Collection of procedural data related to lesion formation during the ablation process could improve understanding and support achievement of ablation success rates.
In this study, we linked the ablation procedural data from CARTO systems with health system EHR data. Machine learning model was developed to assess the incremental accuracy of procedural data in identifying patients with high risk of AF recurrence.
Method
Data sources
This study used the linked EHR data collected using EPIC software with procedural data collected from CARTO systems (Biosense Webster Inc, Irvine, CA) and stored in CARTONET cloud-based platform system during RFA procedure for AF patients in Mercy healthcare system. All CARTONET data were scrubbed of protected health information and transmitted via the Siemens teamplay platform (Siemens, Malvern, PA) to the Microsoft Azure cloud (Microsoft, Redmond, WA). Anonymized procedural data were processed using the CARTONET cloud-based analytical software. CARTONET is the cloud-based platform used to store, analyze, and share patients’ case data from CARTO system. Detailed information on how the CARTONET collect, store, and analyze the data can be found over here [6].
Study population
Mercy healthcare system operates in four states in the Midwest with over 40 hospitals, 12 outpatient surgery centers and 35 urgent care sites, and provides care to approximately 4.2 million patients. Patients (19 years and older) with AF (International Classification of Diseases, 10th revision (ICD-10), Clinical Modification (ICD-10-CM) diagnostic code I48.X) undergoing catheter ablation (identified using ICD-10-procedure codes (ICD-10-PCS) (ICD-10-PCS codes: 02563ZZ, 02573ZZ, 025K3ZZ, 025L3ZZ, 02583ZZ, 02553ZZ, 025M3ZZ, 025S3ZZ, 025T3ZZ) /current procedural terminology (CPT) (CPT code: 93656)) using either the THERMOCOOL SMARTTOUCH® catheter or THERMOCOOL SMARTTOUCH® SF ablation catheter (Biosense Webster Inc, Irvine, CA) in an inpatient or outpatient setting between January 1, 2016 and December 31, 2018 were included. The first occurrence of cardiac ablation procedure was considered as index ablation. Patients needed to be continuously enrolled in Mercy health for at least 12 months pre-index and 12 months post-index period to be included. Patients with catheter ablation procedure for AF within 6 months prior to the index date were excluded.
Study outcome
The outcome of the study included 1-year AF recurrence defined as a composite of repeat ablation, AF related hospitalization, and/or direct current cardioversions (DCCV) post-index ablation.
CARTONET procedural data
Catheter ablations involve the pulmonary vein isolation in right and left wide antral circumferential ablation (WACA) with potential additional ablation lines in right and left carina, roof line, mitral line, inferior line, posterior line, anterior line as shown in Fig 1 [7]. Each ablation lesion was automatically classified into an anatomical line or site using a validated ML algorithm with correction by a human observer as necessary. Right WACA includes lesions from right roof, right posterior, right inferior, and right anterior while left WACA include lesions from left roof, left posterior, left inferior, left anterior, and ridge. Additional ablation lines, such as right and left carina line, anterior line, roof line, posterior line, were performed as needed. The number of ablation lesions and duration of ablation were calculated for each ablation line or site. Averages of the following ablation parameters including stability [mm], average force [gr], and maximum power [W] were derived from the Visitag module for each ablation site/line. Also, total number of ablation lesions and total ablation, mapping, and procedural duration were calculated for each patient.
Right WACA includes right posterior (light blue), right inferior (purple), right anterior (blue) and right roof (green); left WACA includes left posterior (light orange), left inferior (yellow), left anterior (orange), ridge (light purle) and left roof (red). LAA: left atrial appendage; LS: left superior pulmonary vein (PV); LI: left inferior PV; RS: right superior PV; RI: right inferior PV.
Ablations were not conducted at all anatomical sites or lines for every patient. If there was no ablation conducted on the anatomical lines, procedural data for the corresponding anatomical lines were set to zero before analysis.
Statistical analysis
Demographic, clinical, and ablation characteristics were compared for patients with and without AF recurrence. Chi-square test and count/proportion were used for categorical variables. For continuous variables, t-test and mean/standard deviation (SD) were used.
AutoGluon-Tabular, an open-source automate machine learning (AutoML) framework in Python, were used to train AutoML model by ensembling multiple machine learning models (e.g. random forest, neural network, boosted tree, catBoost) and stacking them in multiple layers [8]. The dataset was divided into train data (70%) for model development and test data (30%) for model validation. At first, AutoML model included patients’ demographic and clinical variables. Second, AutoML model was developed with CARTO procedural data, as well as demographic/clinical variables. AutoGluon was run with parameters “eval_metric = ’roc_auc’, presets = ’best_quality’, and time_limit = 1200”, leaving all other parameters were left as default. The AutoGluon function automatically selects 20% of the training data for validation and hyperparameter fine-tuning.
Model performance was assessed on test data. Area under Receiver operating characteristic (AUROC) was calculated for AutoML models without and with procedural data. Feature importance is calculated using permutation method by randomly shuffling the values of each feature in the test data and calculating the resulting decrease in AUROC [9]. The features with the largest decrease in performance are considered the most important. Patients in test data were grouped in three risk categories (tercile): low, medium, and high based on the predictive probability of AF recurrence from AutoML models without procedural data. Same cutoff values of predictive probability were used for AutoML model with procedural data. Net reclassification improvement (NRI) was calculated to compare discrimination between AutoML models without and with procedural data [10].
Partial dependence plots (PDP) and accumulated local effect (ALE) plots were used to visualize how each CARTO feature influences the AF recurrence from AutoML model using test data [9]. Both PDP and ALE plots illustrate the average probability of AF recurrence when values of a study feature are changed, and all other features keep their original values. PDP used all the patients in the test data to calculate the average prediction for a unique value for the study feature. For ALE, it first identifies a subset of patients within a defined value range of the study feature, replace the study feature value with its minimum and maximum value within the subset, then calculate the difference of prediction between the subset with minimum and maximum value. The above process is repeated for all the defined intervals for the study feature and the cumulative sum of differences were calculated for ALE plots. Compared to PDP, ALE is valid if a feature is correlated with other features.
Results
There were 306 patients in this study with 67 (21.9%) patients having the AF recurrence within 1 year post ablation. As shown in Table 1, patients with persistent AF were more likely to have AF recurrence than paroxysmal AF. There were no statistical differences between patients with and without AF recurrence regarding to age, gender, sleep apnea, Elixhauser score, and CHA2DS2-VASc score.
As shown in Table 2. patients with AF recurrence had longer ablation and mapping duration, and higher number of total ablation lesions than patients without AF recurrence. Patients with AF recurrence had higher number of lesions in left anterior, left ridge, left roof, left WACA, and total number of lesions, as well as lower right inferior force than patients without AF recurrence.
Fig 2 shows the ablation patterns of study samples. Ablation pattern is defined as the combinations of ablation status in the anatomical sites/lines. The Y axis shows the ablation sites/lines and the number of patients with ablation conducted at the corresponding sites/lines. Red tiles indicate ablation were conducted. All the patients have ablations conducted at the right and left WACA. However, there were a few patients with incomplete PVI at some anatomical sites, especially for right inferior (25 patients), and left roof (13 patients). Ablation lines were only conducted at subset of patients. The most common ablation lines were right carina (187 patients), left carina (109 patients), and roof line (141 patients). Only 25 patients had anterior line ablation and 68 patients with posterior line ablation.
Outcomes represents the number of AF recurrence among patients with the specific ablation pattern as shown in the bottom of figure.
After adding procedural data to the AutoML model, AUROC increased from 0.66 to 0.78 as shown on Fig 3(A). Patients in the test data were grouped into low, medium, and high-risk categories (tercile) based on predicted probability of AF recurrence from AutoML model. As shown on Fig 3(B), in the low-risk category, AutoML without procedural data had 12.9% vs 0% of AF recurrence for model with procedural data, respectively; For the high-risk category, AutoML model without procedural data had 32.2% of AF recurrence vs 44.8% of AF recurrence for model with procedural data.
(a): AUROC curves, (2) proportion of patients with AF recurrence in the risk categories.
NRI for patients with and without AF recurrence are 32% and 4% respectively, as shown in Table 3. For patients with AF recurrence, there were 8 (42%) patients moved to higher risk categories when adding procedural data to model. There were no patients in low-risk category in the AutoML model with procedural data. For patients without AF recurrence, more patients are in low-risk category in AutoML model with procedural data with the NRI of 4%.
Table 4 shows the top 10 important predictive features for AF recurrence from AutoML model with procedural data. Nine of 10 features were from CARTO procedural data. Right inferior force and AF type had the feature importance of 0.08. It indicates AUROC would decrease by 0.08 if column of right inferior force or AF type were randomly shuffled while all other features were unchanged. The curves from PDP and ALE as shown on Figs 4 and 5 were similar for the top 9 important procedural features. Lower right inferior forces and low number of left inferior and right roof lesions are associated with high risk of AF recurrence. Long ablation duration and high number of left ridge lesions increased the risk of AF recurrence.
Histogram shows the distribution of patients for each feature.
Histogram shows the distribution of patients for each feature.
Discussion
Adding CARTO procedural data to AutoML model improve the identification of patients who would experience AF recurrence. Nine of 10 top predictive features were from CARTO procedural data. AutoML model with procedural data provides important actionable insights to improve the ablation process.
Catheter ablation is a complex process that requires physicians use a diagnostic catheter to map the left atria and use a therapeutic catheter to abate cardiac tissue to block abnormal electrical signals. Successful ablation requires physicians to create lesions with appropriate contact force and power. Location of lesions is also critical as it determines whether lesions can durably block the transmission of abnormal electrical signals [7]. Automated segmentation of lesions into specific anatomical sites and calculation of their ablation parameters provide objective feedback for physicians to review their ablation process. Linking procedural data with EHR data allowed us to build machine learning model to predict AF recurrence. Our goal is to build models that can be used to inform cardiac ablation workflow and thereby improve patient outcomes.
AutoML model identified some important features that predicts the AF recurrence and provides directions for improvement in ablation strategies. Nine out of top 10 predictive features were from procedural data. Low right inferior contact force was associated with high risk of AF recurrence. Contact force provides an actionable direction for physicians to improve success rate of ablation during procedures. In this study, some patients had no ablation conducted at right inferior area. This could be due to great catheter instability around the right sided veins [11]. The ability to deliver force to right inferior atrium may be dependent on operator experience. Left inferior and right roof are from left and right WACA lines. Low number of lesions in left inferior and right roof were associated with high risk of AF recurrence. Therefore, complete PVI with more ablation lesions to completely block the abnormal electrical signals in right and left WACA have potentials to improve the success of ablation.
As shown in other studies, patients with persistent AF are more likely to have AF recurrence. One of the key features predicting AF recurrence was AF type, with persistent AF patients significantly more likely to have recurrence as compared to patients with paroxysmal AF. Persistent AF is the advanced stage of AF with long lasting episodes of arrhythmia than paroxysmal AF. Early ablation (referred as RFA within 1 year of AF diagnosis) can lead to improvements in adverse cardiac remodeling, greater success in maintain sinus rhythm, and decreased need for repeat ablation based on the results of clinical trials and systematic reviews [12, 13]. Based on meta-analysis results of randomized trials, catheter ablation as first-line rhythm control therapy was associated with lower incidence of atrial tachyarrhythmias recurrence and hospitalizations compared with AADs, with similar incidence of death, stroke/transient ischemic attack, and serious adverse effects [14].
Identifying patients with high risk of AF recurrence is an important step to improve the ablation success rate. Various models were developed to predict recurrent atrial fibrillation after catheter ablation [15]. For example, “FLAME score” (range 0–9) were developed for non-paroxysmal atrial fibrillation ablation and included sex, AF type, left atrial diameter, mild to moderate mitral regurgitation (MR), and extreme comorbidity [16]. Extreme comorbidity was defined as the presence of any one of severe mitral regurgitation (MR), moderate or severe mitral stenosis, mitral valve replacement, hypertrophic cardiomyopathy or structural congenital heart disease. C-statistics for FLAME score is 0.64 and patients with score > = 5 had statistically significant high risk of AF recurrence. Similarly, ATLAS score was developed to predict AF recurrence after the first catheter ablation procedures and included age, sex, smoking, AF type, left atrial volume [17]. However, both FLAME and ATLAS score focused on the pre-operative variables and did not include the intraprocedural variables. Our results demonstrate the importance of ablation procedural data in identifying patients with high risk of AF recurrence. Our model could be used to identify patients with high risk of AF recurrence post-ablation. Our model can also be integrated into the cardiac ablation technology system to provide real-time feedback to electrophysiologists during the procedure, enabling them to achieve better results. The model also identified areas for improvement in ablation procedures and provides directions to improve ablation strategies.
Limitation
This study used the linked data between procedural data and EHR data. AF recurrence was defined using diagnosis codes and procedurals codes from EHR data. AF recurrence is more likely underestimated as recurrence are frequently asymptomatic and our AF recurrence definition are more likely to capture severe cases with required encounters with hospitals or physicians. Some procedural data, such as impedance drop, and ablation index had missing values for some patients and were excluded from AutoML model. Future model can be developed by including all ablation parameters. The study results were evaluated on the test data that were not used for model development. External validation should be conducted to use independent validation dataset to evaluate whether model would be generalized to patients outside of Mercy Health. The ablation procedural data comes from patients using Thermocool catheters and CARTO platform. External validation should be conducted to evaluate model performance on catheters and platforms from other manufacturers. Future changes of ablation parameters are anticipated as more physicians use CARTONET to review their ablation process and improve their procedures. Future updates on the model should be conducted to reflect the newest practice of ablation.
Conclusion
AutoML model with procedural data achieved improved discrimination ability than AutoML model without procedural data. There is significant potential for CARTONET and AutoML model with procedural data to provide rich insights into factors influencing AF recurrence and thereby lead to improvements in AF ablation success rate.
References
- 1. Bizhanov K.A., et al., Atrial fibrillation: Epidemiology, pathophysiology, and clinical complications (literature review). J Cardiovasc Electrophysiol, 2023. 34(1): p. 153–165. pmid:36434795
- 2. Darby A.E., Recurrent Atrial Fibrillation After Catheter Ablation: Considerations For Repeat Ablation And Strategies To Optimize Success. J Atr Fibrillation, 2016. 9(1): p. 1427.
- 3. Rattanakosit T., et al., Role of Indices Incorporating Power, Force and Time in AF Ablation: A Systematic Review of Literature. Heart Lung Circ, 2021. 30(9): p. 1379–1388. pmid:33958295
- 4. Nery P.B., et al., Relationship Between Pulmonary Vein Reconnection and Atrial Fibrillation Recurrence: A Systematic Review and Meta-Analysis. JACC Clin Electrophysiol, 2016. 2(4): p. 474–483.
- 5. Hussein A., et al., Use of Ablation Index-Guided Ablation Results in High Rates of Durable Pulmonary Vein Isolation and Freedom From Arrhythmia in Persistent Atrial Fibrillation Patients: The PRAISE Study Results. Circ Arrhythm Electrophysiol, 2018. 11(9): p. e006576. pmid:30354288
- 6. Kreidieh O., et al., Utility of a cloud-based lesion data collection software to record, monitor, and analyze an ablation strategy. Heart Rhythm O2, 2022. 3(3): p. 319–322. pmid:35734303
- 7. Calkins H., et al., 2017 HRS/EHRA/ECAS/APHRS/SOLAECE expert consensus statement on catheter and surgical ablation of atrial fibrillation. Heart Rhythm, 2017. 14(10): p. e275–e444. pmid:28506916
- 8.
Erickson, N., et al., AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data. ArXiv, 2020. abs/2003.06505.
- 9.
Molnar C., Interpretable Machine Learning—A Guide for Making Black Box Models Explainable. 2 ed. 2022.
- 10. Leening M.J., et al., Net reclassification improvement and integrated discrimination improvement require calibrated models: relevance from a marker and model perspective. Stat Med, 2014. 33(19): p. 3415–8. pmid:25042215
- 11. Nair G.M., et al., Three-year outcomes and reconnection patterns after initial contact force guided pulmonary vein isolation for paroxysmal atrial fibrillation. J Cardiovasc Electrophysiol, 2017. 28(9): p. 984–993. pmid:28635046
- 12. D’Angelo R.N., et al., Very early versus early referral for ablation in young patients with newly diagnosed paroxysmal atrial fibrillation. Pacing Clin Electrophysiol, 2022. 45(3): p. 348–356. pmid:35150152
- 13. Proietti R., et al., A Systematic Review on the Progression of Paroxysmal to Persistent Atrial Fibrillation: Shedding New Light on the Effects of Catheter Ablation. JACC Clin Electrophysiol, 2015. 1(3): p. 105–115. pmid:29759352
- 14. Kheiri B., et al., Ablation Versus Antiarrhythmic Drugs as First-Line Treatment of Paroxysmal Atrial Fibrillation: A Meta-Analysis of Randomized Trials. Circ Arrhythm Electrophysiol, 2021. 14(8): p. e009692. pmid:34397264
- 15. Dretzke J., et al., Predicting recurrent atrial fibrillation after catheter ablation: a systematic review of prognostic models. Europace, 2020. 22(5): p. 748–760. pmid:32227238
- 16. Boyalla V., et al., Internationally validated score to predict the outcome of non-paroxysmal atrial fibrillation ablation: the ’FLAME score’. Open Heart, 2021. 8(2). pmid:34348972
- 17. Mesquita J., et al., Development and validation of a risk score for predicting atrial fibrillation recurrence after a first catheter ablation procedure—ATLAS score. Europace, 2018. 20(FI_3): p. f428–f435. pmid:29016770