Article Text
PDF
Abstract
Background Transcutaneous carbon dioxide (TcCO2) monitoring is a non-invasive alternative to arterial blood sampling. The aim of this review was to determine the accuracy and precision of TcCO2 measurements.
Methods Medline and EMBASE (2000–2016) were searched for studies that reported on a measurement of PaCO2 that coincided with a measurement of TcCO2. Study selection and quality assessment (using the revised Quality Assessment of Diagnostic Accuracy Studies tool (QUADAS-2)) were performed independently. The Grading Quality of Evidence and Strength of Recommendation approach was used to summarise the strength of the body of evidence. Pooled estimates of the mean bias between TcCO2 and PaCO2 and limits of agreement with outer 95% CIs (termed population limits of agreement) were calculated.
Results The mean bias was −0.1 mm Hg and the population limits of agreement were −15 to 15 mm Hg for 7021 paired measurements taken from 2817 participants in 73 studies, which was outside of the clinically acceptable range (7.5 mm Hg). The lowest PaCO2 reported in the studies was 18 mm Hg and the highest was 103 mm Hg. The major sources of inconsistency were sensor location and temperature. The population limits of agreement were within the clinically acceptable range across 3974 paired measurements from 1786 participants in 44 studies that applied the sensor to the earlobe using the TOSCA and Sentec devices (−6 to 6 mm Hg).
Conclusion There are substantial differences between TcCO2 and PaCO2 depending on the context in which this technology is used. TcCO2 sensors should preferentially be applied to the earlobe and users should consider setting the temperature of the sensor higher than 42°C when monitoring at other sites.
Systematic review registration number PROSPERO; CRD42017057450.
- respiratory measurement
- clinical epidemiology
Statistics from Altmetric.com
Key messages
What is the key question?
Transcutaneous carbon dioxide (TcCO2) monitoring devices are commercially available and are being used in clinical practice, so it is vital that clinicians have a clear understanding of the accuracy of these devices to ensure they are applied in appropriate circumstances.
What is the bottom line?
A TcCO2 measurement at any single point in time could be as much as 15 mm Hg higher or lower than PaCO2, meaning that an arterial blood gas sample would be required to confirm diagnosis prior to initiation (or cessation) of treatment.
Why read on?
As clinicians would be interested in the accuracy of the type of transcutaneous monitoring device they use and for the population in which they use it, we provide population limits of agreement according to the indication for monitoring (eg, respiratory failure, surgery, intensive care unit, sedation and postoperative recovery), type of device (Sentec and TOSCA), as well as location of sensor placement and sensor temperature.
Introduction
Measurement of PaCO2 in arterial blood is the reference standard for ventilation assessment.1 Arterial puncture is painful and time-consuming, and there is risk of infection as well as tissue and nerve damage.2 Measurement of carbon dioxide (CO2) levels from the skin, which is known as transcutaneous carbon dioxide (TcCO2) monitoring, is a non-invasive alternative to arterial blood sampling. TcCO2 monitors measure PaCO2 that diffuses through the skin by the application of a sensor, which is heated above body temperature (typically to between 40°C and 44°C) to achieve local arterialisation. Local arterialisation, combined with application of an algorithm that corrects the CO2 value detected by the sensor to 37°C, is thought to provide an accurate estimate of PaCO2. False-positive and false-negative indications of worsening ventilation status are both important issues to consider regarding the application of these monitors to the clinical practice context. A false-positive or false-negative indication about ventilation status from TcCO2 monitoring may lead to inappropriate initiation, delay or avoidance of treatment, which could be detrimental for the patient.
The agreement between TcCO2 and PaCO2 has been investigated in a large number of studies. Synthesis of the data through meta-analysis would aid clinical decision-making regarding the appropriate circumstances in which TcCO2 monitoring can be used. We aimed to determine if TcCO2 has clinically acceptable accuracy and precision compared with PaCO2. Accuracy is defined in this context as the average difference between TcCO2 and PaCO2 measurements and precision as the variance (typically reported as SD) in the differences.
Methods
A systematic review was conducted according to a prespecified protocol (PROSPERO trial registration number: CRD42017057450).
Data sources and searches
Published studies were located by searching Medline and EMBASE from January 2000 to December 2016, as well as the reference lists of articles identified to be relevant to the review. This search strategy is an efficient approach for systematic reviews of diagnostic test accuracy studies.3 Unpublished and ongoing studies were located by searching the International Clinical Trials Platform. Published conference abstracts were planned to be included if there was sufficient detail reported to assess study quality. Language restrictions were not imposed for the search. The Cochrane-recommended search strategy combining terms for the ‘target condition’ and ‘index test’ was used.4 The specific search strategy for each database is in online supplementary file 1. Study selection was performed by two independent reviewers.
Supplementary file 1
Studies that reported a measurement of PaCO2 that coincided with a measurement of TcCO2 were included. Studies conducted before the year 2000 were excluded as earlier studies evaluated outdated technology. Only studies that reported on PaCO2 measured either by a point-of-care blood gas analyser or central laboratory that coincided with a measurement of the index test were included.
Data extraction and quality assessment
Information about the study characteristics (author, year of publication, country, design, sample size, clinical setting, number studied and number analysed for each outcome, number of dropouts with reason, and funding source), population characteristics (inclusion/exclusion criteria, mean/median and range of PaCO2) and TcCO2 characteristics (timing and methods of sampling/measurements, method of sampling/calibration) was extracted. Outcomes extracted were the mean bias (ie, accuracy) and variance or SD (ie, precision) in CO2 between transcutaneous and arterial blood gas analyses. Information was extracted about how repeated measurements were handled: (1) analysed each pair of data separately; (2) treated each pair of data as independent; or (3) used either analysis of variance or a random-effects model as a way to control for the dependent nature of the repeated measures data.5
Risk of bias assessment of the included studies was undertaken independently in duplicate using the revised Quality Assessment of Diagnositc Accuracy Studies (QUADAS-2) tool.6 Guiding questions were used to rate the risk of bias for patient selection, conduct of the TcCO2 measurement, conduct of the PaCO2 measurement and flow and timing (eg, timing of TcCO2 or PcCO2 measurements and dropouts) as ‘high’, ‘low’ or ‘unclear’. The risk of publication bias was minimised by comprehensively searching multiple databases and an international clinical trial registry.7 Language restrictions were not imposed for the search. However, we were unable to classify four potentially eligible studies because the full text was not in English. Statistical analyses to detect reporting bias were not conducted due to lack of validated methods.8 Although some meta-analyses of method comparison studies have used tests for detecting funnel plot asymmetry,9 simulations have revealed that such tests will result in publication bias being incorrectly identified too often.10
We applied the Grading Quality of Evidence and Strength of Recommendations methodology to rate the quality of evidence.11 Reasons used to downgrade the evidence were study limitations, inconsistency and imprecision. We did not downgrade for indirectness because this systematic review excluded studies that were not relevant. Publication bias was not formally assessed so the possibility of this bias was not excluded but not considered sufficient to require downgrading the quality of evidence.
Data synthesis and analysis
Our goal for the meta-analysis was to estimate the population limits of agreement between TcCO2 and PaCO2. The framework for meta-analysis of Bland-Altman method comparison studies based on a limits of agreement (LoA) approach was used.12 We selected this method since it mirrors the approach in primary Bland-Altman studies, providing an estimate of the pooled LoAs in the population (not just the samples studied). The ‘population LoA’ is wider than those typically reported in meta-analyses of Bland-Altman studies.12 Here the pooled LoAs are calculated using δ±2√(σ2+τ2), where δ is the average bias across studies, σ2 is the average within-study variation in differences and τ2 is the variation in bias across studies. We estimated δ and σ2 using a weighted least-squares model (similar to a random-effects approach) and estimated their SEs using robust variance estimation (RVE). We used RVE instead of model-based SEs because many studies included in our review used repeated-measures designs without accounting for the correlation between measurements.13–15 The method-of-moments estimator from ref 16 was used for the τ2 parameter. Following ref 12 we also (1) included measures of uncertainty when interpreting the LoA estimates by calculating the outer 95% CIs for pooled LoA; and (2) adjusted repeated measurements which were not properly adjusted in individual studies (by using weights proportional to the number of samples not the total number of measurements). Formulas for these calculations from ref 12 are provided in online supplementary file 1. All analyses were conducted in the R statistical program.17 The R code (provided in ref 12) and all data used in the meta-analyses are available at https://doi.org/10.6084/m9.figshare.6244058.v2.
The results from the individual studies were converted into a standard format to conduct meta-analyses, with bias meaning PaCO2-TcCO2 measured in mm Hg. In two of the studies, the results were reported for two separate groups of participants, so these were treated in the meta-analyses as separate ‘studies’. Other studies reported separate results for analyses conducted using different TcCO2 device types or sensor locations performed on the same patients. Only the result with the largest number of paired measurements between PaCO2 and TcCO2 was selected for inclusion in the main analysis, with others included in subgroup meta-analyses where appropriate. For the studies that reported results for patients while receiving both two-lung and one-lung ventilation during thoracic surgery, we used the result for two-lung ventilation in the main analysis.
The conventionally cited clinically acceptable agreement between TcCO2 and PaCO2 is 7.5 mm Hg (or 1 kPa).18 We deemed that outer confidence bounds for 95% LoA between transcutaneous and arterial CO2 measurements (termed as ‘population limits of agreement’) outside of these bounds would not be clinically acceptable.
We performed sensitivity analysis for the primary meta-analysis based on risk of bias (eg, treating ‘unclear risk of bias’ as ‘high risk’ and removing ‘high risk of bias’ studies from the analyses). As clinicians would be interested in the accuracy of the type of transcutaneous monitoring device they use and for the population in which they use it, we conducted subgroup analyses according to the indication for monitoring (eg, volunteer study, respiratory failure, surgery, intensive care unit (ICU), sedation and postoperative recovery), type of device (Sentec and TOSCA), as well as location of sensor placement and sensor temperature.
Results
Study selection and description
There were 73 studies eligible for inclusion (figure 1). The characteristics of each study are in online supplementary file 1. The 73 studies enrolled 2817 participants predominantly from Europe, USA and UK. Sixteen (22%) studies included adult participants in ICUs, 6 (7%) studies included paediatric participants in ICUs or having surgery, 7 (10%) studies included neonates, 13 (18%) studies were conducted with adults undergoing surgery with general anaesthesia, 13 (18%) studies were focused on acute respiratory failure, 9 (12%) studies included participants with chronic respiratory failure, 5 (7%) studies included patients who were sedated but spontaneously breathing either during or after surgery, and 4 (5%) studies were conducted with patients undergoing lung function testing. Two studies intentionally manipulated the range of PaCO2 by inducing hypoventilation and hyperventilation. The lowest PaCO2 reported in the studies was 18 mm Hg and the highest was 103 mm Hg.
Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.
Several different TcCO2 monitors were evaluated across the studies included in this review, including the TCM3 (n=12), TCM4 (n=11), TOSCA 500 with Sensor 92 (n=7), TOSCA not otherwise classified (n=13), Sentec with V-Sign sensor (n=27), Sentec with V-Sign 2 sensor (n=2), Fastrac (n=1), Microgas (n=2) and PeriFlux (n=1). All studies reported that device manufacturer instructions were followed regarding calibration and stabilisation of the sensor prior to undertaking assessments. Most studies reported that the temperature of the TcCO2 sensor was less than 43°C (n=49; 67%). The earlobe was the most common sensor location site evaluated (n=45). Other sensor location sites included the chest, upper arm, abdomen, forehead, cheek and palmar surface of the forearm.
There was a high risk of bias associated with patient selection for 14 (19%) studies, conduct of TcCO2 and PaCO2 measurements in 7 (10%) and 9 (12%) studies, respectively (mostly due to PaCO2 measurements being taken with knowledge of the TcCO2 measurement and vice versa), and participant flow for 7 (10%) studies. The authors declared that they either had a conflict of interest or had received equipment or funding from the manufacturers of the device being evaluated in 19 (26%) studies.
Agreement between transcutaneous and arterial CO2 measurements
Table 1 presents the results of the primary meta-analysis, sensitivity analysis and subgroup analyses. Data from all 73 studies were included in the primary meta-analysis. Although the pooled estimate of the mean bias between PaCO2 and TcCO2 was small (0.1 mm Hg), the variation in these differences was large (figure 2), resulting in the two methods differing from −15 mm Hg to 15 mm Hg across all patients studied. These population limits of agreement were not in the clinically acceptable range. A summary of findings is presented in table 2. We downgraded the quality of evidence for the primary outcome to low quality due to concerns about study limitations and inconsistency. Population limits of agreement for the sensitivity analysis restricted to studies rated as having low risk of bias were also outside of the clinically acceptable range (−9 mm Hg to 10 mm Hg; 1600 paired measurements from 842 participants in 23 studies).
Comparisons within and across studies. Dotted curves are distributions of the differences between arterial and transcutaneous carbon dioxide (CO2) in individual studies. Solid curve filled with blue is the distribution of the pooled estimate of the difference between arterial and transcutaneous CO2. Dotted vertical lines indicate bounds for the pooled estimates for limits of agreement between arterial and transcutaneous CO2. Solid vertical lines indicate bounds for the outer 95% CIs for the pooled estimates of limits of agreement between arterial and transcutaneous CO2 (ie, population limits of agreement).
Results of meta-analysis of agreement between transcutaneous and arterial carbon dioxide
Summary of findings for accuracy and precision of transcutaneous carbon dioxide monitoring
The population limits of agreement for the TOSCA device were within the clinically acceptable range (−7 to 6 mm Hg; 3313 paired measurements from 1561 participants in 45 studies) but not for the Sentec device (−9 to 9 mm Hg; 3585 paired measurements from 1256 participants in 30 studies). However, population limits of agreement differed according to the location that the sensor was applied and the temperature of the sensor. TcCO2 monitoring was accurate to a clinically acceptable degree in a meta-analysis of 44 studies (3974 paired measurements from 1786 participants) where the sensor was applied to the earlobe with either the TOSCA (20 studies) or Sentec (24 studies) device. The population limits of agreement were −6 to 6 mm Hg. In contrast, population limits of agreement were outside the clinically acceptable range where TcCO2 monitoring was conducted with the sensor on the chest (−11 to 12.7 mm Hg; 1041 paired measurements from 471 participants in 13 studies) and the arm (−8 mm Hg to 5.3 mm Hg; 247 paired measurements from 157 participants in 7 studies). There was a large amount of variation in bias between the 16 studies where TcCO2 sensors were located at other sites (τ2=30.1), resulting in extremely wide estimates for population limits of agreement (−134 to 133 mm Hg). Of note, studies that applied the sensor to the earlobe set the temperature of the device to 42°C, whereas studies that applied sensors to the chest or other sites used a variety of different temperature settings. Population limits of agreement were wider in meta-analysis of studies which set the temperature of the sensor to 42°C (−25 to 26 mm Hg) compared with studies where the sensor temperature was higher (−7 to 7 mm Hg). There was large variation in bias between these studies (τ2=12.0), which was likely due to the location of sensor placement (earlobe in 35 studies, chest in 4 studies and arm/forearm in 3 studies).
TcCO2 monitoring was accurate to a clinically acceptable degree for only a minority of the subgroup meta-analyses conducted according to clinical indication. Population limits of agreement were within the clinically acceptable range for studies that enrolled adults in ICU (16 studies), children undergoing surgery or in ICU (6 studies) and adults undergoing lung function testing (4 studies).
Discussion
It is vital that clinicians have a clear understanding of the accuracy of TcCO2 monitoring devices to ensure they are applied in appropriate circumstances. Both the primary meta-analysis and sensitivity analysis, including only studies at low risk of bias, revealed population limits of agreement outside of the clinically acceptable range. Clinicians using transcutaneous monitoring to assess ventilation status in patients across the broad range of populations included in our systematic review should therefore determine baseline PaCO2 and the TcCO2-PaCO2 gradient and to confirm the diagnosis of hypercapnoea prior to initiation (or cessation) of treatment.
The results from our subgroup analyses have important implications for how TcCO2 monitoring should be applied. No specific recommendations for a preferred site or sites are provided by manufacturers. Similarly, guidelines on transcutaneous monitoring from the American Association for Respiratory Care do not provide a recommendation for the optimal site to place a TcCO2 sensor.18 Our analysis indicates that TOSCA and Sentec TcCO2 device sensors should preferentially be placed on the earlobe because the population limits of agreement were within the bounds of the clinically acceptable range (<7.5 mm Hg). Monitoring on the earlobe had similar LoAs to those reported in a meta-analysis of capillary blood gas LoAs, where the mean bias was −0.1 mm Hg and the SD of bias was 2.9 mm Hg.19 If TcCO2 monitoring is feasible from the earlobe, it should be considered the preferable solution to use for ventilation assessment over capillary blood gas because of its non-invasiveness and ability to provide a continuous assessment of ventilation. Likewise, the results of our meta-analyses suggest that TcCO2 measurements would be more accurate than estimations of PaCO2 derived from venous blood gas samples. The mean bias between PaCO2 and venous blood CO2 measurements was estimated to range from −10.7 mm Hg and 2.4 mm Hg in a meta-analysis of 16 studies.20 Pooled estimates of the LoAs between venous and arterial measurements of CO2 were not reported.20 It should be noted though that not all patients who may benefit from continuous ventilation assessment will be suitable for application of a TcCO2 monitoring sensor to their earlobes. For example, the earlobes of a neonate requiring ventilation assessment may not be large enough to accommodate a TcCO2 sensor. Adult patients with multiple piercings, undergoing surgery to or with trauma around the head and neck would also preclude the application of a TcCO2 sensor to the earlobe.
We found large differences in population limits of agreement in subgroup analyses focusing on sensor temperature. Meta-analysis restricted to studies that used a sensor temperature of 42°C exhibited worse agreement with PaCO2 in comparison with meta-analysis of studies that used higher temperatures. The difference in these results can be explained by the large between-study variation in bias in the subgroup analysis of studies that used a sensor temperature of 42°C but applied the sensor to either the earlobe, chest or arm. Of note, the majority of studies where the sensor was applied to participants’ earlobes set the temperature of the sensor to 42°C but still yielded clinically acceptable population limits of agreement. Together, the findings from both subgroup analyses indicate that if monitoring on the earlobe is not possible, the sensor temperature should be set higher than 42°C.
A strength of this analysis is the incorporation of the variation in bias between studies, the bias associated with repeated measures not accounted for in the analysis of individual studies, as well as measures of uncertainty (CIs) into our estimates of the agreement between transcutaneous and arterial CO2 measurements. Standard meta-analysis approaches focused on providing the average bias and the average precision separately may have erroneously led clinicians to believe that the agreement between TcCO2 and PaCO2 measurements is acceptable. The risk here is clearly evident for the primary meta-analysis where the estimate of the population limits of agreement were within clinically acceptable bounds but the outer 95% CIs were far wider. There was a larger difference between the population limits of agreement and the CIs because the sampling variation in each component of the LoA is taken into account when calculating the CIs (the mean bias, SD and variation in bias between studies). By incorporating the between-study heterogeneity in bias and sampling variation, it is clear that the LoAs in the population are much broader and not clinically acceptable for interchangeable use across the range of situations in which TcCO2 monitoring was tested in the main analysis.
The trending ability of TcCO2 monitors is an important information for clinicians to consider when using TcCO2 in practice. This is because PaCO2 continuously changes in response to a variety of factors. In addition, evaluating trends in TcCO2 may be useful in clinical practice for evaluating the effectiveness of interventions employed to improve ventilation status. Conclusions about trending ability can be drawn from the accuracy and precision of absolute measurements (ie, LoAs) by making a qualitative judgement about whether or not the index test is sufficiently precise. We did not identify strong evidence to support the trending ability of TcCO2 for ventilation assessment in this systematic review because the population limits of agreement for the primary meta-analysis were wide. If part of the imprecision relates to patient-specific characteristics, such as vascularity, it is possible that there will be a systematic measurement error for within-patient readings, and therefore the within-patient trend may have tighter LoAs than individual measurements. Therefore, methods other than the Bland-Altman approach may be more suited to quantitatively assess trending ability. For example, the 4Q, polar analysis and clinical concordance methods have recently been recommended for consideration in method comparison studies evaluating the validity of cardiac output monitors.21 Similar to TcCO2 monitoring, cardiac output monitors provide a continuous estimate of a dynamic physiological parameter, which changes rapidly from various influences. As such, further research aiming to examine the trending ability of TcCO2 monitoring should consider incorporating such assessments.
The results of subgroup analyses according to the indication for monitoring identified specific clinical areas where further research into the accuracy of TcCO2 monitoring would be beneficial because population limits of agreement were outside of the clinically acceptable range. These include acute respiratory failure, thoracic surgery with single lung ventilation and for assessment of ventilation in patients who are sedated during or after surgery. However, it should be noted that studies within these subgroups applied TcCO2 monitoring using different devices (Sentec and TOSCA), sensor locations and temperatures, which may explain the large variation in bias between studies and resulting imprecise estimates of LoAs. Clinicians who use TcCO2 monitoring in these areas should be confident that TcCO2 measurements would be within clinically acceptable agreements if applied to the earlobe or another monitoring site at a temperature above 42°C using either the Sentec or TOSCA device.
Transcutaneous CO2 monitoring is commonly used to reduce the frequency of arterial blood gas analysis in neonates due to the limitations of other methods to estimate PaCO2 in this population, such as end-tidal CO2 and capillary blood gas analysis.22 Yet we identified weak evidence for accuracy and precision with population limits of agreement that were far outside the clinically acceptable range. It should be noted though that we only included studies that compared TcCO2 with PaCO2. Several articles were excluded due to TcCO2 being compared instead with capillary blood gas analysis.23
Limitations
Data were not extracted on adverse events related to transcutaneous monitoring. We cannot rule out the possibility of publication bias. However, this may not be as serious a problem for diagnostic test accuracy studies as it is for randomised trials.8 We did not use meta-regression or tests for interaction between subgroups to investigate for sources of heterogeneity because of our focus on the population limits of agreement, which incorporated the variation in bias between studies into the estimates. This systematic review did not assess the clinical utility of these monitors. Therefore, the evidence to be derived from this systematic review should only be considered within the context of other information about the clinical utility, reliability and ease of use of these devices during normal clinical practice. Of note, the level we set as the limit for clinically acceptable agreement between PaCO2 and TcCO2 (7.5 mm Hg) was chosen based on recommendations for ventilation monitoring made by the American Association for Respiratory Care therapists. If a difference in repeated measurements of PaCO2 less than this magnitude would be important for a given situation in clinical practice, then an arterial blood sample should be drawn to confirm diagnosis when a change in TcCO2 is observed during monitoring.
Conclusion
This meta-analysis has identified that there may be substantial differences between TcCO2 and PaCO2 depending on the context in which this technology is used in clinical practice. Measuring TcCO2 from the earlobe with either the TOSCA or Sentec device would yield clinically acceptable accuracy. As such, this monitoring site is recommended for use in clinical practice. For optimal accuracy and precision, users should set the temperature of the sensor higher than 42°C when monitoring at sites other than the earlobe.
References
Footnotes
Contributors All authors have agreed on the final version. AC: study design, data extraction and quality assessment, statistical analysis, drafted the manuscript, and is responsible for the overall content as the guarantor. ET: study design, statistical analysis, revised the manuscript for important intellectual content. W-HL: study design, data extraction and quality assessment, revised the manuscript for important intellectual content. ZC: data extraction and quality assessment, revised the manuscript for important intellectual content. KS: data extraction and quality assessment, revised the manuscript for important intellectual content. JS: study design, data extraction and quality assessment, revised the manuscript for important intellectual content. JF: study design, data extraction and quality assessment, revised the manuscript for important intellectual content.
Funding Department of Health, Australian Government, National Health and Medical Research Council (APP1091657).
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All data used for the study are available as online supplementary information and at https://doi.org/10.6084/m9.figshare.6244058.v2.
Linked Articles
- Airwaves