Chest
Critical CarePredicting Outcome after ICU Admission: The Art and Science of Assessing Risk
Section snippets
Creating a Predictive Instrument
Most studies of ICU outcome, including those involving APACHE or MPM, attempt to identify “risk factors” or other “predictors” of outcome (Table 1). Usually a specific subset of ICU patients is defined, and potential predictors are evaluated for their association, if any, with a particular outcome. Studies involving the adult respiratory distress syndrome (ARDS), septic shock or multiorgan system failure, nontraumatic coma, cardiopulmonary resuscitation, cancer requiring intensive care support,
Patient and Outcome Selection
Since all studies of ICU outcome focus on just a subset of the total ICU patient population, there is an implicit assumption that the most powerful set of predictors will be specific for a particular type of patient or problem. Interestingly, more general systems like APACHE and MPM challenge this notion, which remains one of the most controversial aspects of their development. Even these systems, however, have excluded some patient groups. For instance, burn patients, patients under 16 years
Variable Selection and Data Collection
Although hundreds of variables can be measured in the ICU, only a subset can be evaluated in any one study. Usually a combination of demographic, clinical, and laboratory variables (eg, age, sex, primary diagnosis, physical signs, blood gas values, and electrolyte concentrations) are included. In effect, the choice of variables reflects a (sometimes unstated) hypothesis that some subset will be related to outcome. For instance, the developers of the APACHE system17, 18 stated that their
Data Analysis
To reduce the number of variables finally evaluated for their association with a particular outcome, APACHE,5, 17, 18 the Simplified Acute Physiology Score (SAPS),19 and the Therapeutic Intervention Scoring System (TISS)20, 21 all summarize a large set of variables into a single “score.” In each case, the score represents the sum of values (“weights”) assigned to the chosen predictors. For continuous (ie, physiologic) variables, selected ranges are defined. The weights assigned to each range
Relating Predictors to Outcome
Regression techniques are often used to relate predictors to outcome. For multiple-variable (multivariate) analyses, an equation (ie, a “model”) such as describes a linear relationship where y is the dependent, “response,” or “outcome” variable; x1 through xi are the individual predictors; b1 through bi are coefficients (analogous to a slope in a simple linear regression model for one independent variable); and b0 is a constant (the y intercept in a simple linear
Validating the Instrument
Predictive models can be validated by comparing the model's predictions, derived from a “training” data base, against the actual observed outcome in a test set. The test group can be the group from which the original model was derived (compared by so-called cross-validation techniques), a portion of the group from which the data were originally collected but from which data were not used to develop the model (the split-sample method), or a completely new sample.24 The last approach, although
Impact of the Instrument
Predictive instruments can be used for research purposes to show that the groups being studied (eg, a test group and a placebo group) are similar with respect to baseline severity of disease. Presumably, similar predicted risks would imply similar severity of disease at baseline. However, a common error in using APACHE (and probably other scoring systems as well) in clinical research studies has been to simply report the raw scores for the different study groups, when those groups include
Updates
Many sets of predictors fail to perform well in subsequent studies.44 In addition to the issues already discussed, changes in therapy may have altered the nature of the ICU population (eg, patients being admitted with new complications not previously encountered in that group) or may have altered the prognosis of either the acute problem or the underlying disease. This problem can be addressed only by periodic updates of the data base, either verifying previous results or modifying the
Conclusions
It is quite clear that clinicians, administrators, and regulators would like an accurate predictive instrument against which to judge and evaluate clinical effectiveness, efficiency, and quality. Numerous isolated studies that report predictors of ICU outcome for selected patient groups have been of little value because they have not been appropriately validated. Nevertheless, in toto, they demonstrate the importance of chronic health, the nature and severity of the acute illness, the response
ACKNOWLEDGMENTS
The author wishes to gratefully acknowledge the helpful comments of Drs Clay Dunagan and Michael Kahn, and the statistical review provided by Dr Michael Province.
A more complete list of references, especially pertaining to the outcome of specific ICU patient populations, is available from the author.
References (62)
- et al.
The APACHE III prognostic system: risk prediction of hospital mortality for critically ill hospitalized adults
Chest
(1991) - et al.
Determinants of weaning and survival among patients with COPD who require mechanical ventilation for acute respiratory failure
Chest
(1989) - et al.
Importance of epidemiology and biostatistics in deciding clinical strategies for using diagnostic tests: a simplified approach using examples from coronary artery disease
J Am Coll Cardiol
(1989) - et al.
Disease severity in the coronary care unit
Chest
(1991) Variability in physician estimates of survival for acute respiratory failure in chronic obstructive pulmonary disease
Chest
(1987)- et al.
Patterns of resource consumption in medical intensive care
Chest
(1991) - et al.
Medical intensive care: indications, interventions, and outcomes
N Engl J Med
(1980) Consensus Development Conference
(1983)- et al.
Critical care units
Severity standardization and hospital quality assessment
Prognosis with mechanical ventilation: the influence of disease, severity of disease, age, and chronic health status on survival from an acute illness
Am Rev Respir Dis
Outcome prediction in adult intensive care
Intensive care unit survival of patients with systemic illness
Am Rev Respir Dis
Evaluating the severity of illness of critically ill patients
Prediction and definition of outcome in a cost-sensitive era
Survival data in patients with acute and chronic lung disease requiring mechanical ventilation
Am Rev Respir Dis
Adult respiratory distress syndrome: prognosis after onset
Am Rev Respir Dis
A method for predicting survival and mortality of ICU patients using objectively derived weights
Crit Care Med
Refining intensive care unit outcome prediction by using changing probabilities of mortality
Crit Care Med
Clinical prediction rules: applications and methodological standards
N Engl J Med
APACHE—acute physiology and chronic health evaluation: a physiologically based classification system
Crit Care Med
APACHE II: a severity of disease classification system
Crit Care Med
A simplified acute physiology score for ICU patients
Crit Care Med
Therapeutic intervention scoring system: a method for quantitative comparison of patient care
Crit Care Med
Therapeutic intervention scoring system: update 1983
Crit Care Med
A comparison of methods to predict mortality of intensive care unit patients
Crit Care Med
Statistical methods (APACHE-IIII study design)
Crit Care Med
Severity-of-illness modeling
Why predictive indexes perform less well in validation studies
Arch Intern Med
Evaluation of APACHE II for cost containment and quality assurance
Ann Surg
Cited by (76)
A model for identifying patients who may not need intensive care unit admission
2010, Journal of Critical CareCitation Excerpt :For categorical variables, the χ2 test was used to look for significant associations. Assessment of a model's accuracy in an out-of-sample patient population is desirable because predictive models tend to be over specific for the database used in their development [24,27,28]. To test the accuracy of the model based on the 2002 to 2003 data set, we compared the observed and predicted number of ICU day 1 monitor patients who received subsequent active treatment in a new cohort of patients.
Severity Scoring Systems: Tools for the Evaluation of Patients and Intensive Care Units
2008, Critical Care Medicine: Principles of Diagnosis and Management in the AdultOptimal prediction of mortality after abdominal aortic aneurysm repair with statistical models
2006, Journal of Vascular SurgeryImproving the ICU: Part 1
2005, ChestCitation Excerpt :The SMR is, thus, a death rate that is case mix-adjusted by comparison with the original inception group.88,183,184 Although unable to predict individual outcomes with sufficient accuracy to evaluate care for individual patients,190191 the SMR performs reasonably well for patient cohorts.183,185,192193,194,195 Other than short-term mortality, the list of performance measures for which validated prediction equations allow case-mix adjustments is brief.
The injury scale - A valuable tool for forensic documentation of trauma
2005, Journal of Clinical Forensic Medicine