Chest
Volume 102, Issue 6, December 1992, Pages 1861-1870
Journal home page for Chest

Critical Care
Predicting Outcome after ICU Admission: The Art and Science of Assessing Risk

https://doi.org/10.1378/chest.102.6.1861Get rights and content

Section snippets

Creating a Predictive Instrument

Most studies of ICU outcome, including those involving APACHE or MPM, attempt to identify “risk factors” or other “predictors” of outcome (Table 1). Usually a specific subset of ICU patients is defined, and potential predictors are evaluated for their association, if any, with a particular outcome. Studies involving the adult respiratory distress syndrome (ARDS), septic shock or multiorgan system failure, nontraumatic coma, cardiopulmonary resuscitation, cancer requiring intensive care support,

Patient and Outcome Selection

Since all studies of ICU outcome focus on just a subset of the total ICU patient population, there is an implicit assumption that the most powerful set of predictors will be specific for a particular type of patient or problem. Interestingly, more general systems like APACHE and MPM challenge this notion, which remains one of the most controversial aspects of their development. Even these systems, however, have excluded some patient groups. For instance, burn patients, patients under 16 years

Variable Selection and Data Collection

Although hundreds of variables can be measured in the ICU, only a subset can be evaluated in any one study. Usually a combination of demographic, clinical, and laboratory variables (eg, age, sex, primary diagnosis, physical signs, blood gas values, and electrolyte concentrations) are included. In effect, the choice of variables reflects a (sometimes unstated) hypothesis that some subset will be related to outcome. For instance, the developers of the APACHE system17, 18 stated that their

Data Analysis

To reduce the number of variables finally evaluated for their association with a particular outcome, APACHE,5, 17, 18 the Simplified Acute Physiology Score (SAPS),19 and the Therapeutic Intervention Scoring System (TISS)20, 21 all summarize a large set of variables into a single “score.” In each case, the score represents the sum of values (“weights”) assigned to the chosen predictors. For continuous (ie, physiologic) variables, selected ranges are defined. The weights assigned to each range

Relating Predictors to Outcome

Regression techniques are often used to relate predictors to outcome. For multiple-variable (multivariate) analyses, an equation (ie, a “model”) such as y=b0+b1x1+bxx2bixi describes a linear relationship where y is the dependent, “response,” or “outcome” variable; x1 through xi are the individual predictors; b1 through bi are coefficients (analogous to a slope in a simple linear regression model for one independent variable); and b0 is a constant (the y intercept in a simple linear

Validating the Instrument

Predictive models can be validated by comparing the model's predictions, derived from a “training” data base, against the actual observed outcome in a test set. The test group can be the group from which the original model was derived (compared by so-called cross-validation techniques), a portion of the group from which the data were originally collected but from which data were not used to develop the model (the split-sample method), or a completely new sample.24 The last approach, although

Impact of the Instrument

Predictive instruments can be used for research purposes to show that the groups being studied (eg, a test group and a placebo group) are similar with respect to baseline severity of disease. Presumably, similar predicted risks would imply similar severity of disease at baseline. However, a common error in using APACHE (and probably other scoring systems as well) in clinical research studies has been to simply report the raw scores for the different study groups, when those groups include

Updates

Many sets of predictors fail to perform well in subsequent studies.44 In addition to the issues already discussed, changes in therapy may have altered the nature of the ICU population (eg, patients being admitted with new complications not previously encountered in that group) or may have altered the prognosis of either the acute problem or the underlying disease. This problem can be addressed only by periodic updates of the data base, either verifying previous results or modifying the

Conclusions

It is quite clear that clinicians, administrators, and regulators would like an accurate predictive instrument against which to judge and evaluate clinical effectiveness, efficiency, and quality. Numerous isolated studies that report predictors of ICU outcome for selected patient groups have been of little value because they have not been appropriately validated. Nevertheless, in toto, they demonstrate the importance of chronic health, the nature and severity of the acute illness, the response

ACKNOWLEDGMENTS

The author wishes to gratefully acknowledge the helpful comments of Drs Clay Dunagan and Michael Kahn, and the statistical review provided by Dr Michael Province.

A more complete list of references, especially pertaining to the outcome of specific ICU patient populations, is available from the author.

First page preview

First page preview
Click to open first page preview

References (62)

  • WA Knaus

    Prognosis with mechanical ventilation: the influence of disease, severity of disease, age, and chronic health status on survival from an acute illness

    Am Rev Respir Dis

    (1989)
  • JE Zimmerman et al.

    Outcome prediction in adult intensive care

  • TA Raffin

    Intensive care unit survival of patients with systemic illness

    Am Rev Respir Dis

    (1989)
  • D Teres et al.

    Evaluating the severity of illness of critically ill patients

  • J Civetta

    Prediction and definition of outcome in a cost-sensitive era

  • LD Hudson

    Survival data in patients with acute and chronic lung disease requiring mechanical ventilation

    Am Rev Respir Dis

    (1989)
  • AA Fowler et al.

    Adult respiratory distress syndrome: prognosis after onset

    Am Rev Respir Dis

    (1985)
  • S Lemeshow et al.

    A method for predicting survival and mortality of ICU patients using objectively derived weights

    Crit Care Med

    (1985)
  • S Lemeshow et al.

    Refining intensive care unit outcome prediction by using changing probabilities of mortality

    Crit Care Med

    (1988)
  • JH Wasson et al.

    Clinical prediction rules: applications and methodological standards

    N Engl J Med

    (1986)
  • WA Knaus et al.

    APACHE—acute physiology and chronic health evaluation: a physiologically based classification system

    Crit Care Med

    (1981)
  • WA Knaus et al.

    APACHE II: a severity of disease classification system

    Crit Care Med

    (1985)
  • JR Gall et al.

    A simplified acute physiology score for ICU patients

    Crit Care Med

    (1984)
  • DJ Cullen et al.

    Therapeutic intervention scoring system: a method for quantitative comparison of patient care

    Crit Care Med

    (1974)
  • AR Keene et al.

    Therapeutic intervention scoring system: update 1983

    Crit Care Med

    (1983)
  • S Lemeshow et al.

    A comparison of methods to predict mortality of intensive care unit patients

    Crit Care Med

    (1987)
  • D Wagner et al.

    Statistical methods (APACHE-IIII study design)

    Crit Care Med

    (1989)
  • Ruttimann UE. Severity of illness indices: development and evaluation. In: Shoemaker WC, Ayres S, Grenvik A, Holbrook...
  • D Teres et al.

    Severity-of-illness modeling

  • ME Charlson et al.

    Why predictive indexes perform less well in validation studies

    Arch Intern Med

    (1987)
  • JM Civetta et al.

    Evaluation of APACHE II for cost containment and quality assurance

    Ann Surg

    (1990)
  • Cited by (76)

    • A model for identifying patients who may not need intensive care unit admission

      2010, Journal of Critical Care
      Citation Excerpt :

      For categorical variables, the χ2 test was used to look for significant associations. Assessment of a model's accuracy in an out-of-sample patient population is desirable because predictive models tend to be over specific for the database used in their development [24,27,28]. To test the accuracy of the model based on the 2002 to 2003 data set, we compared the observed and predicted number of ICU day 1 monitor patients who received subsequent active treatment in a new cohort of patients.

    • Severity Scoring Systems: Tools for the Evaluation of Patients and Intensive Care Units

      2008, Critical Care Medicine: Principles of Diagnosis and Management in the Adult
    • Improving the ICU: Part 1

      2005, Chest
      Citation Excerpt :

      The SMR is, thus, a death rate that is case mix-adjusted by comparison with the original inception group.88,183,184 Although unable to predict individual outcomes with sufficient accuracy to evaluate care for individual patients,190191 the SMR performs reasonably well for patient cohorts.183,185,192193,194,195 Other than short-term mortality, the list of performance measures for which validated prediction equations allow case-mix adjustments is brief.

    View all citing articles on Scopus
    View full text