Clinical opinion
Education
The use of a comparability scoring system in reporting observational studies

https://doi.org/10.1016/j.ajog.2013.09.002Get rights and content

The traditional statistical analyses with adjustment for confounders in observational studies assume that there is perfect similarity in the already-provided medical management between the comparison groups. However, variations in medical management frequently exist because of differences in circumstances of health care. We propose that to minimize the selection bias of observational studies, the degree of similarity or dissimilarity of the comparison groups regarding the circumstances of health care should be considered. Circumstances of health care include the geographic setting, health care setting, type of health care providers, and likelihood in having confounding introduced by differences in the medical management between comparison groups. We propose a comparability scoring system of circumstances of care and provide examples of the application of this system, using recent literature to assess comparability among study groups. In our examples, the presupposed statistical associations disappeared once the analyses accounted for the differences in circumstances of care. Authors of submitted manuscripts using an observational study design may consider incorporating our scoring system or an equivalent in their methods and in reporting of the results. The comparability score should be factored during statistical analysis so that the appropriate analysis can correct for differences in circumstances of care. The use of a comparability scoring system can provide important insights for reviewers and readers that will improve the interpretation of this type of research study.

Section snippets

Comparability scoring criteria

Ideally, comparative analyses should take into consideration the following criteria regarding circumstances of health care. The definition of the criteria is shown in Table 1.

Checklist-based scoring system

We propose a scoring checklist system (Table 1) to evaluate the similarity or comparability of baseline characteristics between the comparison groups, thus allowing for more valid conclusions.

The first 4 criteria may be used for all comparative analyses (maximum score 8), whereas all 6 criteria may be used for historical control studies (maximum score 12). If both the historical control and study groups have the same geographic setting (score 2), the same health care setting (score 2), the same

Examples and statistical analysis

We propose that comparative analyses should properly account for differences in the comparability score (Table 1). In a regression framework to assess the efficacy of the treatment on the outcome, several options are available and one of them is a simple weighting of the analysis by the comparability score, derived from the criteria shown in Table 1. The degree of correction should be inversely related to the total score; weighting the analysis based on the comparability score may have no

Conclusions

We have described a comparability-based scoring system for circumstances of health care to determine the degree of comparability between comparison groups in observational studies. Using examples from the published literature, we showed that this scoring system is useful in assessing baseline similarities or dissimilarities between comparison groups in observational studies or studies using historical controls that then can be used to correct the statistical analysis and produce valid

References (21)

There are more references available in the full text version of this article.

Cited by (0)

The authors report no conflict of interest.

Reprints not available from the authors.

View full text
Copyright © 2014 Mosby, Inc. All rights reserved.