Skip to main content

Advertisement

Log in

Improving the diagnostic accuracy of a stratified screening strategy by identifying the optimal risk cutoff

  • Original Paper
  • Published:
Cancer Causes & Control Aims and scope Submit manuscript

Abstract

Background

The American Cancer Society (ACS) suggests using a stratified strategy for breast cancer screening. The strategy includes assessing risk of breast cancer, screening women at high risk with both MRI and mammography, and screening women at low risk with mammography alone. The ACS chose their cutoff for high risk using expert consensus.

Methods

We propose instead an analytic approach that maximizes the diagnostic accuracy (AUC/ROC) of a risk-based stratified screening strategy in a population. The inputs are the joint distribution of screening test scores, and the odds of disease, for the given risk score. Using the approach for breast cancer screening, we estimated the optimal risk cutoff for two different risk models: the Breast Cancer Screening Consortium (BCSC) model and a hypothetical model with much better discriminatory accuracy. Data on mammography and MRI test score distributions were drawn from the Magnetic Resonance Imaging Screening Study Group.

Results

A risk model with an excellent discriminatory accuracy (c-statistic \(= 0.947\)) yielded a reasonable cutoff where only about 20% of women had dual screening. However, the BCSC risk model (c-statistic \(= 0.631\)) lacked the discriminatory accuracy to differentiate between women who needed dual screening, and women who needed only mammography.

Conclusion

Our research provides a general approach to optimize the diagnostic accuracy of a stratified screening strategy in a population, and to assess whether risk models are sufficiently accurate to guide stratified screening. For breast cancer, most risk models lack enough discriminatory accuracy to make stratified screening a reasonable recommendation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Gareen IF, Gatsonis C, Marcus PM, Sicks JD (2011) Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med 365(5):395–409

    Article  PubMed  Google Scholar 

  2. Amir E, Evans DG, Shenton A, Lalloo F, Moran A, Boggis C, Wilson M, Howell A (2003) Evaluation of breast cancer risk assessment packages in the family history evaluation and screening programme. J Med Genet 40(11):807–814

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Antoniou AC, Pharoah PPD, Smith P, Easton DF (2004) The BOADICEA model of genetic susceptibility to breast and ovarian cancer. Br J Cancer 91(8):1580–1590. https://doi.org/10.1038/sj.bjc.6602175

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  4. Armstrong AC, Evans GD (2014) Management of women at high risk of breast cancer. BMJ 348(apr28 26):g2756–g2756. https://doi.org/10.1136/bmj.g2756

    Article  CAS  PubMed  Google Scholar 

  5. Atkin WS, Edwards R, Kralj-Hans I, Wooldrage K, Hart AR, Northover JMA, Parkin DM, Wardle J, Duffy SW, Cuzick J, UK Flexible Sigmoidoscopy Trial Investigators (2010) Once-only flexible sigmoidoscopy screening in prevention of colorectal cancer: a multicentre randomised controlled trial. Lancet (London, England) 375(9726):1624–1633. https://doi.org/10.1016/S0140-6736(10)60551-X

    Article  Google Scholar 

  6. Baker SG (2000) Identifying combinations of cancer markers for further study as triggers of early intervention. Biometrics 56(4):1082–1087

    Article  CAS  PubMed  Google Scholar 

  7. Barlow WE, White E, Ballard-Barbash R, Vacek PM, Titus-Ernstoff L, Carney PA, Tice JA, Buist DSM, Geller BM, Rosenberg R, Yankaskas BC, Kerlikowske K (2006) Prospective breast cancer risk prediction model for women undergoing screening mammography. J Natl Cancer Inst 98(17):1204–1214. https://doi.org/10.1093/jnci/djj331

    Article  PubMed  Google Scholar 

  8. Berry DA, Iversen ES Jr, Gudbjartsson DF, Hiller EH, Garber JE, Peshkin BN, Lerman C, Watson P, Lynch HT, Hilsenbeck SG, Rubinstein WS, Hughes KS, Parmigiani G (2002) BRCAPRO validation, sensitivity of genetic testing of BRCA1/BRCA2, and prevalence of other breast cancer susceptibility genes. J Clin Oncol 20(11):2701–2712

    Article  CAS  PubMed  Google Scholar 

  9. Brawley O, Byers T, Chen A, Pignone M, Ransohoff D, Schenk M, Smith R, Sox H, Thorson AG, Wender R (2011) New American Cancer Society process for creating trustworthy cancer screening guidelines. J Am Med Assoc 306(22):2495–2499

    Article  CAS  Google Scholar 

  10. Claus E (2000) Risk models in genetic epidemiology. Stat Methods Med Res 9(6):589–601

    Article  CAS  PubMed  Google Scholar 

  11. D’Souza G, Pawlita M, Westra WH (2007) Case–control study of human papillomavirus and oropharyngeal cancer. N Engl J Med 356(19):1944–1956

    Article  PubMed  Google Scholar 

  12. Elmore JG, Barton MB, Moceri VM, Polk S, Arena PJ, Fletcher SW (1998) Ten-year risk of false positive screening mammograms and clinical breast examinations. N Engl J Med 338(16):1089–1096

    Article  CAS  PubMed  Google Scholar 

  13. Gail MH, Pfeiffer RM (2005) On criteria for evaluating models of absolute risk. Biostatistics 6(2):227–239. https://doi.org/10.1093/biostatistics/kxi005

    Article  PubMed  Google Scholar 

  14. Hagen AI, Kvistad KA, Maehle L, Holmen MM, Aase H, Styr B, Vabø A, Apold J, Skaane P, Møller P (2007) Sensitivity of MRI versus conventional screening in the diagnosis of BRCA-associated breast cancer in a national prospective series. Breast (Edinburgh, Scotland) 16(4):367–374. https://doi.org/10.1016/j.breast.2007.01.006

    Article  Google Scholar 

  15. Hartman AR, Daniel BL, Kurian AW, Mills MA, Nowels KW, Dirbas FM, Kingham KE, Chun NM, Herfkens RJ, Ford JM, Plevritis SK (2004) Breast magnetic resonance image screening and ductal lavage in women at high genetic risk for breast carcinoma. Cancer 100(3):479–489. https://doi.org/10.1002/cncr.11926

    Article  PubMed  Google Scholar 

  16. Hendrick RE, Smith RA, Rutledge JH, Smart CR (1997) Benefit of screening mammography in women aged 40–49: a new meta-analysis of randomized controlled trials. J Natl Cancer Inst 22:87–92

    Article  Google Scholar 

  17. Hosmer DW, Lemeshow S (2000) Applied logistic regression, 2nd edn. Wiley series in probability and statistics. Wiley, New York

    Book  Google Scholar 

  18. Kriege M, Brekelmans CTM, Boetes C, Besnard PE, Zonderland HM, Obdeijn IM, Manoliu RA, Kok T, Peterse H, Tilanus-Linthorst MMA, Muller SH, Meijer S, Oosterwijk JC, Beex LVAM, Tollenaar RAEM, de Koning HJ, Rutgers EJT, Klijn JGM (2004) Efficacy of MRI and mammography for breast-cancer screening in women with a familial or genetic predisposition. N Engl J Med 351(5):427–437

    Article  CAS  PubMed  Google Scholar 

  19. Kuhl CK, Schrading S, Leutner CC, Morakkabati-Spitz N, Wardelmann E, Fimmers R, Kuhn W, Schild HH (2005) Mammography, breast ultrasound, and magnetic resonance imaging for surveillance of women at high familial risk for breast cancer. J Clin Oncol 23(33):8469–8476

    Article  PubMed  Google Scholar 

  20. Leach MO, Boggis CRM, Dixon AK, Easton DF, Eeles RA, Evans DGR, Gilbert FJ, Griebsch I, Hoff RJC, Kessar P, Lakhani SR, Moss SM, Nerurkar A, Padhani AR, Pointon LJ, Thompson D, Warren RML (2005) Screening with magnetic resonance imaging and mammography of a UK population at high familial risk of breast cancer. Lancet 365(9473):1769–1778

    Article  CAS  PubMed  Google Scholar 

  21. Lehman CD (2012) Diffusion weighted imaging (DWI) of the breast: ready for clinical practice. Eur J Radiol 81(Suppl 1):S80–81. https://doi.org/10.1016/S0720-048X(12)70032-3

    Article  PubMed  Google Scholar 

  22. Lehman CD, Blume JD, Weatherall P, Thickman D, Hylton N, Warner E, Pisano E, Schnitt SJ, Gatsonis C, Schnall M, DeAngelis GA, Stomper P, Rosen EL, O’Loughlin M, Harms S, Bluemke DA (2005) Screening women at high risk for breast cancer with mammography and magnetic resonance imaging. Cancer 103(9):1898–1905

    Article  PubMed  Google Scholar 

  23. Ma H, Bandos AI, Gur D (2015) On the use of partial area under the ROC curve for comparison of two diagnostic tests. Biom J 57(2):304–320. https://doi.org/10.1002/bimj.201400023

    Article  PubMed  Google Scholar 

  24. Ma H, Bandos AI, Rockette HE, Gur D (2013) On use of partial area under the ROC curve for evaluation of diagnostic performance. Stat Med 32(20):3449–3458. https://doi.org/10.1002/sim.5777

    Article  PubMed  PubMed Central  Google Scholar 

  25. McFarland EG, Levin B, Lieberman DA, Pickhardt PJ, Johnson CD, Glick SN, Brooks D, Smith RA (2008) Revised colorectal screening guidelines: joint effort of the American Cancer Society, U.S. Multisociety Task Force on Colorectal Cancer, and American College of Radiology. Radiology 248(3):717–720

    Article  PubMed  Google Scholar 

  26. Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P (2004) Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol 159(9):882–890. https://doi.org/10.1093/aje/kwh101

    Article  PubMed  Google Scholar 

  27. Pisano ED, Gatsonis C, Hendrick E, Yaffe M, Baum JK, Acharyya S, Conant EF, Fajardo LL, Bassett L, D’Orsi C, Jong R, Rebner M (2005) Diagnostic performance of digital versus film mammography for breast-cancer screening. N Engl J Med 353(17):1773–1783

    Article  CAS  PubMed  Google Scholar 

  28. Quante AS, Whittemore AS, Shriver T, Hopper JL, Strauch K, Terry MB (2015) Practical problems with clinical guidelines for breast cancer prevention based on remaining lifetime risk. J Natl Cancer Inst 107(7):djv124. https://doi.org/10.1093/jnci/djv124

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Ross S (1984) First course in probability, 2nd edn. Macmillan Publishing Company, New York

    Google Scholar 

  30. Sankaranarayanan R, Ramadas K, Thomas G, Muwonge R, Thara S, Mathew B, Rajan B (2005) Effect of screening on oral cancer mortality in Kerala, India: a cluster-randomised controlled trial. Lancet 365(9475):1927–1933

    Article  Google Scholar 

  31. Sardanelli F, Aase HS, Álvarez M, Azavedo E, Baarslag HJ, Balleyguier C, Baltzer PA, Beslagic V, Bick U, Bogdanovic-Stojanovic D, Briediene R, Brkljacic B, Camps Herrero J, Colin C, Cornford E, Danes J, de Geer G, Esen G, Evans A, Fuchsjaeger MH, Gilbert FJ, Graf O, Hargaden G, Helbich TH, Heywang-Köbrunner SH, Ivanov V, Jónsson Á, Kuhl CK, Lisencu EC, Luczynska E, Mann RM, Marques JC, Martincich L, Mortier M, Müller-Schimpfle M, Ormandi K, Panizza P, Pediconi F, Pijnappel RM, Pinker K, Rissanen T, Rotaru N, Saguatti G, Sella T, Slobodníková J, Talk M, Taourel P, Trimboli RM, Vejborg I, Vourtsis A, Forrai G (2017) Position paper on screening for breast cancer by the European Society of Breast Imaging (EUSOBI) and 30 national breast radiology bodies from Austria, Belgium, Bosnia and Herzegovina, Bulgaria, Croatia, Czech Republic, Denmark, Estonia, Finland, France, Germany, Greece, Hungary, Iceland, Ireland, Italy, Israel, Lithuania, Moldova, The Netherlands, Norway, Poland, Portugal, Romania, Serbia, Slovakia, Spain, Sweden, Switzerland and Turkey. Eur Radiol 27(7):2737–2743. https://doi.org/10.1007/s00330-016-4612-z

    Article  PubMed  Google Scholar 

  32. Sardanelli F, Podo F, Santoro F, Manoukian S, Bergonzi S, Trecate G, Vergnaghi D, Federico M, Cortesi L, Corcione S, Morassut S, Di Maggio C, Cilotti A, Martincich L, Calabrese M, Zuiani C, Preda L, Bonanni B, Carbonaro LA, Contegiacomo A, Panizza P, Di Cesare E, Savarese A, Crecco M, Turchetti D, Tonutti M, Belli P, Maschio AD (2011) High breast cancer risk Italian 1 (HIBCRIT-1) Study: multicenter surveillance of women at high genetic breast cancer risk using mammography, ultrasonography, and contrast-enhanced magnetic resonance imaging (the high breast cancer risk italian 1 study): final results. Investig Radiol 46(2):94–105. https://doi.org/10.1097/RLI.0b013e3181f3fcdf

    Article  Google Scholar 

  33. Saslow D, Boetes C, Burke W, Harms S, Leach MO, Lehman CD, Morris E, Pisano E, Schnall M, Sener S, Smith RA, Warner E, Yaffe M, Andrews KS, Russell CA (2007) American Cancer Society guidelines for breast screening with MRI as an adjunct to mammography. CA 57(2):75–89

    PubMed  Google Scholar 

  34. Sickles EA, D’Orsi CJ, Bassett LW (2013) ACR BI-RADS—mammography

  35. Smith RA, Andrews K, Brooks D, DeSantis CE, Fedewa SA, Lortet-Tieulent J, Manassaram-Baptiste D, Brawley OW, Wender RC (2016) Cancer screening in the United States, 2016: a review of current American Cancer Society guidelines and current issues in cancer screening. CA 66(2):96–114. https://doi.org/10.3322/caac.21336

    Article  PubMed  Google Scholar 

  36. Smith RA, Cokkinides V, Brawley OW (2012) Cancer screening in the United States, 2012: a review of current American Cancer Society guidelines and current issues in cancer screening. CA 62(2):129–142

    PubMed  Google Scholar 

  37. Thisted RA (1988) Elements of statistical computing: numerical computation, 1st edn. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  38. Tosteson ANA, Stout NK, Fryback DG, Acharyya S, Herman BA, Hannah LG, Pisano ED, DMIST Investigators (2008) Cost-effectiveness of digital mammography breast cancer screening. Ann Intern Med 148(1):1–10

    Article  PubMed  PubMed Central  Google Scholar 

  39. Tyrer J, Duffy SW, Cuzick J (2004) A breast cancer prediction model incorporating familial and personal risk factors. Stat Med 23(7):1111–1130

    Article  PubMed  Google Scholar 

  40. Wald NJ, Hackshaw AK, Frost CD (1999) When can a risk factor be used as a worthwhile screening test? BMJ 319(7224):1562–1565. https://doi.org/10.1136/bmj.319.7224.1562

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  41. Wang Z, Luo X, Chang YcI (2015) Assessing the predictive power of newly added biomarkers. Biom J 57(5):797–807. https://doi.org/10.1002/bimj.201400210

    Article  PubMed  Google Scholar 

  42. Warner E (2008) The role of magnetic resonance imaging in screening women at high risk of breast cancer. Top Magn Reson Imaging 19(3):163–169. https://doi.org/10.1097/RMR.0b013e31818bc994

    Article  PubMed  Google Scholar 

  43. Warner E, Plewes DB, Hill KA, Causer PA, Zubovits JT, Jong RA, Cutrara MR, DeBoer G, Yaffe MJ, Messner SJ, Meschino WS, Piron CA, Narod SA (2004) Surveillance of BRCA1 and BRCA2 mutation carriers with magnetic resonance imaging, ultrasound, mammography, and clinical breast examination. JAMA 292(11):1317–1325. https://doi.org/10.1001/jama.292.11.1317

    Article  CAS  PubMed  Google Scholar 

  44. Yabuuchi H, Matsuo Y, Sunami S, Kamitani T, Kawanami S, Setoguchi T, Sakai S, Hatakenaka M, Kubo M, Tokunaga E, Yamamoto H, Honda H (2011) Detection of non-palpable breast cancer in asymptomatic women by using unenhanced diffusion-weighted and T2-weighted MR imaging: comparison with mammography and dynamic contrast-enhanced MR imaging. Eur Radiol 21(1):11–17. https://doi.org/10.1007/s00330-010-1890-8

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

This manuscript was submitted to the Department of Biostatistics and Informatics in the Colorado School of Public Health, University of Colorado Denver, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Biostatistics for JTB. Partial funding for DHG was provided by a generous grant from the Lundbeck Foundation, who provided a visiting professorship to the University of Copenhagen. The authors thank the BCSC investigators, participating mammography facilities, and radiologists who provided the relevant data for this study. A list of the BCSC investigators and procedures for requesting BCSC data for research purposes are provided at: http://breastscreening.cancer.gov/.

Funding

Funding was provided by Lundbeckfonden, National Cancer Institute (Grant Nos. 5K07CA088811, 1R03CA136048-01A1), and National Institute of Dental and Craniofacial Research (Grant No. RC2DE020779).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to John T. Brinton.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Data collection and sharing for the BCSC was supported by the National Cancer Institute (U01CA63740, U01CA86076, U01CA86082, U01CA63736, U01CA70013, U01CA69976, U01CA63731, U01CA70040, HHSN261201100031C).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Brinton, J.T., Hendrick, R.E., Ringham, B.M. et al. Improving the diagnostic accuracy of a stratified screening strategy by identifying the optimal risk cutoff. Cancer Causes Control 30, 1145–1155 (2019). https://doi.org/10.1007/s10552-019-01208-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10552-019-01208-9

Keywords

Navigation