Skip to main content
SpringerLink
Log in

Radiation Safety Performance is More than Simply Measuring Doses! Development of a Radiation Safety Rating Scale

  • Scientific Paper (other)
  • Radiation Protection
  • Published:
CardioVascular and Interventional Radiology Aims and scope Submit manuscript

Abstract

Purpose

Radiation safety performance is often evaluated using dose parameters measured by personal dosimeters and/or the C-arm, which provide limited information about teams’ actual radiation safety behaviors. This study aimed to develop a rating scale to evaluate team radiation safety behaviors more accurately and investigate its reliability.

Materials and Methods

A modified Delphi consensus was organized involving European vascular surgeons (VS), interventional radiologists, and interventional cardiologists. Initial items and anchors were drafted a priori and rated using five-point Likert scales. Participants could suggest additional items or adjustments. Consensus was defined as ≥ 80% agreement (rating ≥ 4) with Cronbach’s alpha ≥ .80. Two VS with expertise in radiation safety evaluated 15 video-recorded endovascular repairs of infrarenal aortic aneurysms (EVAR) to assess usability, inter and intrarater reliability.

Results

Thirty-one of 46 invited specialists completed three rating rounds to generate the final rating scale. Five items underwent major adjustments. In the final round, consensus was achieved for all items (alpha = .804; agreement > 87%): ‘Pre-procedural planning’, ‘Preparation in angiosuite/operating room’, ‘Shielding equipment’, ‘Personal protective equipment’, ’Position of operator/team’, ‘Radiation usage awareness’, ‘C-arm handling’, ‘Adjusting image quality’, ‘Additional dose reducing functions’, ‘Communication/leadership’, and ‘Overall radiation performance and ALARA principle’. All EVARs were rated, yielding excellent Cronbach’s alpha (.877) with acceptable interrater and excellent intrarater reliability (ICC = .782; ICC = .963, respectively).

Conclusion

A reliable framework was developed to assess radiation safety behaviors in endovascular practice and provide teams with formative feedback. The final scale is provided in this publication.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Hertault A, Maurel B, Midulla M, et al. Editor’s choice—minimizing radiation exposure during endovascular procedures: basic knowledge, literature review, and reporting standards. Eur J Vasc Endovasc Surg. 2015;50(1):21–36.

    Article  CAS  PubMed  Google Scholar 

  2. Kendrick DE, Miller CP, Moorehead PA, et al. Comparative occupational radiation exposure between fixed and mobile imaging systems. J Vasc Surg. 2016;63(1):190–7.

    Article  PubMed  Google Scholar 

  3. Vano E, Rosenstein M, Liniecki J, Rehani MM, Martin CJ, Vetter RJ. ICRP Publication 113. Education and training in radiological protection for diagnostic and interventional procedures. Ann ICRP. 2009;39(5):7–68.

    Article  CAS  PubMed  Google Scholar 

  4. Bartal G, Vano E, Paulo G, Miller DL. Management of patient and staff radiation dose in interventional radiology: current concepts. Cardiovasc Intervent Radiol. 2014;37(2):289–98.

    Article  PubMed  Google Scholar 

  5. Tzanis E, Tsetis D, Kehagias E, Ioannou CV, Damilakis J. Occupational exposure during endovascular aneurysm repair (EVAR) and aortoiliac percutaneous transluminal angioplasty (PTA) procedures. Radiol Med. 2019;124(6):539–45.

    Article  PubMed  Google Scholar 

  6. Miller C, Kendrick D, Shevitz A, et al. Evaluating strategies for reducing scattered radiation in fixed-imaging hybrid operating suites. J Vasc Surg. 2018;67(4):1227–33.

    Article  PubMed  Google Scholar 

  7. Werner GS, Glaser P, Coenen A, et al. Reduction of radiation exposure during complex interventions for chronic total coronary occlusions: implementing low dose radiation protocols without affecting procedural success rates. Cathet Cardiovasc Interv. 2017;89(6):1005–12.

    Article  Google Scholar 

  8. de Ruiter QM, Moll FL, Gijsberts CM, van Herwaarden JA. AlluraClarity radiation dose–reduction technology in the hybrid operating room during endovascular aneurysm repair. J Endovasc Ther. 2016;23(1):130–8.

    Article  PubMed  Google Scholar 

  9. Spira D, Kirchner S, Blumenstock G, et al. Therapeutic angiographic procedures: differences in dose area product between analog image intensifier and digital flat panel detector. Acta Radiol (Stockholm, Sweden: 1987). 2016;57(5):587–94.

    Google Scholar 

  10. Hull L, Bicknell C, Patel K, et al. Content validation and evaluation of an endovascular teamwork assessment tool. Eur J Vasc Endovasc Surg. 2016;52(1):11–20.

    Article  CAS  PubMed  Google Scholar 

  11. Bech B, Lonn L, Schroeder TV, Rader SB, Ringsted C. Capturing the essence of developing endovascular expertise for the construction of a global assessment instrument. Eur J Vasc Endovasc Surg. 2010;40(3):292–302.

    Article  CAS  PubMed  Google Scholar 

  12. Strom M, Lonn L, Bech B, Schroeder TV, Konge L, Panel ED. Assessment of competence in EVAR procedures: a novel rating scale developed by the Delphi technique. Eur J Vasc Endovasc Surg. 2017;54(1):34–41.

    Article  CAS  PubMed  Google Scholar 

  13. Duran C, Estrada S, O’Malley M, et al. The model for Fundamentals of Endovascular Surgery (FEVS) successfully defines the competent endovascular surgeon. J Vasc Surg. 2015;62(6):1660–1666e1663.

    Article  PubMed  Google Scholar 

  14. Hopmans CJ, den Hoed PT, van der Laan L, et al. Assessment of surgery residents’ operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study. Surgery. 2014;156(5):1078–88.

    Article  PubMed  Google Scholar 

  15. Strom M, Lonn L, Konge L, et al. Assessment of EVAR competence: validity of a novel rating scale (EVARATE) in a simulated setting. Eur J Vasc Endovasc Surg. 2018;56(1):137–44.

    Article  PubMed  Google Scholar 

  16. Bordoli SJ, Carsten CG 3rd, Cull DL, Johnson BL, Taylor SM. Radiation safety education in vascular surgery training. J Vasc Surg. 2014;59(3):860–4.

    Article  PubMed  Google Scholar 

  17. Resch TA, Tornqvist P, Sonesson B, Dias NV. Techniques to reduce radiation for patients and operators during aortic endografting. J Cardiovasc Surg. 2016;57(2):178–84.

    Google Scholar 

  18. Shaw PV, Crouail P, Paynter R, Coeck M. Education and training in radiation protection: improving alara culture. J Radiol Prot. 2015;35(1):223–7.

    Article  PubMed  Google Scholar 

  19. Rehani MM, Ciraj-Bjelac O, Vano E, et al. ICRP Publication 117. Radiological protection in fluoroscopically guided procedures performed outside the imaging department. Ann ICRP. 2010;40(6):1–102.

    Article  CAS  PubMed  Google Scholar 

  20. Doyen B, Maurel B, Cole J, Maertens H, Mastracci T, Van Herzeele I. Defining the key competencies in radiation protection for endovascular procedures: a multispecialty delphi consensus study. Eur J Vasc Endovasc Surg. 2018;55(2):281–7.

    Article  PubMed  Google Scholar 

  21. Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol. 2005;5:37.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Tolsgaard MG, Todsen T, Sorensen JL, et al. International multispecialty consensus on how to evaluate ultrasound competence: a Delphi consensus survey. PLoS ONE. 2013;8(2):e57687.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  23. Palter VN, MacRae HM, Grantcharov TP. Development of an objective evaluation tool to assess technical skill in laparoscopic colorectal surgery: a Delphi methodology. Am J Surg. 2011;201(2):251–9.

    Article  PubMed  Google Scholar 

  24. de Villiers MR, de Villiers PJ, Kent AP. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639–43.

    Article  PubMed  Google Scholar 

  25. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–60.

    Article  CAS  PubMed  Google Scholar 

  26. Giavarina D. Understanding bland altman analysis. Biochem Med. 2015;25(2):141–51.

    Article  Google Scholar 

  27. Graham B, Regehr G, Wright JG. Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol. 2003;56(12):1150–6.

    Article  PubMed  Google Scholar 

  28. Greer T, Dunlap WP, Hunter ST, Berman ME. Skew and internal consistency. J Appl Psychol. 2006;91(6):1351–8.

    Article  PubMed  Google Scholar 

  29. Holey EA, Feeley JL, Dixon J, Whittaker VJ. An exploration of the use of simple statistics to measure consensus and stability in Delphi studies. BMC Med Res Methodol. 2007;7:52.

    Article  PubMed  PubMed Central  Google Scholar 

  30. de Ruiter QM, Gijsberts CM, Hazenberg CE, Moll FL, van Herwaarden JA. Radiation awareness for endovascular abdominal aortic aneurysm repair in the hybrid operating room An instant patient risk chart for daily practice. J Endovasc Ther. 2017;24(3):425–34.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hambleton RK, Jaeger RM, Plake BS, Mills C. Setting performance standards on complex educational assessments. Appl Psychol Meas. 2000;24(4):355–66.

    Article  Google Scholar 

  32. Sailer AM, Paulis L, Vergoossen L, et al. Real-time patient and staff radiation dose monitoring in IR practice. Cardiovasc Intervent Radiol. 2017;40(3):421–9.

    Article  PubMed  Google Scholar 

  33. Hertault A, Rhee R, Antoniou GA, et al. Radiation dose reduction during EVAR: results from a prospective multicentre study (the REVAR study). Eur J Vasc Endovasc Surg. 2018;56(3):426–33.

    Article  PubMed  Google Scholar 

  34. Jung JJ, Juni P, Lebovic G, Grantcharov T. First-year analysis of the operating room black box study. Ann Surg. 2020;271(1):122–7.

    Article  PubMed  Google Scholar 

  35. Martin JA, Regehr G, Reznick R, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273–8.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The members of the PRET group: B. Bech, L. Bertoglio, C. Bicknell, D. Böckler, M. Brodmann, E. Brountzos, T. Carrell, T. Cohnert, J.P. De Vries, F. Dick, R. Ferraresi, Y. Gouëffic, S. Haulon, C. Karkos, I. Končar, J. Lammer, Z. Martin, R. McWilliams, G. Melissano, S. Müller-Hülsbeck, C. Nienaber, T. Resch, V. Riambau, R. Williams, Z. Szeberin, J. Teijink, J. Van Den Berg, J. van Herwaarden, F. Vermassen, F. Verzini, A. Wanhainen. The authors wish to thank K. Bacher for reviewing the final radiation safety rating scale.

Funding

I. Van Herzeele is supported by a Senior Clinical Fellowship of the Fund for Scientific Research—Flanders, Belgium.

Author information

Authors and Affiliations

  1. Department of Thoracic and Vascular Surgery, Ghent University Hospital, 2K12D, Route 1024, Corneel Heymanslaan 10, 9000, Ghent, Belgium

    Bart Doyen & Isabelle Van Herzeele

  2. Department of Vascular Surgery, University Hospital Centre of Nantes, Nantes, France

    Blandine Maurel

  3. Department of Vascular Surgery, Valenciennes General Hospital, Valenciennes, France

    Adrien Hertault

  4. Department of Work, Organisation and Society, Ghent University, Ghent, Belgium

    Peter Vlerick

  5. Department of Vascular Surgery, Royal Free London NHS Foundation Trust, London, UK

    Tara Mastracci

Authors
  1. Bart Doyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Blandine Maurel
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Adrien Hertault
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Peter Vlerick
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tara Mastracci
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Isabelle Van Herzeele
    View author publications

    You can also search for this author in PubMed Google Scholar

Consortia

On behalf of: PRET (Principles of Radiation protection within Endovascular Team) group

  • B. Bech
  • , L. Bertoglio
  • , C. Bicknell
  • , D. Böckler
  • , M. Brodmann
  • , E. Brountzos
  • , T. Carrell
  • , T. Cohnert
  • , J. P. De Vries
  • , F. Dick
  • , R. Ferraresi
  • , Y. Gouëffic
  • , S. Haulon
  • , C. Karkos
  • , I. Končar
  • , J. Lammer
  • , Z. Martin
  • , R. McWilliams
  • , G. Melissano
  • , S. Müller-Hülsbeck
  • , C. Nienaber
  • , T. Resch
  • , V. Riambau
  • , R. Williams
  • , Z. Szeberin
  • , J. Teijink
  • , J. Van Den Berg
  • , J. van Herwaarden
  • , F. Vermassen
  • , F. Verzini
  •  & A. Wanhainen

Corresponding author

Correspondence to Bart Doyen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Informed Consent

“For this type of study (Delphi study) formal ethical approval was not required, nevertheless, written informed consent was obtained from all individual participants included in the study. Consent for publication was obtained for every individual person’s data included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Appendix A:

Comparison of ratings between raters (scatter plot). The dashed lines represent the threshold score (33 points). Two cases were rated on different sides of threshold with rater B rating higher compared to rater A (dots in upper-left quadrant): in one case ratings were 6 points apart, in the second case there was a difference of 12 points (EPS 1301 kb)

Appendix B:

Bland–Altman plot – Mean of ratings plotted against their difference (rater A – rater B). On average rater A scored 2.6 points lower compared to rater B (EPS 1337 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Doyen, B., Maurel, B., Hertault, A. et al. Radiation Safety Performance is More than Simply Measuring Doses! Development of a Radiation Safety Rating Scale. Cardiovasc Intervent Radiol 43, 1331–1341 (2020). https://doi.org/10.1007/s00270-020-02590-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00270-020-02590-7

Keywords

Search

Navigation