Skip to main content
SpringerLink
Log in

Interobserver agreement and efficacy of consensus reading in Kwak-, EU-, and ACR-thyroid imaging recording and data systems and ATA guidelines for the ultrasound risk stratification of thyroid nodules

  • Original Article
  • Published:
Endocrine Aims and scope Submit manuscript

Abstract

Purpose

To investigate the interobserver agreement (IA) and the impact of consensus reading using four risk stratification systems for thyroid nodules (TN).

Methods

Four experienced specialists independently rated US images of 80 TN according to the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines. The cases were randomly extracted from a prospectively acquired database (n > 1500 TN). The observers were blinded to clinical data. This study was divided into two sessions (S1 and S2) with 40 image sets each. After every session, a consensus reading was carried out (C1, C2). Subsequently, the effect of C1 was tested in S2 with 40 new cases followed by C2. Fleiss’ kappa (κ) was calculated for S1 and S2 to estimate the IA and learning curves. The results of C1 and C2 were used as reference for diagnostic accuracy calculations.

Results

IA significantly increased (p < 0.01) after C1 with κ values of 0.375 (0.615), 0.411 (0.596), 0.321 (0.569), and 0.410 (0.583) for the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines in S1 (S2), respectively. ROC analysis (C1 + C2) revealed similar areas under the curve (AUC) for the Kwak-TIRADS, EU-TIRADS, ACR TI-RADS, and ATA Guidelines (0.635, 0.675, 0.694, and 0.654, respectively, n.s.). AUC did not increase from C1 (0.677 ± 0.010) to C2 (0.632 ± 0.052, n.s.). ATA Guidelines were not applicable in five cases.

Conclusions

IA and diagnostic accuracy were very similar for the four investigated risk stratification systems. Consensus reading sessions significantly improved the IA but did not affect the diagnostic accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. C. Reiners, K. Wegscheider, H. Schicha, P. Theissen, R. Vaupel, R. Wrbitzky, P.M. Schumm-Draeger, Prevalence of thyroid disorders in the working population of Germany: ultrasonography screening in 96,278 unselected employees. Thyroid 14(11), 926–932 (2004). https://doi.org/10.1089/thy.2004.14.926

    Article  PubMed  Google Scholar 

  2. G. Russ, S. Leboulleux, L. Leenhardt, L. Hegedus, Thyroid incidentalomas: epidemiology, risk stratification with ultrasound and workup. Eur. Thyroid J. 3(3), 154–163 (2014). https://doi.org/10.1159/000365289

    Article  PubMed  PubMed Central  Google Scholar 

  3. S. Guth, U. Theune, J. Aberle, A. Galach, C.M. Bamberger, Very high prevalence of thyroid nodules detected by high frequency (13 MHz) ultrasound examination. Eur. J. Clin. Invest. 39(8), 699–706 (2009). https://doi.org/10.1111/j.1365-2362.2009.02162.x

    Article  CAS  PubMed  Google Scholar 

  4. F. Bray, J. Ferlay, I. Soerjomataram, R.L. Siegel, L.A. Torre, A. Jemal, Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J. Clin. 68(6), 394–424 (2018). https://doi.org/10.3322/caac.21492

    Article  PubMed  Google Scholar 

  5. C.M. Oh, K.W. Jung, Y.J. Won, A. Shin, H.J. Kong, J.S. Lee, Age-period-cohort analysis of thyroid cancer incidence in Korea. Cancer Res Treat. 47(3), 362–369 (2015). https://doi.org/10.4143/crt.2014.110

    Article  CAS  PubMed  Google Scholar 

  6. L. Liang, X.C. Zheng, M.J. Hu, Q. Zhang, S.Y. Wang, F. Huang, Association of benign thyroid diseases with thyroid cancer risk: a meta-analysis of prospective observational studies. J. Endocrinol. Invest. 42(6), 673–685 (2019). https://doi.org/10.1007/s40618-018-0968-z

    Article  CAS  PubMed  Google Scholar 

  7. J.Y. Kwak, K.H. Han, J.H. Yoon, H.J. Moon, E.J. Son, S.H. Park, H.K. Jung, J.S. Choi, B.M. Kim, E.K. Kim, Thyroid imaging reporting and data system for US features of nodules: a step in establishing better stratification of cancer risk. Radiology 260(3), 892–899 (2011). https://doi.org/10.1148/radiol.11110206

    Article  PubMed  Google Scholar 

  8. E.J. Ha, J.H. Baek, D.G. Na, Risk stratification of thyroid nodules on ultrasonography: current status and perspectives. Thyroid 27(12), 1463–1468 (2017). https://doi.org/10.1089/thy.2016.0654

    Article  PubMed  Google Scholar 

  9. S.M. Ha, H.S. Ahn, J.H. Baek, H.Y. Ahn, Y.J. Chung, B.Y. Cho, S.B. Park, Validation of three scoring risk-stratification models for thyroid nodules. Thyroid 27(12), 1550–1557 (2017). https://doi.org/10.1089/thy.2017.0363

    Article  PubMed  Google Scholar 

  10. S. Schenke, T. Rink, M. Zimny, TIRADS for sonographic assessment of hypofunctioning and indifferent thyroid nodules. Nuklearmedizin 54(3), 144–150 (2015). https://doi.org/10.3413/Nukmed-0712-14-12

    Article  PubMed  Google Scholar 

  11. J.Y. Park, H.J. Lee, H.W. Jang, H.K. Kim, J.H. Yi, W. Lee, S.H. Kim, A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma. Thyroid 19(11), 1257–1264 (2009). https://doi.org/10.1089/thy.2008.0021

    Article  PubMed  Google Scholar 

  12. B.R. Haugen, E.K. Alexander, K.C. Bible, G.M. Doherty, S.J. Mandel, Y.E. Nikiforov, F. Pacini, G.W. Randolph, A.M. Sawka, M. Schlumberger, K.G. Schuff, S.I. Sherman, J.A. Sosa, D.L. Steward, R.M. Tuttle, L. Wartofsky, 2015 American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: the American Thyroid Association Guidelines Task Force on thyroid nodules and differentiated thyroid cancer. Thyroid 26(1), 1–133 (2016). https://doi.org/10.1089/thy.2015.0020

    Article  PubMed  PubMed Central  Google Scholar 

  13. S. Schenke, P. Seifert, M. Zimny, T. Winkens, I. Binse, R. Goerges, Risk stratification of thyroid nodules using Thyroid Imaging Reporting And Data System (TIRADS): the omission of thyroid scintigraphy increases the rate of falsely suspected lesions. J. Nucl. Med. 60(3), 342–347 (2019). https://doi.org/10.2967/jnumed.118.211912

    Article  CAS  PubMed  Google Scholar 

  14. E. Horvath, S. Majlis, R. Rossi, C. Franco, J.P. Niedmann, A. Castro, M. Dominguez, An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management. J. Clin. Endocrinol. Metab. 94(5), 1748–1751 (2009). https://doi.org/10.1210/jc.2008-1724

    Article  CAS  PubMed  Google Scholar 

  15. P. Perros, K. Boelaert, S. Colley, C. Evans, R.M. Evans, G. Gerrard Ba, J. Gilbert, B. Harrison, S.J. Johnson, T.E. Giles, L. Moss, V. Lewington, K. Newbold, J. Taylor, R.V. Thakker, J. Watkinson, G.R. Williams, A. British Thyroid, Guidelines for the management of thyroid cancer. Clin. Endocrinol. (Oxf.) 81(Suppl 1), 1–122 (2014). https://doi.org/10.1111/cen.12515

    Article  CAS  Google Scholar 

  16. E.J. Ha, W.J. Moon, D.G. Na, Y.H. Lee, N. Choi, S.J. Kim, J.K. Kim, A multicenter prospective validation study for the Korean thyroid imaging reporting and data system in patients with thyroid nodules. Korean J. Radio. 17(5), 811–821 (2016). https://doi.org/10.3348/kjr.2016.17.5.811

    Article  Google Scholar 

  17. G. Russ, S.J. Bonnema, M.F. Erdogan, C. Durante, R. Ngu, L. Leenhardt, European thyroid association guidelines for ultrasound malignancy risk stratification of thyroid nodules in adults: the EU-TIRADS. Eur. Thyroid J. 6(5), 225–237 (2017). https://doi.org/10.1159/000478927

    Article  PubMed  PubMed Central  Google Scholar 

  18. F.N. Tessler, W.D. Middleton, E.G. Grant, J.K. Hoang, L.L. Berland, S.A. Teefey, J.J. Cronan, M.D. Beland, T.S. Desser, M.C. Frates, L.W. Hammers, U.M. Hamper, J.E. Langer, C.C. Reading, L.M. Scoutt, A.T. Stavros, ACR thyroid imaging, reporting and data system (TI-RADS): white paper of the ACR TI-RADS committee. J. Am. Coll. Radio. 14(5), 587–595 (2017). https://doi.org/10.1016/j.jacr.2017.01.046

    Article  Google Scholar 

  19. G. Grani, L. Lamartina, V. Cantisani, M. Maranghi, P. Lucia, C. Durante, Interobserver agreement of various thyroid imaging reporting and data systems. Endocr. Connect 7(1), 1–7 (2018). https://doi.org/10.1530/EC-17-0336

    Article  PubMed  Google Scholar 

  20. C.L. Chng, H.C. Tan, C.W. Too, W.Y. Lim, P.P.S. Chiam, L. Zhu, N.V. Nadkarni, A.Y.Y. Lim, Diagnostic performance of ATA, BTA and TIRADS sonographic patterns in the predication of malignancy in histologically proven thyroid nodules. Singap. Med J. 59(11), 578–583 (2018). https://doi.org/10.11622/smedj.2018062

    Article  Google Scholar 

  21. S.H. Choi, E.K. Kim, J.Y. Kwak, M.J. Kim, E.J. Son, Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid 20(2), 167–172 (2010). https://doi.org/10.1089/thy.2008.0354

    Article  PubMed  Google Scholar 

  22. P. Valderrabano, B. McIver, Evaluation and management of indeterminate thyroid nodules: the revolution of risk stratification beyond cytological diagnosis. Cancer Control 24(5), 1–14 (2017). https://doi.org/10.1177/1073274817729231

    Article  Google Scholar 

  23. J.R. Landis, G.G. Koch, The measurement of observer agreement for categorical data. Biometrics 33(1), 159–174 (1977)

    Article  CAS  PubMed  Google Scholar 

  24. R. Wienhold, M. Scholz, J.R. Adler, G.N. C, R. Paschke, The management of thyroid nodules: a retrospective analysis of health insurance data. Dtsch. Arztebl. Int. 110(49), 827–834 (2013). https://doi.org/10.3238/arztebl.2013.0827

    Article  PubMed  PubMed Central  Google Scholar 

  25. P. Seifert, M. Freesmeyer, Preoperative diagnostics in differentiated thyroid carcinoma. Nuklearmedizin 56(6), 201–210 (2017). https://doi.org/10.3413/Nukmed-0924-17-08

    Article  PubMed  Google Scholar 

  26. H. Gharib, E. Papini, J.R. Garber, D.S. Duick, R.M. Harrell, L. Hegedus, R. Paschke, R. Valcavi, P. Vitti, A.A.A.T.Fo.T. Nodules, American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi medical guidelines for clinical practice for the diagnosis and management of thyroid nodules-2016 update. Endocr. Pract. 22(5), 622–639 (2016). https://doi.org/10.4158/EP161208.GL

    Article  PubMed  Google Scholar 

  27. H. Dralle, T.J. Musholt, J. Schabram, T. Steinmuller, A. Frilling, D. Simon, P.E. Goretzki, B. Niederle, C. Scheuba, T. Clerici, M. Hermann, J. Kussmann, K. Lorenz, C. Nies, P. Schabram, A. Trupka, A. Zielke, W. Karges, M. Luster, K.W. Schmid, D. Vordermark, H.J. Schmoll, R. Muhlenberg, O. Schober, H. Rimmele, A. Machens, German Societies of General and Visceral Surgery; Endocrinology; Nuclear Medicine; Pathology; Radiooncology; Oncological Hematology; and the German Thyroid Cancer Patient Support Organization Ohne Schilddrüse leben e.V., German Association of Endocrine Surgeons practice guideline for the surgical management of malignant thyroid tumors. Langenbecks Arch. Surg. 398(3), 347–375 (2013). https://doi.org/10.1007/s00423-013-1057-6

    Article  PubMed  Google Scholar 

  28. F. Pacini, M. Schlumberger, H. Dralle, R. Elisei, J.W. Smit, W. Wiersinga, European Thyroid Cancer Taskforce, European consensus for the management of patients with differentiated thyroid carcinoma of the follicular epithelium. Eur. J. Endocrinol. 154(6), 787–803 (2006). https://doi.org/10.1530/eje.1.02158

    Article  CAS  PubMed  Google Scholar 

  29. L. Leenhardt, M.F. Erdogan, L. Hegedus, S.J. Mandel, R. Paschke, T. Rago, G. Russ, 2013 European thyroid association guidelines for cervical ultrasound scan and ultrasound-guided techniques in the postoperative management of patients with thyroid cancer. Eur. Thyroid J. 2(3), 147–159 (2013). https://doi.org/10.1159/000354537

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. J. Koh, S.Y. Kim, H.S. Lee, E.K. Kim, J.Y. Kwak, H.J. Moon, J.H. Yoon, Diagnostic performances and interobserver agreement according to observer experience: a comparison study using three guidelines for management of thyroid nodules. Acta Radio. 59(8), 917–923 (2018). https://doi.org/10.1177/0284185117744001

    Article  Google Scholar 

  31. J. Koh, H.J. Moon, J.S. Park, S.J. Kim, H.Y. Kim, E.K. Kim, J.Y. Kwak, Variability in interpretation of ultrasound elastography and gray-scale ultrasound in assessing thyroid nodules. Ultrasound Med. Biol. 42(1), 51–59 (2016). https://doi.org/10.1016/j.ultrasmedbio.2015.08.005

    Article  PubMed  Google Scholar 

  32. S.H. Park, S.J. Kim, E.K. Kim, M.J. Kim, E.J. Son, J.Y. Kwak, Interobserver agreement in assessing the sonographic and elastographic features of malignant thyroid nodules. AJR Am. J. Roentgenol. 193(5), W416–W423 (2009). https://doi.org/10.2214/AJR.09.2541

    Article  PubMed  Google Scholar 

  33. S.J. Park, S.H. Park, Y.J. Choi, D.W. Kim, E.J. Son, H.S. Lee, J.H. Yoon, E.K. Kim, H.J. Moon, J.Y. Kwak, Interobserver variability and diagnostic performance in US assessment of thyroid nodule according to size. Ultraschall Med. 33(7), E186–E190 (2012). https://doi.org/10.1055/s-0032-1325404

    Article  CAS  PubMed  Google Scholar 

  34. S.P. Cheng, J.J. Lee, J.L. Lin, S.M. Chuang, M.N. Chien, C.L. Liu, Characterization of thyroid nodules using the proposed thyroid imaging reporting and data system (TI-RADS). Head. Neck 35(4), 541–547 (2013). https://doi.org/10.1002/hed.22985

    Article  PubMed  Google Scholar 

  35. C.S. Park, S.H. Kim, S.L. Jung, B.J. Kang, J.Y. Kim, J.J. Choi, M.S. Sung, H.W. Yim, S.H. Jeong, Observer variability in the sonographic evaluation of thyroid nodules. J. Clin. Ultrasound 38(6), 287–293 (2010). https://doi.org/10.1002/jcu.20689

    Article  PubMed  Google Scholar 

  36. S.H. Kim, C.S. Park, S.L. Jung, B.J. Kang, J.Y. Kim, J.J. Choi, Y.I. Kim, J.K. Oh, J.S. Oh, H. Kim, S.H. Jeong, H.W. Yim, Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules. Korean J. Radio. 11(2), 149–155 (2010). https://doi.org/10.3348/kjr.2010.11.2.149

    Article  Google Scholar 

  37. H.G. Kim, J.Y. Kwak, E.K. Kim, S.H. Choi, H.J. Moon, Man to man training: can it help improve the diagnostic performances and interobserver variabilities of thyroid ultrasonography in residents? Eur. J. Radio. 81(3), e352–e356 (2012). https://doi.org/10.1016/j.ejrad.2011.11.011

    Article  Google Scholar 

  38. C. Anuradha, K. Abhishek, B. Pushpa, A. Deepak, Positive predictive value and inter-observer agreement of TI-RADS for ulrasound features of thyroid nodules. Paper presented at the Abstract presented at the European Congress of Radiology (ECR), Vienna, Austria, 6–10 March. Abstract no. C-0594 (2014).

  39. A. Chandramohan, A. Khurana, B.T. Pushpa, M.T. Manipadam, D. Naik, N. Thomas, D. Abraham, M.J. Paul, Is TIRADS a practical and accurate system for use in daily clinical practice? Indian J. Radio. Imaging 26(1), 145–152 (2016). https://doi.org/10.4103/0971-3026.178367

    Article  Google Scholar 

  40. M.N. Srinivas, V.N. Amogh, M.S. Gautam, I.S. Prathyusha, N.R. Vikram, M.K. Retnam, B.V. Balakrishna, N. Kudva, A prospective study to evaluate the reliability of thyroid imaging reporting and data system in differentiation between benign and malignant thyroid lesions. J. Clin. Imaging Sci. 6, 5 (2016). https://doi.org/10.4103/2156-7514.177551

    Article  PubMed  PubMed Central  Google Scholar 

  41. J.M. Bae, S.Y. Hahn, J.H. Shin, E.Y. Ko, Inter-exam agreement and diagnostic performance of the Korean Thyroid Imaging Reporting and Data System for thyroid nodule assessment: real-time versus static ultrasonography. Eur. J. Radiol. 98, 14–19 (2018). https://doi.org/10.1016/j.ejrad.2017.10.027

    Article  PubMed  Google Scholar 

  42. J.M. Bae, S.Y. Hahn, J.H. Shin, E.Y. Ko, Erratum to “Inter-exam agreement and diagnostic performance of the Korean Thyroid Imaging Reporting and Data System for thyroid nodule assessment: real-time versus static ultrasonography” [Eur. J. Radiol. 98 (2018) 14–19]. Eur. J. Radiol. 101, 193 (2018).

    Article  PubMed  Google Scholar 

  43. J.K. Hoang, W.D. Middleton, A.E. Farjat, S.A. Teefey, N. Abinanti, F.J. Boschini, A.J. Bronner, N. Dahiya, B.S. Hertzberg, J.R. Newman, D. Scanga, R.C. Vogler, F.N. Tessler, Interobserver variability of sonographic features used in the American College of Radiology Thyroid Imaging Reporting and Data System. AJR Am. J. Roentgenol. 211(1), 162–167 (2018). https://doi.org/10.2214/AJR.17.19192

    Article  PubMed  Google Scholar 

  44. A. Persichetti, E. Di Stasio, R. Guglielmi, G. Bizzarri, S. Taccogna, I. Misischi, F. Graziano, L. Petrucci, A. Bianchini, E. Papini, Predictive value of malignancy of thyroid nodule ultrasound classification systems: a prospective study. J. Clin. Endocrinol. Metab. 103(4), 1359–1368 (2018). https://doi.org/10.1210/jc.2017-01708

    Article  PubMed  Google Scholar 

  45. G. Grani, M. D’Alessandri, G. Carbotta, A. Nesca, M. Del Sordo, S. Alessandrini, C. Coccaro, R. Rendina, M. Bianchini, N. Prinzi, A. Fumarola, Grey-scale analysis improves the ultrasonographic evaluation of thyroid nodules. Med. Baltim. 94(27), e1129 (2015). https://doi.org/10.1097/MD.0000000000001129

    Article  Google Scholar 

  46. L. Gao, R. Liu, Y. Jiang, W. Song, Y. Wang, J. Liu, J. Wang, D. Wu, S. Li, A. Hao, B. Zhang, Computer-aided system for diagnosing thyroid nodules on ultrasound: A comparison with radiologist-based clinical assessments. Head. Neck 40(4), 778–783 (2018). https://doi.org/10.1002/hed.25049

    Article  PubMed  Google Scholar 

  47. S. Gitto, G. Grassi, C. De Angelis, C.G. Monaco, S. Sdao, F. Sardanelli, L.M. Sconfienza, G. Mauri, A computer-aided diagnosis system for the assessment and characterization of low-to-high suspicion thyroid nodules on ultrasound. Radio. Med 124(2), 118–125 (2019). https://doi.org/10.1007/s11547-018-0942-z

    Article  Google Scholar 

  48. E.Y. Jeong, H.L. Kim, E.J. Ha, S.Y. Park, Y.J. Cho, M. Han, Computer-aided diagnosis system for thyroid nodules on ultrasonography: diagnostic performance and reproducibility based on the experience level of operators. Eur. Radio. 29(4), 1978–1985 (2019). https://doi.org/10.1007/s00330-018-5772-9

    Article  Google Scholar 

  49. Y.J. Yoo, E.J. Ha, Y.J. Cho, H.L. Kim, M. Han, S.Y. Kang, Computer-aided diagnosis of thyroid nodules via ultrasonography: initial clinical experience. Korean J. Radio. 19(4), 665–672 (2018). https://doi.org/10.3348/kjr.2018.19.4.665

    Article  Google Scholar 

  50. K. Kobayashi, T. Fujimoto, H. Ota, M. Hirokawa, T. Yabuta, H. Masuoka, M. Fukushima, T. Higashiyama, M. Kihara, Y. Ito, A. Miya, A. Miyauchi, Calcifications in thyroid tumors on ultrasonography: calcification types and relationship with histopathological type. Ultrasound Int. Open 4(2), E45–E51 (2018). https://doi.org/10.1055/a-0591-6070

    Article  PubMed  PubMed Central  Google Scholar 

  51. L. Gao, X. Xi, J. Wang, X. Yang, Y. Wang, S. Zhu, X. Lai, X. Zhang, R. Zhao, B. Zhang, Ultrasound risk evaluation of thyroid nodules that are “unspecified” in the 2015 American Thyroid Association management guidelines: a retrospective study. Med. Baltim. 97(52), e13914 (2018). https://doi.org/10.1097/MD.0000000000013914

    Article  Google Scholar 

  52. P. Seifert, F. Guhne, M. Freesmeyer, Hyperfunctioning papillary microcarcinoma diagnosed by 124I PET/ultrasound fusion imaging. Clin. Nucl. Med. (2019). https://doi.org/10.1097/RLU.0000000000002503

    Article  PubMed  Google Scholar 

  53. M.J. Lima, V. Soares, P. Koch, A. Silva, A. Taveira-Gomes, Autonomously hyperfunctioning cystic nodule harbouring thyroid carcinoma - case report and literature review. Int J. Surg. Case Rep. 42, 287–289 (2018). https://doi.org/10.1016/j.ijscr.2018.01.002

    Article  PubMed  PubMed Central  Google Scholar 

  54. G. Yun, Y.K. Kim, S.I. Choi, J.H. Kim, Medullary thyroid carcinoma: application of Thyroid Imaging Reporting and Data System (TI-RADS) Classification. Endocrine 61(2), 285–292 (2018). https://doi.org/10.1007/s12020-018-1594-4

    Article  CAS  PubMed  Google Scholar 

  55. M.J. Liu, Z.F. Liu, Y.Y. Hou, Y.M. Men, Y.X. Zhang, L.Y. Gao, H. Liu, Ultrasonographic characteristics of medullary thyroid carcinoma: a comparison with papillary thyroid carcinoma. Oncotarget 8(16), 27520–27528 (2017). https://doi.org/10.18632/oncotarget.15897

    Article  PubMed  PubMed Central  Google Scholar 

  56. D.S. Kim, J.H. Kim, D.G. Na, S.H. Park, E. Kim, K.H. Chang, C.H. Sohn, Y.H. Choi, Sonographic features of follicular variant papillary thyroid carcinomas in comparison with conventional papillary thyroid carcinomas. J. Ultrasound Med. 28(12), 1685–1692 (2009)

    Article  PubMed  Google Scholar 

  57. E. Chaigneau, G. Russ, B. Royer, C. Bigorgne, M. Bienvenu-Perrard, A. Rouxel, L. Leenhardt, L. Belin, C. Buffet, TIRADS score is of limited clinical value for risk stratification of indeterminate cytological results. Eur. J. Endocrinol. 179(1), 13–20 (2018). https://doi.org/10.1530/EJE-18-0078

    Article  CAS  PubMed  Google Scholar 

  58. H. Baser, B. Cakir, O. Topaloglu, A. Alkan, S.B. Polat, H.T. Dogan, M.O. Yazicioglu, C. Aydin, R. Ersoy, Diagnostic accuracy of Thyroid Imaging Reporting and Data System in the prediction of malignancy in nodules with atypia and follicular lesion of undetermined significance cytologies. Clin. Endocrinol. Oxf. 86(4), 584–590 (2017). https://doi.org/10.1111/cen.13274

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors thank PD Dr med. Martin Freesmeyer of the Department of Nuclear Medicine at Jena University Hospital for providing access to patient data and images.

Author information

Authors and Affiliations

  1. Department of Nuclear Medicine, Jena University Hospital, Jena, Germany

    Philipp Seifert

  2. Department of Nuclear Medicine, Essen University Hospital, Essen, Germany

    Rainer Görges

  3. Joint Practice for Nuclear Medicine, Duisburg/Moers, Duisburg, Germany

    Rainer Görges

  4. Institute for Nuclear Medicine Hanau, Giessen, Germany

    Michael Zimny & Simone Schenke

  5. Division of Nuclear Medicine, Department of Radiology and Nuclear Medicine, Magdeburg University Hospital, Magdeburg, Germany

    Michael C. Kreissl & Simone Schenke

Authors
  1. Philipp Seifert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rainer Görges
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Zimny
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael C. Kreissl
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Simone Schenke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Seifert.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seifert, P., Görges, R., Zimny, M. et al. Interobserver agreement and efficacy of consensus reading in Kwak-, EU-, and ACR-thyroid imaging recording and data systems and ATA guidelines for the ultrasound risk stratification of thyroid nodules. Endocrine 67, 143–154 (2020). https://doi.org/10.1007/s12020-019-02134-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12020-019-02134-1

Keywords

Search

Navigation