Skip to main content
SpringerLink
Log in

The intelligent OR: design and validation of a context-aware surgical working environment

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Interoperability of medical devices based on standards starts to establish in the operating room (OR). Devices share their data and control functionalities. Yet, the OR technology rarely implements cooperative, intelligent behavior, especially in terms of active cooperation with the OR team. Technical context-awareness will be an essential feature of the next generation of medical devices to address the increasing demands to clinicians in information seeking, decision making, and human–machine interaction in complex surgical working environments.

Methods

The paper describes the technical validation of an intelligent surgical working environment for endoscopic ear–nose–throat surgery. We briefly summarize the design of our framework for context-aware system’s behavior in integrated OR and present example realizations of novel assistance functionalities. In a study on patient phantoms, twenty-four procedures were implemented in the proposed intelligent surgical working environment based on recordings of real interventions. Subsequently, the whole processing pipeline for context-awareness from workflow recognition to the final system’s behavior is analyzed.

Results

Rule-based behavior that considers multiple perspectives on the procedure can partially compensate recognition errors. A considerable robustness could be achieved with a reasonable quality of the recognition. Overall, reliable reactive as well as proactive behavior of the surgical working environment can be implemented in the proposed environment.

Conclusions

The obtained validation results indicate the suitability of the overall approach. The setup is a reliable starting point for a subsequent evaluation of the proposed context-aware assistance. The major challenge for future work will be to implement the complex approach in a cross-vendor setting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Kasparick M, Schlichting S, Golatowski F, Timmermann D (2015) New IEEE 11073 standards for interoperable, networked point-of-care medical devices. In: Proceedings of the 37th IEEE engineering in medicine and biology society (EMBC). IEEE, pp 1721–1724

  2. Andersen B, Ulrich H, Rehmann D, Kock A-K, Wrage J-H, Ingenerf J (2015) Reporting device observations for semantic interoperability of surgical devices and clinical information systems. In: Proceedings of the 37th IEEE engineering in medicine and biology society (EMBC). IEEE, pp 1725–1728

  3. Rockstroh M, Franke S, Hofer M, Will A, Kasparick M, Andersen B, Neumuth T (2017) ORNET: multi-perspective qualitative evaluation of an integrated operating room based on IEEE 11073 SDC. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-017-1589-2

    Article  PubMed  Google Scholar 

  4. Katić D, Julliard C, Wekerle A-L, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-015-1222-1

    Article  PubMed  Google Scholar 

  5. Blum T, Padoy N, Feußner H, Navab N (2008) Modeling and online recognition of surgical phases using hidden Markov models. In: Metaxas D, Axel L, Fichtinger G, Székely G (eds) Medical image computing and computer-assisted intervention-MICCAI 2008. Springer, Berlin, pp 627–635

    Chapter  Google Scholar 

  6. Lalys F, Riffaud L, Bouget D, Jannin P (2011) An application-dependent framework for the recognition of high-level surgical tasks in the OR. In: Fichtinger G, Martel A, Peters T (eds) Medical image computing and computer-assisted intervention-MICCAI 2011. Springer, Berlin, pp 331–338

    Chapter  Google Scholar 

  7. Speidel S, Zentek T, Sudra G, Gehrig T, Müller-Stich BP, Gutt C, Dillmann R (2009) Recognition of surgical skills using hidden Markov models. In: Proceedings volume 7261, medical imaging 2009: visualization, image-guided procedures, and modeling, pp 726125-8–726125-8

  8. Kranzfelder M, Schneider A, Fiolka A, Koller S, Reiser S, Vogel T, Wilhelm D, Feussner H (2014) Reliability of sensor-based real-time workflow recognition in laparoscopic cholecystectomy. Int J Comput Assist Radiol Surg 9:941–948. https://doi.org/10.1007/s11548-014-0986-z

    Article  PubMed  Google Scholar 

  9. Lalys F, Riffaud L, Bouget D, Jannin P (2012) A Framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 59:966–976. https://doi.org/10.1109/TBME.2011.2181168

    Article  PubMed  CAS  Google Scholar 

  10. Bouarfa L, Akman O, Schneider A, Jonker PP, Dankelman J (2012) In-vivo real-time tracking of surgical instruments in endoscopic video. Minim Invasive Therapy Allied Technol 21:129–134. https://doi.org/10.3109/13645706.2011.580764

    Article  Google Scholar 

  11. Kranzfelder M, Schneider A, Fiolka A, Schwan E, Gillen S, Wilhelm D, Schirren R, Reiser S, Jensen B, Feussner H (2013) Real-time instrument detection in minimally invasive surgery using radiofrequency identification technology. J Surg Res 185:704–710. https://doi.org/10.1016/j.jss.2013.06.022

    Article  PubMed  Google Scholar 

  12. Glaser B, Dänzer S, Neumuth T (2015) Intra-operative surgical instrument usage detection on a multi-sensor table. Int J Comput Assist Radiol Surg 10:351–362. https://doi.org/10.1007/s11548-014-1066-0

    Article  PubMed  Google Scholar 

  13. Neumuth T (2017) Surgical process modeling. Innov Surg Sci. https://doi.org/10.1515/iss-2017-0005

    Article  Google Scholar 

  14. Schmidt A, Beigl M, Gellersen H-W (1999) There is more to context than location. Comput Graph 23:893–901. https://doi.org/10.1016/S0097-8493(99)00120-X

    Article  Google Scholar 

  15. Franke S, Meixensberger J, Neumuth T (2015) Multi-perspective workflow modeling for online surgical situation models. J Biomed Inform 54:158–166

    Article  PubMed  Google Scholar 

  16. Franke S, Neumuth T (2015) Adaptive surgical process models for prediction of surgical work steps from surgical low-level activities. In: Workshop on modeling and monitoring of computer-aided interventions (M2CAI). Medical Image Computing and Computer Assisted Intervention, Munich, Germany

  17. Franke S, Meixensberger J, Neumuth T (2013) Intervention time prediction from surgical low-level tasks. J Biomed Inform 46:152–159. https://doi.org/10.1016/j.jbi.2012.10.002

    Article  PubMed  Google Scholar 

  18. Franke S, Neumuth T (2015) Towards structuring contextual information for workflow-driven surgical assistance functionalities. In: 49th annual conference of the German society for biomedical engineering (BMT 2015), Lübeck, Germany

  19. Franke S, Neumuth T (2015) Rule-based medical device adaptation for the digital operating room. In: Proceedings of 37th annual international conference of the IEEE engineering in medicine and biology society. IEEE, Milano, pp 1733–1736

  20. Franke S, Rockstroh M, Schreiber E, Neumann J, Neumuth T (2016) Context-aware medical assistance systems in integrated surgical environments. In: 28th conference of the international society for medical innovation and technology (SMIT), Delft, Netherlands, p 98

  21. Meyer MA, Levine WC, Egan MT, Cohen BJ, Spitz G, Garcia P, Chueh H, Sandberg WS (2007) A computerized perioperative data integration and display system. Int J Comput Assist Radiol Surg 2:191–202. https://doi.org/10.1007/s11548-007-0126-0

    Article  Google Scholar 

  22. Pickering BW, Herasevich V, Ahmed A, Gajic O (2010) Novel representation of clinical information in the ICU: developing user interfaces which reduce information overload. Appl Clin Inform 1:116–131. https://doi.org/10.4338/ACI-2009-12-CR-0027

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  23. Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27:1681–1688. https://doi.org/10.1007/s00464-012-2656-y

    Article  PubMed  Google Scholar 

  24. Rockstroh M, Franke S, Neumuth T (2013) A workflow-driven surgical information source management. In: Lemke HU (ed) The international journal for computer assisted radiology and surgery. Springer, Heidelberg, pp 189–191

  25. Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inf Assoc 16(1):72–80

    Article  Google Scholar 

  26. Twinanda A, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86–97

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Funding was provided by Bundesministerium für Bildung und Forschung (DE) (Grant No. 16KT1236).

Author information

Authors and Affiliations

  1. Innovation Center Computer Assisted Surgery, University Leipzig, Semmelweisstraße 14, 04103, Leipzig, Germany

    Stefan Franke, Max Rockstroh & Thomas Neumuth

  2. Department of ENT Surgery, University Leipzig, Liebigstraße 14, 04103, Leipzig, Germany

    Mathias Hofer

Authors
  1. Stefan Franke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Max Rockstroh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mathias Hofer
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Neumuth
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stefan Franke.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Franke, S., Rockstroh, M., Hofer, M. et al. The intelligent OR: design and validation of a context-aware surgical working environment. Int J CARS 13, 1301–1308 (2018). https://doi.org/10.1007/s11548-018-1791-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-018-1791-x

Keywords

Search

Navigation