Bolin Zhang
Abstract
The outbreak of Covid-19 has exposed the lack of medical resources, especially the lack of medical personnel. This results in time and space restrictions for medical services, and patients cannot obtain health information all the time and everywhere. Based on the medical knowledge graph, healthcare bots alleviate this burden effectively by providing patients with diagnosis guidance, pre-diagnosis, and post-diagnosis consultation services in the way of human-machine dialogue. However, the medical utterance is more complicated in language structure, and there are complex intention phenomena in semantics. It is a challenge to detect the single intent, multi-intent, and implicit intent of a patient’s utterance. To this end, we create a high-quality annotated Chinese Medical query (utterance) dataset, CMedQ (about 16.8k queries in medical domain which includes single, multiple, and implicit intents). It is hard to detect intent on such a complex dataset through traditional text classification models. Thus, we propose a novel detect model Conco-ERNIE, using concept co-occurrence patterns to enhance the representation of pre-trained model ERNIE. These patterns are mined using Apriori algorithm and will be embedded via Node2Vec. Their features will be aggregated with semantic features into Conco-ERNIE by using an attention module, which can catch user explicit intents and also predict user implicit intents. Experiments on CMedQ demonstrates that Conco-ERNIE achieves outstanding performance over baseline. Based on Conco-ERNIE, we develop an intelligent healthcare bot, MedicalBot. To provide knowledge support for MedicalBot, we also build a Chinese medical graph, CMedKG (about 45k entities and 283k relationships).
- [1] . 1986. Fuzzy set theory in medical diagnosis. IEEE Transactions on Systems, Man, and Cybernetics 16, 2 (1986), 260–265.Google Scholar
Digital Library
- [2] . 1994. Fast algorithms for mining association rules. In Proc. 20th Int. Conf. Very Large Data Bases, VLDB, Vol. 1215. 487–499.Google Scholar
- [3] . 2022. Incremental intent detection for medical domain with contrast replay networks. In Findings of the Association for Computational Linguistics: ACL, Dublin, Ireland, May 22–27. 3549–3556.Google Scholar
- [4] . 2021. LifeDoc: Availability and monitoring system of online medical consultation. In 11th IEEE International Conference on Control System, Computing and Engineering, ICCSCE 2021, Penang, Malaysia, August 27–28, 2021. 103–108.Google Scholar
- [5] . [n.d.]. An CNN-LSTM attention approach to understanding user query intent from online health communities. In 2017 IEEE International Conference on Data Mining Workshops, ICDM Workshops 2017, New Orleans, LA, USA, November 18–21, 2017. 430–437.Google Scholar
- [6] . 2020. A benchmark dataset and case study for Chinese medical question intent classification. BMC Medical Informatics Decis. Mak. 20-S, 3 (2020), 125.Google ScholarCross Ref
- [7] . 2021. Multi-label text classification with latent word-wise label information. Appl. Intell. 51, 2 (2021), 966–979.Google ScholarDigital Library
- [8] . 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).Google Scholar
- [9] . 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 855–864.Google ScholarDigital Library
- [10] . [n.d.]. Joint semantic utterance classification and slot filling with recursive neural networks. In 2014 IEEE Spoken Language Technology Workshop, SLT 2014, South Lake Tahoe, NV, USA, December 7–10, 2014. 554–559.Google Scholar
- [11] . 2019. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5467–5471.Google Scholar
- [12] . 2015. Bidirectional LSTM-CRF models for sequence tagging. CoRR abs/1508.01991 (2015). http://arxiv.org/abs/1508.01991.Google Scholar
- [13] . 2016. Supervised and semi-supervised text categorization using LSTM for region embeddings. In Proceedings of the 33rd International Conference on Machine Learning, ICML 2016, and (Eds.), Vol. 48. 526–534.Google Scholar
- [14] . 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).Google Scholar
- [15] . 2015. Recurrent convolutional neural networks for text classification. In Twenty-ninth AAAI Conference on Artificial Intelligence.Google ScholarDigital Library
- [16] . 2021. Selecting the most helpful answers in online health question answering communities. Journal of Intelligent Information Systems3 (2021).Google Scholar
- [17] . 2019. Recording daily health status with chatbot on mobile phone - A preliminary study. In Twelfth International Conference on Mobile Computing and Ubiquitous Network, ICMU 2019, Kathmandu, Nepal, November 4–6, 2019. IEEE, 1–6.Google ScholarCross Ref
- [18] . 2021. An LSTM&Topic-CNN model for classification of online Chinese medical questions. IEEE Access (2021), 52580–52589.Google ScholarCross Ref
- [19] . 2021. Deep learning-based text classification: A comprehensive review. ACM Comput. Surv. 54, 3 (2021), 62:1–62:40.Google Scholar
- [20] . 2018. Natural language understanding for task oriented dialog in the biomedical domain in a low resources context. arXiv preprint arXiv:1811.09417 (2018).Google Scholar
- [21] . 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, New Orleans, Louisiana, USA, June 1–6, 2018, Volume 1 (Long Papers). 2227–2237.Google ScholarCross Ref
- [22] . 2021. Building blocks of a task-oriented dialogue system in the healthcare domain. In Proceedings of the Second Workshop on Natural Language Processing for Medical Conversations. 47–57.Google ScholarCross Ref
- [23] . 2014. Circumlocution in diagnostic medical queries. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. 133–142.Google ScholarDigital Library
- [24] . 2020. ERNIE 2.0: A continual pre-training framework for language understanding. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020. AAAI Press, 8968–8975.Google ScholarCross Ref
- [25] . 2021. Encoding syntactic knowledge in transformer encoder for intent detection and slot filling. In Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021. 13943–13951.Google ScholarCross Ref
- [26] . 2018. Task-oriented dialogue system for automatic diagnosis. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 201–207.Google ScholarCross Ref
- [27] . 2020. An attention-based multi-task model for named entity recognition and intent analysis of Chinese online medical questions. J. Biomed. Informatics 108 (2020), 103511.Google ScholarCross Ref
- [28] . 2020. SlotRefine: A fast non-autoregressive model for joint intent detection and slot filling. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16–20, 2020. 1932–1937.Google ScholarCross Ref
- [29] . 2019. End-to-end knowledge-routed relational dialogue system for automatic diagnosis. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7346–7353.Google ScholarDigital Library
- [30] . 2019. CRQA: Credibility retrieval for medical question answer service. In 2019 IEEE International Conference on Real-time Computing and Robotics (RCAR). IEEE, 347–350.Google ScholarCross Ref
- [31] . 2013. A text categorization method using extended vector space model by frequent term sets. Journal of Information Science and Engineering 29, 1 (2013), 99–114.Google Scholar
- [32] . 2017. Bringing semantic structures to user intent detection in online medical queries. In 2017 IEEE International Conference on Big Data (Big Data). IEEE, 1019–1026.Google ScholarCross Ref
- [33] . 2016. Mining user intentions from medical queries: A neural network based heterogeneous jointly modeling approach. In Proceedings of the 25th International Conference on World Wide Web, WWW 2016, Montreal, Canada, April 11–15, 2016. ACM, 1373–1384.Google ScholarDigital Library
- [34] . [n.d.]. CBLUE: A Chinese biomedical language understanding evaluation benchmark. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22–27, 2022. 7888–7915.Google Scholar
- [35] . 2021. Natural language processing for smart healthcare. arXiv preprint arXiv:2110.15803 (2021).Google Scholar
- [36] . 2021. Discovering better model architectures for medical query understanding. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Industry Papers. 230–237.Google ScholarCross Ref
Index Terms
- Conco-ERNIE: Complex User Intent Detect Model for Smart Healthcare Cognitive Bot
Recommendations
Automatic recommendation of medical departments to outpatients based on text analyses and medical knowledge graph
In many countries, outpatients generally visit a major hospital without a referral from health professionals due to the shortage of family physicians. Not knowing at which medical specialty department to register, outpatients have to wait in long queues ...
Intent and Entity Detection with Data Augmentation for a Mental Health Virtual Assistant Chatbot
IVA '23: Proceedings of the 23rd ACM International Conference on Intelligent Virtual AgentsWe report on implementing MIRA, a mental health resource chatbot to support healthcare workers in finding timely and relevant mental health resources. To generate appropriate queries to our carefully curated resource database, the chatbot must correctly ...
Comments