Joint discriminative and collaborative representation for fatty liver disease diagnosis☆
Introduction
Due to the changes in diet and lifestyle in western countries and many Asian countries, the prevalence of obesity and metabolic syndrome have increased dramatically, increasing the number of people suffering from the Fatty Liver disease (Hashimoto, Tokushige, Ludwig, 2015, Hashimoto, Tokushige, Ludwig, 2015, Amarapurkar, Hashimoto, Lesmana, Sollano, Chen, Goh, 2007, Amarapurkar, Hashimoto, Lesmana, Sollano, Chen, Goh, 2007, Deng, Dahmen, Sun, Huang, Sehestedt, Homeyer, Schenk, Dirsch, 2014, Deng, Dahmen, Sun, Huang, Sehestedt, Homeyer, Schenk, Dirsch, 2014, Bucak, Baki, 2010, Bucak, Baki, 2010, Gunasundari, Janakiraman, Meenambal, 2016, Gunasundari, Janakiraman, Meenambal, 2016). The type-B ultrasonic and Computed Tomography (CT) as the general methods of detecting the Fatty Liver disease are widely applied in most hospitals. Although adopting these two methods has achieved a remarkable performance, it is inconvenient for many people to diagnose in the daily life. Thus a novel approach for the Fatty Liver disease diagnosis is necessary.
The tongue or facial diagnosis as a staple has been practiced in the traditional medicine system for many years. Particularly, in Traditional Chinese Medicine (TCM) (Wang, Zhang, Yang, Wang, Zhang, 2013, Wang, Zhang, Yang, Wang, Zhang, 2013, Zhang, Kumar, Zhang, 2014b, Zhang, Kumar, Zhang, 2014b, Kirschbaum, 2000, Kirschbaum, 2000, Wang, Zhang, 2013, Wang, Zhang, 2013, Wang, Zhang, Lu, 2016, Wang, Zhang, Lu, 2016) for instance, the belief is that the color, texture and geometric changes of the patient’s tongue and face are capable of reflecting symptoms of a certain disease. The traditional tongue or facial diagnostic is heavily based on the practitioner’s years of experience, restricting its application in wider fields. Fortunately, some works quantitatively extracting and analyzing different features of tongue and facial images for diagnosis were established in recent years, and have shown their effectiveness in many disease detections. Therefore, in this paper, a novel multi-modal fusion method which considers the tongue and face simultaneously is proposed to detect the Fatty Liver disease.
Six centroids extracted from the facial color gamut were first presented by Zhang et al. (2014b). The facial color feature was then obtained according to the centroids, and achieved a superior performance in Diabetes Mellitus (DM) detection. In addition, the heart and hepatitis disease diagnostic systems based on facial color were proposed by Kim, Lee, Cho, and Oh (2008) and Liu and Guo (2008), respectively. Apart from the color feature, Zhang, Wang, Karray, Yang, and Zhang (2013) also took texture features into account by Gabor filter and achieved a prominent performance. Wang, Zhang, Yang et al. (2013) first designed an image capture device which not only gets a high-resolution image but also reduces the influence of different lights. Similarly to Zhang et al. (2014b), 12 color centroids were statistically studied from a large number of tongue images in Wang’s study (Wang, Zhang, Yang, Wang, Zhang, 2013, Wang, Zhang, Yang, Wang, Zhang, 2013) to reflect the color changes in tongue images caused by diseases. Simultaneously, a novel DM and Nonproliferative Diabetic Retinopathy detection method was presented in Zhang’s research (Zhang, Kumar, Zhang, 2014a, Zhang, Kumar, Zhang, 2014a) by exploiting the color, texture and geometry features of the tongue.
Although a number of tongue or face based diagnostic systems have been established, most of them considered either the tongue or the face independently and ignored the relationship between them. It is reasonable to assume that there is some latent and correlated information between the tongue and face modalities, and these types of factors may have a contribution for the overall classification performance. Specially, a practitioner may be difficult to make a diagnostic decision only based on tongue or facial features, while an integration of these two modalities may contribute to obtaining more accurate detection results. As shown in Fig. 1, we can see that some Healthy or Fatty Liver samples can not be detected by exploiting facial features but tongue features does. Similarly, in some cases, using facial images may fail to get an accurate diagnosis, but using tongue images may succeed. Moreover, there is a case that we can not get an accurate diagnosis no matter using the tongue or the face, but a combination of these two images may be possible. Therefore, the fusion of the tongue and facial features is crucial for diagnosis.
A naive way of fusing the tongue and facial features is to concatenate them as a single one. However, it fails to discover the latent correlation between them. By utilizing both of the tongue features and the facial features for classification, an efficient multi-modal approach is presented in this paper. Specially, our approach effectively exploits the similarity of two different modalities, extracting the relationship between tongue and facial features. Meanwhile, the discriminative information among each class is kept. In this case, the so-called joint discriminative and collaborative representation (JDCR) not only reveals the similar representation, but also encourages the representations of different categories to be more discriminative and de-correlate for the test sample (Xu, Zhong, Yang, You, Zhang, 2016, Xu, Zhong, Yang, You, Zhang, 2016). An optimization algorithm is presented to calculate the closed-form solution in a computationally efficient way. Finally, we apply JDCR in discriminating between Fatty Liver control. To be honest, especially in the Guangdong Provincial TCM Hospital, there is no accurate way to detect the fatty liver disease using only type-B ultrasonic or CT. These two medical imaging examination methods can only give an uncertain result that the patient may suffer from the fatty liver disease, but not an accurate diagnosis. In order to get a reliable detection, further examinations such as the liver function test and the blood fat test are needed. In practice, there are two ways for the accurate fatty liver disease diagnosis. The first one is a combined judgment. For instance, in the Guangdong Provincial Traditional Chinese Medicine Hospital, a patient is detected with the disease if and only if his or her type-B ultrasonic (or CT), liver function, and blood fat tests are uncommon simultaneously. The second method is the needle biopsy of liver (Evans, 1952, Evans, 1952), which needs to locate certain cells in the liver. It can be easily seen that both strategies are inconvenient, invasive and painful. However, our method finds the valuable information in the tongue and facial images to do the fatty liver disease diagnosis, which is quite easy to achieve. Despite the fact that our detection accuracy may be a little lower than that of the two strategies mentioned above, we aim to provide a self reliant device for people to diagnose by themselves at any time. If a person is detected with the fatty liver disease by the proposed method, we would advise him or her to go to the hospital to have a further examination.
We firstly describe the tongue and facial image capture device, as well as the feature extraction in Section 2. The proposed joint discriminative and collaborative representation algorithm is then analyzed in Section 3. The experimental results and the conclusion are illustrated in Section 4 and Section 5, respectively.
Section snippets
Image capture device and feature extraction
In this section, the tongue and facial image capture device is introduced, followed by the feature extraction of images in the two modalities.
Multi-modal joint discriminative and collaborative representation
In this section, we introduce a novel multi-model fusion approach called joint discriminative and collaborative representation (JDCR). As the proposed method is based on the collaborative representation, we first briefly review the collaborative representation based classification (CRC).
Image dataset
The tongue and facial dataset consists of 961 samples containing 500 Healthy samples and 461 Fatty Liver samples. In this dataset, each sample has two different types of images: tongue image and facial image, respectively. These images were collected at the Guangdong Provincial TCM Hospital, Guangdong, China, from the early 2014 to the late 2015. In the sampling process, people who were healthy were verified through a blood test and other examinations. According to the standardized rule
Conclusion
In this paper, an efficient multi-modal fusion approach is presented for Fatty Liver disease diagnosis. Different from existing strategies for Fatty Liver diagnosis, we exploit the valuable information of tongue and facial images. Firstly, the tongue and facial images are captured by exploiting a non-invasive capture device. The color, texture and geometric features are then extracted. In order to exploit the correlation between them, we propose a novel fusion method called joint discriminative
Acknowledgment
This work is supported by the GRF (15224015) fund from the HKSAR Government, the central fund from Hong Kong Polytechnic University, the NSFC fund (61332011, 61272292, 61271344), Shenzhen Fundamental Research fund (JCYJ20150403161923528, JCYJ20140508160910917), and the Science and Technology Development Fund (FDCT) of Macau 124/2014/A3.
References (36)
- et al.
Diagnosis of liver disease by using cmac neural network approach
Expert Systems with Applications
(2010) - et al.
Velocity bounded boolean particle swarm optimization for improved feature selection in liver and kidney disease diagnosis
Expert Systems with Applications
(2016) - et al.
Joint similar and specific learning for diabetes mellitus and impaired glucose regulation detection
Information Sciences
(2017) - et al.
Non-negativity and locality constrained laplacian sparse coding for image classification
Expert Systems with Applications
(2017) - et al.
Facial image medical analysis system using quantitative chromatic feature
Expert Systems with Applications
(2013) - et al.
Computerized facial diagnosis using both color and texture features
Information Sciences
(2013) - et al.
How can a sparse representation be made applicable for very low-dimensional data?
Expert Systems with Applications
(2017) - et al.
Sample pair based sparse representation classification for face recognition
Expert Systems with Applications
(2016) - et al.
How common is non-alcoholic fatty liver disease in the Asia-Pacific region and are there local differences?
Journal of Gastroenterology and Hepatology
(2007) - et al.
Nearest neighbor pattern classification
IEEE Transactions on Information Theory
(1967)
Limited correlation between conventional pathologist and automatic computer-assisted quantification of hepatic steatosis due to difference between event-based and surface-based analysis
IEEE Journal of Biomedical and Health Informatics
Nonlocally centralized sparse representation for image restoration
IEEE Transactions on Image Processing
For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution
Communications on Pure and Applied Mathematics
Needle biopsy of liver
British Medical Journal
Projective dictionary pair learning for pattern classification
Advances in neural information processing systems
Diagnosis and classification of non-alcoholic fatty liver disease and non-alcoholic steatohepatitis: Current concepts and remaining challenges
Hepatology Research
A proposal of heart diseases diagnosis method using analysis of face color
International conference on advanced language processing and web information technology, 2008. ALPIT’08
Atlas of Chinese tongue diagnosis
Cited by (34)
Hierarchical deep learning approach using fusion layer for Source Camera Model Identification based on video taken by smartphone[Formula presented]
2024, Expert Systems with ApplicationsA novel multi-feature learning model for disease diagnosis using face skin images
2024, Computers in Biology and MedicineMultiple color representation and fusion for diabetes mellitus diagnosis based on back tongue images
2023, Computers in Biology and MedicineMissing-view completion for fatty liver disease detection
2022, Computers in Biology and MedicineCitation Excerpt :For fatty liver disease, authors indicate the yellowing of the skin, and the eyes are possible symptoms of fatty liver disease in clinical studies [14]. Li et al. also initially investigated fatty liver disease diagnosis with tongue and facial images and obtained a good performance [15]. In addition, Traditional Chinese Medicine (TCM) also can support this point, as it believes various body regions that can indicate different statuses of candidates’ organs [16].
Double-cohesion learning based multiview and discriminant palmprint recognition
2022, Information Fusion