The IRMA project aims to develop and implement high-level methods for CBIR with prototypal application for medico-diagnostic tasks on radiological image archives[16]. The database for mammograms integrated to the IRMA project was developed based on the union of the DDSM, MIAS, the Lawrence Livermore National Laboratory (LLNL), and routine images from the university hospital of Rheinisch-Westfälische Technische Hochschule (RWTH) Aachen University, Aachen, Germany[15].

The DDSM database[17] officially contains 2,479 studies (695 normal, 870 benign, and 914 cancerous cases). Each study includes two images of each breast, acquired in craniocaudal (CC) and mediolateral (ML) views that have been scanned from the film-based sources by four different scanners with a resolution between 50 and 42 microns, providing a total of 9916 radiographs. Since image coding was originally proprietary, they had to be converted to a standard file format with special software: the C source code provided at the DDSM web page needed extensions to cope with endianness, palettes, and others[15]. For all cases, ground truth is provided in additional ACSII text files including the BI-RADS tissue type classification and type as well as resolution of the scanner used to digitize the film-based mammograms.

The MIAS database[18] contains only 322 mammograms, all of which were acquired in the ML view. Initially scanned from film with a resolution of 50 microns, all images were reduced to 200 microns and clipped/padded so that they fit into a 1024 × 1024 bounding box. The image files are available in the portable network graphics (PNG) format and annotated with the following details: a database reference number indicating left and right breast, character of background tissue, pathology, class of lesion present and coordinates as well as size of these lesions.

The LLNL database[19] contains a total of 197 mammograms that have been digitized at 35 microns per pixel. The images are stored in the image cytometry standard (ICS) format and had to be converted to a standard file format with a program provided as source code. For 190 images, there is a plain text file containing patient status, biopsy results and ground truth comments.

In order to evaluate the extensibility of mammogram reference resources, 170 cases were extracted arbitrarily from the picture archiving and communication system (PACS) at the Department of Diagnostic Radiology, University Hospital, RWTH Aachen University, Aachen, Germany. These images were acquired digitally using a General Electric Senographe operating with low beam energy about 26 to 32 kV and with a phosphor storage system from Fuji/Philips capable of recording 7 lp/mm (approx 70 microns). The cassette was read using a Philips PCR Eleva CosimaX. Where available, a free text diagnosis in German describing the breast examination, pathology, type of tissue and lesion was included along with the digital imaging and communications in medicine (DICOM) files.

To uniformly integrate all data into the IRMA system, the IRMA code[20] was extended for mammography, and based on the meta-information available with all the databases, all images were coded consistently according to the mono-hierarchical, multi-axial IRMA ontology[15]. In particular, there are four axes, each having three to four hierarchical positions, which describe: (1) technique: The imaging modality axis of the coding scheme is used to differ direct digital and secondarily digitized imaging and their resolution; (2) direction: The body orientation axis captures the CC and ML views; (3) anatomy: The body region examined axis holds information on the left and right breast; and (4) biosystem: The biological system examined provides three positions that code the tissue density according to the ACR classes, the tumor staging according to BI-RADS[3], and the type of lesion, i.e. micro or macro calcification, speckled or circumscribed masses, architectural distortions, and asymmetry.

In machine vision, an image is represented numerically by a so-called feature vector (also referred to as signature), preferentially at a low-dimensional space in which the most relevant visual aspects are emphasized[21,22]. Visually, breasts of fatty and dense tissues differ through gray level intensities (Figure 1). Since texture contains information about the spatial distribution of gray levels and variations in brightness, its use for breast density assessment becomes appropriate[23]. However, the high dimensionality of a feature vector that represents texture attributes limits its computational efficiency, so it is desirable to choose a technique that combines the representation of texture with the reduction of dimensionality, in such a way as to make the retrieval algorithm more effective and computationally treatable.

Open in New Tab   Full Size Figure   Download Figure

Figure 1 Mammograms of different breast tissues. From left to right: BI-RADS I to BI-RADS IV. Corresponding Image Retrieval in Medical Applications (IRMA) codes are given below the images.

SVD may satisfy these requirements representing the structure of the original data on a new basis in which the variables are ordered from the largest to the smallest degree of explained variation[24,25]. Wang et al[26] proposed a method of classification based on neural networks, in which the features used are the singular values of face images. For face photography recognition, the relevant technical properties of SVD are (1) stability (if a small disturbance is inserted in the image, the singular values alter only slightly); (2) algebraic properties, and; and (3) invariance of an image by the singular values.

Singular values represent important attributes of a matrix. As images can be observed as matrices, the singular values can serve as important features for evaluation of similarity. Also aiming to reduce the dimensionality and characterizing images in a medical CBIR system, Chen et al[27] applied SVD to represent color images of the stomach. SVD was performed on the color histograms to form a new feature vector. Concluding their study, the authors indicated the need for further studies to determine the optimal parameterization.

In general, let A be an m × n signature representing an image. Then, SVD is expressed as: A = UWV^T (1). where U and V are orthogonal matrices, W = diag(λ₁, λ₂, ¡ , λ_k) and is the matrix of singular values of A with λ₁≥λ₂≥_¡≥λ_k≥ 0 and k is the rank of A. For purposes of dimensionality reduction, suppose that matrix U_k of size m × k is composed of the first k leftmost singular vectors of U, matrix V_k of size n × k is composed of the first k rightmost singular vectors of V and the diagonal matrix W_k of size k × k is composed of the k singular values. Matrix A_k is defined as follows: A_k = U_kW_kV^T_k (2). The parameter k is crucial[27]. The smaller the value of k, the less storage and processing load is required, but if k is too small, important visual information is disregarded by the signature and retrieval results will become worse. In our study, k is determined by systematic experiments (see Experiments).

The support vector machine (SVM) method was initially developed to solve binary classification problems[28]. It guides the construction of classifiers with a good degree of generalization[29], i.e. with the ability of correctly predicting the class of a sample that was not used in the learning process. The use of SVM was extended to CBIR systems[30], where the similarity between images is measured by the relevance of an image to a particular query[31]. For instance, Yang et al[32] have used SVM specifically for CBIR of mammograms. However, SVM requires adjustments when applied to more than two classes, such as the four BI-RADS codes used for breast density classification.

Machine learning techniques may employ an inference principle called induction. The general conclusions are obtained from a particular set of examples[22]. In supervised learning, an external agent is used to indicate the desired answers to the entry patterns. The classifier is trained with a broad set of labeled data. Given a set of labeled examples as (x_i,y_i), where x_i represents an example and y_i denotes its label, one should be able to produce a classifier that can precisely predict the label of the new data. This induction process of a classifier from a sample of data is called training. The obtained classifier may also be seen as a function f that receives a dataset x and associated labels y. The labels or classes represent the phenomenon of interest on which one wants to make predictions. The labels can assume discrete values 1, ¡, p. A classification problem with p = 2 is called binary.

For a binary classification, SVM can be described as follows: given two classes and a set of points that belong to these classes, the SVM classifier determines the hyperplane in the feature space that separates the points in order to place the highest number of points of the same class on the same side, while maximizing the distance of each class to that hyperplane. The hyperplane is determined by a subset of items from the two classes, called support vectors. In most cases, however, the data set cannot be precisely separated by a hyperplane, so a kernel function is used instead. It receives two points x_i and x_j from the input space and computes the product between these data in the feature space. The most commonly used kernels are polynomial and Gaussian, in which the parameters must be empirically adjusted.

For more than two classes, this problem turns into a multi-class problem[33,34], which is the case of the MammoSVx system that works with four classes corresponding to the four BI-RADS categories for breast density.

There are two basic approaches for a multi-class application: (1) one-against-all: A SVM is built for each class through the discrimination of this class against the remaining classes. Successively, i.e. class by class, the decision is made. Let C denote the number of classes (here, C = 4). Then, the number of SVMs used is M = C-1. Hence, the MammoSVx system yields M = 3. Test data x is classified using a decision strategy, i.e. the class with the maximum value of the discriminant function f (x) is assigned to that data. All the n training examples are used to construct the SVM for one class. The SVM for one class p is built using the set of training data (x) and the desired outputs (y); (2) one-against-one: A SVM is built for a pair of classes through its training in the discrimination of two classes. In this way, the number of SVMs used in the method is M = (C-1)(C-2)/2 = 3. Here, one SVM for a pair of classes (p,m) is built using training examples belonging to only these two classes. This approach is a kind of generalization of the binary classification to more than two classes. Advantageously, all training examples are used at the same time[33].

With respect to the four BI-RADS tissue classes, both methods require three SVMs. In order to optimize the training phase providing all the data, we apply the one-against-one method to separate the four BI-RADS categories, two by two, for all experiments.

The MammoSVx system was implemented using MatLab (Matrix Laboratory), using the image processing and symbolic math toolboxes, and the LSSVM library[35]. It was executed on an Intel Core 2 Duo 2GHz processor with 3GB of RAM under the Microsoft Windows operating system.

The IRMA system is implemented in C and operated on a common Linux/Unix system. The PostgreSQL database is used to manage image, feature, and feature transform data[36]. Web-based access is provided by PHP hypertext preprocessor and the Smarty template engine (http://www.smarty.net)[37].

The methodologies were fused to MammoSVx (Figure 2). To remove noise, examination labels and other annotations from all images, regions of interest (ROI) were extracted. In a first step, a standardized image size was obtained shrinking all mammograms into a format of 1024 ×Z pixels, where Z varies according to the aspect ratios of the radiographs between 300 and 800 pixels. In this step, linear interpolation is applied. In the resulting scale, the size of the ROI that was extracted automatically was set to 300 × 300 pixels, which ensures the ROI contains tissue pattern only. Thereafter, the textural features were extracted using the SVD method, and compared to the features in the IRMA reference database using the SVM method.

Open in New Tab   Full Size Figure   Download Figure

Figure 2 Scheme of the MammoSVx content-based image retrieval system. From both the query image and images from the database, features are extracted and an index of similarity between these images is obtained. The most relevant images to the query are retrieved from the database and presented to the user.

Selection of image data: The IRMA database was merged from data of different sources and provides a reliable base for parameterization and evaluation of CBIR and CAD applications. Using the IRMA code, groups of reference images can be easily formed. As a result, the data used in the experiments were uniformly mixed from directly digital acquired and secondarily digitized mammograms of the left or right breast in CC or ML views with and without pathological alteration. However, the frequency of occurrence of tissue type in the IRMA database differs. To ensure an equal distribution for classification experiments, all images from the least frequent BI-RADS class are used, and the same amount is taken arbitrarily from the other classes.

Extraction of features: SVD was performed for all selected mammograms. The first k singular values were kept for the composition of the feature vector. The values for k used in the experiments were 25, 50, 100, 150 and 200. These values were chosen empirically in accordance with Elden and Andrews[25,38]. Optionally, SVD features were combined with the gray level histogram, as histograms have been successfully used in previous work[12,13,39]. In addition, we analyze the impact of the gray level histograms using this information solely for retrieval.

Evaluation of MammoSVx for CBIR: The CBIR task keeps the physician in the loop. Usually, the user presents an image, and the system offers a set of, for instance, ten responses, visually displayed to the physician, who can select appropriate information from the responses, or refine the query.

For the evaluation of the CBIR system, measures of precision and recall were obtained based on the top 10 retrieved images. Precision is the ratio of the number of relevant images retrieved to the total number of irrelevant and relevant images, whereas recall is the ratio of the number of relevant images retrieved to the total number of relevant images in the database[40]. Both measures are usually expressed as a percentage. We apply 5-fold cross validation and variance analysis (ANOVA) to obtain the best configuration of MammoSVx.

Evaluation of MammoSVx as a classifier: Furthermore, one can think of using the MammoSVx system as an automatic classifier. In this setting, the physician is excluded from the loop, and the system is used for automatic decision making. There are several ways of combining the ground truth of a set of retrieved images and forming a decision. The easiest, but usually not the best way, is to return only one image, and simply decide whether it is from the correct class or not. In doing so, the obtained results can be best compared with others.

Hence, this evaluation was performed measuring the accuracy, which is the percentage of correctly classified images of a certain class over the ground truth of the total mammograms in that class. For this experiment, we apply 10-fold cross validation.

Based on international standards such as ACR and BI-RADS, we provided a scheme to integrate available mammogram databases using a standardized description of imaging modality and resolution, orientation and view, left and right position of the breast, tissue type, tumor staging and lesion description. Integrating different resources that are freely available in the Internet, our database currently holds 10 509 images from 232 different code classes.

BI-RADS tissue class II was found to be most common with about 4000 entries, and BI-RADS class I was at least represented with only 1256 images. According to the protocol defined in the previous section, 1256 radiographs were randomly selected from all the groups yielding a total of 5024 mammograms.

Table 1 shows the resulting parameterization of the SVM with polynomial, radial, and linear kernels. Depending on the kernel, k = 25, k = 100, and k = 200 performed best, respectively. Therefore, the polynomial SVM kernel was superior since it needed the least number of data. Table 2 shows the overall results. In general, the combination of SVD and gray level histogram outperformed SVD and histogram feature extraction.

The best average precision of 82.14%, 71.75%, and 76.87% was obtained using the polynomial, radial, and linear kernel functions, respectively (Table 2). The ANOVA variance analysis yielded statistical significance. In order to verify the number of singular values that really represent breast density, the trace of the matrix A_k of singular values was examined. The last column in Table 2 shows the average amount of variables that are represented by the k singular values. The diagonal matrix has k singular values that are significantly larger than the others, and the zero singular values usually appear as small numbers. For all different values of k, only 24 singular values really represented breast density, and so the other values could be considered irrelevant. In conclusion, k = 25 appropriately represented breast texture in a lower-dimensional space with maximized computational savings, and the polynomial kernel significantly outperformed the other configurations.

Figure 3 represents an example of the MamoSVx system, with a query image of BI-RADS category I for breast density. All the top ten retrieved images are from the same category of breast density of the query image. In this experiment, the system was not designed to differentiate the projection (CC or MLO) as only a ROI was selected for characterization, which does not consider the pectoral muscle (signature of the MLO projection). Also, the distance of these retrieved images to the query image is presented, where images with distance values near 1.0 are closer to the query image. Time of retrieval was 3.85 s.

Open in New Tab   Full Size Figure   Download Figure

Figure 3 Retrieval example of the MammoSVx system. The retrieval is based on breast density, with k = 25 as parameter for singular value decomposition (SVD) and support vector machine (SVM) using the polynomial kernel.

Considering the classification experiment, an average accuracy rate of 76.4% was obtained.

Systems assisting radiologists in lesion detection and classification, or just presenting similar images to a sketched query are of great importance in clinical practice. They impact case-based reasoning, evidence-based medicine, and advanced blended learning techniques.

In screening mammography, the compilation of a large data collection with reliable ground truth is important to compile reliable systems for computer-aided diagnosis. Automatic detection of BI-RADS categories for breast tissue has not yet been solved, although several works on this topic has been published.

We provide a content-based image retrieval approach that relies on a large compilation of ground truth data in mammography. For computer-assisted diagnosis, breast tissue patterns are represented using a few singular values and a support vector machine for classification.

The Image Retrieval in Medical Applications framework provides server and client components to interface computer-aided diagnosis in routine clinical practice. A sample application is provided.

The topic is well presented and the article is reasonable. Some corrections need to be made, especially in the experimental section to improve the clarity.