1 Introduction

Medical imaging has drastically improved the clinician’s ability to diagnose a disease by providing an easy method to examine a patient’s body. While there are many different techniques to obtain these images, one of the most effective, computed tomography (CT) scanning, has seen a rapid increase in usage over the past couple of decades (Brenner and Hall 2007). However, due to its use of ionizing radiation, the patient is at a higher risk of encountering harmful side effects, especially induced cancer risks (Brenner et al. 2003; Huang et al. 2014; Pearce et al. 2012). Due to its increased usage, CT imaging was the largest source of medical radiation exposure in 2009, nearly half, and actually constituted 24 % of the collective effective dose to Americans (Schauer and Linton 2009). In order to optimize safety, it is necessary to keep the radiation dose as low as reasonably achievable (ALARA). Dose monitoring ensures that this practice is upheld among all patients.

In most hospital information systems, CT exams are stored in a picture archiving and communication system (PACS) after acquisition by a CT scanner. This is a digital information system that receives images from scanners, and stores and distributes them digitally to medical practitioners (Caffery and Sim 2010). To facilitate the handling of digital medical imaging data, the Digital Image Communication in Medicine (DICOM) standard was developed to store the images in PACS. It describes the standard mechanisms for data exchange and management (Wang et al. 2014). In addition to the image itself, the DICOM file contains a header, which includes data relating to the patient, scanner, and properties of the image. The header from the PACS proves to be a good target for programs, such as dose monitoring ones, that need such information.

As the clinical setting has become increasingly fast paced, clinicians accessing PACS expect that they can quickly and reliably obtain images for their desired exams. Any interruption to this workflow, even a slower acquisition of images, can interfere with patient care and is highly undesirable. Thus, it is essential to ensure that a dose monitoring system is transparent to the workflow. In order to achieve this transparency, a seamless integration of the system with the PACS is needed.

Furthermore, a wide variety of factors can interfere with such systems, and these need to be considered. For instance, there are many types of scanners, and they store their radiation data in a variety of ways. A dose monitoring system would need to be able to handle the differences inherent in these scanners. Not only do the scanners vary, but the clinical protocols do as well. These protocols cannot be judged on the same basis because some, due to their nature, would require higher amounts of radiation, so it is important to handle different protocols individually when identifying dose outliers. Also, each patient is unique, and characteristics such as body size are important when assessing appropriateness of different radiation dosages.

Not only will these systems have to be able to handle the variety in the field, they will need to be capable of working with the data effectively. At the NIH Clinical Center, 30–90 exams are performed each day. The exams have an average of 200 images (min 10, max 4899), and with each image being half a megabyte, a single study is 100 MB large. After even 1 month of dose monitoring, a large amount of data (about 100 GB) is processed. The header information constitutes roughly 1/25th of these data. The engine has to be capable of working with such large amounts of data. This includes determining body volume from the images themselves as well as organizing all of the header data into meaningful categories, so that proper analysis can be performed. After all of the data handling is done, it would also be advantageous if the system could automatically produce images presenting the data in a meaningful way, to aid human review of the outliers.

To facilitate radiation dose analysis, there has been research describing ways to extract and save radiation data from a medical image archive, such as a PACS (Larson et al. 2013a, b; Sodickson et al. 2012; Wang et al. 2011). Sodickson et al. developed an open source toolkit that uses optical character recognition (OCR) on CT dose screens to obtain exposure metrics on a large scale (2012). From personal experience, we have discovered that OCR can have trouble with dose pages from Toshiba scanners. Wang et al. created a system that supports the extraction of dose information from the DICOM header for a multitude of modalities (2011). For CT, they extract CTDIvol and DLP from the header, though DLP is only available in structured reports, which are not always available for each exam. Larson et al. developed and showed the efficacy of a dose optimization model that takes into account the water-equivalent diameter of a patient, CTDIvol, and image noise (2013a, b). Such methods can result in the collection of large amounts of dose-relevant data. It is desirable to analyze these data, similar to what Sodickson had done for chest, abdomen, and pelvis exams, for trends and pinpoint areas for further investigation to reduce doses and optimize scanning protocols.

Data mining provides a great way of analyzing large amounts of data that are already being stored from dose monitoring systems. Many medical specialties have begun to embrace this technique and have demonstrated its utility in a variety of situations, such as the identification of associations between age on drug-associated liver injury to the prediction of the benefits of cochlear implants, among numerous other applications (Hunt et al. 2014; Ramos-Miguel et al. 2014). Recently, Kharat et al. discussed the potential for data mining to be used in radiology, proposing such benefits as identifying the best imaging protocols, and aiding with treatment planning (2014).

Data visualization methods are very useful for this analysis because it allows for an easy and interactive method of noticing any trends or outliers within data. When given raw data, it can be difficult to identify obscure relationships. However, once the data are properly prepared, visualizing data on a variety of parameters can reveal associations that would otherwise be difficult to determine. Indeed, its utility as such is exemplified in a paper where it was combined with data mining and used to distinguish Hepatitis B and C based on different patterns of complex lab results (Takabayashi et al. 2007). This usefulness has also been demonstrated in relation to a variety of other topics, including genomic analysis (Boyle et al. 2012). Here, we demonstrate how useful such techniques can be used to understand dose data.

In this paper, we present novel ways to retrieve, mine and visualize a large amount of radiation dose data from CT to reveal insightful patient specific, protocol specific, and scanner specific relationships. We first discuss a recent engine that we developed to obtain the requisite data, followed by an analysis of a year’s worth of radiation dose data coupled with discussions about the methods to visualize this information. The engine is designed to run fully automatically in the background, and its retrieval can be controlled by either a configuration file via a text editor, or a web interface for a more interactive setting. While the techniques that we use are moderately novel, the system that combines them to form a dose monitoring system based on personal information is.

This system is intended to be used by CT dose monitoring teams that are interested in examining their data in a more accessible fashion. Rather than relying on impractical methods, such as manual transcription of doses, or unreliable ones, such as OCR, we hope to provide a system that automatically obtains the requisite information that allows for patient specific dose analysis. This includes information (scanned body volume) that cannot be gathered via other methods outside of image processing. By allowing dose monitoring teams to more easily obtain relevant information, they can efficiently examine how well their dose initiatives are being enacted, and, through its dose monitoring, ensures that all current exams are consistent, both intra-and inter-institutionally. If any discrepancy arises, the team could take appropriate action to ensure that their doses are ALARA. Furthermore, the patient specific dose analysis will allow clinicians to track the cumulative dose of a patient, allowing them to better judge the risks of further radiation against the benefits of a CT scan over other modalities.

2 Terminology

The following are a few specific terms used in radiation dose that will be referred in the manuscript.

Accession number:

The code used to identify an order for a study

Clinical protocol:

The general exam that is being performed, such as CT Neck, or CT Cerebrum. Compare with scanning protocol, which is more specific

CTDIvol :

Volume weighted CT Dose Index; Standardized measure of radiation exposure from a CT scanner, based on measurements performed on a phantom (McCollough et al. 2011). Measured in mGy. It is now commonly provided in the DICOM header

DLP:

Dose length product; the product of CTDIvol and scan length, and can be considered an expression of total radiation energy imparted to a patient, and can be used to characterize exposure (Jessen et al. 1999). Measured in mGy-cm

Dw :

Water Equivalent Diameter; the diameter of a circle of water that contains the same total attenuation as the slice, thus depending on its attenuation and area (Ikuta et al. 2014). (Fig. 2c). Measured in cm

Effective dose:

A dose descriptor reflecting the differences in sensitivities to radiation of different regions of the body (McCollough et al. 2008). Measured in mSv

Multi-phase exam:

A type of scanning protocol that involves multiple scans of the same body region. The different phases correspond to different temporal phases of contrast, e.g., none, arterial phase (immediately after administration of contrast material), and a delay of several minutes or more after administration

Overscan:

The extra partial or full rotations required at the ends of a scan due to interpolation methods of helical scanners (Tsalafoutas 2011)

SBV:

Scanned body volume; the total body volume, excluding air, of the region being irradiated (Fig. 2a, b)

Single-phase exam:

A type of scanning protocol that only involves a single scan of the region of interest

Scanning protocol:

A protocol that details the specific settings, such as kVp, number of phases, and body region to be used for the exam

3 Data retrieval

We developed a radiation exposure extraction engine (RE3) to collect CT radiation dose data (Weisenthal et al. 2014). The diagram of RE3 is shown in Fig. 1. In short, RE3 first queries the PACS for the desired studies and series. It then downloads the relevant series to the computer, and retrieves the image data and DICOM headers. With this information, dose information is calculated and analyzed, and outlier exams are checked for.

Fig. 1
figure 1

RE3 flow chart. Specific queries at the study and series level, discovers studies in the PACS that are then retrieved to the local drive. Radiation data is extracted from DICOM images, allowing for analysis, monitoring, and reporting. Created using Microsoft Word

Full size image

3.1 System requirements

Using this engine requires the installation of a few other, open source packages. This includes three modules from the DICOM Toolkit (DCMTK): FINDSCU, MOVESCU, and DCMDUMP (http://dicom.offis.de/dcmtk.php.en). The first two help with the interaction between our computer and the PACS, while the latter transfers the DICOM header information into a readable text file. The other major dependency is the command line plotting program, Gnuplot, for its use in the generated reports (http://www.gnuplot.info/).

3.2 Open source availability

The source code of RE3 can be downloaded at our github site.Footnote 1 More specific details on downloading this system can be found there as well. In short, the code has to be downloaded from the site, and DCMTK and the Perl modules will have to be downloaded as well. Both the PACS and server’s IP address, AE title and port needs to be supplied to the configuration file. Paths to the files used, including the DCMTK modules, the Perl path, and the body volume calculation program needs to be recorded in the configuration file as well. Once this has been set-up, the program can be run.

3.3 RE3

A configuration file in xml format exists that controls all of the settings for this engine. Before RE3’s first run on a new computer, essential information needs to be updated in this file, such as the paths to its dependencies, and the AE titles, IPs, and ports of the PACS and the local computer. The configuration file also controls the filter to retrieve studies and the running schedule.

The engine utilizes modules from the open source package DICOM Toolkit (DCMTK) for the retrieval of images. RE3 first uses the DCMTK module FINDSCU to query the PACS for the requested exams. If the engine is running prospectively, this query occurs every n minutes, where n is a user defined number in the configuration file. Once found, the engine looks at each study individually, and starts its examination of a new study by querying for its series.

RE3 employs a two-layer filter to prevent the inclusion of a duplicate series into the dose calculation. DICOM is organized in a hierarchical manner: at the highest level, there is the Patient, followed by the Study, then the Series, and, at the lowest level, the Image. The dose information is located in the images; however, images reconstructed from the same scan, and thus including the same dose information, can be found in different series located both within the same study and different ‘studies’.

Duplicate series within the same study occurs very frequently because the CT information from a scan can be reconstructed with different kernels to highlight certain features better. The first layer of the filter addresses this problem. By examining the ‘image type’ DICOM tag of images within a series, those that contain ‘derived’ images, or those whose pixel values are based on other images, are removed. Removing derived series poses no threat to the integrity of the dose calculation because the presence of such an image implies that a series containing ‘original’ images, those whose pixel values are based on the raw data, is present. The content time of the images within the remaining series are then compared, and if there are series that occur at the same time, then the longer series is kept. If they are of similar length, then the series containing less images is retained in order to minimize download time. The benefit of initially using content time over other DICOM tags, such as acquisition time or irradiation event uid, is based on speed. Unlike most tags, content time can actually be queried by the C-Move operation used in FINDSCU. This allows this filter to work without downloading the images, improving the engine’s speed immensely. However, another filtering does occur after the images are downloaded to ensure that no series are duplicates. Here, it uses acquisition time, in a similar manner to content time, and it ensures that the acquisition numbers of the studies are unique.

Less common are the duplicates among series of different studies. Ideally, each study is unique; however, due to clinical management workflow, series can be duplicated into multiple studies. To ensure that such replicates are not included, the PACS is queried for all of the studies performed on the patient on that day. The engine performs the algorithm used previously to compare these series with the remaining ones from the first pass and marks which ones should be kept or not. It might seem intuitive to have used study level DICOM tags such as study ID or study time for this filter; however, they have proven to be ineffective because they are treated as distinct. Also, when this division occurs, the series in the different studies will occasionally only partially overlap each other. In this case, both series would be taken, but will be analyzed under their own clinical protocol labels. While this may not give an accurate DLP calculation for the patient overall, it allows better analysis for dose outliers. For example, if a scan combine Head and Neck scans and they are split as described above, then the Neck may have higher dose than Neck scans by itself and keeping this series will allow for identification of this outlier.

In the unlikely circumstance that these fail, a backup system is in place that checks to ensure that the chosen series are not duplicated of each other by examining DICOM tags that can be retrieved from the images that are not available in queries, such as acquisition time and acquisition number. The entire filtering system has proven to be valid at selecting the correct series and we have noticed few incorrect selections while it has run. The major problem with this method is that it is dependent on the images that are in the PACS, so if the images from a scan are not uploaded, then it will not be able to calculate the dose properly. To validate this selection, we have examined a week of studies, totaling 256. Of these, only 7 differed from the dose page, and all of these were because they were part of a protocol that did not upload all of their series to the PACS.

Once the unique series have been identified, their images are downloaded to the machine by the MOVESCU module from DCMTK. Another module, DCMDUMP, extracts the DICOM header from the images and saves it to a text file. Many useful tags can be extracted from this file for further analysis, including patient related information (MRN, name, age, and gender) and protocol related information (clinical and scanning protocol name, scanner used, kVp, current, pitch, collimation values, date, and body region).

The tags used for DLP calculation that are extracted from this file include CTDIvol from each slice, the number of images in the series, the exposure of the first and last slice, slice thickness, and the slice positions. As seen in Eq. 1, the DLP is calculated by averaging the CTDIvol of each slice, multiplying it by the scan length (SL), and adding an estimated value for the radiation due to overscan (OS) from an overscan model trained by previous studies, which uses the radiation exposure of the exam as a predictor. There are two values used for the scan length of the series. One is the difference between the slice locations of the first and last slice of a series. This is used for most exams, when slices are obtained contiguously. If this is not the case, then the scan length is calculated by multiplying the slice thickness with the number of images. A rough estimate of the effective dose can be calculated by multiplying the DLP with coefficients, sepcific for the body region, found in (McCollough et al. 2008).

$$DLP_{series} = \frac{{\sum CTDI_{{vol_{slice} }} }}{ImageCount} \times {\text{SL}} + {\text{OS}}$$
(1)

Once determined, this information, along with patient and study relevant data, is saved to comma-separated-value (csv) files, one for the study level and one for the series level; however, it should be noted that any header information can be saved to these files should they be deemed useful. Then, an automated image processing tool is run on the downloaded images to measure the SBV (Fig. 2a, b) and Dw (Fig. 2c) for each study, which is also saved to the files (Yao et al. 2011). The concept of selective, automatic retrieval of images from the PACS and processing them through a computer-aided detection system, similar to what is being done here, is described by Huang et al. (2013). Dw of a slice is calculated by Eq. 2, where the Hounsfield number for water is 0 (Ikuta et al. 2014). Figure 2 shows what these two values look like based on a real case.

$$Dw = \sqrt {\sum \frac{{\left( {pixel HU + 1000} \right)}}{1000}*pixel area*\frac{4}{\pi }}$$
(2)
Fig. 2
figure 2

Examples of body size measurements. a SBV: A representation of the total volume that is being calculated. SL is the scan length. b An axial slice from a shaded red showing exactly what is being considered by the imaging tool when it is calculating body volume. c Dw: A diagram representing the water equivalent circle of the slice in b (Color figure online)

Full size image

With multi-phase exams, there are multiple ways of handling the calculation of SBV: summing each individual scan’s value, averaging them, taking the minimum, or taking the maximum. A summation of the values does not represent the same area being scanned multiple times. Between the three other methods, we chose to take the maximum because it represents the entire region that is being irradiated, while the other two may ignore portions that were actually scanned.

Once these data are gathered, an informative dose report can be generated on either a daily or weekly basis. It includes a general overview of all of the radiation dose outliers (see Sect. 5.1 for more information) for the specified time period, followed by a page with patient and study-related information for each outlier, an example of which is presented in Fig. 3.

Fig. 3
figure 3

An example of a study specific dose page included in the report that is generated weekly or daily when RE3 is run prospectively

Full size image

A series of figures are included at the end of the report to aid its understanding. This includes simple descriptive graphs, such as a histogram of all doses received by the patients, a scatter plot of all doses plotted against age, a pie chart describing the ages of the outliers, and two box plots of the doses, one for children, and the other for adults. Also included is a scatterplot of the studies within a certain protocol, which is described in more detail in Sect. 5.1.

The engine is fully automated and seamlessly integrated with a Clinical PACS with a small overhead (without interrupting the current workflow). The engine can be run in three different modes: on-the-fly, scheduled, and user-request. The mode is selected in the configuration file. If it is set to run on-the-fly, then RE3 constantly runs, checking for any new studies to process. To reduce workload on the PACS, RE3 sleeps for a user-defined interval in between checks. However, if the user wants a tighter control over the times that RE3 runs, it contains a scheduled mode. In the configuration file, the user can specify a certain time period that it is allowed to run, and if it is not within that period, RE3 sleeps. This conveniently allows RE3 to be run overnight, so it does not put an extra burden on the PACS as the studies are being uploaded to it. While these modes allow for dose monitoring of current exams, the user-request mode allows the user to process studies that have already been performed. In this mode, the user can either specify the exact accession numbers of the studies to be examined, or specify a date range. For the latter option, all the exams in the date range are examined. These requests can also be under the scheduled mode, so if there is a large request, such as for months of exams, it is possible to ensure that RE3 is only run at night. These different modes allow the user to fine tune the workload on the PACS during the day, and enable him to collect previous data for analysis.

While these different modes can be manually changed in the configuration file, we have set up a basic web UI using the open source Mojolicious framework for those who do not want to repeatedly tamper with such a file. From this interface, the user can choose to either set up the engine to look for more cases or to only create a report based on collected data. For the former, the user is able to choose among the aforementioned modes. For the latter, the report can be specific for a given date range, clinical protocols, pediatric cases, and/or a certain exam. This interface gives the user much easier access to the engine and its outlier detection capabilities.

3.4 Security

There are three layers of security for the data access: institutional firewall, user login and password, and file encryption. While the first two layers are always in place, we have included an option to toggle the file encryption, in case the users believe that the other layers are sufficient for their purposes. The encryption is done by changing the character encoding before printing them to the results file. The default encoding that it would change to is CP1026, but this can be changed in the configuration file as well.

4 Retrieved radiation dose data

Using RE3, we collected CT radiation dose data from November 2013 through October 2014 for a total of 12,633 studies. 32 (.2 %) data sets were excluded due to failed body volume calculations. This was mostly because the number of images in a series was not within the user-defined range. An upper threshold of 1000 images is used to prevent running out of memory when the volume calculations are done, while a lower threshold of 10 images is used to ensure that there are enough images to reliably calculate the body volume. The studies were from 5543 patients. Information about these patients can be found in Table 1.

Table 1 Patient cohort demographic information
Full size table

There were 61 different clinical protocols, with a total of 266 scanning protocols associated with them; 80 scanning protocols had at least 20 studies. The pie graph in Fig. 4a displays the distribution of the most common clinical protocols, while Fig. 4b shows the top three common clinical protocols and the most frequently used scanning protocols used in those types of studies. In Fig. 4b, regarding the CT CAP clinical protocol, all three scanning protocols are CAP scans with contrast, but they are performed on different scanners. Also, the protocol “chest abd pelvis w” within the CT Neck studies is not a mistake of the data retrieval; rather, these were the studies’ actual labels because they followed a CT CAP exam of the same protocol. Other study information can be found in Table 2.

Fig. 4
figure 4

Common protocols from November 2013 through October 2014. a The five most frequently used clinical protocols during the year. CT CAP refers to Chest, Abdomen, and Pelvis exams. b The most used scanning protocols within the top three clinical protocols. The scanning protocol CHEST_ABD_PELVIS_W in the CT Neck protocol is strangely named because it follows a CT CAP exam. Created using Microsoft Excel

Full size image
Table 2 Scanning protocol general information
Full size table

5 Data mining and visualization

To extract useful information from this large, noisy and high dimensional dosage data, several data visualization and mining tools are explored. Orange, an open source python program, was chosen for its ease at accessing all facets of the data (Demšar et al. 2013). One way it does this is through the use of user friendly widgets. Figure 5 is an example of a data flow using these widgets, starting from a file containing the data, filtering it by a specific scanner and protocol, and ending up at the desired scatter plot. While we used it mostly for these types of visualization, it has the capability of producing many others, including survey plots, and mosaic displays, among others.

Fig. 5
figure 5

A sample workflow used in Orange. Here, data is loaded from a file, filtered by scanner, and filtered again by protocol, with scatter plots available after each filtering. The full data table is visible at any of these points

Full size image

The other major tool used was Gnuplot, a popular, command line plotting program. Using these tools, we can conduct different analysis on the data, including scanner-based analysis, protocol specific analysis, and patient specific analysis. Through these analyses, abnormal patterns and outliers can be discovered. In the following sections, we present several schemes to visualize and mine the large radiation dose data.

While most of these tools and graphical structures have already existed before, we aim to evaluate their utility in the realm of radiation dose monitoring, especially when examining specific scanning protocols. This also demonstrates how RE3 is able to organize the retrieved data into a file format that can be easily used with other visualization interfaces. We also describe an outlier detection system and reflect on its efficacy during its initial trial period.

5.1 Scatter plots

Scatter plots are one of the basic plotting tools used throughout the scientific community, and for good reason. They are easy to understand, able to handle large data, and can easily display any 2-dimensional correlations among variables. Furthermore, they can still be useful at plotting a third discrete variable, by altering the color scheme.

Figure 6 is a scatter plot between SBV and DLP to model the dose distribution of Chest Abdomen Pelvis (CAP) exams acquired by only three different scanners, as any more would result in an illegible graph, which is a slight problem with scatter plots of this size.

Fig. 6
figure 6

DLP of CT CAP exams with respect to body volume for 3 scanners. The SOMATOM Force scanner exposes patients to less radiation in general. The Definition AS has two clusters: one equal to the Flash, the other the Force. Created in Orange

Full size image

Through even a cursory glance, important trends can be identified. The most obvious one is that when taking the scanned volume into account, the SOMATOM Force scanner exposes a patient to the least amount of radiation, compared to the other machines, on average. This trend also occurs when compared with the machines not included in this graph. This difference is to be expected because the Force has many features that are conducive to lower doses, including the use of higher resolution detectors, faster data transmission and an enhancement of the z-flying focal spot approach (SOMATOM Force 2013).

This trend may be overshadowed by the large distribution of points that is present above the major dense clusters of the scanners. A closer examination of these data is necessary to understand what has happened with these points. To reduce any confounding factors associated with the difference between scanners, it is important to examine a specific scanner by itself (Fig. 7).

Fig. 7
figure 7

Radiation exposure on the Flash scanner for exams with different phases. The multi-phase exams tended to have higher DLP values than single phase, except for one cluster (circled). Created in Orange

Full size image

After experimenting with several parameters to determine the difference between the exams, it was discovered that most of these exams were multi-phase exams. As can be seen in the graph, scans that have multiple acquisitions tend to be above those with only one. This conclusion makes logical sense because if the same area is scanned multiple times, that area should be exposed to more radiation. The wide distribution may be due to the way that RE3 handles SBV as discussed in Sect. 3.

By looking at the number of scans taken in a study, an interesting discovery appears: there is a line of studies consisting of two acquisitions that is consistently beneath the majority of single phase studies. Examination of these specific cases revealed that they are all part of the same protocol: DE_UOB_CAP_LiverVNC (a dual energy, liver multi-phase exam where the non-contrast phase is performed virtually). It makes sense that there would be lower radiation exposure as the first acquisition is considerably shorter than the second, and the second was a dual energy scan, which should give lower DLP values than pure 120 kVp exams.

5.2 Heat maps

An alternative to the scatter plot is the heat map. Instead of plotting individual exams, a heat map groups them before plotting, making the data less crowded. A major advantage of a heat map is that it allows a third continuous variable to be plotted as the color of the boxes. In this case, the average DLP can act as this third variable, leaving the other two axes free for other parameters. Due to the size of the boxes and its grid-like nature, certain relationships can appear better.

This set up is ideal for investigating a specific protocol over a variety of variables, such as study date. As mentioned in Fig. 6, the studies performed on the Definition AS scanner demonstrated a strange clustering: two clusters with one higher than the other. However, when this was investigated, only two protocols showed this discrepancy: NIH_CAP_W (Chest, abdomen Pelvis with contrast) and NIH_CAP_WO (without contrast). Knowing this, a single protocol, NIH_CAP_W, was investigated, but the only difference that could be found was that the lower cluster had occurred only during the month of May (Fig. 8a).

Fig. 8
figure 8

Heat maps of NIH_Cap_W exams on the Definition AS, SBV versus Month. a The colors represent DLP intensity. In May, there was a decrease in DLP. b The colors represent number of studies. The later months had a sharp decrease in protocol usage, and indeed, the protocol was not used during September, November, or December because this scanner was being phased out during this time. Created in Gnuplot (Color figure online)

Full size image

With a heat map, the decrease in radiation exposure during the month of May becomes noticeable. However, no data from the DICOM headers has proven useful to identify how this occurred, so investigation into this specific case is still in progress. The identification of this oddity is important because it gives an opportunity to learn about a systematic influence, which may be useful for the future.

While it is able to demonstrate the dip during the month of May, it still has its own drawback: the number of patients in each cell cannot be determined. This requires the creation of a second heat map (Fig. 6b), whose color represents the number of patients contained within the cell, as opposed to the DLP. By comparing these two graphs, it is possible to understand what the initial map showed, with a sense of the density. With this comparison, we can believe that the exhibited change is not due to random chance, as a similar number of patients underwent this exam from February through July, though we do notice that it was becoming infrequent from August onwards.

5.3 Distribution plots

Looking at the specific protocols on a scanner also serves a very useful monitoring purpose. Through the use of distribution plots, all the relevant protocols can be easily assessed without worrying about too much crowding, like in a scatterplot. In order to demonstrate these techniques on protocols other than those for the CAP exams, a distribution plot was created for the CT Cerebrum protocols (Fig. 9).

Fig. 9
figure 9

Distribution plot of CT Cerebrum exams. The PEDS_HEAD exam had lower DLP values, with the force having lower DLP values when used. Created in Orange

Full size image

Such a graph easily demonstrates the differences in DLP among all the protocols. For instance, the PEDS_HEAD protocol (a protocol for pediatric patients) tends to have smaller values, as one would expect, because children both have smaller heads, and should be exposed to less radiation. Also, in the Stealth Brain protocol, there were a few exams with lower DLP than the others, and by coloring the points based on their scanner, it appears that this reduction is due to these scans being run on the Force.

One disadvantage of this graph is that body size (head size in this case) is not taken into account. However, the distribution of sizes among clinical protocols should be similar. The best way to improve this graph would be if size-specific effective doses could be calculated for all of the exams, and then be plotted.

5.4 Bubble plots

One of the most important traits of a patient is their pediatric status, as they should receive lower doses than adults. To ensure that the important aspects of a patient were considered, bubble plots can be created such as Fig. 10.

Fig. 10
figure 10

Bubble Plot of the CAP 120 K protocol on the Force. Dashed line is 18 years, and bubble size represents DLP. Created in Orange

Full size image

The bubble plot includes both a patient’s SBV (x-axis) and pediatric status (age on y-axis), while also allowing easy visualization of the received DLP. It is very easy to identify the pediatric cases by following the y-axis. Furthermore, each can be colored to reveal more useful information. In general, the smaller bubbles should appear below an age of 18 (pediatric), as well as closer to the left, where the body volume is smaller. In this figure, there were bubbles larger than they should be, but through an appropriate coloring schema they appear green, signifying that they were multi-phase exams, suggesting that they were mislabeled. The graph essentially shows four dimensional data.

6 Patient specific analysis

Because radiation doses are heavily impacted by the physical properties of a patient, it is imperative to ensure that a patient is getting a dose ALARA for that specific patient. One of the most defining characteristics that affects a patient’s dose is his or her body volume. Having a larger body volume reduces the dose a person receives from a specified amount of radiation compared to a person with a smaller body volume. Because DLP does not take patient size into account, most of the graphs were shown with respect to the scanned body volume. This property also means that dose monitoring systems cannot be based purely on resulting DLP values. Here, we present a system that considers such personal information.

6.1 Outlier detection

To account for these patient-specific differences, RE3 uses a multivariate, linear regression model whose parameters are the patient’s scanned body volume, age, effective water diameter, gender, and scan length to detect outliers. There is no need to determine the correct parameter weights via training and testing sets because this is a live, unsupervised model. Effectively, all the prior studies constitute the training set, while the study that the model is being run on is the testing set.

RE3 also creates a unique model for each clinical protocol. Clinical protocol is the best choice for the division of overdose models because it effectively splits up the exams by the body part being examined, while also taking into account the type of exam taking place, such as a Chest versus a High-Resolution Chest. As can be seen in Fig. 11, different scanning protocols tend to have distinct patterns, necessitating the use of a multitude of models. Model creation runs iteratively, excluding any outliers, by three standard deviations, from the data set for the regression, until the model converges. This increases the recall, but decreases the precision of this model. All of the outliers are recorded in their own file as well, for ease of access.

Fig. 11
figure 11

Different study protocols on all of the machines. Each tends to have their own distinct clustering. Created in Orange

Full size image

Figure 12 plots the observed DLP versus the DLP predicted by the regression model. It demonstrates how well the model can predict the majority of exams, while also showing how significant the difference between an outlier and the majority of points is. It is easy to spot which exams need to be looked into, such as patient 1, who received 2800 mGy cm more than expected, though this was because it was a mislabeled extremity exam. Another major outlier was patient 2, receiving 2000 mGy cm more than expected. This patient was larger than most, so it is possible that parts of him or her lay out of the image, resulting in an underestimation of the SBV, but it still is possible this is an outlier case. Patient 3 was another outlier, receiving 800 excess mGy cm. This was due to the mislabeling of a multi-phase exam as single-phase.

Fig. 12
figure 12

Graphic representation of the regression model for CT CAP exams in December, 2013. Created in Gnuplot

Full size image

6.1.1 Validation

In order to evaluate our outlier model, we examined all identified outliers, randomly selected 50, and checked to see if there could be cause to discount it as one. There were 544 (4.3 %) studies that were considered outliers. Out of the 50 randomly selected studies, 11 seemed to be legitimate. The 39 false positives could be broken down into four groups.

The largest group consisted of 18 studies labeled as outliers due to mislabeling. The majority of these were multiphase CAP exams that were merely labeled as CAP exams; however, this trial took place during a transition state to ensuring that all multiphase exams were labeled with their own unique clinical protocol, thereby removing this problem for future use. The next largest false positive group with 13 included those due to insufficient data. Each clinical protocol requires at least 20 cases before a sufficient model was created. When a study is encountered before its model is created, the outlier system is based on an old default model, mostly based on CAP exams. Because this was our first run, this lack of models occurred frequently towards the beginning, and these errors should not occur in future trials.

The two remaining groups are limitations of the current system. In six of the selected studies, the field of view of the images was too small for the body size, resulting in incorrect calculation of their body volume. The model incorrectly believes it sees a higher dose for a smaller body size. The two remaining outliers occurred because the CAD system was unable to calculate the body volume for these images, resulting in an immediate flagging for further study. While there were a few problems during this trial period, many of the false positives would not be a problem in future studies.

6.1.2 Comparison to current method

Currently, the way to determine whether a patient has received too much radiation in a study is by using a threshold of 50 mSv. We compared this method to our own for several clinical protocols.

The threshold method is most likely to call outliers in multiphase CAP exams because this protocol results in the largest amounts of radiation. The 50 mSv threshold is roughly 3333.33 mGy cm when converted to DLP via k factors (McCollough et al. 2008). Seventeen exams matched this criterion. However, only two cases were classified as outliers by the regression model, and these cases were in agreement with two that were found by the threshold method. This means that 15 cases were in disagreement between the two models, all claiming to be outliers by thresholding, but not the regression. Looking at these 15 cases, all of them were larger patients who needed a larger exposure in order to get adequate image quality, and so it seems more likely that they are not outliers. However, as k factors are conversion factors for ‘normal’ body size, the DLP threshold should be larger for these individuals. Currently, though, such a system to take body size into account is not in place.

Furthermore, the threshold based system can only discover cases where the radiation is considered harmful; it does not enhance the ability to keep doses ALARA. This is well demonstrated in our experience with high resolution chest CTs. For the chest region, the DLP threshold would be about 35,714 mGy cm via k factors. Out of 268 high resolution chest CTs, not a single exam exceeded this value. Meanwhile, RE3 repeatedly identified exams performed under a specific protocol as to be low outliers. Investigation into this distinction revealed that the scanner with higher doses still performed a helical scan, as opposed to the thin cuts that the other scanner used. By using RE3 we were able to identify such a discrepancy efficiently.

6.2 Cumulative doses

In order to get a comprehensive understanding of a patient’s dose, it is important to not only keep track of doses from a single study, but also from a cumulative standpoint. However, unlike other analyses, it is improper to use DLP to keep track of cumulative dose because a patient may undergo scans of different regions of the body, which have different sensitivities to radiation. Instead, effective dose, which is region-independent, becomes the best alternative.

In Fig. 13, we examined the cumulative dose of a patient with the highest number of studies, 27. An important thing to realize from this graph is just how quickly doses can add up. While each individual bar appears rather small compared to the total, it is easy to understand that this patient has received a large amount of radiation. Indeed, the threshold used for detecting outliers used to be 50 mSv, so while the patient would not have triggered anything by an individual study, his total dose most assuredly would have. In this particular case, the patient has a Scedosporium infection, and so tracking the disease outweighs the risk of the radiation exposure.

Fig. 13
figure 13

A single patient’s cumulative dose during the past year. The line is the cumulative dose, while the bars are the doses from a single study. Created in Microsoft Excel

Full size image

7 Discussion

In this paper, we sought to create a system that combines several standard techniques, such as DICOM retrieval and linear regression models, that is able to automatically retrieve and organize the data, and, from there, use that information to determine outliers based on dose data found in DICOM images. We then wanted to demonstrate how normal visualization techniques can be used to assess this information, based on how this system was able to group the data. To this end, we have successfully developed an automated method for retrieving the dose relevant information from PACS and incorporating it into an outlier detection system. This system is seamlessly integrated with a PACS, so it is able to function without any interruption of current clinical workflow. It is capable of working with any scanner that provides CTDIvol in the DICOM header, and can interact with any PACS system. A large amount of data is retrieved and archived, so that it can be analyzed more thoroughly. Data visualization of this information enables analysis that can detect systematic variation, such as a difference between the radiation exposure of different scanners or protocols. Such variation results in the appearance of distinct trends or clusters on these figures, which allows for an easier understanding of the data. Furthermore, RE3 is set up so that it can easily integrate the calculation of organ doses to get more personalized dose monitoring for when such an option becomes feasible.

Thus, we have presented a variety of visualization methods to mine this large amount of stored data. It represents initial forays into accessing this untapped knowledge, and demonstrates the utility of these techniques to identify and resolve problems that other methods might struggle with. However, it does have some limitations. The engine cannot accurately determine the outlier status if a patient’s body is not contained within the image field of view because SBV cannot be calculated correctly. Data mining for dose monitoring can be resource intensive and should involve multi-disciplinary teams, including radiologists (Siegelman and Gress 2013). Further, these teams need to have a manager to communicate and identify outliers, and ensure that doses are mitigated based on the findings. Some of the data mining is currently performed manually or visually. Automated processes based on artificial intelligence techniques would be necessary to handle larger data sets. Despite these considerations, data mining of dose data seems promising.