View synthesis for depth from motion 3D X-ray imaging
Introduction
The depth from motion or kinetic depth X-ray imaging (KDEX) technique is designed to meet a number of operational constraints, which are of critical importance for its intended application in airport security screening. It is envisaged that the technique would have no impact upon the speed of the conveyor belt or the standard functionality that is currently available on conventional two-dimensional (2D) X-ray machines. The stationary configuration of a single X-ray source and a number of linear or folded linear X-ray detector arrays provide the basis for a relatively simple and low maintenance system. The total number of detector arrays is contingent upon the synthesis of intermediary view X-ray images, which may make intermediate X-ray detector arrays redundant. The X-ray source requirements for the KDEX technique are identical to conventional 2D X-ray machines.
It is useful to compare and contrast the operational requirements of the proposed technique with computed tomography (CT) systems which have been increasingly deployed at airports since 9/11. CT utilises digital X-ray sensors to produce a stack of contiguous slice images (Romans, 1995), which can be combined to produce alternative volumetric views of the object under inspection. This technique requires very high signal/noise ratios and consequently very high X-ray source levels fully to penetrate objects from many different radial positions during the image acquisition process. The resultant high level of radiation requires substantial radiation screening. Additionally, CT systems require complex and bulky mechanical arrangements such as a slip ring to provide data signals (and power to the X-ray tube) to flow between the continuously rotating gantry and the stationary CT components, through the electrical, radio frequency or optical connections on the slip ring (Hsieh, 2003). With increasingly demanding scan speeds, the mechanical requirements on the gantry increase (as the centrifugal force increases with the square of the rotation speed). Note that CT components mounted on a gantry can weigh of the order of a hundred kilos or more. The purchase and running costs of CT scanners are estimated to be at least an order of magnitude greater than anticipated for the KDEX technology. Thus the proposed technique has the potential for greatly increased utility in comparison with 2D technology but at the fraction of the cost, bulk and radiation exposure associated with CT. It is not suggested that the KDEX could replace CT per se but rather that it has the potential to replace 2D technology with a cost effective three-dimensional (3D) imaging technology. It is beyond the scope of this paper to examine this assertion in any greater detail as CT and multiple view KDEX technology are based on different imaging principles.
Past work by the university team in collaboration with the Home Office Scientific Development Branch (HOSDB) UK has produced a novel binocular stereoscopic X-ray technique, (Evans, 2002, Evans and Robinson, 2000, Evans et al., 1996) to aid the detection and identification of objects in X-ray scans of luggage. Imaging technology based on this early work is now commercially available. More recently the university team in collaboration with the HOSDB has developed X-ray imaging techniques that combine binocular stereoscopic imagery with motion or kinetic depth effects. Movement or rotation of an object relative to the observer can produce a vivid depth in a 2D display. Linear “motion parallax” refers to the differential angular velocities of retinal images of points moving laterally with the same speed, but at different distances (from the eye in the case of the real world, and from the sensors in the case of the X-ray scanner). Therefore, this effect can be used to produce motion perspective. Motion perspective enables a viewer to extract depth information from continuous movement occurring in a visual display. Interestingly, the depth effect obtained from motion can exceed that produced by the binocular stereoscopic effect.
It is worth noting that many writers have described the essential isomorphism of depth due to disparity, and apparent (stroboscopic) motion (Ringach et al., 1996). Stereopsis normally depends upon the parallax difference in retinal images (which may be simultaneously present), while apparent motion depends upon a disparity in temporal sequence of spatially separated elements. Similarly, the motion parallax cue is dependent upon the same geometry as that which produces stereopsis. The difference being that the comparison of different viewpoints occurs sequentially in the case of motion parallax rather that synchronously in the case of binocular parallax (Howard and Rogers, 2002, Rock, 1975). When an object is rotated about an axis other than the line of sight, the relative motions of features upon the object can specify the 3D structure of the object to monocular vision. The geometric similarity between disparity, motion parallax, and rotation is practically identical for small rotations/displacements (Durgin et al., 1995). A special case of motion parallax termed the kinetic depth effect or KDE was first systematically investigated by Wallach and O’Connell (1953). This effect involves the recovery of 3D information from a sequence of 2D silhouettes. This effect involves the rotational motion of objects, rather than observers; a figure looks flat when it is stationary and appears to have depth once it moves. They concluded that kinetic depth effect requires: “… the shadows cast to display contours or lines which change their length and their direction simultaneously”. The resultant depth effect is compelling and the observer can work out the shapes of certain objects with remarkable accuracy from the shadows during a full or partial rotation. Kaufman (1974) suggested that motion perspective (which is a form of motion parallax) entails a semi-rotation of elements along an extended terrain, and is therefore a form of KDE. However, in the KDE the direction of depth is inherently ambiguous, since the direction of perceived depth may change spontaneously, even though there is no physical change in the stimulus. Binocular disparity tends to disambiguate the depth produced by motion parallax and KDE. It is interesting to note that no further perspective views are required to display a kinetic stereoscopic sequence (Evans and Hon, 2002). This is achieved by channelling a set of identical, but out of phase, perspective images to each of the observer’s eyes. The combination of binocular disparity and motion parallax allows the object to be viewed from different viewpoints, thus revealing attributes of the object that may not be evident in a simple 2D display.
To produce a smooth image rotation over sufficiently wide angles suitable for security screening applications does require a relatively large number of views. In this paper the number of views under consideration is 15. However, ongoing work is examining the possibility of extending the useful angular range of the technique by using up to 32 views although these studies are still in their early development. The implementation of a large number of folded array detectors presents several serious practical problems for the construction of X-ray collimators and configuration of dual-energy sensor modules. This problem is exacerbated by the small angular increments, of the order of 1°, required between each successive view and the physical bulk of the sensor arrays. These physical constraints currently preclude the development of a practical single pass imaging system. The aim of the work presented in this paper is to synthesise intermediary X-ray views to minimise the number of X-ray sensors and associated hardware required for the collection of the raw image data. If this can be achieved then the world’s first single pass KDEX scanner can be realised.
Section snippets
Image synthesis
Several approaches are available to produce intermediary image views. Simple techniques do not account for changes in the perspective, orientation and other properties that may change between images. Significant research has been conducted within computer science into the comparison of images and the generation of intermediary or alternative views from a sequence of images. Applications of such research include video compression and stereoscopic image synthesis. Correspondence problem
Algorithm
The algorithm applies eight correlation criteria to establish correspondences between pixels in the left and the right perspective X-ray images. Thus for every pixel in the left image a search along the epipolar line in the right image is conducted. The length of the search is bounded by the maximum possible parallax value. The epipolar line constraint indicates that there is nominally no y-position difference (or vertical disparity in the resultant display) exhibited in KDEX image sequences.
Experiment details
Two experimental X-ray machines were employed to produce the perspective images used in this research. The first machine utilises a ‘folded’ linear X-ray detector array (see Fig. 2, Fig. 3) to image typical luggage and is described in Section 4.1. The second machine employs an image intensifier to simulate a set of flat bed linear X-ray detector arrays and is described in Section 4.2. The resultant images from each machine are stored in a monochrome 512 × 512 pixels format with an 8 bit intensity
Experiments
The initial experiment was designed to establish the optimum combination of criteria for image synthesis. The second experiment applies the combination of criteria identified by the first experiment, with appropriately modified disparity constraints, to assess the algorithm performance as a function of the X-ray beam angle between successive views. This latter experiment is important with regard to establishing the optimum X-ray sensor geometry, especially for systems using up to 32 views which
Discussion of results
The images used for the indicative analysis presented in this paper were chosen carefully to include objects composed of different material classes arranged in spatially complex scenes. The inclusion of dense amorphous structure together with overlapping high frequency details exhibits the multi-layered translucency typical of images routinely encountered at security checkpoints. The images from both KDEX scanners, based on LXDAs and FLXDAs respectively, facilitate satisfactory image synthesis.
Conclusions and future work
The narrow angular separation of the order of 1° between successive views in the image sequences is dictated by the constraints concerning the perception of smooth motion, kinetic depth effect and binocular stereoscopic fusion. It has been demonstrated that intermediary images may be successfully synthesised by processing adjacent perspective images produced by X-ray beams separated by angular increments of 2°. This finding is potentially important as for example it renders the middle X-ray
Acknowledgements
The EPSRC Grant (EP/C520351/1; Dynamic 3D Imaging for Security Screening: Crime feasibility Study) funds this research in collaboration with HOSDB, Sandridge, and the US Dept. of Homeland Security (DHS): Transportation Security Administration. The authors would like to thank Richard Doney of the DfT, Professor Dick Lacey of the HOSDB and Dr. Josh Rubinstein of the US DHS for their support of this programme of research.
References (28)
- et al.
Dynamic stereoscopic X-ray imaging
Non-Destruct. Test. Eval. Int. (NDT&E)
(2002) - et al.
Design of a stereoscopic X-ray imaging system using a single X-ray source
Non-Destruct. Test. Eval. Int. (NDT&E)
(2000) - et al.
A new stereoscopic X-ray imaging technique using a single X-ray source: theoretical analysis
Non-Destruct. Test. Eval. Int. (NDT&E)
(1996) - et al.
Binocular eye movement caused by the perception of three-dimensional structure from motion
Vision Res.
(1996) - et al.
Feature-based image metamorphosis
SIGGRAPH
(1992) - et al.
Wire transfer function analysis for castellated dual-energy X-ray detectors
Appl. Opt. Opt. Soc. Amer. (OSA)
(2004) - et al.
View interpolation for image synthesis
SIGGRAPH
(1993) - et al.
Comparing depth from motion with depth from binocular disparity
J. Exp. Psychol.: Hum. Percept. Perform.
(1995) Stereoscopic imaging using folded linear dual-energy X-ray detectors
Inst. Phys. (IOP) J. Meas. Sc. Tech.
(2002)Kinetic depth effect X-ray imaging for security screening
IEE Vis. Inf. Eng. (VIE),
(2003)
Depth from motion 3D X-ray imaging for security screening
IEE International symposium on imaging for crime detection and prevention (ICDP 2005)
Multiple View Geometry in Computer Vision
Cited by (12)
Method for filling and sharpening false colour layers of dual energy X-ray images
2015, IFAC-PapersOnLinePosition determination of scatter signatures - A novel sensor geometry
2010, TalantaCitation Excerpt :As a next step we intend to combine this approach with the novel tomographic approach afforded by kinetic depth effect X-ray imaging [12].
The separation of X-ray diffraction patterns for threat detection
2010, Applied Radiation and IsotopesSporadic absorption tomography using a conical shell X-ray beam
2017, Optics ExpressSharpening filter for false color imaging of dual-energy X-ray scans
2017, Signal, Image and Video ProcessingMethod for Filling and Sharpening False Colour Layers of Dual Energy X-ray Images
2016, International Journal of Electronics and Telecommunications