Skip to main content
SpringerLink
Log in

MIRF: A Multimodal Image Registration and Fusion Module Based on DT-CWT

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Image Fusion is a powerful and necessary tool to incorporate the relevant visual information provided by multiple sensors simultaneously. The quality of the results however, is bounded not only by the quality of the algorithm, but also by the outcome of the required image registration algorithm. Despite this dependency, images are always assumed to be pre-aligned. With 3rd Generation surveillance systems, centralized computations are shifted to distributed visual nodes low on computational and power resources. This article presents a combined approach that is able to register and fuse multimodal images, dubbed MIRF. Combining both algorithms into one image domain not only offers a reduction in complexity making it a better fit for a resource constrained embedded platform, but also improves the response time of the system. Two algorithms for area-based image registration and object-based image fusion are proposed. They are based on Dual-Tree Complex Wavelet Transform. Qualitative and quantitative experimental results show that the proposed registration approach achieves comparable accuracies to its counterparts, with lower-complexity. On the other hand, the developed fusion scheme exhibits higher accuracy and proves its immunity to minor errors in registration

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14

Similar content being viewed by others

References

  1. Zeng, J., Sayedelahl, A., Gilmore, T., & Chouikha, M. (2006). Review of image fusion algorithms for unconstrained outdoor scenes. In Proc. IEEE 8th Int. Conf. on Signal Processing, Vol. 2.

  2. Piella, G. (2002). A region-based multiresolution image fusion algorithm. In Proc. IEEE 5th Int. Conf. on Information Fusion, Vol. 2 (pp. 1557–1564).

  3. Li, Z., Jing, Z., Liu, G., Sun, S., & Leung, H. (2003). A region-based image fusion algorithm using multiresolution segmentation. In Proc. IEEE int. Conf. on Intelligent Transportation Systems, Vol. 1 (pp. 96–101).

  4. Cvejic, N., Lewis, J., Bull, D., & Canagarajah, N. (2006). Adaptive region-based multimodal image fusion using ICA bases. In Proc. IEEE 9th Int. conf. on information fusion (pp. 1–6).

  5. Zhang, Y., & Ge, L. (2007). Region-based image fusion using energy estimation. In Proc. IEEE 8th int. conf. on software engineering, artificial intelligence, networking, and parallel/distributed computing, Vol. 1 (pp. 729–734).

  6. Zitova, B., & Flusser, J. (2003). Image registration methods: a survey. Image and Vision Computing, 21(11), 977–1000.

    Article  Google Scholar 

  7. Ghantous, M., Ghosh, S., & Bayoumi, M. (2009). A multi-modal automatic image registration technique based on complex wavelets. In Proc. IEEE 16th International Conference on Image Processing (pp. 173–176)

  8. Ghantous, M., Ghosh, S., & Bayoumi, M. (2008). A gradient-based hybrid image fusion scheme using object extraction. In Proc. IEEE 15th International Conference on Image Processing (pp. 1300–1303).

  9. Mallat, S. G. (1989). A theory for multiresolution signal decomposition: the wavelet representation. IEEE Transaction Pattern Analysis and Machine Intellegence, 11, 674–693.

    Article  MATH  Google Scholar 

  10. Rockinger, O. (1997). Image sequence fusion using a shift invariant wavelet transform. IEEE Transaction on Image Processing, 3, 288–291.

    Google Scholar 

  11. Kingsbury, N. G. (2000). A dual-tree complex wavelet transform with improved orthogonality and symmetry properties. In Proc. IEEE Conf. on Image Processing (pp. 375–378).

  12. Heather, J. P., Smith, M. I., Sadler, J., & Hickman, D. (2010). Issues and challenges in the development of a commercialised image fusion system. In Proc. SPIE, Vol. 7701.

  13. Heather, J. P., & Smith, M. I. (2010). New adaptive algorithms for real-time registration and fusion of multimodal imagery. In Proc. SPIE Enhanced and synthetic vision, Vol. 76895.

  14. Fonseca, L., & Costa, M. (1997). Automatic registration of satellite images. In Proc. Brazilian Symposium on Computer Graphics and Image Processing (pp. 219–226).

  15. Le Moigne, J., Campbell, W., & Cromp, R. (2002). An automated parallel image registration technique based on the correlation of wavelet features. IEEE Transaction Geoscience and Remote Sensing, 40(8), 1849–1864.

    Article  Google Scholar 

  16. Sarvaiya, J., Patnaik, S., & Bombaywala, S. (2009). Image registration by template matching using normalized cross-correlation. In Proc. Intl. Conf. Advances in Computing, Control and telecommunication technologies, ACT (pp. 819–822).

  17. Viola, P., & Wells, W. M. (1995). Alignment by maximization of mutual information. In Proc. IEEE 5th International conference on Computer Vision (pp. 16–23).

  18. Wu, J., & Chung, A. (2004). Multimodal brain image registration based on wavelet transform using SAD and MI. In Proc. 2nd International Workshop on Medical Imaging and Augmented Reality (pp. 270–277).

  19. Fan, X., Rhody, H., & Saber, E. (2005). Automatic registration of multi-sensor airborne imagery. In Proc. IEEE Applied Imagery and Pattern Recognition workshop (pp. 80–86).

  20. Maes, F., Collignon, A., Vandermeulen, D., & Marchal, G. (1997). Multimodality image registration by maximization of mutual information. IEEE Transaction Medical Imaging, 16, 187–198.

    Article  Google Scholar 

  21. Press, W. H., Flannery, B. P., Teulolsky, S. A., & Vetterling, W. T. (1992). Numerical recipes in C, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, ch. 10, (pp. 412–219).

  22. Xu, R., & Chen, Y. (2007). Wavelet-Based multiresolution medical image registration strategy combining mutual information with spatial information. International Journal of Innovative Computing, Information and Control, Vol. 3, No.2.

  23. Malviya, A., & Bhirud, S. G. (2009). Wavelet based image registration using mutual information. In Proc. IEEE Emerging trends in electronic and photonic devices and systems, ELECTRO (pp. 241–244).

  24. Orchard, J. (2007). Efficient least squares multimodal registration with a globally exhaustive alignment search. IEEE Transaction Image Processing, 16, 2526–2534.

    Article  MathSciNet  Google Scholar 

  25. Woodward, A., Rowland, J., & Kell, D. (2004). Fast automatic registration of images using the phase of a complex transform: application to proteome gels. Analyst, 129(6), 542–552.

    Article  Google Scholar 

  26. Richards, J. A. (1984). Thematic mapping from multitemporal image data using the principal component transformation. Remote Sensing of Environment, 16, 35–46.

    Article  MathSciNet  Google Scholar 

  27. Metwalli, M., Nasr, A., Farag, O., & El-Rabaie, S. (2009). Image fusion based on principal component analysis and high pass filter. In Proc. IEEE computer Engineering and systems, ICCES (pp. 63–70).

  28. Burt, P. J., Hong, T. H., & Rosenfeld, A. (1981). Segmentation and estimation of image region properties through cooperative hierarchical computation. IEEE Transaction Systems, Man, and Cybernetics, 11, 802–809.

    Article  Google Scholar 

  29. Burt, P. J., & Adelson, E. H. (1983). The Laplacian pyramid as a compact image code. IEEE Transaction Communications, 31, 532–540.

    Article  Google Scholar 

  30. Mitianoudis, N., & Stathaki, T. (2007). Pixel-based and region-based image fusion schemes using ICA bases. Information Fusion, 8, 131–142.

    Article  Google Scholar 

  31. Mattes, D., Haynor, D. R., Vesselle, H., Lewellen, T. K., & Eubank, W. (2001). Non-rigid multimodality image registration. In Medical Imaging 2001: Image Processing (pp. 1609–1620).

  32. Klein, S., Starring, M., & Pluim, J. (2005). Comparison of gradient approximation techniques for optimization of mutual information in nonrigid registration. In Proc. SPIE Medical Imaging.

  33. Collins, R. T., Lipton, A. J., Kanade, T., Fujiyoshi, H., Duggins, D., Tsin, Y., Tolliver, D., Enomoto, N., Hasegawa, O., Burt, P., & Wixson, L. (2000). A system for surveillance and monitoring. Tech. report CMU-RI-TR-00-12, Robotics Institute, Carnegie Mellon University, May.

  34. Wang, Y., & Lohmann, B. (2000). Multisensor image fusion: concept, method and applications. Technical report, University of Bremen.

  35. Qu, G., Zhang, D., & Yan, P. (2002). Information measure for performance of image fusion. Electronics Letters, 38, 313–315.

    Article  Google Scholar 

  36. Xydeas, C. S., & Petrovic, V. (2000). Objective image fusion performance measure. Electronics Letters, 36, 308–309.

    Article  Google Scholar 

  37. Petrovic, V. S., & Xydeas, C. S. (2003). Sensor noise effects on signal-level image fusion performance. Information Fusion, 4, 167–183.

    Article  Google Scholar 

  38. Piella, G., & Heijmans, H. (2003). A new quality metric for image fusion. In Proc. IEEE International Conference on Image Processing (pp. 173–176).

  39. Wang, Z., & Bovik, A. C. (2002). A universal image quality index. IEEE Signal Processing Letters, 9, 81–84.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lebanese International University, Mouseitbeh, PO BOX 146404, Mazraa, Lebanon

    Milad Ghantous

  2. University of Louisiana at Lafayette, 104 university circle, Lafayette, LA, USA

    Magdy Bayoumi

Authors
  1. Milad Ghantous
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Magdy Bayoumi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Milad Ghantous.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ghantous, M., Bayoumi, M. MIRF: A Multimodal Image Registration and Fusion Module Based on DT-CWT. J Sign Process Syst 71, 41–55 (2013). https://doi.org/10.1007/s11265-012-0679-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-012-0679-1

Keywords

Search

Navigation