Abstract
Salient object detection in wavelet domain has recently begun to attract researchers’ effort due to its desired ability to provide multi-scale analysis of an image simultaneously in both frequency and spatial domains. The proposed algorithm exploits the inherent multi-scale structure of the double-density dual-tree complex-oriented wavelet transform (DDDTCWT) to decompose each input image into four approximate sub-band images and 32 high-pass detailed sub-band images at each scale. These 32 detailed high-pass sub-bands at each scale are adequate to represent singularities of any geometric object with high precision and to mimic zooming-in and zooming-out process of human vision system. In the proposed model, we first compute a rough segmented saliency map (RSSM) by fusing multi-scale edge-to-texture features generated from DDDTCWT with segmentation results obtained from bipartite graph partitioning-based segmentation approach. Then, each pixel in RSSM is categorized into either background region or salient region based on a threshold. Finally, the pixels of the two regions are considered as samples to be drawn from a multivariate kernel function whose parameters are estimated using expectation maximization algorithm, to generate a saliency map. The performance of the proposed model is evaluated in terms of precision, recall, F-measure, area under the ROC curve and computation time using six publicly available image datasets. Extensive experimental results on six benchmark datasets demonstrate that the proposed model outperformed the existing 29 state-of-the-art methods in terms of F-measure on all five datasets, recall on four datasets and area under ROC curve on two datasets. In terms of mean recall value, mean F-measure value and mean AUC value on all six datasets, the proposed method outperforms all state-of-the-art methods. The proposed method also takes comparatively less computation time in comparison with many existing spatial domain methods.
Similar content being viewed by others
Notes
E-mail at “rinki.arya89@gmail.com” or “navjot.singh.09@gmail.com”.
References
Achanta R (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282
Achanta R, Susstrunk S (2010) Saliency detection using maximum symmetric surround. In: Proceedings of 17th IEEE international conference on image processing (ICIP), pp 2653–2656
Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Proceedings of IEEE conference on Computer vision and pattern recognition, pp 1597–1604
Alpert S, Galun M, Brandt A, Basri R (2012) Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell 34(2):315–327
Arya R, Singh N, Agrawal R (2015) A novel hybrid approach for salient object detection using local and global saliency in frequency domain. Multimed Tools Appl 75(14):8267–8287
Arya R, Singh N, Agrawal R (2017) A novel combination of second-order statistical features and segmentation using multilayer superpixels for salient object detection. Appl Intell 46(2):254–271
Bian P, Zhang L (2008) Biological plausibility of spectral domain approach for spatiotemporal visual saliency. In: Proceedings of the international conference on neural information processing, pp 251–258
Bian P, Zhang L (2010a) Piecewise frequency domain visual saliency detection. In: Proceedings of IEEE third international conference on information and computing (ICIC), pp 269–272
Bian P, Zhang L (2010) Visual saliency:a biologically plausible contourlet-like frequency domain approach. Cogn Neurodyn 4(3):189–198
Borji A, Itti L (2013) State-of-the-art in visual attention modeling. IEEE Trans Pattern Anal Mach Intell 35(1):185–207
Borji A, Sihite DN, Itti L (2013) Quantitative analysis of human-model agreement in visual saliency modeling:a comparative study. IEEE Trans Image Process 22(1):55–69
Borji A, Cheng M-M, Jiang H, Li J (2014) Salient object detection: a survey. In: arXiv preprint arXiv:14115878
Borji A, Cheng M-M, Jiang H, Li J (2015) Salient object detection: a benchmark. IEEE Trans Image Process 24(12):5706–5722
Boykov YY, Jolly M-P (2001) Interactive graph cuts for optimal boundary and region segmentation of objects in ND images. In: Proceedings of the eighth IEEE international conference on computer vision, pp 105–112
Bruce N, Tsotsos J (2006) Saliency based on information maximization. In: Advances in neural information processing systems, pp 155–162
Castleman Kenneth R (1996) Digital image processing. Prentice Hall Press, Upper Saddle River
Chen L-Q, Xie X, Fan X, Ma W-Y, Zhang H-J, Zhou H-Q (2003) A visual attention model for adapting images on small displays. Multimed Syst 9(4):353–364
Cheng M-M, Warrell J, Lin W-Y, Zheng S, Vineet V, Crook N (2013) Efficient salient region detection with soft image abstraction. In: Proceedings of IEEE international conference on computer vision, pp 1529–1536
Cheng M, Mitra NJ, Huang X, Torr PH, Hu S (2015) Global contrast based salient region detection. IEEE Trans Pattern Anal Mach Intell 37(3):569–582
Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619
Daubechies I (1988) Orthonormal bases of compactly supported wavelets. Commun Pure Appl Math 41(7):909–996
Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7(Jan):1–30
Derrac J, García S, Molina D, Herrera F (2011) A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm Evolut Comput 1(1):3–18
Fang Y, Lin W, Lee B-S, Lau C-T, Chen Z, Lin C-W (2012) Bottom-up saliency detection model based on human visual sensitivity and amplitude spectrum. IEEE Trans Multimed 14(1):187–198
Fang Y, Chen Z, Lin W, Lin C-W (2012) Saliency detection in the compressed domain for adaptive image retargeting. IEEE Trans Image Process 21(9):3888–3901
Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181
Fu K, Gong C, Yang J, Zhou Y, Gu IY-H (2013) Superpixel based color contrast and color distribution driven salient object detection. Sig Process Image Commun 28(10):1448–1463
Fu K, Gong C, Gu IY-H, Yang J (2015) Normalized cut-based saliency detection by adaptive multi-level region merging. IEEE Trans Image Process 24(12):5671–5683
Gasparini F, Corchs S, Schettini R (2007) Low-quality image enhancement using visual attention. Opt Eng 46(4):040502
Goferman S, Zelnik-Manor L, Tal A (2012) Context-aware saliency detection. IEEE Trans Pattern Anal Mach Intell 34(10):1915–1926
Gopalakrishnan V, Hu Y, Rajan D (2010) Random walks on graphs for salient object detection in images. IEEE Trans Image Process 19(12):3232–3242
Guo C, Zhang L (2010) A novel multiresolution spatiotemporal saliency detection model and its applications in image and video compression. IEEE Trans Image Process 19(1):185–198
Guo C, Ma Q, Zhang L (2008) Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8
Hadizadeh H, Bajic IV (2014) Saliency-aware video compression. IEEE Trans Image Process 23(1):19–33
Harel J, Koch C, Perona P (2006) Graph-based visual saliency. In: Proceedings of the advances in neural information processing systems, pp 545–552
Hou X, Zhang L (2007) Saliency detection:A spectral residual approach. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8
Hou X, Harel J, Koch C (2012) Image signature: highlighting sparse salient regions. IEEE Trans Pattern Anal Mach Intell 34(1):194–201
Huang K, Tao D, Yuan Y, Li X, Tan T (2011) Biologically inspired features for scene classification in video surveillance. IEEE Trans Syst Man Cybern B Cybern 41(1):307–313
Huang X, Su Y, Liu Y (2016) Iteratively parsing contour fragments for object detection. Neurocomputing 175:585–598
Huo L, Jiao L, Wang S, Yang S (2016) Object-level saliency detection with color attributes. Pattern Recogn 49:162–173
Imamoglu N, Lin W, Fang Y (2013) A saliency detection model using low-level features based on wavelet transform. IEEE Trans Multimed 15(1):96–105
Iman RL, Davenport JM (1980) Approximations of the critical region of the fbietkan statistic. Commun Stat Theory Methods 9(6):571–595
Itti L (2000) Models of bottom-up and top-down visual attention. In: Doctoral dissertation California Institute of Technology
Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Pattern Anal Mach Intell 20(11):1254–1259
Jian M, Dong J, Ma J (2011) Image retrieval using wavelet-based salient regions. Imaging Sci J 59(4):219–231
Jian M, Lam K-M, Dong J, Shen L (2015) Visual-patch-attention-aware saliency detection. IEEE Trans Cybern 45(8):1575–1586
Jiang H, Wang J, Yuan Z, Wu Y, Zheng N, Li S (2013) Salient object detection: a discriminative regional feature integration approach. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 2083–2090
Judd T, Durand F, Torralba A (2012) A benchmark of computational models of saliency to predict human fixations. In: MIT technical report technical report
Kannan R, Ghinea G, Swaminathan S (2015) Salient region detection using patch level and region level image abstractions. IEEE Signal Process Lett 22(6):686–690
Karssemeijer N, te Brake GM (1996) Detection of stellate distortions in mammograms. IEEE Trans Med Imaging 15(5):611–619
Kim J, Han D, Tai Y-W, Kim J (2016) Salient region detection via high-dimensional color transform and local spatial support. IEEE Trans Image Process 25(1):9–23
Kingsbury N (1999) Image processing with complex wavelets. Philos Trans R Soc Lond A Math Phys Eng Sci 357(1760):2543–2560
Ko BC, Nam J-Y (2006) Object-of-interest image segmentation based on human attention and semantic region clustering. JOSA A 23(10):2462–2470
Koch C, Ullman S (1987) Shifts in selective visual attention: towards the underlying neural circuitry. In: Proceedings of the matters of intelligence, pp 115–141
Kumar K (2017) An efficient SOM technique for event summarization in multi-view surveillance videos. In: Proceedings of 5th international conference on advanced computing networking and informatics (ICACNI-17), pp 1–6
Kumar K, Shrimankar DD, Singh N (2016) Equal partition based clustering approach for event summarization in videos. In: Proceedings of IEEE conference on signal-image technology and internet-based systems (SITIS), pp 119–126
Kumar K, Shrimankar DD, Singh N (2017) Eratosthenes sieve based key-frame extraction technique for event summarization in videos. Multimed Tools Appl 77(6):7383–7404
Li Z, Itti L (2011) Saliency and gist features for target detection in satellite images. IEEE Trans Image Process 20(7):2017–2029
Li Z, Wu X-M, Chang S-F (2012) Segmentation using superpixels: a bipartite graph partitioning approach. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 789–796
Li J, Levine MD, An X, Xu X, He H (2013) Visual saliency based on scale-space analysis in the frequency domain. IEEE Trans Pattern Anal Mach Intell 35(4):996–1010
Li J, Duan L-Y, Chen X, Huang T, Tian Y (2015) Finding the secret of image saliency in the frequency domain. IEEE Trans Pattern Anal Mach Intell 37(12):2428–2440
Liang J, Zhou J, Tong L, Bai X, Wang B (2018) Material based salient object detection from hyperspectral images. Pattern Recogn 76:476–490
Liu Z (2014) Saliency tree: a novel saliency detection framework. IEEE Trans Image Process 23(5):1937–1952
Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X et al (2007) Learning to detect a salient object. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 1–8
Liu T, Yuan Z, Sun J, Wang J, Zheng N, Tang X et al (2011) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367
Ma Y-F, Zhang H-J (2003) Contrast-based image attention analysis by using fuzzy growing. In: Proceedings of ACM international conference on Multimedia, pp 374–381
Marchesotti L, Cifarelli C, Csurka G (2009) A framework for visual saliency detection with applications to image thumbnailing. In: Proceedings of 12th IEEE international conference on computer vision, pp 2232–2239
Murray N, Vanrell M, Otazu X, Parraga CA (2011) Saliency estimation using a non-parametric low-level vision model. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 433–440
Naqvi SS, Browne WN, Hollitt C (2016) Salient object detection via spectral matting. Pattern Recogn 51:209–224
Navalpakkam V, Itti L (2006) An integrated model of top-down and bottom-up attention for optimizing detection speed. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 2049–2056
Ninassi A, Meur OL, Callet PL, Barbba D (2007) Does where you gaze on an image affect your perception of quality? Applying visual attention to image quality metric. In: Proceedings of IEEE international conference on image processing, pp II-169
Park J, Lee J-Y, Tai Y-W, Kweon IS (2012) Modeling photo composition and its application to photo re-arrangement. In: Proceedings of IEEE international conference on image processing (ICIP), pp 2741–2744
Perazzi F, Krahenbuhl P, Pritch Y, Hornung A (2012) Saliency filters: contrast based filtering for salient region detection. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 733–740
Rother C, Bordeaux L, Hamadi Y, Blake A (2006) Autocollage. ACM Trans Graph (TOG) 25(3):847–852
Rutishauser U (2004) Is bottom-up attention useful for object recognition? In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp II-37
Santella A, Agrawala M, DeCarlo D, Salesin D, Cohen M (2006) Gaze-based interaction for semi-automatic photo cropping. In: Proceedings of SIGCHI conference on human factors in computing systems, pp 771–780
Selesnick IW (2001) The double density DWT. In: Proceedings of wavelets in signal and image analysis, pp 39–66
Selesnick IW (2004) The double-density dual-tree DWT. IEEE Trans Signal Process 52(5):1304–1314
Selesnick IW, Baraniuk RG, Kingsbury NG (2005) The dual-tree complex wavelet transform. IEEE Signal Process Mag 22(6):123–151
Shen X, Wu Y (2012) A unified approach to salient object detection via low rank matrix recovery. In: Proceedings of IEEE conference on computer vision and pattern recognition (CVPR), pp 853–860
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Mach Intell 22(8):888–905
Singh N, Arya R, Agrawal R (2014) A novel approach to combine features for salient object detection using constrained particle swarm optimization. Pattern Recogn 47(4):1731–1739
Singh N, Arya R, Agrawal R (2016) A convex hull approach in conjunction with Gaussian mixture model for salient object detection. Digit Signal Process 55:22–31
Singh N, Arya R, Agrawal R (2016b) A novel position prior using fusion of rule of thirds and image center for salient object detection. Multimed Tools Appl 76:1–18
Singh N, Arya R, Agrawal RK (2017) Performance enhancement of salient object detection using superpixel based Gaussian mixture model. Multimed Tools Appl 77:1–19
Sun J, Lu H, Liu X (2015) Saliency region detection based on Markov absorption probabilities. IEEE Trans Image Process 24(5):1639–1649
Tian Q, Sebe N, Lew MS, Loupias E, Huang TS (2001) Content-based image retrieval using wavelet-based salient points. In: Photonics west 2001-electronic imaging, pp 425–436
Wang Y-S, Tai C-L, Sorkine O, Lee T-Y (2008) Optimized scale-and-stretch for image resizing. ACM Trans Graph (TOG) 27(5):118
Wang K, Lin L, Lu J, Li C, Shi K (2015) PISA:pixelwise image saliency by aggregating complementary appearance contrast measures with edge-preserving coherence. IEEE Trans Image Process 24(10):3019–3033
Xie Y, Lu H, Yang M-H (2013) Bayesian saliency via low and mid level cues. IEEE Trans Image Process 22(5):1689–1698
Yang C, Zhang L, Lu H, Ruan X, Yang M-H (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 3166–3173
Yu Y, Wang B, Zhang L (2009) Pulse discrete cosine transform for saliency-based visual attention. In: Proceedings of IEEE 8th international conference on development and learning, pp 1–6
Yu J-G, Xia G-S, Samal A, Tian J (2016) Globally consistent correspondence of multiple feature sets using proximal Gauss-Seidel relaxation. Pattern Recogn 51:255–267
Zhang L, Tong MH, Marks TK, Shan H, Cottrell GW (2008) SUN: a Bayesian framework for saliency using natural statistics. J Vis 8(7):32
Zhu Z, Wahid K, Babyn P, Yang R (2013) Compressed sensing-based MRI reconstruction using complex double-density dual-tree DWT. Int J Biomed Imaging 2013:907501. https://doi.org/10.1155/2013/907501
Zhu L, Klein DA, Frintrop S, Cao Z, Cremers AB (2014) A multisize superpixel approach for salient object detection based on multivariate normal distribution estimation. IEEE Trans Image Process 23(12):5094–5107
Acknowledgements
The authors express their gratitude to the University Grant Commission (UGC), India, and DST-Purse, India, for the obtained financial support in performing this research work.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Arya, R., Agrawal, R.K. & Singh, N. A novel approach for salient object detection using double-density dual-tree complex wavelet transform in conjunction with superpixel segmentation. Knowl Inf Syst 60, 327–361 (2019). https://doi.org/10.1007/s10115-018-1243-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10115-018-1243-5
Keywords
- Salient object detection
- Visual saliency
- 2D double-density discrete wavelet transform (DDDWT)
- 2D dual-tree complex wavelet transform (DTCWT)
- 2D dual-tree real wavelet transform (DTRWT)
- Double-density dual-tree complex wavelet transform (DDDTCWT)
- Inverse double-density dual-tree complex wavelet transform (IDDDTCWT)
- Gaussian mixture model (GMM)
- Expectation maximization (EM)
- Saliency map