Abstract
In this paper, a novel objective quality metric is proposed for individual object segmentation in images. We analyze four types of segmentation errors, and verify experimentally that besides quantity, area and contour, the distortion of object content is another useful segmentation quality index. Our metric evaluates the similarity between ideal result and segmentation result by measuring these distortions. The metric has been tested on our subjectively-rated image segmentation database and demonstrated a good performance in matching subjective ratings.
The original version of this chapter was revised: The copyright line was incorrect. This has been corrected. The Erratum to this chapter is available at DOI: 10.1007/978-3-319-02895-8_64
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Powers, D.: Evaluation: From Precision, Recall and F-Factor to ROC, Informedne, Markedness & Correlation. Journal of Machine Learning Technologies 2(1), 37–63 (2011)
Ge, F., Wang, S., Liu, T.: New benchmark for image segmentation evaluation. Journal of Electronic Imaging 16(3), 33011 (2007)
Villegas, P., Marichal, X.: Perceptually-weighted evaluation criteria for segmentation masks in video sequences. IEEE Trans. Image Process 13(8), 1092–1103 (2004)
Erdem, C., Sankur, B.: Performance evaluation metrics for objectbased video segmentation. In: Proc. X Eur. Signal Process Conf., Tampere, Finland, vol. 2, pp. 917–920 (2000)
Strasters, K., Gebrands, J.: Three-dimensional image segmentation using a split, merge and group approach. Pattern Recognit. Lett. 12(5), 307–325 (1991)
McGuinness, K., O’Connor, N.: A comparative evaluation of interactive segmentation algorithms. Pattern Recognition 43(2), 434–444 (2010)
Gelasca, E.D.: Full-reference objective quality metrics for video watermarking, video segmentation and 3D model watermarking. In: Ph.D. dissertation, EPFL, Lausanne, Switzerland (2005)
Correia, P., Pereira, F.: Objective evaluation of video segmentation quality. IEEE Trans. Image Process 12(2), 186–200 (2003)
Zhang, Y.J.: A Survey on Evaluation Methods for Image Segmentation. Pattern Recognition 29(8), 1335–1346 (1996)
Li, S., Mak, L.C.-M., Ngan, K.N.: Visual Quality Evaluation for Images and Videos. In: Lin, W., Tao, D., Kacprzyk, J., Li, Z., Izquierdo, E., Wang, H. (eds.) Multimedia Analysis, Processing and Communications. SCI, vol. 346, pp. 497–544. Springer, Heidelberg (2011)
Achanta, R., Hemami, S., Estrada, F., Susstrunk, S., Frequency-tuned, M.S.: salient region detection. In: Proc. IEEE CVPR, Miami, USA, pp. 1597–1604 (2009)
Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting salient objects from images and videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 366–379. Springer, Heidelberg (2010)
Internal Telecommunication Union Radio communication Sector, C.: ITU-R Recommendation BT.500-13, Methodology for the Subjective Assessment of the Quality of Television Pictures (2012)
Video Quality Expert Group (VQEG) S.: Final Report From the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment I (2010)
Zadeh, L.: Fuzzy sets and systems. Information and Control 8(3), 338–353 (1965)
Li, S., Zhang, F., Ma, L., Ngan, K.N.: Image quality assessment by separately evaluating detail losses and additive impairments. IEEE Trans. Multimedia 13(5), 935–949 (2011)
Huynh-Thu, Q., Garcia, N.N., Speranza, F., Corriveau, P., Raake, A.: Study of rating scales for subjective quality assessment of high-definition video. IEEE Trans. Broadcasting 57(1), 1–14 (2011)
Gelasca, E.D., Karaman, M., Ebrahimi, T., Sikora, T.: A Framework for Evaluating Video Object Segmentation Algorithms. In: Proc. IEEE CVPR Workshop, New York, USA, pp. 198–198 (2006)
Internal Telecommunication Union Telecommunication Standardization Sector, C.: ITU-T Recommendation P.910, Subjective video quality assessment methods for multimedia applications (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shi, R., Ngan, K.N., Li, S. (2013). The Objective Evaluation of Image Object Segmentation Quality. In: Blanc-Talon, J., Kasinski, A., Philips, W., Popescu, D., Scheunders, P. (eds) Advanced Concepts for Intelligent Vision Systems. ACIVS 2013. Lecture Notes in Computer Science, vol 8192. Springer, Cham. https://doi.org/10.1007/978-3-319-02895-8_42
Download citation
DOI: https://doi.org/10.1007/978-3-319-02895-8_42
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-02894-1
Online ISBN: 978-3-319-02895-8
eBook Packages: Computer ScienceComputer Science (R0)