Skip to main content
SpringerLink
Log in
Book cover

International Conference on Multimedia Modeling

MMM 2018: MultiMedia Modeling pp 267–278Cite as

Image Aesthetic Distribution Prediction with Fully Convolutional Network

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10704))

Abstract

Image aesthetics assessment emerges as a hot topic in recent years for its potential in numerous applications. In this paper, we propose to quantify the image aesthetics by a distribution over multiple quality levels. The distribution representation can effectively characterize the disagreement among users’ aesthetic perceptions regarding the same image. We realize an end-to-end framework of aesthetic distribution prediction with fully convolutional network, which accepts input images of arbitrary sizes. In this way, we circumvent the requirement of fixed-sized inputs from prevalent convolutional neural network, and thereby avoid the risk of impairing the intrinsic aesthetic appeal of images. Experiments on two benchmark datasets well verified the effectiveness of our approach in both scenarios of aesthetic distribution prediction and aesthetic label prediction.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.dpchallenge.com/.

References

  1. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., Torr, P.H.S.: Fully-convolutional siamese networks for object tracking. In: Hua, G., Jégou, H. (eds.) ECCV 2016. LNCS, vol. 9914, pp. 850–865. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-48881-3_56

    Chapter  Google Scholar 

  2. Chollet, F., et al.: Keras. https://github.com/fchollet/keras (2015)

  3. Cui, C., Fang, H., Deng, X., Nie, X., Dai, H., Yin, Y.: Distribution-oriented aesthetics assessment for image search. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1013–1016 (2017)

    Google Scholar 

  4. Cui, C., Shen, J., Ma, J., Lian, T.: Social tag relevance learning via ranking-oriented neighbor voting. Multimedia Tools Appl. 76(6), 8831–8857 (2017)

    Article  Google Scholar 

  5. Datta, R., Joshi, D., Li, J., Wang, J.Z.: Studying aesthetics in photographic images using a computational approach. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3953, pp. 288–301. Springer, Heidelberg (2006). https://doi.org/10.1007/11744078_23

    Chapter  Google Scholar 

  6. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  7. Deng, Y., Loy, C.C., Tang, X.: Image aesthetic assessment: an experimental survey. IEEE Sig. Process. Mag. 34(4), 80–106 (2017)

    Article  Google Scholar 

  8. Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1657–1664 (2011)

    Google Scholar 

  9. Dong, Z., Shen, X., Li, H., Tian, X.: Photo Quality assessment with DCNN that understands image well. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8936, pp. 524–535. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14442-9_57

    Google Scholar 

  10. Gao, B.B., Xing, C., Xie, C.W., Wu, J., Geng, X.: Deep label distribution learning with label ambiguity. IEEE Trans. Image Process. 26(6), 2825–2838 (2017)

    Article  MathSciNet  Google Scholar 

  11. Geng, B., Yang, L., Xu, C., Hua, X.S., Li, S.: The role of attractiveness in web image search. In: Proceedings of the 19th ACM International Conference on Multimedia, pp. 63–72 (2011)

    Google Scholar 

  12. Geng, X.: Label distribution learning. IEEE Trans. Knowl. Data Eng. 28(7), 1734–1748 (2016)

    Article  Google Scholar 

  13. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)

    Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Spatial pyramid pooling in deep convolutional networks for visual recognition. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8691, pp. 346–361. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10578-9_23

    Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  16. Kao, Y., He, R., Huang, K.: Deep aesthetic quality assessment with semantic information. IEEE Trans. Image Process. 26, 1482–1495 (2017)

    Article  MathSciNet  Google Scholar 

  17. Ke, Y., Tang, X., Jing, F.: The design of high-level features for photo quality assessment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 419–426 (2006)

    Google Scholar 

  18. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)

    Google Scholar 

  19. Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)

  20. Lo, K.Y., Liu, K.H., Chen, C.S.: Assessment of photo aesthetics with efficiency. In: Proceedings of the 21st International Conference on Pattern Recognition, pp. 2186–2189 (2012)

    Google Scholar 

  21. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  22. Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rating image aesthetics using deep learning. IEEE Trans. Multimedia 17(11), 2021–2034 (2015)

    Article  Google Scholar 

  23. Luo, Y., Tang, X.: Photo and video quality evaluation: focusing on the subject. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 386–399. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88690-7_29

    Chapter  Google Scholar 

  24. Marchesotti, L., Murray, N., Perronnin, F.: Discovering beautiful attributes for aesthetic image analysis. Int. J. Comput. Vis. 113(3), 246–266 (2015)

    Article  Google Scholar 

  25. Marchesotti, L., Perronnin, F., Larlus, D., Csurka, G.: Assessing the aesthetic quality of photographs using generic image descriptors. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1784–1791 (2011)

    Google Scholar 

  26. Murray, N., Marchesotti, L., Perronnin, F.: AVA: a large-scale database for aesthetic visual analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2408–2415 (2012)

    Google Scholar 

  27. Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 91–99 (2015)

    Google Scholar 

  28. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  29. Springenberg, J.T., Dosovitskiy, A., Brox, T., Riedmiller, M.: Striving for simplicity: the all convolutional net. arXiv preprint arXiv:1412.6806 (2014)

  30. Tang, X., Luo, W., Wang, X.: Content-based photo quality assessment. IEEE Trans. Multimedia 15(8), 1930–1943 (2013)

    Article  Google Scholar 

  31. Tian, X., Dong, Z., Yang, K., Mei, T.: Query-dependent aesthetic model with deep learning for photo quality assessment. IEEE Trans. Multimedia 17(11), 2035–2048 (2015)

    Article  Google Scholar 

  32. Wang, L., Qiao, Y., Tang, X., Van Gool, L.: Actionness estimation using hybrid fully convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2708–2717 (2016)

    Google Scholar 

  33. Wang, L., Wang, L., Lu, H., Zhang, P., Ruan, X.: Saliency detection with recurrent fully convolutional networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 825–841. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_50

    Chapter  Google Scholar 

  34. Wu, O., Hu, W., Gao, J.: Learning to predict the perceived visual quality of photos. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 225–232 (2011)

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (61573219, 61671274, 61701281), NSFC Joint Fund with Guangdong under Key Project (U1201258), China Postdoctoral Science Foundation (2016M592190), Shandong Provincial Natural Science Foundation (ZR2017QF009), and the Fostering Project of Dominant Discipline and Talent Team of Shandong Province Higher Education Institutions.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Shandong University, Jinan, China

    Huidi Fang, Xiang Deng & Yilong Yin

  2. School of Computer Science and Technology, Shandong University of Finance and Economics, Jinan, China

    Chaoran Cui, Xiushan Nie & Muwei Jian

Authors
  1. Huidi Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaoran Cui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiang Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiushan Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Muwei Jian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yilong Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Chaoran Cui or Yilong Yin .

Editor information

Editors and Affiliations

  1. Alpen-Adria-Universität Klagenfurt, Klagenfurt, Austria

    Klaus Schoeffmann

  2. Chulalongkorn University, Bangkok, Thailand

    Thanarat H. Chalidabhongse

  3. City University of Hong Kong, Hong Kong, China

    Chong Wah Ngo

  4. Chulalongkorn University, Bangkok, Thailand

    Supavadee Aramvith

  5. Dublin City University, Dublin, Ireland

    Noel E. O’Connor

  6. Gwangju Institute of Science and Technology, Gwangju, Korea (Republic of)

    Yo-Sung Ho

  7. Tampere University of Technology, Tampere, Finland

    Moncef Gabbouj

  8. Rutgers University, Piscataway, New Jersey, USA

    Ahmed Elgammal

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Fang, H., Cui, C., Deng, X., Nie, X., Jian, M., Yin, Y. (2018). Image Aesthetic Distribution Prediction with Fully Convolutional Network. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-73603-7_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-73602-0

  • Online ISBN: 978-3-319-73603-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation