Skip to main content
SpringerLink
Log in

Real-time emotion retrieval scheme in video with image sequence features

  • Special Issue Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

In the current Internet environment, a lot of multimedia information is navigated on the on-line computer systems. Among the multimedia information, video sequence has the most valuable and meaningful influence on human emotions. Therefore, one human’s emotions to see and feel the same video can be different from that of others depending on the person’s mental state. In this research, we propose a new real-time emotion retrieval scheme in video with image sequence features. The features of image sequence consist of color information, key frame extraction, video sound, and optical flow. Each video feature is combined with the weight for the emotion retrieval. The experimental results show the new approach of real-time emotion retrieval in video with the better results compared to the previous studies. The proposed scheme will be applied to the many multimedia fields: movie, computer game, video conference, and so on.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Wang, S.F.: Research on emotion information processing and its application in image retrieval. Doctoral dissertation, University of Science and Technology of China (2002)

  2. Chang, J.-K., Ryoo, S.-T.: Image-based emotion retrieval approach with multi-machine learning schemes, future information technology, application, and science. LNEE 179, 177–185 (2012)

  3. Smeulders, A.W.M., et al.: Content-based image retrieval: The end of the early years. IEEE Trans. PAMI 22(12), 1349–1380 (2000)

    Google Scholar 

  4. Mojsilovic, A., Gomes, J., Rogowitz, B.: Semantic-friendly indexing and querying of images based on the extraction of the objective semantic cues. Intl. J. Comput. Vis. 56, 79–107 (2004)

    Google Scholar 

  5. Colombo, C., Bimbo, A.D., Pala, P.: Semantics in visual information retrieval. IEEE Multimed. 6(3), 38–53 (1999)

    Article  Google Scholar 

  6. Cho, S.B., Lee, J.Y.: A human-oriented image retrieval system using interactive genetic algorithm. IEEE Trans. SMC Part A 32(3), 452–458 (2002)

    MathSciNet  Google Scholar 

  7. Lee, J. et al.: Emotional evaluation of color patterns based on rough sets. Proc. 3(h ICNC 1), pp. 140–144 (2007)

  8. Lee J., et al.: Emotional evaluation of color patterns based on rough sets. Proc. 3th ICNC, pp. 1140–144 (2007)

  9. Li, J., Wang, J.Z.: Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. PAMI 25(9), 1075–1088 (2003)

    Article  Google Scholar 

  10. Olkiewicz K.A., Markowska-Kaczmar U.: Emotion-based image retrieval an artificial neural network approach. Proc. International Multiconference on CSIT, pp. 89–96 (2010)

  11. Lee, J., Zhang, L., Park, E.: An emotion-based image retrieval system by using fuzzy integral with relevance feedback. Proc. IEEK 31, 683–688 (2008)

    Google Scholar 

  12. Yoo, H.W.: Visual-based emotional descriptor and feedback mechanism for image retrieval. J. Inf. Sci. Eng. 22, 1205–1227 (2006)

    Google Scholar 

  13. Salway, A., Graham, M.: Extracting Information about emotions in films. In: Proc. of the 11th ACM Int. Conf. on Multimedia, pp. 299–302 (2003)

  14. Knautz, K. et al.: Finding emotional-laden resources on the World Wide Web. J. Inf. 2, 217–246 (2011)

    Google Scholar 

  15. Singh, K.V., Tripathi, A.K.: Emotion based contextual semantic relevance feedback in multimedia information retrieval. Int. J. Comput. Appl. 55(15), 38–49 (2012)

    Google Scholar 

  16. Yoo, H.W., Cho, S.B.: Video scene retrieval with interactive genetic algorithm. Multimed. Tools Appl. 34, 317–336 (2007)

    Article  Google Scholar 

  17. Canini, L., Benini, S., Leonardi, R.: Affective analysis on patterns of shot types in movies. IN: 7th Int. Sym. Of Image and Signal processing and Analysis, pp. 253–258 (2011)

  18. Russell, J.: Pancultural aspects of the human conceptual organization of emotions. J. Pers. Soc. Psychol. 45, 1281–1288 (1983)

    Article  Google Scholar 

  19. Chang, J.K., Ryoo, S.T.: Implementation of key frame extraction system in video sequence. J. Korean Soc. Comput. Game 26(1), 79–85 (2013)

    Google Scholar 

  20. Won, H.K.: A study on setting adaptive thresholds and weighting factors for real-time shot change detection, Department of Computer Engineering, The Graduate School, Pukyoung National University (2009)

  21. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–204 (1981)

    Article  Google Scholar 

  22. Chang, J.K., Huntsberger, T.L.: Dynamic motion analysis using wavelet flow surface images, Pattern Recognition Letters, 20 (1999)

  23. Jeon, Y.W., Cho, A.: Effect of 1/f fluctuation sound on comfort sensibility. J. Ergon. Soc. Korea 25(4), 9–22 (2006)

    Article  Google Scholar 

  24. Ishikawa,Y.; Zhao, H. A.: A study on speech quality improvement system by 1/f fluctuation theory. In: Proc. Of IEICE, pp. 4–22 (2002)

Download references

Acknowledgments

This research was supported by Hanshin University Research Grant.

Author information

Authors and Affiliations

  1. School of Computer Engineering, Hanshin University, Osan, South Korea

    Jae-Khun Chang & Seung-Taek Ryoo

Authors
  1. Jae-Khun Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Seung-Taek Ryoo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Seung-Taek Ryoo.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chang, JK., Ryoo, ST. Real-time emotion retrieval scheme in video with image sequence features. J Real-Time Image Proc 9, 541–547 (2014). https://doi.org/10.1007/s11554-013-0366-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-013-0366-x

Keywords

Search

Navigation