Skip to main content
SpringerLink
Log in

Time and space for segmenting personal photo sets

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

A personal collection of photos shows large variability in the depicted items, making difficult a fully automated solution to cope with sensory and semantic gaps. Emotions and non-visual contextual information can be very important to address those problems. Manual annotations are key, but their time-consuming nature alienate users from doing them. One solution is to lower the annotation effort, building solutions on top of algorithms that prepare a context separation, making possible the reuse of annotations. In this paper we present a segmentation algorithm that uses spatio-temporal information to segment personal photo collections. The algorithm is assessed in a user study, using the participants own photos. The results show users make none or few changes to the proposed segmentations, indicating an acceptance of the algorithm outcome.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23

Similar content being viewed by others

Article 28 December 2016

Davi Oliveira Serrano de Andrade, Luis Fernando Maia, … Cláudio de Souza Baptista

Notes

  1. http://www.flickr.com/.

  2. Considering a person on foot.

  3. Photos with spatial coordinates neither null nor outliers.

  4. The effect size values are for a student’s t-distribution.

  5. The values for f t and f g were settled after testing with 39 personal collections of photos, publicly available at Picasa Web Albuns. Those collections are different from the ones provided by the participants in the study.

  6. No. of segments greater than the median.

References

  1. Allen J (1983) Maintaining knowledge about temporal intervals. Commun ACM 26(11):832–843

    Article  MATH  Google Scholar 

  2. Breunig M M, Kriegel H P, Ng R T, Sander J (2000) Lof: identifying density-based local outliers. SIGMOD Rec 29(2):93–104. doi:10.1145/335191.335388

    Article  Google Scholar 

  3. Bruneau P, Pigeau A, Gelgon M, Picarougne F (2010) Geo-temporal structuring of a personal image database with two-level variational-bayes mixture estimation. In: Detyniecki M, Leiner U, Nrnberger A (eds) adaptive multimedia retrieval. Identifying, summarizing, and recommending image and music, lecture notes in computer science, vol 5811. Springer, Berlin Heidelberg, pp 127–139

    Chapter  Google Scholar 

  4. Cao L, Luo J, Kautz HS, Huang TS (2008) Annotating collections of photos using hierarchical event and scene models. In: CVPR, IEEE Computer Society. doi:10.1109/CVPR.2008.4587382

  5. Cobley P, Haeffner N (2009) Digital cameras and domestic photography: communication, agency and structure. Vis Commun 8(2):123–146

    Article  Google Scholar 

  6. Cohen J (1988) Statistical power analysis for the behavioral sciences. Psychology Press

  7. Cohen J (1992) A power primer. Psychol Bull 112(1):155

    Article  Google Scholar 

  8. Comaniciu D, Meer P (2002) Mean shift: A robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  9. Cooper M, Foote J, Girgensohn A, Wilcox L (2005) Temporal event clustering for digital photo collections. ACM Transactions on Multimedia Computing. Communicat Appl (TOMCCAP) 1(3):269– 288

    Google Scholar 

  10. Cooper ML (2011) Clustering geo-tagged photo collections using dynamic programming. In: Proceedings of the 19th ACM International Conference on Multimedia, MM ’11. ACM, New York, pp 1025–1028. doi:10.1145/2072298.2071929

  11. Datia N, Moura-Pires J, Correia N (2014) Summarised presentation of personal photo sets. In: Gurrin C, Hopfgartner F, Hurst W, Johansen H, Lee H, OConnor N (eds) MultiMedia modeling, lecture notes in computer science, vol 8325. Springer International Publishing, pp 195–206

  12. Datta R, Joshi D, Li J, Wang J Z (2008) Image retrieval: Ideas, influences, and trends of the new age. ACM Comput Surv 40 (2):1–60. doi:10.1145/1348246.1348248

    Article  Google Scholar 

  13. Do TMT, Blom J, Gatica-Perez D (2011) Smartphone usage in the wild: a large-scale analysis of applications and context. In: Proceedings of the 13th international conference on multimodal interfaces, ICMI ’11. ACM, New York, pp 353–360. doi:10.1145/2070481.2070550

  14. Foote J (2000) Automatic audio segmentation using a measure of audio novelty. In: International Conference on Multimedia and Expo, ICME 2000, vol. 1, pp 452–455

  15. Friedman W (2004) Time in autobiographical memory. Social Cognition 22(Special issue):591–605. doi:10.1521/soco.22.5.591.50766

  16. Gargi U (2003) Consumer media capture: Time-based analysis and event clustering. Tech. rep., Technical Report HPL-2003-165, HP Laboratories

  17. Georgescul M, Clark A, Armstrong S (2006) An analysis of quantitative aspects in the evaluation of thematic segmentation algorithms. In: Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue, Association for Computational Linguistics, pp 144–151

  18. Gozali J, Kan M, Sundaram H (2012) Hidden markov model for event photo stream segmentation. In: 2012 IEEE international conference on Multimedia and Expo Workshops (ICMEW). IEEE, pp 25–30

  19. Graham A, Garcia-Molina H, Paepcke A, Winograd T (2002) Time as essence for photo browsing through personal digital libraries. In: Proceedings of the second ACM/IEEE-CS joint conference on Digital libraries, pp 326–335

  20. Gye L (2007) Picture this: the impact of mobile camera phones on personal photographic practices. Continuum 21(2):279–288

    Article  Google Scholar 

  21. House N A V (2009) Collocated photo sharing, story-telling, and the performance of self. International Journal of Human-Computer Studies 67(12):1073–1086. doi:10.1016/j.ijhcs.09.003

    Article  Google Scholar 

  22. Janssen S, Chessa A, Murre J (2006) Memory for time: how people date events. Mem Cogn 34(1):138

    Article  Google Scholar 

  23. Johnson J, et al. (2010) Designing with the mind in mind: simple guide to understanding user interface design rules. Morgan Kaufmann

  24. Kang H, Bederson B B, Suh B (2007) Capture, annotate, browse, find, share: Novel interfaces for personal photo management. Int J Hum Comput Interact 23 (3):315–337. doi:10.1080/10447310701702618

    Article  Google Scholar 

  25. Kellerman A (1989) Time, space, and society: geographical societal perspectives. Kluwer Academic Pub

  26. Kirk D S, Sellen A (2010) On human remains: Values and practice in the home archiving of cherished objects. ACM Transactions on Computer-Human Interaction (TOCHI) 17(3):10. doi:10.1145/1806923.1806924

    Article  Google Scholar 

  27. Kwok S C, Shallice T (2012) Macaluso, E. Functional anatomy of temporal organisation and domain-specificity of episodic memory retrieval. Neuropsychologia 50 (12):2943–2955. doi:10.1016/j.neuropsychologia2012.07.025

    Google Scholar 

  28. Latif K, Mustofa K, Tjoa A (2006) An approach for a personal information management system for photos of a lifetime by exploiting semantics. In: Bressan S, Kng J, Wagner R (eds) Database and expert systems applications, lecture notes in computer science, vol 4080. Springer Berlin Heidelberg, pp 467–477. doi:10.1007/11827405_46

  29. Lietz P (2010) Research into questionnaire design. Int J Mark Res 52(2):249–272

    Article  Google Scholar 

  30. Loui A, Savakis A (2003) Automated event clustering and quality screening of consumer pictures for digital albuming. IEEE Trans Multimedia 5:390–402

    Article  Google Scholar 

  31. Lux M, Kogler M, del Fabro M (2010) Why did you take this photo: a study on user intentions in digital photo productions.. In: Proceedings of the 2010 ACM workshop on Social, adaptive and personalized multimedia interaction and access, SAPMIA ’10. ACM, New York, pp 41–44. doi:10.1145/1878061.1878075

  32. McGill R, Tukey J W, Larsen W A (1978) Variations of box plots. Am Stat 32(1):12–16

    Google Scholar 

  33. Naaman M, Song Y J, Paepcke A, Garcia-Molina H (2004) Automatic organization for digital photographs with geographic coordinates. In: Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, JCDL ’04. ACM Press, New York, pp 53–62

  34. Nielsen J (1994) Usability inspection methods. In: Conference companion on Human factors in computing systems. ACM, pp 413–414

  35. Pevzner L, Hearst M A (2002) A critique and improvement of an evaluation metric for text segmentation. Computational Linguistics 28(1):19–36

    Article  Google Scholar 

  36. Platt J, Czerwinski M, Field B (2003) Phototoc: automatic clustering for browsing personal photographs. Information, Communications and Signal Processing, 2003 and the Fourth Pacific Rim Conference on Multimedia Proceedings of the 2003 Joint Conference of the Fourth International Conference on 1 1:6–10

    Article  Google Scholar 

  37. Reeves L M, Lai J, Larson J A, Oviatt S, Balaji T S, Buisine S, Collings P, Cohen P, Kraal B, Martin J C, McTear M, Raman T, Stanney K M, Su H, Wang Q Y (2004) Guidelines for multimodal user interface design. Commun ACM 47(1):57–59

    Article  Google Scholar 

  38. Seltman HJ (2012) Experimental design and analysis. Online at: http://www.stat.cmu.edu/hseltman/309/Book/Book.pdf

  39. St Jacques P, Rubin D, LaBar K, Cabeza R (2008) The short and long of it: Neural correlates of temporal-order memory for autobiographical events. J Cognitive Neurosci 20(7):1327–1341, cited By (since 1996)29

  40. Sun F, Li H, Wang X (2013) Photo 4w: mobile photo management on what, where, who and when. Neurocomputing intelligent Processing Techniques for Semantic-based Image and Video Retrieval 119:59–64. doi:10.1016/j.neucom.2012.03.038

  41. Sun Y, Zhang H, Zhang L, Li M (2002) Myphotos: a system for home photo management and processing. In: Proceedings of the 10th ACM international conference on Multimedia ’02. ACM Press, New York, pp 81–82. doi:10.1145/641007.641022

  42. Tulving E (2002) Episodic memory: from mind to brain. Annual Review of Psychology 53(1):1–25

    Article  Google Scholar 

  43. Viana W, Bringel Filho J, Gensel J, Villanova-Oliver M, Martin H (2008) PhotoMap: from location and time to context-aware photo annotations. Journal of Location Based Services 2(3):211–235

    Article  Google Scholar 

  44. von Watzdorf S, Michahelles F (2010) Accuracy of positioning data on smartphones. In: Proceedings of the 3rd International Workshop on Location and the Web, ACM, p 2

  45. Whittaker S, Bergman O, Clough P (2010) Easy on that trigger dad: a study of long term family photo retrieval. Pers Ubiquit Comput 14(1):31–43

    Article  Google Scholar 

  46. Zerubavel E (1985) Hidden rhythms: schedules and calendars in social life. University of California Press

  47. Zerubavel E (1996) Social memories: steps to a sociology of the past. Qual Sociol 19(3):283–299

    Article  Google Scholar 

  48. Zhao M, Teo Y, Liu S, Chua TS, Jain R (2006) Automatic person annotation of family photo album. In: Sundaram H, Naphade M, Smith J, Rui Y (eds) Image and video retrieval, lecture notes in computer science, vol 4071. Springer, Berlin Heidelberg, pp 163–172. doi:10.1007/11788034_17

    Chapter  Google Scholar 

  49. Zuzanek J, Smale J (1993) Life-cycle variations in across-the-week allocation of time to selected daily activities. SOCIETY AND LEISURE-MONTREAL- 15:559–559

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. ISEL, Instituto Politécnico de Lisboa, Rua Conselheiro Emídio Navarro, 1959-007, Lisboa, Portugal

    Nuno Datia

  2. FCT, Universidade Nova de Lisboa, 2829-516, Caparica, Portugal

    João Moura Pires & Nuno Correia

Authors
  1. Nuno Datia
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. João Moura Pires
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nuno Correia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nuno Datia.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Datia, N., Moura Pires, J. & Correia, N. Time and space for segmenting personal photo sets. Multimed Tools Appl 76, 7141–7173 (2017). https://doi.org/10.1007/s11042-016-3341-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3341-2

Keywords

Search

Navigation