ABSTRACT
Multimedia data is by nature heterogeneous, conveying semantic information through multiple cues. Text analysis of closed captions already brought us understanding of the spoken information. Today's advances in computer vision now enable us to look for relevant semantic information from the visual content of real-world archives. Combining these two levels of extracted information to make sense of an archive still remains a challenge. Multiplex net- works, which model multiple families of interactions in a graph, can capture and combine both sources of semantics. We can leverage on these objects to extract hierarchies and integrate them in an interactive heterogeneous "visual cloud". Inspired by word clouds, these clouds allow to grasp visual and textual semantic information captured from a multimedia collection all at once. The interaction then enables direct access to the relevant video. We demonstrate our system with the exploration of a Japanese news archive.
Supplemental Material
Available for Download
PDF File
- Brandon Amos, Bartosz Ludwiczuk, and Mahadev Satyanarayanan. 2016. OpenFace: A general-purpose face recognition library with mobile applications. Technical Report. Technical report, Carnegie Mellon University-CS-16-118, Carnegie Mellon University School of Computer Science.Google Scholar
- Nancy A Chinchor, James J Thomas, Pak Chung Wong, Michael G Christel, and William Ribarsky. 2010. Multimedia analysis visual analytics= multimedia analytics. IEEE computer graphics and applications Vol. 30, 5 (2010), 52--60. Google ScholarDigital Library
- Ichiro Ide et al.. 2004. Topic threading for structuring a large-scale news video archive. Image and Video Retrieval Vol. 1, 1 (2004), 123--131.Google Scholar
- Kuno Kurzhals, Markus John, Florian Heimerl, Paul Kuznecov, and Daniel Weiskopf. 2016. Visual Movie Analytics. IEEE Transactions on Multimedia Vol. 18, 11 (2016), 2149--2160. Google ScholarDigital Library
- D. D. Le and S. Satoh. 2011. Indexing Faces in Broadcast News Video Archives. 2011 IEEE 11th ICDM Workshops. 519--526. Google ScholarDigital Library
- Thanh Duc Ngo et al.. 2013. Face retrieval in large-scale news video datasets. IEICE Trans. on Information and Systems Vol. 96, 8 (2013), 1811--1825.Google ScholarCross Ref
- Benjamin Renoust, Guy Melancon, and Tamara Munzner. 2015. Detangler: Visual Analytics for Multiplex Networks Computer Graphics Forum, Vol. Vol. 34. Wiley Online Library, 321--330. Google ScholarDigital Library
- Benjamin Renoust, Guy Melanccon, and Marie-Luce Viaud. 2014. Entanglement in multiplex networks: understanding group cohesion in homophily networks. Social Network Analysis-Community Detection and Evolution. Springer, 89--117.Google Scholar
- Benjamin Renoust, Haolin Ren, Guy Melanccon, and Marie-Luce Viaud. 2017. Layer Hierarchization in Multiplex Networks for Word Cloud Visualization Visualization Symposium (PacificVis), 2017 IEEE Pacific. IEEE.Google Scholar
- Jianbo Shi and Carlo Tomasi. 1994. Good features to track. In Proceedings CVPR'94. IEEE, 593--600.Google Scholar
- Fernanda B Viegas, Martin Wattenberg, and Jonathan Feinberg. 2009. Participatory visualization with Wordle. IEEE transactions on visualization and computer graphics, Vol. 15, 6 (2009), 1137--1144. Google ScholarDigital Library
- Paul Viola and Michael J Jones. 2004. Robust real-time face detection. International journal of computer vision Vol. 57, 2 (2004), 137--154. Google ScholarDigital Library
- Weixin Wang, Hui Wang, Guozhong Dai, and Hongan Wang. 2006. Visualization of Large Hierarchical Data by Circle Packing Proc. of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, NY, USA, 517--520. Google ScholarDigital Library
Index Terms
- FaceCloud: Heterogeneous Cloud Visualization of Multiplex Networks for Multimedia Archive Exploration
Recommendations
News Archive Exploration Combining Face Detection and Tracking with Network Visual Analytics
MM '16: Proceedings of the 24th ACM international conference on MultimediaVisual analytics helps analytical reasoning and exploration of complex systems, for which it combines the means of interactive visualization, with the power of data analytics. The recent progress in computer vision techniques opens wide applications in ...
Big data exploration through visual analytics
VAST '12: Proceedings of the 2012 IEEE Conference on Visual Analytics Science and Technology (VAST)SAS® Visual Analytics Explorer is an advanced data visualization and exploratory data analysis application that is a component of the SAS Visual Analytics solution. It excels at handling big data problems like the VAST challenge. With a wide range of ...
Enhanced Visualization of News Shot Cloud with Employing Circular Layout
VINCI '15: Proceedings of the 8th International Symposium on Visual Information Communication and InteractionTV news programs are one of major information sources for daily lives. Since the computer storage cost has reduced significantly, today we can digitally archive a huge amount of TV news programs. On the other hand, as the archive size grows larger, the ...
Comments