Dominika Wanat
ABSTRACT
Most user studies in the QoE multimedia domain are done by asking users about quality. This approach has advantages: it obtains many answers and reduces variance by repeated measurements. However, the results obtained in this context may be different from those obtained in the real application, since quality is not asked about this often in everyday life. It is more natural is to focus on user behavior. The proposed PhD focuses on a method for performing experiments based on observations of a participant’s behavior. We address two main challenges that exist in any new experiment design: how to calculate the interval validity of the proposed method and how to analyze the obtained data. The data analysis we propose is based on psychometric functions. We propose two different experiments, one of which is already ongoing.
- Donald T Campbell and Julian C Stanley. 1963. Experimental and quasi-experimental designs for research. Rand McNally, Chicago, IL.Google Scholar
- Thomas D Cook, Donald Thomas Campbell, and Arles Day. 1979. Quasi-experimentation: Design & analysis issues for field settings. Vol. 351. Houghton Mifflin Boston.Google Scholar
- L. Danon, A. Diaz-Guilera, J. Duch, and A. Arenas. 2005. Comparing community structure identification. J. Stat. Mech.-Theory Exp. (2005), P09008.Google Scholar
- Lorin Dole and Wendy Ju. 2019. Face and Ecological Validity in Simulations: Lessons from Search-and-Rescue HRI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3290605.3300681Google ScholarDigital Library
- Håkan Fischer, Tomas Furmark, Gustav Wik, and Mats Fredrikson. 2000. Brain representation of habituation to repeated complex visual stimulation studied with PET. Neuroreport 11, 1 (2000), 123–126.Google ScholarCross Ref
- Boni García, Luis López-Fernández, Francisco Gortázar, and Micael Gallego. 2019. Practical evaluation of VMAF perceptual video quality for WebRTC applications. Electronics 8, 8 (2019), 854.Google ScholarCross Ref
- ITU-T. 1998. Subjective assessment of video quality using Expert Viewing Protocol. Technical Report BT.710. Geneva.Google Scholar
- ITU-T. 1998. Subjective audiovisual quality assessment methods for multimedia applications. Recommendation P.911. International Telecommunication Union, Geneva.Google Scholar
- ITU-T. 2007. Methodology for the subjective assessment of video quality in multimedia applications. Technical Report BT.1788. Geneva.Google Scholar
- ITU-T. 2016. Subjective assessment methods for 3D video quality. Recommendation P.915. International Telecommunication Union, Geneva.Google Scholar
- ITU-T. 2017. Subjective assessment of video quality using Expert Viewing Protocol. Technical Report BT.2095. Geneva.Google Scholar
- ITU-T. 2017. Subjective quality evaluation of audio and audiovisual multiparty telemeetings. Technical Report P.1301. Geneva.Google Scholar
- ITU-T. 2019. Dimension-based subjective quality evaluation for video content. Technical Report P.918. Geneva.Google Scholar
- ITU-T. 2019. Methodologies for the subjective assessment of the quality of television images. Technical Report BT.500. Geneva.Google Scholar
- ITU-T. 2020. Subjective test methodologies for 360º video on head-mounted displays. Technical Report P.919. Geneva.Google Scholar
- ITU-T. 2020. Subjective test methodology for assessing impact of initial loading delay on quality of experience. Technical Report P.917. Geneva.Google Scholar
- ITU-T. 2021. Methods for the subjective assessment of video quality, audio quality and audiovisual quality of Internet video and distribution quality television in any environment. Recommendation P.913. International Telecommunication Union, Geneva.Google Scholar
- ITU-T. 2022. Subjective video quality assessment methods for multimedia applications. Recommendation P.910. International Telecommunication Union, Geneva.Google Scholar
- John F Kihlstrom. 2021. Ecological validity and “ecological validity”. Perspectives on Psychological Science 16, 2 (2021), 466–471.Google ScholarCross Ref
- F.A.A. Kingdom. 2012. Psychophysics. In Encyclopedia of Human Behavior (Second Edition) (second edition ed.), V.S. Ramachandran (Ed.). Academic Press, San Diego, 234–239. https://doi.org/10.1016/B978-0-12-375000-6.00296-2Google Scholar
- Michael Langer. 2018. COMP 546 - Computational Perception lecture notes. https://www.cim.mcgill.ca/ langer/546/13-psychophysics-notes.pdfGoogle Scholar
- Letitia Lew, Truc Nguyen, Solomon Messing, and Sean Westwood. 2011. Of course I wouldn’t do that in real life: advancing the arguments for increasing realism in HCI experiments. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. 419–428.Google Scholar
- Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward A Practical Perceptual Video Quality Metric. https://netflixtechblog.com/toward-a-practical-perceptual-video-quality-metric-653f208b9652Google Scholar
- Hyunwoo Nam, Kyung-Hwa Kim, and Henning Schulzrinne. 2016. QoE matters more than QoS: Why people stop watching cat videos. In IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications. 1–9. https://doi.org/10.1109/INFOCOM.2016.7524426Google ScholarDigital Library
- Cheryl Olman (Ed.). 2022. Introduction to Sensation and Perception. University of Minnesota Libraries Publishing, Chapter Basics: neuroscience and psychophysics.Google Scholar
- Marta Orduna, César Díaz, Lara Muñoz, Pablo Pérez, Ignacio Benito, and Narciso García. 2019. Video multimethod assessment fusion (VMAF) on 360VR contents. IEEE Transactions on Consumer Electronics 66, 1 (2019), 22–31.Google ScholarDigital Library
- Ulrich Reiter, Kjell Brunnström, Katrien De Moor, Mohamed-Chaker Larabi, Manuela Pereira, Antonio Pinheiro, Junyong You, and Andrej Zgank. 2014. Factors Influencing Quality of Experience. Springer International Publishing, Cham, 55–72. https://doi.org/10.1007/978-3-319-02681-7_4Google Scholar
- Werner Robitza, Alexander M. Dethof, Steve Göring, Alexander Raake, André Beyer, and Tim Polzehl. 2020. Are You Still Watching? Streaming Video Quality and Engagement Assessment in the Crowd. In 2020 Twelfth International Conference on Quality of Multimedia Experience (QoMEX). 1–6. https://doi.org/10.1109/QoMEX48832.2020.9123148Google Scholar
- Werner Robitza and Alexander Raake. 2016. (Re-) actions speak louder than words? A novel test method for tracking user behavior in Web video services. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). IEEE, 1–6.Google ScholarCross Ref
- Werner Robitza and Alexander Raake. 2016. (Re-)actions speak louder than words? A novel test method for tracking user behavior in web video services. In 2016 Eighth International Conference on Quality of Multimedia Experience (QoMEX). 1–6. https://doi.org/10.1109/QoMEX.2016.7498926Google ScholarCross Ref
- Colin Robson and Kieran McCartan. 2016. Real world research. Wiley Global Education.Google Scholar
- Srisakul Thakolsri, Wolfgang Kellerer, and Eckehard Steinbach. 2011. QoE-Based Cross-Layer Optimization of Wireless Video with Unperceivable Temporal Video Quality Fluctuation. In 2011 IEEE International Conference on Communications (ICC). 1–6. https://doi.org/10.1109/icc.2011.5963296Google Scholar
- Khalil ur Rehman Laghari, Rishabh Gupta, Jan-Niklas Antons, Robert Schleicher, Sebastian Möller, and Tiago H. Falk. 2013. Objective characterization of human behavioural characteristics for QoE assessment: A pilot study on the use of electroencephalography features. In 2013 IEEE Globecom Workshops (GC Wkshps). 1168–1173. https://doi.org/10.1109/GLOCOMW.2013.6825151Google ScholarCross Ref
- Stuart A. Wallis, Daniel H. Baker, Tim S. Meese, and Mark A. Georgeson. 2013. The slope of the psychometric function and non-stationarity of thresholds in spatiotemporal contrast vision. Vision Research 76 (2013), 1–10. https://doi.org/10.1016/j.visres.2012.09.019Google ScholarCross Ref
- Nikolas Wehner, Michael Seufert, Sebastian Egger-Lampl, Bruno Gardlo, Pedro Casas, and Raimund Schatz. 2020. Scoring High: Analysis and Prediction of Viewer Behavior and Engagement in the Context of 2018 FIFA WC Live Streaming. In Proceedings of the 28th ACM International Conference on Multimedia (Seattle, WA, USA) (MM ’20). Association for Computing Machinery, New York, NY, USA, 807–815. https://doi.org/10.1145/3394171.3414016Google ScholarDigital Library
- Di Yuan, Tiesong Zhao, Yiwen Xu, Hong Xue, and Liqun Lin. 2019. Visual JND: A Perceptual Measurement in Video Coding. IEEE Access 7 (2019), 29014–29022. https://doi.org/10.1109/ACCESS.2019.2901342Google ScholarCross Ref
- Jingwen Zhu and Patrick Le Callet. 2022. Just Noticeable Difference (JND) and Satisfied User Ratio (SUR) Prediction for Compressed Video: Research Proposal. In Proceedings of the 13th ACM Multimedia Systems Conference (Athlone, Ireland) (MMSys ’22). Association for Computing Machinery, New York, NY, USA, 393–397. https://doi.org/10.1145/3524273.3533933Google ScholarDigital Library
Index Terms
- Behavior as a Function of Video Quality in an Ecologically Valid Experiment
Recommendations
Studying the Perceived Quality Variation over Time for Video Quality Assessment
PIVP '14: Proceedings of the 1st International Workshop on Perception Inspired Video ProcessingReliably predicting how humans assess video quality aspects remains an academic challenge, partly due to our limited understanding of human visual system and of how it affects the perception of spatial and temporal distortions. In current video chains, ...
Towards Behavior-oriented Quality of Experience Assessment for Online Video Services
TVX '17 Adjunct: Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online VideoWhen users decide to cancel a video playback because the video is not loading, this is a first step in their abandonment of a service. Internet or video service providers however are strongly interested in engaging people to continue using their offers. ...
Describing Subjective Experiment Consistency by p-Value P--P Plot
MM '20: Proceedings of the 28th ACM International Conference on MultimediaThere are phenomena that cannot be measured without subjective testing. However, subjective testing is a complex issue with many influencing factors. These interplay to yield either precise or incorrect results. Researchers require a tool to classify ...
Comments