Sébastien Lallé
ABSTRACT
User performance and satisfaction when working with an interface is influenced by how quickly the user can acquire the skills necessary to work with the interface through practice. Learning curves are mathematical models that can represent a user's skill acquisition ability through parameters that describe the user's initial expertise as well as her learning rate. This information could be used by an interface to provide adaptive support to users who may otherwise be slow in learning the necessary skills. In this paper, we investigate the feasibility of predicting in real time a user's learning curve when working with ValueChart, an interactive visualization for decision making. Our models leverage various data sources (a user's gaze behavior, pupil dilation, cognitive abilities), and we show that they outperform a baseline that leverages only knowledge on user task performance so far. We also show that the best performing model achieves good accuracies in predicting users' learning curves even after observing users' performance only on a few tasks. These results are promising toward the design of user-adaptive visualizations that can dynamically support a user in acquiring the necessary skills to complete visual tasks.
Supplemental Material
- Akaike, H. A new look at the statistical model identification. IEEE Transactions on Automatic Control 19, 6 (1974), 716--723.Google Scholar
Cross Ref
- Amar, R., Eagan, J., and Stasko, J. Low-Level Components of Analytic Activity in Information Visualization. Proceedings of the 2005 IEEE Symposium on Information Visualization, IEEE Computer Society (2005), 15--21. Google ScholarDigital Library
- Anderson, J. R. Cognitive Skills and Their Acquisition. Psychology Press, 1981.Google Scholar
- Anderson, J. R. Rules of the Mind. Psych. Press, 2014.Google ScholarCross Ref
- Bailey, C. D. and McIntyre, E. V. The relation between fit and prediction for alternative forms of learning curves and relearning curves. IIE Transactions 29, 6 (1997), 487--495.Google ScholarCross Ref
- Beatty, J. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin 91, 2 (1982), 276--292.Google ScholarCross Ref
- Beck, J. E. and Sison, J. Using knowledge tracing in a noisy environment to measure student reading proficiencies. International Journal of Artificial Intelligence in Education 16, 2 (2006), 129--143. Google ScholarDigital Library
- Beck, J. E., Woolf, B. P., and Beal, C. R. ADVISOR: a machine-learning architecture for intelligent tutor construction. University of Massachusetts (2000).Google Scholar
- Bednarik, R., Eivazi, S., and Vrzakova, H. A Computational Approach for Prediction of Problem-Solving Behavior Using Support Vector Machines and Eye-Tracking Data. In Eye Gaze in Intelligent User Interfaces. Springer, 2013, 111--134.Google Scholar
- Bixler, R., Kopp, K., and D'Mello, S. Evaluation of a Personalized Method for Proactive Mind Wandering Reduction. Proc. of the 4th Workshop on Personalization Approaches for Learning Environments, UMAP (2014), 33--41.Google Scholar
- Bondareva, D., Conati, C., Feyzi-Behnagh, R., Harley, J. M., Azevedo, R., and Bouchet, F. Inferring Learning from Gaze Data during Interaction with an Environment to Support Self-Regulated Learning. Proc. of the 16th Int. Conference on Artificial Intelligence in Education, Springer (2013), 229--238.Google ScholarCross Ref
- Bunt, A., Conati, C., and McGrenere, J. Supporting Interface Customization Using a Mixed-initiative Approach. Proc. of the 12th International Conference on Intelligent User Interfaces, ACM (2007), 92--101. Google ScholarDigital Library
- Carenini, G., Conati, C., Hoque, E., Steichen, B., Toker, D., and Enns, J. T. Highlighting Interventions and User Differences: Informing Adaptive Information Visualization Support. Proc. of the SIGCHI Conf. on Human Factors in Computing Systems, ACM (2014). Google ScholarDigital Library
- Carenini, G. and Loyd, J. ValueCharts: analyzing linear models expressing preferences and evaluations. Proceedings of the Working Conference on Advanced Visual Interfaces, ACM (2004), 150--157. Google ScholarDigital Library
- Chamberlain, B. C., Carenini, G., Oberg, G., Poole, D., and Taheri, H. A Decision Support System for the Design and Evaluation of Sustainable Wastewater Solutions. IEEE Trans. on Comp. 63, (2014), pp. 129--141.Google ScholarDigital Library
- Çöltekin, A., Fabrikant, S. I., and Lacayo, M. Exploring the efficiency of users' visual analytics strategies based on sequence analysis of eye movement recordings. International Journal of Geographical Information Science 24, 10 (2010), 1559--1575. Google ScholarDigital Library
- Conati, C., Carenini, G., Steichen, B., and Toker, D. Evaluating the Impact of User Characteristics and Different Layouts on an Interactive Visualization for Decision Making. Proceedings of the 16th Eurographics Conf. on Visualization, (2014), 371--380.Google ScholarCross Ref
- Desmarais, M. Performance comparison of item-toitem skills models with the IRT single latent trait model. Proc. of UMAP, (2011), 75--86. Google ScholarDigital Library
- D'Mello, S., Olney, A., Williams, C., and Hays, P. Gaze tutor: A gaze-reactive intelligent tutoring system. Int. J. of Human-Comp. Studies 70, 5 (2012), 377--398. Google ScholarDigital Library
- Ekstrom, R. B. and Harman, H. H. Manual for Kit of Factor-referenced Cognitive Tests, 1976. Educational Testing Service, 1976.Google Scholar
- Ghazarian, A. and Noorhosseini, S. M. Automatic detection of users' skill levels using high-frequency user interface events. User Modeling and User-Adapted Interaction 20, 2 (2010), 109--146. Google ScholarDigital Library
- Granholm, E. and Steinhauer, S. R. Pupillometric measures of cognitive and emotional processes. Int. Journal of Psychophysiology 52, 1 (2004), 1--6.Google ScholarCross Ref
- Green, T. M. and Fisher, B. Towards the Personal Equation of Interaction: The impact of personality factors on visual analytics interface interaction. Proc. of the 2010 IEEE Symposium on Visual Analytics Science and Technology, IEEE (2010), 203--210.Google ScholarCross Ref
- Hess, E. H. and Polt, J. M. Pupil Size in Relation to Mental Activity during Simple Problem-Solving. Science 143, 3611 (1964), 1190--1192.Google ScholarCross Ref
- Holmes, G., Donkin, A., and Witten, I. H. Weka: A machine learning workbench. Second Australian and New Zealand Conference on Intelligent Information Systems, (1994), 357--361.Google ScholarCross Ref
- Hurst, A., Hudson, S. E., and Mankoff, J. Dynamic Detection of Novice vs. Skilled Use Without a Task Model. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2007), 271--280. Google ScholarDigital Library
- Iqbal, S. T., Adamczyk, P. D., Zheng, X. S., and Bailey, B. P. Towards an index of opportunity: understanding changes in mental workload during task execution. ACM Press (2005), 311. Google ScholarDigital Library
- Jaques, N., Conati, C., Harley, J. M., and Azevedo, R. Predicting Affect from Gaze Data during Interaction with an Intelligent Tutoring System. Proceedings of ITS, Springer (2014), 29--38.Google ScholarDigital Library
- Kardan, S. and Conati, C. Comparing and Combining Eye Gaze and Interface Actions for Determining User Learning with an Interactive Simulation. Proceedings of UMAP, (2013), 215--227.Google ScholarCross Ref
- Kruskal, W. Relative Importance by Averaging Over Orderings. American Statistician 41, 1 (1987), 6--10.Google Scholar
- Lallé, S., Mostow, J., Luengo, V., and Guin, N. Comparing Student Models in Different Formalisms by Predicting their Impact on Help Success. Proceedings of the 16th International Conference on Artificial Intelligence in Education, (2013), 161--170.Google ScholarCross Ref
- Luck, S. J. and Vogel, E. K. The capacity of visual working memory for features and conjunctions. Nature 390, 6657 (1997), 279--281.Google ScholarCross Ref
- Martínez-Gómez, P. and Aizawa, A. Recognition of understanding level and language skill using measurements of reading behavior. Proceedings of the 19th International Conference on Intelligent User Interfaces, ACM (2014), 95--104. Google ScholarDigital Library
- Ooms, K., De Maeyer, P., and Fack, V. Study of the attentive behavior of novice and expert map users using eye tracking. Cartography and Geographic Information Science 41, 1 (2014), 37--54.Google ScholarCross Ref
- Pascual-Cid, V., Vigentini, L., and Quixal, M. Visualising Virtual Learning Environments: Case Studies of the Website Exploration Tool. Proceedings of the 14th International Conference on Information Visualisation, IEEE (2010), 149--155. Google ScholarDigital Library
- Pavlik, P. I., Cen, H., and Koedinger, K. R. Performance Factors Analysis - A New Alternative to Knowledge Tracing. Proc. of the International Conf. on Artificial Intelligence in Education, IOS Press (2009), 531--538. Google ScholarDigital Library
- Prendinger, H., Hyrskykari, A., Nakayama, M., Istance, H., Bee, N., and Takahasi, Y. Attentive interfaces for users with disabilities: eye gaze for intention and uncertainty estimation. Universal Access in the Information Society 8, 4 (2009), 339--354. Google ScholarDigital Library
- Rotter, J. B. Generalized expectancies for internal versus external control of reinforcement. Psychological Monographs: General and Applied 80, 1 (1966), 1--28.Google ScholarCross Ref
- Saraiya, P., North, C., and Duca, K. An Insight-Based Methodology for Evaluating Bioinformatics Visualizations. IEEE Transactions on Visualization and Computer Graphics 11, 4 (2005), 443--456. Google ScholarDigital Library
- Speelman, C. P. and Kirsner, K. Beyond the Learning Curve: The Construction of Mind. Oxford University Press, 2005.Google ScholarCross Ref
- Steichen, B., Conati, C., and Carenini, G. Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data. ACM Trans. on Interactive Intelligent Systems 4, 2 (2014). Google ScholarDigital Library
- Toker, D., Steichen, B., Gingerich, M., Conati, C., and Carenini, G. Towards Facilitating User Skill Acquisition: Identifying Untrained Visualization Users Through Eye Tracking. Proceedings of the 19th International Conference on Intelligent User Interfaces, ACM (2014), 105--114. Google ScholarDigital Library
- Turner, M. L. and Engle, R. W. Is working memory capacity task dependent? Journal of Memory and Language 28, 2 (1989), 127--154.Google ScholarCross Ref
- Yudelson, M. V., Koedinger, K. R., and Gordon, G. J. Individualized Bayesian Knowledge Tracing Models. Proceedings of AIED, Springer (2013), 171--180.Google ScholarCross Ref
- Zhu, Y. Measuring Effective Data Visualization. Proceedings of the 3rd International Symposium on Advances in Visual Computing, Springer (2007), 652-- 661. Google ScholarDigital Library
Index Terms
- Prediction of Users' Learning Curves for Adaptation while Using an Information Visualization
Recommendations
User-adaptive information visualization: using eye gaze data to infer visualization tasks and user cognitive abilities
IUI '13: Proceedings of the 2013 international conference on Intelligent user interfacesInformation Visualization systems have traditionally followed a one-size-fits-all model, typically ignoring an individual user's needs, abilities and preferences. However, recent research has indicated that visualization performance could be improved by ...
Prediction of individual learning curves across information visualizations
Confident usage of information visualizations is thought to be influenced by cognitive aspects as well as amount of exposure and training. To support the development of individual competency in visualization processing, it is important to ascertain if ...
Predicting Visual Search Task Success from Eye Gaze Data as a Basis for User-Adaptive Information Visualization Systems
Information visualizations are an efficient means to support the users in understanding large amounts of complex, interconnected data; user comprehension, however, depends on individual factors such as their cognitive abilities. The research literature ...
Comments