Skip to main content
SpringerLink
Log in

Hint Generation Under Uncertainty: The Effect of Hint Quality on Help-Seeking Behavior

  • Conference paper
  • First Online:
Book cover Artificial Intelligence in Education (AIED 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10331))

Included in the following conference series:

Abstract

Much research in Intelligent Tutoring Systems has explored how to provide on-demand hints, how they should be used, and what effect they have on student learning and performance. Most of this work relies on hints created by experts and assumes that all help provided by the tutor is correct and of high quality. However, hints may not all be of equal value, especially in open-ended problem solving domains, where context is important. This work argues that hint quality, especially when using data-driven hint generation techniques, is inherently uncertain. We investigate the impact of hint quality on students’ help-seeking behavior in an open-ended programming environment with on-demand hints. Our results suggest that the quality of the first few hints on an assignment is positively associated with future hint use on the same assignment. Initial hint quality also correlates with possible help abuse. These results have important implications for hint design and generation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Complete rubric and dataset available at: http://go.ncsu.edu/aied2017-rubric.

  2. 2.

    The Mann-Whitney U test was used were data were not normally distributed.

References

  1. Aleven, V., Koedinger, K.: Limitations of student control: do students know when they need help? In: International Conference on Intelligent Tutoring Systems, pp. 292–303 (2000)

    Google Scholar 

  2. Aleven, V., Mclaren, B., Roll, I., Koedinger, K.: Toward meta-cognitive tutoring: a model of help seeking with a cognitive tutor. Int. J. Artif. Intell. Educ. 16(2), 101–128 (2006)

    Google Scholar 

  3. Aleven, V., Roll, I., Mclaren, B.M., Koedinger, K.R.: Help helps, but only so much: research on help seeking with intelligent tutoring systems. Int. J. Artif. Intell. Educ. 26(1), 1–19 (2016)

    Article  Google Scholar 

  4. Aleven, V., Stahl, E., Schworm, S., Fischer, F., Wallace, R.: Help seeking and help design in interactive learning environments vincent. Rev. Educ. Res. 73(3), 277–320 (2003)

    Article  Google Scholar 

  5. Anderson, J.R.: Rules of the Mind. Lawrence Erlbaum Associates, Hillsdale (1993)

    Google Scholar 

  6. Barnes, T., Stamper, J.: Toward automatic hint generation for logic proof tutoring using historical student data. In: International Conference on Intelligent Tutoring Systems, pp. 373–382 (2008)

    Google Scholar 

  7. Corbett, A., Anderson, J.: Locus of feedback control in computer-based tutoring: impact on learning rate, achievement and attitudes. In: SIGCHI Conference on Human Computer Interaction, pp. 245–252 (2001)

    Google Scholar 

  8. Garcia, D., Harvey, B., Barnes, T.: The beauty and joy of computing. ACM Inroads 6(4), 71–79 (2015)

    Article  Google Scholar 

  9. Kluger, A.N., Denisi, A.: The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 119(2), 254–284 (1996)

    Article  Google Scholar 

  10. Luckin, R., Du Boulay, B.: Ecolab: the development and evaluation of a Vygotskian design framework. Int. J. Artif. Intell. Educ. 10, 198–220 (1999)

    Google Scholar 

  11. Piech, C., Sahami, M., Huang, J., Guibas, L.: Autonomously generating hints by inferring problem solving policies. In: ACM Conference on Learning @ Scale, pp. 1–10 (2015)

    Google Scholar 

  12. Price, T.W., Dong, Y., Barnes, T.: Generating data-driven hints for open-ended programming. In: International Conference on Educational Data Mining (2016)

    Google Scholar 

  13. Price, T.W., Dong, Y., Lipovac, D.: iSnap: towards intelligent tutoring in novice programming environments. In: ACM Technical Symposium on Computer Science Education (2017)

    Google Scholar 

  14. Rivers, K., Koedinger, K.R.: Data-driven hint generation in vast solution spaces: a self-improving python programming tutor. Int. J. Artif. Intell. Educ. 27(1), 37–64 (2017). https://link.springer.com/article/10.1007/s40593-015-0070-z

    Article  Google Scholar 

  15. Roll, I., Aleven, V., McLaren, B.M., Ryu, E., Baker, R.S.J., Koedinger, K.R.: The help tutor: does metacognitive feedback improve students’ help-seeking actions, skills and learning? In: Ikeda, M., Ashley, K.D., Chan, T.-W. (eds.) ITS 2006. LNCS, vol. 4053, pp. 360–369. Springer, Heidelberg (2006). doi:10.1007/11774303_36

    Chapter  Google Scholar 

  16. Stamper, J., Eagle, M., Barnes, T., Croy, M.: Experimental evaluation of automatic hint generation for a logic tutor. Int. J. Artif. Intell. Educ. 22(1), 3–17 (2013)

    Google Scholar 

  17. Wood, D.: Scaffolding, contingent tutoring and computer-supported learning. Int. J. Artif. Intell. Educ. 12, 280–292 (2001)

    Google Scholar 

  18. Wood, H., Wood, D.: Help seeking, learning and contingent tutoring. Comput. Educ. 33(2–3), 153–169 (1999)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. North Carolina State University, Raleigh, NC, 27606, USA

    Thomas W. Price, Rui Zhi & Tiffany Barnes

Authors
  1. Thomas W. Price
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rui Zhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tiffany Barnes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas W. Price .

Editor information

Editors and Affiliations

  1. Human-Centered Multimedia, University of Augsburg, Augsburg, Germany

    Elisabeth André

  2. University of Pennsylvania, Philadelphia, Pennsylvania, USA

    Ryan Baker

  3. Department of Psychology, University of Memphis, Memphis, Tennessee, USA

    Xiangen Hu

  4. Ateneo de Manila University, Quezon City, Philippines

    Ma. Mercedes T. Rodrigo

  5. University of Sussex, Brighton, United Kingdom

    Benedict du Boulay

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Price, T.W., Zhi, R., Barnes, T. (2017). Hint Generation Under Uncertainty: The Effect of Hint Quality on Help-Seeking Behavior. In: André, E., Baker, R., Hu, X., Rodrigo, M., du Boulay, B. (eds) Artificial Intelligence in Education. AIED 2017. Lecture Notes in Computer Science(), vol 10331. Springer, Cham. https://doi.org/10.1007/978-3-319-61425-0_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-61425-0_26

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-61424-3

  • Online ISBN: 978-3-319-61425-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation