Abstract
The growing use of computer-like tablets and PCs in educational settings is enabling more students to study online courses featuring computer-aided tests. Preparing these tests imposes a large burden on teachers who have to prepare a large number of questions because they cannot reuse the same questions many times as students can easily memorize their solutions and share them with other students, which degrades test reliability. Another burden is appropriately setting the level of question difficulty to ensure test discriminability. Using magic square puzzles as examples of mathematical questions, we developed a method for automatically preparing puzzles with appropriate levels of difficulty. We used crowdsourcing to collect answers to sample questions to evaluate their difficulty. Item response theory was used to evaluate the difficulty of the questions from crowdworkers’ answers. Deep learning was then used to build a model for predicting the difficulty of new questions.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Coughlan, S.: Tablet computers in ‘70% of schools’, BBC NEWS, 3 December 2014. https://www.bbc.com/news/education-30216408
Ministry of Education, Culture, Sports, Science and Technology: “The research for situation of educational informatization in school in 2017”. https://www.e-stat.go.jp/stat-search/files?page=1&cycle_facet=cycle
Hoshino, A., Nakagawa, H.: A real-time multiple-choice question generation for language testing – a preliminary study. In: Proceedings of the 2nd Workshop on Building Educational Applications, Using NLP, pp. 17–20 (2005)
Hill, J., Simba, R.: Automatic generation of context-based fill-in-the-blank exercises using co-occurrence likelihoods and Google n-grams. In: Proceedings of the 11th Workshop on Innovative Use of NLP for Building Educational Applications, pp. 23–30 (2016)
Sakaguchi, K., Arase, Y., Komachi, M.: Discriminative approach to fill-in-the-blank quiz generation for language learners. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pp. 238–242 (2013)
Liu, M., Rus, V., Liu, L.: Automatic Chinese factual question generation. IEEE Trans. Learn. Technol. 10(2), 194–204 (2017)
Rocha, O.R., Zucker, C.E.: Automatic generation of educational quizzes from domain ontologies. In: EDULEARN 2017-9th International Conference on Education and New Learning Technologies, pp. 4024–4030 (2017)
Papasalouros, A., Kanaris, K., Kotis, K.: Automatic generation of multiple choice questions from domain ontologies. In: Proceeding of the 8th International Conference on Web Intelligence, Mining and Semantics, Article No. 32 (2018)
Takano, A., Hashimoto, J.: Drill exercise generation based on the knowledge base. The Special Interest Group Technical Reports of Information Processing Society of Japan, NL-160, pp. 23–28 (2003)
Hurudate, M., Takagi, M., Takagi, T.: A proposal and evaluation on a method of automatic construction of knowledge base for automatic generation of exam questions. The Special Interest Group Technical Reports of Information Processing Society of Japan, vol. 128, no. 14 (2015)
Lord, F.M.: A Theory of Test Scores (Psychometric Monograph). Psychometric Society, Iowa City (1952)
Mislevy, R.J.: Bayes model estimation in item response theory. Psychometrika 51(2), 177–195 (1986)
Sakumura, T., Tokunaga, M., Hirose, H.: Making up the complete matrix from the incomplete matrix using the EM-type IRT and its application. Inf. Process. Soc. Japan J. Trans. Math. Modeling Appl. 7(2), 17–26 (2014)
Lancers Homepage. https://www.lancers.jp
Acknowledgements
This work was partially supported by JSPS KAKENHI Grant Numbers JP15H02782 and JP18H03337, and by the Telecommunications Advancement Foundation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Sekiya, R., Oyama, S., Kurihara, M. (2019). User-Adaptive Preparation of Mathematical Puzzles Using Item Response Theory and Deep Learning. In: Wotawa, F., Friedrich, G., Pill, I., Koitz-Hristov, R., Ali, M. (eds) Advances and Trends in Artificial Intelligence. From Theory to Practice. IEA/AIE 2019. Lecture Notes in Computer Science(), vol 11606. Springer, Cham. https://doi.org/10.1007/978-3-030-22999-3_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-22999-3_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22998-6
Online ISBN: 978-3-030-22999-3
eBook Packages: Computer ScienceComputer Science (R0)