Chatchai Wangwiwattana
ABSTRACT
The past decade has witnessed enormous advancement in online educational resources. One noteworthy advancement has been the development of automatic learning platforms. The introduction of this new technology has raised questions about its effectiveness in aiding educators to improve the engagement of students and evaluate their achievement of learning outcomes. While the use of open-ended questions to assess learners' outcomes is valuable, the workload demanded of educators can increase considerably when open-ended questions are used in large classes. We have experimented with a semi-automatic method to help grade short open-ended questions answered in Thai language. Our method employed Keyword Matching and unsupervised document grouping. Fixed types of questions were tested using different algorithms. Keyword Matching was found to be an effective method for a relatively fixed, yet open-ended set of answers. For non-fixed types of answers, Document Clustering proved suitable. In generating grading tools, we adopted three methods: Keyword Matching; Sentence Vector Similarity Ranking; and Document Clustering with TF-IDF and K-Means. The algorithms were found to be useful for online learning and grading specific content-based answers which, in turn, may be used as a guide in directing educators who wish to elicit information to provide feedback.
- M. M. M. Abdelmalak and J. L. Parra, “Expanding Learning Opportunities for Graduate Students with HyFlex Course Design,” International Journal of Online Pedagogy and Course Design, vol. 6, no. 4, pp. 19–37, Oct. 2016, doi: 10.4018/IJOPCD.2016100102.Google Scholar
Digital Library
- B. Beatty, “Hybrid courses with flexible participation: The hyflex course design,” Practical Applications and Experiences in K-20 Blended Learning Environments, pp. 153–177, Dec. 2013, doi: 10.4018/978-1-4666-4912-5.CH011.Google ScholarCross Ref
- M. Detyna, R. Sanchez-Pizani, V. Giampietro, E. J. Dommett, and K. Dyer, “Hybrid flexible (HyFlex) teaching and learning: climbing the mountain of implementation challenges for synchronous online and face-to-face seminars during a pandemic,” Learn Environ Res, Apr. 2022, doi: 10.1007/s10984-022-09408-y.Google ScholarCross Ref
- N. Thi, H. Minh, and T. T. Linh, “Designing Online English Grammar Exercises 10th Graders via Learning Management System Chamilo,” Journal of English Language Teaching and Applied Linguistics , vol. 3, no. 5, pp. 55–63, May 2021, doi: 10.32996/JELTAL.2021.3.5.6.Google ScholarCross Ref
- B. L. Moorhouse, Y. Li, and S. Walsh, “E-Classroom Interactional Competencies: Mediating and Assisting Language Learning During Synchronous Online Lessons:,” https://doi.org/10.1177/0033688220985274, Feb. 2021, doi: 10.1177/0033688220985274.Google ScholarCross Ref
- M. A. Hussein, H. Hassan, and M. Nassef, “Automated language essay scoring systems: A literature review,” PeerJ Comput Sci, vol. 2019, no. 8, 2019, doi: 10.7717/PEERJ-CS.208.Google ScholarCross Ref
- V. S. Kumar and D. Boulanger, “Automated Essay Scoring and the Deep Learning Black Box: How Are Rubric Scores Determined?,” Int J Artif Intell Educ, vol. 31, no. 3, pp. 538–584, Sep. 2021, doi: 10.1007/S40593-020-00211-5.Google ScholarCross Ref
- D. Ramesh and S. K. Sanampudi, “An automated essay scoring systems: a systematic literature review,” Artif Intell Rev, vol. 55, no. 3, p. 2495, Mar. 2022, doi: 10.1007/S10462-021-10068-2.Google ScholarDigital Library
- V. Kumar and D. Boulanger, “Explainable Automated Essay Scoring: Deep Learning Really Has Pedagogical Value,” Front Educ (Lausanne), vol. 5, Oct. 2020, doi: 10.3389/FEDUC.2020.572367.Google ScholarCross Ref
- B. S. Bloom, M. D. Engelhart, E. J. Furst, and D. R. Krathwohl, Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook I: Cognitive Domain. 1956.Google Scholar
- J. Hoblos, “Experimenting with Latent Semantic Analysis and Latent Dirichlet Allocation on Automated Essay Grading,” in 2020 Seventh International Conference on Social Networks Analysis, Management and Security (SNAMS), Dec. 2020, pp. 1–7. doi: 10.1109/SNAMS52053.2020.9336533.Google ScholarCross Ref
- S. Ludwig, C. Mayer, C. Hansen, K. Eilers, and S. Brandt, “Automated Essay Scoring Using Transformer Models,” Psych, vol. 3, no. 4, pp. 897–915, Dec. 2021, doi: 10.3390/psych3040056.Google ScholarCross Ref
- E. Mayfield and A. W. Black, “Should You Fine-Tune BERT for Automated Essay Scoring?,” in Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, 2020, pp. 151–162. doi: 10.18653/v1/2020.bea-1.15.Google ScholarCross Ref
- J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” NAACL HLT 2019 - 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Proceedings of the Conference, vol. 1, pp. 4171–4186, Oct. 2018, doi: 10.48550/arxiv.1810.04805.Google ScholarCross Ref
- L.-H. Chang, I. Rastas, S. Pyysalo, and F. Ginter, “Deep learning for sentence clustering in essay grading support,” Apr. 23, 2021. https://arxiv.org/abs/2104.11556 (accessed Oct. 20, 2022).Google Scholar
- D. S. Morris, “Automatically grading Java programming assignments via reflection, inheritance, and regular expressions,” Proceedings - Frontiers in Education Conference, vol. 1, 2002, doi: 10.1109/FIE.2002.1157985.Google ScholarCross Ref
- J. Eisenstein, Introduction to Natural Language Processing. USA: Westchester Publishing Services, 2019.Google Scholar
- P. Singhapreecha, “Review of the book: A reference grammar of Thai,” Lingua, vol. 117, no. 8, pp. 1497–1512, Aug. 2007, doi: 10.1016/J.LINGUA.2006.09.005.Google ScholarCross Ref
- R. Arreerard, S. Mander, and S. Piao, “Survey on Thai NLP Language Resources and Tools,” in Proceedings of the Thirteenth Language Resources and Evaluation Conference, Jun. 2022, vol. 4695, pp. 6495–6505.Google Scholar
- J. Howard and S. Ruder, “Universal Language Model Fine-tuning for Text Classification,” ACL 2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (Long Papers), vol. 1, pp. 328–339, Jan. 2018, doi: 10.48550/arxiv.1801.06146.Google ScholarCross Ref
- M. A. Hussein, H. Hassan, and M. Nassef, “Automated language essay scoring systems: A literature review,” PeerJ Comput Sci, vol. 2019, no. 8, 2019, doi: 10.7717/PEERJ-CS.208.Google ScholarCross Ref
- D. J. C. Mackay, Information theory, inference, and learning algorithms Cambridge. 2003.Google Scholar
- L. Logeswaran and H. Lee, “An efficient framework for learning sentence representations,” in 6th International Conference on Learning Representations, ICLR 2018 - Conference Track Proceedings, 2018.Google Scholar
- O. Sychev, A. Anikin, and A. Prokudin, “Automatic grading and hinting in open-ended text questions,” Cogn Syst Res, vol. 59, 2020, doi: 10.1016/j.cogsys.2019.09.025.Google ScholarDigital Library
Index Terms
- Semi-Automatic Short-Answer Grading Tools for Thai Language using Natural Language Processing
Recommendations
Predicting math performance using natural language processing tools
LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge ConferenceA number of studies have demonstrated links between linguistic knowledge and performance in math. Studies examining these links in first language speakers of English have traditionally relied on correlational analyses between linguistic knowledge tests ...
Comments