Abstract
Online judges are Web-based platforms where people can solve programming challenges and have their solutions automatically evaluated, in real time. They can be used for teaching, self study, or recruitment purposes. Online judges are a great resource for students in particular, as a means of practicing for algorithmic competitions, exams, and interviews. Numerous computer science departments from institutions around the world try to integrate online judges into their teaching systems, as a solution for automatic assessment. Some have even developed custom judges, and published papers which elaborate the implementation details and review the impact on students’ performance. Unfortunately, none of them clarifies the methods used to achieve language-agnostic judges. The aim of this paper is to fill in this gap, by surveying different approaches of designing a judge which supports multiple programming languages.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kurnia, A., Lim, A., Cheang, B.: Online judge. Comput. Educ. 36(4), 299–315 (2001)
Wasik, S., Antczak, M., Badura, J., Laskowski, A., Sternal, T.: A survey on online judge systems and their applications. ACM Comput. Surv. (CSUR) 51(1), 1–34 (2018)
Cheang, B., Kurnia, A., Lim, A., Oon, W.C.: On automated grading of programming assignments in an academic institution. Comput. Educ. 41(2), 121–131 (2003)
Douce, C., Livingstone, D., Orwell, J.: Automatic test-based assessment of programming: a review. J. Educ. Resour. Comput. (JERIC) 5(3), 4–es (2005)
Brito, M., Gonçalves, C.: Codeflex: a web-based platform for competitive programming. In: 2019 14th Iberian Conference on Information Systems and Technologies (CISTI), pp. 1–6. IEEE (2019)
Petit, J., Roura, S., Carmona, J., Cortadella, J., Duch, J., Gimnez, O., Mani, A., Mas, J., Rodrguez-Carbonell, E., Rubio, E., et al.: Jutge. org: Characteristics and experiences. IEEE Trans. Learn. Technol. 11(3), 321–333 (2017)
Leetcode. https://leetcode.com/. Last accessed 10 Jan 2020
Hackerrank. https://www.hackerrank.com/dashboard. Last accessed 10 Jan 2020
Codechef. https://www.codechef.com/. Last accessed 10 Jan 2020
Hackerearth. https://help.hackerearth.com/hc/en-us. Last accessed 10 Jan 2020
Codeforces. https://codeforces.com/. Last accessed 10 Jan 2020
Topcoder. https://www.topcoder.com/. Last accessed 10 Jan 2020
Jutge.org. https://jutge.org/. Last accessed 10 Jan 2020
Verdú, E., Regueras, L.M., Verdú, M.J., Leal, J.P., de Castro, J.P., Queirós, R.: A distributed system for learning programming on-line. Comput. Educ. 58(1), 1–10 (2012)
Kaya, M., Özel, S.A.: Integrating an online compiler and a plagiarism detection tool into the moodle distance education system for easy assessment of programming assignments. Comput. Appl. Eng. Educ. 23(3), 363–373 (2015)
Mu, L.: gLua: A modern Lua transpiler in scheme (2019)
Langlois, B., Jitia, C.E., Jouenne, E.: Dsl classification. In: OOPSLA 7th Workshop on Domain Specific Modeling (2007)
Czarnecki, K., O’Donnell, J.T., Striegnitz, J., Taha, W.: Dsl implementation in metaocaml, template haskell, and c++. In: Domain-Specific Program Generation, pp. 51–72. Springer (2004)
Humer, C., Wimmer, C., Wirth, C., Wöß, A., Würthinger, T.: A domain-specific language for building self-optimizing ast interpreters. In: Proceedings of the 2014 International Conference on Generative Programming: Concepts and Experiences, pp. 123–132 (2014)
Freeman, S., Pryce, N.: Evolving an embedded domain-specific language in java. In: Companion to the 21st ACM SIGPLAN Symposium on Object-Ooriented Programming Systems, Languages, and Applications, pp. 855–865 (2006)
Hackerrank domain specific language documentation. https://help.hackerrank.com/hc/en-us/articles/360008561014-HackerRank-DSL-Domain-Specific-Language-Documentation. Last accessed 26 Jan 2020
Voelter, M., Kolb, B., Birken, K., Tomassetti, F., Alff, P., Wiart, L., Wortmann, A., Nordmann, A.: Using language workbenches and domain-specific languages for safety-critical software development. Softw. Syst. Model. 18(4), 2507–2530 (2019)
Lambdachecker. https://lambdachecker.com/. Last accessed 17 May 2020
Acknowledgements
This survey was conducted in order to assist the development of LambdaChecker, an online judge which strives to go beyond simple test-based grading. We would like to thank all who contributed to project: Stefan Balas and Catalin Dragutescu, for implementing the prototype as their graduation project; Andrei Medar, Serban Ciofu and Gabriel Tuculina, who are currently working to improve the user experience for both teachers and students; George Milescu and his colleagues at AWS for their valuable advice.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tica, IT., Olteanu, AC., Racec, E. (2021). Supporting Multiple Programming Languages in an Online Judge. In: Mealha, Ó., Rehm, M., Rebedea, T. (eds) Ludic, Co-design and Tools Supporting Smart Learning Ecosystems and Smart Education. Smart Innovation, Systems and Technologies, vol 197. Springer, Singapore. https://doi.org/10.1007/978-981-15-7383-5_21
Download citation
DOI: https://doi.org/10.1007/978-981-15-7383-5_21
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-7382-8
Online ISBN: 978-981-15-7383-5
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)