Abstract
Finding effectively open educational resources and open courseware that are the most relevant and that have the best quality for a specific user’s need, in a particular context, becomes more and more demanding. Hence, even though teachers and learners (enrolled students or self-learners as well) get to a greater extent support in finding the right educational resources, they still cannot rely on support for evaluating their quality and relevance, and, therefore, there is a stringent need for effective search and discovery tools that are able to locate high quality educational resources. We propose here a multi-agent system for evaluation and classification of open educational resources and open courseware (called MASECO) based on our socio-constructivist quality model. MASECO supports learners and instructors in their quest for the most appropriate educational resource that fulfills properly their educational needs in a given context. Faculty, educational institutions, developers, and quality assurance experts may also benefit from using it.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Atkins D., Seely Brown J., Hammonds A.: A review of the open educational resources (OER) movement: achievements, challenges, and new opportunities. www.hewlett.org/uploads/files/Hewlett_OER_report.pdf (2007)
OECD: Giving knowledge for free—the emergence of open educational resources (2007)
Hylén, J.: Open educational resources: opportunities and challenges, pp. 49–63, Utah State University, Logan, UT. www.oecd.org/edu/ceri/37351085.pdf (2006)
Kernohan, D., Thomas A: Open educational resources—a historical perspective. http://repository.jisc.ac.uk/4915/
Albright, P.: Discussion highlights, pp. 61-83. In: D’Antoni, S. (ed.) OERs—conversations in cyberspace, UNESCO, Paris. www.col.org/SiteCollectionDocuments/country…/OER_Full_Book.pdf (2009)
UNESCO: Paris OER declaration. http://www.unesco.org/pv_obj_cache/pv_obj_id_EEF3C7E6694B8B91C31F5EA3340D484EF03A0100/filename/Paris%20OER%20Declaration_01.pdf (2012)
Vlădoiu, M.: Quality criteria for open courseware and open educational resources. In: 11th International Conference on Web based Learning 2012 (ICWL 2012), Workshops—2nd International Symposium on Knowledge Management and E-Learning (KMEL 2012), LNCS 7697, Sinaia, Romania (2012)
Vladoiu, M., Constantinescu, Z.: Evaluation and comparison of three open courseware based on quality criteria. In: Grossniklaus, M., Wimmer, M. (eds.) 12th International Conference on Web Engineering 2012 (ICWE 2012) Workshops—3rd Workshop on Quality in Web Engineering 2012 (QWE 2012), LNCS vol. 7703, pp. 204–215. Springer, Heidelberg (2012)
Vlădoiu, M.: Towards assessing quality of open courseware. In: 11th International Conference on Web Based Learning 2012 (ICWL 2012) Workshops—2nd International Workshop on Creative Collaboration through Supportive Technologies in Education (CCSTED 2012), LNCS 7697, Sinaia, Romania (2012)
Bloom, B.S., Englehart, M.D., Furst, E.J., Hill, W.H., Krathwohl, D.R.: Taxonomy of Educational Objectives: The Classification of Educational Goals. David McKay, New York (1956)
Custard, M., Sumner, T.: Using machine learning to support quality judgments. D-Lib Mag. 11(10), 1082. http://www.dlib.org/dlib/october05/custard/10custard.html
Nikoi, S., Armellini, A.: The OER mix in higher education: purpose, process, product, and policy. Distance Educ. 33(2), 165–184 (2012)
Wiley, D.: On the sustainability of open educational resource initiatives in higher education, OECD-CERI. www.oecd.org/edu/ceri/38645447.pdf (2007)
Geser, G.: Open educational practices and resources. OLCOS Roadmap 2012 Recommendations, Salzburg Research, EduMedia Group. Salzburg. http://www.olcos.org/cms/upload/docs/olcos_roadmap_recommendations.pdf (2007)
Geser, G.: Open educational practices and resources. OLCOS Roadmap 2012, Revista de Universidad y Sociedad del Conoscimiento 4(1), 4–13 (2007)
Downes, S.: Models for sustainable open educational resources. Interdisc. J. Knowl. Learn. Objects 3, 29–44 (www.oecd.org/edu/ceri/36781698.pdf) (2007)
Bethard, S., Wetzer, P., Butcher, K., Martin, J. H., Sumner, T.: Automatically characterizing resource quality for educational digital libraries. In: 9th ACM/IEEE-CS Joint Conference on Digital Libraries, Austin, TX, USA (2009)
Pawlowski, J.M., Hoel, T.: Towards a global policy for open educational resources: the Paris OER Declaration and its implications, White Paper, Version 0.2, Jyväskylä, Finland (2012)
Larsen, K., Vincent-Lancrin, S.: The impact of the ICT on tertiary education: advances and promises. In: Kahin, B., Foray, D. (eds.) Advancing Knowledge and the Knowledge Economy. MIT Press, Cambridge (2006)
Taylor, P.: Quality and web-based learning objects: towards a more constructive dialogue, in Quality Conversations. In: 25th HERDSA Annual Conference, Perth, Western Australia, pp. 655–662 (2002)
Littlejohn, A.: Reusing Online Resources: A Sustainable Approach to E-Learning. Routledge, London (2003)
Richter, T., McPherson, M.: Open educational resources: education for the world. Distance Educ. 33(2), 201–219 (2012)
UNESCO: Forum on the impact of open courseware for higher education in developing countries (final report), Paris. http://www.wcet.info/resources/publications/unescofinalreport.pdf (2002)
Camilleri, A.F., Ehlers, U.D., Conole, G.: Mainstreaming OEP—recommendations for policy, OPAL Consortium. http://www.oer-quality.org/publications/project-deliverables/ (2011)
MERLOT: http://www.merlot.org
NLN Materials: www.nln.ac.uk
Kelty, C.M., Burrus, C.S., Baraniuk, R.G.: Peer review anew: three principles and a case study in postpublication quality assurance. Proc. IEEE 96(6), 1000–1011 (2008)
Connexions: http://cnx.org
Rosewell, J., Ferreira, G.: QA in open educational resources (OER): open access to quality teaching resources. European Seminar on QA in e-learning, UNESCO, Paris. http://www.slideshare.net/J.P.Rosewell/qa-in-elearning-and-open-educational-resourcesoer-8398956 (2011)
Devedzic, V.: Education and the semantic web. Int. J. Artif. Intell. Educ. 14, 39–65 (2004)
Johnson, W.L., Rickel, J., Lester, J.C.: Animated pedagogical agents: face-to-face interaction in interactive learning environments. Int. J. Artif. Intell. Educ. 11, 47–78 (2000)
Hassan, S., Mihalcea, R.: Learning to identify educational materials. In: Conference in Recent Advances in Natural Language Processing, pp. 123–127, Borovets, Bulgaria(2009)
Hassan, S., Mihalcea, R.: Learning to identify educational materials. ACM Trans. Speech Lang. Process. 8(2), 2–18 (2011)
Meyer, M., Hannappel, A., Rensing, C., Steinmetz, R.: Automatic classification of didactic functions of e-learning resources. In: 15th International Conference on Multimedia’07 (MM’07), pp. 513–516, Augsburg, Germany (2007)
Meder, N.: Didaktische Ontologien. Globalisierung und Wissensorganisation: Neue Aspekte für Wissen,Wissenschaft und Informationssysteme, 401–416 (2000)
Sanz-Rodriguez, J., Dodero Beardo, J., Sánchez-Alonso, S.: Ascertaining the relevance of open educational resources by integrating various quality indicators, RUSC—Revista de Universidad y Sociedad del Conocimiento, 8(2), 211–224 (2011)
Han, K.: Quality rating of learning objects using Bayesian belief networks. PhD thesis, Simon Fraser University , Canada (2004)
Nesbit, J.C., Li, J.Z., Leacock, T.L.: Web-based tools for collaborative evaluation of learning resources. J. Systemics Cybern. Informatics, 3(5) (http://www.iiisci.org/journal/sci/Contents.asp?var=&previous=ISS2829) (2005)
Burgos Aguilar, J.V.: Rubrics to evaluate OERs. www.temoa.info/sites/default/files/OER_Rubrics_0.pdf (2011)
Kumar, V., et al.: Quality rating and recommendation of learning objects. In: Pierre, S. (ed.) E-Learning Networked Environments and Architectures—A Knowledge Processing Perspective, pp. 337–373. Springer, London (2007)
ACHIEVE: http://www.achieve.org
OER Commons: http://www.oercommons.org
Learning Registry: http://www.learningregistry.org
MIT OCW: http://ocw.mit.edu/index.htm
OCW Consortium: http://www.ocwconsortium.org
The Saylor Foundation: http://www.saylor.org
University of Washington Courses: http://www.cs.washington.edu/education/courses
Coursera: https://www.coursera.org
Webcast.Berkeley: http://webcast.berkeley.edu
Universia: http://ocw.universia.net/es
ParisTech Libres Savoirs: http://graduateschool.paristech.fr
Open.Michigan: http://open.umich.edu
OCW University of California, Irvine: http://ocw.uci.edu/courses/index.aspx
OCW University of Southern Queensland: http://ocw.usq.edu.au
OCW Utah State University: http://ocw.usu.edu
Textbook Revolution: http://textbookrevolution.org/index.php/Main_Page
Google custom OER/OCW Search: http://www.google.com/cse/home?cx=000793406067725335231%3Afm2ncznoswy
OCW search, http://www.ocwsearch.com
MIT OpenCourseWare on Database Systems: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-830-database-systems-fall-2010
Saylor Foundation’s Introduction to Modern Database Systems open courseware: http://www.saylor.org/courses/cs403
Stanford’s Professor Jennifer Widom Introduction to Databases open courseware: https://www.coursera.org/course/db
Introduction to Database Systems courseware, Nguyen Kim Anh, in Connexions: http://cnx.org/content/m28135/latest/
King Fahd University’s KFUPM OpenCourseWare on Database Systems: http://ocw.kfupm.edu.sa/BrowseCourse.aspx?dname=Info.+%26+Computer+Science&did=ICS&cid=ICS324
University of Washington’s Introduction to Data Management open courseware: http://www.cs.washington.edu/education/courses/cse344/12au
Universidad Charlos III de Madrid’s Database Fundamentals: http://ocw.uc3m.es/ingenieria-informatica/fundamentos-de-las-bases-de-datos
Universidad Politecnica de Madrid’s Database Administration: http://ocw.upm.es/lenguajes-y-sistemas-informaticos/administracion-de-bases-de-datos
Vladoiu, M.: State-of-the-art in open courseware initiatives worldwide. Informatics Educ. 10(2), 271–294 (2011)
Vladoiu, M.: Open courseware initiatives—after 10 years. In: 10th International Conference Romanian Educational Network—RoEduNet, pp. 183–188. IEEE Press, Iasi (2011)
Brockbank, A., McGill, I.: Facilitating Reflective Learning in Higher Education. SRHE and Open University Press, Buckingham (1998)
Light, G., Cox, R.: Learning and Teaching in Higher Education. The Reflective Professional. Paul Chapman Publishing, London (2001)
Loughran, J.J.: Developing Reflective Practice. Learning About Teaching and Learning Through Modelling. Falmer Press, London (1996)
Schunk, D.H., Zimmerman, B.J.: Self-regulated Learning—from Teaching to Self-reflective Practice. Guilford Press, New York (1998)
Panait, L., Luke, S.: Cooperative multi-agent learning: the state of the art. Auton. Agents Multi-Agent Syst. 11(3), 387–434 (2005)
Mitchell, T.M.: Machine Learning. McGraw-Hill Higher Education, New York (1997)
Sebastiani, F.: Machine learning in automated text categorization. ACM Comput. Surv. 34(1), 1–47 (2002)
Rao, A.S., Georgeff, M.P.: Modeling rational agents within a BDI-architecture. In: Fikes, R., Sandewall, E. (eds.) Knowledge Representation and Reasoning (KR&R-91), pp. 473–484. Morgan Kaufmann Publishers, San Mateo (1991)
Rao, A.S., Georgeff, M.P.: BDI agents: from theory to practice. In: International Conference on Multi-Agent Systems (ICMAS-95), San Francisco, USA (1995)
Müller, J.P.: The Design of Intelligent Agents: A Layered Approach, LNCS, vol. 1177: Lecture notes in artificial intelligence. Springer, Berlin (1996)
Moise, G.: A software system for online learning applied in the field of computer science. Int. J. Comput. Commun. Control II(1), 84–93 (2007)
Wooldridge, M., Jennings, N.: Intelligent agents: theory and practice. Knowl. Eng. Rev. 10(2), 115–152 (1995)
Sycara, K.P.: Multiagent systems. AI Mag. 19(2), 79–92 (1998)
Young T.Y., Calvert, T.W.: Classification, Estimation, and Pattern Recognition. American Elsevier Pub. Co., Amsterdam (1974)
Archer, N.P., Wang, S.: Application of the back propagation neural network algorithm with monotonicity constraints for two-group classification problems. Decis. Sci. 24(1), 60–75 (1993)
Guobin, O., Yi, LuM: Multi-class pattern classification using artificial neural networks. Pattern Recogn. 40, 4–18 (2007)
McCulloch, W.S., Pitts, W.: A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys. 5(4), 115–133 (1943)
Rosenblatt, F.: The perceptron: a probabilistic model for information storage and organization in the brain. Psychol. Rev. 65(6), 386–408 (1958)
Zilouchian, A.: Intelligent control systems using soft computing methodologies. CRC Press, Boca Raton (2001)
Moise, G., Netedu, L., Toader, F.A.: Bio-inspired E-learning systems. A simulation case: english language teaching. In: Pontes, I., Silva, A., Guelfi, A., Takeo Kofuji, S. (eds.) Methodologies, Tools and New Developments for E-Learning, p. 14. InTech, Rijeka (2012)
Pearl, J.: Probabilistic reasoning in intelligent systems: networks of plausible inference. Morgan Kaufmann, San Mateo (1988)
Bayes, T., Price, R.: An essay towards solving a problem in the doctrine of chance. Philos. Trans. R. Soc. London 53, 370–418. www.stat.ucla.edu/history/essay.pdf (1763)
Friedman, N., Geiger, D., Goldszmidt, M.: Bayesian network classifiers. Mach. Learn. 29, 131–163 (1997)
Baesens, B., et al.: Bayesian network classifiers for identifying the slope of the customer lifecycle of long-life customers. EJOR 156(2), 508–523 (2004)
Minsky, M.: Steps toward artificial intelligence. Proc. Inst. Radio Eng. 49(1), 8–30 (1961)
Piedra, N., Chicaiza, J., Tovar, E., Martinez, O.: Open educational practices and resources based on social software: UTPL experience. In: 9th IEEE International Conference on Advanced Learning Technologies—ICALT 2009, pp. 497–498, Riga (2009)
Acknowledgments
The authors are very grateful to both the editors and the anonymous reviewers for their valuable comments and ideas to improve this chapter.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix: The Quality Scores Obtained by the Eight Open Courseware on Databases
Appendix: The Quality Scores Obtained by the Eight Open Courseware on Databases
1 MIT OCWDB | 2 Saylor DB | 3 St WidDB | 4 Cnx NKA | 5 KF DBSs | 6 UW DMg344 | 7 UC3M DADB | 8 UPM BD | |
|---|---|---|---|---|---|---|---|---|
CR1 | 2.5 | 2.5 | 5 | 5 | 5 | 3.5 | 5 | 3 |
CR2 | 2.5 | 2.5 | 5 | 5 | 5 | 5 | 4 | 4 |
CR3 | 5 | 5 | 3 | 5 | 5 | 5 | 4 | 4 |
CR4 | 4 | 5 | 3 | 5 | 5 | 4 | 4 | 5 |
CR5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR6 | 3 | 5 | 5 | 1 | 3 | 3 | 5 | 4 |
CR7 | 3 | 5 | 5 | 1 | 3 | 3 | 5 | 5 |
CR8 | 5 | 5 | 5 | 3 | 5 | 3 | 5 | 5 |
CR9 | 2 | 5 | 2 | 2 | 0 | 3.5 | 2 | 3.5 |
CR10.1 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR10.2 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR10.3 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR10.4 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR10.5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR10.6 | 0 | 5 | 5 | 0 | 0 | 0 | 0 | 0 |
CR10.7 | 0 | 5 | 0 | 0 | 0 | 0 | 0 | 0 |
CR10.8 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CR10.9 | 5 | 1 | 1 | 1 | 0 | 5 | 5 | 1 |
CR10.10 | 0 | 5 | 5 | 0 | 0 | 0 | 5 | 0 |
ID1 | 1 | 5 | 0.5 | 2.5 | 3.5 | 1 | 4 | 4 |
ID2 | 1 | 5 | 0 | 1 | 1 | 0 | 1 | 1 |
ID3 | 3 | 5 | 5 | 3 | 3 | 3 | 3.75 | 3 |
ID4 | 2.5 | 5 | 5 | 1 | 0 | 5 | 2 | 2.5 |
ID5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
ID6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
ID7 | 0 | 5 | 5 | 0 | 0 | 1 | 0 | 0 |
TR1 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
TR2 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
TR3 | 2.5 | 2.5 | 2.5 | 4 | 2.5 | 2.5 | 2.5 | 2.5 |
TR4 | 2 | 3 | 2 | 5 | 2 | 2 | 2 | 2 |
TR5 | 5 | 5 | 5 | 5 | 5 | 0 | 0 | 0 |
TR6 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
TR7 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
TR8 | 5 | 5 | 5 | 5 | 0 | 5 | 2 | 0 |
CW1.1 | 4 | 5 | 4 | 2 | 5 | 4 | 5 | 4 |
CW1.2 | 4 | 5 | 4 | 0 | 5 | 4 | 5 | 4 |
CW1.3 | 5 | 4 | 0 | 0 | 0 | 0 | 1 | 3 |
CW1.4 | 5 | 4 | 0 | 0 | 5 | 0 | 0 | 3 |
CW1.5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
CW1.6 | 5 | 2.5 | 5 | 2.5 | 4.75 | 4.75 | 2.5 | 2.5 |
CW1.7 | 5 | 5 | 5 | 3 | 3 | 5 | 5 | 5 |
CW1.8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
CW1.9 | 0 | 0 | 2 | 0 | 0 | 0 | 0 | 0 |
CW1.10 | 0 | 2 | 2 | 0 | 0 | 2 | 0 | 0 |
CW1.11 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CW1.12 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CW1.13 | 0 | 5 | 5 | 0 | 0 | 0 | 0 | 0 |
CW1.14 | 0 | 5 | 0 | 0 | 0 | 0 | 0 | 0 |
CW1.15 | 2 | 5 | 5 | 2 | 2 | 2 | 0 | 2 |
CW1.16 | 4 | 5 | 5 | 0 | 0 | 0 | 0 | 0 |
CW1.17 | 5 | 5 | 5 | 4 | 3 | 5 | 5 | 5 |
CW1.18 | 1 | 5 | 1 | 1 | 0 | 5 | 2 | 2 |
CW2 | 5 | 5 | 5 | 0 | 0 | 0 | 5 | 0 |
CW3 | 5 | 5 | 5 | 0 | 0 | 0 | 0 | 0 |
CW4 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CW5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 0 |
CW6 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CW7 | 2 | 5 | 5 | 2 | 2 | 2 | 2 | 2 |
CW8.1 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
CW8.2 | 2 | 5 | 4 | 3.75 | 2 | 2 | 2 | 2 |
CW8.3 | 0 | 5 | 3 | 0 | 0 | 0 | 0 | 0 |
CW8.4 | 0 | 5 | 1 | 0 | 0 | 0 | 0 | 0 |
CW8.5 | 2 | 5 | 2 | 5 | 2 | 2 | 2 | 2 |
CW9 | 0 | 5 | 5 | 0 | 0 | 0 | 0 | 0 |
CW10.1 | 0 | 2 | 0 | 3 | 0 | 0 | 0 | 0 |
CW10.2 | 2 | 5 | 5 | 5 | 2 | 0 | 0 | 0 |
CW10.3 | 0 | 3 | 3 | 0 | 0 | 0 | 0 | 0 |
CW10.4 | 0 | 3 | 5 | 4 | 0 | 0 | 0 | 0 |
CW10.5 | 0 | 3 | 5 | 4 | 0 | 0 | 0 | 0 |
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Moise, G., Vladoiu, M., Constantinescu, Z. (2014). MASECO: A Multi-agent System for Evaluation and Classification of OERs and OCW Based on Quality Criteria. In: Ivanović, M., Jain, L. (eds) E-Learning Paradigms and Applications. Studies in Computational Intelligence, vol 528. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41965-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-41965-2_7
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41964-5
Online ISBN: 978-3-642-41965-2
eBook Packages: EngineeringEngineering (R0)