Skip to main content
SpringerLink
Log in

Standard Setting in a Formative Assessment of Digital Responsibility Among Norwegian Eighth Graders

  • Chapter
  • First Online:
Standard Setting in Education

Part of the book series: Methodology of Educational Measurement and Assessment ((MEMA))

  • 999 Accesses

Abstract

Students’ digital responsibility is an important topic in a digital society. Since 2016, a learning supportive 50-item test in digital responsibility is available for Norwegian eighth-graders. Rasch model was used to select tasks to the test. Our study addresses experiences from using two standard setting methods, Angoff and bookmark, to set the cut-off scores and to define the proficiency levels of digital responsibility. In this process, as this is a learning supportive assessment, the decision makers had to take into consideration both the implied expectations from teachers who would use the assessment in their classroom practices and the results of psychometric analyses. A sample test with 1026 students from 26 schools was used to define three proficiency levels. The use of two standard setting methods, Angoff and bookmark, gave different results, and this difference addresses uncertainty about where to set the cut-off score. What are the validity arguments for considering different expectations when setting cut scores?

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  • Afzar, A. (2006). A systems theoretical critique of international comparisons. Norsk pedagogisk tidsskrift, 27, 253–264.

    Google Scholar 

  • Ainley, J., Fraillon, J., Freeman, C. (2007). National assessment program: ICT literacy years 6 & 10 report 2005. Australia: MCEETYA.

    Google Scholar 

  • Ala-Mutka, K. (2011). Mapping digital competence: Towards a conceptual understanding. Luxembourg: European Union.

    Google Scholar 

  • Angoff, W. A. (1971). Scales, norms, and equivalent scores. In R. I. Thorndike (Ed.), Educational measurement (2nd ed., pp. 508–600). Washington, DC: American Council of Education.

    Google Scholar 

  • Binkley, M., Erstad, O., Herman, J., Raizen, S., Ripley, M., & Rumble, M. (2012). Defining 21st century skills. In P. Griffin, B. McGaw, & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 17–66). Dordrecht: Springer. doi:10.1007/978-94-007-2324-5.

    Chapter  Google Scholar 

  • Black, P. J., & Wiliam, D. (1998). Inside the black box: Raising standards through classroom assessment. London: King’s College.

    Google Scholar 

  • Calvani, A., Fini, A., Ranieri, M., & Picci, P. (2012). Are young generations in secondary school digitally competent? A study on Italian teenagers. Computer & Education, 58, 797–807.

    Article  Google Scholar 

  • Cizek, G. J. (Ed.). (2012a). Setting performance standards: Concepts, methods and perspectives. Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Cizek, G. J. (2012b). An introduction to contemporary standard setting: Concepts, characteristics, and contexts. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 3–14). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Cizek, G. J., & Bunch, M. B. (2007). Standard setting. Thousand Oaks: Sage.

    Book  Google Scholar 

  • Claro, M., Preiss, D. D., San Martín, E., Jara, I., Hinostroza, J. E., Valenzuela, S., et al. (2012). Assessment of 21st century ICT skills in Chile: Test design and results from high school level students. Computers & Education, 59, 1042–1053.

    Article  Google Scholar 

  • Crocker, L., & Algina, J. (2008). Introduction to classical & modern test theory. New York: Holt, Rinehart, and Winston.

    Google Scholar 

  • Deunk, M. I., van Kuijk, M. F., & Bosker, R. J. (2014). The effect on small group discussion on cut-off scores during standard setting. Applied Measurement in Education, 27, 77–97.

    Article  Google Scholar 

  • Educational Testing Service (ETS). (2001). Digital transformation. A framework for ICT literacy. A report of the international ICT literacy panel. www.ets.org/Media/Tests/Information_and_Communication _Technology_Literacy/ictreport.pdf. Accessed 15 April 2016.

  • Embretson, S. E., & Reise, S. P. (2009). Item response theory for psychologists. Mahwah: L. Erlbaum Associates.

    Google Scholar 

  • Ferrari, A. (2012). Digital competence in practice: An analysis of frameworks. JRC technical reports. Seville: European Commission.

    Google Scholar 

  • Ferrari, A. (2013). DIGCOMP: A framework for developing and understanding digital competence in Europe. Luxembourg. http://ipts.jrc.ec.europa.eu/publications/pub.cfm?id=6359. Accessed 15 April 2016.

  • Fraillon, J., et al. (2014). Preparing for life in a digital age: The IEA International Computer and Educational Literacy Study International Report. Switzerland: Springer.

    Book  Google Scholar 

  • Hambleton, R. K. (2001). Setting performance standards on educational assessments and criteria for evaluating the process. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 89–116). Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Hatlevik, O. E., Egeberg, G., Guðmundsdóttir, G. B., Loftsgarden, M., & Loi, M. (2013). Monitor skole 2013 - Om digital kompetanse og erfaringer med bruk av IKT i skolen Læring for framtiden. Oslo: Senter for IKT i utdanningen.

    Google Scholar 

  • Hatlevik, O. E., Ottestad, G., & Throndsen, I. (2015). Predictors of digital comp. in 7th grade: Students’ motivation, family background, and culture for professional development in schools. Journal of Computer Assisted Learning, 31(3), 220–231.

    Article  Google Scholar 

  • Hsieh, M. (2013). Comparing yes/no Angoff and bookmark standard setting methods in the context of English assessment. Language Assessment Quarterly, 10, 331–350.

    Article  Google Scholar 

  • International Society for Technology in Education. (2007). ISTE standards students. https://www.iste.org/docs/pdfs/20-14_ISTE_Standards-S_PDF.pdf. Accessed 15 Apr 2016.

  • Jaeger, R. M. (1989). Certification of student competence. In R. L. Linn (Ed.), Educational measurement (3rd ed.). Washington, DC: American Council on Education.

    Google Scholar 

  • Kane, M. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64(3), 425–461.

    Article  Google Scholar 

  • Kim, H. S., Kil, H. J., & Shin, A. (2014). An analysis of variables affecting the ICT literacy level of Korean elementary school students. Computers & Education, 77, 29–38.

    Article  Google Scholar 

  • Krumsvik, R. (2011). Digital competence in Norwegian teacher education and schools. Högre utbilding, 1, 39–51.

    Google Scholar 

  • Kuhlmeier, H. A., & Hemker, B. (2007). The impact of computer use at home on students’ internet skills. Computer & Education, 49, 460–480.

    Article  Google Scholar 

  • Matzat, U., & Sadowski, B. (2012). Does the “do-it-yourself approach” reduce digital inequality? Evidence of self-learning of digital skills. The Information Society, 28(1), 1–12.

    Article  Google Scholar 

  • Mediatilsynet. (2014). Barn og media 2014. Barn og unges (9-16 år) bruk og opplevelse av media. Fredrikstad: Mediatilsynet.

    Google Scholar 

  • Mehrens, W. A., & Cizek, G. J. (2012). Setting standards for decision making: Classifications, consequences, and the common good. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 33–46). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Mitzel, H. C., Lewis, D. M., Patz, R. J., & Green, D. R. (2001). The bookmark procedure: Psychological perspectives. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 249–281). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • OECD. (2011). OECD reviews of evaluation and assessment in education: Norway. Paris: OECD Publishing.

    Google Scholar 

  • Pant, H. A., Rupp, A. A., Tiffin-Richards, S. P., & Köller, O. (2009). Validity issues in standard setting studies. Studies in Educational Evaluation, 35, 95–101.

    Article  Google Scholar 

  • Plake, B. S., & Cizek, G. J. (2012). Variations on a theme: the modified Angoff, extended Angoff, and yes/no standard setting methods. In G. J. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives (pp. 181–199). Mahwah: Lawrence Erlbaum.

    Google Scholar 

  • Sadler, D. R. (2009). Grade integrity and the representation of academic achievement. Studies in Higher Education, 34(7), 807–826. doi:10.1080/03075070802706553.

    Article  Google Scholar 

  • Søby, M. (2013). Learning to be: Developing and understanding digital competence. Nordic Journal of Digital Literacy, 8(03), 134–138.

    Google Scholar 

  • The Norwegian Directorate for Education and Training. (2012). Framework for basic skills. Oslo: The Norwegian Directorate for Education and Training.

    Google Scholar 

  • Tiffin-Richards, S. P., & Pant, H. A. (2013). Setting standards for English foreign language assessment: Methodology, validation, and a degree of arbitrariness. Educational Measurement: Issues and Practice, 32(2), 15–25.

    Article  Google Scholar 

  • Tveit, S. (2014). Educational assessment in Norway. Assessment in Education: Principles, Policy & Practice, 21(2), 221–237.

    Article  Google Scholar 

  • Zhong, Z. J. (2011). From access to usage: The divide of self-reported digital skills among adolescents. Computers & Education, 56, 736–746.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Teacher Education and School Research, The University of Oslo, Oslo, Norway

    Ove Edvard Hatlevik

  2. Skills Norway, Oslo, Norway

    Ingrid Radtke

Authors
  1. Ove Edvard Hatlevik
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ingrid Radtke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ove Edvard Hatlevik .

Editor information

Editors and Affiliations

  1. Centre for Educational Measurement at the University of Oslo (CEMO), Oslo, Norway

    Sigrid Blömeke

  2. Department of Education and Special Education, University of Gothenburg, Gothenburg, Sweden

    Jan-Eric Gustafsson

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Hatlevik, O.E., Radtke, I. (2017). Standard Setting in a Formative Assessment of Digital Responsibility Among Norwegian Eighth Graders. In: Blömeke, S., Gustafsson, JE. (eds) Standard Setting in Education. Methodology of Educational Measurement and Assessment. Springer, Cham. https://doi.org/10.1007/978-3-319-50856-6_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-50856-6_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-50855-9

  • Online ISBN: 978-3-319-50856-6

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation