Activities with Educational Robotics: Research Model and Tools for Evaluation of Progress

Abstract

The use of robots in the learning process has been popular since S. Papert developed his LOGO Turtle idea and argued that students can construct their own knowledge, test their constructive solutions and be motivated to learn if they use robotics in the learning process. Today, the idea of using elements of robotics in the learning process is no longer new and innovative but there are still elements that can be developed and issues that should be discussed. In this chapter, the authors provide the research model and five research tools (structured observation protocol, evaluation of the possible risks of early school leaving to be filled in by teachers before and after activities, students’ questionnaires to be filled in before and after activities) for evaluating the outcomes of organized after-school robotics activities. The research model and tools were tested and approbated with students who are at risk of early school leaving and students who participate in robotics activities to develop computational thinking.

Appendices

Appendix 1

EVALUATION INSTRUMENT for detecting students who are at risk of ESL

EVALUATION INSTRUMENT

General information about student

Student ____________________________________________ (code of the student)

____ (age)

Subject/s you teach for particular student _________________________________

☐ subject teacher ☐ class coordinator/class teacher ☐ robotics teacher

Average evaluation grade for the 1st term for all subjects _______

Three learning subjects with LOWEST grade for the 1st term (starting from the lowest):	average mark	Three learning subjects with HIGHEST grade for the 1st term (starting from the highest):	average grade
1.		1.
2.		2.
3.		3.

Missed lessons 3 months before starting participation in activities	Number of missed lessons
1. Excused by doctor
2. Excused for other reasons (parents etc.)
3. Without an excuse

Difficulties which influence learning Mark with ‘X’ if the statement characterizes the student
Has learning difficulties connected with reading
Has learning difficulties connected with calculation
Has difficulties in understanding graphs and schemes
Has other different learning difficulties
Has other special needs
Has attention concentration problems
Attitude to learning Please evaluate these statements about the student on a scale of 1−5, where 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always
Completes homework
Cooperates with teachers in a positive way
Cooperates with classmates during lessons in a positive way
Is ready for work in lessons
Understands the connection between learning and achievements
Is ready to do extra assignments to improve achievements
Obeys behavioural rules in classroom
Is ready to join out-of-class/school activities together with other classmates
Is involved in sport/art activities not connected with learning at school
Knows that the learning is important for him/her
Problem-solving skills Please evaluate these statements about the student on a scale of 1−5, where 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always
Solves the learning problems by himself/herself
Asks for help from teachers
Solves conflicts in a calm way
Motivation Please evaluate these statements about the student on a scale of 1−5, where 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always
Is motivated to learn the subject you teach
Is motivated to understand his/her mistakes to correct them
Is motivated to improve achievements
Is motivated to overcome difficulties in learning
Has an aim and works to achieve it
Observed problems Please evaluate these statements about the student on a scale of 1−5, where 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always
Is late for the beginning of lessons
Has problematic behaviour during recess (break)
Is aggressive to other students
Is aggressive to teachers
Uses rude language with classmates
Uses rude language with teachers
Refuses doing assignments during the lessons
In situation of conflict reacts aggressively

Appendix 2

Hello! You are going to learn how to work with Robots! Congratulations! It is fun! Before starting learning, can you answer some questions? Your responses are anonymous!

• 1. I am:

  ☐ boy ☐ girl

  I am ______years old (your age)

  2. Learning and achievements Please evaluate these statements on a scale of 1−5, where 1 = completely disagree, 2 = rarely agree, 3 = sometimes agree, 4 = mostly agree, 5 = completely agree
  3.1. Learning is fun
  3.2. My achievements depend on my learning
  3.3. I do all the homework
  3.4. I like to cooperate with my classmates in lessons
  3.5. I like to work individually to do assignments
  3.6. I like to do extra assignments
  3.7. I like it when there are different activities in lessons
  3.8. I like it when I can do something active in lessons
  3.9. I like to solve learning problems by myself
  3.10. I like to look for extra information needed for learning
  3. If you miss lessons it happens because Please evaluate these statements on a scale of 1−5, where 1 = always, 2 = often, 3 = sometimes, 4 = rarely, 5 = never
  4.1. I was sick or had an appointment with the doctor
  4.2. I had to participate in another activity – sports, art, etc.
  4.3. I had to help my parents
  4.4. It was an unpredictable situation
  4.5. I had to work
  4.6. I had to babysit my sister/brother
  4.7. I did not want to go to school
  4.8. I overslept
  4.9. I did not want to meet my classmates
  4.10. I do not like learning

• 5. Please write here the three learning subjects you like the most

  • 5.1. _______________________________________

  • 5.2. _______________________________________

  • 5.3. _______________________________________

• 6. Please write here the three learning subjects you like the least

  • 6.1. _______________________________________

  • 6.2. _______________________________________

  • 6.3. _______________________________________

Appendix 3

Protocol of observation

Teacher_________________________

Legend: 0 = can’t be observed, 1 = low level, 2 = can be observed almost in all situations, 3 = can be observed during all projects, 4 = does more than expected

Appendix 4

Dear Teacher,

Please assess the changes in the attitude of the student who has participated in the activities. Use the same code for the student as was used before the activities. This survey is very important. It will take approximately 5 min to fill in the questionnaire. Thank you in advance for your time.

Student _______________________________________________ (code)

Gender_____

Subject/s you teach _________________________________________________

Attitude to learning (statements are the same as in the first questionnaire, but evaluation is based on changes in the student’s attitude) Please evaluate these statements about the student’s attitude on a scale of 1−5, where: 1 = no changes at all 2 = some signs of improvement observed occasionally/rarely 3 = some signs of improvement observed sometimes 4 = signs of improvement observed in most situations 5 = strong improvement observed in all situations
Preparation of homework
Cooperation with teachers in a positive way
Cooperation with classmates during lessons in a positive way
Readiness for work in lessons
Understanding of the connection between learning and achievements
Readiness to do extra assignments to improve achievements
Following of the behavioural rules in the classroom
Readiness to join out-of-class/school activities together with other classmates
Readiness to join activities led by other classmates
Readiness to reach learning aims
Motivation (statements are the same as in the first questionnaire, but evaluation is based on changes in the student’s motivation) Please evaluate these statements about the student’s motivation on a scale of 1−5, where: 1 = no changes at all 2 = some signs of improvement observed occasionally/rarely 3 = some signs of improvement observed sometimes 4 = signs of improvement observed in most situations 5 = strong improvement observed in all situations
Motivation to learn the subject you teach
Motivation to understand his/her mistakes to correct them
Motivation to improve achievements
Motivation to overcome difficulties in learning
Readiness to work hard to achieve the aim
Observed problems (statements are the same as in the first questionnaire, but evaluation is based on changes in the student’s behaviour) Please evaluate these statements about the student’s behaviour on a scale of 1-5, where: 1 = no changes at all 2 = some signs of positive improvement observed occasionally/rarely 3 = some signs of positive improvement observed sometimes 4 = signs of positive improvement observed in most situations 5 = strong positive improvement observed in all situations
Being late for the beginning of lessons
Problematic behaviour during recess (break)
Aggressiveness to other students
Aggressiveness to teachers
Using rude language with classmates
Using rude language with teachers
Refuses to do assignments during the lessons
Aggressive reaction in situations of conflict
Problem-solving skills Please evaluate these statements about the student on a scale of 1−5, where 1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always
Solves the learning problems by himself/herself
Asks for help from teachers
Solves the conflicts in a calm way

Thank you!

Appendix 5

You had a wonderful opportunity to learn how to work with robotics. We hope you enjoyed that! Can you answer some questions about your experience? Your responses are anonymous!

• 1. I am:

  ☐ boy ☐ girl

• 2. I am ______years old (your age)

• 3. Which robotics activities did you like most? Please name at least three of them

____________________________________________

____________________________________________

____________________________________________

____________________________________________

• 4. Which robotics activities were most challenging? Please name them

  ____________________________________________

  ____________________________________________

  ____________________________________________

  ____________________________________________

  5. Learning with robots Please evaluate these statements on a scale of 1−5, where 1 = completely disagree, 2 = rarely agree, 3 = sometimes agree, 4 = mostly agree, 5 = completely agree
  5.1. Learning by using robots was fun
  5.2. I have learned how to program robots
  5.3. I liked to work in groups to do assignments with robots
  5.4. I liked to make calculations for programming
  5.5. I can use this knowledge in other activities
  5.6. I liked to solve problems with programming by myself
  5.7. I liked that others helped me to solve problems with programming
  5.8. I liked to look for extra information needed for using robots
  5.9. Other outcome (please name it)
  6. Activities with robots helped me to improve my: Please evaluate these statements on a scale of 1−5, where 1 = completely disagree, 2 = rarely agree, 3 = somehow agree, 4 = mostly agree, 5 = completely agree
  6.1. understanding of maths
  6.2. understanding of physics
  6.3. understanding of informatics and technologies
  6.4 attitude to learning
  6.5. cooperation skills with my classmates
  6.6. cooperation skills with teachers
  6.7. other outcome (please name it)

• 7. Please write here three learning subjects where your learning outcomes improved

  • 7.1. _______________________________________

  • 7.2. _______________________________________

  • 7.3. _______________________________________