1 Introduction and Motivation

It is quite important to know learners’ perceptions of Computer Science (CS) concepts to plan and design CS courses. But even though the special role that students’ perspectives play for the design and arrangement of CS lessons and courses has been pointed out in the model of educational reconstruction for Computer Science Education (CSE) in 2012 [1], there is still a lot of work to do in this field, since perceptions of many CS concepts are not investigated yet.

But how do researchers get information on students’ conceptions usually? The common methods (cf. Sect. 1.2) range from questionnaires to structured interviews, which are two ways bringing their own disadvantages like the lack of response or the wanting flexibility. The Repertory Grid MethodFootnote 1 (RGM) combines their advantages: The assessors are allowed to use their own vocabulary; thus a comparability of various Repertory Grid Interviews is warranted by providing the elements of the grid (cf. Sect. 2). It is also possible to flexibly focus on interesting points during the interviews using the RGM.

We chose to study students’ conceptions of the Internet since there is already a great amount of information on how children and teenagers think the Internet works (cf. Sects. 1.2 and 3.3). This enables us to discuss whether or not this method is capable of studying learners’ perceptions because we are able to compare our results to previous studies (cf. Sect. 3.3). We introduce the basics and the possibilities of the RGM and briefly present the design and the results of our study in the following, to finally discuss the suitability of this technique for the purpose of elaborating learners’ perceptions.

1.1 Kelly’s Personal Construct Psychology

The Personal Construct Psychology (PCP) came into existence as an answer to the critique on the decisive psychological concepts in the 1940’s and 1950’s. Its founder, the American psychologist Kelly, presented his theory in his major work “The psychology of Personal Constructs” for the first time in 1955 [2]. One of his main points is that we as human beings develop our own theories and “come to understand the world […] by erecting a personally organized system of […] constructs of experienced events” [3]. He argues “that we make sense out of our world by simultaneously noting likenesses and differences” [4]. The systematically organized constructs then are “our way of distinguishing similarity from difference” [3]. “They are linked, related and integrated into a complex hierarchical structure or system containing many sub-systems. Through our system of […] constructs we are able to predict and control our […] world. Thus they are our guidelines for living” [3].

The Repertory Grid Method (RGM) that is introduced in the following Sect. 2 allows us “to elicit personal constructs and examine the relationships between them within a specific domain” [3].

1.2 Related Work and Our Goals

Most researchers get information on learners’ perceptions of CS concepts using standardized or (semi-)structured Interviews [5,6,7] or questionnaires  [8, 9]. Only little work in this field is making use of more ‘exotic’ methods like drawings e.g. [9, 10]. A common approach to get information on the way learners view on CS is through the methodology of Grounded Theory [11,12,13].

However, in this paper we want to introduce the Repertory Grid Method (RGM), which has already been used in a few didactic studies and investigations of childrens’ perceptions successfully. Lengnink and Prediger applied the method in a qualitative evaluation study of a pre-service teacher course “to explore teacher students’ individual conceptions about learning and teaching mathematics” [14] for example. Williams used the technique “to describe the informal models of the limit concept held by two college calculus students” [15]. Baxter, Jack and Schröder used the RGM “to elicit constructs from 23 children aged between 8 and 11 years of age relating to their perceptions of eight common vegetables” [16].

Although the work mentioned represents only an excerpt of the available didactic publications that utilize the RGM, we do not know of any authors that applied it to the domain of CSE yet. This leads us to the aim of our study, which is mainly not to confirm learners’ views already carved out by other authors, but to look into the suitability of the RGM for this purpose and to encourage this qualitative approach in this field.

2 Methodology

2.1 Origins of the Repertory Grid Technique

The most common Personal Construct Psychology (PCP) method is the “Role Construct Repertory Test”. It is no standardized test with a set procedure in its usual sense, but a structured interview [3]. In the 1980’s the RGM was increasingly growing in popularity after 30 years of existence [3]. The benefits that result from the grid interviews providing structured data despite being flexible and open for the personal constructs of the interviewees “encouraged many people [of the most different fields of work] to design and develop their own applications” [17]. Thereby the basic idea of Repertory Tests is to “elicit personal constructs and examine the relationships between them within a specific domain” [3], by structuring the conversation.

Since Kelly’s basic work, the evolution of the RGM generated a multitude of different variations [3]. Nonetheless, this thoroughly suits Kelly’s mind: In his opinion the RGM is an open and variable method. However, all the Repertory Grid Interviews have in common that the interviewees are asked to differentiate between objects of their experience (elements) and that these differentiations (constructs) are then recorded in a data grid. Within the single procedures the preparation, post-processing and evaluation vary considerably [18].

2.2 Modifications of the Technique

As already mentioned there is no particular RGM. However, all the corresponding studies share the fact that the subjects are asked to differentiate between various elements. Thereby these – mostly up to twelveFootnote 2 [17] – elements can range from important persons within the radius of the interviewed person to material things such as chocolate bars, “flowers in the garden, the meals of the day or other aspects of life that are important to the person” [4]. So there is no restriction in the choice of elements. Furthermore, the elements can be obtained by different strategies: They are either elaborated together with the interviewee during the conversation or completely provided by the investigator. Even a mixed strategy, where some of the elements or a pool or area of interests is provided and others are generated by the interviewee, is possible [3, 17].

Groups of elements are chosen by the interviewer and provided to the interviewee to gain the so called constructs [3]. While it is the proband’s task to think about how the presented elements are similar to and different from one another, the attributes describing similarities and differences are denominating the constructs.

The most common method of gaining constructs is the so called triad method, since a construct at least has to be build out of one similarity relation between two elements and a dissimilarity relation between those two elements and the third one according to Kelly’s dichotomy corollary [2].

“But since one is eliciting constructs already established in the person’s repertoire there is no reason why three elements need to be used” [4]. So, the dyad method, where only two elements are given when comparing them, often is applied when working with children or other persons that are overstrained with the task of comparing groups of three elements [4].

The combinations of elements to be compared are usually gathered by randomly drawing two or three elements. Therefore, the elements need to be homogeneous, so that it would not carry weight, if single elements did appear in none of the triads, while others were consulted more often [17]. During the Repertory Grid Interview, the interviewer usefully ensures that the elements occur equally. Furthermore, it is even possible to decide “which combinations will bring out the greatest contrast in the elements available” [17].

As for the elements, even for the constructs “quite a literature has developed over the question whether or not supplied or provided constructs give the same answers as do elicited constructs” [4]. But since it is our goal to detect students’ perceptions of IT topics we will not go into detail of this alternative.

2.3 Analysis and Interpretation of Repertory Grids and the Importance of Explanations During Repertory Grid Interviews

Easterby-Smith points out “that the interpretation of grid data is very much an art and not a technology” [17]. It is also a common misconception “that it cannot be analyzed adequately without a computer” [17]. While he refers to correlation matrices as the most common technique for manual analysis, he reminds that computer analysis “does not add anything to the information available in a grid, nor does it provide any indication of the meaning of a grid; it simply reduces the amount of work required for interpretation by summarizing and condensing the data available” [17]. An interesting alternative to factor or cluster analysis was used by Lengnink and Prediger: They visualized the structures of their grids in line diagrams to make the logical structures explicit [14]. We wanted to briefly name these two possibilities, because we applied them in our study (cf. Sect. 3.2).

Originally the RGM was thought of as a structured interview, in which the interviewees shall provide information about their subjective meanings of the reviewed elements [18]. Not only Kelly but also the authors of the following decades completely lose sight of the detailed conversation that needs to develop when elaborating personal constructs [18], though he must have been aware of the interview situation of this technique right from the beginning. Kelly almost by the way mentions a crucial requirement when it comes to understanding people’s personal constructs: the explanations of the terms used by the subjects. But the only thing you learn from Kelly about how such explanations look like and how they come off is that referring to this somehow the interviewers’ skills matter [18]. While it might have been completely self-evident to him, that a simple retrieval of terms belonging together on its own has nothing to do with understanding other people’s personal constructs, many scientists worked off such lists of terms, looked at that as an application of the Repertory Grid Technique, and published their work after him [18]. Instead of that it is essential to attend to the explanations, comments and statements the subjects make while elaborating constructs and to consult these explanations in the analysis of the interviews.

2.4 Developing the Repertory Grid Interviews for this Particular CSE Study

At the beginning of the interviews for this study the elements were provided by the investigator to guarantee a comparability of the interviews among each other. If a set of elements is provided, it is important to ensure that these elements “provide representative coverage of the area to be investigated” [17]. Once the Repertory Grid Technique is used to gain information about learners’ views on the Internet it means that the elements have to conform to the framework of terms that are needed to describe the functionality of the Internet scientifically. At the same time these elements should represent the phenomena and artifacts that teenagers associate the Internet with. Considering these aspects, the following twelve elements were chosen for this Repertory Grid study: the Internet, (my) computer, server, web page, IP address, e-mail, cable, Google, data, router, (data) packet, (chat) application. Before the actual interviews started these elements were printed on index cards to guarantee an optimal flow of the interviews.

Since this study mainly wants to examine whether the RGM is appropriate for investigating learners’ views, we decided to design two variations of the method, that differ in the way the constructs are gained:

  1. 1.

    In the first variation the order of comparisons of elements (e.g. first compare the triad (my) computer, cable, the Internet then compare the triad router, IP address, server and so on) is determined.

  2. 2.

    In the second variation the subject is asked to sort the index cards first of all, so that elements belonging together lie on one pile. After that the proband has to assemble commonalities in the single piles, which then set the constructs. While the construct poles result directly from the descriptions of commonalities in the elements of the single piles, the contrast poles then have to be asked for (“What is the opposite of this characteristic?”) in a second step.

3 Results

3.1 Execution of this Investigation

Within the framework of this study five male students at eleven to thirteen years that participated in a voluntary “robotics” study group at a northern German secondary school were interviewed. Because of the students’ voluntary participation in the study group we can assume an over-average interest in topics around computers and their functionalities. In addition to that all of them mention their daily usage of the Internet for mainly social (social media) or entertaining (online games) purposes.

Variation 1 was conducted by three students while two students participated in Repertory Grid Interviews based on variation 2 (cf. Sect. 2.4). At the beginning of the interviews an introductory example on chocolate bars, which was inspired by a market researchFootnote 3 and impressively shows the variety of possible applications of the technique, was worked through with the interviewees each time. Thereby, the sample was adjusted according to the actual variation of the procedure. The individual interviews lasted between 30 and 50 min. During the process of planning the interviews the possibility of switching from the triad to the dyad method was retained, as long as the relevant interviewee is overstrained with the task of comparing three elements (cf. Sect. 2.3).

All of the interviews were recorded to be able to go into substantial statements of the subjects during the evaluation of the interviews. Figure 1 shows an exemplary Repertory Grid as it was worked out during one of the interviews that were led in the context of this study.

Fig. 1.
figure 1

An exemplary Repertory Grid as it was worked out during one of the interviews that were led in the context of this study. This particular Repertory Grid comes from variation 1 of the method. The dots indicate the triad or pair of elements that led to the particular construct. Students’ quotes were translated from German.

Full size image

3.2 Evaluating the Repertory Grid Interviews

The participants of this investigation were actively prompted to deal with their cognitive organization structures by finding similarities and differences between the provided elements. Thereby the explanations that were vehemently called for provide the basis of the learners’ views on the Internet that were identified in this study. However, the actual Repertory Grids might move into background as each interview progressed, though the organization structures of the grids are still reflected in the students’ explanations. In order to evaluate the interviews, we listened back to the audio tracks, transcribed meaningful passages and then wrote down the mentioned perceptions while keeping an eye on the original Repertory Grids.

We created correlation matrices and line diagrams from the Repertory Grids but saw no mentionable advantage. Especially the advantage of line diagrams only seems to work, if the subjects are directly confronted with them during the interviews.

3.3 A Comparison of the Perceptions of the Internet We Won in this Study and the Results of Other Works

Comparing the single interviews with one another we found out that everyone of the five participants of this study holds concise misconceptions of servers, IP addresses and Google. In this paper we want to focus on these three aspects and summarize them briefly due to the lack of space.

Servers:

The subjects of our study were of the mind that servers generally are not touchable and that they are something purely virtual. They consider servers as a possibility to play online games with friends and they are completely not aware of them being the central elements of the Internet. Correspondingly the model of a single, central computer that runs the whole Internet has already been worked out in an older study [5]. One of our participant’s visualization of servers as “huge hay bales” of cables with numerous crossroads is similar to a model that was also elaborated in the mentioned paper: In this particular model markers on the Internet show where to go next [5].

IP addresses:

While some of the participants of this study believe that Google is responsible for finding the IP address belonging to the query, others consider that IP addresses are used in combination with self-selected passwords to log in to various websites. There is reason to suspect that the former simply mix up IP and URL addresses or that they even use both abbreviations synonymously. The latter likely mix IP addresses up with user names. Again, parallels between the results of this study and previous ones can be found: It has already been attested that the majority of pupils indeed is sensible of the need for a one-time address on the Internet although many students believe that such an addressing could be realized by proper (user) names  [5].

Google:

60% of the subjects explained that Google is the first thing you get to automatically when connecting to the Internet. Furthermore, it is quite striking that two students even are of the opinion that Google would provide the IP addresses of other websites. Another student explains, that he gets to other websites by putting their names into Google’s search field. These are results that seem to be plausible since they have been worked out by others, too [6].

One basic similarity is the functional, application-oriented view on the Internet that all of the interviewed students share: They mainly describe what you can do on the Internet and more scarcely wherefore or even how. Furthermore, the impression unfolds that all of the five students obviously never asked themselves how single processes on the Internet work and that they just as recently as they were asked to find similarities and differences during the Repertory Grid Interviews developed their own theories to this. Again, this fact has been elaborated by Seifert et al. [6] before.

4 Discussion

The results of this study mainly were not surprising and we could confirm many perceptions that other authors already worked out using common techniques like interviews. In addition to that some conceptions were elaborated that have not been studied by other authors yet but seem to be conclusive. Basically it can already be recapped at this point that Repertory Grids are definitely suitable for gaining information on learners’ views on CS concepts. In the following Sect. 4.1 we want to point out the advantages and disadvantages we saw in using the RGM for our purpose.

4.1 Advantages of Using the Repertory Grid Method Instead of Other Instruments

The interviewees were permanently asked to wonder about their thoughts on their subjective organization structures because of the demanded comparisons of the elements. In this regard it would be interesting to look into the method when it is adapted in a way that two persons can be interviewed at once. Then two students had to respond to each other to come to an agreement about the characteristics of the elements. But basically the individual interviews made it possible to gain insights into the world of ideas of the students, by asking them to explain their choices. We do not recommend using the most original variation of the RGM to gain information on learners’ views after our pre-tests though (cf. Sect. 2.3) because letting the subjects elaborate the elements by themselves by naming things they connect with the Internet complicates the later comparisons probably due to a lack of comparability. Both of the variations that were used in this study (cf. Sect. 2.4) have their own advantages and disadvantages: Thus the “motivational barrier” that results from the task of sorting the elements to piles of similar ones is a disadvantage of variation 2, since both students that took part in this variation of the study stopped carving out constructs after they completed each pile by only naming one characteristic shared by elements of the pile. However, it seemed as if the task of sorting piles and afterwards finding characteristics (variation 2) was a lot easier for the children than variation 1, where they had to compare triads that were given by the interviewer. The initial classification of elements belonging together (as it is the case with variation 2) results in a structural order that is not manipulated by a previously determined order of elements (as is the case with variation 1). Using the first variation it is in fact possible for students to avoid elements being compared with one another, which prevents the researcher from drawing conclusions about students’ perceptions. But you cannot completely ignore the fact that the interviewees are challenged with the task of comparing elements they are unfamiliar with, which is indeed the huge disadvantage of variation 1. Accordingly, two of three students that participated in variation 1 of the study were overstrained being asked to compare triads of elements. When working with children it is obviously a lot easier to simply ask them to find similarities, which provides variation 2 with an advantage over variation 1. However, the task had to be formulated a lot more precisely, so that the interviewees do not carve out only as many constructs as they find piles of elements. Practically, the task could be expressed as sorting the cards several times according to various criteria.

Finally, the Repertory Grid Interviews enabled us to get a detailed insight into the learners’ perceptions of the CS concepts investigated, by carving out similarities and differences between the elements. In conclusion Repertory Grids are suitable for qualitative analysis of students’ perspectives indeed. The only disadvantage that we saw was the difficulty of finding meaningful terms for the constructs. The students had no problems working out similarities and differences at all, but they had a hard time to describe these constructs with short terms for the grid. And that is why the explanations during the Repertory Grid interviews are the important basis for the perception analysis. For the evaluation process, the grids originally worked out move into background.

5 Conclusion

It has been this study’s aim to adapt the RGM with the objective of investigating learners’ views on the functionality of the Internet and then to discuss the effectiveness of this method for this particular application purpose. The students’ perceptions that were elaborated in this study and mainly have also been worked out by the authors of previous investigations already prove that it is actually possible to use this method for qualitative research on students’ perceptions. However, this method is as time-consuming as the more original research methods (cf. Sect. 1.2) and the evaluation also is as tedious. Thus there is no advantage regarding the duration of execution and analysis. In addition to that most of the times it is hard for the subject and the interviewer to come to an agreement about expressing the constructs as precisely yet concisely as possible. Therefore, it is often hard to find out what the subject wanted to express by only looking at the constructs written on the log sheet. That is why it is so important to record the interviews on tape to be able to refer to decisive explanations that turned out to be the crucial source of information in this study when it comes to elaborate perceptions.

Finally the RGM does not necessarily enjoy such great popularity because of the possibility to perform numerous calculations on grids or to visualize relationships in line diagrams. It is rather the researchers’ task to work out similarities and differences that forces participants to actively deal with their perceptions and concepts and to express them properly. We wanted to present our way of designing Repertory Grid Interviews to study learners’ views on the functionality of the Internet and hopefully encourage you to use the RGM as a skeleton for the interviews of your future plans on studying learners’ conceptions. For us the method worked out as a promising approach.