1 Introduction

Gamification of online surveys has been proposed to make questionnaire filling a more enjoyable experience and to improve the accuracy of survey results [6, 9]. This is an important goal because online surveys have been criticized for their dullness resulting in negative respondent behavior such as speeding, random responding, premature termination, and lack of attention [9, 17, 23]. In contrast to these negative effects, evaluations of gamified surveys have reported diverse benefits regarding user experience, motivation, participation, amount and quality of data [6, 8, 9, 23]. These prior works confirm the usefulness of gamified online surveys, but have remained unclear about suitable design processes. More recent work  [13] has proposed (but not evaluated) a design process that unifies process models from the related disciplines of form design and gamification. This work employs and evaluates the process in a case study where two designers gamified a survey about sports and leisure activities amongst teenagers and young adults. The goals and contributions of this work are firstly, to document our application of the process and the resulting gamified design (Sect. 4). This will also provide qualitative results (Sect. 5) regarding the process’s applicability and usefulness. And secondly, to evaluate the psychological and behavioral outcomes of the gamified design (Sects. 6–8) in an empirical study.

2 Related Work

Gamification of online surveys builds on many disciplines [13]. The following section briefly discusses relevant backgrounds, concepts and methods.

Tradition and Innovation in Surveys.

The use of forms for surveying information has a long historical tradition dating back to the 16th century when officers in Spanish provinces were equipped with questionnaires to standardize interviewing and observations [10]. These questionnaires enabled bureaucratic processes by abstracting individual life experiences into consistent, standardized representations [3, 10]. This characteristic is shared with today’s digital forms and online surveys, albeit with a different purpose of enabling automated data processing. Understanding the history of online surveys provides ample opportunity for innovation, as demonstrated by related work that has linked today’s forms with their historic predecessors in order to derive research goals for form design [12]. The goal of this work can be described accordingly: Gamification of online surveys seeks to avoid negative historical entailments of the ‘form’ UI metaphor (in particular, the connotations that forms are bureaucratic and dull [9]) by adding interactive game elements to the survey.

Form design.

The discipline of form design is highly relevant to survey gamification because online surveys typically employ form-based UIs to enable data entry. Related work has captured best practices for form design in guidelines  [2] and books  [16, 28]. Relevant aspects have been structured into three layers of a form design process [16]. In the relationship layer, designers analyze the relationship with users, their tasks, and the usage context. In the conversation layer, designers seek to create interactions that make the conversations between users and the survey flow easily. The appearance layer describes detailed UI and graphical design.

Gamification.

Gamification has been defined as “the use of design elements characteristic for games in non-game contexts” [7]. In this definition, the “design elements characteristic for games” can more shortly be termed game elements. The MDA framework  [15] provides a way to understand game elements as MDAs, i.e., as either game mechanics, dynamics, or aesthetics. Mechanics describe the basic building blocks (data representations, algorithms, rules, interactive elements) that make up a game. Dynamics refer to the resulting run-time behavior over time. Aesthetics characterize a player’s emotional response and experience. The “non-game contexts” include business, education, health, many more listed in  [11, 14], and online surveys, as examined in this work.

Gamified Online Surveys.

As a potential benefit, gamification provides motivational affordances that produce psychological (e.g., user experience, emotion, fun) and behavioral (e.g., participation, performance) outcomes [11]. In Fig. 1, we additionally included a priori factors such as context of use, user characteristics, and affect because these have been shown to significantly influence the outcomes of gamification [11]. Related work has aimed at exploring possible designs for gamified surveys and at evaluating their impact [6, 8, 9, 23]. E.g., four designs have been compared in [9]: text-only, decoratively visual, functionally visual, and fully gamified. Evaluations have reported beneficial psychological outcomes such as a better user experience  [8, 9] and increased motivation  [6]. Beneficial behavioral outcomes have included more participation and engagement  [6, 8], more feedback  [23], and better data quality  [8]. Despite these experienced benefits, not all gamified surveys have produced significantly positive results  [9]. Furthermore, a recent literature review has shown benefits to be strongly influenced by users and context [11]. There is also a lack of comparisons of the required effort and subsequent benefits of specific game elements  [11]. This calls for future studies to clearly describe the influence of survey domain and target user group(s), the game elements provided as motivational affordances, and the effort that was required for designing and implementing the gamified survey.

Fig. 1.
figure 1

Gamification provides game elements as motivational affordances to produce psychological and behavioral outcomes  [11, 13]. These outcomes are influenced by a priori factors such as context, tasks, user characteristics, and affect [11].

Full size image

3 A Design Process for Gamifying Online Surveys

To design a gamified online survey in this work, we chose to follow the process originally proposed in [13]. This process integrates and unifies the MDA (mechanics-dynamics-aesthetics) gamification framework [15, 26] and the three layers of form design [16], applying them to the various survey areas, as visualized in Fig. 2a. In addition to prior work, this work contributes a more detailed description of the process, complemented with examples (this Section), and a qualitative evaluation (Sect. 5).

Fig. 2.
figure 2

Design process for gamifying online surveys. The process (a) unifies the disciplines of gamification and form design and applies them to the survey areas to be gamified, as originally proposed in  [13]. Its iterations (b) follow the steps proposed in the MDA framework  [15, 26] and in the “three layers of form design”  [16].

Full size image

1. Game Elements for Inspiration.

Prior to starting with the gamification process, designers should collect game elements that can inspire their further design activities [13]. As a quick and easy starting point, they can use the pre-compiled catalogues of game elements suited for survey gamification from [23, 27]. Further game elements (not all of them necessarily suited for surveys) are provided in the “gamification toolkit” [26], “ingredients of great games” [24], game mechanics listed in [1], “motivational game design patterns” [19], “game flow criteria” [25], “playful experiences” in [18], and the aesthethics in [15]. The designers should familiarize themselves with the game elements so they can inspire the subsequent design steps.

2. Aesthetics and the Relationship Layer.

As a first step of the proposed process, designers should analyze the intended users (i.e., the survey’s target population), tasks (the form schema to be filled), and context, as described in the relationship layer of form design [16]. Based on this knowledge, they can set goals regarding intended aesthetics, i.e., the intended emotional responses and user experiences that shall be elicited by the survey. Designers may set different aesthetic goals for different survey areas (introduction, questions, answers, navigation, and submission, compare the “survey areas” dimension in Fig. 2a). Nonetheless, gamification should result in one coherent design; therefore a single process is proposed for all survey areas, see Fig. 2b. Aesthetics from the previously compiled catalogue of MDAs can serve as inspiration. Designers can rank and choose aesthetics as deemed suitable.

For example, designers may consider the aesthetics of challenge and sensation to be suited for a survey’s target users, but may deem the fellowship aesthetic unsuited for an intended single-user experience. Regarding the various survey areas, they could aim at arousing curiosity and interest in a survey’s introduction page. They could seek to provide visual and auditory sensation to enhance questions and answers, but refrain from making questions challenging to answer because perceived intellectual difficulty has been shown to adversely influence respondent behavior [17]. They could decide to design navigation with a target aesthetic of gameful exploration. The submission page could be designed to reward users for their effort. Note that the above aesthetics are provided as illustrative examples – other target aesthetics are of course possible.

3. Dynamics and the Conversation Layer.

Designers can use the MDA framework [15] to reason about which game dynamics are suited for producing the intended aesthetics. This creative thinking can be inspired by the catalogue of MDAs. Note that since game dynamics refer to the run-time behavior of a gamified system [15], the considerations in this step of the process correspond to the conversation layer of form design [16], i.e., the flow of interactions that a user is going to have with the survey.

For example, the game dynamic of time pressure has been recommended for motivating users to provide lengthy free-text answers [23], but designers should avoid creating time pressure throughout the entire survey because this could motivate users to speed. Designers may also implement feedback loops, i.e., dynamics wherein user actions affect the overall state of gameplay [15]. Feedback loops may visualize concepts such as a user’s progress, status, wealth, health, points, etc.

4. Mechanics and the Conversation and Appearance Layers.

To produce the intended dynamics and aesthetics, designers can employ suitable game mechanics and playful elements. Again, they can use the catalogue of MDAs for inspiration. Since game mechanics are the detailed building blocks and rules that make up a game  [15], this step relates to detailed design activities in the conversation and appearance layers of form design. As an overall goal, re-designed questions should still represent the construct of interest and the interactive UI elements should not bias the answers given by respondents.

For example, designers may choose to employ the mechanics of points and badges to implement the dynamic of feedback, which in turn can produce the aesthetic of challenge. They may further choose to visualize a stopwatch next to free-text fields to implement the dynamic of time pressure and the same aesthetic of challenge. They may choose to employ the avatar mechanic and allow users to freely move their avatar throughout the survey and thus produce an aesthetic of exploration.

5. Prototyping, Evaluation, and Iteration.

As typical for creative design processes [5, 21], designers should work in a team, explore multiple designs in parallel, prototype, and evaluate prototypes. The overall gamification process will typically progress from deliberate vagueness during brainstorming, ideation, and sketching (primarily in steps 2–3) to increasing detail and specifity during prototyping and evaluation (primarily in step 4). Evaluations should consider both intended outcomes for the user (e.g., subjective experience) and outcomes for those who create the survey (e.g., completion rate, truthful answers). Formative evaluations can be performed with relatively few users, using test observation methods such as thinking-aloud [22]. In the authors’ experience, paper prototyping and digital mockups have worked well in the first iteration, whereas later iterations have required digital, interactive prototypes. Three iterations have sufficed to create a pleasant design with good usability.

4 Case Study: Gamification of a Sports Survey

An existing online survey about sports and leisure activities amongst teenagers and young adultsFootnote 1 was chosen as a case study because of its beneficial characteristics: The survey’s questions are easy to understand and answer; therefore domain-specific knowledge amongst test users is unlikely to bias evaluation results. It employs state-of-the art survey design using survey-monkey’sFootnote 2 default style and functionality. Furthermore, the survey addresses children and teenagers as target population; related work has shown this target group to react well to gamification [20].

4.1 Application of the Gamification Process

Methodologically, two designers (one senior with over five years in HCI and form design, one student in HCI) employed the design process presented in this work to gamify the sports survey. In summary, the designers held three workshops and took three iterations (each including prototyping and evaluation) to work through the different phases of the process, thus converting the conventionally-designed sports survey into a gamified one. In the first workshop, they discussed the aesthetics available in the catalogue in [27] and finally chose the three aesthetics of sensation, challenge, and exploration as suitable goals for the relationship layer of form design, i.e., for the intended relationship with teenage users. More specifically, they aimed at producing a design that elicits a rich visual sensation (in contrast to typical, text-only surveys), that includes small challenges in the form of micro-games (albeit without making questions too difficult to answer because this could potentially bias results), and that allows users to freely explore and discover the various survey areas. In the next workshop, they brainstormed possible designs using the catalogue of MDAs from [27] as inspiration. Their design activities iterated rapidly between explorative, abstract thinking (i.e., which dynamics and mechanics can produce the intended aesthetics) and specific, increasingly detailed design (i.e., sketching ideas and elaborating the conversation and appearance layers of form design). Regarding dynamics, they chose to implement feedback systems about the respondent’s progress and about beneficial user actions. They further chose to implement the dynamic of time pressure to produce an aesthetic of challenge when users enter free-text answers. They sketched a design with the following mechanics: Users should steer an avatar through the survey. Feedback should be given using progress indicators and using coins as rewards for beneficial actions. In the third workshop, they produced mockups, thus addressing detailed UI design and the appearance layer of form design. An initial paper prototype was employed in an early, formative usability test and was subsequently replaced by a web-based prototypes and a final implementation.

4.2 Resulting Gamified Design

The resulting, gamified survey contains the same questions as the original sports survey, but features a novel design. The addition of many game elements resulted in a highly game-like appearance, as shown in Figs. 3 and 4.

Fig. 3.
figure 3

The gamified survey features multiple areas. Respondents (a) create an avatar, (b) freely navigate between survey areas, (c-f) play mini-games to answer questions, and (g) may buy accessories for their avatar in a shop using rewarded coins. Upon completion, they win the sports competition and are (h) rewarded a gold medal.

Full size image
Fig. 4.
figure 4

Input controls were visually decorated as shown in the above example.

Full size image

Visual Design.

The overall theme of the gamified survey was designed to reflect the survey’s topic of sports. The graphical appearance was designed to remind of jump’n’run games (such as Super Mario) that members of the target population are likely to be familiar with from their childhood. Survey elements such as input controls were graphically decorated in order to produce the intended aesthetic of sensation. For example, radio buttons were re-designed to include the respondent’s avatar along with pictures that each represent one possible answer, as shown in Fig. 4. All survey areas maintain a similar visual style but feature different interactions, as explained in the following subsections.

Avatar.

In the first survey area, an avatar is automatically assigned to each respondent. The avatar’s visual appearance depends on the demographic data that respondents provide about themselves, see Fig. 3a for an example.

Free Exploration.

The survey allows respondents to navigate freely between four sports disciplines that each represent a different survey area. Navigation is implemented through a map shown in the second survey area, see Fig. 3b. When respondents click on a sports discipline, their avatar walks to the specified place on the map and the according survey area is subsequently shown. Once they complete a survey area, they return back to the map.

Questions and Answers.

The survey areas of soccer, javelin throwing, long jump, and sprint, see Fig. 3c–f, are micro-games that each afford and require different interactions through which respondents can answer questions. For example, the soccer game (Fig. 3c) instructs respondents to perform a penalty kick by dragging and then releasing their avatar. When released, the avatar kicks the ball in the specified direction into the goal and thus selects one of two options. The other survey areas are designed in similar ways. The javelin throwing and long jump games (Fig. 3d–e) map length (of jump or throw) unto answers. The sprint game (Fig. 3f) creates time pressure by visualizing a decreasing amount of time during which respondents shall provide a maximum of free-text answers. To avoid bias through unintentionally wrong answers, each survey area provides instructions about the required interactions. Furthermore, respondents are asked to practice and then demonstrate their skill by providing a pre-specified answer before they can start answering real questions. Respondents can correct every answer before confirming it by clicking a “next”-button that leads to the next question.

Feedback Mechanisms.

Various mechanisms provide positive feedback about the respondents’ progress. While filling the survey, they are awarded coins. The map allows respondents to enter a shop (Fig. 3g) where they can buy accessories such as sunglasses and hats for their avatar. The shop has no other purpose than to strengthen positive reward. The last survey area – shown upon completion of the entire survey – was designed as a medal ceremony (Fig. 3h) where each respondent is honored as winner of a sports competition.

5 Lessons Learned About the Gamification Process

The two designers reported qualitative feedback regarding the applicability and usefulness of the gamification process, as experienced by them in the case study. Their overall opinion was positive. They both found the process served them as a helpful guideline about how to proceed with the gamification. This was especially important for one of them whose background was more in user interface design than in gamification. Both designers liked the structure provided by the process. They said they could follow the steps proposed in the process and found no need to deviate from its structure. We asked specifically if, in their experience, the structure of the process succeeded in combining the MDA-phases (aesthetics, dynamics, mechanics) with the layers of form design (relationship, conversation, appearance). They found that MDA-aesthetics fit very well with the relationship layer because both relate to setting design goals. One of them experienced the combinations of dynamics and mechanics with the conversation and appearance layers to be rather intermingled. Regarding the first step of the process, they found that using a catalogue of MDAs provided ideas and useful inspiration. They often consulted it during their design activities and wished for a more extensive catalogue. Regarding subsequent steps, they highlighted the need to carefully avoid bias. For example, they were aware that their chosen target aesthetic of challenge should not result in overly complex interactions that could bias answers. They had therefore decided to make questions easy to answer but to produce the aesthetic of challenge by designing a narration of sports competition. In a similar way, the designers reflected on their decision to include an avatar, expressing concerns that users taking on foreign roles are likely to bias survey results. They had therefore personalized the avatars based on respondent characteristics, thus communicating that the avatar represents the actual respondent, and not a fictitious role. Within this context, they stressed the need for formative evaluations and said they had discovered and fixed many usability problems through formative usability testing and subsequent design iterations. Both designers stated that iterative design and implementation of the gamified survey took a lot of time and effort – more than they had anticipated, and significantly more than the non-gamified variant, see Table 1 for a quantitative comparison. They found that – in addition to the gamification process – they could have used technical guidance and better development tools for their prototyping and implementation activities. They further suggested that future work should examine methods for reducing the implementation effort.

Table 1. Working time needed for design and implementation of the gamified vs. conventional survey. Note that since the designers gamified an already existing survey, the numbers do not include the time needed to plan and formulate survey questions.
Full size table

6 Study Design

The gamified survey’s psychological and behavioral outcomes were evaluated in a remote, comparative, between-subject usability test. Invitations were sent via Email and Facebook, asking to participate in a survey about sports activities. Since invitations could be forwarded without restriction, we had limited control over sample demographics. The invitations did not promise any extrinsic reward and did not disclose the study’s true purpose of evaluating a gamified design. Participants were unaware of the existence of two different survey designs and were automatically assigned to one of two test conditions (gamified vs. conventional design) using a round-robin algorithm. After completing the survey, respondents were asked to also fill out a post-test questionnaire. Duplicate responses were prevented through technical measures, i.e., by setting a browser cookie. Respondent behavior and answers were logged into a database. The survey was completed after 60 participants, resulting in an equal distribution of 30 participants in both the conventional and gamified survey.

7 Results

A total of N = 60 participants accessed the sports survey. The participants’ age distribution was higher than the intended target audience of the original sports survey, see Table 2. A total of 47 participants completed the survey (24 female, 23 male). A test session was considered complete if users finished the sports survey, no matter if they also filled (N = 40 out of 47) the subsequent post-test questionnaire. Quantitative results are shown in Tables 3–6, including means (M), medians (MD) and standard deviations (SD). Significant differences (p < 0.05), as tested using Mann-Whitney U-Tests, are highlighted in a bold font. Qualitative results are reported in Sect. 7.4.

Table 2. Age distribution of participants
Full size table
Table 3. Respondent behavior.
Full size table

7.1 Respondent Behavior and Engagement

Respondent behavior was automatically logged during use. The gamified survey had a lower response rate of 70 % (21 out of 30 persons), as opposed to the conventional survey with a response rate of 86 % (26 out of 30 persons). We also measured the amount of time spent in the survey and the question where participants cancelled the survey. Amongst respondents who completed the survey, those working with the gamified design spent about twice as much time (19:20 ± 04:42) in comparison to those working with the conventional design (09:18 ± 04:39), see Table 3c. Amongst those who cancelled the survey, we found no significant differences regarding the question after which participants cancelled and the time after which they quit, as shown in Table 3a and b. Besides response rate and time spent, we took an additional measure of engagement by evaluating the amount of plain-text answers that respondents were willing to provide, but found no significant difference between the gamified and conventional survey, see Table 3d. We additionally investigated if respondent behavior was influenced by the following demographic factors: gender, age, self-rated health and sportiness, county, size of city, highest education and profession, relationship status, has children, living condition (i.e., lives with parents/friends/own family). None of these factors proved to have a statistically significant influence.

7.2 Answers Given

We compared the answers given in response to the gamified versus non-gamified survey. For this purpose, all answers to the survey’s 61 closed questions were numerically coded. The answers to 4 of 61 questions were significantly different between the two survey designs (Table 4); all other questions revealed no such influence. Interestingly, all four of these questions were negatively worded Likert questions, part of large blocks of radiobuttons in the text-only survey and part of the javelin-throwing survey area in the gamified design. All four questions got higher answers (i.e., “agree more fully”) in the conventional survey. We further investigated the possibility of answers being systematically influenced by the gamified survey’s microgames. The javelin-throwing micro game did produce significantly different answers, as compared between the gamified (2.50 ± 1.072) vs. conventional (2.65 ± 1.130) design. There were no significant differences any of the other micro-games.

Table 4. Answers given. Amongst the survey’s 61 closed questions, answers to the above four questions were significantly influenced by the survey’s gamified vs. conventional design. All other questions showed no such influence.
Full size table

7.3 Self-rated User Experience

Perceived usability and user experience were assessed upon completion of the sports survey through a post-test questionnaire that included System Usability Scale (SUS) [4] questions. It was filled out by an overall number of 40 respondents (21 gamified, 19 conventional). Pair-wise comparison of individual SUS questions (see Table 5) revealed that respondents were significantly more inclined to frequently use the gamified survey (2.81 ± 0.75), compared to the conventional version (1.16 ± 1.02). However, respondents felt significantly less confident using the gamified survey (3.1 ± 0.7), compared to the conventional version (3.79 ± 0.42). There was no significant effect of survey design on any other SUS question. Overall SUS scores for both survey versions were comparable as well, with the gamified survey scoring 77.98 points and the conventional survey scoring 79.08. Answers to further questions in the post-test questionnaire (Table 6) showed that respondents found the gamified survey (3.29 ± 0.56) significantly more fun to use than the conventional survey (2.32 ± 1.01). They were also significantly more inclined to recommend the gamified survey (3.38 ± 0.67), compared to the conventional survey (2.42 ± 0.96).

Table 5. System Usability Scale (SUS) scores from the post-test questionnaire.
Full size table
Table 6. Self-rated fun and likeliness of recommending the survey.
Full size table

7.4 Qualitative Results

Qualitative comments were collected from respondents using open-ended questions in the post-test questionnaire. The comments were analyzed and grouped into structured categories (see Table 7). Of 21 respondents that finished the gamified survey, every single one answered the post-test questionnaire, while 19 of 26 respondents that finished the conventional survey answered the post-test questionnaire. Respondents were also much more inclined to provide comments (both positive and negative) for the gamified survey (N = 21 of 21) compared to the conventional survey (N = 9 of 26).

Table 7. Qualitative results. The table shows answers given to open-ended questions in the post-test questionnaire, structured into coded categories.
Full size table

Respondents positively commented on the novelty (9), variety (4) and interactivity (2) of the gamified survey. They found it playful (4) and fun (3). Graphics and animation also garnered positive comments (5), as well as the personalized and customizable avatar (4). Some respondents complained that the gamified survey took much longer to answer than a conventional survey might have taken (4). There also were complaints about the controls (4) and responsiveness of individual mini games (3). Several respondents also commented that they would have liked to continue playing after finishing the survey (4), which is an interesting complaint insofar as it highlights the heightened level of engagement and joy compared to a conventional survey.

Comments regarding the conventional survey were less varied: Respondents found the survey easy to use (5) and easy to answer (3), while complaining about vague or ambigious questions (4) and boredom (3).

8 Discussion

Results indicate that gamification successfully increased the users’ perceived fun, the average time spent, as well as their willingness to use and recommend the survey, without introducing a strong bias in the survey results, albeit with a lower overall response rate. This improvement in user experience is in line with related studies on gamified online surveys [6, 9].

Quantitative results show that respondents found the gamified survey more fun and spent significantly more time. While an increase in time spent is in itself not necessarily a sign of heightened engagement, the fact that participants also found the gamified survey more fun and voluntarily spent more time suggests that the increased duration is the result of an improved user experience. This may prove beneficial for marketing surveys that aim at exposing users to a certain brand in a pleasant way. Furthermore, the respondents’ higher willingness to use and recommend the gamified survey can be useful for viral marketing.

Given these positive outcomes, the lower overall response rate for the gamified survey was surprising and warrants further examination. Results provide two possible explanations. Firstly, higher engagement and positive feedback from those who did finish the gamified survey suggest that the gamified design may have caused polarized reactions among participants, turning those away who did not approve of the chosen design. This issue requires further examination in future work. Secondly, respondents stated they felt less confident using the gamified survey; the lack of familiarity with the novel design may have caused them to cancel. Future long-term studies are needed to investigate effects of novelty in gamified online surveys.

Overall, the gamified design barely influenced answers given by participants, as there was no significant effect of survey style for 57 of 61 questions. However, there was a significant difference for answers given in the javelin-throwing micro game that requires further examination. One interpretation is that the gamified survey successfully reduced negative respondent behavior and thus reduced bias – but the gamification may also have introduced new bias. To clarify the issue, we suggest that future work should develop automated measures of speeding, straightlining, random responding, lack of attention, conflicting and empty answers, compare  [9, 17, 23], and use these measures to quantify negative respondent behavior in gamified and conventional surveys.

The qualitative comments given by respondents reaffirm our initial expectations and motivation: That conventional surveys are often perceived as somewhat dull and boring, and that gamification is a suitable approach to make surveys more fun and engaging. Some of the comments validate specific design decisions made during the gamification process, such as the use of a customizable avatar to represent survey respondents, as well as implementation details such as graphics and animation. However, other comments demonstrate the difficulty of getting every design detail right, as demonstrated by scattered complaints about controls or the responsiveness of individual micro-games. Additionally, the abrupt ending of the gamified survey drew a large number of complaints of users who would have liked to continue playing. While an abrupt ending might be appropriate for a conventional survey, it seems inappropriate for more playful, open-ended experiences, as in our gamified survey. Comparing the comments between the gamified versus conventional survey, it becomes apparent that the gamified survey garnered both a larger number as well as more varied comments. One possible explanation for this difference in quantity and quality of respondent’s comments would be the novelty of the gamified survey raising awareness of specific survey design aspects, compared to the dull familiarity of a conventional survey spurring less reflection and comment.

9 Conclusion and Future Work

Gamification is a promising way of improving user experience and increasing engagement in online surveys. This work extends prior research by making the following two contributions.

Firstly, this work documents the successful application of a recently proposed design process for gamifying online surveys and describes the resulting design. The process was applied in a case study where two designers gamified a survey about sports and leisure activities amongst teenagers and young adults. The designers reported qualitative results supporting the practical usefulness and applicability of the process. This indicates that other survey gamification projects can benefit from the same or a similar process.

As a second contribution, the resulting gamified design was evaluated in a remote online study with 60 participants. The gamified survey achieved better psychological outcomes (respondents found the gamified survey more fun, they were more inclined to use and recommend the gamified design, and provided more, and more positive, qualitative feedback) and better behavioral outcomes (regarding engagement: respondents voluntarily spent more time in the gamified survey). These positive results are, however, accompanied by critical issues including a lower response rate in the gamified survey and possibly biased answers in one specific survey area. These issues warrant further empirical investigation.

Our future work in this area will continue in the following direction. Since a survey’s gamification takes a lot of effort, we intend to examine ways of increasing benefits (e.g., by identifying best practices and by seeking ways of improving behavioral outcomes) and of reducing the required efforts (e.g., by creating re-usable design patterns and component libraries) in order to improve the return on investment of future survey gamifications.