Elsevier

Entertainment Computing

Volume 5, Issue 4, December 2014, Pages 441-449
Entertainment Computing

A methodological approach to identifying and quantifying video game difficulty factors

https://doi.org/10.1016/j.entcom.2014.08.004Get rights and content

Highlights

  • Simulate gameplay between Pac-Man and ghosts using a variety of parameter values.

  • Examine changes to game parameters and the impact on performance variables.

  • Use a full-factorial analysis to identify factor effects and interaction levels.

  • Summarize factors influence on the performance results and player’s strategy.

  • Summarize benefits and limitations of the methodology to evaluate game difficulty.

Abstract

Video games which are unable to identify and adjust the level of difficulty to the player’s ability have no mechanism to engage a player who is beginning to become bored or overly frustrated with their gaming experience. Players who are overly bored or frustrated by a game might completely abandon the game; as most players purchase games on the recommendations of other players, this can damage the commercial side of the video game (Bateman and Boon, 2006). Our research formalizes and evaluates a methodology for identifying and quantifying the effects of different game factors on a player’s performance. To evaluate our methodology, we have designed an experimental testbed based on the classic video game Pac-Man. The evaluation of player performance is subdivided into response variables based on collecting items, the overall score, and the ability to remain alive. In the testbed, the behavior of Pac-Man and the ghosts is simulated, and every parameter and algorithm within the testbed is capable of being adjusted. Our experiment simulates game sessions for Pac-Man and the ghosts, observing the impact of adjusting the different game factors on the player’s performance. The impact of a game factor and its interaction with other factors is evaluated using a full factorial analysis. The results of our analysis identify factors with varying degrees of impact on the player’s performance. In doing so, these results identify a small sets of game factors which have a prominent impact on nearly all players, in addition to a unique sets of factors which impact only specific player types.

Introduction

Properly identifying the ideal level of game difficulty for an individual is critically important to the success of the game, as it would allow adjustments for new players to slowly engage in the game at their own pace, and could increase the difficulty quickly for more experienced players. Correctly matching the difficulty of a game to the player’s abilities does not guarantee it will not be abandoned by the player, as a player may reject a video game for other reasons; such as a lack of interest in the story or the genre. However, the level of difficulty of a video game is consistently listed as one of the most important aspects for it to be considered engaging [2], [3].

The importance of dynamically identifying the difficulty of a game and adapting the challenge level to the player’s ability is becoming even more important. In recent years, the gaming community has experienced an influx of new player demographics introduced to gaming by the paradigm shift of how players can be actively engaged [1], as well as a surge in popularity of so-called casual games made more accessible through the rise of mobile, web, and social platforms for gaming. The paradigm shift towards non-standard controller systems has further promoted the use of video game systems as tools to support other research areas such education and rehabilitation. As the user demographics of video games expands, so too will the range of player abilities and entertainment needs. Players will have varying levels of experience with interfacing with technology as well as varying skill levels in terms of personal characteristics such as reaction times, hand eye coordination, and tolerance for failure. The variation in players’ abilities will increase the difficulty for game designers to sort players into the usual static and preset difficulty settings of easy, medium and hard.

As the applications of video games continue to broaden into other research areas and new demographics of users, the importance of having customized learning, training, and feedback could become critical to the success of the application. To overcome the complexity and effort involved in designing for a significant number and variety of players, there is a growing need to more thoroughly understand game difficulty, especially the impact of various factors and design decisions on difficulty and outcomes. To form this understanding, experimental methods are required to collect and analyze the necessary data in a rigorous fashion. This, unfortunately, is a daunting task for any game of reasonable size and complexity. Consequently, the goal of our current work is to formalize an approach to studying game difficulty, in particular identifying prominent factors and determining their impact on player experience. This approach has applications to offline analysis during production to support game balancing, level tuning, and issues with playability and usability. Online applications for adaptive game systems include determining suitable factors and the granularity of adjustments necessary to optimize the player’s experience. In this paper, we present our approach, based on a full factorial analysis methodology, and demonstrate its usefulness through applying an experimental assessment to a testbed version of the classic game Pac-Man.

Section snippets

Related work

At a high level, an adaptive gaming system includes the following components: player type database, player preference modeler, game performance monitor, and a component to adjust game parameters [4]. The adaptive game system consistently monitors the player’s performance then makes adjustments to the game based on the individual’s preferences and player type models. After adjusting the game based on the player’s individual needs, the adaptive game system continues monitoring to identify if the

Methodology for game difficulty assessment

In designing our experimental methodology, the design focus was on developing an adaptive game system that is relatively independent of game choice and had the flexibility to adjust the player’s attributes, the level design, or attributes of the NPCs. The system design was intended to account for the potential use cases of game designers; thus, additional emphasis was placed on the system design to be a modular component allowing game designers to quickly integrate this methodology into any

Application of our approach

Pac-Man is a classic 2D game; the object of the game is to navigate through a fully visible maze and collect all of the tokens. While in the maze, four ghosts attempt to stop Pac-Man from collecting all of the tokens; the ghosts capture Pac-Man when at least one of the ghosts occupy the same square as Pac-Man. Pac-Man and the ghosts move one square at a time. Ghosts attempt to eat Pac-Man when in predator mode; if Pac-Man collects a power-pellet, the balance of power is shifted and Pac-Man

Conducting experimentation

In each simulated game, Pac-Man has 3 lives and is unable to gain additional lives via points or bonus items. Due to the fact that our experiment continuously runs using the same level, if the player exceeded 350 steps, that particular life would be halted. This restriction helped normalize situations where Pac-Man was significantly better than the opponents or where Pac-Man is unable to complete the level but remains alive. The experiment consists of four cases; one for each algorithm pair,

Experimental results

Presenting the massive volume of data collected and organized from the simulations would be verbose and would detract from the experiment goals of evaluating the application of our methodology to the domain of gaming and dynamic difficulty. Instead, we focus the discussion of our results on the main effects of selected cases from the experiments; in-depth results can be reviewed in [18]. For SSS_FLOCK, we selected the fruit frequency at a low level, as the two cases have nearly identical

Conclusions and future work

Our research investigated a methodology to identify game factors that altered the simulated player’s performance on a set of response variables and the difficulty of the game. Understanding the relationship between game factors and their impact on level of difficulty and thus the player’s performance is the first step towards customizing gameplay to improve the player’s emotional investment and experience. The methodology used in this experiment examines factors from the three main types of

References (18)

  • C. Bateman et al.

    21st Century Game Design

    (2006)
  • L. Ermi, F. Mäyrä, Fundamental Components of the Gameplay Experience: Analysing Immersion, Changing Views: Worlds in...
  • P. Sweetser et al.

    GameFlow: a model for evaluating player enjoyment in games

    Comput. Entertainment (CIE)

    (2005)
  • B. Cowley et al.

    User-system-experience model for user centered design in computer games

    Lect. Notes Comput. Sci.

    (2006)
  • S. Miller, Auto-Dynamic Difficulty, Published in Scott Miller Game Matters Blog...
  • J. Togelius et al.

    Towards automatic personalised content creation for racing games

    Comput. Intell. Games

    (2007)
  • J. Togelius et al.

    Evolving controllers for simulated car racing

    Evol. Comput.

    (2005)
  • G. Andrade et al.

    Challenge-sensitive action selection: an application to game balancing

    Intell. Agent Technol.

    (2005)
  • G. Yannakakis, J. Hallam, A Generic Approach for Generating Interesting Interactive Pac-Man Opponents, IEEE Symposium...
There are more references available in the full text version of this article.

Cited by (0)

This paper has been recommended for acceptance by Prof. Matthias Rauterberg.

View full text