Elsevier

Food Quality and Preference

Volume 22, Issue 6, September 2011, Pages 507-520
Food Quality and Preference

Sort and beer: Everything you wanted to know about the sorting task but did not dare to ask

https://doi.org/10.1016/j.foodqual.2011.02.004Get rights and content

Abstract

In industries, the sensory characteristics of products are key points to control. The method commonly used to characterize and describe products is the conventional profile. This very efficient method requires a lot of time to train assessors and to teach them how to quantify the sensory characteristics of interest. Over the last few years, other faster and less restricting methods have been developed, such as free choice profile, flash profile, projective mapping or sorting tasks. Among these methods, the sorting task has recently become quite popular in sensory evaluation because of its simplicity: it only requires assessors to make groups of products perceived as similar. Previous studies have shown that this method produces sensory spaces similar to those obtained with conventional profiles but that the descriptions of the products are coarser than the descriptions yielded by sensory profiles. The aim of the present paper is to further evaluate the efficiency of the sorting task as a sensory tool. We present a series of studies highlighting the advantages and delineating the limits of the sorting task and illustrate advantages and limits using beer as the common type of stimuli. These studies underline the main issues encountered when designing sorting tasks. More precisely, we examine the potential of the sorting task to describe beer sensory characteristics, we determine the type of assessors able to perform a sorting task and we evaluate the stability of the results as well as some important methodological points (e.g. number of beers to be sorted, instructions given to the judges) that might impact the efficiency of the task.

Research highlights

► We present the current state of knowledge on sorting tasks, using beers as stimuli. ► The sorting task gives satisfactory results with both trained and untrained assessors. ► The sorting task is a robust tool even when used with as few as 20 assessors. ► Discrimination power is limited to stimuli sets containing no more than 20 beers. ► Descriptions of the sorted groups are not precise and might be difficult to interpret.

Introduction

For industries, the sensory characteristics of products are essential criteria in various areas including R&D, quality control, and marketing. For example in product development, it is crucial to understand the sensory characteristics of a product in order to evaluate the relationship between raw material and/or process parameters and the quality of the product. Likewise, monitoring the sensory characteristics in routine control is essential to maintain and control product quality.

A classical way to describe these sensory characteristics is to select a small group of panelists and train them to identify and quantify the main sensory dimensions of the products. This type of method, called sensory profile, is quite efficient but also very expensive and time consuming and therefore most industries cannot routinely use this technique (Kemp et al., 2009, Meilgaard et al., 1999, Stone and Sidel, 1993). Thus, it is necessary to develop other sensory methods to obtain sensory information about products. Among these new methods, the sorting task has been one of the most popular in the domain of product descriptions (see Abdi, Valentin, Chollet, & Chrea, 2007, for a review of sorting tasks applied on food and nonfood products). Several recent papers deal with this method, but they address only one aspect of this method, namely the comparison with conventional profile (Blancher et al., 2007, Cartier et al., 2006, Faye et al., 2004, Faye et al., 2006, Lelièvre et al., 2008, Lelièvre et al., 2009, Saint-Eve et al., 2004, Soufflet et al., 2004, Tang and Heymann, 1999). The goal of this article is to synthesize the current state of knowledge about the sorting task and to delineate its main advantages and limits. After an overview of the different methodologies available to perform sensory descriptions (conventional profile, free choice profile, flash profile, projective mapping and sorting task) we will review the results from several sorting experiments carried out between 1998 and 2008 on different sets of beers. These experiments examined the potential of the sorting task: (1) to describe beer sensory characteristics (Experiments 1 and 2), (2) to describe the type of assessors able to perform sorting task (Experiment 3) and (3) to evaluate the stability of the results as well as the main factors (e.g. number of beers that are sorted, instructions given to the judges) that might impact the efficiency of the task (Experiments 4–6).

The most frequently used method to determine the sensory characteristics of a set of products is certainly the sensory profile. This method belongs to the quantitative descriptive methods and can be performed using different procedures. Among these procedures the most widely used for describing food products are the Quantitative Descriptive Analysis or QDA (Stone, Sidel, Oliver, Woolsey, & Singleton, 1974), the Quantitative Flavor Profiling (Stampanoni, 1994) and the Spectrum™ method (Munoz & Civille, 1992). All these procedures require a small number (6–18) of assessors who have been preselected for their good sensory abilities and trained to describe the products. Training includes several steps. First, assessors consensually generate a list of objective, unique, unambiguous, and independent terms that will be used to describe the products. Usually each term is associated to a physical or chemical reference and to a precise protocol of assessment. Then assessors are trained to rate, on a scale, the intensity of each attribute. Finally before using the panel for describing products, the assessors’ performance is checked in terms of repeatability, discrimination and agreement. The quantitative data obtained for each attribute are generally analyzed using parametric statistics such as analysis of variance (ANOVA) and the relationship between attributes are often described with multivariate methods such as Principal Component Analysis (PCA).

Sensory profile is the only method designed to analyze products with a high degree of reliability and precision. This method provides quantifiable and relevant information on the sensory characteristics of the products but requires trained assessors. Training may vary widely because it depends on the objectives of the study in terms of precision and sensitivity but it always requires a substantive amount of time and so sensory profile is always costly and time consuming. In particular, language development and calibration are likely to require a long time to develop. In fact, these two steps can last from a few weeks to several months. Other limitations come from the use of conventional profile in the industrial environment. Training in conventional profile is generally limited to a specific type of products and thus the vocabulary generated by the panel for a given type of product is specific to it and cannot be generalized to other products. As a consequence different panels need to be trained to describe different types of products (Bitnes, Rødbotten, Lea, Ueland, & Martens, 2007) but it is not always possible for companies’ sensory analysts to set up as many panels as there are types of products to analyze. One way out of this problem is to form a panel that is able to analyze all types of products. But this kind of panel takes even more time to train and involves a long pre-training step to adapt the vocabulary to each type of products. Finally, this method is completely based on language and this creates potential comprehension and agreement problems. To alleviate these problems though, and to facilitate product description, some researchers have developed some consensual vocabulary for some families of products (see, e.g. for beers, Meilgaard, Dalgliesh, & Clapperton, 1979; for whiskies, Shortreed, Rickards, Swan, & Burtles, 1979, for wines, Guinard & Noble, 1986; for cheeses, Guerra, Méndez, Taboada, & Fernandez-Albalat, 1999). However even though these terminologies facilitate the communication, some comprehension and interpretation problems remain and in particular, panelists still require a long time to be “calibrated.” So despite its qualities, but because it often requires too much time and resources, the sensory profile evaluation is often dropped out when results are urgently needed.

In order to palliate some of the drawbacks of the conventional sensory profiling, some recent alternative sensory evaluation methods have been developed. These methods have the advantage of bypassing the training stage and thus might be an economical way of describing sensory properties.

One of the first alternatives that appeared in the literature was free choice profiling. Williams and Langron (1984) described a radically different approach to descriptive analysis in which no screening and training of assessors were required and in which assessors could use any words they wanted to describe and evaluate the products (Guy et al., 1989, Marshall and Kirby, 1988, Oreskovich et al., 1991). The data obtained with free choice profiling were initially analyzed by generalized procrustes analysis (GPA, Gower, 1971) but multiple factor analysis (MFA, Abdi and Valentin, 2007a, Escofier and Pagès, 1990) or any 3-way type multivariate analyses (e.g. STATIS, Abdi and Valentin, 2007b, Escoufier, 1980, Lavit et al., 1994) could also be used. These types of multivariate analyses give product maps similar to those obtained with PCA. The main difference is that on the map we find the specific terms of all the assessors rather than common terms. The main advantage of this technique is that it saves much time because it does not require training other than 1 h of explanation of the testing procedure to the assessors (i.e., generation of attributes, scoring of the attributes and use of the chosen scale). A second advantage of free profiling is that the assessors—which have not been trained—can still be regarded as representing naive consumers. However, free choice profiling is not problem-free. The large diversity of vocabulary used by the assessors makes the product map difficult to interpret. In order to provide reliable guidance for product researchers, the sensory analyst has to decide upon the meaning of each attribute. Therefore the resulting descriptions of the product sensory characteristics can come more from the sensory analyst than from the assessors.

Another alternative method is the flash profile (Dairou & Sieffermann, 2002) which is a combination of free choice profiling and a comparative evaluation of the whole product set. The flash profile was initially developed as a method providing a quick access to the relative sensory positioning of a set of products. The main advantage of this method is to provide a product map in a very short time because the phases of familiarization with the product space, attribute generation, and rating have been integrated into a single step. Assessors simply rank the products from the least intense to the most intense for each attribute that they have themselves chosen. This method forces assessors to focus on the perceived differences and to solely use discriminative attributes. Since the structure of the data is comparable to that of the free choice profile, the same multivariate analyses can be applied to both techniques. In some cases the simultaneous presentation of the whole set of products could be a drawback, when only one product is available at a time—as, for example, in control quality. Another weakness of the flash profile is to require expert assessors. According to Delarue and Sieffermann (2004), expert assessors have previously participated in several descriptive evaluation tasks and are able to understand panel leader’s instructions and generate discriminative and non-hedonic attributes, even if these assessors do not need to be trained on a specific product set. Moreover, as in free choice profiling, it could be difficult to interpret the sensory characteristics of the products because of the diversity of the vocabulary (Dairou & Sieffermann, 2002).

In parallel in the 1980’s and 1990’s, Risvik and collaborators developed projective mapping (Kennedy and Heymann, 2009, Risvik et al., 1994, Risvik et al., 1997), also recently re-labeled—with an intriguing blend of French and English—“Napping” (“nappe” in French means “tablecloth” Pagès, 2003, Pagès, 2005). In this method, assessors are asked to draw a map (which could be drawn on a tablecloth) in two dimensions and to position the products according to the similarity and dissimilarity between these products. The coordinates of each product on the map constitute the data. Originally, projective mapping data were analyzed with PCA but more recently Pagès, 2003, Pagès, 2005 proposed to use MFA because this technique takes into account the differences between assessors but as previously other equivalent methods could be used. As in flash profile the advantages and the drawbacks are linked to the comparative basis of this method: all the products have to be available at the same time. Moreover this method constrains the assessors to use two dimensions to discriminate between the products (Perrin et al., 2008).

The sorting task is a simple procedure for collecting similarity data in which each assessor groups together stimuli based on their perceived similarities. Sorting is based on categorization which is a natural cognitive process routinely used in everyday life and it does not require a quantitative response. The final objective of sorting task is to reveal—via statistical analyses—the structure of the product space and to interpret the underlying dimensions. Practically the assessors are in front of a set of products and are asked to compose different groups of products such that the products in a group are similar to each other. The groups should be homogenous and coherent. The sorting task can be stopped as this point or can be followed by a description step where assessors are asked to describe each group of products (for beers: Lelièvre et al., 2008, Lelièvre et al., 2009; for other products: Blancher et al., 2007, Cartier et al., 2006, Faye et al., 2006, Lawless et al., 1995, Lim and Lawless, 2005, Saint-Eve et al., 2004, Santosa et al., 2010, Tang and Heymann, 1999). Concerning statistical interpretation, the collected data are distance matrices which can be analyzed with two main sets of methods. The first set gives (Euclidean) map representations and comprises techniques such as multidimensional scaling analysis (MDS: Schiffman, Reynolds, & Young, 1981), DISTATIS (Abdi et al., 2005, Abdi et al., 2007), multiple correspondence analysis (MCA: Cadoret et al., 2009, Takane, 1981, Takane, 1982), common components and specific weights analysis (Qannari, Cariou, Teillet, & Schlich, 2009). The second set gives tree representation and comprises clustering techniques (Miller, 1969) and additive trees (Abdi, 1990).

The sorting task is simple and easy to perform but raises several practical and methodological issues that we explore in a series of experiments described below.

Section snippets

Beers

Six sets of beers were used in the different experimentations (Table 1). All the beers were presented in brown plastic tumblers and served between 8 and 10 °C under red light to mask the possible color differences between them and to avoid that assessors use this type of information in the sorting task. We used “blind tasting” because previous work has showed that the color of the beers has such an importance in these tasks that chemo-sensory characteristics were neglected by most assessors (

Conclusion

The review of the results from different sorting experiments suggests answers to several questions about the methodology of the sorting task – in particular a sorting task could be used with about 20 novice assessors, is a robust tool, is efficient with no more than 20 beers and provides groups similar to those obtained from a profile. However the sorting task can, in some cases, provide descriptions difficult to interpret and not as precise as those obtained from a standard profile.

From a

Acknowledgment

This work was supported by the Institut Supérieur d’Agriculture, CESG, the region of Nord-Pas-de-Calais, Qualtech and Danone. The authors would also like to gratefully thank the anonymous reviewers who for their helpful comments on a previous version of this paper.

References (75)

  • P. Faye et al.

    An alternative to external preference mapping based on consumer perceptive mapping

    Food Quality and Preference

    (2006)
  • A.L. Hughson et al.

    The knowing nose: The role of knowledge in wine expertise

    Food Quality and Preference

    (2002)
  • C. Lavit et al.

    The ACT (STATIS) method

    Computational Statistics & Data Analysis

    (1994)
  • H.T. Lawless et al.

    Multidimensional scaling of sorting data applied to cheese perception

    Food Quality and Preference

    (1995)
  • M. Lelièvre et al.

    What is the validity of the sorting task for describing beers? A study using trained and untrained assessors

    Food Quality and Preference

    (2008)
  • G.A. Miller

    A psychological method to investigate verbal concepts

    Journal of Mathematical Psychology

    (1969)
  • J. Pagès

    Collection and analysis of perceived product inter-distances using multiple factor analysis: Application to the study of 10 white wines from the Loire Valley

    Food Quality and Preference

    (2005)
  • L. Perrin et al.

    Comparison of three sensory descriptive methods for use with the Napping® procedure: Case of ten wines from Loire Valley

    Food Quality and Preference

    (2008)
  • E. Risvik et al.

    Projective mapping: A tool for sensory analysis and consumer research

    Food Quality and Preference

    (1994)
  • E. Risvik et al.

    Evaluation of sensory profiling and projective mapping data

    Food Quality and Preference

    (1997)
  • A. Saint-Eve et al.

    Impact of the olfactory quality and chemical complexity of the flavouring agent on the texture of low fat stirred yogurts assessed by three different sensory methodologies

    Food Quality and Preference

    (2004)
  • M. Santosa et al.

    A modified sorting task to investigate consumer perceptions of extra virgin olive oils

    Food Quality and Preference

    (2010)
  • I. Soufflet et al.

    A comparison between industrial experts’ and novices’ haptic perception organization: A tool to identify descriptors of handle of fabrics

    Food Quality and Preference

    (2004)
  • D. Valentin et al.

    Expertise and memory for beers and beer olfactory compounds

    Food Quality and Preference

    (2007)
  • H. Abdi

    Additive-tree representations

    Lecture Notes in Biomathemathics

    (1990)
  • H. Abdi

    The RV coefficient and the congruence coefficient

  • H. Abdi

    Metric multidimensional scaling

  • H. Abdi

    Congruence: Congruence coefficient, RV coefficient, and mantel coefficient

  • Abdi, H., Valentin, D., O’Toole, A.J., & Edelman, B. (2005). DISTATIS: The analysis of multiple distance matrices....
  • H. Abdi et al.

    STATIS

  • H. Abdi et al.

    Multiple factor analysis

  • M.D. Augustin et al.

    Art expertise: A study of concepts and conceptual spaces

    Psychology Science

    (2006)
  • T.C. Avancini de Almeida et al.

    Same-different discrimination tests with interstimulus delays up to one day

    Journal of Sensory Studies

    (1999)
  • J. Bitnes et al.

    Effect of product knowledge on profiling performance comparing various sensory laboratories

    Journal of Sensory Studies

    (2007)
  • A. Chatard-Pannetier et al.

    Représentation, catégorisation et évaluation: Différence entre experts et novices dans la domaine des meubles d’antiquités

    L’Année Psychologique

    (2002)
  • J. Chauhan et al.

    Descriptive profiling versus direct similarity assessments of soft drinks

    Journal of Food Technology

    (1986)
  • C. Chollet et al.

    Impact of training on beer flavor perception and description: Are trained and untrained assessors really different?

    Journal of Sensory Studies

    (2001)
  • Cited by (107)

    • Free sorting task method to optimize the development of smoked bacon: A case study

      2022, Sensory Analysis for the Development of Meat Products: Methodological Aspects and Practical Applications
    • Alternative methods of sensory testing: Working with chefs, culinary professionals and brew masters

      2022, Rapid Sensory Profiling Techniques: Applications in New Product Development and Consumer Research, Second Edition
    • Free sorting as a sensory profiling technique for product development

      2022, Rapid Sensory Profiling Techniques: Applications in New Product Development and Consumer Research, Second Edition
    View all citing articles on Scopus
    View full text