Sort and beer: Everything you wanted to know about the sorting task but did not dare to ask
Research highlights
► We present the current state of knowledge on sorting tasks, using beers as stimuli. ► The sorting task gives satisfactory results with both trained and untrained assessors. ► The sorting task is a robust tool even when used with as few as 20 assessors. ► Discrimination power is limited to stimuli sets containing no more than 20 beers. ► Descriptions of the sorted groups are not precise and might be difficult to interpret.
Introduction
For industries, the sensory characteristics of products are essential criteria in various areas including R&D, quality control, and marketing. For example in product development, it is crucial to understand the sensory characteristics of a product in order to evaluate the relationship between raw material and/or process parameters and the quality of the product. Likewise, monitoring the sensory characteristics in routine control is essential to maintain and control product quality.
A classical way to describe these sensory characteristics is to select a small group of panelists and train them to identify and quantify the main sensory dimensions of the products. This type of method, called sensory profile, is quite efficient but also very expensive and time consuming and therefore most industries cannot routinely use this technique (Kemp et al., 2009, Meilgaard et al., 1999, Stone and Sidel, 1993). Thus, it is necessary to develop other sensory methods to obtain sensory information about products. Among these new methods, the sorting task has been one of the most popular in the domain of product descriptions (see Abdi, Valentin, Chollet, & Chrea, 2007, for a review of sorting tasks applied on food and nonfood products). Several recent papers deal with this method, but they address only one aspect of this method, namely the comparison with conventional profile (Blancher et al., 2007, Cartier et al., 2006, Faye et al., 2004, Faye et al., 2006, Lelièvre et al., 2008, Lelièvre et al., 2009, Saint-Eve et al., 2004, Soufflet et al., 2004, Tang and Heymann, 1999). The goal of this article is to synthesize the current state of knowledge about the sorting task and to delineate its main advantages and limits. After an overview of the different methodologies available to perform sensory descriptions (conventional profile, free choice profile, flash profile, projective mapping and sorting task) we will review the results from several sorting experiments carried out between 1998 and 2008 on different sets of beers. These experiments examined the potential of the sorting task: (1) to describe beer sensory characteristics (Experiments 1 and 2), (2) to describe the type of assessors able to perform sorting task (Experiment 3) and (3) to evaluate the stability of the results as well as the main factors (e.g. number of beers that are sorted, instructions given to the judges) that might impact the efficiency of the task (Experiments 4–6).
The most frequently used method to determine the sensory characteristics of a set of products is certainly the sensory profile. This method belongs to the quantitative descriptive methods and can be performed using different procedures. Among these procedures the most widely used for describing food products are the Quantitative Descriptive Analysis or QDA (Stone, Sidel, Oliver, Woolsey, & Singleton, 1974), the Quantitative Flavor Profiling (Stampanoni, 1994) and the Spectrum™ method (Munoz & Civille, 1992). All these procedures require a small number (6–18) of assessors who have been preselected for their good sensory abilities and trained to describe the products. Training includes several steps. First, assessors consensually generate a list of objective, unique, unambiguous, and independent terms that will be used to describe the products. Usually each term is associated to a physical or chemical reference and to a precise protocol of assessment. Then assessors are trained to rate, on a scale, the intensity of each attribute. Finally before using the panel for describing products, the assessors’ performance is checked in terms of repeatability, discrimination and agreement. The quantitative data obtained for each attribute are generally analyzed using parametric statistics such as analysis of variance (ANOVA) and the relationship between attributes are often described with multivariate methods such as Principal Component Analysis (PCA).
Sensory profile is the only method designed to analyze products with a high degree of reliability and precision. This method provides quantifiable and relevant information on the sensory characteristics of the products but requires trained assessors. Training may vary widely because it depends on the objectives of the study in terms of precision and sensitivity but it always requires a substantive amount of time and so sensory profile is always costly and time consuming. In particular, language development and calibration are likely to require a long time to develop. In fact, these two steps can last from a few weeks to several months. Other limitations come from the use of conventional profile in the industrial environment. Training in conventional profile is generally limited to a specific type of products and thus the vocabulary generated by the panel for a given type of product is specific to it and cannot be generalized to other products. As a consequence different panels need to be trained to describe different types of products (Bitnes, Rødbotten, Lea, Ueland, & Martens, 2007) but it is not always possible for companies’ sensory analysts to set up as many panels as there are types of products to analyze. One way out of this problem is to form a panel that is able to analyze all types of products. But this kind of panel takes even more time to train and involves a long pre-training step to adapt the vocabulary to each type of products. Finally, this method is completely based on language and this creates potential comprehension and agreement problems. To alleviate these problems though, and to facilitate product description, some researchers have developed some consensual vocabulary for some families of products (see, e.g. for beers, Meilgaard, Dalgliesh, & Clapperton, 1979; for whiskies, Shortreed, Rickards, Swan, & Burtles, 1979, for wines, Guinard & Noble, 1986; for cheeses, Guerra, Méndez, Taboada, & Fernandez-Albalat, 1999). However even though these terminologies facilitate the communication, some comprehension and interpretation problems remain and in particular, panelists still require a long time to be “calibrated.” So despite its qualities, but because it often requires too much time and resources, the sensory profile evaluation is often dropped out when results are urgently needed.
In order to palliate some of the drawbacks of the conventional sensory profiling, some recent alternative sensory evaluation methods have been developed. These methods have the advantage of bypassing the training stage and thus might be an economical way of describing sensory properties.
One of the first alternatives that appeared in the literature was free choice profiling. Williams and Langron (1984) described a radically different approach to descriptive analysis in which no screening and training of assessors were required and in which assessors could use any words they wanted to describe and evaluate the products (Guy et al., 1989, Marshall and Kirby, 1988, Oreskovich et al., 1991). The data obtained with free choice profiling were initially analyzed by generalized procrustes analysis (GPA, Gower, 1971) but multiple factor analysis (MFA, Abdi and Valentin, 2007a, Escofier and Pagès, 1990) or any 3-way type multivariate analyses (e.g. STATIS, Abdi and Valentin, 2007b, Escoufier, 1980, Lavit et al., 1994) could also be used. These types of multivariate analyses give product maps similar to those obtained with PCA. The main difference is that on the map we find the specific terms of all the assessors rather than common terms. The main advantage of this technique is that it saves much time because it does not require training other than 1 h of explanation of the testing procedure to the assessors (i.e., generation of attributes, scoring of the attributes and use of the chosen scale). A second advantage of free profiling is that the assessors—which have not been trained—can still be regarded as representing naive consumers. However, free choice profiling is not problem-free. The large diversity of vocabulary used by the assessors makes the product map difficult to interpret. In order to provide reliable guidance for product researchers, the sensory analyst has to decide upon the meaning of each attribute. Therefore the resulting descriptions of the product sensory characteristics can come more from the sensory analyst than from the assessors.
Another alternative method is the flash profile (Dairou & Sieffermann, 2002) which is a combination of free choice profiling and a comparative evaluation of the whole product set. The flash profile was initially developed as a method providing a quick access to the relative sensory positioning of a set of products. The main advantage of this method is to provide a product map in a very short time because the phases of familiarization with the product space, attribute generation, and rating have been integrated into a single step. Assessors simply rank the products from the least intense to the most intense for each attribute that they have themselves chosen. This method forces assessors to focus on the perceived differences and to solely use discriminative attributes. Since the structure of the data is comparable to that of the free choice profile, the same multivariate analyses can be applied to both techniques. In some cases the simultaneous presentation of the whole set of products could be a drawback, when only one product is available at a time—as, for example, in control quality. Another weakness of the flash profile is to require expert assessors. According to Delarue and Sieffermann (2004), expert assessors have previously participated in several descriptive evaluation tasks and are able to understand panel leader’s instructions and generate discriminative and non-hedonic attributes, even if these assessors do not need to be trained on a specific product set. Moreover, as in free choice profiling, it could be difficult to interpret the sensory characteristics of the products because of the diversity of the vocabulary (Dairou & Sieffermann, 2002).
In parallel in the 1980’s and 1990’s, Risvik and collaborators developed projective mapping (Kennedy and Heymann, 2009, Risvik et al., 1994, Risvik et al., 1997), also recently re-labeled—with an intriguing blend of French and English—“Napping” (“nappe” in French means “tablecloth” Pagès, 2003, Pagès, 2005). In this method, assessors are asked to draw a map (which could be drawn on a tablecloth) in two dimensions and to position the products according to the similarity and dissimilarity between these products. The coordinates of each product on the map constitute the data. Originally, projective mapping data were analyzed with PCA but more recently Pagès, 2003, Pagès, 2005 proposed to use MFA because this technique takes into account the differences between assessors but as previously other equivalent methods could be used. As in flash profile the advantages and the drawbacks are linked to the comparative basis of this method: all the products have to be available at the same time. Moreover this method constrains the assessors to use two dimensions to discriminate between the products (Perrin et al., 2008).
The sorting task is a simple procedure for collecting similarity data in which each assessor groups together stimuli based on their perceived similarities. Sorting is based on categorization which is a natural cognitive process routinely used in everyday life and it does not require a quantitative response. The final objective of sorting task is to reveal—via statistical analyses—the structure of the product space and to interpret the underlying dimensions. Practically the assessors are in front of a set of products and are asked to compose different groups of products such that the products in a group are similar to each other. The groups should be homogenous and coherent. The sorting task can be stopped as this point or can be followed by a description step where assessors are asked to describe each group of products (for beers: Lelièvre et al., 2008, Lelièvre et al., 2009; for other products: Blancher et al., 2007, Cartier et al., 2006, Faye et al., 2006, Lawless et al., 1995, Lim and Lawless, 2005, Saint-Eve et al., 2004, Santosa et al., 2010, Tang and Heymann, 1999). Concerning statistical interpretation, the collected data are distance matrices which can be analyzed with two main sets of methods. The first set gives (Euclidean) map representations and comprises techniques such as multidimensional scaling analysis (MDS: Schiffman, Reynolds, & Young, 1981), DISTATIS (Abdi et al., 2005, Abdi et al., 2007), multiple correspondence analysis (MCA: Cadoret et al., 2009, Takane, 1981, Takane, 1982), common components and specific weights analysis (Qannari, Cariou, Teillet, & Schlich, 2009). The second set gives tree representation and comprises clustering techniques (Miller, 1969) and additive trees (Abdi, 1990).
The sorting task is simple and easy to perform but raises several practical and methodological issues that we explore in a series of experiments described below.
Section snippets
Beers
Six sets of beers were used in the different experimentations (Table 1). All the beers were presented in brown plastic tumblers and served between 8 and 10 °C under red light to mask the possible color differences between them and to avoid that assessors use this type of information in the sorting task. We used “blind tasting” because previous work has showed that the color of the beers has such an importance in these tasks that chemo-sensory characteristics were neglected by most assessors (
Conclusion
The review of the results from different sorting experiments suggests answers to several questions about the methodology of the sorting task – in particular a sorting task could be used with about 20 novice assessors, is a robust tool, is efficient with no more than 20 beers and provides groups similar to those obtained from a profile. However the sorting task can, in some cases, provide descriptions difficult to interpret and not as precise as those obtained from a standard profile.
From a
Acknowledgment
This work was supported by the Institut Supérieur d’Agriculture, CESG, the region of Nord-Pas-de-Calais, Qualtech and Danone. The authors would also like to gratefully thank the anonymous reviewers who for their helpful comments on a previous version of this paper.
References (75)
- et al.
Analyzing assessors and products in sorting tasks: DISTATIS, theory and applications
Food Quality and Preference
(2007) - et al.
Comparison of free choice profiling, direct similarity measurements and hedonic data for ewes’ milk cheeses sensory evaluation
International Dairy Journal
(2003) - et al.
French and Vietnamese: How do they describe texture characteristics of the same food? A case study with jellies
Food Quality and Preference
(2007) - et al.
A factorial approach for sorting task data (FAST)
Food Quality and Preference
(2009) - et al.
Sorting procedure as an alternative to quantitative descriptive analysis to obtain a product sensory map
Food Quality and Preference
(2006) - et al.
Categorization and representation of physics problems by experts and novices
Cognitive Science
(1981) - et al.
Do trained assessors generalize their knowledge to new stimuli?
Food Quality and Preference
(2005) - et al.
Sensory mapping using Flash profile Comparison with a conventional descriptive method for the evaluation of the flavour of fruit dairy products
Food Quality and Preference
(2004) - et al.
Perceptual variation among drinking waters: The reliability of sorting and ranking data for multidimensional scaling
Food Quality and Preference
(1997) - et al.
Perceptive free sorting verbalization tasks with naive assessors: An alternative to descriptive mappings
Food Quality and Preference
(2004)
An alternative to external preference mapping based on consumer perceptive mapping
Food Quality and Preference
The knowing nose: The role of knowledge in wine expertise
Food Quality and Preference
The ACT (STATIS) method
Computational Statistics & Data Analysis
Multidimensional scaling of sorting data applied to cheese perception
Food Quality and Preference
What is the validity of the sorting task for describing beers? A study using trained and untrained assessors
Food Quality and Preference
A psychological method to investigate verbal concepts
Journal of Mathematical Psychology
Collection and analysis of perceived product inter-distances using multiple factor analysis: Application to the study of 10 white wines from the Loire Valley
Food Quality and Preference
Comparison of three sensory descriptive methods for use with the Napping® procedure: Case of ten wines from Loire Valley
Food Quality and Preference
Projective mapping: A tool for sensory analysis and consumer research
Food Quality and Preference
Evaluation of sensory profiling and projective mapping data
Food Quality and Preference
Impact of the olfactory quality and chemical complexity of the flavouring agent on the texture of low fat stirred yogurts assessed by three different sensory methodologies
Food Quality and Preference
A modified sorting task to investigate consumer perceptions of extra virgin olive oils
Food Quality and Preference
A comparison between industrial experts’ and novices’ haptic perception organization: A tool to identify descriptors of handle of fabrics
Food Quality and Preference
Expertise and memory for beers and beer olfactory compounds
Food Quality and Preference
Additive-tree representations
Lecture Notes in Biomathemathics
The RV coefficient and the congruence coefficient
Metric multidimensional scaling
Congruence: Congruence coefficient, RV coefficient, and mantel coefficient
STATIS
Multiple factor analysis
Art expertise: A study of concepts and conceptual spaces
Psychology Science
Same-different discrimination tests with interstimulus delays up to one day
Journal of Sensory Studies
Effect of product knowledge on profiling performance comparing various sensory laboratories
Journal of Sensory Studies
Représentation, catégorisation et évaluation: Différence entre experts et novices dans la domaine des meubles d’antiquités
L’Année Psychologique
Descriptive profiling versus direct similarity assessments of soft drinks
Journal of Food Technology
Impact of training on beer flavor perception and description: Are trained and untrained assessors really different?
Journal of Sensory Studies
Cited by (107)
Influence of taste sensitivity on preference and sensory perception of mezcal
2024, Food Research InternationalAnalyzing larger sample sets with rapid methods: Incomplete-block designs with free-sorting and free-linking tasks
2024, Food Quality and PreferenceFree sorting task method to optimize the development of smoked bacon: A case study
2022, Sensory Analysis for the Development of Meat Products: Methodological Aspects and Practical ApplicationsAlternative methods of sensory testing: Working with chefs, culinary professionals and brew masters
2022, Rapid Sensory Profiling Techniques: Applications in New Product Development and Consumer Research, Second EditionFree sorting as a sensory profiling technique for product development
2022, Rapid Sensory Profiling Techniques: Applications in New Product Development and Consumer Research, Second EditionPreferred beer styles influence both perceptual maps and semantic descriptions of dry hops
2021, Food Quality and Preference