PROBLEMS WITH THE GRADUATED FREQUENCY APPROACH TO MEASURING ALCOHOL CONSUMPTION: RESULTS FROM A PILOT STUDY IN TORONTO, CANADA

Abstract

Aims: To evaluate advantages and disadvantages of the graduated frequency (GF) approach, which asks about the frequency of alcohol consumption at mutually exclusive quantity levels (i.e. 12 or more drinks, at least eight drinks but less than 12, etc.).

Methods: Telephone survey of 464 adults aged 18 and older in Toronto, Canada, using random digit dialling and computer-assisted telephone interviewing.

Results: Respondents reported higher frequency and volume of drinking on the GF compared to overall and beverage-specific quantity–frequency type measures; however, at least 16% of GF responses included double counting on their frequency estimates using the GF. When these cases were excluded or corrected, differences between the GF and quantity–frequency measures mostly disappeared. The GF was superior to quantity–frequency measures for identifying heavy episodic drinkers. However, the GF had little advantage over the weekly recall method except for identifying very infrequent (i.e. less often than twice a month) heavy drinkers. Conclusions: Because the GF has a high rate of response errors in terms of measuring frequency of alcohol consumption, other combinations of measures, including alternate measures of heavy episodic drinking should be considered.

(Received 31 December 2003; first review notified 4 February 2004; in revised form 7 May 2004; accepted 9 May 2004)

INTRODUCTION

Surveys of alcohol consumption provide important information for a variety of purposes, including estimating risk and burden of disease Rehm et al., 2004, monitoring trends over time (Greenfield and Kerr, 2003) and evaluating policies and prevention strategies (Babor et al., 2003). There is currently no consensus regarding the best way to measure alcohol consumption, and it is unlikely that a single all-purpose ‘best’ measure will be found given that the advantages of any particular measure depend on the research question to be addressed and the population or culture being studied (Knibbe and Bloomfield, 2001). At the same time, the field continues to work toward identification of good measures, and a number of measurement principles have been recommended (Dawson and Room, 2000).

Increasingly, the graduated frequency (GF) approach that asks about the frequency of consuming alcohol at different quantity levels, usually starting at the highest quantity consumed by the respondent (e.g. frequency of drinking 12-plus drinks, eight to 12 drinks, six to eight drinks) is recommended for survey research (Hilton, 1989; Greenfield, 2000). Theoretically, the GF provides one of the best measures of drinking pattern and volume to date. In practice, however, the GF requires estimates that many respondents find difficult and burdensome, and some problems have been encountered. For example Greenfield reported that nine percent of respondents in the 1995 National Alcohol Survey reported drinking more than 365 days of the year, although some of this overestimate may have been partly due to the algorithm for calculating total frequency which used the middle range for each frequency category (Greenfield, 2000).

Commonly used alternatives to the GF include: overall quantity–frequency (QF) (i.e. usual frequency of drinking by usual number of drinks consumed per drinking occasion (Rehm, 1998), beverage-specific quantity–frequency (BQF) [i.e. QF for each type of beverage (Serdula et al., 1999)] and methods involving the last several drinking occasions (Wyllie et al., 1994; Dawson and Room, 2000), past 7 days (WR) (Rehm, 1998) or detailed questions regarding the number of drinks consumed yesterday (Knibbe and Bloomfield, 2001).

Each of these measures has some strengths and weaknesses. As noted by Midanik, the basic QF, while brief and easy to use, underestimates overall volume of consumption and does not address drinking pattern (Midanik, 1994); however, the QF may provide a reasonable estimate of drinking frequency and, because of its widespread use, it is useful to include for comparisons to other studies (Dawson and Room, 2000). The BQF has some major advantages over the simple QF, particularly in terms of comparing drinking by sex and culture (Dawson, 1993) and in estimating overall volume (Williams et al., 1994; Serdula, 1999); however, the BQF does not provide an overall measure of frequency. The recent occasions, WR and yesterday's consumption have the advantage over most other measures in that they do not require mental averaging or estimating usual drinking; instead, actual drinks consumed are reported. The WR also captures drinking pattern, at least to some extent (Lemmens, 1988). The ‘yesterday’ method (Knibbe and Bloomfield, 2001) allows very detailed questions about drink size and content, relies least on memory and mental averaging and produces overall estimates closest to sales data, but does not allow estimates of drinking pattern. The major disadvantages of the WR and the yesterday method are that past week's drinking (or yesterday) may be atypical for a subset of respondents, and that the restricted time frame overestimates the proportion of abstainers and daily drinkers.

Recently, the GF was adopted by the cross-national collaborators on the GENACIS project (Gender, Alcohol and Culture: An International Study: Wilsnack et al., 2002–2005) as the primary measure of alcohol consumption to be used by all countries entering into the collaboration along with the two-question QF. As part of conducting pilot research for the Canadian GENACIS national survey, we included several other measures of alcohol consumption in addition to the overall QF and the GF, namely, beverage-specific QF (BQF) and weekly recall (WR) in order to identify the best combination of measures to use in the national survey. In the present paper, we first compare the GF to the QF and BQF for drinking frequency, volume and quantity per drinking day, exploring the reasons for inconsistencies between the GF and other measures and the impact of these inconsistencies on the overall estimates. To our knowledge, an analysis of this type has never been conducted before. Although Midanik (1994) examined characteristics of respondents who provided inconsistent responses to the GF versus the QF, she did not examine the nature of these inconsistencies. The WR was not used in these comparisons as it is not considered an appropriate measure of drinking frequency, volume or quantity because it provides no information for those who did not consume alcohol in the past week.

Second, because assessing drinking pattern (especially heavy episodic drinking) is a major strength of the GF, we compare the GF to the WR, QF and BQF in terms of identification of persons who drink five-plus, eight-plus and 12-plus drinks at various frequency levels. Although the QF and BQF are not designed to measure heavy episodic drinking in that they ask the respondent about usual consumption, comparing these methods with the GF can be used to identify the extent that heavy episodic drinkers are identified by the GF but missed by QF type methods. Similarly, the WR is not designed to measure heavy episodic drinking in that it only measures whether a specific number of drinks was consumed on a single day during the past week but it has the advantage of asking about actual rather than usual consumption. Thus, comparison of the WR and the GF can assess the extent that the GF outperforms the WR in assessing heavy episodic drinking.

MATERIALS AND METHODS

Data were collected as part of two pilot studies (n = 201 and n = 263) conducted with a random sample of adults aged 18 and older living in the greater Toronto (Canada) area using random digit dialling (RDD) and computer-assisted telephone interviewing (CATI). In the first pilot, the BQF was asked first, followed by the QF, the GF and the WR. In the second pilot, a two-stage survey was being tested with the BQF moved to the second part of the survey. Therefore, in the second pilot, the QF was asked first, followed by the GF and the WR, with the BQF administered in a second part of the survey that was not given to all respondents. Separate analyses of the two pilots identified no significant differences on relevant measures; therefore, data from the two pilots were combined for the present analyses. The questions on alcohol were preceded by a description of a standard drink size. The overall QF was measured using the following two questions.

‘During the last 12 months, how often did you usually have any kind of drink containing alcohol, whether it was wine, beer, liquor or any other alcohol? Every day, 4–6 days a week, 2–3 days a week, once a week, 2–3 days a month, once a month, less than once a month, never in the last 12 months?’

‘In the past 12 months when you had any kind of beverage containing alcohol including beer, wine, liquor and other drinks, how many drinks did you have on a typical day?

The BQF used the same question format and response options as the QF but asked questions separately for wine, beer, spirits and 'coolers' (i.e. premixed drinks).

The GF was preceded by a question asking the most drinks the respondent had consumed on a single day during the past year. This question determined the level of drinking that started the GF process, with 12 or more drinks being the highest quantity for starting the frequency questions. Light infrequent drinkers (maximum of fewer than three drinks on a single day and usual frequency once a month or less on the QF) were not asked the GF. In addition, because respondents reacted negatively in pretesting when asked about frequency levels that exceeded their overall frequency level reported on the QF, response categories for the GF were scaled to match the overall frequency on the QF (e.g. if the person reported drinking once a week on the QF, the frequency response options for the quantity categories on the GF began with once a week). Finally, as soon as a respondent replied ‘every day’ to any quantity category on the GF, the subsequent quantity categories were not asked.

The WR used a standard format employed in most other Canadian alcohol surveys involving asking the respondent how many drinks he/she consumed yesterday, the day before yesterday, and so on for the previous seven days.

Calculations of frequency, volume, and quantity of drinking and heavy episodic drinking

Annual frequency of drinking based on the GF was calculated by summing the frequency responses at each quantity level using the following values: every day = 365, 4–6 days per week = 260 (i.e. 5 days per week × 52 weeks), 2–3 days per week = 130, 1 day per week = 52, 2–3 days per month = 30, once per month = 12 and less than once per month = 6. The same values were used for calculating annual frequency from the QF and annual beverage-specific frequencies on the BQF. It is not possible to calculate a measure of overall frequency for the BQF.

Average volume was calculated for the GF by multiplying the midpoints of each quantity (for the category of 12-plus drinks per occasion, 13 drinks was used for the calculation of volume as recommended by Greenfield, 2000) by the annual frequency and summing these products. Volume on the QF was calculated by multiplying usual quantity by usual (annual) frequency. Volume for the BQF was calculated by multiplying the quantity for each beverage by the annual frequency for that beverage and summing the values obtained.

Usual quantity per drinking day was calculated from the GF by dividing the total volume per year by total frequency and from the BQF by dividing the total volume per year by overall frequency based on the maximum frequency provided for the QF or BQF. Usual quantity was asked as part of the QF.

Three levels of heavy drinking were defined (five-plus, eight-plus, 12-plus) for three levels of frequency of heavy drinking (at least once a week, at least twice a month, at least once a month). For the GF, these values were calculated by summing frequencies for relevant quantity categories. For the QF and the BQF those reporting usual consumptions of five-plus, eight-plus or 12-plus at usual frequencies of weekly or more frequent, twice monthly or once a month were identified. The QF and BQF were combined into a single measure by including those who scored positive for frequent high quantity drinking using either measure. For the WR, those who had consumed each quantity on at least one day in the previous week were identified.

Analyses

Matched t-tests were used to assess significance of pairwise comparisons of mean levels of consumption on the GF with the QF and BQF for frequency, volume and quantity; Pearson and Spearman rank order correlations were also used to illustrate the relationships between the measures. McNemar's test of proportions was used to assess the significance of differences (and the Kappa statistic used to assess agreement) between the GF and other measures in terms of identifying heavy episodic drinkers.

RESULTS

Of the 464 respondents to both pilots, 103 reported that they had not consumed alcohol during the past year, 348 provided responses to the QF, 281 to the GF, 228 to the BQF, and 220 to the WR.

Frequency of drinking

As shown in Table 1, the Spearman rho rank order correlations indicated fairly high agreement between the GF and QF; the Pearson correlations were similar except slightly lower for men and higher for women. These correlations are probably underestimates because light infrequent drinkers were not asked the GF. Mean number of drinking days per year was significantly higher for the GF compared to the QF. Inspection of the actual responses, however, suggested that at least some of the higher scores on the GF were due to double-counting in the following ways. In total (1 and 2 above), 16% were identified as double counting, with a higher proportion of double counting for men (22%) than for women (8%). The problem rate for the GF is even higher when it is taken into consideration that many respondents (23%) were only asked one frequency question on the GF (e.g. those who reported a maximum of two drinks per occasion were asked only the frequency of consuming one to two drinks as part of the GF), thus double counting would not be possible for these respondents. When those who reported a frequency value for only one quantity level on the GF were excluded from the total, the proportion of respondents with double counting increased to 20.8%.

1. Providing responses that summed to more than 365 drinking days per year. It should be noted that a conservative method was adopted for identifying this error; namely, using the low end of the frequency ranges (e.g. 4 days per week for a response of 4–6 days per week) rather than the usual algorithm (Greenfield, 2000) of using the mid-point (i.e. 5 days for 4–6 days per week); This error was found for 13 respondents, 4.6% of those responding to the GF.

2. Responding with an overall frequency of drinking for more than one quantity level on the GF. As an example of the latter error, one respondent who reported usual frequency of drinking on the QF as 2–3 days per month, reported this same frequency for consuming eight to 11 drinks, five to seven drinks, three to four drinks and one to two drinks. Thus, the frequency of drinking on the QF was computed to be 30 days per year while the frequency on the GF was estimated as four times as high (120 days per year). It should be noted that multiple responses of less than once per month were not considered an error because it was possible to have this as an overall frequency and still report this frequency for multiple levels of consumption. This error was found for 32 respondents, 11.4% of GF respondents.

It should be noted that this estimate probably underestimates double counting as it does not include cases of possible double counting that did not exceed 365 days or did not use multiple application of the overall frequency scale. For example, if a person responded to the overall QF that he/she usually drank 2–3 days per week and then responded to the GF that he/she drank 12-plus drinks less than once per month, eight to 12 drinks once per month, five to eight drinks once per week, two to four drinks once per week and one to two drinks 2–3 days a week, this pattern on the GF would not have been identified as an error despite evidence that the person was probably over-estimating frequency by ignoring that the GF quantity categories were intended to be mutually exclusive.

When cases with double counting on the GF were excluded from the analyses, the mean frequency of drinking on the GF was no longer significantly different from the mean frequency on the QF (see Table 1). However, there were differences between sexes in the comparison of the GF with the QF. For men, the pattern was the same as that for the total sample; whereas for women, the mean frequency on the QF was the same as that of the GF before removing double counting and significantly higher than the GF when cases with double counting were removed.

Volume of alcohol consumption

As shown in Table 1, the GF correlated quite highly with the QF and BQF on volume. The mean volume of consumption was significantly higher on the GF than on the QF or BQF; however, when cases with double counting were excluded, the GF was no longer significantly different from the BQF and the difference between the GF and the QF was substantially reduced.

It is possible that excluding cases with double counting overestimates the comparability of the GF with other measures. Therefore, where sufficient information was available, estimates were developed on the GF for the excluded cases in the following manner. Each case was examined individually, including responses to the GF, QF, BQF and WR. Where data were consistent in suggesting that the respondents erroneously applied usual frequency to all relevant quantity levels, the frequencies on the GF were divided by the number of quantity levels for which overall frequency was reported (nine cases). Where it appeared that each lower level was subsumed under higher levels (e.g. frequency of consuming five to seven drinks included those times when more than seven drinks were consumed) and other data suggested that the overall frequency on the QF was consistent with the BQF and the WR, each previous frequency was subtracted from the subsequent frequency so that the total frequency was the same as the overall frequency (11 cases). There were a number of cases, however, that seemed to involve a combination of errors. For example, once person reported consuming 12-plus drinks less than once per month, eight to 11 drinks once per month, five to seven drinks once per month, three to four drinks 2–3 days per week and one to two drinks 2–3 days per week, with an overall frequency of 2–3 days per week and usual quantity of four drinks per day. Subtracting each higher frequency would have made the frequency of drinking one to two drinks zero. Yet, the past week's drinking measure showed the person consuming two drinks per day on 2 days. Therefore, the frequency of consuming three to four drinks was adjusted before the calculation so that the remaining frequency after subtracting frequencies for higher levels was split between three to four and one to two drinks. After similar changes, it was possible to produce estimates for an additional 12 cases. For 13 cases, the data were too inconsistent to make estimates. When these additional estimates were included, the mean volume was 313 (SD = 393) versus 262 (SD = 320) for the GF versus QF respectively (t = 3.2, d.f. = 262, P = 0.002) and 290 (SD = 373) versus 285 (SD = 368) for the GF versus BQF (t = 0.2, d.f. = 188, P = 0.820). These results essentially show the same pattern as those excluding all cases with double counting.

Quantity per drinking day

The number of drinks per drinking day as measured by the GF was moderately correlated with the average quantities from the QF and the BQF, although these correlations were generally lower than for frequency and volume (possibly related to the lower variability on quantity compared to frequency or volume). There was an overall pattern for the GF to result in higher mean values on drinks per drinking day than the QF but lower or equal quantities when compared with the BQF, regardless of whether cases with double counting were excluded.

Identifying heavy episodic drinkers

The following analyses were conducted on the whole sample, that is, did not exclude cases with double counting. As shown in Table 2, the GF was not significantly better than the BQF/QF in terms of identifying persons who consumed five-plus drinks at least weekly. However, the GF was significantly better at identifying those who consumed five-plus drinks at least twice per month and at least once per month. The superiority of the GF over the BQF/QF in measuring frequency of high risk drinking is also evident for consuming eight or more drinks and 12 or more drinks even for weekly frequency.

The GF was not clearly superior to the WR for identifying heavy episodic drinkers, especially for weekly and two-plus monthly frequencies. A significantly larger proportion reported consuming five-plus and eight-plus drinks during the past week on the WR than reported consuming these amounts at least once per week on the GF. Even at the frequency level of at least twice per month on the GF, the difference between the GF and the WR was only significant for consuming five-plus (not for eight-plus and 12-plus) and this applied only to men. For frequency of at least once per month, however, the GF identified many more heavy episodic drinkers than the WR.

In terms of sex, the comparison of the GF with the BQF/QF and the WR showed the same trend for men as found for the whole sample. For women, differences between the GF and the BQF/QF were only significant for those identified as drinking five-plus or eight-plus at a frequency of at least once per month, and the only significant difference between the GF and the WR was that significantly more female respondents reported consuming eight-plus drinks on the WR than reported consuming eight-plus drinks at least weekly on the GF. Thus, the WR actually outperformed the GF for identifying female heavy episodic drinkers.

Overall, there was considerable disagreement among the methods in terms of which respondents were identified as hazardous drinkers at each of the frequency levels; most Kappa statistics comparing alternate options of identifying high risk drinkers were less than 0.40 (results not shown).

DISCUSSION

Consistent with previous research (Knibbe and Bloomfield, 2001), we found that a sizable proportion (4.6%) of respondents reported a frequency of alcohol consumption of more than 365 days per year. However, we found that substantially more respondents (more than 11%) over-estimated frequency, by responding with overall frequency to multiple quantity levels on the GF. To our knowledge, this type of error has not been identified in the alcohol measurement literature, although Poikolainen et al. (2002) found that the GF overestimated consumption when compared to prospective diary recording. It should be noted that the high rate of double counting found in the present study occurred despite safeguards in the CATI programming (e.g. starting GF frequencies ranges no higher than the overall frequency reported on the QF, discontinuing questions as soon as the respondent gave a frequency response of ‘every day’ to any level of consumption). Moreover, even this rate of problems is probably an underestimate given the conservative methods used to identify double counting. On the other hand, the rate of problems may be higher for telephone surveys than for other methods such as in-person interviews where clarification of the GF might be provided during the interview. In the present study, the telephone interviewers followed the question protocol strictly, clearly stating the mutually exclusive levels of consumption on the GF but not offering further clarification.

It is possible that the method used in the present study of beginning the GF at a maximum of 12 or more drinks and using 13 drinks in the computation actually resulted in underestimates on the GF for some heavy drinkers. However, in the present study, 75% of the 56 respondents who reported drinking 12 or more drinks reported consuming this amount once per month or less; therefore, the impact of this factor on overall consumption of the sample was likely to be small. Ideally, the GF questions would start at a higher quantity level (e.g. 20-plus drinks) in order to fully capture amounts consumed by heavier drinkers. At the same time, this benefit needs to be weighed against the overall goals of the survey and repetitiveness of GF questions when numerous levels are included.

The pattern of results by sex indicated that double counting had a substantial impact on estimates of frequency and volume of drinking for men but relatively little impact for women. This differential impact by sex suggests that differences on frequency and volume of drinking between sexes might be over-estimated when the GF is used.

The fact that the GF tends to result in higher reported volume of drinking than other measures has been considered a strength of the measure because survey estimates of overall alcohol consumption are generally lower than production and sales figures. However, if the higher estimate on the GF is due to double counting on frequency, the GF may be less accurate than other measures for analyses at the individual level even though it produces aggregate estimates that are closer to sales and production estimates. Given that the BQF appears to overcome underestimates of volume found with the overall QF (Serdula et al., 1999), there appears to be no advantage of using the GF in terms of measuring volume, while the BQF has the added advantage of measuring beverage-specific consumption, a feature that is important in terms of sex and cultural comparisons (Dawson, 1993).

It is clear that the GF has advantages over the BQF/QF in identifying heavy episodic drinkers, especially for men. However, even though the WR misses heavy drinking by some occasional drinkers who did not drink in the week prior to the survey, it does considerably better than the GF in terms of the total number of people identified as consuming high levels of alcohol at a frequency of at least once per week and does as well as the GF for identifying those consuming at high levels at least twice per month. Thus, although the WR may not be as good as the GF or even the QF for identifying hazardous drinkers based on volume of consumption (Rehm, 1998), the WR appears to be equally effective as the GF in terms of identifying heavy episodic drinkers, except for those who consume high quantities very rarely (i.e. less frequently than twice per month). However, the lack of agreement among the measures in these analyses suggests that there is still work to be done to develop reliable and valid measures of heavy episodic drinking. It may be that such identification needs to be based on multiple measures rather than relying on a single measure.

In conclusion, the GF led the field as an innovative and comprehensive method to capture not only frequency, quantity and volume of alcohol consumption but also drinking pattern. Nevertheless, as our analyses show, in practice a substantial proportion of respondents, at least in telephone interviews, err by double counting frequency on this measure. Thus, an alternative approach might be to combine the BQF/QF with less cognitively complex alternatives to the GF for measuring heavy episodic drinking, such as asking the frequency of consuming five-plus, eight-plus and 12-plus drinks (Midanik, 1994; Dawson and Room, 2000). Greenfield (2000) has suggested that an advantage of the GF is that by using mutually exclusive categories that start the GF questions at the highest level of quantity, the approach 'legitimizes' high quantity drinking and makes the respondent more willing to report higher quantities. If this is the case, using the alternative questions that ask about levels in ascending order may reduce response errors but may also underestimate high quantity drinking compared to the GF. The extent that this is a problem with alternatives to the GF needs to be evaluated in future research.

This research was supported by funding to K.G. and A.D. from the Canadian Institutes of Health CIHR, specifically, the Gender and Health Institute and the Institute on Neurosciences, Mental Health and Addictions.

REFERENCES

Author notes

1Centre for Addiction and Mental Health, London, Ontario

2Department of Psychology, University of Western Ontario, London, Ontario, Canada

3Groupe de Recherche sur les Aspects Sociaux de la Santé et de la Prévention (GRASP)

4Department of Sociology, Université de Montréal, Montréal, Québec, Canada

5Centre for Addiction and Mental Health

6Department of Public Health Sciences, Faculty of Medicine, University of Toronto

7Department of Psychiatry, University of Toronto, Toronto, Canada

8Addiction Research Institute, Zurich, Switzerland

9Swiss Institute for the Prevention of Alcohol and Drug Problems, Lausanne, Switzerland