Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The Reliability and Validity of Short Online Questionnaires to Measure Fruit and Vegetable Intake in Adults: The Fruit Test and Vegetable Test

Abstract

The first aim of this study was to investigate the stability of the Fruit Test and Vegetable Test over time and whether the Fruit Test and Vegetable Test are capable of measuring fruit and vegetable intake with consistency. Second, the study aimed to examine criterion (concurrent) validity of the Fruit Test and Vegetable Test by testing their agreement with 7-day food diary-derived measures of fruit and vegetable intake. In total 58 adults (31% male, mean age = 30.0±12.09y) completed the Flemish Fruit and Vegetable test by indicating the frequency of days that they ate fruit and vegetables and the number of portions during the past week. Validity was tested by using a 7-day food diary as a golden standard. Adults were asked to register their fruit and vegetable intake daily in a diary during one week. Spearman correlations were measured to compare total intake reported in the Fruit and Vegetable Test and in the 7-day diary. Agreement plots were used to illustrate absolute agreement. Test-retest reliability was evaluated by having participants completing the Fruit Test and Vegetable Test twice. The Fruit Test (ICC = 0.81) and Vegetable Test (ICC = 0.78) showed excellent and substantial reliability. The Fruit Test (ρ = 0.73) and Vegetable Test showed good validity. Agreement plots showed modest variability in differences between vegetable and fruit intake as measured by the Vegetable and Fruit Test and the 7-day food diary. Also a small underestimation of fruit intake in the Fruit test and vegetable intake in the Vegetable test against the 7-day food diary was shown. Based on the results, it is suggested to include portion size pictures and consumption of mixed vegetables to prevent underestimation. To prevent overestimation, it is concluded to add a moderate number of representative fruit and vegetable items, questions on portion size, household sizes with sufficient detail and food items highly tailored to the dietary behaviors and local food items of the population surveyed. The questionnaires can easily be adapted for the use in other diets (e.g. Asian diet), but reliability and validity should then be examined again. Also, validity remains to be tested in other population groups (i.e. low socio economic status groups, other age groups).

Introduction

The World Health Organisation (WHO) recommends adults to consume a minimum of 400 g of fruits and vegetables per day to prevent chronic diseases (e.g. diabetes) [1]. However, in most Western countries a large part of the adult population does not meet this recommendation. In 2013, only 30% of Belgian adults ate two pieces of fruit per day (i.e. 250 gram), and only 39% ate two portions (i.e. 300 gram) of vegetables every day[2]. To address these public health concerns, effective interventions promoting fruit and vegetable intake (FV intake)in a large population are necessary. Computer-tailored interventions have shown to be effective in promoting dietary intake and are able to reach large populations at a low cost[3]. In computer-tailored interventions, online self-administered questionnaires are used to provide tailored feedback[4]. To ensure feasibility of computer-tailored interventions, attractive and brief online questionnaires are required[510]. These online questionnaires, however, also need to measure FV intake in a valid and reliable manner to allow adequate feedback based upon a comparison of adults’ mean intake of fruit and vegetables with the health guidelines (i.e. consuming a minimum of 400 g of fruits and vegetables per day) [5]. Kim and Holowaty (2003) conducted a literature review of brief, validated survey instruments measuring self-reported fruit and vegetable consumption. They identified survey instrument characteristics that are associated with greater validity and/or reliability[10]. These instruments were validated by comparing them to other methods (e.g. extended food frequency questionnaires, weighted dietary records, single or multiple 24-hour recalls). The authors found 10 instruments with less than 17 items, with correlations ranging from r = 0,29 to 0,84[10]. Through a further search using the International Register of Validated Short Dietary Assessment Instruments[11], 16 other short (<20 items) questionnaires that were not included in the review of Kim and Holowaty (2003) were identified. A variation in correlations coefficients was found, showing moderate [8, 1214] to good validity [5, 8, 1518] of the questionnaires. However, none of these instruments or the instruments reviewed by Kim and Holowaty (2003) were computerised or online questionnaires. Internet use is strongly increasing and new technologies for dietary assessment offer important advantages (e.g. easy processing of data, producing immediately results and increasing privacy and confidentiality)[19]. Therefore, it is necessary to also develop valid and reliable, brief online questionnaires that measure FV intake. The Fruit Test (FT) and Vegetable Test (VT) evaluated in this study, are brief, online questionnaires developed by the Flemish institute for Health Promotion and Disease Prevention (VIGeZ) to measure vegetable intake and fruit intake in Flemish adults[20]. A pre-test of the FT and VT was conducted to ensure clarity and ease of completion/interpretation of the FT and VT[20]. Furthermore, as suggested by Kim and Holowaty (2003), a moderate number of representative fruit and vegetable items, questions on portion size, food items highly tailored to the dietary behaviors and local food items of the population surveyed were included in the VT and FT[10]. In Belgium, these questionnaires include different kinds of fruit and vegetables frequently eaten in a Western diet, based on Belgian food consumption data [2123]. Which makes the questionnaire highly tailored to local dietary behaviors and food items. Still, the measures are short (<16 items) and assess FV intake over seven days (the past week). Both the FT and VT were already evaluated by 116 and 70 adults, respectively. Adults positively scored the FT and VT with regard to feasibility and acceptability (good length, time needed, good structure, clear and relevant questions, easy to fill in and easy to understand)[20]. Despite these promising results, the reliability and validity of the FT and VT needs further scrutiny, to justify their use in computer-tailored programs. The first objective of this study was to investigate the stability and consistency of the FT and VT over time. Therefore, the test-retest reliability of the FT and the VT in Flanders was examined[24]. The second objective of the present study was to examine concurrent criterion validity of the FT and VT in Flemish adults by testing their agreement with 7-day food diary-derived measures of FV intake.

If the questionnaires would be sufficiently reliable and valid to measure FV intake, they can be considered useful for future research and interventions. Since suggestions to enhance reliability and validity were followed[10], we hypothesised the FT and VT to be valid and reliable online questionnaires.

Methods

Participants and procedure

Convenience sampling was used to recruit adults (> = 18 years) by handing out information letters to friends, family and university students. Participants who agreed to participate, signed an informed consent and received an email with more information on the study. The study lasted for three weeks and was conducted in September 2014. In week one, adults completed the FT and VT for the first time. In the second week, adults completed the FT and VT for the second time. Each time, adults first completed the FT and then immediately the VT. In week three, adults recorded their FV intake in a 7-day food diary. Email reminders were used to remind participants to fill in the 7-day food diary, the FT and VT. Test-re-test reliability was evaluated by having participants fill in the FT and VT twice with a time-interval of one week. Based on previous studies, the questionnaires were validated against a previously validated 7-day food diary [25, 26]. The study was approved by the Ghent University Ethics Committee (approval number: B670201422475).

The Fruit test and the Vegetable test

The FT and the VT were developed by the Flemish institute for Health Promotion and disease prevention (VIGeZ) in Belgium. This study was conducted in the context of the eHealth intervention ‘MyPlan 1.0’, a computer-tailored intervention based on self-regulation that aims to increase FV intake of Flemish adults. We aim to incorporate the FT and VT in ‘MyPlan 1.0’ to measure mean FV intake per week and to provide feedback in which the mean FV intake of adults’ is compared with health guidelines[27].

Figs 1 and 2 illustrate the FT and VT, respectively. First, participants were asked on how many days in the past week (past seven days) they ate fruit (FT) and vegetables (VT). Next, if participants ate fruit/vegetables on one or more days, a list with frequently eaten fruits/vegetables in a Western diet was displayed on the screen. The displayed fruits and vegetables were based on the Belgian food table and consumption data[21]. For each type of fruit or vegetable, portion sizes and household sizes were mentioned (e.g. 1 cherry = 4 gram, 1 dessert plate of berries weighs about 100 grams). Participants were instructed to indicate for each type of fruit or vegetables the number of portions they ate during the past seven days.

thumbnail
Fig 1. Fruit test.

Items in the Fruit Test are grouped based on similar portion sizes. Via the dropdown, participants can indicate the amount of portions of fruit they ate.

https://doi.org/10.1371/journal.pone.0159834.g001

thumbnail
Fig 2. Vegetable Test.

Items in the Vegetable Test are grouped based on similar portion sizes. Via the dropdown, participants can indicate the amount of portions of vegetables they ate.

https://doi.org/10.1371/journal.pone.0159834.g002

The reported portions of fruit/vegetables were multiplied with the portion size of the corresponding types of fruit/vegetables to calculate the average grams per week. To calculate the average portion size per day, the average grams per week were divided by seven (see formulas in Table 1). Portion sizes and household sizes were based on prescriptions of the Belgian Superior Health council[23].

thumbnail
Table 1. formula total portion of vegetables and fruit per day.

https://doi.org/10.1371/journal.pone.0159834.t001

7-day food diary

Participants were instructed to weigh fruit and vegetables (in grams) and to record this every day at home in a 7-day food diary. Two dietitians reviewed the diaries with participants for completeness and coded the diaries. To ensure construct validity, definitions of fruit and vegetables types included in the FT and the VT were similar to definitions used to code vegetables and fruits species in the 7-day food diary. These definitions were derived from the Flemish active food triangle[22, 28]. Items in the FT and VT were grouped together based on similar portion sizes. If it was impossible to weigh vegetables or fruit (e.g. at a restaurant), participants were allowed to use household sizes (e.g. a spoon, half of a plate, slices, …). In case household sizes were used, the same portion sizes as used in the VT and FT were used to calculate the amount of grams (See Table 1). Total FV intake per day was measured by summing all recorded grams and calculated grams (based on household sizes) and by dividing this sum by seven.

Statistics

SPSS21 (SPSS Inc., Chicago, IL, USA) was used to perform statistical analyses. The outcome variables (total fruit intake per day and total vegetable intake per day) were first checked for normality by plotting a normal score plot.

Intra class correlations (ICC) of total FV intake measured at week one and week two were used to evaluate reliability. To interpret the ICC values, ratings by Landis and Koch[29]: 0.00–0.20 (poor), 0.21–0.40 (fair), 0.41–0.60 (moderate), 0.61–0.80 (substantial), 0.81–1.00 (excellent) were used. Participants (n = 2) who did not complete the FT/VT two times were not included in the analysis.

Because the distribution of the data was not normal, non-parametric spearman rank correlations coefficients (ρ) were used to evaluate validity of the questionnaires. Correlations of 0.30 to 0.40 were considered as good validity, correlations of 0.20 to 0.30 were considered as moderate validity and correlations lower than 0.20 were considered as not valid[30]. In addition, absolute agreement between FV intake measured by the FT/VT and FV intake measured by the 7-day food diary was calculated. As the data were not normally distributed, we used a non-parametric approach, as suggested by Bland-Altman [31, 32]. A non-parametric plot was interpreted, one for fruit intake, and one for vegetable intake. For the plot of FT, the difference between fruit intake measured by the FT and measured by the 7-day food diary was calculated and expressed as a percentage of difference (reported fruit by the FT as a percentage of 7-day food diary data). This percentage of difference was plotted on the y-axis against the average of fruit intake derived via the 7-day food diary on the x-axis. The 5th and 95th percentiles of the percentage-of-difference measures were calculated and subsequently inserted on the scatter plot, reflecting 90% limits of agreement (LOA). An identical procedure was followed for vegetable intake.

Results

Participant characteristics

In total, 58 adults (31% male, M age = 30y, ±12.09) participated in this study. Only two participants did not fill in the FT and VT for the second time. Fruit was consumed on a daily basis by 43.1% of the adults. Daily average intake of fruit was 166.0 (±94.80) grams. About half of the participants (53.4%) ate vegetables on a daily basis and adults ate on average 139.9 (±82.99) grams of vegetables per day.

Test-retest reliability

The test-retest reliability study (n = 56) indicated that the FT showed excellent reliability (ICC = 0.81) for vegetable consumption. The VT also showed substantial reliability (ICC = 0.78) for vegetable consumption.

Criterion (concurrent) validity

Spearman rank correlation coefficients between self-reported fruit intake in the diary and self-reported fruit intake via the FT showed good validity (ρ = 0.73). In terms of percentage of difference between fruit intake reported in the FT and the 7-day food diary, the median difference was -17,61%. This indicates a small under-estimation of fruit intake in the FT against the 7-day food diary. Fig 3 provides the results on the criterion validity for the FT. Results indicate a modest variability in differences in fruit intake measured by the FT and the 7-day food diary. There is a higher variability at the lowest measured fruit intake in the 7-day food diary and lower variability at the highest measured fruit intake in the 7-day food diary.

thumbnail
Fig 3. Non-parametric plot for fruit intake.

Difference between fruit intake reported in the FT and the 7-day food diary. Y-axis represent these differences as a percentage of difference (reported fruit by the FT as a percentage of 7-day food diary data); x-axis represent fruit intake as measured by the 7-day food diary. Full lines represent median (M) percentage of difference; dotted lines show the 90% nonparametric limits of agreement (LOA), representing 5th and 95th percentiles (P5 and P95): M = -17,61; P5 = -60,68; P95 = 160,00).

https://doi.org/10.1371/journal.pone.0159834.g003

Comparison between self-reported vegetable intake in the diary and self-reported vegetable intake in the VT showed good validity (ρ = 0.52). In terms of percentage of difference between vegetable intake reported in the VT and the 7-day food diary, the median difference was -28,27%. This indicates a small under-estimation of vegetable intake in the VT against the 7-day food diary. Fig 4 illustrates the criterion validity results regarding the VT in more detail. It shows a modest variability in differences in vegetable intake measured by the VT and the 7-day food diary. There is a higher variability at the lowest measured vegetable intake in the 7-day food diary and lower variability at the highest measured vegetable intake in the 7-day food diary.

thumbnail
Fig 4. Non-parametric plot for vegetable intake.

Difference in vegetable intake as measured by the VT and the 7-day food diary. Y-axis represent these differences as a percentage of difference (reported vegetables by the FT as a percentage of 7-day food diary data); x-axis represent vegetable intake as measured by the 7-day food diary. Full lines represent median (M) percentage of difference; dotted lines show the 90% nonparametric limits of agreement (LOA), representing 5th and 95th percentiles (P5 and P95): M = -28,27; P5 = -69,77; P95 = 91,82).

https://doi.org/10.1371/journal.pone.0159834.g004

Discussion

The short online FT and VT have already shown to be feasible questionnaires to measure FV intake in adults[20]. The aim of the current study was to test their test-retest reliability and criterion validity among Flemish adults.

Reproducibility of a questionnaire can be influenced by the questionnaire itself, respondents’ memory and actual changes in food intake[8]. Nevertheless, our results indicated that test-retest reliability of the FT and VT questionnaires was good.

Average fruit intake (166 gram/day) measured by the FT was comparable to average fruit intake in Europe (European Food Information Council (EUFIC, 2012) (166 gram/day)[33], but higher than in the Belgian Health Survey (average intake of 118 gram/day)[34]. Average vegetable intake measured with the VT (140 gram/day) was comparable to average vegetable intake in the Belgian health survey study (138 gram/day)[34], but lower than in the European Food Information Council (EUFIC) study in 2012 (220 gram/day)[33]. Good correlations ((FT (ρ = 0.73); VT (ρ = 0.52)) were found between FV intake consumption over seven days reported in the diet records and the FT and VT. A review on validated survey instruments of FV intakes in adults observed correlations around 0.40 to 0.50. However, these results should be cautiously interpreted, since they can only suggest that the FT and VT were relatively good at ranking individuals by their reported FV intake, when compared to FV intake reported in the 7-day food diary[35, 36]. Therefore, to evaluate validity of the VT and FT, Spearman rank order correlations were also combined with agreement plots. The plots for the VT and the FT showed a modest variability in differences in FV intake measured by the FT/VT and the 7-day food diary. These results also indicate a good validity of the FT and VT. Other validity studies of FFQs have reported that detailed food questionnaires containing many food items were not valid due to over-reporting of food consumption[5]. In contrast, the plots in our study showed a small under-reporting of FV intake measured by the FT and VT against the 7-day food diary. This may indicate that overestimation can be tempered by using brief questionnaires that include a moderate number of representative fruit and vegetable items, questions on portion size, mentioning household sizes with sufficient detail (e.g. 1 serving of raw vegetables = about 50 grams = a dessert plate of lettuce) and by including food items highly tailored to the dietary behaviors and local food items of the population surveyed[10]. Based on this study and the review of Kim and Holowaty (2003), we also suggest to further incorporate portion size pictures and consumption of mixed vegetables to decrease under-estimation in order to further improve validity of the FT and VT[10]. To use the FT and VT as measurement instruments in other countries, it might be that some adaptations are needed, since only commonly eaten fruits and vegetables in a Western diet (based on Belgian consumption and household purchase data) were included in the FT and VT[2123]. The questionnaires can easily be adapted for the use beyond Europe (for example in an Asian diet), by adding other commonly eaten fruit or vegetables. However, to ensure generalizability, we recommend to re-evaluate reliability and validity of the questionnaires when adapting or translating it.

Some study limitations need to be acknowledged. In this study, both the FT and VT and the diet record (golden standard) were subjective methods to measure FV intake. This can lead to social desirability and recall bias [12, 37, 38]. To prevent social desirability and recall bias, it is often suggested to measure fruit and vegetable objectively, by using biomarkers(e.g. plasma concentrations of vitamin C)[8, 38]. However, biomarkers also have some limitations. Participants need to provide blood samples and plasma concentrations of carotenoids and vitamin C are also influenced by biologic factors, plasma cholesterol, body mass index, sex, smoking and vitamin supplement use, which can lead to modest validity[8, 38, 39].

Another limitation of our study is that we did not take into account certain individual factors that may have affected the results, such as socio-economic status. Since we used a convenience sample of adults, our sample may have consisted mainly of adults with a high socio-economic status. Adults with lower SES might experience more difficulties to fill in the FT and VT, which can influence validity of the questionnaires when they are used in these populations.

The FT/VT and the 7-day food diary were also not assessed at the same time, which makes it impossible to disentangle what is the variation in the two measures that is caused by the instrument itself and what is the variation that may have been caused by variations in diet in two different weeks [12, 40, 41]. But, if the diary is used before or at the same time when the FT and VT are assessed, the recording process in the diary could improve the recall of fruits and vegetables in the VT and FT. This may augment the correlations between results of the two methods[24, 42, 43]. To limit variation in diet in two different weeks due to other influences (e.g. changes in seasons, influences from advertisements), we kept the study period as short as possible. The study was conducted in the winter only and a short period (1 week) was used between the test-retest administrations.

In conclusion, this study shows that the FT and the VT are valid and reliable tools to measure FV intake in adults. Since these online questionnaires were already been evaluated as feasible (i.e. having a good length, easy to fill in and clear), they can be considered as valuable tools to measure FV intake of adults in future eHealth interventions. However, when using the questionnaire in different population groups, a re-evaluation of the reliability and validity of the questionnaire is recommended.

Acknowledgments

We would like to thank the Flemish institute for Health Promotion and Disease Prevention (VIGeZ) for developing the Fruit Test and Vegetable Test. Furthermore we thank Kimberley Breugelmans and Eva Meeus (dietitian) for conducting the field work.

Author Contributions

Conceived and designed the experiments: JP. Performed the experiments: JP. Analyzed the data: JP MV. Contributed reagents/materials/analysis tools: SS LD EV. Wrote the paper: JP MV GC IDB.

References

  1. 1. WHO. Diet, Nutrition and the prevention of Chronic diseases, Report of Joint WHO/FAO Expert Consultation. 2003.
  2. 2. Drieskens S. Voedingsstatus. [Health Status]. In Gezondheidsenquête 2013. Rapport 2: Gezondheidsgedrag en leefstijl. [Health survey 2013. Report 2: Health behaviour and life style]. Gisle L, Demarest S (editors). WIV-ISP, Brussel, 2014.
  3. 3. Broekhuizen K, Kroeze W, van Poppel MN, Oenema A, Brug J. A systematic review of randomized controlled trials on the effectiveness of computer-tailored physical activity and dietary behavior promotion programs: an update. Ann Behav Med. 2012;44(2):259–86. pmid:22767052; PubMed Central PMCID: PMC3442159.
  4. 4. Vandelanotte C, De Bourdeaudhuij I, Philippaerts R. Reliability and Validity of a Computerized and Dutch Version of the International Physical Activity Questionnaire (IPAQ). Journal of Physical Activity and Health. 2005; 2; 63–75.
  5. 5. Kristjansdottir AG, Andersen LF, Haraldsdottir J, de Almeida MD, Thorsdottir I. Validity of a questionnaire to assess fruit and vegetable intake in adults. European journal of clinical nutrition. 2006;60(3):408–15. pmid:16306927.
  6. 6. Haerens L, Deforche B, Vandelanotte C, Maes L, De Bourdeaudhuij I. Acceptability, feasibility and effectiveness of a computer-tailored physical activity intervention in adolescents. Patient education and counseling. 2007;66(3):303–10. pmid:17298873.
  7. 7. Vandelanotte C. Acceptability and feasibility of a computer-tailored physical activity intervention using stages of change: project FAITH. Health education research. 2003;18(3):304–17. pmid:12828232
  8. 8. Bogers RP. Reproducibility, Validity, and Responsiveness to Change of a Short Questionnaire for Measuring Fruit and Vegetable Intake. American Journal of Epidemiology. 2004;159(9):900–9. pmid:15105183
  9. 9. Brouwer W, Oenema A, Crutzen R, de Nooijer J, de Vries NK, Brug J. What makes people decide to visit and use an internet-delivered behavior-change intervention? Health Education. 2009;109(6):460–73.
  10. 10. Kim DJ, Holowaty EJ. Brief, validated survey instruments for the measurement of fruit and vegetable intakes in adults: a review. Preventive medicine. 2003;36(4):440–7. pmid:12649052
  11. 11. Applied Research: Register of validated short dietary assessment instruments. Available: http://appliedresearch.cancer.gov/diet/shortreg/register.php. Accessed May 2016.
  12. 12. Yaroch AL, Tooze J, Thompson FE, Blanck HM, Thompson OM, Colon-Ramos U, et al. Evaluation of three short dietary instruments to assess fruit and vegetable intake: the National Cancer Institute's food attitudes and behaviors survey. Journal of the Academy of Nutrition and Dietetics. 2012;112(10):1570–7. pmid:23017567; PubMed Central PMCID: PMC3775662.
  13. 13. Lean ME, Anderson AS, Morrison C, Currall J. Evaluation of a dietary targets monitor. European journal of clinical nutrition. 2003;57(5):667–73. pmid:12771967.
  14. 14. Resnicow K, Odom E, Wang T, Dudley WN, Mitchell D, Vaughan R, et al. Validation of Three Food Frequency Questionnaires and 24-Hour Recalls with Serum Carotenoid Levels in a Sample of African-American Adults. Am J Epidemiol. 2000; 152(11). pmid:11117617.
  15. 15. Mainvil LA, Horwath CC, McKenzie JE, Lawson R. Validation of brief instruments to measure adult fruit and vegetable consumption. Appetite. 2011;56(1):111–7. pmid:20970470.
  16. 16. Greene GW, Resnicow K, Thompson FE, Peterson KE, Thomas GH, Hebert JR. Correspondence of the NCI Fruit and Vegetable Screener to Repeat 24-H Recalls and Serum Carotenoids in Behavioral Intervention Trials. J. Nutr. 2008; 138: 200–204. pmid:18156425.
  17. 17. Thompson FE, Midthune D, Subar AF, Kahle LL, Schatzkin A, Kipnis V. Performance of a short tool to assess dietary intakes of fruits and vegetables, percentage energy from fat and fibre. Public health nutrition. 2004;7(8):1097–105. pmid:15548349.
  18. 18. Alcantara I, Haardorfer R, Gazmararian JA, Hartman TJ, Greene B, Kegler MC. Relative validation of fruit and vegetable intake and fat intake among overweight and obese African-American women. Public health nutrition. 2015;18(11):1932–40. pmid:25412867.
  19. 19. Albar SA, Alwan NA, Evans CE, Greenwood DC, Cade JE. Agreement between an online dietary assessment tool (myfood24) and an interviewer-administered 24-h dietary recall in British adolescents aged 11–18 years. The British journal of nutrition. 2016:1–9. pmid:26975650.
  20. 20. VIG Gezondheidsbevordering op het werk. Resultaten en verslag van de productevaluatie Voeding: fruittest, groententest en vochttest. [Health promotion at work. Results and report of product evaluation Nutrition: fruit and vegetable test]. Available: http://www.gezondheidstest.be/testjes/VD_productevaluatie_tests.pdf. Accessed June 2015.
  21. 21. NUBEL vzw (2004) Belgische voedingsmiddelentabel. [Belgian food table]. Brussel.
  22. 22. Vanhauwaert E (2012) De Actieve Voedingsdriehoek. [Active Food Triangle]. Acco: Leuven.
  23. 23. Gezondheidsraad Hoge [Belgian Superior Health council] (2009) Voedingsaanbevelingen voor België. [Food recommendations for Belgium] Brussel. Available: http://www.vigez.be/files/voedingenbeweging/publicatie_hoge_gezondheidsraad_2009.pdf. Accessed June 2015.
  24. 24. Portney LG, Watkins MP. Foundations of Clinical Research. 3rd ed. New Jersey: Pearson Education, Inc.; 2009.
  25. 25. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires—a review. Public health nutrition. 2002;5(4):567–87. pmid:12186666.
  26. 26. De Keyzer W, Huybrechts I, De Vriendt V, Vandevijvere S, Slimani N, Van Oyen H, et al. Repeated 24-hour recalls versus dietary records for estimating nutrient intakes in a national food consumption survey. Food & nutrition research. 2011;55. pmid:22084625; PubMed Central PMCID: PMC3215303.
  27. 27. Plaete J, De Bourdeaudhuij i, Verloigne M, Oenema aA, Crombez G. Systematic development of an ehealth intervention that targets self-regulation processes to increase physical activity and fruit and vegetable intake through general practice. jmir res protoc 2015 (dec 22); 4(4):e141.
  28. 28. VIGeZ. De voedingsdriehoek: een praktische voedingsgids. [The Food Triangle: a practical food guide]. Flemish institute for Health Promotion and Disease Prevention (VIGeZ); 2016. Available: http://www.vigez.be/projecten/actieve-voedingsdriehoek.
  29. 29. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics. 1977; 33; 159–174. pmid:843571
  30. 30. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. New Jersey: Lawrence Erlbaum Associates, 1988.
  31. 31. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986;1:307–10. pmid:2868172
  32. 32. Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8:135–60. pmid:10501650
  33. 33. European food information council (EUFIC). Fruit and vegetable consumption in Europe: Do Europeans get enough? 2012. Available: http://www.eufic.org.
  34. 34. Vandevijvere S, De Vriese S, Huybrechts I, Moreau M, Temme E, De Henauw S, et al. The gap between food-based dietary guidelines and usual food consumption in Belgium, 2004. Public health nutrition. 2009;12(3):423–31. pmid:18426635.
  35. 35. Van Holle V, De Bourdeaudhuij I, Deforche B, Van Cauwenberg J, Van Dyck D. Assessment of physical activity in older Belgian adults: validity and reliability of an adapted interview version of the long International Physical Activity Questionnaire (IPAQ-L). BMC public health. 2015;15:433. pmid:25928561; PubMed Central PMCID: PMC4427934.
  36. 36. Cerin E, Cain KL, Oyeyemi AL, Owen N, Conway TL, Cochrane T, et al. Correlates of Agreement between Accelerometry and Self-reported Physical Activity. Medicine and science in sports and exercise. 2016. pmid:26784274.
  37. 37. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutrition research reviews. 2004;17(1):5–22. pmid:19079912.
  38. 38. Natarajan L, Flatt SW, Sun X, Gamst AC, Major JM, Rock CL, et al. Validity and systematic error in measuring carotenoid consumption with dietary self-report instruments. Am J Epidemiol. 2006;163(8):770–8. pmid:16524958.
  39. 39. Mayne ST. Antioxidant Nutrients and Chronic Disease: Use of Biomarkers of Exposure and Oxidative Stress Status in Epidemiologic Research. J. Nutr. 2003; 133, 933S. pmid:12612179.
  40. 40. Laviolle B, Froger-Bompas C, Guillo P, Sevestre A, Letellier C, Pouchard M, et al. Relative validity and reproducibility of a 14-item semi-quantitative food frequency questionnaire for cardiovascular prevention. European Journal of Cardiovascular Prevention and Rehabilitation 2005, 12:587–595. pmid:16319550.
  41. 41. Johnson B, Hackett A, Roundfield M, Coufopoulos A. An investigation of the validity and reliability of a food intake questionnaire. J Hum Nutr Dietet. 2001; 14, pp. 457–465. pmid:11906588.
  42. 42. Rangan AM, Tieleman L, Louie JC, Tang LM, Hebden L, Roy R, et al. Electronic Dietary Intake Assessment (e-DIA): relative validity of a mobile phone application to measure intake of food groups. The British journal of nutrition. 2016;115(12):2219–26. pmid:27121045.
  43. 43. Glover JA. The "Testing" Phenomenon: Not Gone but Nearly Forgotten. Journal of Educational Psychology 1989; 81(3), 392–399.