Relative Validation of a Four Weeks Retrospective Food Frequency Questionnaire versus 7-Day Paper-Based Food Records in Estimating the Intake of Energy and Nutrients in Adults

Introduction

To properly explore short-term nutrition intakes, accurate and reproducible instruments for short-term dietary assessment are needed. The accuracy of methods estimating dietary intakes has also to be validated in different populations and regions due to the potential biases introduced by different cultural and geographical backgrounds and feeding practices.¹

Food records, typically 4–7 days, are one of the recognized methods for capturing dietary intake,^2,3 but it is difficult to be used in large epidemiological studies since the procedure requires adequate literacy on behalf of responders and it is, also, time and resources consuming on behalf of the investigators. Alternatively, food frequency questionnaires (FFQs) have been successfully developed and used to estimate dietary intakes, for all age categories of people^4–6 and in different languages.^7–11 Compared to other methods of dietary intake assessment, the completion of FFQs takes significantly less time and resources, especially in the self-administered form. Although sometimes prone to bias by over- or underestimating some nutrient intakes, FFQs remain a facile instrument for the purpose for which they were validated (eg complete dietary assessment for energy/macro/micronutrients or partial assessment for selected macro/micronutrients in population studies).¹²

Presently, few dietary assessment instruments have been developed and validated for the Romanian population, although such instruments are needed to plan nutritional education programs and healthy eating interventions and for research purposes. Moreover, recent studies have demonstrated a low theoretical level of nutrition knowledge in the Romanian adult population¹³ and overall dietary patterns characterized by overconsumption of fat-derived energy intake and inadequacy of intake for most micronutrients, not only in the general population¹⁴ but also among overweight and obese people who are trying to lose weight.¹⁵

This study determined whether, for short-term dietary assessment, FFQs and 7-day food records can be used interchangeably in Romanian participants, and if so, what are the limitations of energy, macronutrients, micronutrients and food groups assessment. That is, for which outcomes these two methods can be used with similar results and in which circumstances significant differences may prevent the indiscriminate use of either method, but instead would require additional insight in selecting a certain method when designing a nutritional study.

Patients and Methods

The FFQ applied by Haftenberger et al⁷ in the German National Nutrition Monitoring (NEMONIT) was used in this study. This FFQ was translated into Romanian and then back to German by two independent translators. The two versions, the German original and the backward-translated version, were found to be similar by a native German speaker with a medical background. A questionnaire coming from the German-speaking space was used since it was recognized by the authors, during the initial feasibility stage of the study that the 53 food items, their description and the portion sizes, as were used by the German team are fitted to be easily understood by Romanian participants.^16,17 Therefore, all the questions were kept as in the original questionnaire.⁷ Briefly, the questionnaire explores the intake of 53 common food items, consumed during the previous four weeks. Each food item contains 11 categories of frequency, an ordinal quantity per serving and, for some foods, additional questions about fat content, added sugar or types of cooking (Supplementary material –English and Romanian Version of the Food Frequency Questionnaire – En-FFQ/Ro-FFQ).

To assure high-quality data collection, the paper-based Romanian version of the questionnaire was used to develop a Google Forms online tool, which was further pre-tested. A test group consisting of 12 volunteers assessed the compatibility of the questionnaire in computer/hand-held device (telephone, tablet) environments and the conditional function attached to the main questions. For ease of completion, if, for a given food, the frequency of intake was “never” during the last 4 weeks, the tool would skip the questions related to the quantity and any supplementary question on that particular food, reducing the total number of questions a responder would answer, by employing the function of conditional answers.

For 7-day food record a paper-based collection method. These two methods of dietary assessment were compared concerning energy, macro- and micronutrients intakes and food groups consumption.

Ethics Approval

The research protocol was approved in January 2019 by the Institutional Review Board of the Victor Babes University of Medicine and Pharmacy Timisoara, Romania. Before any study procedure, all subjects provided written informed consent.

Study Population

Participants were recruited from February 2019 until October 2019, with a pause around Easter and summer holidays, to avoid food-abundant periods and different patterns of feeding. The final sample size of 118 participants was in agreement with Willet’s recommendations for validation studies (between 100 and 200 participants).¹⁸ Participants were recruited, in the first wave, from the social network of researchers, followed by other waves of recruitment from enlarged circles. Participants were incentivized into participating by being offered a full nutritional analysis of their 7-day food records. Each participant acted in his/her control, as subjects also completed the FFQ. No specific inclusion/exclusion population criteria were formulated, except the need for signed informed consent, which was obtained at the recruitment visit from 190 participants (Figure 1).

Figure 1 Flow chart of the subjects’ recruitment process.

Study Design and Measurements

Instructions were issued at the recruitment visit, along with the printed forms for semi-weighted food record collection for 7 days, on how to document weight, volume and/or standard measurements of food and beverage. For each entry in the record, the following information was requested: time, name of the food or brand name, composition for complex foods and quantity in grams or volume by use of written instructions and images representing standard servings. Dietary supplements taken by participants for the week included in the food record were also self-declared by indicating the brand, the quantity and frequency of intake, and were added to the final nutrient intake database. Demographic variables included sex, age and educational level. Instructions on how to self-measure weight and height were also given to participants and were further used to calculate body mass index (BMI). Participants with a BMI equal to or greater than 25 kg/m2 were classified as being overweight and/or obese. To increase the quality of the food records,¹⁹ an experienced dietitian kept close contact via phone with all participants who required assistance during the week of reporting and, at the end of the week, the dietitian checked the completion of the records. Food records that were incomplete (not having 7 consecutive days) and/or were low in details about complex food composition (eg missing fat content for dairy and meats, no composition of certain recipes such as soups or stews), were rejected and the participants were excluded from the study. The exclusion was performed at this step and not during the analysis stage, and these rejected food records were never converted to energy and nutrients, to spare the overload of the team. Participants were further invited to fill in the online FFQ in a maximum period of 3 weeks after the completion of the record so that the week of record overlaps with the 4 weeks for the retrospective FFQ. 116 participants have succeeded in offering both complete 7-day records and timely answers to FFQ and were, therefore, included in the final analysis (Figure 1).

Coding of Nutrient Intakes from Consecutive 7-Day Food Records

For each participant and each day reported in the record, all foods and drinks, including their amounts, were converted to energy, macro- and micro-nutrients using a web-application (Nutritio, Naturalpixel SRL, Bucharest, Romania, https://nutritioapp.com).²⁰ The procedure was described in detail elsewhere.¹⁵ Briefly, the platform uses algorithms based on the USDA (United States Department of Agriculture) Food and Nutrient Database for Dietary Studies, European Databases and other databases for local foods. For those foods that are folate-fortified in the USDA database, the app used identical items from European food databases. For each of the 7 days, energy content and macro- and micronutrient composition of reported intakes were calculated and further averaged as mean estimated daily intakes that were used in subsequent analyses. The reported intakes from 7-day food records were also transformed into quartiles and energy-adjusted, then further used in statistical analyses.

The food items declared in the food records were also assigned to food groups with the purpose of further using the results for the calculation of healthy eating index (Fruits, Natural juices, Vegetables, Legumes, Potatoes, Whole grain, Refined grains, Dairy, Eggs, Poultry, Red meat, Fish, Nuts and seeds, Alcoholic, Non-alcoholic bottled drinks, processed meat), according to a procedure described elsewhere.²¹

Obtaining Estimated Nutrient Intakes from Semi-Quantitative FFQ

To estimate nutrient intakes for each of the 53 food items, one or more representative food items, from the USDA Food and Nutrient Database for Dietary Studies, were selected. The selection was carefully done, excluding foods that were folate fortified. Total estimated energy, macro- and micronutrient monthly intakes, obtained by summation of values obtained from all food groups, were used to obtain daily intake. The reported intakes from the FFQ were also transformed into quartiles and energy-adjusted, then further used in statistical analysis.

For both nutritional assessment methods, macro- and micronutrients were presented with raw and the energy-adjusted values per 1000 kcal of reported intake.

To compare the two intake assessment methods, some of the food items of the FFQ were merged so they would fit with the major food groups defined by the food records. Each item of the 53 items was assigned by 2 independent researchers to food groups. Differences were discussed, and the final assignment was further used for the calculation of intake per food groups as follows: Fruits (2 items), Natural juices (2 items), Vegetables (2 items), Legumes (1 item), Potatoes (4 items), Whole grain (3 items), Refined grains (8 items), Dairy (4 items), Eggs (1 item), Poultry (1 item), Red meat (1 item), Fish (2 items), Nuts and seeds (1 item), Non-alcoholic bottled drinks (7 items), Pure alcohol g/day (4 items), Processed meat (g/day) (3 items). Examples are as follows: fresh fruits and conserved fruits were merged as fruits, juices from fruits and juices from vegetables were merged as natural juices, fresh and cooked vegetables as coloured vegetables, legumes as legumes, boiled, fried, baked potatoes and potato chips as potatoes, whole-grain breakfast cereals, whole-grain bread and whole-grain pasta as whole grain cereals, etc. This procedure was done according to the previous method used in computing a Healthy Eating Index.²¹

For either method (food records or FFQ) no implausible food intakes were found (defined as <800 kcal/day or >4200 kcal/day for men and <600 kcal/day or 3500 kcal/day for women), as potential participants with missing information and implausible food records (25 participants with missing information) were excluded in the subject recruitment stage and their food records were not further converted to energy and nutrients, as described above (Figure 1).

Statistical Analysis

The IBM-SPSS software version 18 (IBM, Armonk, New York, USA) was used for statistical analyses. Numerical variables were presented as median (interquartile range), because they failed normality assumptions, as tested using the Kolmogorov-Smirnov test. Categorical variables were presented as frequency (%/n) of participants in the categories. Mann–Whitey test and chi-square test were used to compare the demographic characteristics in men versus women. Raw nutrient intakes and the amount of food in each food group were transformed into quartiles and the proportion of agreement and kappa coefficients were further computed.

Spearman correlation coefficients were computed for nutrient intakes and food groups from the FFQ and 7-day records and interpreted as follows: <0.30 poor or no correlation, 0.3–0.49 fair, 0.5–0.69 moderate and >0.7 large.²² Cohen’s kappa statistic (K) values were used for comparing the degree of agreement of quartiles of intake for each nutrient and food group from FFQ and 7-day food records. The following criteria were used to evaluate agreement by K between the dietary methods: >0.80 – very good agreement, 0.61–0.80 good agreement, 0.41–0.60 moderate agreement, 0.21–0.40 fair agreement and <0.20 poor agreement, as described elsewhere.²³ For each participant, the agreement between quartiles for each of the two assessment methods was calculated as the percentage of subjects in the same quartile, in the adjacent (one quartile difference) and/or in the opposite quartile (two or more quartile difference). Bland–Altman plots were used to evaluate the visual agreement between the two dietary methods of assessment; the mean unadjusted intakes of selected macro/micronutrients from both intake assessment methods (7-day food records and FFQ) were plotted against the difference between the two methods and regression was used to estimate the magnitude of disagreement and to spot outliers and potential trends, by assessing the number of individuals outside the limit of agreement ±1.96 SDs from the mean of the difference between the methods of intake assessment.

Results

Demographic and anthropometric characteristics of the 116 participants who provided full data (Figure 1) are presented in Table 1. The sample with a median age of 25.5 years and an interquartile range of 15 years were largely composed of women, who represented nearly 70%. Overweight and obese participants accounted for 26.8% of the sample and the proportion of participants with at least some college degree represented 73.3% of all participants. Except for BMI, which was higher in males than in females and significant proportions of males were found in higher BMI categories, for other demographic characteristics of the study population, the differences were not statistically significant.

Table 1 Characteristics of Study Participants by Sex (N=116)

Since no significant interaction effects of sex and age were noted (data not shown), participants were not stratified by sex and age in further analyses.

Central tendency (Table 2) was expressed as the median (interquartile range) of energy, nutrient intakes and energy-adjusted intakes from the 7-day food records and the 4-week retrospective FFQ. Spearman correlation coefficients and energy-adjusted correlation coefficients are also reported in Table 2. Correlation coefficients between food records and FFQ for energy and nutrients ranged from 0.286 for calcium to 0.782 for total fat. Energy-adjusted correlation coefficients ranged from 0.223 for magnesium to 0.614 for choline.

Table 2 Unadjusted and Adjusted Intakes of Energy and Nutrient Intakes from the 7-Day Food Records and 4-Week Retrospective FFQ (n = 116)

Table 2 also shows the results of the cross-classification analysis of study participants by quartiles of absolute intakes, obtained from the food records and the FFQ. Cohen’s kappa values for energy and macronutrients are indicated in Table 3 and ranged between 0.402 for protein and 0.470 for fat, while for micronutrients, the kappa values were between 0.08 for calcium and 0.33 for iron. Regarding cross-classification into quartiles using data from FFQ and 7-day food records, for energy and macronutrients, on average, less than 10% of observations were grossly misclassified. Most of the participants were classified in the same quartile (ranging from 55.17% for protein to 60.52% for energy) or in the adjacent quartile (ranging from 31.1% for fat to 35.3% for carbohydrates). For micronutrients, a higher degree of misclassification was found, with 22.4% of participants being grossly misclassified. For micronutrients, the same quartile percentage ranged from a minimum of 31% for calcium to a maximum of 50% for iron.

Table 3 Adjusted and Unadjusted Spearman’s rho, Percentage of Participants Classified into the Same, Adjacent or Opposite Quartile by the 7-Day Food Records and 4-Week Retrospective FFQ and the Cohen’s Kappa (n = 116)

Bland–Altman plots were used to visually assess the overall agreement between FFQ and 7-day food records for energy (Figure 2A) and macronutrient intakes (Figure 2B–D). No more than 9 observations were outside the limits of agreement for any energy and macronutrients. Slopes (95% CI) of the regression, using intakes from food records as independent variables and those of the difference between food records and FFQ, were: 0.028 (−0.031, 0.086), 0.058 (−0.067, 0.183), 0.047 (−0.097, 0.191), −0.025 (−0.226, 0.175) for energy, protein, fat and carbohydrate, respectively.

Figure 2 Bland–Altman plot with the difference in energy and nutrient intakes measured with the 7-day food records and 4 weeks retrospective FFQ plotted against the mean intakes from the two methods. (A) energy intake (kcal); (B) protein intake (g); (C) fat intake (g); (D) carbohydrate intake (g). Notes: The solid line indicates the regression line, the serried dashed line indicates the mean difference between the methods, while the spaced dashed lines indicate ±1.96 SDs.

Bland–Altman plots were also generated for the subclasses of fat: saturated fatty acids (Figure 3A), polyunsaturated fatty acids (Figure 3B) and monounsaturated fatty acids (Figure 3C) and, as well, for total sugars (Figure 3D), to visually assess the agreement between FFQ and 7-day food records. No more than 7 observations were outside the limits of agreement for any fractions of fatty acids and total sugars (Figure 3D). Slopes (95% CI) of the regression, using intakes from food records as independent variables and the differences between food records and FFQ as dependent variables, were: 0.090 (−0.094, 0.275), −0.057 (−0.254, 0.140), 0.028 (−0.162, 0.218), 0.592 (0.330, 0.854) for saturated fatty acids, polyunsaturated fatty acids and monounsaturated fatty acids and total sugars, respectively.

Figure 3 Bland–Altman plots for the difference in fractions of fatty acid intakes measured with the 7-day food records and 4 weeks retrospective FFQ, plotted against the mean intakes from the two methods. (A) SFA intake (g); (B) PUFA intake (g); (C) MUFA intake (g); (D) total sugars intake (g). Notes: The solid line indicates the regression line, the serried dashed line indicates the mean difference between the methods, while the spaced dashed lines indicate ±1.96 SDs.

Table 4 presents the median (interquartile range) of food groups, as assessed by the two methods: 4 weeks retrospective FFQ and 7-day food records, along with their Spearman’s rho, which are ranging from 0.15 for legumes to 0.68 for pure alcohol. Table 5 contains the concordance of quartile classification by the two methods in the same, next and opposite and the Cohen’s Kappa values. The evaluation of intakes, based on food groups, confirmed fair relative relation (Cohen’s kappa > 0.2) for fruits, potatoes, whole grain, fish and alcoholic and non-alcoholic bottled drinks. For the rest of the food groups, Cohen’s kappa values indicated low agreement. Approximately 40% of participants were classified in the same quartile and an average of 38.5% were classified into the next quartile, while grossly misclassification was observed for 20% of participants. Lower levels of misclassification (<20%) were calculated for natural juices, dairy, fish and nuts and seeds. The highest percentage of concordance is seen for pure alcohol with 63.8% of participants in the same quartile and the highest discordance is observed in diary, eggs and read meat in each 31.9% of participants being in the same quartile. Kappa ranges from 0.088 for legumes to 0.517 for pure alcohol.

Table 4 Intakes of the Food Groups as Assessed by 4-Week Retrospective FFQ and 7-Day Food Records Methods (n = 116)

Table 5 Spearman’s rho, Percentage of Participants Classified into the Same, Adjacent or Opposite Quartile by the 7-Day Food Records and 4-Week Retrospective FFQ and Cohen’s Kappa (n = 116)

Discussion

This study describes the relative validation between estimated intakes using 53-item FFQ and 7-day food records, which were used to assess the short-term intake of nutrients and categories of foods consumed by Romanian adults. The FFQ method estimated the dietary intakes for 4 weeks. To reduce recall bias, a type of bias highly associated with investigating dietary intakes, it has been recommended that a shorter recall period is preferable to a longer one, especially when investigating routine or frequent events.²⁴ With shorter recall instruments, higher validity is expected, since it has been previously indicated that shorter than 70 items FFQs have a higher validity, as compared to longer than 97 items instruments.¹¹ However, longer instruments have the advantage of including more food items, which could provide a better estimation of the habitual intake of macro- and micronutrients.

Several measures of concordance between the FFQ and the 7-day food record methods were used. The FFQ performed fair to moderate when comparing energy and nutrient intakes with those estimated by the 7-day food records. Moderate agreement between the two methods of assessment, quantified by quartile distribution between assessments, with kappa above 0.4, was observed for energy and macronutrients, as recommended by Willet.¹⁸ Jackson et al²⁹ reported similar to our results a range for kappa from 0.17 for protein to 0.40 for energy. A fair agreement was observed for all tested micronutrients. Raw and energy-adjusted Spearman correlation coefficients for energy and macronutrients were comparable with other studies that reported a range of coefficients from 0.025 to 0.84.^25–28 For other nutrients, correlation coefficients were lower than 0.4 in both raw and energy-adjusted analyses, except for fibre, cholesterol and choline, similar to other reported studies, with the reported correlation coefficients in 0.34–0.65 range.^27–29

The Bland–Altman plots suggested that a small number of individuals were placed outside the recommended limits, but the overall assessment confirmed an acceptable level of agreement between methods. Regression analysis using the differences between the two methods as a dependent variable and the mean of two methods as an independent variable, performed along with Bland–Altman plots, indicated that the two methods were comparable, without proportional bias, for energy and all macronutrients, saturated fatty acids, mono- and polyunsaturated fatty acids. However, for total sugars, the Bland–Altman plot and the regression suggested a proportional bias between FFQ and 7-day food records. The reported total sugar intake in FFQ was higher than in food records and the difference might be explained by the hypothesis that sugar intake is likely to be underreported in food records, especially by women and individuals with excess weight.^30,31

For macronutrients, other studies have reported similar percentages. Talegawkar et al²⁷ have reported for macronutrients that 80% of the participants were classified in the same or within a quartile difference. Higher degrees of misclassification for macronutrients into the opposite quartile were reported by Jackson et al²⁹, ranging from 8.6% for carbohydrates to 24.3% for fat.

Gross misclassification of some micronutrients such as calcium and, to a lesser extent, magnesium, potassium and folates can be regarded due including, under the same items of the questionnaire, different types of foods such fruits, vegetables or legumes, with different amounts of micronutrients in composition.³² Similar to our results, the study by Talegawkar et al²⁷ reported for micronutrients, on average, that 77.9% of the participants were classified within a quartile and the study by Jackson et al²⁹ reported, for the opposite quartile, percentages ranging from 4.3% for alcohol to 23.8% for vitamin E.

The weak degree of agreement for some food groups may be due in part to the within-subject variance in the 7-day food records. Certain foods, such as legumes, for which a lower frequency intake is usually reported, were perhaps not covered adequately in the 7-day food records, similar to previously reported studies.^8,33

The sample size of 116 participants, each of them providing a complete set of 7-day food records, totalled 812 days for the entire study. Although the increased time length of food records collection and the large time overlap tried ensuring good correspondence between the two assessment methods, it is virtually impossible to get the exact results, due to daily variations of food intakes. However, as some food items in the FFQ may not have been consumed during the week when the food diaries were kept, this comparison between the two methods could be inherently limited concerning its degree of agreement for these specific items, which could partially explain the differences observed.

The FFQ selected for this study is appropriate to explore seasonal variations of dietary intake since diets with different patterns and nutrient composition have been reported in different seasons of the year.³⁴ However, using a shorter instrument could lead to a misallocation of food items by the responders, due to a lower number of items and some foods being included in larger categories. To capture a longer period, the questionnaire would need to be completed several times, so all seasons could be represented. Another limitation of the study is that all participants completed the FFQs after completing the food records and this bias could not be addressed through statistical procedures.

In the present study, 61% of participants returned a 7-day food record and completed the FFQ (Figure 1), similar to another study that recruited the subjects from an academic environment.³⁵ The use of an online tool to capture FFQ answers has the advantage of fast answer retrieval. Paper-based questionnaires have a long history of use and are more likely to be accepted and used in some settings, but the validity of electronic versus paper-based forms was found to be similar.³⁶ The major limitation of the current study is the lack of recovery biomarkers, for the unbiased estimation of nutrient intakes at an individual level. However, the use of multiple days food diaries has been recognized as a good proxy for recovery biomarkers, offering a stronger estimate of energy and protein than the 24-hour recalls.³⁷ Nonetheless, this is not the case for testing the validity of micronutrient intake assessments, where biomarkers are needed.

To validate a specific method for estimating nutrient intakes, several validation methods were described previously, such as the use of biomarkers, the 24-hour dietary recall and dietary records.³⁸ In our study, due to the lack of funding and technical limitations, the use of biomarkers could not be employed.

The participants were volunteers, with a proportion of females/males of 2:1. The disproportionate acceptance of participation among sexes could be explained by the fact that, in general, women are more concerned about food, diet and health^39,40 and have a higher general nutritional knowledge,¹³ as compared to men. Our sample’s socio-economic and demographic structure differs from the general population since higher levels of motivation and interest are related to better coping with difficult tasks,⁴¹ such as detailed reporting of all food and beverages intake for 7 days, explained by internal and external influences on shared decision-making.⁴²

This study indicated that the correlation between the two methods, for different classes of intakes (whether nutrient categories or food groups) was not homogenous. While the best agreement was found for energy and macronutrient intakes, various micronutrient intakes had variable strengths of agreement (with various degrees of misclassification), while food groups had a poor agreement between methods. The results of this study suggest that, while both methods could be employed with similar results for energy and macronutrients, method-specific differences are likely to occur when estimating many micronutrient intakes and food groups consumption. Therefore, when designing studies that estimate nutritional intakes, careful consideration should be given to method selection for assessing such intakes, depending on which nutrient categories are sought and whether food groups assessment is involved or not.

Conclusion

This study compared the agreement between 4 weeks retrospective FFQ and 7-day food records, concerning the estimated energy, macro- and micronutrient intakes and food groups consumed. The described FFQ method was intended to be a short-term dietary assessment tool for seasonal intake in the Romanian population. Our results indicated that the comparison of this FFQ with 7-day food records was moderate in estimating energy and macronutrient intakes. Since this instrument is intended to measure the food intake at a population, not an individual level, a moderate agreement between the two methods indicated that both methods can be used, with similar outcomes for energy and macronutrient intakes. The agreement was fair for fractions of fatty acids and the following micronutrients: iron, sodium, potassium, vitamins B6 and B12 and choline, while poor for the rest of the micronutrients. With food groups, the agreement between the two methods was fair for some of them and poor for others, especially for foods with a low frequency of intakes. These findings suggest that, while the two methods could be interchangeable for estimating energy and macronutrient intakes at a population level, careful consideration should be given to method selection when the aims are to estimate other categories of nutrients or food groups, for which the degree of agreement varied considerably.