Validation of energy intake from a web-based food recall for children and adolescents

The purpose of this study was to validate estimated energy intake from a web-based food recall, designed for children and adolescents. We directly compared energy intake to estimates of total energy expenditure, calculated from accelerometer outputs, combined with data on weight and sex or resting energy expenditure prediction equations. Children (8–9 years) and adolescents (12–14 years) were recruited through schools in Norway in 2013 (N = 253). Results showed that more than one third (36–37%) were identified as under-reporters of energy. In contrast, only 2–4% were defined as over-reporters of energy. The mean energy intake was under-reported with -1.83 MJ/day for the entire study sample. Increased underestimation was observed for overweight and obese participants, the oldest age group (12–14 years), boys, those with parents/legal guardians with low educational level and those living in non-traditional families. In conclusion, energy intake from the web-based food recall is significantly underestimated compared with total energy expenditure, and should be used with caution in young people.


Introduction
A healthy diet and normal body weight are key factors for preventing non-communicable diseases (NCDs), which are the leading causes of deaths worldwide [1]. Generally it takes time for NCDs to develop [2], and it is recognized that risk factors present in childhood may increase the risk of developing NCDs in adulthood [3]. Information regarding dietary exposure and energy intake in the first parts of humans' lives is therefore of large interest in a public health perspective.
Dietary self-report methods such as food records, recalls and food frequency questionnaires have been widely used to assess total dietary or energy intake in children and adolescents [4], despite being prone to reporting bias [5]. Unfortunately, few alternatives exist, due to the fact that there are only a small number of recovery biomarkers available [6], observation of dietary intake for entire days are often not feasible [7], and the double-portion technique is both burdensome and expensive [8]. Thus, self-reported dietary methods need further refinement. New technology or altering and mixing elements from different methods have been suggested as a possible way forward [9]. Over the last years several new dietary assessment tools for a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 children and adolescents have been developed [10]. Use of new technology (e.g. computers, mobile phones) provides clear advantages in terms of reduced data handling, and is preferred over paper-based methods among the young [11]. Yet, it is still not clear how the accuracy of dietary assessment is affected by using new technology [12], and more validation studies of new tools in the younger age groups are needed.
When validating a new dietary assessment tool, a comparison with an objective reference method, with non-correlated measurement errors, is preferred over a comparison with another dietary assessment method [13]. In weight stable individuals, there is a good agreement between energy intake (EI) and total energy expenditure (TEE) [14]. Hence, estimated EI can be evaluated against estimates of TEE [15]. Accelerometers measures physical activity fairly accurate and may be used to estimate TEE in combination with measured or estimated resting energy expenditure (REE) [16].
The aim of this study was to validate children's and adolescents' EI estimated from the webbased food recall (WebFR). This was done by a direct comparison of EI to estimates of TEE, calculated from accelerometer outputs combined with data on weight and sex or REE prediction equations. Furthermore, the proportion of acceptable-and non-acceptable reporters of EI was defined.

Design
In this validation study of the WebFR, developed for use in a national dietary survey among 4 th and 8 th graders (8-9 and 12-14 years) in Norway, a total of 414 children, in these age groups, were invited through schools the fall of 2013 in the municipality of Baerum, Norway. Information regarding the study was provided orally and in writing to all children and their parents/legal guardians. Children were eligible for inclusion if they had Internet access at home and provided a valid email address to one of their parents / legal guardians. Two hundred seventy-six children got parental consent and were included, of which nine withdrew before study start or during the data collection. Participants used the WebFR concurrently with an ActiGraph GT3X+ accelerometer (ActiGraph LLC, Pensacola, FL, USA). Out of the 267 who completed the study, 14 had to be excluded due to incomplete data, of which 13 had less than two valid accelerometer recording days and one lacked entries in the WebFR. Thus, data from 253 (61.1% of all invited) was used in subsequent analyses.

Ethics statement
Child assent and written parental consent were obtained from all participants. The Norwegian Data Protection Official for Research (NSD) approved the study (Project No. 32968). The study was conducted in accordance with the Declaration of Helsinki. A personal gift card containing two cinema tickets were given to all participants who completed the study.

The WebFR
The web-based food recall (WebFR), described extensively in a previous paper [17], is a modified version of the Danish Web-based Dietary Assessment Software for Children (WebDASC) [18]. The interface includes an interactive character guiding the participants through each day's eating occasions, in chronological order, using both audio and text in speech bubbles. To facilitate data entering, participants use a search function field or drop-down-lists with different categories containing around a total of 550 foods and beverages. A free text field is also available if the appropriate item does not exist among those listed. Pop-up elements are incorporated to remind the participants to enter in-between snacks, supplements, or other items often omitted from reports. All participants were instructed to enter everything they consumed for four consecutive days, including one weekend day retrospectively every evening at home after the last eating occasion. Parents/ legal guardians were instructed to assist the youngest participants (8-9 years).

The accelerometers
The ActiGraph accelerometer is a small triaxial accelerometer used to provide objective measurements of physical activity and sedentary behaviors in free-living conditions. In this study all participants were instructed to wear the ActiGraph for seven consecutive days, including a weekend, and only to remove it during water activities (swimming, showering etc) and at night. They were given a demonstration on how to wear the ActiGraph on the right hip with an elastic band. In order to avoid recordings of any possible reactivity at startup, the participants were not informed that the accelerometers were programed to start the recordings the day after they started wearing them.

Anthropometry
Height and body weight (TANITA TBF-300, Tanita Corporation, Tokyo, Japan) were measured, to the nearest 0.1kg and one millimeter, respectively, without shoes and in light clothing using standard procedures. To define overweight and obese participants, the age and sex-specific body mass index (ISO-BMI) from Cole et al. [19] was applied.

Sex, age and family background
Information regarding sex and age, parental education level, parental ethnicity and family structure was provided from questions included in the written consent form completed by the parents/legal guardians. Accelerometer counts and individual physical activity level. Activity counts from the ActiGraph were used to calculate the rates of individual metabolic equivalents (METs) by using published algorithms (2005) [20]. The acceleration data was sampled at 30 Hz, and data from the vertical axis were used in the analyses. Non-wear periods were defined as periods of at least 20 minutes of consecutive zeroes. All activity between midnight and 6 am was excluded. Inclusion criteria were at least eight hours of recordings each day, for a minimum of two days. Individual physical activity levels (PALs) were expressed as average METS over 24-hour period time. For the non-valid accelerometer time, which included non-wear time (i.e. sleeping); a MET value of 1.2 was used.

Data handling and statistics
Resting metabolic rate. Age, sex, weight and height specific equations from Henry [21] were used to estimate REE for each individual.
Prediction equations for total energy expenditure. The mean of the following three different equations were used to calculate TEE in MJ/day:
REE from Henry [21] and equation for activity energy expenditure (AEE) from Ekelund et al. [22]. Boys = 0; Girls = 1. Activity counts are expressed in counts per minute (CPM). [21]. PAL expressed as average METS over 24-hour period time. Estimated energy intake. Mean estimated EI from the WebFR recordings were calculated for the average of the four recording days, and for each of the four recording days. A one-way repeated measures ANOVA was conducted to compare EI across the recording days.
Pearson's correlation between energy intake and total energy expenditure. Pearson's correlations were calculated between EI and TEE for all participants and for subgroups of the sample.
Definition of acceptable-, under-and over-reporters of energy. Two different approaches were used in order to identify acceptable and non-acceptable reporters of energy intake. Participants were defined as either acceptable reporters (AR), under-reporters (UR), or over-reporters (OR). A theoretically impeccable reporter of energy intake, if weight stable, would fulfill the following: 1. EI/TEE = 1

EI/REE = PAL
However, such a perfect agreement cannot be anticipated. Thus, in the first approach, AR were defined as those within the 95% confidence limits (CL) of the agreement between reported EI and TEE, that takes into account the within-subject variation in reported EI and TEE in addition to the number of days of the dietary assessment method, as proposed by Black [15]. In this study, AR had EI/TEE from 0.72-1.28, UR had EI/TEE <0.72 and OR had EI/TEE >1. 28. TEE is expressed as the mean of TEE1 , TEE2 and TEE3 , fully described earlier in this paper. Secondly, the well-established Goldberg cut-off approach [23] was used, in which AR were defined as those having a reported EI/REE within the 95% CL of agreement with their individual measured PAL, incorporating the within-subject variation in reported EI and REE, in addition to between-subject variation in PAL. That is, using the Goldberg cut-off approach, an AR with a PAL of e.g. 1.5 would have EI/REE between 1.07-2.11, UR would have EI/REE <1.07 and OR would have EI/REE>2.11. The within-subject coefficient of variation (CV) for reported EI was set to 23%, as suggested by Black [23]. A within-subject CV for TEE of 8.2%, based on doubly labelled water [24], was used when calculating the 95% CL for the EI/TEE agreement. For the Goldberg cut-offs, the standard CV for BMR of 8.5% was used to account for the variation in REE [23], in addition to a between-subject CV for PAL of 9.23% given from our own study-sample.
Bland-Altman plot. In order to explore and visualize if the agreement between EI and TEE differed across the mean scores of EI and TEE, a Bland-Altman plot was created.
Linear regression. Linear multiple regression analysis was used to investigate which variables contributed significantly to misreporting of EI, using 'difference between EI and TEE (EI minus TEE)' as the outcome. The variables 'sex', 'age-group', 'weight status', 'parental educational level', 'parental ethnicity' and 'family structure' were initially tested in univariate regression analysis. All were statistically significant at the 10% level, and were included in a multiple linear regression model. Subsequently, one variable, 'parental ethnicity', did not significantly contribute to the explained variance and was omitted from the model. No statistically significant interactions were found and all assumptions of normality and linearity were met.
Sensitivity analysis. Finally, a sensitivity analysis was conducted, to assess the validity of the reported EI after using a recommended approach to exclude implausible reporters of energy in nutrition epidemiology studies [25]; any individual with EI <2.09 MJ (500 kcal) or >14.64 MJ (3500 kcal) were excluded before running the previously described analysis.

Results
The characteristics of the study sample are shown in Table 1. Forty-nine % of participants were 4 th graders (8-9 years) and 51% were 8 th graders (12-14 years). There were slightly fewer boys than girls, and 14% of all participants were either overweight or obese. The level of parental educational was high for most participants (77%); the majority had at least one parent/legal guardian with Norwegian ethnicity (86%), and lived in a traditional family (73%). Table 1 shows that the participants had a mean PAL of 1.57. Moreover, they had a mean EI of 6.85 MJ/day, and the mean TEE was 8.67 MJ/day. The mean under-reporting of EI was -1.83 MJ/day for all participants, and -4.13 MJ/day among the overweight and obese. Pearson's correlation between EI and TEE was 0.16 for the entire sample.
There was a significant difference in EI across the four recording days (Wilk's Lamda gave p<0.001, and eta squared was 0.20). A steady increase in EI was observed from day one till four: 6.17 MJ, 6.47 MJ, 6.91 MJ and 7.84 MJ, respectively. Fig 1 shows that the proportion of AR varied between 59-62%, UR varied between 36-37% and OR varied between 2-4% when using two different calculation techniques. Thus, the differences between the two approaches are negligible.
The Bland-Altman plot in Fig 2 gives a visual description of the difference between the reported EI from the WebFR and estimated TEE plotted against the mean of the two. The plot shows that the difference between EI and TEE deviate largely from 0, and more individuals are under-reporting, than over-reporting their energy intake. There is a tendency for more underreporting at lower mean values of the two methods, and more over-reporting at higher mean values, suggesting possible bias.
A multiple linear regression model including variables associated with misreporting of EI is shown in Table 2. This model explains 24% of the variation in misreporting, defined by the difference between EI and TEE (EI minus TEE). BMI-category has the strongest impact: overweight or obese children under-reported their EI with -2.35 MJ/day more than the normal weight individuals. Moreover, increased under-reporting of EI was found for boys, the older children (12-14 years), those with parents/legal guardians with low educational level and those living in non-traditional families. These results are in line with the misreporting of EI in subgroups presented in Table 1.
Sensitivity analyses showed that the overall results were not affected, as only one participant was excluded using the recommended cut-offs.

Main findings
More than one third of all participants (36-37%) were identified as under-reporters (UR) in this study, when comparing estimated EI from a web-based food recall (WebFR) to TEE calculated from objective accelerometer counts, combined with data on weight, sex or REE. In Table 1. Characteristics of study sample, measures of physical activity, reported energy intake, resting-and total energy expenditure, energy balance, and Pearson's correlation between energy intake and total energy expenditure. contrast, only 2-4% were defined as over-reporters (OR). The mean under-reporting of EI was -1.83 MJ/day for the entire study sample. Increased underestimation was observed for overweight and obese participants, the oldest age group (12-14 years), boys, those with parents/ legal guardians with low educational level and those living in non-traditional families.

Comparisons with previous work
EI estimated from the Danish WebDASC system has previously been evaluated against TEE estimated from accelerometer counts and data on age, sex, height and weight [26]. Due to the similarities between instruments, we expected similar results. However, data from the Danish WebDASC suggested approximately 20% under-reporters and 20% over-reporters (Biltoft-Jensen et al.), compared with 36-37% UR, and 2-4% OR in this study. Moreover, the mean reported EI and the estimated TEE in the Biltoft-Jensen study [26] were not significantly different with a mean under-reporting of only -0.04 MJ/day, compared to our under-reporting of -1.83 MJ/day. About 50% of our participants were adolescents, who are known to be more influenced by social desirability that reduces their reporting accuracy [27], whereas the age range in Biltoft-Jensen et al.'s study was 8-11 years [26]; this may partly explain differences between studies. Besides, we had a higher proportion of overweight and obese individuals, in addition to individuals with a diverse ethnic background, compared with Biltoft-Jensen et al. [26], who described their study population as relatively homogeneous in respect of ethnic, social and cultural background [28].
Results from other studies, in which accelerometers have been used to validate estimated EI from more traditional paper-based methods among children, are in line with our study: suggesting the proportion of under-reporters is a large problem [29][30][31]. For example, estimated The percentage of AR, UR and OR, identified using two different approaches. AR, acceptable reporters; UR, under-reporters; OR, over-reporters; EI, energy intake; TEE, total energy expenditure. (A) AR were defined as those within the 95% confidence limits of the agreement between estimated EI from a webbased food recall (WebFR) and TEE calculated based on accelerometer counts, combined with data on weight, sex or REE. AR had EI:TEE from 0.72-1.28, UR had EI:TEE <0.72 and OR had EI:TEE >1.28. (B) The Goldberg cut-off approach was used, in which AR were defined as those having a reported EI:REE within the 95% CL of agreement of their individual physical activity level (PAL) measured by accelerometers. UR and OR were defined as those under and over this 95% CL, respectively. https://doi.org/10.1371/journal.pone.0178921.g001 Validation of a web-based food recall EI using a paper-based pre-coded food diary for four days in nine year old children suggested under-reporting of -1.8 MJ/day [30]. This is similar to our observations among the eight-nine year olds showing under-reporting of -1.4 MJ/day. Moreover, underestimation was larger in boys compared with girls [30], this is also in line with findings from the present study. Severe under-reporting was also observed in a similar validation study of a paper-based pre-coded food diary among 13 year olds; the mean difference between EI and TEE showed underreporting that varied from -1.3 to -4.8 MJ/day [29]. Rothausen et al. report a difference between EI from a seven days food diary and TEE of -2.7 and -2.1 MJ/day for 12-13 year old boys and girls, respectively [31], which is comparable to our observations. However, these authors report better reporting accuracy for the same individuals using 2 x 24 hour recalls, and in seven to eight year old children. The effect of sex on misreporting of EI seems inconsistent in the literature, also when using the gold standard doubly labeled water as the reference [5]. In summary, our results corroborate previous observations using the traditional paper-based methods. A possible explanation for this may be that the inherent challenges with the dietary Fig 2. Bland-Altman plot displaying the difference between EI and TEE plotted against their mean. EI, energy intake; TEE, total energy expenditure; MJ, megajoule; SD, standard deviation. This visual plot demonstrates how the difference between estimated EI from a web-based food recall (WebFR) and TEE estimated based on accelerometer counts, combined with data on weight, sex or REE (Y-axis) varies with increasing levels on the scale (X-axis). The mean difference between EI and TEE is given by the solid thick line, together with the 95% CI for the mean, displayed in long stippled lines. The short stippled lines show +/-1.96 SD of the mean difference between EI and TEE. assessment methodology is not necessarily bypassed by the new technology alone, as suggested by Illner et al. [12].
Low reported intakes have been observed in the last days of long recording periods (>4 days) in adults [32,33], explained by participant fatigue. We observed higher reported EI during the last recording days; hence, it is unlikely that participant fatigue have contributed to the under-reporting in this study. We speculate if the observed increase in EI over the recording days was caused by a learning effect, or if it reflects the day-of-the-week variation between recording days (day 3 and 4 held all Fridays and Saturdays, respectively). Supportive of the latter, Lillegaard et al. reported significantly higher EI Fridays and Saturdays, compared to weekdays, in 9-year old children [34].
Under-reporting was greater in magnitude in overweight and obese children. This finding was expected, and has been reported previously, for example in a review by Burrows et al. in which doubly labelled water was used as the reference method [5]. Svensson et al. found under-reporting of EI of -2.84 MJ/day, when they assessed EI using a food record combined with digital cameras, and compared it to estimated TEE based on accelerometer counts and data on temperature, weight, height, sex, and age, among overweight and obese 8-12 year olds [35]. The under-reporting of EI was -4.1 MJ/day in our sample of overweight and obese participants. It is likely that the participants' young age may have been contributing to higher reporting accuracy in the study of Svensson et al., as children's dietary reporting is less biased before entering adolescence [36]. Additionally, the innovative element of using digital cameras in addition to the food record, may have improved the reporting accuracy. This is supported by a review paper on image-assisted dietary assessment among adults [37], in which the use of digital images as the primary record or in addition to traditional methods reduced underreporting. Methodologically new dietary assessment methods in the younger age groups have also been developed [10]. An example of a tool being more than just technologically new is the Technology Assisted Dietary Assessment (TADA) food record application, in which users take images of foods and beverages at all eating occasions using a mobile device [38]. However, no validation studies of the TADA, or similar methods, using objective reference methods in children or youth have been published, to our knowledge.

Strengths
The use of accelerometers is a strength in this study, as accelerometers has demonstrated to be objective, accurate and reliable tools to measure physical activity, and is also considered as a preferred choice when estimating energy expenditure [16]. Moreover, the ActiGraph, which was used in this study, is the most commonly used accelerometer in physical activity research for children and adolescents [16]. All participants in this study were instructed to wear the accelerometer for seven consecutive days, and the sample achieved a mean of five valid measurement days. This strengthens the estimates of the individually estimated TEE and PAL. Further, in order to reduce the challenge with reactivity, the first day of wearing the accelerometer was omitted, that is, the accelerometers were programmed to start the day after the participants started wearing them.
We estimated TEE as the mean of three different algorithms, based on the statistical principle 'wisdom of select crowds', which postulates that averaging a small number of selected estimates based on expertise, will often outperform a single estimate taken as the best [39]. Moreover, aggregate prediction equations have shown to perform better than single prediction equations, when predicting body composition and resting metabolic rate [40]. The rationale is that by averaging, errors randomly distributed between uncorrelated estimates will cancel each other out [39,40]. Consistent with this, our use of two different approaches to assess the proportion of AR, UR and OR, also ensure more robust data.

Limitations
The equations from Ekelund et al. [22], used to estimate TEE and AEE from accelerometer counts and data on weight and sex, were developed in nine year old children. Therefore these equations may be less accurate for the 8 th graders (12-14 years) included in this study.
Moreover, although accelerometers have the advantage that they can assess physical activity objectively, and thus estimate AEE and TEE, if combined with data on weight, sex or REE, there are also several well-known limitations to their ability to assess AEE and thus TEE at the individual level [16].
Because participants were only weighed once in this study, we cannot discriminate between those who were truly in negative or positive energy balance, and thus under eating or over eating compared to their true energy needs, and those who were in energy balance and simply misreported their EI.

External validity
The activity counts in our study were 664 and 571 CPM for 4 th graders (8-9 years) and 8 th graders (12-14 years), respectively. In comparison, a nationally representative Norwegian physical activity survey [41] observed a mean CMP of 653 for nine year olds, 421 CPM for 15 year old girls and 494 CPM for 15 year old boys. Given the age-related decline in physical activity level [42], the physical activity level of our sample is similar to the nationally representative data. Moreover, parental educational level, ethnic background and weight status in our participants are comparable to the population from urban and semi-urban areas in Norway, demonstrated in a previous paper covering the same study sample [43]. Therefore the results in this paper can probably be extrapolated to children and adolescents living in similar areas in Norway.

Conclusions
The WebFR significantly underestimated EI compared with TEE estimated from accelerometer counts, combined with data on weight, sex or REE, in children (8-9 years) and adolescents (12-14 years). The magnitude of underestimation was influenced by overweight and obesity, sex, age, parental education level and family structure. The level of under-reporting of energy is in line with traditional paper based methods, and the estimated EI from the WebFR should be used with caution, equally as estimated EI from traditional dietary self-reports.