Validation of the food frequency questionnaire for the assessment of dietary vitamin D intake

fnut-09-950874 September 16, 2022 Time: 16:25 # 1

TYPE Original Research

PUBLISHED 23 September 2022

DOI 10.3389/fnut.2022.950874

OPEN ACCESS

EDITED BY

Christophe Matthys,

KU Leuven, Belgium

REVIEWED BY

Rosalind Fallaize,

University of Hertfordshire,

United Kingdom

Irena Keser,

University of Zagreb, Croatia

*CORRESPONDENCE

Igor Pravst

igor.pra[email protected]

SPECIALTY SECTION

This article was submitted to

Nutrition Methodology,

a section of the journal

Frontiers in Nutrition

RECEIVED 23 May 2022

ACCEPTED 26 August 2022

PUBLISHED 23 September 2022

CITATION

Hribar M, Žlavs K, Pravst I and Žmitek K

(2022) Validation of the food frequency

questionnaire for the assessment

of dietary vitamin D intake.

Front. Nutr. 9:950874.

doi: 10.3389/fnut.2022.950874

Žmitek. This is an open-access article

distributed under the terms of the

Creative Commons Attribution License

(CC BY). The use, distribution or

reproduction in other forums is

permitted, provided the original

author(s) and the copyright owner(s)

are credited and that the original

publication in this journal is cited, in

accordance with accepted academic

practice. No use, distribution or

reproduction is permitted which does

not comply with these terms.

Validation of the food frequency

questionnaire for the

assessment of dietary vitamin D

intake

Maša Hribar

1,2

, Katarina Žlavs

1,2

, Igor Pravst

1,2,3

and

Katja Žmitek

1,3

Nutrition and Public Health Research Group, Nutrition Institute, Ljubljana, Slovenia,

Biotechnical

Faculty, University of Ljubljana, Ljubljana, Slovenia,

VIST–Faculty of Applied Sciences, Ljubljana,

Slovenia

Vitamin D and its adequate status are related to many aspects of human

health; therefore, an appropriate tool is needed for the valid assessment of

vitamin D status. The main contributor to vitamin D status is endogenous

synthesis after cutaneous exposure to ultraviolet B light (UVB), but in the

absence of UVB radiation, vitamin D intake becomes an important source of

vitamin D. Various tools are available for vitamin D intake assessments, with the

Food Frequency Questionnaire (FFQ) being among the fastest, cheapest, and

most convenient; however, until now, these tools have not been adapted for

the Slovenia (SI). To enable valid vitamin D intake estimation, we developed

a simple one-page semi-quantitative FFQ (sqFFQ/SI) and tested its validity

using a 5-day dietary record (DR) as a reference method. The reproducibility

was tested with the second sqFFQ/SI (sqFFQ/SI2) 6 weeks after the ﬁrst

(sqFFQ/SI1). The validity and reproducibility of this method were tested on

54 participants using Bland–Altman plots, Spearman’s correlation, and Kappa

analyses of tertiles. The mean daily vitamin D intake was 3.50 ± 1.91 µg

according to the 5-day DR, and 2.99 ± 1.35 and 3.31 ± 1.67 µg according

to the sqFFQ/SI1 and repeated sqFFQ/SI (sqFFQ/SI2), respectively. When

analyzing for validity, the sqFFQ/SI1 was found to be signiﬁcantly correlated

(p < 0.05) with the 5-day DR, with an acceptable correlation coefﬁcient of

0.268 and a Bland–Altman index of 3.7%. For reproducibility, the correlation

between the sqFFQ/SI1 and sqFFQ/SI2 was highly signiﬁcant (p < 0.001),

with a good correlation coefﬁcient of 0.689 and a Bland–Altman index

of 3.7%. Kappa analyses of tertiles showed a poor validity and acceptable

reproducibility. Overall, we observed a higher reproducibility than validity.

Validation and reproducibility analyses demonstrated that the proposed

sqFFQ/SI is acceptable and is, therefore, an appropriate tool for the effective

Frontiers in Nutrition 01 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 2

Hribar et al. 10.3389/fnut.2022.950874

assessment of habitual vitamin D intake on an individual level. With this

consideration, this tool will be used in further population studies to assess

vitamin D intake and for the development of a screening tool for the

assessment of the risk for vitamin D deﬁciency, which will be used as a

foundation for evidence-based policy-making decisions.

KEYWORDS

vitamin D, nutrient intake, validation, reproducibility, Slovenia, Food Frequency

Questionnaire (FFQ), dietary record

Introduction

Vitamin D is a fat-soluble vitamin that is, due to its many

functions in the body, crucial for the growth and maintenance

of health in all life stages (1–3). For humans, the sources of

vitamin D are endogenous synthesis in the skin when exposed

to ultraviolet B (UVB) radiation and dietary intake (either

with foods that are naturally rich in vitamin D, fortiﬁed foods,

or pharmaceutical preparations) (4). Although endogenous

synthesis is the main source of vitamin D for most people, in

the absence of suﬃcient UVB exposure, vitamin D becomes an

essential nutrient and suﬃcient dietary intake is required (5–7).

The dietary vitamin D intake is usually well below

recommendations (5), mainly because very few foods are rich in

vitamin D, and, at the same time, they are seldom consumed.

The recommended dietary vitamin D intake for the adult

population is 5 µg/day (19–50 years), 10 µg/day (51–65 years),

and 15 µg/day (>65 years) according to the recommendations

of the World Health Organization (WHO) (8), and 15 and

20 µg/day (in the absence of endogenous synthesis) according

to the recommendations of the European Food Safety Authority

(EFSA) and the Nutrition Societies of Germany, Austria, and

Switzerland (D-A-CH), respectively (9, 10). On the other hand,

the nutrient reference value (NRV), as deﬁned in the European

union food labeling regulation, is 5 µg (11), while the threshold

of 2.5 µg is sometimes used as a lower reference nutrient intake

(LRNI) (12).

Studies have reported a high prevalence of inadequate

dietary intakes of vitamin D in European populations and

around the world (6, 13, 14), including Slovenia (15, 16). In

most European countries, the daily intake of vitamin D is

lower than 5 µg (6, 14, 16, 17); exceptions are Scandinavian

countries, where oil-rich ﬁsh consumption is relatively high,

and both fortiﬁcation and supplementation policies have also

been implemented (6, 18). Only a few studies have evaluated the

dietary intake of vitamin D in the Slovenian population, using

various methods to record the dietary intake (15, 16).

Due to insuﬃcient UVB exposure and simultaneous

inadequate vitamin D intake, an important public health task

is the rapid identiﬁcation of individuals exposed to the risk

of inadequate vitamin D status. Furthermore, the accurate

assessment of dietary vitamin D intake is important for the

application of evidence-based public health measures in order

to prevent poor vitamin D status in diﬀerent population groups.

To achieve this, a suitable screening procedure is necessary (19).

The optimal and most objective method for evaluating

vitamin D status is a laboratory determination of serum 25-

hydroxyvitamin D [25(OH)D] (20), but this method is invasive

and not recommended for screening in large populations (19,

21). Because vitamin D status is also aﬀected by dietary intake

of vitamin D, a valid dietary assessment method that is easy

to use is needed (22). Determining dietary vitamin D intake

with the 24-h recall or the dietary record (DR) method, a gold

standard for dietary intake assessments, is not the most well

suited (23); because of large day-to-day variations in vitamin

D intake (dependent on, e.g., ﬁsh intake and the diversity of

fortiﬁed foods), an extended time period is necessary for data

collection (22, 24). On the other hand, the Food Frequency

Questionnaire (FFQ) is less useful for measuring the absolute

dietary intake, but it can better reﬂect one’s typical diet (25).

Additionally, the FFQ may be more reliable for the estimation

of micronutrient intake, such as vitamin D, as it covers a longer

period and can focus on speciﬁc foods that are relevant to

vitamin D intake (26).

When assessing dietary intake, the research method must

be simple and fast, for both the subject and the researcher,

and, at the same time, it must be valid and reproducible (25,

27). Various FFQs for the assessment of vitamin D intake

were designed and validated in several countries and studies

around the world (23, 26, 28–34). However, such tools need

to be tailored for use in speciﬁc regions; country-speciﬁc food

consumption patterns and foods need to be considered (25, 27,

35). The typical reference methods for the validation of the FFQ

are DR or the 24-h recall method, and biomarkers are sometimes

also used (25).

The objective of this study was to assess the validity and

reproducibility of a semi-quantitative FFQ on the Slovenian

population (sqFFQ/SI) for the assessment of the dietary

intake of vitamin D, using 5-day DR as a reference method.

The sqFFQ/SI was developed by the Nutrition Institute

Frontiers in Nutrition 02 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 3

Hribar et al. 10.3389/fnut.2022.950874

(Slovenia) in cooperation with the National Institute of Public

Health (Slovenia) within the national research project Nutri-D

“Challenges in achieving adequate vitamin D status in the adult

population” (L7-1849).

Materials and methods

Study design and data collection

The study protocol was approved by the Nutrition

Research Ethics Committee (Biotechnical Faculty, University of

Ljubljana), under the identiﬁcation number KEP-1-2/2020 on

10 February 2020. The study was conducted in full compliance

with the principles laid out in the Declaration of Helsinki.

Participation in the study was voluntary. All of the subjects

signed a written informed consent form before participating.

They were informed that they can withdraw from the study

at any time with no consequences. The study was conducted

between February and April 2020 and included data collection

using the FFQ and a 5-day DR. The participants received all

the required information (and instructions) in oral and written

format at individual meetings. To assess reproducibility, the

participants were asked to ﬁll out the sqFFQ/SI two times: the

ﬁrst one (sqFFQ/SI1) was ﬁlled out at the beginning of the study,

and the second one (sqFFQ/SI2) was ﬁlled out approximately

6 weeks later (Figure 1). The participants were asked not to alter

dietary habits between sqFFQ/SI1 and sqFFQ/SI2 if possible.

It should be noted that the second one was conducted during

the SARS-CoV-2 epidemic. To evaluate the validity of the

questionnaire, the participants were requested to complete a 5-

day DR during the time between both administered sqFFQ/SI.

The participants were free to choose any 3 week/2 weekend days

in that time period.

Study population

The sqFFQ/SI was validated among a group of Slovenian

adults, aged between 18 and 65 years, mainly from central

Slovenia and the Savinja statistical region. The subjects were

enrolled with the use of invitations via social media proﬁles from

the oﬃcial Nutrition Institute proﬁle, and personal invitations.

The exclusion criteria were diagnosis of chronic disease,

pregnant or breastfeeding women, and speciﬁc diets (vegan diet,

ketogenic diet, energy-restricted diets, and diets due to medical

reasons). It should be noted that vegetarians were not excluded.

All the required information regarding inclusion/exclusion

criteria was presented before the beginning of the study.

Semi quantitative food frequency

questionnaire for Slovenian population

For this study, a semi-quantitative FFQ adapted for the

Slovenian population was used (sqFFQ/SI), in which the

frequency of food consumption and the size of portions

are deﬁned (36). The tool included food products that were

previously identiﬁed as important sources of vitamin D in

Slovenia (15). Although Slovenia does not have a mandatory

vitamin D fortiﬁcation of foods, some food groups are

commonly fortiﬁed (37) and were therefore included. The ﬁnal

sqFFQ/SI consisted of 22 food items that contain at least 0.03 µg

of vitamin D per 100 g, according to the reviewed literature

(38) and the selected food composition databases: Slovenian

Open Platform for Clinical Nutrition (OPEN) (39), McCance

and Widdowson’s The Composition of Foods (38), and the

United States Department of Agriculture (USDA) database (40).

The included food groups are presented in Table 1. For each

food group, we identiﬁed all the relevant food records in the

abovementioned food composition datasets and calculated the

category average content of vitamin D. We did not include the

use of pharmaceutical preparations.

The subjects were asked to rank their consumption

frequencies during the past year. Previously reported (17)

frequency options were implemented: multiple times a day,

daily, 4–6 times per week, 1–3 times per week, 1–3 times per

month, and rarely or never. Further, subjects were asked to

rank their usual portion sizes (in comparison to the indicated

reference portion size): (a) as indicated, (b) less than indicated

(speciﬁed as at least one-half smaller than the normal portion

size), and (c) more than indicated (speciﬁed as at least one-half

FIGURE 1

Study design.

Frontiers in Nutrition 03 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 4

Hribar et al. 10.3389/fnut.2022.950874

larger than the normal portion size). The complete sqFFQ/SI

is provided in the Supplementary material. The sqFFQ/SI was

carried out online and took approximately 10 min to complete.

Five-day dietary record

In line with the previously reported approach (31), the 5-

day DR was conducted on ﬁve typical random non-consecutive

days (3 weekdays and 3 days during the weekend). At the

ﬁrst meeting, the participants were given detailed instructions

on how to complete the DR. Participants were asked to

maintain their usual eating habits and record all consumed

foods and beverages in as much detail as possible (describing

the type/brand of food, the amount of food, the method

of preparation, and the recipes of composited dishes where

applicable). The amounts were preferably weighted and written

down in grams when participants had access to a kitchen scale.

Exceptionally, the amounts were estimated using illustration

material for diﬀerent portion sizes of typical foods using a

previously developed nationally adapted picture book (41). The

participants returned their completed 5-day DR via a pre-paid

postal service or in person.

Data processing and statistical analysis

The data collected by both sqFFQ/SI were used to calculate

the daily vitamin D intake (µg/day) based on the method

described in detail by Biro and Gee (42). The calculations were

performed using the selected serving size and average vitamin D

contents in 100 g of foods, as shown in Table 1.

Vitamin D intake (µg/day) was further determined using a

5-day DR using the online nutrition analysis software OPEN,

which is linked to the food composition database (43). Due to

some missing information regarding the vitamin D content in

some foods, the OPEN database was updated in cooperation

with the software owner, the Jožef Stefan Institute (JSI). The

missing data were updated with data available in the USDA

database (40), the National Food Composition Database in

Finland (Fineli) (44), and McCance and Widdowson’s The

Composition of Foods (38).

The obtained data were statistically analyzed with the

IBM SPSS version 27, Statistics program (IBM SPSS,

IBM Corp., Armonk, NY, USA) (45). We investigated the

validity (external validation compared with the results of

the 5-day DR) and reproducibility of the method (internal

validation comparing results obtained two times: sqFFQ/SI1

and sqFFQ/SI2) (46). Descriptive characteristics (means,

median, and proportions) for the daily vitamin D intakes

were calculated.

The estimated daily vitamin D intakes were grouped for

cross-classiﬁcation according to tertiles. In the analyses, we

regarded the estimations as good if less than 10% of the

participants were grossly misclassiﬁed into the opposite tertiles

and at least 50% of the participants were correctly classiﬁed

(47). In the Kappa analyses, we considered Kappa values below

0.20 to have a poor agreement, between 0.20 and 0.60 as

having an acceptable agreement, and over 0.60 as having good

agreement (48).

The normality of distribution was tested with the Shapiro–

Wilk test. The analysis of correlations between the results

obtained in the assessment of validity (sqFFQ/SI1 compared

with a 5-day DR) and the assessment of reproducibility

(comparison between sqFFQ/SI1 and sqFFQ/SI2) was used,

where Spearman’s correlation was applied. Correlation

coeﬃcients of less than 0.20 were a poor outcome; those

between 0.20 and 0.49 were acceptable, and those of 0.50 or

higher was considered a good outcome (48).

TABLE 1 Reference serving sizes and vitamin D content in 100 g of

the foods used in the semi-quantitative Food Frequency

Questionnaire (sqFFQ/SI).

Food group Reference serving

size (g/ml)

Vitamin D

(µg/100 g)

Sardines, trout, salmon, and carp 120 7.84

Sea bass, tuna, cod, common sole,

blue tilapia, and other ﬁsh

120 3.23

Canned ﬁsh 80 4.31

Plant-based milk alternatives: rice

milk, soy milk, etc.

250 0.47

Semi-skimmed milk (1.5%

milkfat), cocoa drink, and milk

drinks

200 0.03

Whole milk (3.5% milkfat), a

cocoa drink containing whole

milk, milk drinks

200 0.09

Semi-skimmed (1.5% milkfat)

ﬂavored or plain yogurt

150 0.03

Whole milk (3.5% milkfat)

ﬂavored or plain yogurt

150 0.06

Hard cheese: Gouda cheese,

Edam cheese, etc.

30 0.9

Blue cheese 20 0.39

Cottage cheese, mozzarella, other

types of processed cheese

50 0.28

Ice cream 40 0.25

Butter 6 1.66

Margarine 6 2.5

Eggs 50 2.9

Egg pasta 100 0.28

Red meat 100 0.48

Poultry 100 0.26

Meat products 40 0.86

Calf’s liver 60 1.2

Mushrooms 100 0.18

Cakes, pastry, and muﬃns 70 0.31

Frontiers in Nutrition 04 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 5

Hribar et al. 10.3389/fnut.2022.950874

In all of the comparisons, signiﬁcance was considered

at p < 0.05. A Bland–Altman plot was further used for

the validation and reproducibility assessment. Since the data

were not normally distributed, we used log transformation, as

previously proposed (49). A Bland–Altman index below 5% was

interpreted as good, as suggested before in similar research (23,

28, 49–51).

Results

A total of 55 participants volunteered to participate. The

ﬁnal sample included 54 participants, as one of the individuals

withdrew from the study (due to lack of time). The sample

was represented by 37 women (69%) and 17 men (31%). The

average age was 32.7 years (±13.6 years). Other characteristics

of the population [age and body mass index (BMI)] are shown in

Table 2. The participants completed two sqFFQ/SIs on average

46 days apart (sqFFQ/SI1 and sqFFQ/SI2, respectively), and a 5-

day DR according to a study design, presented in Figure 1. The

mean daily vitamin D intake was 3.50 ± 1.91 µg according to

the 5-day DR, and 2.99 ± 1.35 and 3.31 ± 1.67 µg according

to the sqFFQ/SI1 and sqFFQ/SI2, respectively (Table 3). Since

none of the participants achieved the nationally recommended

daily intake of vitamin D (20 µg), we analyzed the data with a

cut-oﬀ value for the LRNI (2.5 µg) and NRV (5 µg). Overall,

the NRV threshold was not met by 87.0% of subjects according

TABLE 2 Study population description.

Parameter Criteria Number (%)

Participants (total) 54 (100)

Sex Men 17 (31.5)

Women 37 (69.5)

Age 19–24 28 (52.9)

25–65 26 (48.1)

Body mass index

categories

<18.5 kg/m

(underweight) 2 (3.7)

18.5–24.9 kg/m

(normal weight) 37 (68.5)

25.0–29.9 kg/m

(overweight) 10 (18.5)

>30.0 kg/m

(obese) 5 (9.3)

TABLE 3 Daily vitamin D intake estimated with a 5-day dietary record

(DR) and semi-quantitative Food Frequency Questionnaires (sqFFQ/SI)

administered 6 weeks apart.

sqFFQ/SI1 sqFFQ/SI2 5-day DR

Mean ± SD (µg) 2.99 ± 1.35 3.31 ± 1.67 3.50 ± 1.91

Median (µg) 2.61 2.94 3.04

Minimum (µg) 0.44 0.58 0.97

Maximum (µg) 7.08 8.19 10.31

<2.5 µg (%) 42.6 40.7 35.2

<5 µg (%) 90.7 83.3 87

to the 5-day DRs, 90.7% according to the sqFFQ/SI1, and 83.3%

according to the sqFFQ/SI2. On the other hand, the lower LRNI

threshold was not met by 35.2, 42.6, and 40.7%, respectively.

Validity

The validity of the sqFFQ/SI1 for the estimation of daily

vitamin D intake was analyzed with comparison to the 5-day

DR. The estimated intakes were analyzed with Spearman’s rank

correlation for the sqFFQ/SI1 and 5-day DR (Figure 2). The

sqFFQ/SI1 was signiﬁcantly correlated (p < 0.05) with the 5-day

DR, with a correlation coeﬃcient of 0.268; the mean diﬀerence

between both methods was 0.514 µg (SD: 0.318 µg). Due to the

non-normal distribution, further comparison of the 5-day DR

and sqFFQ/SI1 using Bland–Altman plots were carried out with

log-transformed data (Figure 3). The Bland–Altman index of

the logarithmic model was good (3.70%). Further, we analyzed

the percentages of subjects classiﬁed into the same vitamin D

intake tertile (Table 4). When comparing the sqFFQ/SI1 and the

5-day DR, 42.6% of the participants were categorized into the

same tertile, and 16.7% into the opposite tertile. This indicates a

low agreement; the Kappa coeﬃcient was 0.139.

Reproducibility

To investigate the reproducibility of the sqFFQ/SI, we

compared the daily vitamin D intake as estimated with

the sqFFQ/SI1 and sqFFQ/SI2, which were administered

approximately 6 weeks apart. The correlation between the

sqFFQ/SI1 and sqFFQ/SI2 was highly signiﬁcant (p < 0.001),

with a correlation coeﬃcient of 0.689 (Figure 4). The mean

diﬀerence between measurements was 0.318 µg (SD: 0.291 µg).

Furthermore, the log-transformed Bland–Altman plot showed

good reproducibility with an index of 3.70% (Figure 5). When

testing the sqFFQ/SI1 for reproducibility, the analysis of the

tertiles showed acceptable agreement; 59.3% of the subjects

were categorized into the same tertile, and there were no

classiﬁcations into opposite tertile, while the Kappa value was

acceptable (0.389) (Table 4).

Discussion

Vitamin D is a crucial micronutrient for optimal human

health in all life stages, and we should strive to achieve optimal

status across all populations. Besides UVB-induced cutaneous

synthesis, food intake is an important source of vitamin D (5–7).

Vitamin D intake can be estimated using various methods, with

the FFQ being one of the less burdensome methods. The FFQ

is user friendly and time/cost eﬃcient (25). Convenient tools

for intake estimation are important for the eﬃcient assessment

Frontiers in Nutrition 05 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 6

Hribar et al. 10.3389/fnut.2022.950874

FIGURE 2

Analysis of correlation for daily vitamin D intake estimated with semi-quantitative Food Frequency Questionnaire 1 (sqFFQ/SI1) and 5-day dietary

record (correlation coefﬁcient = 0.268; p < 0.05).

FIGURE 3

Bland–Altman plot comparing daily vitamin D intake estimated with semi-quantitative Food Frequency Questionnaire 1 (sqFFQ/SI1) and a 5-day

dietary record (Bland–Altman index: 3.70%).

Frontiers in Nutrition 06 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 7

Hribar et al. 10.3389/fnut.2022.950874

TABLE 4 Count and percentages of subjects classiﬁed into the

same/opposite vitamin D intake tertile.

Category sqFFQ/SI1

vs. 5-day DR

sqFFQ/SI1 vs.

sqFFQ/SI2

Subjects classiﬁed

into the same tertile

N 23 32

% 42.6 59.3

Subjects

misclassiﬁed into the

opposite tertile

N 9 0

% 16.7 0

DR, dietary record; sqFFQ/SI1, semi-quantitative Food Frequency Questionnaire 1;

sqFFQ/SI2, semi-quantitative Food Frequency Questionnaire 2.

of the risk of vitamin D deﬁciency, particularly in the absence

of endogenous synthesis. To accurately assess the dietary intake

of vitamin D in the Slovenian population we developed a semi-

quantitative FFQ and tested its validity and reproducibility using

5-day DR and repeated sqFFQ/SI, respectively. The estimated

mean daily vitamin D intakes in our study were 3.50, 2.99,

and 3.31 µg for the 5-day DR, sqFFQ/SI1, and sqFFQ/SI2,

respectively. We did not observe a higher mean intake with

the FFQ (in comparison to the 5-day DR), unlike some other

validation studies (29, 52).

The validity and reproducibility were tested using Bland–

Altman plots, a recommended “gold-standard” approach by

which to compare results from diﬀerent methods observing

the same variable (53). Our results show that the developed

sqFFQ/SI is fairly valid and reproducible; only 3.70% of the

data points were outside the 95% limits of agreements for both

validity and reproducibility. Other research investigating similar

a topic reported from 2.7 to 6.3% of data points outside the

95% limits of agreement using Bland–Altman plot (23, 28).

Additionally, Spearman’s correlation was signiﬁcant both for

validity (<0.05) and reproducibility (<0.001). The correlation

coeﬃcients were acceptable and good (0.268 and 0.689,

respectively). In similar studies comparing multiple day dietary

vitamin D intake with FFQ, signiﬁcant correlation coeﬃcients

ranged from 0.21 to 0.83 for validity and from 0.62 to 0.82 for

reproducibility (23, 26, 28, 29, 52). It should be noted that due

to the complexity of the estimation of micronutrient intakes,

correlation coeﬃcients above 0.2 are considered acceptable, and

coeﬃcients above 0.7 are rarely reported (32). However, the

thresholds for acceptable correlations are not well harmonized

(25, 48), and we should take caution when evaluating the

outcomes. Analyses of terciles in our case showed less agreement

than in some other studies. Altogether, in the validity study,

42.6% of subjects were classiﬁed in the same tercile (Kappa

coeﬃcient: 0.139; poor agreement), while some other studies

reported up to 64% (28, 29), but we must note that the

cross-classiﬁcation is a relatively crude measurement (29). On

the hand, we observed better results in analyses of terciles

in the reproducibility study (59.3%; Kappa coeﬃcient: 0.389;

acceptable). Other studies also reported lower diﬀerences in the

reproducibility of FFQs, in comparison to validity testing with

DRs (23, 54), which might be aﬀected by the limited ability of

DRs to capture dietary patterns, related to vitamin D intake.

In a recent study, it was shown that in Slovenia vitamin

D deﬁciency is highly prevalent, particularly in the wintertime

when dietary intake becomes the main source of vitamin D.

In the winter months, ca. 80% of adults and elderly people

were shown to be vitamin D deﬁcient (55), and the mean

daily vitamin D intakes were 2.9 and 2.5 µg, respectively (15).

Globally, various FFQs were developed and regionally adapted

to estimate vitamin D intakes (23, 26, 28–34); however, to the

best of our knowledge, there is no such tool available for use in

the Slovenian population.

The intake of nutrients can be estimated using a variety

of methods that have diﬀerent levels of accuracy for diﬀerent

nutrients. For nutrients that are found in a limited number

of foods, the use of short-period DRs can pose a risk of

not capturing a typical dietary pattern, and it is therefore

recommended to follow food intake over a period of several

days. On the contrary, although the FFQ is much simpler

to use, this method can better capture food consumption

patterns over a longer period (26). In the case of vitamin

D, the intake estimation is particularly challenging due to

notable day-to-day variations as vitamin D-rich foods (i.e.,

ﬁsh) are seldom consumed (24, 26). This, of course, aﬀects the

estimation of daily vitamin D intake when diﬀerent methods

are used. For example, in a nationally representative Slovenian

SI. Menu study, 72.8% of adults were recognized as sea ﬁsh

consumers when two 24 h dietary recalls were used, while

the Food Propensity Questionnaire method identiﬁed 80.8%

as true consumers (15).We developed an FFQ that covers the

most important contributors to vitamin D intake in Slovenia,

including eggs, ﬁsh, and ﬁsh products, meat and meat products,

milk and milk products, and commonly fortiﬁed foods, such

as plant-based milk alternatives (15, 37). The validation of the

FFQ (sqFFQ/SI1) was conducted on 54 participants using a 5-

day DR as a reference method. Despite the abovementioned

limitations, the DR is a commonly used reference method in

such validation studies (56). We also tested the reproducibility,

using a repeated FFQ (sqFFQ/SI2) administered 6 weeks after

the ﬁrst measurement.

To evaluate the validity and reproducibility we used

various approaches. The results are showing that validity varied

from poor to good, and good for reproducibility. We have

demonstrated that the proposed FFQ is acceptable and is

therefore an appropriate tool for the eﬀective assessment of

habitual vitamin D intake on an individual level. Overall,

we observed higher reproducibility than validity. However,

such tools are also commonly used in population studies.

Therefore, we further compared the estimated mean vitamin

D intakes between the tested methods and literature data. The

diﬀerence between the mean vitamin D intake according to

Frontiers in Nutrition 07 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 8

Hribar et al. 10.3389/fnut.2022.950874

FIGURE 4

Analysis of correlation for daily vitamin D intake estimated with semi-quantitative Food Frequency Questionnaire 1 (sqFFQ/SI1) and 2

(sqFFQ/SI2) (correlation coefﬁcient = 0.689; p < 0.001).

FIGURE 5

Bland–Altman plot comparing daily vitamin D intake estimated with a semi-quantitative Food Frequency Questionnaire 1 (sqFFQ/SI1) and 2

(sqFFQ/SI2) (Bland–Altman index: 3.70%).

both of the tested methods was small (0.51 µg) and statistically

insigniﬁcant. With consideration of the recommended daily

vitamin D intake (20 µg), the clinical importance of such a

diﬀerence is minimal. Similar diﬀerences were also observed in

other similar studies, for example, in the study by Kiely et al.

in their comparison of the FFQ and 14-day DR results (29).

Furthermore, our results are comparable with mean vitamin D

intakes reported for the general Slovenian population. Vitamin

Frontiers in Nutrition 08 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 9

Hribar et al. 10.3389/fnut.2022.950874

D intake was recently investigated in a nationally representative

SI. Menu study (15). The weighted population mean intake

was estimated with the multiple source method (MSM), using

two 24 h recalls and the Food Propensity Questionnaire. The

estimated mean vitamin D intake in adults (18–64 years) was

2.85 µg (15), comparable to the results in our study (sqFFQ/SI1:

2.99 µg). A recent systematic review also highlighted that

vitamin D intakes in other studies in the Slovenian population

were below 5 µ g (16).

Although the developed tool was shown as valid and

reproducible, some limitations need to be noted. While we

followed the recommendation that validity studies should be

conducted on at least 50 subjects (31), a bigger sample would

be beneﬁcial to check the validity in more speciﬁc population

groups. Furthermore, we did not use biological biomarkers

of vitamin D status [serum 25(OH)D concentration], but we

should note that this biomarker is seriously aﬀected by UVB-

induced endogenous vitamin D biosynthesis, which results in

major inter-individual diﬀerences. The limited use of blood

biomarkers for such validation studies in the case of vitamin D

was noted also in other studies (26, 33). Moreover, we should

note that while majority (72.2%) of our study participants were

with BMI < 25 kg/m

, we also had some overweight/obese

subjects (18.5 and 9.3%, respectively), where food intake

misreporting might be more common. We have not excluded

those from the analyses, because vitamin D intake screening is

also very relevant in this population group. At last, it should

be said that we tested the tool on healthy, non-pregnant, no-

lactating, adult, omnivore populations. We suggest that the

described sqFFQ/SI is further tested on other populations of

public health interest.

Conclusion

The estimation of one’s usual daily vitamin D intake is

a challenging task, regardless of the method used, due to its

major day-to-day variability. Building on previously established

methods and major contributors to vitamin D intake in the

Slovenian population, we developed a simple one-page semi-

quantitative FFQ (sqFFQ/SI) for the quick estimation of one’s

usual daily vitamin D intake. To the best of our knowledge,

the described tool is the ﬁrst FFQ adapted for the Slovenian

population. The Bland–Altman plot analyses showed a good

level of agreement between the developed sqFFQ/SI and the

standard 5-day DR method, as well as a good reproducibility,

with less than 5% of the outliers falling outside of the

agreement limit and a signiﬁcant correlation being observed.

Further analyses of correlation showed acceptable and good

correlation, whereas Kappa analyses of terciles showed poor

and acceptable agreement tor validity and reproducibility,

respectively. Considering the analyses results, this tool will be

used in further population studies and for the development

of a screening tool for the assessment of the risk for vitamin

D deﬁciency in healthy non-pregnant, no-lactating, adult, and

omnivore populations. Due to the high prevalence of vitamin D

deﬁciency, such a method is important not only for researchers

but also for clinical practice and policymakers. It should be

noted that the developed tool is very valuable for use in other

countries in the Central European region due to similar food

policies and dietary patterns. However, minor modiﬁcations

might be appropriate for speciﬁc populations.

Data availability statement

The raw data supporting the conclusions of this article will

be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed

and approved by the Nutrition Research Ethics Committee

(Biotechnical Faculty, University of Ljubljana), KEP-1-2/2020.

The patients/participants provided their written informed

consent to participate in this study.

Author contributions

KŽm and IP: conceptualization, funding acquisition,

methodology, and supervision. MH and KŽl: data curation,

formal analysis, investigation, and writing—original draft. KŽm:

project administration and resources. MH: validation and

visualization. MH, KŽm, KŽl, and IP: writing—review and

editing. All authors contributed to the article and approved the

submitted version.

Funding

This study was conducted within the national research

program P3-0395 “Nutrition and Public Health” and project

Nutri-D “Challenges in achieving adequate vitamin D

status in the adult population” (L7-1849), supported by

the Slovenian Research Agency and Ministry of Health of the

Republic of Slovenia.

Acknowledgments

We acknowledge the support of other member of the

research group, particularly Hristo Hristov (Nutrition Institute,

Ljubljana, Slovenia) and Urška Blaznik (National Institute of

Public Health, Ljubljana, Slovenia).

Frontiers in Nutrition 09 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 10

Hribar et al. 10.3389/fnut.2022.950874

Conﬂict of interest

The authors declare that the research was conducted in the

absence of any commercial or ﬁnancial relationships that could

be construed as a potential conﬂict of interest.

Publisher’s note

All claims expressed in this article are solely those of the

authors and do not necessarily represent those of their aﬃliated

organizations, or those of the publisher, the editors and the

reviewers. Any product that may be evaluated in this article, or

claim that may be made by its manufacturer, is not guaranteed

or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be

found online at: https://www.frontiersin.org/articles/10.3389/

fnut.2022.950874/full#supplementary-material

References

1. Holick MF, Binkley NC, Bischoﬀ-Ferrari HA, Gordon CM, Hanley DA,

Heaney RP, et al. Evaluation, treatment, and prevention of vitamin D deﬁciency:

an endocrine society clinical practice guideline. J Clin Endocrinol Metab. (2011)

96:1911–30. doi: 10.1210/jc.2011-0385

2. Autier P, Boniol M, Pizot C, Mullie P. Vitamin D status and ill health: a

systematic review. Lancet Diabetes Endocrinol. (2014) 2:76–89. doi: 10.1016/S2213-

8587(13)70165-7

3. Molina P, Carrero JJ, Bover J, Chauveau P, Mazzaferro S, Torres PU, et al.

Vitamin D, a modulator of musculoskeletal health in chronic kidney disease. J

Cachexia Sarcopenia Muscle. (2017) 8:686–701. doi: 10.1002/jcsm.12218

4. Holick MF. Vitamin D deﬁciency. New Engl J Med. (2007) 375:266–81. doi:

10.1056/NEJMra070553

5. Calvo MS, Whiting SJ, Barton CN. Vitamin D intake: a global perspective of

current status. J Nutr. (2005) 135:310–6. doi: 10.1093/jn/135.2.310

6. Spiro A, Buttriss JL. Vitamin D: an overview of vitamin D status and intake in

Europe. Nutr Bull. (2014) 39:322–50. doi: 10.1111/nbu.12108

7. O’Mahony L, Stepien M, Gibney MJ, Nugent AP, Brennan L. The potential

role of vitamin D enhanced foods in improving vitamin D status. Nutrients. (2011)

3:1023–41. doi: 10.3390/nu3121023

8. World Health Organization [WHO]. Vitamin and Mineral Requirements in

Human Nutrition. 2nd ed. Geneva: World Health Organization (2005).

9. German Nutrition Society [GNS]. New reference values for vitamin D. Ann

Nutr Metab. (2012) 60:241–6. doi: 10.1159/000337547

10. EFSA Panel on Dietetic Products and Allergies. Dietary reference values for

vitamin D. EFSA J. (2016) 14:e04547. doi: 10.2903/j.efsa.2016.4547

11. European Commission [EC]. Commission Regulation (EC) No 1170/2009 of

30 November 2009 Amending Directive 2002/46/EC of the European Parliament and

of Council and Regulation (EC) No 1925/2006 of the European Parliament and of

the Council as Regards the Lists of Vitamin and Minerals and Their Forms that Can

Be Added to Foods, Including Food Supplements. Brussels: European Commission

(2009).

12. Mensink GB, Fletcher R, Gurinovic M, Huybrechts I, Lafay L, Serra-Majem

L, et al. Mapping low intake of micronutrients across Europe. Br J Nutr. (2013)

110:755–73. doi: 10.1017/S000711451200565X

13. European Food Safety Authority [EFSA]. Scientiﬁc opinion on the tolerable

upper intake level of vitamin D. EFSA J. (2012) 10:2813. doi: 10.2903/j.efsa.2012.

2813

14. Roman Viñas B, Ribas Barba L, Ngo J, Gurinovic M, Novakovic R, Cavelaars

A, et al. Projected prevalence of inadequate nutrient intakes in Europe. Ann Nutr

Metab. (2011) 59:84–95. doi: 10.1159/000332762

15. Hribar M, Hristov H, Lavriša Z, Korouši

c Seljak B, Matej G, Blaznik U, et al.

Vitamin D intake in slovenian adolescents, adults, and the elderly population.

Nutrients. (2021) 13:3528. doi: 10.3390/nu13103528

16. Hribar M, Benedik E, Gregori

c M, Blaznik U, Kukec A, Hristov H, et al.

A systematic review of vitamin D status and dietary intake in various Slovenian

populations. Zdarvstveno Varstvo. (2022) 61:55–72. doi: 10.2478/sjph-2022-0009

17. Lichthammer A, Nagy B, Orbán C, Tóth T, Csajbók R, Molnár S, et al. A

comparative study of eating habits, calcium and vitamin D intakes in the population

of Central-Eastern European countries. New Med. (2015) 19:66–70.

18. Mithal A, Wahl DA, Bonjour JP, Burckhardt P, Dawson-Hughes B, Eisman JA,

et al. Global vitamin D status and determinants of hypovitaminosis D. Osteoporos

Int. (2009) 20:1807–20. doi: 10.1007/s00198-009-0954-6

19. LeFevre ML, U.S. Preventive Services Task Force. Screening for vitamin

D deﬁciency in adults: U.S. Preventive Services Task Force recommendation

statement. Ann Intern Med. (2015) 162:133–40. doi: 10.7326/M14-2450

20. Kennel KA, Drake MT, Hurley DL. Vitamin D deﬁciency in adults: when to

test and how to treat. Mayo Clin Proc. (2010) 85:752–7; quiz 7-8. doi: 10.4065/mcp.

2010.0138

21. Ross AC, Taylor CL, Yaktine AL, Del Valle HB. (eds). Institute of Medicine

(US) Committee to Review Dietary Reference Intakes for Vitamin D and Calcium.

Dietary Reference Intakes for Calcium and Vitamin D. Washington, DC: National

Academic Press (2011).

22. Willett WC. Nutritional Epidemiology. New York, NY: Oxford University

Press (1998). doi: 10.1093/acprof:oso/9780195122978.001.0001

23. Glabska D, Uroic V, Guzek D, Pavic E, Bival S, Jaworska K, et al. The

possibility of applying the vitamin D brief food frequency questionnaire as a tool

for a country with no vitamin D data in food composition tables. Nutrients. (2018)

10:1278. doi: 10.3390/nu10091278

24. Verbeke W, Vackier I. Individual determinants of ﬁsh consumption:

application of the theory of planned behaviour. Appetite. (2005) 44:67–82. doi:

10.1016/j.appet.2004.08.006

25. Cade J, Thompson R, Burley V, Warm D. Development, validation and

utilisation of food-frequency questionnaires - a review. Public Health Nutr. (2002)

5:567–87. doi: 10.1079/PHN2001318

26. Bärebring L, Amberntsson A, Winkvist A, Augustin H. Validation of dietary

vitamin D intake from two food frequency questionnaires, using food records and

the biomarker 25-Hydroxyvitamin D among pregnant women. Nutrients. (2018)

10:745. doi: 10.3390/nu10060745

27. Perez Rodrigo C, Aranceta J, Salvador G, Varela-Moreiras G. Food frequency

questionnaires. Nutr Hosp. (2015) 31(Suppl. 3):49–56.

28. Glabska D, Guzek D, Sidor P, Wlodarek D. Vitamin D dietary intake

questionnaire validation conducted among young polish women. Nutrients. (2016)

8:36. doi: 10.3390/nu8010036

29. Kiely M, Collins A, Lucey AJ, Andersen R, Cashman KD, Hennessy A.

Development, validation and implementation of a quantitative food frequency

questionnaire to assess habitual vitamin D intake. J Hum Nutr Diet. (2016) 29:495–

504. doi: 10.1111/jhn.12348

30. Wu H, Gozdzik A, Barta JL, Wagner D, Cole DE, Vieth R, et al.

The development and evaluation of a food frequency questionnaire used in

assessing vitamin D intake in a sample of healthy young Canadian adults

of diverse ancestry. Nutr Res. (2009) 29:255–61. doi: 10.1016/j.nutres.2009.

03.006

31. Pritchard JM, Seechurn T, Atkinson SA. A food frequency

questionnaire for the assessment of calcium, vitamin D and vitamin K:

a pilot validation study. Nutrients. (2010) 2:805–19. doi: 10.3390/nu20

80805

32. Taylor C, Lamparello B, Kruczek K, Anderson EJ, Hubbard J, Misra M.

Validation of a food frequency questionnaire for determining calcium and vitamin

D intake by adolescent girls with anorexia nervosa. J Am Diet Assoc. (2009)

109:479–85, 485.e1–3. doi: 10.1016/j.jada.2008.11.025

Frontiers in Nutrition 10 frontiersin.org

fnut-09-950874 September 16, 2022 Time: 16:25 # 11

Hribar et al. 10.3389/fnut.2022.950874

33. Djekic-Ivankovic M, Weiler HA, Nikolic M, Kadvan A, Gurinovic M, Mandic

LM, et al. Validity of an FFQ assessing the vitamin D intake of young Serbian

women living in a region without food fortiﬁcation: the method of triads model.

Public Health Nutr. (2016) 19:437–45. doi: 10.1017/S136898001500138X

34. Park Y, Kim SH, Lim YT, Ha YC, Chang JS, Kim IS, et al. Validation of a new

food frequency questionnaire for assessment of calcium and vitamin d intake in

korean women. J Bone Metab. (2013) 20:67–74. doi: 10.11005/jbm.2013.20.2.67

35. Thompson, FE, Subar AF. Dietary assessment methodology. In: Coulston

AM, Boushey CJ, Ferruzzi MG, Delahanty LM editors. Nutrition in the Prevention

and Treatment of Disease. Amsterdam: Elsevier (2017). p. 5–48. doi: 10.1016/B978-

0-12-802928-2.00001-1

36. Gibson RS. Principles of Nutritional Assessment. 2nd ed. New York, NY:

Oxford University Press (2005). 908 p.

37. Kruši

c S, Hribar M, Hafner E, Žmitek K, Pravst I. Use of branded food

composition databases for the exploitation of food fortiﬁcation practices: a case

study on vitamin D in the slovenian food supply. Front Nutr. (2022) 8:775163.

doi: 10.3389/fnut.2021.775163

38. Roe M, Pinchen H, Church S, Finglas P. McCance and Widdowson’s the

composition of foods seventh summary edition and updated composition of foods

integrated dataset. Nutr Bull. (2015) 40:36–9. doi: 10.1111/nbu.12124

39. Korošec M, Golob T, Bertoncelj J, Stibilj V, Seljak BK. The Slovenian food

composition database. Food Chem. (2013) 140:495–9. doi: 10.1016/j.foodchem.

2013.01.005

40. United States Department of Agriculture [USDA]. United States Department

of Agriculture: Food data central Washington: United States Department of

Agriculture, Agricultural Research Service. (2020). Available online at: https://fdc.

nal.usda.gov/ (accessed April, 2021).

41. Vede T. Slikovno Gradivo s Prikazom Velikosti Porcij. Ljubljana: Nacionalni

inštitut za javno zdravje (2016).

42. Biró L, Gee J. Development of a ﬂexible, updatable, user-friendly electronic

food frequency questionnaire. Acta Alimentaria. (2011) 40:117–27. doi: 10.1556/

AAlim.40.2011.1.14

43. Odprta Platforma za Klini

cno Prehrano [OPKP]. Odprta Platforma za

Klinièno Prehrano [Spletno Orodje]. Ljubljana: Institut Jožef Stefan (2021).

44. Fineli, Nutrition Unit of the National Institute for Health and Welfare. Fineli

- the National Food Composition Database Helsinki: National Institute for Health

and Welfare. (2019). Available online at: https://ﬁneli.ﬁ/ﬁneli/en/index (accessed

August 2021).

45. IBM Corp. IBM SPSS Statistics for Windows. Version 22.0 ed. New York, NY:

IBM Corp. (2013).

46. Willet, W, Lenart E. Reproducibility and validity of food frequency

questionnaires. 3rd ed. In: Walter Willet editor. Nutritional Epidemiology. Oxford,

UK: Oxford University Press (2013). doi: 10.1093/acprof:oso/9780199754038.003.

0006

47. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al.

Statistical approaches for assessing the relative validity of a food-frequency

questionnaire: use of correlation coeﬃcients and the kappa statistic. Public Health

Nutr. (2003) 6:313–21. doi: 10.1079/PHN2002429

48. Lombard MJ, Steyn NP, Charlton KE, Senekal M. Application and

interpretation of multiple statistical tests to evaluate validity of dietary intake

assessment methods. Nutr J. (2015) 14:40. doi: 10.1186/s12937-015-0027-y

49. Giavarina D. Understanding bland altman analysis. Biochem Med (Zagreb).

(2015) 25:141–51. doi: 10.11613/BM.2015.015

50. Lee Y, Park K. Reproducibility and validity of a semi-quantitative FFQ for

trace elements. Br J Nutr. (2016) 116:864–73. doi: 10.1017/S0007114516002622

51. Bland JM, Altman DG. Statistical methods for assessing agreement between

two methods of clinical measurement. Lancet (London, England). (1986) 1:307–10.

doi: 10.1016/S0140-6736(86)90837-8

52. Perreault M, Xu VY, Hamilton S, Wright D, Foster W, Atkinson SA.

Validation of a food frequency questionnaire for bone nutrients in pregnant

women. Can J Diet Pract Res. (2016) 77:133–9. doi: 10.3148/cjdpr-2016-002

53. Do

gan N. Bland-Altman analysis: a paradigm to understand correlation and

agreement. Turkish J Emerg Med. (2018) 18:139–41. doi: 10.1016/j.tjem.2018.09.001

54. Itkonen ST, Erkkola M, Skaﬀari E, Saaristo P, Saarnio EM, Viljakainen HT,

et al. Development and validation of an interview-administered FFQ for assessment

of vitamin D and calcium intakes in Finnish women. Br J Nutr. (2016) 115:1100–7.

doi: 10.1017/S0007114515005474

55. Hribar M, Hristov H, Gregori

c M, Blaznik U, Zaletel K, Oblak A, et al.

Nutrihealth study: seasonal variation in vitamin D status among the slovenian

adult and elderly population. Nutrients. (2020) 12:1838. doi: 10.3390/nu120

61838

56. Potosky AL, Block G, Hartman AM. The apparent validity of diet

questionnaires is inﬂuenced by number of diet-record days used for comparison.

J Am Diet Assoc. (1990) 90:810–3. doi: 10.1016/S0002-8223(21)01632-1

Frontiers in Nutrition 11 frontiersin.org