The Ukrainian version of the pediatric Canadian acute respiratory illness and flu scale: a linguistic validation study

Background

Due to universal prevalence, acute respiratory tract infections (ARTIs) require considerable health care and social resources that include medical visits, hospitalizations, a sick person/caregiver working day missed, and costs for medication. However, the assessment of ARTIs is difficult because it is based on the subjective perceptions of patients and their caregivers.

Until recently, there were no validated patient-related surveys for ARTIs either in adults or in children. Nowadays, the Wisconsin upper respiratory symptom survey is a recognized instrument for the assessment of ARTIs in adults, while others are emerging.^1,2 Despite ARTIs in adults sharing common features with pediatric illness, the above-mentioned surveys include many items inconsistent with pediatric applications.

The only identifiable scale initially developed and validated as an instrument for the assessment of pediatric ARTIs has been the Canadian acute respiratory illness and flu scale (CARIFS).³ The CARIFS was successfully used in clinical trials conducted in native English-speaking countries.^4,5 Meanwhile, foreign language versions of the CARIFS are not available.

The objective of our trial was to adapt the English version of the CARIFS to the Ukrainian language. The present study describes the translation process and evaluates the translated version in a Ukrainian pediatric population in terms of consistency, sensitivity, reliability, and validity.

Materials and methods

CARIFS questionnaire

The original CARIFS included the perception of 18 items describing the child’s ARTI (in order as they appeared in the questionnaire): 1) “poor appetite”; 2) “not sleeping well”; 3) “irritable, cranky, fussy”; 4) “feels unwell”; 5) “low energy, tired”; 6) “not playing well”; 7) “crying more than usual”; 8) “needing extra care”; 9) “clinginess”; 10) “headache”; 11)“sore throat”; 12) “muscle aches or pains”; 13) “fever”; 14) “cough”; 15) “nasal congestion/runny nose”; 16) “vomiting”; 17) “not interested in what’s going on”; and 18) “unable to get out of bed”. Symptoms were rated as “none”, “minor”, “moderate”, “major” problem or “do not know/not applicable”, with corresponding values of “0”, “1”, “2”, “3”, or “missing value”. The maximum score indicated the worst disease was 54 units (18×3) (4). The items were classified into three domains: activity (1–7, 17, and 18), parental impact (8 and 9), and symptoms (10–16).

Translation

The translation of the CARIFS was performed according to the recommendation of Guillemin et al.⁶ Two bilingual pediatricians with Ukrainian as their first language translated the English version into Ukrainian. In the translation process, priority was given to the meaning rather than equality of words. During a primary meeting, consensus was achieved on the initial Ukrainian version based on those two translations. Subsequently, six bilingual speakers with English as their native language performed backward translation into English. Neither of these translators had any relevant medical knowledge or knew anything about the CARIFS. Where there was good correspondence between the source and backward translations, the respective Ukrainian item was approved for the final Ukrainian version.

Patients and assessments

The patients were otherwise healthy Ukrainian children of both sexes aged 3–12 years with symptoms and signs corresponding to ARTI. This infection was defined as the manifestation of rhinitis, cough, sore throat, and fever or a combination of these.

Children were excluded from the study in the case of chronic adenotonsillar pathology, recurrent otitis, sinusitis, respiratory allergy, gastroesophageal reflux disease, moderate-to-severe or severe disease of any organ systems, or families with cognitive or linguistic impairment that compromised their ability to complete the questionnaires.

The CARIFS questionnaire was administered four times. On the first day, the questionnaire was completed three times: 1) by a caregiver; 2) by a caregiver in 3–6 hours after the initial completion; 3) by a physician just after the second attempt was made by a caregiver. To evaluate the sensitivity and effect size (ES), the questionnaire was additionally completed by a caregiver on the day 5.

Statistical analysis

The descriptive statistics included median, interquartile range (IQR), maximum and minimum values, number and percent, skewness and kurtosis of the frequency distribution, and standard error. The nonparametric Mann–Whitney U test was used to assess the difference between the CARIFS scores recorded on the first and the second attempts, and the difference between the baseline and the fifth day scores.

The content validity assessed whether the items measured the full range of ARTI. We used the “floor and ceiling effect” to assess this. The distribution of the results were evaluated with the “floor” (the best) score to be between 0 and 3 and the “ceiling” (the worst) score to be between 51 and 54. The floor and the ceiling scores were calculated as the 5% tails of the full range of the scale (54×0.05=2.6 rounded 3).

Internal consistency was evaluated using correlation analysis between the items of the CARIFS and assessed using Cronbach’s α.⁷ This described the extent to which all the items in the CARIFS measured the same concept of ARTI in children. Acceptable values of the Cronbach’s α coefficient ranged from 0.70 to 0.95, with lower figures signifying insufficient number of questions or poor relationship between items, and higher figures suggesting excessive number of questions or their similarity.⁸ Cronbach’s α was recalculated, when an individual CARIFS item was deleted to evaluate the contribution of each to the internal consistency. Item–total correlation coefficient was calculated to assess the independence of the items in the CARIFS.

Means and standard deviations (SDs) were calculated from a square root of the CARIFS score as data were skewed. The sensitivity of the CARIFS was analyzed using the standardized response mean (SRM), calculated as the difference between the baseline mean score and the mean score on the day 5, divided by the SD of the difference. The ES was calculated as the difference between the mean score on the day 5 and the baseline mean score divided by the baseline SD.⁹

The test–retest reliability was measured as the agreement between two measurements taken 3–6 hours apart and expressed as the intra-class correlation coefficient (ICC). The ICC was measured for the total score and for three domains. Pearson’s correlation coefficient was used to correlate the responder’s and physician’s measurements of the CARIFS score and to assess the construct validity.

The analysis was performed with the use of STATISTICA 8 (StatSoft, Inc., Tulsa, OK, USA). The level of statistical significance was set at 0.05.

Ethics

The study received approval of the institutional review board at Lviv National Medical University. Parents signed the informed consent form before their children entered the study.

Results

Most items of the CARIFS questionnaire met their core meanings as it was seen from the backward translation process (Table 1).

Table 1 Backward translation of the CARIFS questionnaire into Ukrainian language Abbreviation: CARIFS, Canadian acute respiratory illness and flu scale.

Excellent agreement was achieved between translators for the backward translation of the illness-specific symptoms.

One hundred forty-nine Ukrainian questionnaires were dispensed to children. The typical patient participated in the study was a 5–6-year-old child (Table 2).

Table 2 Baseline characteristics of children with ARTI Note: Duration of ARTI before enrollment. Abbreviations: ARTI, acute respiratory tract infection; IQR, interquartile range; BMI, body mass index.

Children had an equal sex presentation and normal BMI. The questionnaires were mainly completed on the second day of ARTI, when body temperature, as measured in the armpit, was mildly hyperthermic.

All responders had completed the CARIFS questionnaire initially. On the first time, the median duration for completion of the questionnaire was 4.5 (IQR: 3.5–5.5) minutes. Seventy-three percent of the responders who filled in the questionnaire were mothers (n=109), 15% fathers (n=22), 10% grandfather or grandmother (n=15), and 2% other guardians (n=3). Only 66% (n=99) of families completed the questionnaire again in 3–6 hours. Fifty-three percent of CARIFS questionnaires (n=79) were completed by a physician.

The baseline CARIFS median score was 18.0 (IQR: 13.0–24.0). The skewness and kurtosis in the score distribution analysis were 0.641 and 0.225, respectively (Figure 1).

Figure 1 Distribution histogram of the CARIFS score in children. Abbreviation: CARIFS, Canadian acute respiratory illness and flu scale.

Taking a square root from the CARIFS score resulted in reduction of the skewness to 0.089, making the distribution of the score more normal. Minimum baseline CARIFS score was 5 and the maximum was 44, and none of the responders reported the ceiling or the floor scores. Most of the scores occupied 24%–44% of the full scale.

The disease-specific symptoms were the most reportable items in the CARIFS (Table 3).

Table 3 Baseline scores of the constituent items and Cronbach’s α coefficient for internal consistency of the Ukrainian version of the CARIFS Notes: Cronbach’s a, if the specific item is deleted; bold type numbers denote significant coefficient; CARIFS items present as they appear in the English version, but data are given for the Ukrainian version. Abbreviations: CARIFS, Canadian acute respiratory illness and flu scale; IQR, interquartile range; CI, confidence interval.

Fewer responders rated constitutional items, with “muscle pains” and “not interested in what’s going on” being the least mentioned. The median rate of individual scores ranged from “no” to “minor problem”, while the impact of fever, cough, and nasal symptoms was frequently recorded as a moderate problem.

The Cronbach’s α coefficient for the Ukrainian CARIFS was 0.805, and inter-item correlation coefficient was 0.191. The Cronbach’s α varied if the specific item was deleted. The minimal loss of the internal consistency of the CARIFS questionnaire took place, if the “cough” item was deleted, and the maximal loss occurred, when “not playing well” was deleted. Item to total correlation coefficients varied from 0.185 for the “cough” to 0.665 “for not playing well”.

One hundred percent of responders (n=149) reported a decrease in the CARIFS score on the day 5 with a median change of 12.00 (IQR: 8.0–18.0, P<0.0001). The SRM and ES for the squared root CARIFS score and its domains were high (Table 4).

Table 4 Sensitivity of the Ukrainian version of the CARIFS questionnaire as compared between the days 1 and 5 in children with ARTI Abbreviations: CARIFS, Canadian acute respiratory illness and flu scale; ARTI, acute respiratory tract infection; SRM, standardized response mean; ES, effect size.

In the test–retest reliability analysis of 99 questionnaires, the median CARIFS score for the first and the second measurement was 19.0 (IQR: 14.5–25.0) and 19.0 (IQR: 15.0–25.0), respectively, with a median change of 0.0 (IQR: −1.0 to 0.0, P=0.996). The ICC for the total CARIFS was 0.992, for the domain activity was 0.988, for symptoms was 0.984, and for parental impact was 0.981. All values were highly statistically significant (P<0.001). The Pearson’s correlation coefficient between the CARIFS scores completed by a responder and by a physician was 0.832 (P=0.004).

Discussion

This was the first attempt to evaluate the CARIFS questionnaire in a foreign language. The Ukrainian version was tested in a standard approach to translation, using a statistical analysis that explored consistency, sensitivity, reliability, and validity.

During forward translation, particular difficulties were encountered with “clinginess”, which had no single-word synonym in Ukrainian, and, therefore, it was translated as a phrase “holds close nearby him or her”. Another concern rose up with the item “unable to get out of bed”. In the Ukrainian translation, it meant physical disability rather than a voluntary inability caused by fatigue. Therefore, we replaced “unable to get out” with Ukrainian “difficulty to get out”, as a child was able to get out. Further, “low energy, tired” was changed to Ukrainian “easy fatigability”, and “irritable, cranky, fussy” met the Ukrainian word that embraced all these features. Backward translation demonstrated good correspondence of the sense of the Ukrainian version. Convergence was high both for the activity and the parental impact domains. There was a complete correspondence in interpretation of the disease-specific items.

Most responders experienced no difficulties while filling in the questionnaire. Time taken for completion of the questionnaire was relatively short, and the absence of missing values suggested good comprehensibility of the CARIFS items. Some misunderstandings appeared with interpretation for items 3, 6, 7, and 9 in application to older children and for items 4, 10–12 to younger children. Indeed, assessment of feelings or pains requires interviewing a good responder, which does not include a small child. Conversely, “crying more than usual” and “clinginess” may not be appropriate for the elementary school children, whose advancing socialization replaces more primitive reactions.

The baseline median score of the Ukrainian version of the CARIFS was 18, while in the original English version, it was 28 or around 25 (approximation from the chart).^3,4 In another study, the CARIFS median score was reported between 15 and 20 (approximation from the chart) depending on the age of children.¹⁰ Such a discrepancy could not be explained by differences in duration of illness before enrollment, a major determinant of the disease severity, as it was similar in the referenced studies. Variables that affected the baseline score might be symptomatic medication and season-dependent etiologies of ARTI. Indeed, the common cold caused by rhinoviruses and manifested with nasal discharge and obstruction with mild or absent systemic symptoms peaks in early autumn.¹¹ In temperate climates, more aggressive influenza infection causing prominent local and systemic symptoms occurs in winter.¹² So that, the study of ARTIs attempted in these different seasons may significantly differ in terms of the baseline CARIFS score. Finally, over- or underestimation of the symptoms might be due to differences in personality of the caregivers, while cultural differences cannot also be excluded.

The analysis of the frequency histogram of the CARIFS score exhibited moderate deviation from the normal distribution both for symmetry and amplitude. Skewness of the score was 0.641 vs 0 (shifted to the right) and kurtosis was 0.225 vs 3 (flattened peak) in the normal distribution. Skewness of the CARIFS data was also reported in a study by Butler et al.¹³ In a case of moderate right skewness, taking a log or square root of a data set is often enough to make data suitable for the analysis based on the assumption of the normality. In our study, taking a square root from CARIFS score considerably reduced skewness from 0.641 to 0.089. In comparison analysis, an alternative approach may be the use of non-parametric methods based on ranks without the assumption of the normality. Among such methods, the Mann–Whitney U-test, Wilcoxon signed, or rank sum tests can be suitable.

In our study, Cronbach’s a of the Ukrainian CARIFS items was relatively higher than the one originally reported for the English version.³ This finding may be at least partially explained by the two facts that: 1) different language versions will never be ideally the same; 2) alpha is a property of the scores for a test from a specific sample of responders, meaning that the variation in alpha may be expected even within the same language versions of the test. Therefore, it was proposed that investigators should not rely on published alpha estimates and should measure alpha each time the test was administered.¹⁴

Every CARIFS item contributed almost equally to the Cronbach’s a coefficient, suggesting that the questionnaire is constructed of equally important terms. However, individual item to total correlation coefficients varied considerably. The best correlation was documented for “not playing well” with some weaker relationships for other items that described activities of a child. Surprisingly, the disease-specific items, such as cough, nasal congestion, and sore throat, correlated less with the total CARIFS score. This could be partly explained by the fact that these local symptoms constituted a smaller part of the questionnaire, which was mostly “occupied” by the “systemic” attributes of the disease. Furthermore, the CAFIRS scores were measured on the second day of the disease, when systemic presentation of ARTI is profound, especially in younger children, who dominated the study.

The sensitivity of the Ukrainian version of the CARIFS was excellent with total and domain-specific SRM and ES varying within magnitudes of the large effect.⁸ The SRM was not calculated in the source study but ES for the total CARIFS was 0.99 in contrast to 2.50 in the present research, suggesting the higher difference in the CARIFS scores between the baseline and follow-up observation measurements.³ The latter could be explained by the fact that reassessment of the CARIFS score was virtually made on the third day of the disease in the Canadian study, while our responders completed the questionnaire on the fifth day, when the disease could be less intense. Additionally, as it is shown, the CARIFS scores demonstrated a fair extent of skewness, which might corrupt calculations based on untransformed means and SDs.

The test–retest reliability was high with ICC for the total CAFIFS score, activity, symptom, and parental impact domains being greater than 0.98. Such a good correspondence of the paired data was due to the relatively short time between the first and second measurements ranging from 3 to 6 hours. The description of the CARIFS items in such a setting was rather due to the effective recall of the previously recorded scores than to a true independent re-assessment of the score. Increasing the time between the tests, instead, might mirror the course of the disease or the efficacy of the symptomatic treatment making test–retest issues problematic in the fast changing acute illness.

We were not able to evaluate the construct validity of the CARIFS questionnaire as there was no “gold standard” instrument with which the data could be compared. In the original work, the surrogate gold standard measures were parent global assessments with a visual analogue scale and body temperature.³ We did not adopt these measures as the former would have a generalized level of subjectivity for attitudes, beliefs, or psychological characteristics, and the latter would measure partial and not necessarily inherent aspect of ARTI. In this study, the “gold standard” measure was a questionnaire completed by the physician. Almost immediate rescoring demonstrated the high and significant Pearson’s correlation coefficient between responder’s and physician’s scores.

Thus, the Ukrainian version of the CARIFS-based English questionnaire proved to be a consistent, sensitive, reliable, and valid instrument in assessment of ARTI in the preschool and elementary school children in the Ukrainian population.

Acknowledgments

The study was performed under the program “Optimization of the prognosis, prophylaxis and treatment of the most common pediatric illnesses and functional disorders” approved by the Health Ministry of Ukraine (0113U000209). We thank the family doctors, who recruited children into this study, and the caregivers and children for taking part.

Disclosure

The authors have no conflicts of interest in this work.

---

References

1.	Barrett B, Brown R, Mundt M, et al. The Wisconsin upper respiratory symptom survey is responsive, reliable, and valid. J Clin Epidemiol. 2005;58(6):609–617.
2.	Aabenhus R, Thorsen H, Siersma V, Brodersen J. The development and validation of a multidimensional sum-scaling questionnaire to measure patient-reported outcomes in acute respiratory tract infections in primary care: the acute respiratory tract infection questionnaire. Value Health. 2013;16(6):987–992.
3.	Jacobs B. The Canadian Acute Respiratory Illness and Flu Scale (CARIFS): the Design and Assessment of a Pediatric Disease Severity Measure [master’s thesis]. Toronto: Clinical Epidemiology Graduate Department of Community Health University of Toronto; 1999:1–88.
4.	Butler C, Robling M, Prout H, Hood K, Kinnersley P. Management of suspected acute viral upper respiratory tract infection in children with intranasal sodium cromoglicate: a randomised controlled trial. Lancet. 2002;359:2153–2158.
5.	Mitra A, Hannay D, Kapur A, Baxter G. The natural history of acute upper respiratory tract infections in children. Prim Health Care Res Dev. 2011;12:329–334.
6.	Guillemin F, Bombardier C, Beaton D. Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol. 1993;46(12):1417–1432.
7.	Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.
8.	Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–55.
9.	Husted JA, Cook RJ, Farewell VT, Gladman DD. Methods for assessing responsiveness: a critical review and recommendations. J Clin Epidemiol. 2000;53:459–468.
10.	Damiani V, Di Carlo M, Grappasonni G, Di Domenico R, Dominici P. Efficacy of a new medical device based on colloidal silver and carbossimetyl beta glucan in treatment of upper airways disease in children. Minerva Pediatr. 2011;63(5):347–354.
11.	Monto AS. The seasonality of rhinovirus infections and its implications for clinical recognition. Clin Ther. 2002;24(12):1987–1997.
12.	Cox N. Influenza seasonality: timing and formulation of vaccines. Bull World Health Organ. 2014;92(5):311.
13.	Butler CC, Hood K, Kinnersley P, Robling M, Prout H, Houston H. Predicting the clinical course of suspected acute viral upper respiratory tract infection in children. Fam Pract. 2005;22:92–95.
14.	Streiner D. Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess. 2003;80:99–103.