Psychometric Evaluation of the HIV Knowledge Questionnaire-18 in a Black Canadian Population

Introduction

The World Health Organization (WHO) estimated that in 2023 there were over 39 million people living with HIV worldwide including an estimated total of 1.3 million people who were newly diagnosed.¹ In Canada, the Public Health Agency of Canada (PHAC) reported an HIV incidence rate of 4.0 per 100,000 in 2020 with 1520 new diagnoses.² In 2023, among new diagnoses in Canada where race was reported, 18% of the cases were Black people. ² In Ontario, the Ontario HIV Epidemiology and Surveillance Initiative (OHESI) reported that in 2020, Black communities accounted for approximately 25% of new HIV diagnoses,³ despite comprising only about 5% of the total population in Ontario.^4,5 Due to this disproportionate burden, Black communities have been recognized as a priority population in Ontario by the Ontario Advisory Committee on HIV/AIDS.⁶

HIV knowledge has been identified as a determinant of preventative or self-protective sexual behaviour. The Health Belief Model (HBM) is a commonly used conceptual framework used to understand HIV risk-taking behaviours. Applying the HBM to HIV prevention, this framework theorizes that individuals are motivated to act based on their perceived susceptibility to HIV, the perceived severity of HIV, the perceived benefits about the effectiveness of preventative measures or actions and the perceived barriers to taking preventative action.^7,8 These domains are informed by HIV knowledge. Similarly, the Information Motivation Behavioural Skills (IMB) model identifies information about HIV transmission and prevention as a prerequisite to preventative actions.⁹

The HIV Knowledge Questionnaire (HIV-KQ) is one of the most widely used HIV knowledge scales.¹⁰ The 45-item HIV-KQ was developed to measure HIV knowledge about HIV transmission, prevention and knowledge of the consequences of HIV infection in general populations.¹¹ Subsequently, the brief HIV Knowledge Questionnaire (HIV-KQ-18) is an 18-item instrument adapted from the original 45-item questionnaire.¹² The HIV-KQ-18 has high levels of internal consistency (Cronbach’s alpha = 0.75 to 0.89) and a high correlation with the 45-item HIV-KQ (r = 0.93 to 0.97).¹²

The HIV-KQ-18 has not been tested among Black people in Ontario; therefore, an examination of the psychometric properties of the questionnaire is necessary. Validity and reliability are the main psychometric properties of an instrument or scale.¹³ A reliable instrument measures the same construct consistently and an instrument is considered valid if it measures the intended construct.¹³ Previous research has shown that HIV knowledge instruments may perform differently across subpopulations, highlighting the importance of examining scale performance within groups disproportionately affected by HIV.^14,15 Among Black communities in Canada, structural inequities and differential access to health information may shape both HIV knowledge acquisition and interpretation of survey items.^16,17 This supports the need for a psychometric examination of the HIV-KQ-18 to ensure adequate psychometric performance when measuring HIV knowledge among Black communities. The validation of this instrument in this population will inform a more contextualized interpretation of study results, which allows researchers to better identify knowledge gaps to inform future education and prevention interventions.

The HIV-KQ-18 was used to measure the HIV knowledge of participants in a cross-sectional study of HIV epidemiology among Black communities in Ontario, the A/C study.¹⁸ Among the participants of the A/C study, we used the HIV-KQ-18 and previously reported that only 56% had high knowledge. However, the psychometric properties of the HIV-KQ-18 have not been evaluated in this population. The A/C study found and reported that only 56.5% of survey participants had high knowledge about HIV transmission and acquisition.¹⁹

The objective of this study is to examine the psychometric properties of the HIV-KQ-18 questionnaire in a Black population with a focus on construct validity and internal consistency.

Methods

Study Design and Sample

The A/C study recruited participants that met the following criteria: born in a Caribbean or sub-Saharan African country, or born elsewhere (including Canada) with a parent born in one of those countries; aged 15–64 years old at the time of the survey; residence in the Greater Toronto Area (Toronto) or Ottawa, Ontario; capacity to communicate in English or French; and capable of providing informed consent. Participants were recruited by peer recruiters from January to December 2019.

The study was approved by the following affiliated institutional Research Ethics Boards (REBs): University of Ottawa Research Ethics Board; Ottawa Public Health Research Ethics Board; Toronto Public Health Research Ethics Board; Laurentian University Research Ethics Board; and University of Toronto. All methods were carried out in accordance with relevant guidelines and regulations.

Instrument

In this study, we used data collected through the HIV-KQ-18 questionnaire (the study instrument) to measure participants’ HIV knowledge. The questionnaire (see Supplementary File 1) consists of 18 statements about HIV and participants were asked to indicate whether they think the statement is true or false.¹² Responses to the questionnaire were then computed into a knowledge score with one point for every correct answer for a maximum of 18 points.

Statistical Analysis

Descriptive statistics were calculated using Stata version 17.²⁰ We computed frequencies and percentages for categorical variables. Although the HIV‑KQ‑18 was developed as a continuous measure and does not have empirically established cut‑off scores, categories of HIV knowledge were created for descriptive purposes to aid interpretation. We categorized participants into four knowledge levels: No HIV knowledge (score = 0), Low HIV knowledge (score = 1–6), Moderate HIV knowledge (score = 7–12), and High HIV knowledge (score = 13–18). For each category, the number and proportion of participants [n (%)] were reported. Consistent with prior practice, all psychometric analyses were conducted using continuous scores. We calculated the overall mean HIV-KQ-18 score with its standard deviation (SD) to summarize central tendency and variability in HIV knowledge across the sample. Missing data were minimal and were handled using complete‑case analysis. The sample size was determined by the available data from the A/C study. With 1302 participants, the sample size exceeds commonly recommended thresholds for psychometric validation studies and was considered adequate to evaluate internal consistency and construct validity of the HIV‑KQ‑18.

Internal Consistency

We estimated internal consistency of this scale using Cronbach’s alpha score. Internal consistency describes how closely the items on a scale are related to each other or the degree to which all items assess the same concept or construct.²¹ Acceptable values of alpha range from 0.70 to 0.95.^21,22 We estimated the corrected item-total correlation which is the correlation between a single item and the sum of all the other items in a scale excluding the item itself.²³ The commonly accepted cut-off for this measure is 0.30.²³ The Cronbach’s alpha for the scale with the removal of individual items was also calculated.

Construct Validity

Construct validity refers to the agreement of the scale with theoretical constructs.²⁴ Known-groups validity is a type of construct validity which is used to measure whether a scale is able to distinguish between groups that are known to differ on the construct.²⁵ We hypothesized that we would find differences in HIV knowledge in the sampled population by age, education and employment. In previous international research, these factors have been associated with HIV knowledge.^26–29 We hypothesized that older age groups, those with higher levels of education and those with full-time employment had higher levels of HIV knowledge. We also examined the relationship between gender, city, and language as exploratory constructs and hypothesized that we would find differences in scores in these additional categories. We used the ANOVA test and t-tests to compare HIV knowledge levels for the subgroups of interest, followed by post-hoc pairwise comparisons using the Bonferroni correction where necessary.³⁰

Results

Out of 1380 participants who participated in the A/C study, 1302 completed the HIV-KQ-18 questionnaire. Most participants were women (n = 813, 62%) and were employed either full-time or part-time (n = 718, 55%). The majority were from Toronto (n = 816, 63%) and completed the questionnaire in English (n = 1208, 93%). The complete sociodemographic information of the included participants is presented in Table 1.

Table 1 Socio-Demographic Characteristics of Participants (N=1302)

The mean knowledge score for all participants in this sample was 12.45 or 69.1% (95% confidence interval [CI]: 12.23, 12.67) out of a total of 18 potential points, meaning on average participants provided a correct response to ~70% of the items in the questionnaire. The overall performance of participants to the questions in the HIV-KQ-18 is presented in Supplementary File 1. Out of 1302 participants, 2.38% (n = 31) had no HIV knowledge (0 out of 18 points), 7.60% (n = 99) had low HIV knowledge (1 to 6 out of 18 points), 30.18% (n = 393) had moderate HIV knowledge (7 to 12 out of 18 points) and 59.83% (n = 779) had high HIV knowledge (13 to 18 out of 18 points). The socio-demographic characteristics of participants by HIV knowledge level with the mean score of each group are presented in Table 2.

Table 2 Socio-Demographic Characteristics of Participants by HIV Knowledge Level (Score Out of 18 Points)

Internal Consistency

The Cronbach’s alpha for the scale was 0.82 (95% CI 0.80–0.84) suggesting good internal consistency. The “alpha if removed” for each item varied slightly. The corrected item-total correlation values ranged from 0.24 to 0.58 indicating a moderate relationship between individual items and the total scale score. The full item total statistics are reported in Table 3.

Table 3 Item-Total Statistics

Construct Validity

A one-way ANOVA to compare the effect of age on HIV knowledge scores revealed that there was a statistically significant difference in mean scores between at least two age groups (F(5, 1286) = [30.30]; p = < 0.001). Post hoc analysis using the Bonferroni correction indicated those aged 15 to 19 scored significantly lower than all other age groups except for those aged 60 to 64 (mean difference [MD] = −1.16; 95% CI [−1.16, 3.49]; p > 0.999). Participants ages 20 to 29 scored significantly higher than those aged 15 to 19 (MD = 3.00; 95% CI: [1.93, 4.08]; p < 0.001) and significantly lower than those aged 30 to 39 (MD = −1.39; 95% CI: [0.55, 2.23]; p < 0.001). Those aged 30 to 39 also had significantly higher scores than those aged 60 to 64 (MD = 3.23; 95% CI: [1.00, 5.46]; p < 0.001). Participants aged 40 to 49 had significantly higher scores than those aged 60 to 64 (MD = 2.62; 95% CI: [0.38, 4.87]; p = 0.009).

We found statistically significant differences in mean scores by employment status (F(2, 1299) = [39.93]; p < 0.001). Post-hoc analysis showed statistically significant differences in scores between all groups. Participants who reported that they were employed full time had significantly higher scores than those employed part time (MD = 1.38; 95% CI: [0.64, 2.11]; p < 0.001) and unemployed participants (MD = 2.18; 95% CI: [1.60, 2.77]; p < 0.001).

We found statistically significant differences in mean scores by education level (F(3, 1276) = [75.76]; p < 0.001). Post-hoc analysis showed statistically significant differences in scores between all education levels. Participants who reported that they completed university had significantly higher scores than those who reported completing college (MD = 0.88; 95% CI: [0.15, 1.61]; p = <0.001), high school (MD = 3.32; 95% CI: [2.65, 4.00]; p < 0.001) and less than high school (MD = 6.17; 95% CI: [4.32, 8.02]; p < 0.001).

We also found statistically significant differences in mean scores by gender (F(2, 1292) = [6.76]; p = 0.001). Post-hoc analysis revealed significantly higher scores among participants who identified themselves in gender categories other than men (MD = 2.98; 95% CI: [0.87, 5.08]; p < 0.001) or women (MD = 2.51; 95% CI: [0.43, 4.60]; p = 0.012). There was no statistically significant difference between the scores of men and women (MD = 0.47, 95% CI: [−0.1, 1.03]; p = 0.144).

Independent two sample t-tests revealed higher HIV knowledge scores among those living in Toronto compared to Ottawa (MD=1.13, 95% CI: 0.68–1.58; p = <0.001). We also found higher scores among those who completed the questionnaire in English compared to French (MD = 1.08, 95% CI: 0.24–1.94; p = 0.012). Table 4 presents a summary of all the psychometric properties we examined in this study.

Table 4 Summary of Psychometric Properties

Discussion

Our findings suggest that the HIV-KQ-18 demonstrates acceptable psychometric performance for use in a Black Canadian population. The mean knowledge score of 69.1% for the overall sample was similar to that of the populations sampled in the development study for the HIV-KQ-18 which ranged from 69.2% to 72.5%.¹² The Cronbach’s alpha score of 0.82 is well within the range of acceptable values.²¹ In our examination of the internal consistency of the questionnaire, we found that the value of alpha varied slightly with the removal of items, however the range of values was between 0.81 and 0.82 confirming that the instrument is well constructed. The corrected item-total correlation values ranged from 0.24 to 0.58 indicating a moderate relationship between individual items and the total scale score. One item (Item 17) fell below the threshold of 0.30; however, the removal of that item does not significantly change the Cronbach’s alpha of the scale. Future qualitative studies may be necessary to provide additional insight into the cultural appropriateness of the items and to determine whether any modifications of items are necessary to improve clarity.

Our findings indicate that the HIV-KQ-18 effectively differentiates participants’ levels of HIV knowledge based on their education level, employment status, and age, consistent with our a priori hypotheses. Participants with higher education levels and full-time employment had higher HIV knowledge than those with lower education levels and those who reported being employed part-time or unemployed. We also hypothesized that HIV knowledge would increase with age, and we found this to be true; however, participants ages 60 to 64 had a statistically significant lower score than all other age groups except those aged 15 to 19 indicating that the oldest age groups and youngest have similar HIV knowledge. In a study conducted in the United States using this instrument in a population of older adults, with an average age of 73.7, the researchers reported an average knowledge score of 10.8 which is similar to the average score of 10.46 in this study among those in the oldest age group.³¹ A Canadian study examining predictors of HIV knowledge in rural and urban Canadian populations in Ontario, British Columbia and Newfoundland found that age was a significant predictor, and those in age groups over 50 were less likely to be categorized as having high knowledge of HIV/AIDS.³² This might indicate the need for targeted interventions for older adults, as education campaigns are often designed for younger populations. This gap in outreach efforts might lead older populations to assume they are not at risk for contracting HIV leading to lapses in the most up to date information about HIV.

We also examined gender, language and the city of questionnaire administration as exploratory constructs. The relationship between HIV knowledge and gender varies in different populations. A systematic review examining the relationship between gender and HIV knowledge among adolescents in low and middle income countries found that males were consistently found to have higher HIV knowledge scores than females.³³ Similarly, in a systematic review of HIV knowledge and attitudes in the Middle East and North Africa, researchers found that while male populations often had higher HIV knowledge, there were exceptions to that overall trend in some populations in Afghanistan, Iran and Pakistan where female populations had more HIV knowledge.³⁴ There are no similar evidence syntheses including Canadian populations; however, other Canadian studies have not found significant differences in HIV knowledge by gender.^32,35 In this study, participants who identified themselves in gender categories other than men or women had significantly higher scores. This finding warrants further examination. This may suggest that interventions targeting those who identify outside of the gender binary have been more effective.

Participants from Toronto and those who completed the questionnaire in English had higher scores than those who participated in Ottawa and completed the questionnaire in French respectively. Canada is a bilingual country, however, in Ontario most francophones reside in Eastern Ontario, a region which includes the city of Ottawa.³⁶ In this study, 4.4% (n = 38) of Toronto participants took the questionnaire in French compared to 12.5% (n = 66) of participants in Ottawa. Our findings suggest that there may be a need for targeted initiatives focused on Francophone populations and those in Ottawa.

Our study is the first validation study of the HIV-KQ-18 in a Black Canadian population with a robust sample of over 1300 participants from two large Canadian cities. However, there are limitations to our findings. First, due to the cross-sectional design, we were not able to assess other psychometric properties such as test–retest reliability. Second, due to our recruitment in urban cities, there may be additional considerations for Black Canadians who live in rural communities which we are unable to assess. Finally, there is no validated French version of this instrument, therefore additional validation studies to develop a French version of this instrument may be necessary.

Conclusion

Our psychometric analysis shows that the HIV-KQ-18 is a valid and reliable instrument to assess HIV knowledge among Black Canadians. We encourage the use of the instrument in future Canadian research. Future research should assess additional psychometric properties of the HIV‑KQ‑18, to further strengthen evidence for its use among Black Canadians.