Validity and Reliability of the Knowledge, Attitudes and Practices Instrument Regarding Monkey Pox in Peru

Introduction

The virus known as Monkey Pox (Mpox), a member of the genus Orthopoxvirus, is the cause of the disease known as Mpox.¹ Orthopoxviruses also include the virus that causes chickenpox and avian smallpox. The same orthopoxvirus that causes the disease is a member of the poxviridae family. The morphology of the poxvirus is like a brick containing linear, double-stranded, lipoprotein-coated deoxyribonucleic acid (DNA).²

A new concern has been generated by the emergence of a viral threat to public health, with a fatality rate of 11% in people who are not vaccinated against smallpox; likewise, in May 2022,³ the World Health Organization (WHO) reported an outbreak of Mpox in several countries around the world.⁴

Potential routes may include interactions with animals and relationships among asymptomatic individuals. Due to the cross-immunity produced by Orthopoxviruses, vaccination against Mpox virus can provide up to 85% protection against it. However, since vaccine use against this virus has been reduced, outbreaks have increased, suggesting a change in its evolutionary course.⁵

Mpox manifests with skin rashes, malaise, headaches, and significant lymphadenopathy. The severity of the disease depends on the route of exposure of the causative organism and whether or not the infected person has a coexisting disease.^6,7

On the other hand, it is known that the vaccine against Mpox is safe and highly effective, recommended for all people who have not had the disease or have not been previously vaccinated. Likewise, its spread is avoided by adequate hygiene and isolation in cases of the presence of skin lesions, while symptomatic treatment is given with analgesics to relieve pain and antihistamines to relieve itching.⁷

The WHO and the European Centre for Disease Prevention and Control (ECDC) have emphasized the importance of raising awareness and providing appropriate guidance for immediate recommended actions.⁸ Indeed, one of the challenges posed by the current Mpox epidemic is the lack of knowledge about this virus, particularly among the population that may contribute to its evolution into a global pathogen.⁹

Previous studies have shown that the state of knowledge among health care workers themselves was quite unsatisfactory, with substantial knowledge gaps in all aspects of MPOX.^10,11 In turn, analysis of risk perception suggested that it was substantially overlooked as a pathogen, particularly in comparison to SARS-CoV-2, tuberculosis, HIV, and HBV.¹⁰ In the general population in China, only about half of them had a hi MPOX gh level of knowledge about MPOX (56.5%) and related symptoms (49.7%).¹² In Saudi Arabia the problem was not very different, where general knowledge about Mpox infection was deficient in more than half of the population.⁹

Likewise, a study published in the Journal of Infection and Public Health in 2019 shows how a questionnaire of knowledge was developed and validated in a previous pilot, with a content validity ratio of 0.78 being satisfactory. It was applied to university students in the United Arab Emirates, with approximately 77% found to have poor to moderate knowledge of human Mpox. The questionnaire was also found to be reliable in measuring knowledge about the origin, signs and symptoms, transmission, prevention and treatment of Mpox.¹³

For their part, a team of researchers in Pakistan used information from the official websites of the WHO and the Centers for Disease Control and Prevention for the elaboration of a questionnaire of knowledge, attitudes and practices, obtaining a reliability of Cronbach’s alpha of 0.76; after a pilot sample of 80 participants, it was applied to a sample of residents in Pakistan, among whom inadequate levels of knowledge and attitudes regarding MPOX were obtained.¹⁴

Recently, in 2022, a study in the Kurdistan Region of Iraq applied an online questionnaire through social networks to a sample of 382 people from this region, concluding that they had moderate knowledge, a neutral attitude, and a moderate level of concern about MPOX, suggesting the implementation of preventive measures and timely actions, given the increase in cases worldwide, in order to safeguard not only the physical health of the population but also their psychological wellbeing.¹⁵

Consequently, specific health education programs are an urgent need. Measuring the magnitude of the infection due to the presence of human MPOX involves evaluating knowledge and attitudes towards prevention measures and identifying gaps in public awareness regarding this virus. In this sense, it is necessary to have a validated questionnaire to provide scientific evidence for the network of prevention and control of MPOX at community level.¹⁶

In addition, no study has developed a valid and reliable instrument among the population in general in the Peruvian context. Therefore, the objective of this study was to analyze the validity and reliability of the instrument of knowledge, attitudes and practices of Mpox in Peru. The specific objectives were to evaluate the content validity, construct validity, and reliability of the questionnaire on knowledge, attitudes, and practices regarding Mpox and its dimensions.

Materials and Methods

Study Design

The study was instrumental, cross-sectional and its objective was to analyze the psychometric properties of the instrument.¹⁷

Population and Sample

The population was made up of citizens living in three sectors on the coast of Peru (La Libertad, Lima and Piura), obtaining data from 178 citizens.

The sampling technique was non-probabilistic, by snowball. The sample size was calculated taking into account the total population of the chosen sectors, using a margin of error of 7.5% and 95% confidence interval. Each of the citizens studied responded to the questionnaire, with a multiplier effect on other citizens who met the selection criteria: citizens over 18 years of age, of both sexes, residing in the three study sectors, excluding those who did not complete the form and/or did not provide informed consent.¹⁸

Data Collection

For data collection, the technique used was the survey, and as an instrument the questionnaire prepared by the authors. The instrument was built taking into account a previous questionnaire on knowledge, attitudes and preventive practices of COVID-19,¹⁹ as well as the epidemiological records provided on the pages of the Ministry of Health of Peru.

The sociocultural diversity of the Peruvian population was taken into account, making a prior survey of the inhabitants of the different educational levels, also the validators suggested changes in the initial wording of the instrument, to match the language level of the study population.

This questionnaire is divided into four sections:

The first section identified the characteristics of the respondents, such as sector of origin/residence, gender, marital status, education level, age, and whether they had secure employment.

The second section is made up of 32 items from the Mpox knowledge scale. This, in turn, is made up of 10 items that assess knowledge about the signs, eight items that assess the forms of contagion, eight about the disease, and six about actions in case of infection.

The third section evaluates the attitudes of citizens regarding Mpox and is made up of 10 items: four for attitudes regarding fears about the disease, three for staying informed, and three regarding actions to be taken.

The fourth and last section evaluates preventive practices towards Mpox and consists of nine items: four for practices regarding individual hygiene and five to measure preventative practices in their environment.

The questionnaire was applied virtually using Google Form^®, with dissemination through social networks such as WhatsApp^®, Facebook^® and email in the period January-February 2023, filtered by allowing only one response per person.

Statistical Analysis

The database was extracted to an Excel spreadsheet, and subsequently analyzed using the JAMOVI statistical package. A descriptive analysis of the characteristics of the citizens was made, using statistical measures such as mean, standard deviation and percentages expressed in statistical tables.

Two types of validity were analyzed: content and construct. Content validity is a measure of the ability of an instrument to measure the construct that is intended to be evaluated. This measure is based on the evaluation of the degree to which the items of the instrument adequately represent the content of the construct being measured. This is obtained through a process of expert judgment, where the relevance, clarity and representativeness of the items of the instrument are evaluated in relation to the construct that is intended to be measured.²⁰

Therefore, to analyze the content validity, expert judgment was performed and using Aiken’s V technique, it was verified whether it was higher than 0.90, to be acceptable.

Construct validity is the degree to which a measurement instrument accurately and systematically measures the psychological dimension that it is intended to assess, and that this dimension conforms to a theory or conceptual model. It can be established through a variety of methodological approaches, including confirmatory factor analysis, multigroup invariance analysis, analysis of convergent and divergent correlations, and extreme cluster analysis, among others.²¹

Consequently, for construct validity, an exploratory factor analysis was used with the factoring technique according to the main axis, to identify the items that make up each of the dimensions, verifying compliance with the requirements for the application of this analysis.^22–24

The reliability of an instrument refers to the consistency and stability of the responses obtained by the instrument, ie, the extent to which the results obtained are accurate and reproducible.²⁵

For this purpose, Cronbach’s Alpha and McDonald’s Omega statistics were used, which are suitable statistics for reliability testing. McDonald’s Omega and Cronbach’s Alpha are two reliability measures commonly used in psychometrics and social research. Both coefficients assess the internal consistency of the responses through the correlation among the items of the instrument. Cronbach’s Alpha coefficient measures the internal consistency of an instrument’s items and is commonly used to assess the reliability of psychometric and survey tests. Alpha values can range from 0 to 1, where a value of 0 indicates no consistency among the items of the instrument and a value of 1 indicates perfect consistency. The McDonald Omega coefficient is a more general measure of reliability that can handle complex factor models and can provide a more accurate estimate of reliability when the instrument has multiple latent dimensions. Omega values can also range from 0 to 1, where a value of 1 indicates perfect reliability.^26,27

Ethical Considerations

This research considered the principles mentioned in the Declaration of Helsinki,²⁸ such as respect, credibility, reliability, privacy, applicability and autonomy, which take into account the protection of the health, integrity and identity of the person, for which reason each participant was asked for their informed consent before continuing with answering the questionnaire.

Likewise, this study was submitted for evaluation to the Ethics Committee of the School of Medicine of the Universidad César Vallejo. It was approved by report 009-CEI-EPM-UCV-2023.

Results

Table 1 shows that, of the participants in the study, 43.82% were from the city of Piura, followed by 41.01% from La Libertad and just 15.17% from Lima department. In addition, 66.29% of respondents were female. The mean age of the respondents was 35.08 years, with a variability of 13.54 years with respect to their mean age. Additionally, 56.18% were single, 47.19% had children, and 96.07% had university education.

Table 1 Sociodemographic Data of Participating Citizens

Content validity was evaluated by means of expert judgment. The expert professionals who evaluated the instrument were four health professionals and a methodologist. The items were rated with scores of one for the essential item and zero for the nonessential one. The judges placed observations on the instrument, which were then lifted. The results of the statistical analysis by Aiken’s V were higher than 0.90 for the whole instrument, for each variable and its dimensions, concluding that the instrument is acceptable and can be applied (See Table 2).

Table 2 Evaluation of the Content Validity of the Mpox Knowledge, Attitudes and Practices Scale

Construct validity was assessed using exploratory factor analysis. The extraction method was principal axis factorization, since it is a statistical technique where the observed variables are influenced by latent or underlying factors that are not directly observable, and uses a correlation matrix to calculate the principal components and select the factors. In addition, this technique was used in combination with an Oblimin rotation, which is a method used in factor analysis to obtain a more interpretable and realistic factor solution that allows factors to be correlated with each other and variables to be influenced by multiple factors. To select the number of factors, 4 factors were taken into account initially, considering knowledge about: Factor 1: Knowledge of signs (10 items: 1 to 10), Factor 2: Knowledge of methods of infection (8 items: 11 to 18), Factor 3: Knowledge about the disease (8 items: 19 to 26) and Factor 4: Actions to take in cases of infection (6 items: 27 to 32). (See Table 3).

Table 3 Evaluation of the Construct Validity of the Mpox Knowledge Scale

Likewise, indicators such as the root mean square error of approximation (RMSEA), Tucker-Lewis index (TLI), Bayesian Information Criterion (BIC) are presented. They are measures of model fit, from which it can be deduced that having an RMSEA of 0. 05243 indicates an acceptable fit, the TLI value is acceptable, and the BIC of −1379 suggests a good model fit. Finally, when analyzing the model fit, we observed a p-value < 0.05, indicating that the model fits significantly better than the null model. Bartlett’s Sphericity test is used to verify if the variables included in the analysis are correlated, as the p-value < 0.05. Then it is concluded that the variables are significantly correlated and it is appropriate to continue with the exploratory factor analysis (See Table 3).

The analysis of the KMO sampling adequacy measure is considered a measure of sampling adequacy used in exploratory factor analysis to assess whether the data are adequate to perform a factor analysis. Given that the value of the overall sampling adequacy measure (SAM) is 0.7325, therefore, it is greater than 0.7, which indicates acceptable sampling adequacy and the factor analysis can be continued (See Table 3).

To analyze the construct validity of the scale of attitudes and the scale of practices, exploratory factor analysis was also used with the factorization extraction method according to the main axis in combination with an Oblimin rotation. Three factors were identified for the scale of attitudes; Factor 1: Attitudes regarding fears towards the disease (4 items: 1 to 4), Factor 2: Attitude to keep informed (3 items: 5 to 7) and Factor 3: Attitude regarding actions to take (3 items: 8 to 10), as evidenced in Table 4. On the other hand, in the scale of practices, 2 latent factors were formed: Factor 1: Practices with respect to individual hygiene (4 items: 1, 2, 3 and 4) and Factor 2: Prevention practices in their environment (5 items: 5.6, 7, 8 and 9), as shown in Table 5.

Table 4 Evaluation of the Construct Validity of the Mpox Attitudes Scale

Table 5 Evaluation of the Construct Validity of the Mpox Practice Scale

Likewise, the measures of model fit indicate that having an RMSEA of 0.058 and 0.1665 for the scales of attitudes and practices respectively, indicate an acceptable fit. The TLI value is acceptable for both scales, being 0.9658 for the attitude scale and 0.8424 for practices. The BIC for attitudes was better, obtaining a value of −64.21, and 14.43 for practices, suggesting a good fit of both models. Finally, when analyzing the fit of the model, both for attitudes and practices, by means of significance, we observed a p-value < 0.05, indicating that the model fits significantly better than the null model. Bartlett’s Sphericity test confirms that it is appropriate to continue with the exploratory factor analysis for both scales, given that p < 0.05 results were found, indicating that the variables are significantly correlated. (See Table 4 and 5).

Observing the analysis of the Suitability measure of the KMO sampling, we can see that the MSA Global value is 0.7896 for the attitudes scale and the MSA Global for practices is 0.8924, therefore, since both are greater than 0.7, it indicates an acceptable adequacy of the sampling to make use of factorial analysis, as evidenced by the results in Table 4 and 5.

With respect to the reliability analysis of the scales for knowledge, attitudes and practices, it can be seen in Table 6 that in the application of both Cronbach’s α statistic and McDonald’s ω, the scales of the variables in general are highly reliable (above 0.7). In addition, when analyzing by dimension solely the dimensions of knowledge of the methods of infection, knowledge about the disease and knowledge of actions in cases of infection, these gave reliability values below what was expected. However, since the general knowledge scale has a high reliability, and having been endorsed by the judgment of experts in content validity and construct validity through exploratory factor analysis, the analysis was continued.

Table 6 Reliability Statistics of Mpox Knowledge Scale and Its Dimensions

Figure 1 shows the correlations among items of each dimension of the Mpox knowledge scale. It can be seen that in the dimension of knowledge of the signs are those with the highest correlations; while among the items of the dimensions, knowledge of the disease, knowledge of actions to be taken in cases of infection and knowledge of the methods of infection there are items with low or moderate correlation. However, in some cases the relationship was considered to be inverse.

Figure 1 Correlation heat maps by dimension of the Mpox knowledge scale. Notes: CS: Knowledge of signs, FC: Knowledge of ways of infection, CE: Knowledge of the disease and CA: Knowledge of actions to be taken in case of infection. (from Spanish initials).

Figure 2 shows the intensity of the association among the items of the attitude scale, reaching values higher than 0.8, as is the case between item 7 and items 5 and 6. In the practices scale, these relationships reach values of 0.78, indicating a high association among some items.

Figure 2 Correlation heat map of attitudes and practices scales. Abbreviations: AC, attitudes; PR, practice.

Discussion

The results of the research allowed the attainment of a questionnaire that complies with the adequate psychometric characteristics to be applied. The content validity confirmed that the indicators used in the questionnaire to measure knowledge were adequate, establishing 32 items that evaluate four dimensions of this variable. It is important to mention that the validated items were based on the epidemiological data sheet of the Ministry of Health,²⁹ and coincide with studies carried out in different parts of the world, such as Israel and Japan.^7,30

Content validity also confirmed the relevance of the items measuring attitudes and preventive practices for Mpox, establishing 11 items for attitudes divided into three dimensions, and nine items with two dimensions for the practices variable. The evaluated items coincide with the recommendations provided by the WHO and scientific articles in various parts of the world, where they point to hygiene practices, personal care and keeping informed, among others, as preventive measures.^6,9,31

As for construct validity, it indicates whether the items fall within the established dimensions. The exploratory factor analysis used indicated that the knowledge items are divided into four dimensions; for attitudes, three dimensions; and for the practices variable, two dimensions with their respective items mentioned in the results. Research conducted in various parts of the world indicates that among the attitudes towards Mpox is the fear of acquiring the disease, coinciding with the items set out in the questionnaire.^20,32,33 Similarly, two studies conducted in Peru elaborated questionnaires proposing items expressing physiological and emotional reactions to fear and dread of Mpox.^24,32 These attitudes of the population were also noted in the coronavirus pandemic, when they expressed fear of infection, isolation and keeping informed to avoid acquiring the virus.³⁴

Reliability was evaluated by two techniques, Cronbach’s Alpha and McDonald’s Omega, both of which reported adequate reliability for the instrument to be applied. These results coincide with studies of other questionnaires established to measure this type of variable. Thus, in the questionnaire of knowledge, attitudes and preventive practices used in a population in Pakistan, questions were used on knowledge of signs and symptoms of the disease plus questions of preventive practices similar to those used in Peru, such as avoiding crowds, using masks and washing hands frequently to avoid the spread of the disease, obtaining a reliability higher than 0.70.^20,32 On the other hand, when evaluating the attitude scale, the scale of fear of the disease was found to have good reliabilities, higher than 0.70, similar to the results of the scale of fear, where good reliabilities higher than 0.80 were reported.³⁴

Regarding the limitations experienced in the development of the research, the size and method of obtaining the sample can be mentioned, which could make it difficult to be able to generalize the results.

As a strength of the study, multivariate statistical techniques were used, the same as have been executed in a free access program, obtaining adequate results in such a way that the instrument constitutes a contribution and can be replicated by other researchers.

Conclusions

The study presents a first questionnaire with evidence of content, construct and reliability validity in a sample of Peruvian adults, which will serve to assess knowledge, attitudes and prevention practices regarding Mpox disease.