Development and Psychometric Validation of a Questionnaire to Evaluate Knowledge and Attitude Towards Medication Error Reporting Among Pharmacists
Authors Mamat R, Awang SA, Ab Rahman AF
Received 10 February 2020
Accepted for publication 8 May 2020
Published 14 May 2020 Volume 2020:12 Pages 95—101
Checked for plagiarism Yes
Review by Single-blind
Peer reviewer comments 2
Editor who approved publication: Dr Rajender R. Aparasu
Ruzmayuddin Mamat,1 Siti Asarida Awang,2 Ab Fatah Ab Rahman3
1Department of Pharmacy, Klinik Kesihatan Bandar Kuantan, Kuantan, Pahang, Malaysia; 2Department of Pharmacy, Pejabat Kesihatan Daerah Kuantan, Kuantan, Pahang, Malaysia; 3Faculty of Pharmacy, Universiti Sultan Zainal Abidin, Besut, Terengganu 22200, Malaysia
Correspondence: Ab Fatah Ab Rahman
Faculty of Pharmacy, Universiti Sultan Zainal Abidin, Besut, Terengganu 22200, Malaysia
Purpose: Assessment of medication errors (ME) is crucial to improving the quality of health care. A questionnaire that can be used to explore pharmacists’ perspectives regarding ME would be very useful as part of an ongoing process of quality improvement in patient care. The aim of this study was to develop and validate a questionnaire to measure perceived causes of ME and attitude towards ME reporting among pharmacists.
Methods: The questionnaire was developed from the literature together with outcomes from focus group discussions. It was divided into two domains which are knowledge on ME and attitude towards ME reporting. Content validity index (I-CVI), exploratory factor analysis (EFA), Cronbach alpha and intraclass correlation coefficient (ICC) to assess test–retest reliability were obtained during the validation process.
Results: Overall Cronbach alpha for internal consistency was good (0.742), where subscale of the questionnaire demonstrated adequate internal consistency, with Cronbach alpha value 0.83 for knowledge and 0.70 for reporting behaviour attitude. The I-CVI showed good scores (knowledge=0.88) and (attitude=0.81), while ICC was moderately accepted with a value of 0.77. Two factors were extracted from the 16 items in EFA.
Conclusion: The questionnaire to assess knowledge on ME and attitude towards ME reporting among pharmacists is valid and reliable. It demonstrates good psychometric properties.
Keywords: medication error, pharmacist, psychometric validation, hospital, attitude
Medication error (ME) is defined as any preventable event that may cause or lead to inappropriate medication use or patient harm while the medication is in the control of the health care professional, patient or consumer.1 It can occur at any of the five stages of a medication use process. In some studies, the majority of medication error have been found to occur in the prescribing stage,2,3 while others have shown that ME predominantly occurred in the administration stage.4,5 In each stage of a medication process, different types of MEs can occur. For example, in settings without electronic prescriptions, illegible handwriting may contribute to prescribing error.6,7 In ambulatory care setting, the types of ME at the dispensing stage include labelling and filling errors.8 In the inpatient setting, nurses are usually involved in medication administration activities and the most common type of administration error reported was wrong time error.9
Pharmacists are essential health-care providers who can help ensure patients receive their medications safely. Although they are not usually directly involved with prescribing and administering medications, they play important roles to ensure that each patient receives the right medication in the right dose via the right route and at the right time. To perform these duties, they should be able to rapidly detect and prevent ME before they reach and possibly harm patients. At the same time, they should also see ME reporting as a learning opportunity to improve health care delivery process and patient safety.
ME documentation and reporting in Southeast Asian countries are inadequate.10 In Malaysia, MEs are reported online to the national medication error reporting system. Compared to other health professionals, pharmacists have contributed significantly to the overall ME reporting.2 Nevertheless, there exist strong barriers to it including poor knowledge and wrong attitude.11,12 For a successful implementation of patient safety program, an appropriate reporting culture and feedback mechanisms should be in place.13 Therefore, an assessment tool to explore pharmacists’ perspectives regarding ME would be very useful as part of an ongoing process of quality improvement. The aim of this study was to develop a valid and reliable questionnaire to measure perceived causes of ME and attitude towards ME reporting among pharmacists working in hospitals and healthcare clinics. Psychometric data are also presented.
Materials and Methods
Development of the Questionnaire
The questionnaire was developed in two phases. Phase I was for questionnaire development, while Phase II was for psychometric validation.
A focus group discussion (FGD) with five senior pharmacists, each with more than 10 years working experience, was conducted to obtain in-depth information related to ME. All the discussions were recorded and transcribed for further analysis. Two themes were identified, which were knowledge on ME and attitude towards ME reporting. Using feedback from the FGD and questionnaires used in other similar studies,11,12,14,15 we developed 10 questions on perceived causes of ME and six questions on attitude towards barriers to ME reporting. The questionnaire used a 5-point Likert scale, ranging from “strongly agree” to “strongly disagree”.
Validation of the Questionnaire
Psychometric validation was conducted in Phase II. This process was divided into three stages, (i) Content validity by utilizing items-content validity index (I-CVI) technique, (ii) Test retest in 40 respondents within 14 days apart and (iii) Exploratory factor analysis (EFA) to explore pattern of domains and also to act as a pilot study. Pharmacists were selected by convenient sampling from two public hospitals and 10 primary health-care clinics in the state of Pahang, Malaysia. Data were collected using a web-based survey at www.surveymonkey.com, which was emailed to 200 pharmacists. Within the survey, information explaining the purpose and methodology of the study together with consent statements were provided. Pharmacists were required to provide informed consent by clicking on the radio button/check box before they could proceed with the survey.
Reliability: Internal Consistency and Temporal Stability
To measure the reliability of the questionnaire, Intraclass Correlation Coefficient (ICC) was used to assess temporal stability while Cronbach alpha was used to analyze internal consistency. Individual item analysis was utilized by calculating the corrected item-total for ICC and Cronbach alpha value. Items that were below references value were removed. For Cronbach alpha, items within the range of 0.60 (moderately good) till 0.95 (excellent) were retained in the questionnaire.16
Content Validity: Content Validity Index for Items (I-CVI) and Construct Validity
Content validity index is an index of interrater agreement that expresses the proportion of agreement among a team of expert.17 I-CVI was defined as the number of experts giving a rating either 3 or 4 divided by the total number of experts. I-CVI was established by gaining expert agreement from seven experts. Overall CVIs were calculated based on the average of I-CVIs.17 Construct validity was determined by utilizing the factorial structure with exploratory factor analysis (EFA). EFA was performed in 186 respondents. The principal axis factoring method was used and the rotated matrix was extracted with varimax orthogonal extraction.
Data were imported into Microsoft excel to calculate I-CVI. All other statistical analyses were performed using SPSS software version 23.
This study was approved by the Malaysia Research Ethics Committees (MREC), Ministry of Health, Malaysia [NMRR-19-476-45627 (IIR)].
Characteristics of Respondents
Different phases of this study had different number of respondents. Table 1 summarizes the characteristic of the respondents in this study from the development until the validation of the questionnaire. EFA was based on data collected from 186 respondents (93% response rate). The majority were female (74.2%). The mean age (± SD) of these respondents was 28.5 ± 3.17 years old. There were more respondents working in hospital (52.7%) than those in primary health-care clinics (47.3%). The majority of respondents (67.2%) had less than 8 years working experience.
Table 1 Respondents Characteristics for Different Phases of the Study
Development of the Questionnaire
Experts in FGD agreed that pharmacists who have good knowledge on ME will tend to have good attitude towards reporting. Findings obtained from FGD were used to construct the questionnaire. The first formulated questionnaire for this study was divided into two parts. The first part was for demographic information consisting of six items. The second part was for knowledge on ME (10 items), and for attitude towards ME reporting (six items).
Reliability: Internal Consistency
The internal consistency to assess reliability was conducted among 186 respondents. The Cronbach α as shown in Table 2 demonstrates that knowledge domain was excellent (α=0.85), while for attitude it was acceptably good (α=0.67). The overall Cronbach alpha was 0.74. Intraclass Correlation Coefficient conducted using test-retest methods among 40 respondents was found to be 0.77 (95% CI, 0.69–0.83).
Table 2 Cronbach Alpha and Corrected Item Total Correlation
Content Validity (I-CVI)
I-CVI obtained for this study was good. The average I-CVI in knowledge domain was 0.88 (Table 3). There were four items where all seven experts were in agreement for those items. For the domain on attitude towards ME reporting (Table 4), the I-CVI value was 0.81 and there were two items that had the same agreement between all experts. Both domains obtained values more than 0.78, hence, all items in both domains were retained for exploratory factor analysis (EFA).
Table 3 Rating on a 10-Item Scale for Knowledge on Medication Error by Seven Experts: Items Rated 3 or 4 on 4-Point Relevant Scales
Table 4 Rating on a 6-Item Scale for Attitude on Reporting the Medication Error by Seven Expert: Items Rated 3 or 4 on 4-Point Relevant Scales
Table 5 shows the results of EFA. Two factors were extracted among 16 items in this validation questionnaire process. These factors were “Knowledge on ME” and “Attitude towards ME reporting”. For “Knowledge”, items were found to have communalities ranging from 0.310 to 0.653. Meanwhile for “Attitude towards ME reporting”, the communalities for each item varied from 0.266 to 0.528. Factor loading ranged from 0.409 to 0.808 for “Knowledge”, and 0.315 to 0.724 for “Attitude towards ME reporting”. All items were retained in the EFA model as no deletion was required.
Table 5 Exploratory Factor Analysis for Medication Error Questionnaire
This study provides sufficient face validity, content validity, construct validity, internal consistency and test-retest reliability of a self-administered questionnaire on knowledge and attitude towards ME reporting among pharmacists. To the best of our knowledge, this is the first psychometric validation of a self-administered tool to assess the level of knowledge and attitude towards ME reporting among pharmacists working in government hospitals and health centers in Malaysia.
In the development phase of the questionnaire, a focus group discussion (FGD) was conducted between five senior pharmacists who had at least 10 years of experience in pharmacy practice. FGD is commonly used as a method for face validity where crucial information related to the study objective are discussed thoroughly between all experts. FGD is a very crucial and important method during questionnaire development phase where a researcher can use FGD as a tool to identify and discuss main issues related to the study objectives. Findings from the FGD in combination with those from the literature can be utilized to develop a new questionnaire.18,19
Content validity was obtained for this study by measuring the value of I-CVI. Sufficient values were obtained for both domains which indicated the overall content in each domain were acceptably good and retained in the questionnaires. I-CVI is a method to measure the interrater argument calculated for each item in the questionnaire. Hence, this method enables researchers to amend or retain the items according to the scale given by the experts.20 Polit and Beck described that I-CVI is a good method to study the validity of the items in the questionnaires and have suggested the value for I-CVI must exceed 0.78 if more than 6 experts were involved, whilst if the number of experts was five or fewer, all experts must be in agreement and the value will become 1.00.17 Findings from this study have shown that the I-CVI values for both domains are knowledge = 0.88 and attitude = 0.81, hence all the items in both domains are acceptable.
Meanwhile, for construct validity, the outcome was analyzed to determine the number of factors in the questionnaires by using exploratory factor analysis (EFA). The value of this analysis represents the direct correlations between the items and the factors. From 16 items in the questionnaire construct, all items were extracted into two factors which confirmed the discriminant validity between items in both domains. According to Hair et al, the recommended value for factor loading are 0.5 but 0.3 is the minimum value for the item to be accepted.21 The EFA obtained from this study shows all items were above 0.5 factor loading except three items which were between 0.32 and 0.47. All items in the questionnaires were retained and no deletion were required.
In terms of construct validity for hypothesis testing, the relationship between the score on knowledge and attitude on ME reporting was assessed. Spearman’s Rank correlation coefficient was utilized to determine the relationship between these domains (r = 0.10; p<0.05). This correlation was in positive direction, unfortunately the relationship was very weak. This finding shows that, regardless of the knowledge on ME, the relationship is just minimal towards the attitude of reporting the ME. A similar study by George et al has shown that good knowledge on ME did not give significant effect on attitude on reporting the ME.22
Regarding the reliability, the values for Cronbach alpha for both domains; knowledge and attitude demonstrate an acceptable internal consistency which allows the use of this questionnaire in descriptive population studies. The analysis of each item in both domains affirmed that none of the items were needed to be removed in order to give significant increase of α as suggested by some researchers.22–25
Temporal stability, calculated using ICC from this study shows a high degree of reliability where the ICC value was 0.77 (95% CI, 0.69–0.83). A few authors have suggested that ICC values of less than 0.5 are indicative of poor reliability, values between 0.5 and 0.75 indicate moderate reliability, values between 0.75 and 0.9 indicate good reliability, and values greater than 0.90 indicate excellent reliability. Thus, ICC value for this study demonstrates good reliability, and all items in this study are acceptable.16,26,27 Polit et al have suggested that ICC can also be used to reliably determine the chance of agreement between respondents and measure the redundancy of items in the questionnaires.27
The study has several limitations. Firstly, the questionnaires were conveniently distributed among pharmacist in hospitals and primary health clinics, thus the convenience sampling employed in this study might have resulted in sampling bias. Since the aim of this study was to assess the psychometric properties of a self-administered measure, this type of sampling method will not significantly affect the findings. Secondly, due to the nature of data collection in this study which was self-administered, the results could be exposed to recall and social-desirability biasness. Thirdly, this study did not measure criterion validity which can be associated with the study objective. This limitation occurred due to limited “gold standard” questionnaire for pharmacists on ME which can be used in the validation process. We believe that further studies can use this validated questionnaire as a criterion to develop new questionnaires.
The questionnaire developed from this study is a valid and reliable instrument to assess the knowledge on ME and attitude on ME reporting among pharmacists as it has shown a good psychometric value. Policy maker, researchers and pharmacists can use this questionnaire as a tool to measure perceived causes of ME and attitude towards ME reporting among pharmacists working in hospitals and healthcare centers.
We thank all pharmacists who participated in the study.
This study received no funding. The authors declare that they have no competing interests.
2. Samsiah A, Othman N, Jamshed S, et al. Medication errors reported to the National Medication Error Reporting System in Malaysia: a 4-year retrospective review (2009 to 2012). Eur J Clin Pharmacol. 2016;72(12):1515–1524. doi:10.1007/s00228-016-2126-x
3. Abdel-Latif MM. Knowledge of healthcare professionals about medication errors in hospitals. J Basic Clin Pharm. 2016;7(3):87–92. doi:10.4103/0976-0105.183264
4. Santell JP, Hicks RW, McMeekin J, et al. Medication errors: experience of the United States Pharmacopeia (USP) MEDMARX reporting system. J Clin Pharmacol. 2003;43(7):760–767. doi:10.1177/0091270003254831
5. Ernawati DK, Lee YP, Hughes JD. Nature and frequency of medication errors in a geriatric ward: an Indonesian experience. Ther Clin Risk Manag. 2014;10:413–421. doi:10.2147/TCRM.S61687
6. Khoo EM, Lee WK, Sararaks S, et al. Medical errors in primary care clinics – a cross sectional study. BMC Fam Pract. 2012;13:127. doi:10.1186/1471-2296-13-127
7. Ashcroft DM, Lewis PJ, Tully MP, et al. Prevalence, nature, severity and risk factors for prescribing errors in hospital inpatients: prospective study in 20 UK hospitals. Drug Saf. 2015;38(9):833–843. doi:10.1007/s40264-015-0320-x
8. Rajah R, Hanif AA, Tan SSA, et al. Contributing factors to outpatient pharmacy near miss errors: a Malaysian prospective multi‑center study. Int J Clin Pharm. 2019;41(1):237–243. doi:10.1007/s11096-018-0762-1
9. Keers RN, Williams SD, Cooke J, et al. Prevalence and nature of medication administration errors in health care settings: a systematic review of direct observational evidence. Ann Pharmacother. 2013;47(2):237–256. doi:10.1345/aph.1R147
10. Salmasi S, Khan TM, Hong YH, et al. Medication errors in the Southeast Asian countries: a systematic review. PLoS One. 2015;10(9):e0136545. doi:10.1371/journal.pone.0136545
11. Teoh BC, Alrasheedy AA, Hassali MA, et al. Perceptions of doctors and pharmacists towards medication error reporting and prevention in Kedah, Malaysia: a rasch model analysis. Adv Pharmacoepidemiol Drug Saf. 2015;4:192. doi:10.4172/2167-1052.1000192
12. Samsiah A, Othman N, Jamshed S, et al. Perceptions and attitudes towards medication error reporting in primary care clinics: a qualitative study in Malaysia. PLoS One. 2016;11(12):e0166114. doi:10.1371/journal.pone.0166114
13. Allard J, Carthey J, Cope J, et al. Medication errors: causes and prevention. Br J Haematol. 2002;116(2):255–265. doi:10.1046/j.1365-2141.2002.03272.x
14. Unver V, Tastan S, Akbayrak N. Medication errors: perspectives of newly graduated and experienced nurses. Int J Nurs Pract. 2012;18(4):317–324. doi:10.1111/j.1440-172X.2012.02052.x
15. Wakefield BJ, Uden-Holman T, Wakefield DS. Development and validation of the medication administration error reporting survey. In: Henriksen K, Battles JB, Marks ES, et al., editors. Advances in Patient Safety: From Research to Implementation (Volume 4: Programs, Tools, and Products). Rockville (MD): Agency for Healthcare Research and Quality (US);February 2005.
16. Hallgreen KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23–34. doi:10.20982/tqmp.08.1.p023
17. Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489–497. doi:10.1002/nur.20147
18. Costa-Tutusaus L, Guerra-Balic M. Development and psychometric validation of a scoring questionnaire to assess healthy lifestyles among adolescents in Catalonia. BMC Public Health. 2016;16:89. doi:10.1186/s12889-016-2778-6
19. Kumar CS, Harikiran AG, Development VD. Validation of questionnaire to assess the faculty perception on the dental public health curriculum in India. Int J Sci Stud. 2015;3(4):103–110. doi:10.17354/ijss/2015/316
20. Lynn MR. Determination and quantification of content validity. Nurs Res. 1986;35(6):382–385. doi:10.1097/00006199-198611000-00017
21. Hair JF
22. George D, Hss AS, Hassali A. Medication error reporting: underreporting and acceptability of smartphone application for reporting among health care professionals in Perak, Malaysia. Cureus. 2018;10(6):e2746. doi:10.7759/cureus.2746
23. Taber KS. The use of Cronbach’s alpha when developing and reporting research instruments in science education. Res Sci Educ. 2018;48:1273–1296. doi:10.1007/s11165-016-9602-2
24. Bolarinwa OA. Principles and methods of validity and reliability testing of questionnaires used in social and health science researches. Niger Postgrad Med J. 2015;22(4):195–201. doi:10.4103/1117-1936.173959
25. Tavakol M, Dennick R. Making sense of Cronbach’s alpha. Int J Med Educ. 2011;2:53–55. doi:10.5116/ijme.4dfb.8dfd
26. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. doi:10.1016/j.jcm.2016.02.012
27. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459–467. doi:10.1002/nur.20199
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]