Validation of a glaucoma knowledge assessment in glaucoma patients
Authors Rao VS, Peralta EA, Rosdahl JA
Received 20 July 2016
Accepted for publication 3 September 2016
Published 4 October 2016 Volume 2016:10 Pages 1913—1918
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 4
Editor who approved publication: Dr Scott Fraser
Veena S Rao, Esteban A Peralta, Jullia A Rosdahl
Department of Ophthalmology, Duke University, Durham, NC, USA
Summary: To develop metrics to identify knowledge deficits and barriers to learning in glaucoma patients, this study seeks to validate a glaucoma knowledge assessment to use in the evaluation of knowledge in glaucoma patients.
Purpose: Glaucoma treatment adherence appears to improve when patients demonstrate a greater knowledge of the disease and its treatment. This study seeks to validate a glaucoma knowledge assessment in a glaucoma patient population to assist in patient educational assessments and interventions.
Patients and methods: The National Eye Health Education Program’s (NEHEP) glaucoma knowledge assessment has previously been suggested as a useful measure to assess glaucoma patient’s knowledge. This questionnaire was administered in glaucoma patients along with a more comprehensive 49-question examination. Demographic data and health literacy were assessed. Statistical analyses were performed to assess the validity of the assessments.
Results: A total of 12 glaucoma patients completed the knowledge assessments. The mean ± standard deviation age of the patients was 69±14 years, and the duration of glaucoma was 14±13 years. The participants’ mean score on the NEHEP assessment was 7.3±0.8 (of 10, 73% correct) vs 29.3±7.3 (of 49, 60% correct) on the comprehensive assessment. The value of coefficient α was 0.592 for NEHEP and 0.872 for the cumulative assessment. The P-value (proportion of examinees answering correctly) ranged from 0 to 1 for individual questions. Item point–biserial correlation values for each question ranged from –0.402 to 0.813.
Discussion: The NEHEP quiz may be a good starting point for the development of a reliable knowledge assessment tool to measure and monitor glaucoma knowledge, due to its concise nature and reasonable level of difficulty. This study suggests that questions included in the currently available questionnaires vary widely in difficulty and ability to differentiate knowledge level, which may ultimately compromise reliability and utility of existing examinations. Further examination of individual questions and teaching points included in the current assessments may help to construct increasingly reliable and useful knowledge assessments in the future.
Keywords: ophthalmology, patient education, educational tools, knowledge questionnaire, knowledge quiz
Glaucoma is a chronic, progressive blinding disease that typically requires lifelong treatment and follow-up care. Despite the lack of symptoms in early and moderate disease, it is key for patients to adhere to treatment plans, including physician visits and treatments to prevent vision loss.1 This can be especially challenging, considering the time burden of physician visits and potential side effects from medical and surgical treatments. Adherence to medical therapy in glaucoma, like other chronic diseases, has been documented to be poor.2,3 Up to 80% of glaucoma patients on medications have been shown to have adherence issues,4 and issues with non-adherence appear to be worse in patients on multiple medications.5,6
The current literature suggests that glaucoma patient adherence is improved with better patient education.7 Greater knowledge of the disease process may allow patients to better understand their treatment plan and actively participate in their care. Surveys of glaucoma patients indicate that there is room for improvement in patient education; specifically, patients have demonstrated lack of knowledge regarding the disease process in glaucoma, medication side effects, and the natural history of glaucoma without treatment.8 A lack of understanding of the severity of glaucoma has been correlated with poor adherence.9
Considering these difficulties, it can be inferred that improved patient education, leading to improved understanding of glaucoma, might improve adherence to treatment and, ultimately, lead to better outcomes. In order to better direct patient education, it might be useful to have a method to formally assess glaucoma knowledge during routine clinical care. This could be useful to assess both baseline knowledge and the effectiveness of educational interventions. We previously identified four existing glaucoma knowledge assessment quizzes available to the public:10 the National Eye Health Education Program (NEHEP) Eye-Q test,11 the Prevent Blindness America (PBA) Eye-Q test,12 Gray Glaucoma Knowledge Assessment,13 and Hoevenaars Glaucoma Knowledge Assessment.14 The Gray and Hoevenaars assessments were developed as research instruments. In that study, glaucoma clinicians were surveyed on the quality of the four different assessments and determined a preference for the 10-question NEHEP Eye-Q test for use in clinical practice.
Here, we seek to validate this questionnaire in glaucoma patients and determine the appropriateness of its use in assessing knowledge of glaucoma in these patients. The results of this quiz are compared to the results of a 49-question cumulative examination comprising all questions from the abovementioned four knowledge assessments.
Patients and methods
This prospective study was approved by the Duke University Medical Center’s institutional review board and was conducted in accordance with the standards of Health Insurance Portability and Accountability Act and the Declaration of Helsinki.
The participants were recruited from the Glaucoma Service of a tertiary care academic practice. Written informed consent was obtained from all participants. The inclusion criteria were: capable and willing to provide consent, history of diagnosed glaucoma or history of ocular hypotensive therapy (eg, high-risk glaucoma suspects or ocular hypertensives) based on glaucoma specialist’s documented assessment in the medical record, at least 18 years of age, English speaking (the glaucoma knowledge assessments were in English), and willingness to complete the study activities. Notably, there were no minimum criteria for visual acuity. If the study participant was unable to see to complete an activity (eg, unable to complete the literacy instrument), the study staff read the question or item aloud to the study participant. No compensation was offered for this study.
The potential participants were approached by the study team after an introduction of the study by a member of their care team (per institutional policy). The participants completed the study activities while waiting for their glaucoma doctor during a clinic visit; the study activities included the completion of two glaucoma quizzes, a demographic questionnaire, and a health literacy assessment (the Rapid Estimate of Adult Literacy in Medicine – Short Form (REALM-SF) instrument). The demographic and educational information collected included age, race/ethnicity, gender, education level, glaucoma treatments received (medications, laser, surgery), and presence of any other eye disease. The medical record was reviewed to confirm the information provided by the study participants. The glaucoma quizzes consisted of: 1) the 10-question NEHEP Eye-Q test and 2) a 49-question quiz consisting of questions (in random order) from the NEHEP Glaucoma Eye-Q test, the PBA Glaucoma Eye-Q test, the Gray Glaucoma Knowledge Assessment, and the Hoevenaars Glaucoma Knowledge Assessment. The format of each assessment was as follows: NEHEP, 10 true–false questions; PBA, 9 true–false questions; Gray, 11 short-answer questions; and Hoevenaars, 19 true–false questions.
Statistical analyses of the quiz questions were performed using the Iteman 4 (Assessment Systems) validation software: 1) the mean scores, the alpha value, and the standard error of measurement (SEM) were calculated for the NEHEP quiz, the 49-question quiz (total), and the subtotals of the PBA, NEHEP, Gray, and Hoevenaars questions within the 49-question quiz; 2) intercorrelation values were calculated for PBA, NEHEP, Gray, and Hoevenaars subtotals; and 3) the P-value (the proportion of participants who answered the question correctly, evaluating how easy or hard a question is) and the point–biserial correlation value rpbis (which correlates the response with the total score, evaluating how well each question correlated to the overall score) were calculated for each individual question. Descriptive statistics were used for the demographic information of the participants.
A total of twelve glaucoma patients were recruited in this study. The mean age of the patients was 69±14 years, and 58% (n=7) of patients were female (Table 1). The patients had been diagnosed with glaucoma for a mean of 14.3±12.7 years, and most (92%, n=11) were taking glaucoma medications. The majority of patients had undergone glaucoma surgery (83%, n=10). The educational level ranged from no high school education to graduate degree; 25% of patients (n=3) had less than a high school diploma and 50% (n=6) had at least some college education. The mean REALM-SF score to assess health literacy was 5.5±2.1, which is consistent with a seventh to eighth grade level of education.15
Table 1 Demographics of participants
The highest mean score of knowledge assessments was for the NEHEP assessment (73%), followed by the PBA (67%), Hoevenaars (64%), and Gray (35%; Table 2). The cumulative assessment of all quizzes had a mean score of 60%. The value of coefficient α was 0.592 for NEHEP and 0.872 for cumulative assessment; the SEM for the NEHEP quiz was 1.1, less than that for the cumulative quiz (2.6). The P-value (proportion of examinees answering correctly) ranged from 0 to 1 for all questions. Item point–biserial correlation values for individual questions ranged from −0.402 to 0.813. The PBA assessment correlated most with the other assessments (Table 3) with NEHEP second in quiz intercorrelation.
Table 2 Summary of participant scores and reliability
Table 3 Quiz intercorrelation
The P-value (proportion of patients answering each question correctly) for each question is presented in Table 4. Notably, the Gray and Hoevenaars assessments were the only two assessments with “hard” questions (P<0.334). “Easy” questions (P>0.667) were present in each of the four assessments. There was no particular subject category of question that was consistently either “easy” or “hard” when examining the proportion of patients answering the questions correctly.
Table 4 Analysis of individual questions
In order to better assess glaucoma patient’s knowledge and monitor learning during routine clinical care, a standardized glaucoma knowledge assessment would be instrumental. This would be especially useful in light of literature that supports that adherence in glaucoma patients is often poor4 and tends to improve with better knowledge of the disease.7 The purpose of this study is to assess the validity of commonly available glaucoma knowledge assessments in an effort to identify useful means to evaluate patient’s knowledge in a busy clinical setting. As our recent report suggested the NEHEP glaucoma knowledge assessment as the most useful of these quizzes in this setting,10 we focused on this assessment in this study.
In this group of glaucoma patients, the mean score on the NEHEP knowledge assessment was 7.3±0.8 (of 10). This mean score may suggest an appropriate level of difficulty for most patients. However, the value of coefficient α for the NEHEP quiz was 0.5, suggesting that the quiz may have poor internal consistency. The value of coefficient α for the cumulative 49-question quiz was more robust at 0.8, which suggests that the more comprehensive quiz might be more reliable and internally consistent for determining patient’s knowledge. Interestingly, the Gray assessment also had a high α value, which could be attributed to its short answer, more interactive nature. Despite the high α coefficient value for the Gray assessment, it should be noted that multiple short-answer questions were either omitted or inappropriately answered with a single word by patients in this study. It is not clear if these questions were omitted due to lack of knowledge or due to the extra effort the short-answer question required over the true–false questions (all true–false questions were completed appropriately in the study). This may suggest that although short–answer questions may be more revealing about patient’s knowledge, they may not be the most efficient and effective means to assess knowledge in a busy clinic setting, as suggested by the glaucoma specialists in our prior study.
Furthermore, when examining the item point–biserial correlation, which serves as an indicator of differentiating power for individual questions, this varied widely from −0.402 to 0.813 (possible range from −1.0 to 1.0, with higher values suggesting greater discriminating ability). This suggests that some questions used in each questionnaire would have limited utility in determining the level of glaucoma patient’s knowledge. For example, 3 of 10 questions in the NEHEP quiz had an item point–biserial value of 0.0, suggesting no ability to discriminate between low-knowledge and high-knowledge assessment takers.
The results of this study suggest that the longer, cumulative knowledge assessment may provide a more reliable indicator of glaucoma patient’s knowledge. Although intuitively it is reasonable to assume a more thorough test would provide more reliable results, it would also be pragmatic to identify a concise means to assess patient’s knowledge during a busy clinical encounter. For example, the three NEHEP questions that received a 100% correct score and the item point–biserial value of 0.0 appear to add little to the differentiation of higher and lower knowledge patients in this study. These questions may still have a role in educating the patient, for example, to reinforce important information.
The strength of this study includes the examination of four commonly available knowledge assessments in a single patient population. The patient population in this study had a diverse range of ages (44–88 years), an equal distribution of genders, a wide range of time since glaucoma diagnosis (1–40 years), and a diverse level of education (from no high school to graduate degree). The questions included in this study consisted of a wide range of topics, and each of the 49 questions was individually analyzed for its appropriateness. The weaknesses of this study are the small sample size and the overall high level of glaucoma severity, considering 10 of the 12 patients have undergone glaucoma surgery, which may limit its generalizability. There was no minimum number for assessing validity; thus, a small group of diverse patients was used to preserve the larger population of quiz-naïve potential participants in our patient pool for future studies; the limited sample size likely contributed to the lower alpha values and larger SEM values. Additionally, there was no known “gold standard” for glaucoma knowledge; hence, the topics covered in this study may not be most relevant to all patients and all clinicians. Additional limitations include completion at a single institution and only English-speaking patients were included.
Overall, this study aims to assist in the educational initiatives of glaucoma patients by assessing the validity of currently available glaucoma knowledge assessments. The NEHEP knowledge assessment, previously ranked by glaucoma clinicians as most useful for clinical practice, here was suggested to have a reasonable level of difficulty but a potential lack of internal consistency. Approximately 50 glaucoma knowledge questions assessed here showed varying levels of difficulty and internal consistency.
Clinicians can use the data on each of the questions as a starting point to create their own assessments for their patients, potentially allowing glaucoma knowledge assessments to be tailored to individual patient and provider needs. In the future, it might be useful to tailor not only a general glaucoma quiz but also assessments targeted for patients with varying educational backgrounds and/or glaucoma severity. By developing more accurate means to assess glaucoma knowledge, we ultimately hope to better implement and assess educational interventions for our patients.
The authors thank Christina Bunn, Department of Ophthalmology, Duke University, for IRB regulatory assistance.
JAR serves on the planning committee of the NEHEP but was not involved in the development of the NEHEP Eye-Q questionnaire. The other authors report no conflicts of interest in this work.
Vass C, Hirn C, Sycha T, Findl O, Bauer P, Schmetterer L. Medical interventions for primary open angle glaucoma and ocular hypertension. Cochrane Libr. 2007;(4):CD003167.
Haynes RB, McDonald HP, Garg AX. Helping patients follow prescribed treatment: clinical applications. JAMA. 2002;288(22):2880–2883.
Nordstrom BL, Friedman DS, Mozaffari E, Quigley HA, Walker AM. Persistence and adherence with topical glaucoma therapy. Am J Ophthalmol. 2005;140(4):598.e1–598.e11.
Olthoff CM, Schouten JS, van de Borne BW, Webers CA. Noncompliance with ocular hypotensive treatment in patients with glaucoma or ocular hypertension: an evidence-based review. Ophthalmology. 2005;112(6):953–961.e7.
Robin AL, Covert D. Does adjunctive glaucoma therapy affect adherence to the initial primary therapy? Ophthalmology. 2005;112(5):863–868.
Claxton AJ, Cramer J, Pierce C. A systematic review of the associations between dose regimens and medication compliance. Clin Ther. 2001;23(8):1296–1310.
Waterman H, Evans JR, Gray TA, Henson D, Harper R. Interventions for improving adherence to ocular hypotensive therapy. Cochrane Libr. 2013;(4):CD006132.
Danesh-Meyer HV, Deva NC, Slight C, et al. What do people with glaucoma know about their condition? A comparative cross-sectional incidence and prevalence survey. Clin Exp Ophthalmol. 2008;36(1):13–18.
Kosoko O, Quigley HA, Vitale S, Enger C, Kerrigan L, Tielsch JM. Risk factors for noncompliance with glaucoma follow-up visits in a residents’ eye clinic. Ophthalmology. 1998;105(11):2105–2111.
Rosdahl J, Muir K. Finding the best glaucoma questionnaire: a qualitative and quantitative evaluation of glaucoma knowledge assessments. Clin Ophthalmol. 2015;9:1845–1852.
National Eye Health Education Program Eye Q Test. Available from: https://nei.nih.gov/sites/default/files/nehep-pdfs/EyeQTest_for_Toolkit.pdf. Accessed September 27, 2016.
Prevent Blindness America (PBA) Eye Q Test. Available from: http://www.preventblindness.org/sites/default/files/national/documents/fact_sheets/MK19_GlaucEyeQ_0.pdf. Accessed September 27, 2016.
Gray TA, Fenerty C, Harper R, et al. Preliminary survey of educational support for patients prescribed ocular hypotensive therapy. Eye (Lond). 2010;24(12):1777–1786.
Hoevenaars JG, Schouten JS, Van den Borne B, Beckers HJ, Webers CA. Knowledge base and preferred methods of obtaining knowledge of glaucoma patients. Eur J Ophthalmol. 2004;15(1):32–40.
Arozullah AM, Yarnold PR, Bennett CL, et al. Development and validation of a short-form, rapid estimate of adult literacy in medicine. Med Care. 2007;45(11):1026–1033.
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]