Implementation of PROMIS^® in an Optometry Clinic

Introduction

The National Institutes of Health-funded Patient-Reported Outcomes Measurement Information System^® (PROMIS) consists of a set of computerized adaptive testing (CAT) questionnaires used to create a standardized set of patient-reported outcomes (PROs) which are freely available for use in clinical practice and research.¹ Based on recent research in psychometrics, item response theory, and qualitative health surveys, PROMIS has become a standardized set of CAT-administered PROs. It is utilized widely across clinical practice and research in the US and internationally as a validated measure of general health and wellbeing in both adult and pediatric patients, reported in more than 400 peer-reviewed publications to date.^2,3 This body of research validates the importance of such PROs as crucial to understanding a patient’s health-related quality of life, which has been demonstrated to require patient, rather than physician, estimation.⁴ PROMIS is available to researchers and clinicians free of charge, and provides quick and reliable evaluations of these quality of health measures.^5–8

In many institutions, PROMIS scores are linked to the electronic health record.⁹ This linked interface allows providers to view a variety of PROMIS scores and compare them with reference standards to support decision-making while remaining informed of the patient’s perspective. It also allows tracking of patient responses within functional domains over time and across visits to their various health care providers.⁹ Ideally, providers would have access to this resource, and know that these data are applicable to their specialty. Research into the feasibility and usefulness of PROMIS in many specialties has consistently found it to be of benefit when utilized appropriately.^10,11

The utility of PROMIS has not been documented in the eye care setting. Prior studies have described several vision-oriented questionnaires, some of which have been shown to reliably correlate vision-specific medical diagnoses with visual function and detriments to vision-related quality of life;^4,12–27 however, few studies have used non-vision-specific PROs in the eye care setting,^28–31 and the use of general PROs to identify individuals at greater risk for comorbid medical or psychosocial life disturbances due to visual impairment, or of visual impairment due to other medical or psychosocial factors, remains less certain.

This report details the implementation of computer adapted PROMIS surveys in an optometry clinic, analyzes patient factors which may relate to the feasibility of completing such surveys, and discusses the implications of our results on the equitable administration of PROMIS for patients with vision problems. At the University of Rochester Medical Center (URMC), more than 80% of the patient population has participated in PROMIS due to a University-wide effort to promote patient-centered care through the implementation of a standardized PRO system. Each department was asked to implement PROMIS in its own clinical setting and required to administer at least three domains of PROMIS: physical function, pain interference, and depression. An additional two domains could be chosen at the discretion of the clinical department. Based on the potential benefits to providers’ approach to treatment and patient supports, the URMC Department of Ophthalmology chose the additional domains of social isolation and cognition, as these are aspects commonly impacted by increased age and poorer visual acuity, both of which are prevalent in the general optometry clinic. We piloted these five PROMIS domains in the department’s optometry clinics as part of the medical center’s PRO system implementation.

Methods

Between October 28 and December 30, 2016, PROMIS surveys were distributed in an outpatient optometric clinic of the URMC Flaum Eye Institute in Rochester, NY. These surveys were offered to all patients by office staff members at check-in and were self-administered on electronic tablets following brief instruction by check-in staff. The tablet automatically recorded the time between opening the survey and finishing the survey, which we used directly as time to completion. Staffing and equipment constraints prevented the use of assistive technologies or other means of improving access to the questionnaire for patients with poorer visual acuity in this initial implementation period. In sum, 258 patients (of 651 patients to visit the clinic in total) agreed to complete the PROMIS survey. Some patients also participated in PROMIS in subsequent visits; in these cases (n = 14), we selected only the 1st encounter in which the survey was completed (or began, if the survey was never completed) to include in our data. The URMC Office of Human Protection granted Institutional Review Board approval for the administration of and collection of data from the surveys and the electronic medical record for said patients after obtaining informed consent for the use of this data from participating subjects. This report and the PROMIS implementation itself are adherent to the Health Insurance Portability and Accountability Act, the World Medical Association Declaration of Helsinki, and all federal and New York State laws.

Additional data extracted from the patients’ electronic medical record specifically include race (white or nonwhite), ethnicity (Hispanic or non-Hispanic), age (continuous), diagnosis of diabetes mellitus (presence or absence), and visual acuity. Visual acuity was abstracted from physician notes and was categorized into a 3-tier impairment score. If the distance visual acuity in either eye were 20/20 or better, then visual acuity was categorized as high; acuities worse than 20/20 in both eyes but at least 20/30 in either eye were categorized as moderate acuity, and those with vision poorer than 20/30 in both eyes were categorized as low acuity.

Data regarding those given and those who completed the survey in comparison with categorical demographic data and diabetes diagnosis were analyzed using Cochran-Mantel-Haenszel tests. Survey completion time in comparison with categorical demographic variables and diabetes diagnosis were analyzed using pooled t-tests. Data regarding those given and those who completed the survey in comparison with age were also analyzed using a pooled t-test. Age in comparison with time to completion was analyzed by linear regression. All statistical tests used a significance level of α=0.05. Data were analyzed using SAS^® version 9.4.³²

Results

Of the 258 patients who attempted the PROMIS survey, 78.8% were White. Due to the small percentage of patients in all other race categories, we chose to combine those categories into a single category, Non-White (21.2%) for the purposes of analysis. Hispanic patients made up only 3.1% of the sample; the Hispanic/Non-Hispanic ethnicity variable was not included in subsequent analyses due to the small sample size of Hispanic participants. Mean age was 58.9 years with a range of [20, 97] years. Most patients in our sample did not have a diagnosis of diabetes mellitus (80.6%).

Patients with low visual acuity were less likely to participate (χ²=3.86, p=0.049) in PROMIS, and the association of poorer visual acuity with not completing the survey once started trended towards significance (p=0.077). There was no significant association between age and participation in PROMIS (t=−0.62, DF=645, p=0.534), but older patients were less likely to complete the survey after starting (t=2.25, DF=254, p=0.025). There were no statistically significant differences in race, ethnicity, sex, nor diagnosis of diabetes between those who did and did not participate in PROMIS, nor between those who did and did not completed PROMIS once started. A summary of these data may be found in Table 1.

Table 1 Demographic Table: Characteristics of the Population of All Patients to Visit the Clinic, Patients Who Participated in PROMIS, and Patients Who Completed PROMIS, by Percentage of the Respective Population or Sample

Among the 258 patients who chose to participate in PROMIS, 193 (74.8%) completed the entire survey. Mean time to completion was 6.36 minutes, with a range of [1.43, 51.92] minutes and a standard deviation of 5.62 minutes. Of those who completed the survey, time to completion was not significantly different across any groups (race, ethnicity, sex, diagnosis of diabetes mellitus, age, nor visual acuity).

Discussion

Here we report an implementation of general health and functioning PROMIS surveys in an optometry clinic, and examine patient factors which may have led to less equitable participation in PROMIS or difficulties completing the surveys. General health and functioning PROs yield self-reported information on multiple domains, which could be of great importance to eye care providers. While only 39.6% of clinic patients agreed to participate in the PROMIS survey, our completion rate of 74.8% among those who agreed to participate indicates that PROMIS is feasible to implement in the optometry clinic.

Previous patient outcome studies in ophthalmology or optometry clinics report completion times between 15 and 30 minutes,^16,19,21,23 compared to an average of just over 6 minutes for our PROMIS surveys. Our low time-to-completion and high completion rate are especially encouraging as PROMIS is not a paid, proprietary resource, but rather a freely available resource, open to researchers and clinicians alike,¹ with a significant history of research to date validating its use in other settings.^2,3,5–8 However, our data does indicate a large variability in time-to-completion, suggesting that, for at least some of our subjects, and particularly older individuals, survey completion may have been difficult. The difference in completion based on age may suggest that CAT questionnaires, or perhaps the methodology of administrating the surveys via electronic tablets, are of lesser utility in older patients. The use of different means of survey administration that accounts for challenges older adults face when completing surveys may allow for greater survey completion rates, and should be addressed in future research.

Patients with better visual acuity also appeared more likely to complete the PROMIS survey. While this trend was not found to be significant in this report (p=0.077), it is noteworthy that only 4% of those who completed the survey were categorized as poor acuity, as those with poor vision were less likely to participate in the survey. While staff selection biases are one possible explanation, this would only be likely in the case of patients with profound vision loss requiring assistive devices or help from others to perform visual tasks, as check-in staff have no way of becoming aware of a patient’s visual acuity beyond that which is obvious to the general public. Further, staff were instructed to offer PROMIS to all patients at check-in, and it seems unlikely that if they forgot to do so, it would be more prevalent specifically in patients of poor visual acuity. This could be clarified by allowing office staff a means to document declined participation; however, patients who declined participation did not give consent for any information to be included in the present report, and thus it is not possible to report such information here. The more likely explanation of the correlation between visual acuity and choosing to participate in PROMIS is patient self-selection. Delivering PROMIS in a format that allows for those with poorer visual acuity to feel more comfortable attempting PROMIS, such as by utilizing assistive technologies or having a staff member offer to read the questions and enter the patient’s verbal responses, may have increased the number of those with lower visual acuity who participated. Such aids may also have increased the completion rate among older adults attempting the PROMIS survey. As such, we recommend future studies to include such aids so as to avoid any inherent bias in the distribution of PROMIS, or in the clinical benefits its use may provide, due to the means of its distribution. This is perhaps especially important in optometry and ophthalmology clinics, where the prevalence of poor vision in patients is expected to be higher than in most clinical settings.

As little information was available for those patients declining to participate, it is difficult to speculate on the reasons for non-participation. Many studies do not report their overall completion rate, making direct comparison difficult.^16,19 It is also difficult to put this in context as this report is one of the few from our institution that reports the number of patients declining to participate. Our rates of attempting and completing PROMIS once attempted are well within the ranges reported at the University of Rochester and elsewhere.^33–41 It is important to qualify this, however, as PROMIS is a customizable survey instrument, and we could find no other study reporting the exact PROMIS survey used here.

Finally, PROMIS was only available in English for our implementation, which likely limited non-English speakers’ participation. Future research incorporating the use of PROMIS in other languages would help to diversify the population participating in PROMIS and further generalize results to more diverse populations.

Conclusion

Our high completion rate and short time-to-completion demonstrate the feasibility of implementing such general PROMIS surveys in the eye care setting; however, the lower participation rate in patients of poorer visual acuity indicates a need for the use of alternate means of survey administration such as verbal aids or assistive devices in order not to neglect this vulnerable population. As our institution has captured both clinical and PROMIS data from this initial implementation, our next study aims to compare PROMIS results and clinical data relevant to eye health to determine if general wellness and overall functional ability is associated with vision and eye health status – which, if present, may highlight the further utility of using PROMIS in eye care settings.