Setting Measurement-Based Care in Motion: Practical Lessons in the Implementation and Integration of Measurement-Based Care in Psychiatry Clinical Practice

Introduction

Measurement-based care (MBC) is an integral part of routine clinical medicine. Patients and clinicians expect to have regular measurements of blood pressure to monitor hypertension and hemoglobin A1C to monitor diabetes. Without these measures, quality care is impossible. Psychiatry, though, has been slow to incorporate MBC into routine practice. To do so would mean systematically using standard clinical tools to aid in individualized clinical decision-making regarding diagnosis, treatment selection, side-effect management and relapse prevention.

Using MBC in psychiatry has clearly been shown to improve quality of care and clinical outcomes.^1–5 MBC also helps engage and retain patients in treatment.⁶ MBC engages patients in the acquisition of therapeutic and side effect outcomes to inform clinical decision-making and facilitate patient-clinician collaboration.

Despite this wealth of evidence, MBC is rarely incorporated in routine behavioral health care. A review of the MBC literature states that “typically less than 20% of behavioral health practitioners (are) integrating it into their practice”.⁷ Even regulatory pressure has been slow to change practice. For example, the Joint Commission’s 2018 standards for addiction and behavioral health-care programs include the use of a standardized tool to monitor a patient’s progress and evaluate outcomes of care. Failure to integrate this MBC standard was the third most frequently cited deficiency, with 70% of the citations noting failure to even select an instrument.⁸

Recognizing the value of MBC, the Department of Psychiatry at UT Southwestern Medical Center in Dallas, under the leadership of the Chair (CT) and the Vice-Chair (HI), decided to implement MBC in our general Psychiatric Outpatient Clinic. This report presents the strategies, challenges, solutions, and subsequent adaptations we used to implement MBC in this clinic.

The Health System Context

In 2016, the Psychiatry Clinic at UT Southwestern Medical Center faced rapidly increasing demands for timely behavioral health care from our expanding medical and surgical practice. This growing need required us to quickly transform our care team and clinical processes. In three years, we tripled the number of staff and developed multidisciplinary care teams composed of psychiatrists, psychologists, nurse practitioners, physician assistants, and master’s level psychotherapists including licensed clinical social workers (LCSWs) and licensed professional counsellors (LPCs). Our aim was to make the clinic a “one-stop-shop” where multiple treatment options would be offered, including individual and group psychotherapies, medication management, subspecialty care such as addiction and geriatric psychiatry, as well as expert psychopharmacological consultations and interventions, such as esketamine and transcranial magnetic stimulation (TMS).

The increase in patient volume, diversity and acuity was substantial. In 2015, our Psychiatry Clinic received approximately 2050 referrals and provided care to 2746 unique patients. Care was provided by clinician time of 6.38 full-time equivalents (FTEs). Based on the growing needs of the UT Southwestern health-care system, clinician time was expanded by 2020 to 44 FTEs. With this increased staff, we were able to offer care to a much wider patient population, with around 11,936 referrals and serving 11,017 unique patients in the 2020 academic year. Due to the coronavirus disease 2019 (COVID-19) pandemic, we made a shift to providing care virtually to 85% of patients in March 2020, requiring further adaptation of our processes. This paper highlights the lessons learned from our implementation of MBC into an existing multidisciplinary, multispecialty, academic psychiatry clinical practice.

Measurement-Based Care: Lessons Learned

Administrative Direction, Support, Oversight, and Management of MBC Implementation

During this expansion which began in 2016, clinical leadership decided to implement a standard set of measures to be used across all patients by clinicians at all clinic visits to enhance diagnosis, systematically personalize treatment delivery, and improve communication amongst clinicians and clinical settings (ie, primary care or specialty care to psychiatry). These measures would also provide information to assist us in launching quality improvement projects.

Implementing MBC requires substantial commitment of time, energy, resources and leadership, who must take an active, enthusiastic role in moving the project forward. We established a clear administrative structure with both the Vice Chair and Medical Director taking on the role of project champions. The Vice Chair created the charter for the project, solicited assistance from the electronic medical record (EMR) and quality improvement teams at UT Southwestern to assist with incorporation of measures into the EMR and kept us moving forward with a mandate to implement a comprehensive MBC plan. The Medical Director primarily worked with clinicians to address their specific concerns about the impact of implementing MBC on patient schedules and clinic workflows. These efforts helped to overcome resistances, share project ownership, and develop consensus amongst clinicians with diverse opinions about measure selection and how to best adjust clinic operations. For example, the Medical Director addressed ongoing MBC-related problems during monthly meetings with each provider, championed collection compliance and implemented a group incentive to encourage clinicians to allow patients time for completion before starting each session, which resulted in a high completion rate (with measures collected on 80% of visits). The Vice Chair, Medical Director and Practice Administrator formed the MBC committee for development and implementation, bringing in additional expertise from the EMR, workflow analysis and quality improvement teams throughout the development and implementation phases of the project.

The most substantial time commitment was that of the EMR analyst, who spent time researching methods for building out the questionnaires and the display in the EMR. The questionnaire build took around 3 months, to determine the computational logic required for questionnaire population and to ensure the interface was patient friendly. While the EMR analyst was building the digital version, we planned the workflow and began staff training using the paper form and discussing the changes that would occur once the digital workflow was in place on the tablets. Once the tablets were set up with the questionnaires, we worked one on one with staff using role-playing exercises until the process was clear and included the project in our weekly staff meetings to troubleshoot ongoing issues that arose and make certain that all staff were comfortable with the process.

The primary cost of the project was in personnel time as described above. There were additional costs associated with the tablets used for electronic measure administration,; however, these were not as substantial as staff time and with the advent of tele-medicine we are exploring more cost-effective alternative delivery methods for MBC (further described in Add to the MBC Platform for Subspecialty Care).

Engaging the Clinical Staff

Clinician engagement, understanding, and willingness to implement this new process were pivotal to its success.^2,9,10 Focusing on better patient care appealed to professional core values and made the work and change seem worthwhile. In addition, having a solid and well delineated MBC action plan seemed to facilitate this process.^11,12

To increase clinician enthusiasm and project ownership, we initiated several formal discussions with the clinical staff in advance of MBC implementation. These meetings were crucial for understanding the practical and psychological barriers to use and designing shared solutions that clinicians could embrace. After implementation, we continued to evaluate the MBC process in monthly provider meetings with the goal of improving the process through provider feedback.

We initiated clinician engagement by outlining and agreeing on the ways in which access to measures could improve patient care including: (1) Providing direction for the visit (2) assisting with targeting the evaluation portion of the visit which allows more time for treatment discussions, and, (3) tracking of symptoms to assist with needed changes in treatment to reach remission. While our clinicians were aware of the reported utility of MBC, they were concerned about adding more bureaucracy which would impede clinic flow without clear and immediate clinical benefit. We identified the practical issues of concern (Table 1) as stated by the clinicians and arranged a series of meetings for the MBC development team to tackle each area and explore solutions.

Table 1 Practical Concerns in Implementing MBC

Our clinicians suggested that they were less likely to use MBC if reviewing the data entailed an extra step outside the EMR. Therefore, we chose questionnaires that could be directly entered on a tablet and populated into the EMR in real time. Clear, easily readable measure score displays were essential to enhancing clinician buy-in and utilization of MBC. In addition, our MBC data display in the EMR was designed to allow longitudinal monitoring of outcomes and treatment progress. For each measure, we developed a line graph that includes the last three total scores and that displays on the sidebar of the patient visit (Figure 1). Furthermore, to make both item level responses and total scores easily accessible and visible to our clinicians, we displayed the individual item responses and total scores in separate windows (see Figure 1).

Figure 1 Example of measurement-based care scale display in patient electronic medical record.

Selection of Measures

We now turned to selecting which measures to use. Of the utmost concern was time. Time spent in the clinic waiting room or total visit time influences willingness to seek care, adherence to visit schedules/workflows and patient satisfaction. Unlike the inclusion of MBC in research trials where time constraints are less of an issue and a diagnosis is specified, the use of brief, self-report measures over a diverse set of symptoms in clinical care is crucial for minimizing disruptions to the clinical schedule.^5,13,14

The first task was to identify the purpose/focus for the overall measurement process, as this would drive decisions about which scales to include. Since a primary goal of adding MBC to our clinic was to assist with diagnostic evaluations, we decided to start with symptom-based screening measures for the most common outpatient conditions seen in a large general psychiatry clinic: depression, anxiety, suicidality, substance abuse, obsessive-compulsive thinking, panic, attention deficit, mania, psychosis, sleep and pain. We conducted a literature review to identify two or three nonproprietary, reliable, commonly accepted, valid, self-report measures of symptoms germane to each of the conditions of interest, and presented these findings to the clinicians to gain consensus about our final battery of measures (Table 2). To make our final selection, special consideration was given to the brevity and ease of use of every measure. In addition, the desire to speak a common language with our medical clinics, make comparisons across clinical settings, integrate psychiatry practices with primary care, and make direct comparisons across visits steered the selection of measures. If a measure was already in use in the Health System or within the Department of Psychiatry, we chose that measure to facilitate collaboration and communication across the system. For example, three self-report measures for depression were initially identified by our literature search, including the 21-item Beck Depression Inventory (BDI),¹⁵ the 16-item Quick Inventory of Depressive Symptomatology-Self-Report (QIDS-SR),¹⁶ and the 9-item Patient Health Questionnaire (PHQ-9).¹⁷ The PHQ-9 was ultimately selected because it is relatively shorter than the other two measures and easy to use, but also because it was already in use throughout our health system as a screening tool. This measure was therefore deemed the most suitable measure for depression given its wide support and reliability from our literature review, combined with the use and translational opportunities afforded through consistent use of the PHQ-9 within our Health System.

Table 2 MBC Measurement Battery

We started by building a “core” group of measures that the clinicians felt were important across all diagnoses and visits. Next, we identified a group of measures related to specific diagnoses of interest. As we were selecting measures, we conducted several timed trials to ensure that the battery could be completed within 30 minutes, even for a slow responder. Some of these measures were subsequently adapted and truncated due to time constraints resulting from delivery in a clinical context (see Table 2).

When to Measure

Since our primary objective was to assist in clinical diagnosis, we decided to deploy all the measures at the initial visit. At subsequent visits, each patient would complete only those questionnaires pertinent to the diagnoses they had received (“Add-on scales”, see Table 2) in addition to the core set of questionnaires. Since MBC was to help in clinical decision-making treatment selection, and treatment adjustment (thereby enhancing quality of care), we decided that medication management visits should routinely be accompanied by MBC measures pertinent to that patient, regardless of time between visits,³⁴ while patients engaged in psychotherapy would only complete the questionnaires monthly. For example, a patient with the diagnosis of “Bipolar Disorder” would find both the “core battery” and the “add-on” measure for mania. The questionnaire battery was set up in this format to help alleviate “questionnaire fatigue”. The established patient battery was designed so that it could be typically completed within 15 minutes for most patients.

The questionnaire assignment in the EMR was built on two factors: visit type and diagnosis. The visit type triggered questionnaire deployment, either all questionnaires for a “new” patient or core questionnaires plus diagnosis-specific measures for an “established” patient. The diagnosis as indicated in the EMR problem list drove which “add-on scales” populate each “established” visit.

Expect and Plan for Patient Variability

In our large general psychiatric outpatient clinic, patients vary greatly with regard to the number of concurrent general medical and psychiatric comorbid conditions, which impacts the choice of measures and the allocation of clinician time. Some patients are referred from a medical specialist without discussion, while others may have been in psychiatric care in the community for years but are now being referred to the medical school for a specialized evaluation or treatment.

Several months after implementing MBC, clinicians and front office staff provided feedback that while some patients were completing the “core battery” of questionnaires in 10 minutes, many were spending so much time on the tablet questionnaires that it was impacting their appointment time. To address this variability, we prioritized and ordered our “core battery” measures with the first 3 scales ordered, aimed at depression (Patient Health Questionnaire, PHQ-9),¹⁷ anxiety (Generalized Anxiety Disorder, GAD-7),¹⁹ and suicidality (Concise Health Risk Tracking, CHRT-7).¹⁸ While the entire battery is delivered to patients in the hope that they will complete all scales, this prioritization ensured we would be able to collect information about at least these symptoms (the most common complaints among our clinical population) from the majority of patients at each visit prior to the provider needing to start the visit.

Patient Engagement

Treatment engagement, retention, and adherence are the biggest challenges in delivering effective interventions,⁶ and MBC.^10,35 These facts influenced our decision regarding how to deliver the questionnaires to patients. While we were building the electronic MBC platform in our EMR for delivery on tablets in the clinic, we introduced the concept and process to patients in paper form. We created a one-page patient handout which explained the benefits of MBC, emphasizing how routine measurement of symptoms would help the patient and clinician improve evaluation and treatment. Alongside this handout, we included the PHQ-9, GAD-7 and CHRT-7 self-report measures. We titled the form “Care Eval” and used this tool to educate patients about the potential benefits of engagement with MBC as part of their clinic visits.

We chose to implement MBC via tablets in our clinic due to low rates of engagement with the patient portal/secure EMR email system. However, in response to the COVID-19 pandemic and the shift to telehealth for care, we now send the MBC battery through the patient portal/EMR secure email 24 hours in advance of the telehealth appointment (this is explored further in Add to the MBC Platform for Subspecialty Care). For patients who are seen in-person, the questionnaires can still be completed on the tablet if not answered in advance through the patient portal.

A major concern for providers was the patient’s willingness to arrive early enough to complete the measures prior to the initiation of the treatment visit. Therefore, we built in a pre-visit check in time for patients to arrive for questionnaire completion and labeled this time “Care Eval,” so patients would understand that the time required to complete these measures was part of their care. Working within the constraints of our EMR scheduling templates and appointment reminder system led us to show patients the arrive by time (30 minutes in advance for new patient appointments and 15 minutes for established appointments), rather than the appointment time (the actual time the provider will see the patient). The inclusion of the paper form and “Care Eval” process allowed us to not only introduce the concept of MBC to our existing patients, but gave patients practice at what would be an expanded process when the tablets were introduced with the larger MBC battery, which occurred approximately 6 months after the introduction of the paper form. From the start, we provided scripting and training to staff and clinicians to standardize conversations around the purpose and process. Later, we also created posters for the clinic to reinforce the benefits of MBC (Supplemental Figure 1).

We strongly advised clinicians to explicitly introduce their patients to the purpose and processes involved in MBC, which was found to be very helpful. In addition, clinicians were encouraged to refer to and make use of the data during the patient’s appointment, in order to highlight the importance and express appreciation for the patient’s efforts in completing these measures. The incorporation of patient questionnaires into the visit by the provider helps with understanding the patient’s current functioning as sometimes the questionnaires might tell a different story from what the patient is endorsing and lead the visit in an alternative direction, such as in the following clinical case studies shared by providers:

Clinical Case Study 1: A 38-year-old woman with a history of recurrent depression was treated with a selective serotonin reuptake inhibitor (SSRI). After 4 weeks, she reported that she was doing better and was pleased by her improvement. Her PHQ-9 had decreased from 18 to 6. Over the next few months, she continued to report that she was doing well and denied any symptoms or stressors on direct questioning. Her PHQ-9, though, gradually increased back into the low teens. When this discrepancy was brought to her attention, she tearfully admitted that she had been under a lot of stress at work, that this was affecting her marriage, and that she had, in fact, been more depressed. This disclosure opened up an opportunity to make a medication adjustment and to engage the patient in psychotherapy.

Clinical Case Study 2: A 40-year-old man presented with depression. He was reluctant to accept treatment and was very negative about himself and his life. However, he eventually agreed to antidepressant medication treatment. After 2 months, he denied any improvement and felt that the medication was not working. His PHQ-9, though, had decreased from 22 to 14. When this was brought to his attention, he agreed he might be a little better and accepted a dose increase. Two months later, his PHQ-9 was 4. He reported that he was feeling much better, more confident and hopeful, and was glad he had increased the medication.

The above scenarios underline the importance of MBC integration into patient care by providing critical information to clinicians, encouraging patient engagement, disclosure and collaboration in treatment, and detecting change that patients may overlook.

Revise and Fine-Tune the Measures and Process

Lessons learned from the implementation process led to new discussions and a few changes. It became apparent early on that patients with dementia or cognitive impairment should not be required to complete the MBC battery, as often the data was unreliable, of questionable validity, and often completed by caregivers, or the technology was frustrating. After receiving feedback from providers that the initial new patient battery was taking longer than expected for many patients, we reviewed the data for several of the “add-on” questionnaires to determine how to revise. Based on data suggesting that most items included on the attention deficit hyperactivity disorder (ADHD) scale were positive across diagnoses and not specific to the diagnosis of ADHD, we removed this measure from the initial visit and included it only in established visits where a diagnosis of ADHD was listed on the problem list. In order to further reduce the time required, we followed the same process for the Obsessive-compulsive disorder (OCD) measure and the Post-traumatic stress disorder (PTSD) measure (see Table 2). These revisions were a compromise to reduce the time required at the initial screening visit, but still offer the measures for follow-up care when the measure was relevant to the diagnosis and treatment plan, resulting in positive patient and clinician feedback.

Add to the MBC Platform for Subspecialty Care

Having worked out the delivery process for the standardized MBC platform in the general outpatient clinic, specialty clinicians requested to add additional measures for specific needs. The development of a treatment resistant depression (TRD) Interventional Psychiatry program led to the inclusion of an additional measure of depression. The team requested that we add the Quick Inventory of Depression Symptomatology – Self Report (QIDS-SR)¹⁶ to TMS and esketamine procedure visits to further document treatment outcomes which could be utilized for supporting the payer authorization process for these specialty treatments. Because the delivery of measures is tied to visit types, ie, all screening measures populate the tablet any time a new patient visit type is started and specific measures are added when the established visit types are initiated, it was easy to link these specific procedure visits to a new measure. As a further example, the psychotherapy team who run the group for cognitive behavioral therapy (CBT) for Insomnia asked about adding specific sleep measures to assist with the group curriculum. Attaching these measures to the visit type group for this specific clinician, allows for these measures to be sent out only when a patient is scheduled for this group and provider.

Adapting to Tele Psychiatry

We frequently revisited the option of sending measures in advance via secure email in our EMR along with current pre-visit forms, however, given the low completion rates of these forms we decided to maintain the system of an early arrival time for collecting the MBC battery to ensure a higher rate of completion. However, due to the COVID-19 Pandemic, in March 2020 we had a rapid shift to virtual/video visits with an increase from just a handful of telehealth visits per week to close to over 800 per week, in the span of around 5–7 days. Since no patients were being seen in person initially, we could not use our tablets to collect the data resulting in the termination of MBC temporarily.

Our clinicians immediately asked that MBC be integrated into the virtual visits highlighting how much MBC had become a part of their routine practice. Care seemed incomplete without the measures. We had a new mandate, this time from the providers themselves, to find a way to collect the measures on all patient visits. We began collecting the PHQ-9 and GAD-7 via secure EMR email after making the shift to video visits in April 2020 and are working currently to begin telehealth delivery of the entire MBC battery according to the same rules as our in clinic tablet procedure, and standardize collection across both in-clinic and remote care.

Dealing with Suicidal Risk in the Clinic and Remotely

Since responses indicating suicidal risk are vitally important, we built an alert system that would flash on the screen when the provider opened the patient’s chart if there is endorsement of this construct within the battery (Table 3). The red alert warning says, “your patient responded positively to suicide risk.” In order to close the box, the provider is required to enter a phrase acknowledging the message and attesting to making a safety plan.

Table 3 Suicide Risk Items: Suicide Items That Trigger the Red Box Alert “Your Patient is Responding Positively to Suicide Risk” for the Provider When the Chart is Opened, Therefore Requiring Timely Assessment

When completing measures remotely, we were concerned about sending questionnaires via secure email prior to sessions as a patient might answer questions indicating suicide risk, but the data could sit unnoticed in the EMR and not be assessed in a timely manner. We decided to implement the following process to mitigate risk related to the suicide items. Rather than following the health systems standard policy of sending electronic questionnaires 7 days prior to an upcoming visit, we sent our measures out to patients only 24 hours in advance. By delaying the assessment questionnaires until 24 hours prior to the visit with the clinician, along with the pop-up alert highlighting suicide risk, we hoped to insure timely assessment and intervention. If patients screen positive for suicide risk but are a no-show at either the in-person or virtual visit, our policy is to call the patient immediately and pursue a welfare check if we cannot reach them.

Challenges and Future Directions

MBC has become a valued part of our practice, but we continue to face challenges and questions. For example, we quickly learned that our core measures did not fit the needs of certain patient populations. Our addiction team needed measures specific to relapse prevention; our CBT-Insomnia group needed sleep hygiene scales as well as a sleep diary. While some flexibility exists based on provider and visit type, issues of standardization and questionnaire fatigue are a concern. The wide adoption of telehealth, prompted by the COVID-19 pandemic, allowed us to explore ad hoc delivery of provider-assigned questionnaires specific to particular treatment plans or groups such as our eating disorder patients. We are currently refining a new process that will allow our therapists to manually send specific measures aimed at a more individualized treatment focus rather than being automatically triggered in a more global fashion. These issues bring up the question of how to effectively use MBC to genuinely personalize care.

Another challenge was how best to share scores with patients graphically during telehealth visits. Our EMR currently can only display item by item scores directly to patients. Providers report that actively sharing the scores with patients during visits results in increased patient engagement in the visit and with the MBC process, as well as, enhanced clinical conversation between patient and provider about symptoms and treatment planning. We are continuing to explore the best ways to similarly share the MBC scores and changes during telehealth visits and by secure mail within our EMR for patients to later reference. This brings up the question of how best to use MBC to engage patients in self-monitoring and management beyond the care visit. The use of biomedical informatics data embedded in smartphone applications which allow the patient to self-monitor health and well-being between visits with their provider offers an enhanced opportunity to get even more out of MBC in future, for example.³⁶

Another issue we have faced is how best to address patients who persistently report high scores. Some of these patients are refractory to treatment, while others appear to over-report. In both cases, symptomatic measures do not always capture clinical change. For example, we see many complex patients who may be much more functional and at much less risk for suicide, but their core scores remain high and unchanged. Brief scales such as the Mini-Quality of Life and Enjoyment (Mini-Q-LES-Q)³⁷ to assess quality of life or the Work and Social Adjustment Scale (WSAS)³⁸ may be more helpful in such cases.

MBC generates a wealth of data which provides opportunities for quality improvement and other clinical research projects. By Fall of 2020, over 400,000 questionnaires on over 17,000 patients had been collected across numerous diagnoses and multiple clinicians. To promote the development of research questions and database inquiries, we obtained UT Southwestern Institutional Review Board (IRB) approval and exported the measures into a de-identified database which is accessible to clinicians in support of clinical research activities. We have encouraged interested faculty to use this data and have offered mentorship opportunities. MBC offers rich ground for research creativity and quality outcome projects. For clinicians who use the data for providing patient-centered care, the database offers an opportunity to engage in clinical research in a way they might not have previously.

Conclusion

Using MBC in routine psychiatric practice was implemented and adopted by providers in our clinic. Including and engaging providers was crucial to creating a process that complimented our clinic operations and adapting the process based on both patient and provider experience over time contributed to successful integration of MBC into patient care in a meaningful way. We have found that despite growing pains and continued challenges, MBC now seems like a natural part of our practice. Our clinicians demanded we make it part of telehealth during the COVID-19 pandemic, and we are excited by the research possibilities offered by the wealth of data we have collected. We hope this enthusiasm inspires others to integrate MBC into their clinical care.