Addiction Medicine Practice-Based Research Network (AMNet): Assessment Tools and Quality Measures

Background/Rationale

The opioid epidemic in the United States (US) led to the overdose death of nearly 450,000 people from 1999 to 2018.¹ Adverse outcomes attributable to opioid misuse impart significant personal, social, and economic burden in the US.^1–3 Despite the enormity of this burden, however, the majority of the 1.6 million Americans with opioid use disorder (OUD) are left untreated.⁴

One approach to addressing the country’s opioid epidemic is the systematic implementation of measurement-based care (MBC) in substance use disorder (SUD) treatment. MBC refers to the use of standardized patient-reported outcome measures (PROMs) to identify and monitor patients’ symptoms and functioning and to adjust treatment to maximize improvements in patient outcomes.⁵ MBC can also facilitate the detection of emerging symptoms and comorbid diagnoses, which can impact the prognosis of SUD.⁶ MBC can be considered a “framework” since it can be implemented with many different treatment modalities or approaches across different settings and patient populations, and as such, can be considered a core component of numerous evidence-based practices.^7,8 Therefore, the opioid epidemic and the principles of treatment identified by payers and funders provide a strong rationale for MBC in the treatment of people with SUD, including OUD.^6,9–12 The combined use of MBC and quality measures to inform treatment planning in evidence-based treatment has been shown to improve the quality of care and outcomes for patients with SUD.^6,7

Amid growing interest and momentum in value-based care in the US, clinicians are increasingly expected to measure and track outcomes to demonstrate the value of their service at the patient and population levels.¹³ Further, this demonstration of value should include input from patients who are informed and selective about their healthcare options.¹⁴ The systematic uptake of MBC and the assessment of performance via quality measures can help clinicians meet these expectations. In addition, a quality improvement program that includes systematic uptake of MBC can help ensure patients receive the same level of high-quality, evidence-based care regardless of the treatment setting. However, many clinicians do not consistently implement MBC or a quality measurement system in their practices, thus missing a potential opportunity to improve patient outcomes.⁵

SUD treatment is provided in specialized SUD treatment programs and outpatient practices. While most clinical research has been conducted in specialized programs, particularly opioid treatment programs (OTP), little is known about the patients and treatment outcomes in other outpatient practice settings in which buprenorphine and naltrexone are provided for OUD. Further, the clinicians in non-OTP outpatient settings are rarely engaged in clinical SUD research, which impedes the translation of research findings to clinical practice. The lack of implementation of MBC and quality measurement in OUD treatment, uptake of evidence-based treatment, and insufficient research occurring in outpatient practices create an opportunity to address the OUD epidemic.

The American Psychiatric Association (APA), American Society of Addiction Medicine (ASAM), and Friends Research Institute (FRI) have partnered to develop the Addiction Medicine Practice-based Research Network (AMNet). AMNet is a novel practice-based research network that aims to develop a repository of information on addiction treatment and outcomes and facilitate performance improvement in addiction medicine and addiction psychiatry.¹⁵ This network is focused on the systematic uptake of MBC and quality metrics in non-OTP outpatient practices that treat SUDs. In the future, it will also provide a platform for practice-based research using standardized assessments and other data in a robust database. AMNet has adapted the APA’s mental health patient registry, PsychPRO, for SUDs through the incorporation of addiction-specific assessments and quality measures.¹⁶ PsychPRO provides an electronic application for efficient implementation of MBC and serves as a robust quality improvement tool, including the technological platform for quality data aggregation and reporting. A concerted multi-stakeholder effort has been used to guide the selection of addiction-focused PROMs and quality measures with a focus on OUD. The purpose of the present paper is to describe the process by which PROMS and quality of care performance measures were selected for AMNet following a scoping review and to discuss their potential clinical utility.

Methods

The AMNet Executive and Steering Committees used a multi-step, consensus-based process to identify and select OUD and other SUD standardized assessment tools and quality measures. These measures will facilitate the implementation of MBC and quality improvement for AMNet participants in non-OTP outpatient treatment settings, including office-based opioid treatment practices, certified intensive outpatient programs, outpatient programs, and community health and mental health centers. The Executive Committee comprises the investigators from APA, ASAM, and FRI as well as the AMNet grant’s Science Officer at the National Institute on Drug Abuse (NIDA). The Steering Committee consists of the members of the Executive Committee plus three clinicians, two federal advisors, and one consumer representative.

The Committees followed a multi-step process as illustrated in Figure 1 and involved the following steps: 1) identify domains of interest for OUD and other SUDs; 2) search for assessment tools and quality measures using the domains of interest and other keywords related to substance use (ie, use, misuse, abuse, dependence, disorder); 3) initial review of the tools and quality measures; 4) final review and consensus-based selection of tools and quality measures; and 5) categorize tools as core versus optional. We delineate this process in the following sections.

Figure 1 AMNet Standardized Assessment Tool Selection Flowchart. Abbreviations: *BAM, Brief Addiction Monitor ; COWS, Clinical Opiate Withdrawal Scale; TEA, Treatment Effectiveness Assessment; **C-SSRS, Columbia-Suicide Severity Rating Scale.

AMNet Standardized Assessment Tools

Identify Domains of Interest for Assessment Tools

The AMNet Executive Committee sought domains that would 1) be of interest to treating clinicians and patients with OUD/SUD; and 2) have a potential clinical relevance to OUD/SUD and its treatment. Committee members proposed domains, discussed their importance in the field of addiction medicine, and reached a unanimous agreement on the final domains of interest during a regularly scheduled meeting. The Executive Committee, drawing on its clinical and research expertise in addiction medicine/psychiatry and public health, identified ten domains of interest for the AMNet assessment tools: 1) substance consumption/frequency, quantity, and mode of administration; 2) withdrawal; 3) insomnia; 4) craving; 5) overdose; 6) prescription opioid misuse; 7) quality of life; 8) recovery; 9) functioning; and 10) mental health. The domains of consumption and prescription opioid misuse were selected to characterize the opioid use pattern. The domains of withdrawal, insomnia, craving, and overdose were selected as effects and consequences of interest. The domains of quality of life, recovery, functioning, and mental health were considered to be important to assess both the current state of the patient and any improvements with treatment. Although the Executive Committee focused its discussions on domains that would be of great use to clincians, the Committee noted many of these domains would be of interest to patients (such as withdrawal, insomnia, and quality of life) and the addiction field (such as craving) as they are considered important endpoints for demonstrating the efficacy of drugs for the treatment of OUD by the Food and Drug Administration.¹⁷

Search for Candidate Assessment Tools

A subcommittee of the Executive Committee conducted a scoping review to identify candidate patient- and clinician-rated assessment tools related to OUD and other SUDs. The subcommittee searched PubMed and PsycINFO databases for records published through November 2019 as well as the online catalogs of PROMs, including the National Institutes of Health Patient-Reported Outcomes Measurement Information System (PROMIS), the PhenX Toolkit, and NIDA’s Clinical Trials Network Common Data Elements.^18–21 Search terms were related to: all dimensions of substance use (ie, use OR misuse OR disorder OR abuse OR dependence); the domains of interest identified in Step 1; assessment (ie, measure OR tool OR instrument OR screen); and—for specifically finding tools for OUD—terms related to the substance (opioid OR opiate). The inclusion criteria were as follows: 1) focused on assessment tools used for SUD or OUD; 2) provided information on scoring and interpretation; 3) reported psychometric properties of tools or described tools derived from national surveys; and 4) freely available or in the public domain. The exclusion criteria were as follows: 1) letters, editorials, commentaries, posters, and conference proceedings; and 2) articles not published in the English language. Records from the peer-reviewed literature were screened in two stages—first on titles/abstracts and then full-text. The reference lists of applicable articles were hand-searched for additional resources. Using this approach, the subcommittee located 45 candidate assessment tools.

Initial Review of Candidate Assessment Tools

The Executive Committee conducted an initial review of the 45 tools found in Step 2 against four criteria: 1) brief enough to be a low burden to patients and practices; 2) clinically useful (ie, provide information that is useful in clinical decision-making for both the clinician and the patient); 3) in the public domain; and 4) possessing acceptable or greater reliability (ie, any combination of internal consistency, test-retest, or inter-rater reliability) and validity (ie, any combination of face, criterion, concurrent, or discriminant validity). The Executive Committee sought to identify preferred candidate tools for each of the ten domains of interest identified in Step 1. The Executive Committee shared the list of 45 tools with preferences with the Steering Committee at its face-to-face meeting on December 16, 2019, for consideration and final consensus selection.

Final Review and Consensus-Based Selection of Assessment Tools

The Executive Committee presented the Steering Committee with an overview of the 45 candidate assessment tools as well as its review process and deliberations (Step 3) to arrive at the 13 preferred candidate tools. The Steering Committee selected the final AMNet assessment tools (Table 1) by consensus. This involved at least 7 of 12 Steering Committee members voting “yes” to include the assessment tool. To eliminate redundancies and reduce patient burden, the Steering Committee suggested replacing four of the tools recommended by the Executive Committee with alternate tools that captured multiple domains. Specifically, the Steering Committee suggested using the 17-item Brief Addiction Monitor (BAM) to replace four other candidate tools—PhenX Quantity/Frequency Alcohol & Drug, PROMIS Sleep Disturbance & Sleep-related Impairments, Brief WHO-Quality of Life (WHOQOL-BREF), and the Brief Assessment of Recovery Capital (BARC-10)—to capture substance consumption/frequency, quantity, and mode; insomnia; quality of life; functioning; and recovery for patients with OUD and other SUDs.^{18,19,22–24} Also, the Steering Committee suggested the Treatment Effectiveness Assessment (TEA) replace the WHOQOL-BREF and the BARC.²⁵ According to the developers of the TEA, the tool provides a brief (ie, four items), patient-centered, clinically useful, and internally reliable measure of treatment progress and outcome for patients with SUD.²⁵ The TEA “directly assesses the patient’s personal perspectives on substance use, personal health, lifestyle, and responsibility to the community,” attesting to its patient-centeredness because it captures “common elements of recovery.”²⁵ During the meeting, the APA’s Principal Investigator recommended the Steering Committee discuss the relationship between suicide and suicide attempts and addictions, including OUD and alcohol use disorder (AUD).²⁶ Specifically, studies have found that AUD and OUD are two SUDs that are strongly associated with increased risk for suicide ideation, suicide attempt, and death by suicide.²⁷ As such, the Steering Committee approved the inclusion of the Screen Version of the Columbia-Suicide Severity Rating Scale (C-SSRS).²⁸ The final selected battery includes assessment tools that are applicable to all SUDs (eg, PhenX Injection Drug Use, BAM, TAPS Tool) and some that are specific to OUD (eg, SOWS and COWS). This combination of general SUD and OUD-specific tools were selected because OUD tends to co-occur with other SUDs, such as AUD. Although tools for the domains of overdose and prescription opioid misuse were identified and reviewed, they focused only on scales to assist prescribers of opioid analgesics and were, therefore, not included in the final battery for AMNet clinicians who are treating OUD, and other SUDs, and are not likely to be prescribing opioids for analgesia.

Table 1 AMNet Assessment Tools

Categorize Assessment Tools

To minimize burden, the Executive and Steering Committees limited the number of assessment tools recommended for use with all patients. The final list of assessment tools was categorized as either core/Tier 1 or optional/Tier 2. For Tier 1, the Committees wanted tools that directly assess and monitor SUD and corresponding treatment and would, therefore, be useful for all patients being treated for SUD. The Tier 2 tools assess relevant co-morbidities, outcomes associated with substance use (eg, impaired functioning), and a supplemental clinician-rated opioid withdrawal scale. These tools were considered optional for appropriate patients at the clinicians' discretion.

AMNet Quality of Care Performance Measures

Identify Domains of Interest for Quality of Care Performance Measures

The Executive Committee identified three domains of interest for selecting AMNet quality of care performance measures: initiation, adherence, and retention. These domains were selected because they are pertinent to OUD as well as AUD treatment across populations and treatment settings. A measure focused on the initiation of treatment was considered essential because pharmacotherapy is underutilized in non-OTP outpatient opioid treatment, especially in rural areas.²⁹ Domains related to follow-up—adherence and retention—were selected because studies show continuity of care leads to better outcomes.^30,31 Further, the Executive Committee considered AMNet clinicians would find quality measures in these domains aligned with their treatment practices and, therefore, be likely to use them to drive improvements in quality of care for OUD over time.

Search for Candidate Quality Measures

Following a process similar to that for the assessment tools, the subcommittee conducted a scoping review of quality measures for OUD treatment to implement in AMNet. The subcommittee searched PubMed and PsycINFO databases for records published through November 2019 using the terms “opioid” OR “opiate” and terms related to quality measure (quality measure OR performance measure OR quality indicator OR performance indicator). The subcommittee used the same terms to also search the online directories of the National Quality Forum, Agency for Healthcare Research and Quality, National Committee for Quality Assurance, and the Physician Consortium for Performance Improvement.^32–35 Measures were included if they 1) focused on OUD, OUD treatment, or use of prescription opioids; and 2) contained detailed specifications (eg, precisely defined numerator and denominator). Using these parameters, the subcommittee located 25 candidate quality measures.

Initial Review of Candidate Quality Measures

The Executive Committee conducted an initial review of the 25 candidate quality measures to examine suitability for AMNet (Figure 2). Desirable characteristics of quality measures for AMNet included being: 1) extractable from electronic health records into PsychPRO; 2) in use in the field of addiction medicine; and 3) associated with favorable treatment outcomes. In consideration of each quality measure against these review criteria and to ensure coverage of the prioritized domains, the subcommittee selected 7 candidate measures for AMNet. Five of the measures were developed by the Center for Care Innovation (CCI), one measure by ASAM, and one measure by the Physician Consortium for Performance Improvement.³⁶ The CCI measures evaluated treatment initiation, engagement, retention, and toxicological monitoring. These measures, along with ASAM’s Standard of Care measure quantifying the prescription of medication for OUD among patients with an OUD diagnosis, were selected by the Executive Committee as they are associated with positive patient outcomes and can be easily extracted from the medical record.³⁷ The Physicians Consortium for Performance Improvement measure, pertaining to counseling on treatment types for those with OUD, takes into consideration both pharmacological and psychosocial treatment and was chosen by the Executive Committee due to its comprehensive assessment of treatment.³⁸ Although not selected for recommendation to the Steering Committee due to their lack of generalizability to individuals with OUD, the Executive Committee took into consideration measures from Mathematica Policy Research (Medicaid beneficiaries receiving buprenorphine-alone or in combination with naloxone) and Pharmacy Quality Alliance (concurrent use of prescription opioids and benzodiazepines).^39,40

Figure 2 AMNet Quality Measure Selection Flowchart. Abbreviations: *OUD, opioid use disorder; **CCI, Center for Care Innovations.

Final Review and Consensus-Based Selection of Quality Measures

The Executive Committee presented the Steering Committee with an overview of the 25 candidate quality measures (Step 2) and the rationale for the 7 selected by its subcommittee (Step 3). As with the selection of the AMNet assessments, a consensus-based approach was used by the Steering Committee to select the quality measures. The CCI Initiation, Engagement, and Retention measures were chosen because of their face validity and the ease with which they could be extracted from medical records. Specifically, the initiation and engagement measures were selected because patient monitoring is particularly important early in treatment when there is a high risk of drop-out. The retention measure was selected because of its association with positive patient outcomes. The Steering Committee also noted that the CCI Initiation and Engagement measures are an OUD-specific adaptation of the general Healthcare Effectiveness Data and Information Set (HEDIS) measures on initiation and engagement and, thus, would be suitable for inclusion in the AMNet battery.

Results

AMNet Standardized Assessment Tools

AMNet assessment tools were selected using a consensus-based process (Table 1). The Tier 1 measures include the Tobacco, Alcohol, Prescription Medications, and other Substances (TAPS) Tool, Brief Addiction Monitor (BAM), PhenX Cigarette Smoking Status, PhenX Injection Drug Use, Treatment Effectiveness Assessment (TEA), Short Opiate Withdrawal Scale (SOWS), single-item craving Visual Analog Scale (VAS), and the Patient Health Questionnaire (PHQ)-2+1. The Tier 2 measures include the Clinical Opiate Withdrawal Scale (COWS), PHQ-9, Columbia-Suicide and Severity Rating Scale (C-SSRS), and the PROMIS Pain Interference Instrument. The SOWS and COWS are the only AMNet assessment tools limited to patients with OUD. The VAS for craving, although predominately used for OUD, can also be used for other SUDs. These assessment tools, when used as a MBC framework, are useful for guiding clinical care and tracking and improving patient outcomes in a number of domains of interest to clinicians and patients alike. In addition, they provide a common set of standardized data that can be collected with routine clinical care in support of delivery of evidence-based treatment and quality measurement and improvement. In support of this MBC framework, a detailed resource guide was developed to help clinicians implement the practice, including suggested timing of baseline or initial and follow-up assessments.

AMNet Quality Measures

The consensus-based process resulted in the selection of three quality of care performance process measures that met the review criteria of being extractable from electronic health records, currently in use in addiction medicine practice settings, and associated with desired treatment outcomes. These quality of care performance measures were developed by the CCI and align with the prioritized domains of initiation, engagement, and retention in treatment:

1. Percent of patients with one follow-up visit within 14 days of starting OUD pharmacotherapy [CCI Initiation];
2. Percent of patients with two follow-up visits within 30 days of the date of the initial prescription for OUD pharmacotherapy [CCI Engagement];
3. Percent of patients prescribed OUD pharmacotherapy six months prior who have adhered to this medication continuously for six consecutive months. Continuous adherence will be measured by identifying patients who have attended practice visits and continued with medication refills for six months without interruption [CCI Retention].

Discussion

The US opioid epidemic and public health consequences of SUD underscore the need to improve the outcomes of patients in SUD treatment. SUD treatment, including OUD treatment, is routinely provided in outpatient practices outside of the specialty treatment system. Yet, little is known about the outcomes of patients treated in such practices. The Addiction Medicine Practice-based Research Network (AMNet) was created to address this issue by improving the implementation of MBC via standardized patient-reported assessment tools and quality measures while also serving as a research platform to examine effective treatments and implementation.

A key step in the development of AMNet was the selection of standardized assessment tools and quality measures for incorporation into the APA’s qualified clinical data registry, PsychPRO. A stakeholder group comprising researchers, clinicians, federal advisors, and a consumer representative followed a multi-step, consensus-based process to select the assessment tools and quality measures, with a focus on those most relevant to patients with OUD. The process resulted in the selection of 12 assessment tools and 3 quality measures. The selected assessment tools cover numerous domains pertinent to patients with SUD, such as substance consumption, withdrawal, insomnia, craving, pain, depressive symptoms, suicidal ideation and behavior, and recovery. The selected quality measures are companion measures for treatment initiation, engagement, and retention. By offering an array of assessment tools and quality measures and incorporating them into PsychPRO, AMNet will facilitate participating clinicians' systematic uptake of MBC and delivery of evidence-based treatment for patients with SUD.

MBC can be clinically useful to patients and clinicians by improving the quality of care and patient outcomes.^8,41,42 MBC can help patients report, understand, and track their progress over time by validly quantifying symptoms and other domains of interest. In turn, MBC can help clinicians meaningfully track individual patient progress while facilitating comparison to group averages or norms. Clinicians can also use MBC in conjunction with information obtained in clinical interviews to recognize and assess treatment needs, inform clinical decision-making, and explore factors associated with change or progress. Further, at the population or practice level, MBC can improve the programmatic efficiency of care by adjusting the course, frequency, or duration of encounters according to group need and anticipated benefit.⁴³

Specifically, MBC can be used to personalize and adapt OUD treatment to increase patient engagement, retention, and outcome.⁶ For example, administering the BAM, which covers important substance use-related behaviors, could be used at the beginning of treatment to determine the severity of a patient’s problem and at subsequent time points to chart patient status over time.²² Further, the craving symptom can be an important and actionable item for MBC as distressing craving experiences may trigger relapse and persist long into recovery. Using an assessment tool to assess and track craving over time can prompt a discussion that could help build therapeutic alliance, increase change motivation and interest to engage with treatment.⁶

Quality of care performance process measures, on the other hand, provide a means to aggregate and review data to assess the delivery of healthcare and are especially useful when processes can be linked to improved outcomes. Initiation, as well as engagement and retention in evidence-based treatment interventions, are important to reduce OUD and other SUD. However, only 18.1% of individuals with OUD and 7.6% of those with AUD receive evidence-based treatment each year, and even fewer are retained in treatment.^4,44,45 AMNet aims to address this gap by incorporating quality measures for each of these domains. Specifically, PsychPRO’s quality dashboard has been adapted for AMNet by including the CCI quality measures for initiation, engagement, and retention in OUD treatment. This is an important step for augmenting MBC in non-OTP outpatient practices because quality of care performance measures play a crucial role in guiding quality improvement efforts for the delivery of effective care and, subsequently, for improving patient outcomes.²⁰ Therefore, access to CCI quality of care performance measures, which can also be adapted to include patients with AUD, in the PsychPRO dashboard will allow AMNet clinicians to use their performance information for quality improvement and to guide care over time.³⁷ Underscoring the benefits of including the CCI quality measures in AMNet are its potential links to improved patient outcomes. Medication initiation represents a benchmark in the recovery process for patients. Once medication is initiated, it is important to keep the patient engaged in treatment to monitor their experience and make modifications if necessary, such as titrating up or down or addressing any side effects.⁴⁶ Engagement and retention in treatment are necessary to achieve optimal patient outcomes, including decreased involvement with the criminal justice system, improved employment outcomes, and lower substance use.³⁸ Regarding OUD, retention on medication for OUD, particularly methadone and buprenorphine, is associated with decreased mortality both from overdose and all-cause mortality, with the risk of overdose increasing significantly after discontinuation of medication.⁴⁷

The development of other networks has also demonstrated the need for careful selection of assessments to identify common standardized data to address important clinical questions for improvement of patient outcomes. This function for AMNet is expected in the future, through its use of the APA’s PsychPRO Registry to leverage patient data from routine care or the implementation of MBC. For example, NIDA’s Clinical Trials Network, which was established in 1999 to expedite the translation of evidence-based substance use treatment into community-based practice, focused on selecting assessments that are useful to researchers and feasible to implement in busy practice settings. The network recognized the need for a common assessment battery to standardize data collection and formed a committee of researchers and clinicians  to select assessments for this battery.⁴⁸ The process for selecting tools and measures for AMNet reflects the importance the field places on the involvement of multiple stakeholder groups in developing network projects and on measure standardization and its implications for research and, ultimately, clinical practice.

Given challenges to implementing MBC in practice (eg, time restrictions, material resource limitations, patient complexity), AMNet has planned feasibility testing of the selected tools and measures. The feasibility testing will entail the collection of quantitative and qualitative data from pilot test sites to explore data capture, clinical utility, workflow impact, and the potential burden on clincians and patients. These data will be used to increase uptake of the tools and measures by AMNet’s participating clincians while minimizing disruption to workflows and burden.

Limitations

The current review has several limitations. First, we are uncertain whether the use of a systematic review methodology, as opposed to a scoping review with Steering Committee consensus decisions, would have produced different results in terms of the assessment tools and quality of care performance measures identified and included in AMNet. Although it is possible that additional tools or measures could have been located with a systematic review covering more databases, our approach was informed by an a priori protocol with a wide search strategy and decision-making steps to increase reliability. Another limitation is that selected assessment tools must have been freely available or in the public domain to avoid copyright issues, or could be obtained from the developers for use in AMNet at no cost. This selection criterion eliminated a number of assessment tools that are clinically useful, reliable and valid. Lastly, we selected English-language only articles and there may have been relevant articles not published in English.

Conclusion

Quality registries have not been harnessed previously to collect data in non-OTP outpatient treatment settings regarding OUD patients’ characteristics, treatments, outcomes, and quality care, or data used to examine comparative effectiveness of treatment modalities, other patient-centered outcomes research, or implementation studies. AMNet as a novel research network embedded within the infrastructure of a quality registry will provide a centralized repository including common data elements collected from patients and clinicians in non-OTP outpatient addiction medicine treatment practices. AMNet will serve as a network for enhancing the quality of care and for innovative opioid treatment research that ultimately underpins improved patient outcomes. AMNet can be envisioned as a “Quality Improvement community” that could, in the future, be funded to innovate measurement-based care and develop quality outcome measures in addiction medicine to benchmark and improve population outcomes.