Review of Clinical Outcome Assessments in Pediatric Attention-Deficit/Hyperactivity Disorder

Introduction

Attention-deficit/hyperactivity disorder (ADHD), a neurodevelopmental condition affecting both children and adults, comprises behavioral symptoms and attentional dysregulation (eg, inattentiveness, impulsivity, overactivity).¹ The prevalence of ADHD in the United States is about 6.1 million among children ages 2–17 years.² ADHD is a lifelong and impairing disorder, resulting in significant challenges to educational and vocational attainment as well as increased risks for negative physical, psychosocial, and financial outcomes, particularly when untreated.³

Although effective interventions for ADHD exist – including stimulant and nonstimulant medications and nonpharmacologic therapies^4,5 – evaluation of treatment response can be difficult because of the subjectivity of ADHD symptoms.⁶ Outcome measures in clinical trials for ADHD treatments are often rating instruments based on symptom criteria defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM).⁷ The symptoms of ADHD listed in both the Fourth and Fifth Editions (DSM-IV and DSM-5, respectively) are the same; changes between the 2 diagnostic editions were limited to additional examples of symptom manifestations, criteria for age of onset, and minor wording changes (eg, ADHD “types” in the DSM-IV are called “presentations” in the DSM-5).⁸ However, individuals with ADHD are typically referred for evaluation and treatment because of dysfunction in social, emotional, familial, academic, and/or occupational roles rather than ADHD symptoms alone.⁹ Therefore, it is necessary to evaluate treatment benefits beyond symptom reduction by using additional clinical outcome assessments (COAs).

COAs, as defined by the US Food & Drug Administration (FDA), are measures that “describe or reflect how a patient feels, functions, or survives.”¹⁰ COAs include outcome measures that are patient-reported, observer-reported, or clinician-reported as well as performance outcome measures.¹⁰ COA-based outcomes may be influenced by personal choices, judgment, and motivation.¹¹ They should measure treatment benefit on meaningful aspects of how a patient functions, feels, or survives in his/her life.¹¹ As such, the improvement of the concept(s) measured should be meaningful to the patient in his/her usual or typical life (outside of the clinical trial setting).¹¹

Despite a need to assess outcomes meaningful to pediatric patients with ADHD and their caregivers, there has not yet been a formal evaluation of the relevance of concepts measured by COAs commonly used within clinical trials in ADHD. Whether these COAs measure meaningful aspects of pediatric patients’ and their caregivers’ typical lives has also not been examined. In addition, the degree to which COAs used in pediatric ADHD clinical trials to support label claims capture the concepts important to patients has not been examined. The aim of this literature review is to: 1) identify concepts that are important to pediatric patients with ADHD and their caregivers; 2) identify COAs that have been used in pediatric ADHD research and explore the extent to which they align with the identified concepts; and 3) identify the COAs and their frequency of use in clinical trials, including those that have undergone regulatory review to support product label claims.

Methods

We performed separate literature reviews to address 3 questions:

1. What concepts should be assessed by pediatric ADHD COAs?
2. How well are these concepts measured by existing COAs used to assess ADHD symptoms and impacts in children/adolescents?
3. Which COAs are currently used in pediatric ADHD clinical trials, including those that support label claims and have been reviewed by regulatory bodies?

After applying the search strategy, article filtering was conducted in 2 stages: first, on a title/abstract level, followed by a full-text level review of articles retained after the first filtering. We extracted data on the population, methods, and patient- or caregiver-reported concepts from the full-text articles. Each article was analyzed by one reviewer, and the resulting data were assessed by a second reviewer. If any aspect of the data was unclear, both reviewers reassessed the article. For each of the concepts that required additional review, the reviewers discussed the concept until a consensus was reached.

Literature Review of ADHD Concepts

Two searches were conducted in PubMed to identify publications reporting qualitative research on the symptoms or impacts of ADHD in pediatric patients. Search terms for both searches included “quality of life” or “burden” to capture multiple aspects of the ADHD experience. The first search was conducted to assess the experience of ADHD from the perspective of the patient, using the search terms “ADD OR ADHD” AND “Qualitative OR interview OR focus group” AND “Quality of life OR burden.” Results were limited to human studies published in English from 2000 through 2018 and to patients ≤18 years of age. The second literature search explored the experience of ADHD from the viewpoint of the parent, caregiver, or teacher. This search included the terms “ADD OR ADHD” AND “Qualitative OR interview OR focus group” AND “Quality of life OR burden” AND “Parent OR caregiver OR teacher.” Results were limited to human studies published in English from 2000 through 2018.

Concepts reported by patients and their caregivers were analyzed and organized into a conceptual framework¹² to describe the experience of ADHD, including symptoms and impacts. The identified concepts were then compared with concepts from each pediatric ADHD COA. COAs were assessed to understand how well each captured the experience of ADHD.

The conceptual framework was first organized according to concepts underlying DSM diagnostic criteria for ADHD,¹ including: 1) predominantly inattentive; and 2) predominantly hyperactive/impulsive. Concepts that did not fit within DSM criteria were listed as additional behavioral symptoms or as specific to certain concepts within those categories. A final category of general functional impacts was also created to capture impacts that applied to ADHD more broadly.

Literature Review of Existing ADHD COAs and the Concepts They Measure

A search was conducted in PubMed to identify COAs used in ADHD research. Search terms included “Attention deficit disorder with hyperactivity” [MeSH Major Topic] AND “Patient-reported outcome [Title/Abstract] OR scale [Title/Abstract] OR instrument [Title/Abstract] OR questionnaire [Title/Abstract]” AND “Symptom OR impact OR quality of life.” We limited the search to articles in the English language published within the last 10 years. We extracted specific COA names from the abstracts and then assessed each COA for its relevance to pediatric ADHD. We also examined information about each COA’s development, psychometrics, and interpretation when available.

Literature Review of COAs Used in Clinical Trials and COAs Used to Support Labeling Claims

We conducted a search on ClinicalTrials.gov to identify and review COAs currently used in clinical trials for pediatric ADHD. Individual searches were conducted for each of the COAs identified through the literature review to document their use as outcome measures in clinical trials. We extracted the endpoints (primary, secondary, and exploratory) derived from these COAs.

To ascertain which COAs have been used to make label claims, we examined the medical reviews and product labels for pediatric ADHD medications approved in the past 10 years by the FDA or the European Medicines Agency (EMA). We assessed the reviews and labels of each medication for information about the use of COAs in clinical trials.

Results

ADHD Concepts from the Perspective of Pediatric Patients and Their Caregivers

The initial stage of the literature search yielded 130 articles. After excluding articles deemed to be out of scope, 5 articles were included in our analysis (Figure 1).

Figure 1 Literature review of ADHD concepts (patient perspective). Abbreviations: ADHD, attention-deficit/hyperactivity disorder; ICF, International Classification of Functioning, Disability, and Health; WHO, World Health Organization.

The second stage of the first literature search yielded 69 articles. After excluding articles that were out of scope or had incomplete information, 5 articles were included in our analysis (Figure 2).

Figure 2 Literature review of ADHD concepts (parent, caregiver, or teacher perspective). Abbreviations: ADHD, attention-deficit/hyperactivity disorder; ICF, International Classification of Functioning, Disability, and Health; WHO, World Health Organization.

Three articles were captured in both search strategies and reported perspectives from both the patients and caregivers (eg, both children and their teachers provided information), and thus these articles were included in both reviews. In all, 7 unique full-text articles were reviewed (Table 1). Studies were conducted across the United States, the United Kingdom, Canada, Puerto Rico, and South Africa.

Table 1 Full-Text Articles from Literature Review of ADHD Concepts

The conceptual framework was developed using the concepts identified from the literature searches (Table 2). The final conceptual framework comprised 4 areas: symptoms of ADHD as listed in the DSM, additional behavioral symptoms not in the DSM, specific impacts of ADHD symptoms (eg, task-level functioning), and general impacts of ADHD (eg, overall functioning in broad domains such as self-esteem and socialization).

Table 2 Conceptual Framework: Impacts on Pediatric Patients with ADHD

Existing ADHD COAs and the Concepts Assessed

A total of 483 articles were identified through the PubMed search for COAs. After excluding abstracts that were not relevant to pediatric ADHD or did not report the use of any COAs, 344 full-text articles were reviewed and 177 unique COAs were identified (Figure 3). After excluding those that were not relevant to the symptoms or impacts of pediatric ADHD, those that were not written in English, and those that could not be identified, 27 COAs remained. (See Supplemental Table 1 for a listing of identified measures.) Of the 27 relevant COAs, 12 were easily and fully accessible (ie, the full COA was available for review) and were mapped onto the conceptual framework.

Figure 3 Search of existing ADHD COAs. Abbreviations: ADHD, attention-deficit/hyperactivity disorder; COAs, clinical outcome assessments; PERF-O, performance outcome measures.

The identified COAs differed in their scope and content; some strictly assessed DSM criteria while others also covered additional behavioral symptoms and impacts.

COAs Evaluating DSM Symptoms

We identified 4 of 12 COAs that assessed only the 18 DSM symptoms of ADHD: ADHD Current Symptoms Scale (ADHD CSS),²⁰ ADHD Rating Scale–DSM-5 (ADHD-RS-5),²¹ the Adult ADHD Self-Report Scale–Adolescent Edition (ASRS-Adolescent),²² and the Strengths and Weaknesses of ADHD Symptoms and Normal Behavior (SWAN) Rating Scale.²³

ADHD Current Symptoms Scale (ADHD CSS)

The ADHD CSS,²⁰ also known as the Barkley Current Symptoms Scale, is both a self-report and parent-report measure of current ADHD symptoms. Each item provides 4 answer choices about experienced/observed behaviors: never or rarely, sometimes, often, and very often.

ADHD Rating Scale–DSM-5 (ADHD-RS-5)

The ADHD Rating Scale–DSM-IV (ADHD-RS-IV) and the updated version based on the DSM-5 (ADHD-RS-5)²¹ both measure the frequency of ADHD symptoms. Although the ADHD-RS-IV is more frequently cited in the literature than the ADHD-RS-5 because of the publication dates of the studies (ie, DSM-5 was released in 2013), the ADHD-RS-5 was used for this review because the purpose of the ADHD-RS-5 is to supersede the ADHD-RS-IV. The ADHD-RS-5 includes separate scales for children and adolescents in both the home (parent-report) and the school (teacher-report) setting. There are 4 answer choices for each item, ranging from 0 to 3. Depending on the phrasing of the question, these scores are described as never or rarely, sometimes, often, and very often, or as no problem, minor problem, moderate problem, and severe problem.

Adult ADHD Self-Report Scale–Adolescent Edition (ASRS-Adolescent)

The ASRS-Adolescent²² is a self-report measure of current ADHD symptoms. It is typically used in adults but has also been used in adolescent populations. Item responses include never, rarely, sometimes, often, and very often.

Strengths and Weaknesses of ADHD Symptoms and Normal Behavior (SWAN) Rating Scale

Lastly, the SWAN²³ is a self-, observer- (parent or teacher), or clinician-reported scale to assess ADHD symptom severity. Unlike the other 3 COAs that assess DSM symptoms, the SWAN uses positively worded scale items (eg, “Gives close attention to detail” rather than “Struggles to give attention to detail”). In addition to assessing DSM criteria, it also asks the rater to consider the child’s behavior in comparison with other children. Item responses include far below average, below average, slightly below average, average, slightly above average, above average, and far above average.

COAs Evaluating Additional Concepts

Eight of 12 measures assessed symptoms and impacts not included in the DSM. These COAs include items referring to specific behavioral symptoms and/or specific or general impacts related to the ADHD subtypes. Some of these measures also included items measuring symptom criteria from the DSM, to varying degrees. Where available, we collected information on the item generation process, item reduction process, classical test theory criteria, responsiveness criteria, and Rasch measurement theory criteria.

The ADHD-FX is a 32-item, observer-reported measure of the level of functional impairment a child experiences related to his or her ADHD.²⁴ Response options include not at all, a little, quite a bit, a lot, and do not know. As shown in Table 3, the scale assesses 3 DSM inattention symptoms and 3 hyperactivity/impulsivity symptoms as well as a few specific and general impacts. It has demonstrated adequate reliability and convergent and divergent validity.²⁵

Table 3 Mapping of Pediatric ADHD COAs Onto the Pediatric ADHD Conceptual Framework

The AIM-C is a parent-completed measure of the impact of ADHD on a child’s functioning and at-home quality of life.⁴² The scale is divided into 2 sections: Child Scale and Home Scale. The Child Scale consists of 8 items and measures the child’s well-being, including emotional functioning and school performance. The Home Scale has 10 items and measures the impact of the child’s ADHD on family life. The AIM-C also includes general questions regarding the child’s treatment history and other demographic information. Items were generated using patient input and literature reviews and were refined with additional patient and expert input. Mean scores have been shown to be significantly worse for children with ADHD-combined subtype than for those with ADHD-inattention subtype.²⁶ Items on the AIM-C have little conceptual coverage of ADHD symptoms as listed in the DSM (Table 3).

The DAYAS scale is a parent- or teacher-completed form (DAYAS-P and DAYAS-T, respectively) that measures the intensity of ADHD symptoms and problematic behaviors.²⁷ The measure consists of 17 (DAYAS-P) or 15 (DAYAS-T) items. Answer choices are not at all, just a little, pretty much, and very much. Information about the original item generation was not reported, but patient input was used to reduce the number of items included in the final scale. As shown in Table 3, it provides better coverage of DSM symptoms than the ADHD-FX and AIM-C but is not comprehensive. In addition to nondiagnostic behavioral symptoms, the DAYAS addresses impairment in social relationships but not in other general or specific areas.

The IRS is a parent- or teacher-completed measure of child functioning across multiple clinically relevant domains: peer, sibling, parent, teacher, academics, self-esteem, classroom/family, and global.²⁸ The IRS Parent consists of 8 items and the IRS Teacher has 7 items. Items are scored on a 7-point scale ranging from no problem (definitely does not need treatment or special services) to extreme problem (definitely needs treatment or special services). Each item on the IRS also includes a space for the respondent to provide an open-ended written answer. It is unclear what sources were used to generate the items on this scale. The IRS does not measure DSM symptoms or additional behavioral symptoms for either ADHD subtype. Items on the IRS assess only 1 specific impact of hyperactivity/impulsivity and 3 of the 4 identified general impacts of ADHD from the conceptual framework (Table 3).

The SKAMP is a 13-item observer-rated measure of the child’s impairment on various ADHD-related behaviors.^29–33 It is used in laboratory classrooms and scored by trained observers. Responses are given on a 7-point scale, ranging from 0 (no impairment) to 6 (maximal impairment). The SKAMP measures some but not all DSM symptoms as well as some nondiagnostic behaviors and general impacts identified in the conceptual framework (Table 3).

The SNAP-IV³⁴ measures the child’s problematic behavior. It is derived from the DSM-IV ADHD criteria and includes 10 questions assessing behaviors typical of oppositional defiant disorder (ODD), as the 2 disorders are often comorbid. Items on the scale were developed from the older SNAP questionnaire, with additional input from the Conners Index Questionnaire and the IOWA Conners Questionnaire. The scale must be completed by a parent or teacher and consists of 90 items in total. Item response choices include not at all, just a little, quite a bit, and very much. The SNAP-IV covers nearly all of the concepts included in the conceptual framework, including all DSM-specific symptoms for both ADHD subtypes, as well as many nondiagnostic behavioral symptoms and specific and general ADHD impacts (Table 3).

The VADRS is based on DSM-IV symptoms³⁵ and is used by teachers or parents to determine whether a child should be evaluated for a diagnosis of ADHD.³⁶ The latter half of the scale addresses the presence of ODD symptoms as well as symptoms of anxiety or depression. There are 45 items in total. The first 35 items are answered on a 4-point scale: never, occasionally, often, and very often. The remaining 10 items ask for a rating on a 5point scale, from problematic to above average. The VADRS includes all 18 DSM-5 criteria for both subtypes of ADHD, numerous nondiagnostic behavioral symptoms, one specific impact (ie, disruption in class), and several general impacts (Table 3).

Finally, the WFIRS measures a child’s functioning in a variety of domains, including family, school, social activity, and life skills.³⁷ Items on this parent- and self-report form were developed using a conceptual framework. There are 46 total items; item responses are never or not at all, sometimes or somewhat, often or much, very often or very much, and N/A (not applicable). The parent form may be used to differentiate 5-to-19-year-olds with and without ADHD based on functional impairment.³⁸ The WFIRS includes none of the DSM diagnostic criteria, nor does it focus on any nondiagnostic behavioral symptoms; rather, it includes items that address both the specific and general impacts listed in our conceptual framework (Table 3).

Although not part of our mapping to the pediatric ADHD conceptual framework due to lack of access to the full COA, the Conners 3³⁹ is a commonly used measure in pediatric ADHD clinical trials (Table 6). This COA assesses ADHD symptoms and common comorbid conditions (eg, full criteria for DSM-IV conduct disorder and ODD; screening questions for anxiety and depression) in children and adolescents ages 6–18 years. It includes all 18 items of both subtypes of DSM-IV ADHD. Subscales also provide assessments of ADHD-related problems or areas of concern, including learning difficulties, problems with executive functioning, aggression, and peer and family relations. Versions are available for parents (Conners 3-P), teachers (Conners 3-T), and the child self-reports (Conners 3-SR) and come in both long and short forms.

Table 4 Review of COAs in Clinical Trials

Table 5 COAs Cited in Labels

Table 6 Efficacy COAs in Pediatric ADHD Used in Clinical Trials and Regulatory Information

COAs Used in Clinical Trials and Regulatory Labels

We searched ClinicalTrials.gov for each of the 27 COAs identified through the PubMed review. Eighteen of these COAs were used in at least one clinical trial as a primary or secondary endpoint. We extracted data on each study, including a description of the endpoint measured by the COA (Table 4). We did not include information on the informant completing the COA (ie, self, parent, or teacher), as it was not consistently reported on ClinicalTrials.gov.

We reviewed product labels of pediatric ADHD products approved by the FDA. Table 5 summarizes the COAs used in clinical trials that support FDA-approved product labels.

Table 6 summarizes the COAs identified through our literature search that were also used in clinical trials (from ClinicalTrials.gov) and regulatory information.

Discussion

A number of COAs are available to assess symptoms and impacts of pediatric ADHD, and some have been used to assess treatment benefits in clinical trials. The need to measure treatment benefits meaningful to the patient in his/her normal life within clinical trials has been increasingly recognized.^4,9,11 However, there has not yet been a formal evaluation of whether the concepts measured by the various ADHD COAs are of relevance to pediatric patients with ADHD and/or their caregivers. In this review, we set out first to identify the conceptual areas of the ADHD experience important to pediatric patients and their caregivers based on existing qualitative research and then to evaluate the extent to which COAs used in pediatric ADHD research align with these identified concepts. Finally, we identified the subset of COAs that have been used in clinical trials, including those that have undergone regulatory review to support product label claims.

We found that most outcome measures used in ADHD studies are a reflection of DSM symptoms, a finding that extends those of previous literature reviews of outcome assessments in pediatric ADHD.^7,9 Epstein and Weiss reviewed measures used to assess treatment response in individuals with ADHD across the life span.⁹ Although findings were not stratified into adults versus children/adolescents, the review reported that most studies assessing ADHD treatment effects used DSM-based symptom measures. The investigators also proposed that additional domains, including functional impairment, quality of life, adaptive life skills, and executive functioning, should be evaluated to determine treatment effect.⁹ Another literature review corroborated that DSM-based rating scales were most frequently used to measure treatment outcomes.⁷

Although DSM criteria measure symptom severity, they may not assess the full range of symptoms and/or impacts important to patients and their caregivers. Hence, relying on DSM-based outcome measures may result in a failure to assess treatment benefit in ADHD comprehensively. The American Academy of Pediatrics recommends that patients’ and families’ perspectives be considered to sustain appropriate treatments and achieve long-term successful treatment outcomes.⁴ In contrast with previous literature reviews, our approach was to first determine symptoms and impacts important to the patient and caregiver (eg, teacher, parent) through a review of existing qualitative literature in order to construct a more comprehensive conceptual framework against which assessment tools could be mapped.

In addition to DSM-defined symptoms, we identified many patient-, parent-, and teacher-reported concepts that may be important to consider when evaluating treatment benefits in pediatric ADHD. For the inattention subtype category of ADHD, we identified 5 additional concepts: poor listening, poor planning, frequently procrastinating, needing reminders to perform activities, and poor performance on/neglecting to do tasks. For the hyperactivity and Impulsivity subtype category, there were 8 additionally identified concepts: easy to anger, frustration, difficulty regulating emotion, disrespectful/argumentative, clumsy/accident-prone, disruptions in class, difficulty getting to school on time, and difficulty settling down. Additional general impacts included poor academic performance, difficulty communicating, difficulty in social situations/poor relationships with others, and low self-confidence/self-esteem.

Our work on patient- and caregiver-reported ADHD concepts and the resulting conceptual framework based on these concepts (Table 1) is important because it elucidates the extent to which certain COAs accurately and thoroughly capture patient and caregiver experiences of ADHD, which may inform their potential use in clinical trials.

We found that some COAs were used as endpoints in the registration trials of currently marketed pediatric ADHD medications. Among DSM-based scales, the ADHD-RS-IV was most frequently used in pediatric ADHD clinical trials and was mentioned in 4 pediatric ADHD product labels. The SKAMP was also mentioned in many product labels (n = 6) and was used to assess outcomes in 31 clinical trials. The SKAMP may be preferred because it is a rating scale that is completed in a classroom setting;^30,31 ADHD medications are often designed to reduce symptoms during the school day or improve other activities that require focus and concentration that a parent may not be able to observe.⁴³ The SKAMP may have been used in clinical trials of ADHD medications, including registration trials, specifically to obtain evidence of these treatment benefits (ie, attention and behaviors in simulated academic settings, which may permit an understanding of time-course effects). The use of the ADHD-RS-IV or ADHD-RS-5 and the SKAMP in registration studies may suggest their acceptance by regulatory bodies.

Of the outcome measures we evaluated; the SNAP-IV appears to have the most comprehensive coverage of the concepts listed in our conceptual framework. The SNAP-IV not only assesses DSM symptoms but also most of the additional concepts identified in our framework (Table 3). Although the SNAP-IV was widely used in clinical trials (n=50), it has not been used in registration trials and it was not found in any product labels.

The FDA has published guidance on the use of COAs, which include patient- and observer-reported outcomes, as clinical trial efficacy endpoints to support labeling claims.^10,12 Whether some of the COAs evaluated in this review meet the criteria stated in the FDA guidance will need to be further examined.

Limitations

Our study is limited by the use of publications instead of collecting data directly from patients and their caregivers. For example, we were not able to assess how questions were asked or the environment in which the qualitative studies were conducted. Instead, we assumed the studies used to form our conceptual framework were conducted with good qualitative methods. Our study also did not include a review of the psychometric properties of each COA, but rather aimed at assessing content validity from a patient-centric perspective.

Conclusion

The need to measure treatment benefits meaningful to the patient in his/her usual or typical life has been increasingly recognized.^4,9,11 A number of COAs are available to assess symptoms and/or impacts of pediatric ADHD, but the concepts they measure vary. In this review, we set out to first identify a conceptual framework of the ADHD experience important to pediatric patients and their caregivers based on existing qualitative research. We mapped the existing ADHD COAs used in pediatric ADHD research to the conceptual framework and summarized the extent to which these COAs align with the identified concepts. The registration trials of currently available pediatric ADHD medications frequently used ADHD-RS-IV or ADHD-RS-5 to assess DSM symptoms and the SKAMP, administered in the classroom to assess symptoms and functional impairments during the school day. Future research is needed to understand the best way to measure all concepts important to patients and caregivers within clinical trials in ADHD.