Factors Influencing Long-Term Behavioral Intervention Outcomes in Preschool Children with Attention-Deficit Hyperactivity Disorder in Southeast China

Attention-deficit hyperactivity disorder (ADHD) is a childhood-onset neurodevelopmental disorder that is characterized by a persistent pattern of inattention, hyperactivity, and impulsivity¹ and is often accompanied by comorbid conditions such as autism spectrum disorder (ASD), oppositional defiant disorder (ODD) or conduct disorder (CD).² Cross-sectional studies with large samples have found reduced brain indicators among children with ADHD, with major differences in surface area measures.³ Multiple studies have shown that ADHD can result in impaired behavioral functioning across home and school settings and is associated with negative academic outcomes,^4,5 leading to dropout in severe cases.⁶ There are three subtypes of ADHD: inattention type (ADHD-I), hyperactive-impulsive type (ADHD-H), and combined type (ADHD-C). Recent studies suggest that ADHD diagnosed in childhood predicts the development of adverse childhood experience (ACE) and that the inattentive subtype may have the greatest impact on this risk.⁷ Children diagnosed with ADHD in preschool have more functional impairment than those not diagnosed with ADHD at preschool age.⁸

The purpose of ADHD treatment is to improve core symptoms and reduce behavioral problems. The current evidence-based treatment for ADHD involves the use of psychostimulant medications and behavioral interventions^9–11 and their combination. Psychostimulant medications are first-line medications for ADHD. Recent mechanistic studies have suggested that psychostimulants increase static FC and decrease dynamic FC (DFC), and DFC is associated with therapeutic effects.¹² However, drug treatments can lead to potential adverse side effects.¹³ Because behavioral interventions may improve outcomes, reduce treatment costs, and minimize side effects,¹⁴ they are the first treatment choice for preschool children.^15–17 The American Academy of Pediatrics (AAP) recommends behavioral parent training (BPT) as the first line of treatment for preschool-aged children with ADHD.¹³ China’s latest guidelines for ADHD diagnosis and treatment also emphasize the use of behavioral interventions in children under the age of 12 years.¹⁸

Parent training is designed to train parents to use behavioral or cognitive behavioral techniques to manage their children’s behavior. Parent-based behavior interventions have been shown to reduce disruptive behavior in children, and therapeutic effects persist after treatment completion.¹⁹ In preschoolers with ADHD, the goals of this training may include increased focus and compliance, as well as the management of hyperactive-impulsive behaviors through effective parenting. The programmes vary in style and content but are generally manual-based and may involve discussion and the use of videos and role play.²⁰ Reviews of the literature showed that parent behavioral training demonstrates the strongest evidence of efficacy for the treatment of preschool children with ADHD.¹⁹ Studies in the early phase of the project²¹ also confirmed that behavioral interventions show positive effects. However, another meta-analysis showed that the positive impact of parent training for ADHD ranged from no impact to strong.²² The AAP has noted that if a behavioral intervention does not lead to significant improvement, medication can be prescribed, but it is the secondary treatment for preschoolers.¹⁶ The national studies conducted after the AAP guidelines were published showed that approximately three-fourths of insured children aged 2–5 years received clinical care for ADHD and medication.²³ This suggests the need to study factors related to the effects of behavioral interventions to make better treatment decisions and to carry out personalized treatment. For children who are predicted to have poor efficacy of behavioral intervention, other treatment methods, such as drug therapy, should be given in time to reduce symptoms, prevent secondary problems and enable a better start to school.

At a research hospital in Southeast China, pediatricians, child mental health physicians and psychotherapists offered behavioral interventions to children with ADHD for a period of one year. Data were collected from the children before and after the behavioral interventions, and a case‒control study was conducted according to the effectiveness of the behavioral intervention. We hypothesized that the effectiveness of the behavioral interventions would be influenced by parent implementation skills and teacher cooperation. The first aim of this study was to analyze the factors that influence the effects of behavioral intervention. The second aim of the study was to predict the effect of the behavioral intervention.

Methods

Procedure

Preschool children diagnosed with ADHD received behavioral interventions that continued for 12 months.

Study Setting and Design

This case–control study was conducted from May 2020 to August 2021 in a tertiary health care facility in Southeast China.

Participants

Preschoolers with ADHD were recruited to participate in this behavioral intervention study. The inclusion criteria were as follows: preschool children first diagnosed with ADHD according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). None of the children had enrolled in an early intervention center or received treatment via medication or behavioral interventions.

The exclusion criteria were as follows: (a) a development quotient (DQ) score <75, (b) a history of seizures and/or taking medication to prevent seizures, (c) a childhood history or concurrent diagnosis of pervasive developmental disorder or organic mental disorders, and (d) placement in special education classrooms.

Group Situation

ADHD Rating Scale IV–parent version (ADHD-RS-IV): This scale has been validated as a useful instrument to assess ADHD symptoms among children and adolescents.²⁴ The response to ADHD treatment was defined as an improvement in the ADHD-RS-IV score of ≥30%.^25,26 In this study, the effective group was defined as children whose rates of decrease in ADHD-RS IV score and improvement in attention retention time were both greater than 30%; the ineffective group was defined as all other children.

Behavioral Intervention

The participating physicians designed the behavioral interventions according to the Chinese guidelines for the diagnosis and treatment of ADHD.¹⁸ The details were as follows: (a) Face-to-face parent training: Professional child psychologists offered weekly one-hour parent sessions for three months in the outpatient clinic. The participants were parents who had lived with a child with a confirmed diagnosis of ADHD for a long time and were responsible for the child’s primary and outpatient care. The purpose of the training was to help parents systematically understand ADHD, improve the parenting skills of parents and master the methods of family behavior management. Part of the course was background knowledge of ADHD, taking a close look at our problems, learning to like each other again, clear daily structures, making effective requests and not skimp on praise, learning how to play attentively, getting to know each other and exploring current externalizing symptoms of the child, focusing on positive experiences with the child, implementation of clear rules, social reinforcement and positive consequences of rules, examples of coping skills (eg, coping skills for reaction to stress and conflict with children) and interaction skill training for parents (eg, behavior management, self-regulation, improving organization, improving relationship, and use of encouragement). At the end of each session, parents received home assignments that were discussed at the beginning of the next session. Where parents failed to fully understand topics discussed before or experienced difficulties with techniques from particular sessions, an extra session was given. b) Attention training: After parent training, parents were trained at home under the guidance of physicians. Parents were trained to observe and record the children’s attention time at baseline and formulate an attention training goal (such as 10 minutes) based on the children’s symptoms and the results of the baseline scale assessment. Specifically, choose a quiet room in the family, children have a separate desk and chair, put the alarm clock on the children’s desk, and set to ring after 10 minutes. Parents arranged tasks that the child could perform independently at his or her age and told the child to perform independently within 10 minutes. Parents looked for signs of inattention, hyperactivity and other symptoms before the alarm. If not, it was recorded as achieving the training goal. The training was performed twice a day in two relatively fixed periods. Training was considered effective if the training target was met 90% of the time over a consecutive period. (c) Behavioral therapy: Behavioral therapy was provided at home by parents under the guidance of a physician. Parents and physicians worked together to develop and prioritize goals for behavior improvement. Parent-administered interventions attempt to enhance a parent’s understanding of child behavior management and the quality of parent‒child interactions, with the ultimate goal of optimizing the child’s developmental course. The behavioral training program included positive reinforcement, extinction and punishment methods:²⁷ 1) The positive reinforcement method involved strengthening good behavior through reward and other ways and giving positive reinforcement immediately when children had good behavior. 2) The extinction method: before treatment, it was necessary to determine what factors had a reinforcing effect on their children’s bad behaviors and reduce or eliminate some bad behaviors by planned neglect (stopping reinforcement). 3) The punishment method involved enforcing the appropriate consequences or punishment when the children failed to achieve the goal. In the process of behavioral intervention, the reward-outcome correspondence strategy was used until the completion of the task, and the expectation value of each task was gradually increased until the child’s behavior changed. The target behaviors were the change in ADHD symptom score after treatment.

Parents recorded the effect of children’s behavioral training in the form of daily report cards, implementation of token economy or points strategy, and cash points once every half month.

Teachers were also invited if conditions permitted. Behavioral intervention strategies in schools included priority seating arrangements, adjustments of class assignments, increased interaction/reminder, and classroom behavior management.²⁷ The physicians also trained the teachers on how to deal with emergencies at school and encouraged the children with ADHD to follow rules and stick to tasks.

Children were followed up every half a month in the first 3 months of the intervention and once a month thereafter. The outpatient interview follow-up lasted for 33.793 minutes on average (SD=2.610).

Assessment

(a) Attention retention timings: In a quiet environment, children were required to complete drawing, writing or learning assignments based on their age. The parents observed the children and noted their completion times.

(b) ADHD-RS-IV:²⁴ The higher the score, the more symptomatic the child. Su et al confirmed the reliability and validity of the Chinese scale in 2006.²⁸ The assessments were performed by trained experienced physicians.

(c) Swanson, Nolan, and Pelham (SNAP-IV) checklist: This is an assessment tool for ADHD.²⁹ Zhang et al²⁰ confirmed the reliability and validity of the Chinese scale for children before school age in 2016. Children were assessed by two parents, usually the father and mother, at the same time and averaged.

(d) Performance score for parents: Physicians evaluated the implementation of parental behavioral interventions during outpatient interviews. This study assigned a score of 1–3; higher scores indicated better parent implementation, which was assessed monthly for 12 months and expressed as a total score.

Data Collection

Information on the children was collected retrospectively from the medical record system. If missing data or unqualified data in logical examination were found, the treating physician was immediately contacted for verification and correction.

Sample Size

The sample size was calculated using PASS 15.0 software. Based on the results of earlier studies, we concluded that a moderate impact amount of 0.5 should be detected in the clinically assessed ADHD-RS-IV, with a one-sided 5% significance level and 85% power. At least 25 participants were required for both the effective group and the ineffective group.

Quality Control

The physicians who participated in the behavioral interventions and outcome assessment were trained to a uniform standard. We kept a complete list of encoding, security and storage plans, including any associated processes, to improve data quality. The data monitoring committee was composed of personnel with no competing interests who reviewed all data records and statistical results at the end of the intervention.

Statistical Analyses

Statistical analyses were performed using SPSS 26.0. Continuous variables were expressed as the means ± standard deviations (SDs) or as medians [interquartile range], and Student’s t-tests or Wilcoxon rank-sum tests were performed depending on the normality of the variables’ distributions, as appropriate. The categorical variables were expressed as n (%), and the chi-square test was performed for comparison. Odds ratios (ORs) and 95% confidence intervals (CIs) were calculated by multivariate analysis using backward logistic regression analysis. We used the Wald statistic, in which the variables with p≥0.15 were eliminated from the model one by one. Receiver operating characteristic (ROC) curves, sensitivity and specificity were utilized to assess the predictive performance. The level of significance was 0.05 for all analyses. Nomogram construction and C-index calculation were performed using R 4.0.5 software (http://www.R-project.org).

Results

During the recruitment period, a total of 98 preschool-age children with ADHD were enrolled, and 8 children were excluded (3 children with DQ scores <75, 1 child taking antiepileptic drugs, and 4 children in special education classrooms). After 12 months of the behavioral intervention, 4 children were lost to follow-up, and 86 were ultimately included in the analysis (Figure 1).

Figure 1 Technical road map. Ninety-eight preschool children with ADHD, excluding 3 with DQ scores <75, 1 with antiepileptic drug use and 4 in special education classrooms, were ultimately included in the study and completed the behavioral intervention for 12 months. Four children dropped out during the intervention. At the end of the intervention, 50 children had good effects, and 36 children had poor effects.

There were 50 children (58.140%) in the effective group and 36 children (41.860%) in the ineffective group. Among the 86 participants, 52 (60.465%) children were diagnosed with the inattention type (ADHD-I), 8 (9.302%) with the hyperactive-impulsive type (ADHD-H), and 26 (30.233%) with the combined type (ADHD-C). The children had a mean age of 59.675 months (SD=7.462) and a mean DQ score of 96.665 (SD=7.602). There were no significant group differences in age, gender, DQ scores or subtypes of ADHD (Table 1).

Table 1 Descriptive Information of the Children in Both Groups

Table 2 presented the interquartile ranges of the main evaluation indices of the children in the two groups. The ADHD-RS-IV and SNAP scores at baseline and the post-intervention ADHD-RS-IV scores of the effective group were lower than those of the ineffective group, but the attention retention times both at baseline and postintervention were higher than those of the ineffective group.

Table 2 Interquartile Ranges of the Main Evaluation Indices of the Children in the Two Groups

Table 3 presented the means, standard deviations, and bivariate correlations of all primary study variables. The attention deficit and hyperactivity impulsivity dimensions of the children’s ADHD scale were positively correlated with the corresponding dimensions of the SNAP scale, with correlation coefficients of 0.669 (p<0.001) and 0.812 (p<0.001), respectively.

Table 3 Mean Standard Deviation and the Correlation Between the ADHD and SNAP-IV Scales

Univariate analysis showed that compared with the effective group, the ineffective group had shorter attention retention times and higher SNAP scores at baseline, poor performance scores for parents, more than one complication, lack of teacher cooperation, and lower parental education levels, with significant differences (p<0.001). Variables with statistical significance in the univariate analysis (exclusion criteria p=0.15) were included in the multivariate analysis, which revealed that attention retention times, SNAP total scores at baseline, performance scores for parents, and teacher coordination with the intervention were the factors that influenced the intervention effect (Table 4).

Table 4 The Analysis of the Effect of the Behavioral Intervention Using Univariable and Multivariable Logistic Regression

The estimated equation of the behavioral intervention effect for children with ADHD was as follows:

The p value represented the probability of poor behavioral intervention effects. x₁, x₂, x₃ and x₄ represented attention retention time at baseline, the SNAP total score at baseline, performance scores for parents, and teacher coordination, respectively.

The ROC curve (Figure 2) was constructed by using the predicted probability of logistic regression as the state variable and the efficacy as the test variable, and the AUC value was 0.979 (95% CI: 0.955–1.000, P<0.001), indicating that the diagnostic value was good. The sensitivity and specificity were 86.0% and 97.2%, respectively.

Figure 2 The receiver operating characteristic curve of the efficacy of behavioral interventions in preschool children with ADHD in China. The abscissa is 1-specificity, and the ordinate is sensitivity.

The Hosmer‒Lemeshow goodness of fit test was used to evaluate the calibration ability of the prediction model. The results showed that the likelihood ratio 2 was 4.694 and the P value was 0.790, indicating that there was no significant difference between the predicted value and the observed value, and the calibration degree of the model had a good degree of fit.

Based on the results of multivariable analyses, a nomogram for the risk of ineffective treatment was formulated, as shown in Figure 3A. The performance score for parents was found to have the highest impact on the prediction of efficacy.

Figure 3 Nomogram and calibration curve for the incident risk of the nonefficacy of behavioral interventions in preschool children with ADHD in China: (A) nomogram and (B) calibration plot.

The discrimination of the nomogram yielded a c-index of 0.979. The mean absolute error of the calibration curve of the nomogram was 0.023, as shown in Figure 3B.

Discussion

This study analyzed factors influencing the effectiveness of a 12-month behavioral intervention in preschool children diagnosed with ADHD. We found that short attentional retention time at baseline, poor performance scores by parents, high SNAP total score, and noninvolvement of teachers in behavioral interventions were associated with a risk of low efficacy. In addition, although no statistical significance was shown in the multivariate analysis, the comorbidities and parental education level were significant in the univariate analysis. A prediction model of the behavioral intervention effect for children with ADHD was constructed. The AUC was 0.979, which reflected the good quality of the ROC curve. The nomogram and calibration curve of factors influencing the efficacy of behavioral interventions were plotted.

ADHD is a chronic health condition requiring long-term adherence to treatment. A study has shown that providing a behavioral intervention before initiating medication can lead to better outcomes than if medication is administered first.³⁰ Moreno²⁰ conducted a randomized controlled study and found that behavioral interventions had a greater effect on response control and attention improvement than neurofeedback therapy and pharmacological therapy. Franke conducted a randomized controlled trial to evaluate the effect of an online self-help program for parents in the treatment of ADHD in preschool children and found that hyperactivity/inattention, agitation/impulsivity, resistance/aggression, social functioning, and teacher-rated prosocial behavior in children were significantly improved.³¹ Teasdale³² studied the effect of a brief behavioral intervention for preschool children with ADHD for up to one year. The results demonstrated significant improvements in parent- and teacher-reported child hyperactivity and inattention, with sustained improvement at 6 months and 1 year postintervention. In preschool children, behavioral interventions are generally delivered directly to parents via BPT.³³ The reported behavioral interventions included instruction manuals for parents and regular teleconferencing with professionals.³⁴ Our study was different from the above. First, we further analyzed the factors affecting the effectiveness of behavioral interventions based on the long-term effectiveness shown in previous randomized controlled trials. Second, regular face-to-face outpatient interviews were conducted to understand parents’ intervention implementation skills and to guide them to improve their skills. Third, outpatient physicians evaluated and recorded the implementation of parental behavioral interventions, which were quantified by scores. Fourth, children’s attention time accurately recorded by parents was used as an objective indicator. Finally, for eligible children, we also invited teachers to participate in behavioral intervention through classroom management and other means.

The BPT intervention recommended by the AAP focuses on modifying parenting to treat children’s problem behaviors and has been consistently recognized as an effective treatment for disruptive behavior and ADHD.³⁵ However, significant improvements in BPT were not universal. Clinical work has shown that not all children respond similarly to interventions or positive parental behaviors. For example, a recent German³⁶ randomized trial of the first-line capacity of a behavioral intervention to reduce the need for medication in children with ADHD showed that 63% of the nonbehavioral intervention group was medicated at the endpoint, and 26% of the behavioral intervention group was also taking medication. It can be concluded that children with ADHD have different symptoms or comorbidities, which makes it unlikely that behavioral interventions will be equally effective for all individuals, so it is necessary to emphasize the need for individualized treatment programs.

A number of parent, child, and family factors were explored as factors that potentially influence BPT efficacy. child symptom characteristics may influence parents’ ability to implement parenting skills – a link that may be explained, in part, by increases in parenting-related stress.³⁷ Parent involvement is critical to the success of BPT. Suboptimal parent engagement undermines the potency and effectiveness of the intervention and limits improvement in parent and child outcomes. It is estimated that 40% to 60% of parents of children with ADHD have difficulty fully participating in treatment.³⁸ In classic BPT, parents attend treatment sessions without their child, and changes in child outcomes are dependent on changes in parent knowledge, beliefs, and behavior.³⁹ This study found that attention retention time, baseline Swanson, Nolan, and Pelham total scores, performance score for parents, and teachers’ cooperation were the influencing factors of the effect of behavioral intervention. We conclude that parental understanding and implementation of the behavioral intervention is one of the main factors affecting the efficacy of the intervention in children. In the process of behavioral intervention, it is necessary to fully consider the understanding, cooperation and implementation of parents to improve the compliance of parents. Our interventions included on-site parent training, a combination of intensive discussions and individual counseling, and regular one-on-one outpatient interviews and assessments. These measures had the advantage of strengthening the interactions among physicians, parents and children, allowing the observation and collection of data regarding changes in the children’s symptoms in a timely manner, allowing the identification of and improvement in inappropriate parenting behaviors, providing parents with counseling and treatment strategies, and adjusting individualized behavior intervention programs. We used on-site interviews with outpatient physicians and recorded parents’ participation and implementation in a unified standard quantitative scoring method every month to ensure the reliability of the data analysis results. In the future, children’s baseline attention level and comorbidities should be routinely assessed before initiating behavioral interventions, and children with severe symptoms should have more frequent visits to closely monitor their behavioral improvement. For parents with low education levels or insufficient executive ability, more attention should be given to increasing the amount of training for parents, and in-depth interviews should be conducted to understand what aspects of parents cannot effectively implement behavior management. Individual parental guidance or additional behavioral attention management sessions for children were provided when necessary. In this study, children’s baseline attention time recorded by parents was used as an objective indicator to help parents recognize the degree of their children’s symptoms by comparing the attention time of normal children. On the other hand, we take the baseline attention level as the target basis for formulating attention training, which can assist parents in conducting attention training and effect evaluation at home so that children and parents can grasp the effect of behavioral intervention for the first time, and parents can give positive reinforcement in time to improve children’s enthusiasm and confidence and improve parents’ cognitive level and compliance.

The collaborative life skills (CLS) program⁴⁰ was developed in the US and implemented by school mental health providers (SMHPs) with the participation of schools, parents and students. In randomized controlled trials with children with ADHD and ODD symptoms, parent- and teacher-reported posttreatment CLS gains, organizational skills and academic competence were maintained into the next school year. The difference in these studies was that the CLS programs were for school-age children, including those who were on medication, while in our study, the program was for preschool children who had not started medication. Because children are in kindergarten for approximately 6–7 hours every day, the influence of the teacher is a key factor. Recent findings highlight the need to strengthen teacher training for children with ADHD and to promote school involvement in children’s academic life.⁴¹ Combined with our study conclusions, the structure of behavioral interventions should be improved in the future, and school participation should be emphasized. Teachers, parents, and physicians should jointly develop behavioral intervention plans and regularly exchange information. A systemic framework should integrate the family, school and society in ADHD prevention and treatment.⁴²

There is a clear need for behavioral therapies in clinical practice, as they may be the first choice for patients⁴³ and families and can reduce the need for medications.^43,44 The heterogeneity of symptoms and comorbidities in children with ADHD makes it unlikely that behavioral treatments will be equally effective for all individuals, emphasizing the need for more individualized treatment plans. However, there is a lack of a holistic approach to identify the predictors of behavioral interventions. Based on logistic regression, nomograms can estimate the probability of failure of behavioral therapy by integrating various risk predictors, which meets our needs for visualization tools. Through a user-friendly digital interface, improved accuracy, and easier-to-understand risk predictions compared to traditional mathematical formulas, rapid computational nomograms can seamlessly integrate risk assessment into clinical decision making.⁴⁵ With the help of these simple, rapid, inexpensive and noninvasive tools, we hope to be able to timely and effectively identify children who do not respond to behavioral intervention alone. For children with severe baseline symptoms, difficulties for parents to implement behavioral intervention, and low probability of predicted effect, there is no need to continue to wait to complete all cycles of behavioral intervention. Other treatment options, such as drug therapy, can be considered in advance to improve the prognosis of children.

Advantages

The children with ADHD were recruited through the centers’ outpatient clinics, which were highly experienced in the treatment of ADHD children. We observed a very low lost-to-follow-up rate, probably due to regular outpatient visits. In the daily form, we developed an easy-to-operate behavioral intervention program that was easy to implement and promote. We used strictly measured attention retention times as an objective indicator. On the one hand, this measure can help parents observe their children’s performance at any time. On the other hand, this measure coordinates with a reduction in the ADHD-RS-IV behavior scale score and combined subjective and objective evaluations as an indicator to test the effect of interventions, with good reliability. We developed nomograms that can be used in the outpatient setting to quickly calculate the efficacy in children. The sample size of this study was relatively large, which provided sufficient statistical power in detecting correlations.

Limitations

Our data were collected at one hospital, and the parents in the study may have greater awareness of health care, which limits the generalization of the conclusion.

Conclusions

Our data showed that possible factors affecting the effect of behavioral interventions in preschool children with ADHD included basic attention retention time, performance score for parents, total SNAP score, and teacher cooperation. In the process of behavioral intervention, outpatient physicians should conduct in-depth interviews, regularly evaluate the implementation of parents, recommend that teachers participate in behavioral intervention, collect the baseline level of children, predict the probability of effective behavioral intervention based on the nomogram, adjust the individualized treatment plan in time, and start drug treatment as soon as necessary to provide a basis for clinical decision-making and improve the prognosis of children.