A Predictive Model Offor Attention Deficit Hyperactivity Disorder Based on Clinical Assessment Tools

Introduction

Attention deficit hyperactivity disorder (ADHD) is one of the most common diseases in child psychiatric clinics, with a prevalence rate of 7.2% in school-age children.^1,2 The pathogenesis of ADHD involves deficits in attention, working memory and executive inhibition. Furthermore, a recent research found that children with ADHD exhibit a deficit both in automatic and controlled cognitive processes.³ ADHD is involved in a variety of neuropsychological mechanisms. Therefore, the symptoms of ADHD are highly heterogeneous and are influenced by multiple factors, including physiological, psychological and social factors. At present, clinicians diagnose ADHD in children mainly based on the information provided by their parents, the behaviour of children at clinics and the assessments of clinical rating scales and neuropsychological tests. Conners' parent rating scale (PSQ) and Child Behavior Checklist (CBCL) are two of the most used clinical rating scales for evaluating ADHD.⁴ In recent years, increasing attention has been focused on the development of objective evaluation tools (such as various neuropsychological tests) for managing ADHD. Previous study has demonstrated that Integrated Visual and Auditory Continuous Performance Testing (IVA-CPT) could accurately discriminate children with ADHD.⁵ At present, there is no unified standard tool for analysing the results of various measurement tools used for ADHD. Therefore, clinicians analyse the results of clinical rating scales and neuropsychological tests entirely based on their clinical experience.

Machine learning provides an opportunity to parse the mechanisms and symptoms of mental disorders with multi-factorial and complex impact factors.⁶ Support vector machine (SVM), a machine learning method with high prediction accuracy, was first proposed by Vapnik et al.⁷ Machine learning involves means the establishment of a data classification model by through analysing a known data set with multi-dimensional parameters to predict the classification of new unknown data sets classified in accordance with unknown laws.⁸ Among various machine learning techniques, SVM is best suited to solve pattern recognition of small samples with high-dimensional parameters.⁹ SVM has been successfully applied in the diagnosis of clinical diseases and the prediction of treatment effect. In 2015, Kim et al used SVM to predict the response of ADHD children with ADHD to methylphenidate with 84.6% accuracy.¹⁰ Meanwhile, several studies have adopted SVM in the diagnosis of ADHD.

In the present study, two clinical rating scales (PSQ and CBCL) and one neuropsychological test (IVP-CPTin diagnosing ADHD) were adopted as measuring tools. SVM was used to analyse the results of these measurements and establish a classification prediction model. The present study aimed to offer suggestions for clinicians on how to analyse the results of clinical scales and neuropsychological tests for ADHD.

Methods and Subjects

Subjects

From January 2017 to June 2018, school-aged children (aged 6–12 years) who were admitted to The Affiliated Hospital of Hangzhou Normal University for various learning problems were recruited. The diagnosis was conducted by an experienced psychiatrist in accordance with The Diagnostic and Statistical Manual of Mental Disorders (DSM-5) after a semi-structured interview from the ADHD module of Schedule for Affective Disorders and Schizophrenia in School-Age Children–Present and Lifetime Version (K-SADS-PL).^11,12 The children diagnosed with ADHD were assigned to the ADHD group, whereas those who failed to meet any criteria for mental disorders were assigned to the control group. In this study, all children with brain damage, neurological disorders, genetic disorders, substance dependence, epilepsy, IQ of 70 or less or any other mental disorders were excluded. All the subjects volunteered to participate in the present study and the parents or legal guardians of these subjects provided written informed consent. All the protocols were approved by the Research Ethics Review Board of The Affiliated Hospital of Hangzhou Normal University. The present study was conducted in accordance with the Declaration of Helsinki. Finally, at the end of this study, 159 subjects (95 in the ADHD group and 64 in the control group) were recruited, and no significant difference was observed in the age and sex gender ratio between the two groups (Table 1).

Table 1 Demographic Characteristics of ADHD Group and Control Group

Measurements of Clinical Rating Scales

SQ consists of 48 questions of four grades and was completed by the parents. Six rating parameters (impulsion/hyperactivity, conduct problem, ADHD index, cognitive problems, psychosomatic problems and anxiety) were selected for analysis.¹³ CBCL consists of three parts and 128 questions, and it was completed by the parents. Fifteen parameters (social function, activity ability, social ability, learning ability, behaviour problem, internalisation, externalisation, delinquent behaviour, social problems, thought problems, anxiety/depression, somatic complaints, attention problems, withdrawn and aggressive behaviour) were selected for the analysis.¹³

Measurement by Neuropsychological Test

IVA-CPT was selected to assess the functions of attention and self-control ability to auditory and visual stimuli4. When the visual or auditory stimuli were presented to the subjects, four cognitive variables, including omission number, error response number, reaction time and stability, were monitored. A total of 29 parameters were obtained by analysing these cognitive variables.

Statistical Methods

The results of PSQ, CBCL and IVA-CPT were analysed using a two-sample t-test (false discovery rate corrected, FDR) to screen out the parameters with a significant difference between the two groups, which was shown by mean ± standard deviation. In case of significant effects, Cohen’s d was computed for the effect size.^3,14 Based on the screened parameters, LibSVM was used to develop the classification model. The four kernel functions of LibSVM (linear, polynomial, radial basis function (RBF) and sigmoid) were used for classification modelling. The specific steps are as follows: 1) training of the four classification models using the four kernel functions; 2) using the four classification models to classify the subjects; 3) using leave-one-out method to evaluate the accuracy, sensitivity (true positive rate) and specificity (true negative rate) of the four classification models; 4) applying Plotroc.py in LibSVM toolbox to draw a receiver operating characteristic (ROC) curve to evaluate the classification accuracy (a high area under the ROC curve (AUC) value suggests a good classification accuracy; 5) calculation of the weight of evaluation parameters based on the classification model with the highest accuracy. A high weight value suggests an important classification prediction.

Results

Comparative Analysis Between ADHD and Control Groups

The comparative analysis results showed significant differences in 16 parameters of PSQ and CBCL between the ADHD and control groups (p < 0.05, FDR corrected) (Table 2). Significant differences were observed in three parameters of IVA-CPT between the ADHD and control groups (p < 0.05, FDR corrected) (Table 3).

Table 2 Comparative Analysis of Clinical Rating Scales

Table 3 Comparative Analysis of Neuropsychological Test Results

LibSVM Classification Model

Firstly, 16 parameters of the clinical rating scales were selected for classification modelling. Among the four classification models using the four kernel functions, the prediction accuracy of linear classification model was the highest (61.635%), and its sensitivity and specificity were 71.579% and 46.875%, respectively (Table 4). The AUC of the linear classification model approximated 0.7839 (Figure 1). Secondly, three parameters obtained from IVA-CPT were selected for classification modelling. Among the four kernel functions, the classification prediction accuracy of the linear model was also the highest (59.748%), and its sensitivity and specificity were 100% and 0%, respectively (Table 4). The AUC of the linear model was about 0.6100 (Figure 1). Finally, 19 parameters (16 parameters of PSQ and CBCL and three parameters of IVA-CPT) were used in classification modelling. The classification prediction accuracy of the linear was the highest (70.44%), and its sensitivity and specificity were 74.737% and 64.063%, respectively (Table 4). The AUC amounted to 0.8222 (Figure 1).

Figure 1 ROC curve of LibSVM classification model.

Table 4 LibSVM Classification Model

In addition, the classification modelling analysis results of LibSVM showed the weight values of 19 parameters of the clinical rating scale and neuropsychological test for classification prediction (Table 5).

Table 5 Weight Values of the Parameters

Discussion

To date, clinical rating scales have been used as the primary assessment tools for ADHD, and the information required in these tools was provided by the primary caregiver of children based on their subjective memory. The information required in the neuropsychological tests, such as IVA-CPT, which have been developed in recent years, was provided by children themselves. Neuropsychological tests are relatively objective. However, it still remains uncertain as to how the parameters obtained by these evaluation tools must be analysed and which parameters are the most reliable ones. In this study, t-test was conducted to compare the 50 parameters obtained from PSQ, CBCL and IVA-CPT between the ADHD and control groups, and 19 parameters with significant differences were screened out (4 parameters of PSQ, 12 parameters of CBCL and three parameters of IVA-CPT). Then, 16 clinical rating scale parameters and 3 neuropsychological test parameters were used for SVM classification modelling. The accuracy of the classification model using 16 parameters of clinical rating scales is higher than that of the classification model using 3 parameters of the neuropsychological test. This finding implies that the clinical rating scales might be more accurate than the neuropsychological tests in the diagnosis of ADHD. Finally, the accuracy of the classification model using all 19 parameters of the two kinds of assessment methods was higher than that of the models using the parameters obtained from clinical rating scales or neuropsychological test alone. This result suggests that a comprehensive reference of the results of various assessment methods could improve the accuracy of ADHD diagnosis. In 2011, the prediction accuracy of 62.52% on ADHD diagnosis was achieved by a research team from Johns Hopkins University using machine learning methods and based on the data of the brain functional and structural magnetic resonance imaging from the ADHD-200 competition.^15,16 In 2019, Kaur et al used SVM method to predict ADHD in accordance with electroencephalography data, and the prediction accuracy reached 93.3%, which was higher than the accuracy obtained in this study.¹⁷ This finding suggests that in the future, the diagnosis of ADHD should be a comprehensive reference for a variety of assessment results.

The weights of 19 parameters in the classification model were also analysed. The top three parameters with the highest weight value were “learning problem” and “Hyperactivity/impulsivity parameters” of PSQ and the parameters “activity ability” of CBCL. The possible reason is that most children with ADHD were taken to the clinic by their parents only after they start schooling and caused serious concern because of their deficit in learning ability caused by attention deficit or hyperactivity symptoms. Therefore, the parents and clinicians are more concerned about “learning problem”, “hyperactivity/impulsivity parameters” and “activity ability”. Thus, these parameters play the most important roles in the diagnosis of clinical ADHD. Notably, the three parameters of the neuropsychological test showed the lowest weighted value. This result indicates that the main reference for the current clinical diagnosis of ADHD is still the results of the clinical rating scale and the parents’ description of children’s behaviour, academic performance and social adaptability. The current clinical diagnostic criteria for ADHD (such as DSM-5) are also closely related to the symptoms reflected by the parameters of these clinical rating scales.

In conclusion, the findings suggest that the diagnosis of ADHD should be based on multidimensional assessment. For a more accurate diagnosis of ADHD, more assessment parameters should be developed from different dimensions of physiology or psychology in the future. Furthermore, the predictive model for ADHD may lead to a better understanding and optimisation of the treatment of such a condition.