A Predictive Model Offor Attention Deficit Hyperactivity Disorder Based on Clinical Assessment Tools
Received 11 January 2020
Accepted for publication 6 May 2020
Published 25 May 2020 Volume 2020:16 Pages 1331—1337
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Yuping Ning
Dai Han, 1– 3 Yantong Fang, 2 Hong Luo 1, 2
1Institutes of Psychological Sciences, Hangzhou Normal University, Hangzhou, Zhejiang, People’s Republic of China; 2Children and Adolescents Mental Health Joint Clinic, The Affiliated Hospital of Hangzhou Normal University, Hangzhou, Zhejiang, People’s Republic of China; 3Zhejiang Key Laboratory for Research in Assessment of Cognitive Impairments, Hangzhou, Zhejiang, People’s Republic of China
Correspondence: Hong Luo
Children and Adolescents Mental Health Joint Clinic, The Affiliated Hospital of Hangzhou Normal University, No. 126 Wenzhou Road, Gongshu District, Hangzhou, Zhejiang Province 311121, People’s Republic of China
Email [email protected]
Background: At present, clinicians diagnose that the clinical diagnosis of attention deficit hyperactivity disorder (ADHD) in children is mainly on the basis of the information provided by their parents, the behaviour of children in clinical clinics and the assessments of clinical rating scales and neuropsychological tests. Notably, no unified standard exists currently for analysing the results of various measurement tools for diagnosing ADHD. Therefore, clinicians interpret the results of clinical rating scales and neuropsychological tests entirely based on their clinical experience.
Methods and subjects:: To provide guidance for clinicians on how to analyse the results of various clinical assessment tools when diagnosing ADHD, this study assessed children with ADHD and children in the control group using two clinical assessment scales—parent rating scale (PSQ) and Child Behavior Checklist (CBCL)—and one neuropsychological test (Integrated Visual and Auditory Continuous Performance Testing). The two-sample t-test (FDR correction) screened the parameters of the three assessment tools with significant inter-group differences. LibSVM was used to establish a classification prediction model for analysing the accuracy of ADHD prediction using parameters of the three assessment tools and weight values of each parameter for classification prediction.
Results: A total of 19 parameters (16 from clinical rating scales, 3 from neuropsychological tests) with significant inter-group differences were screened. The accuracy of classification modelling was higher for the clinical rating scales (61.635%) than for the neuropsychological test (59.784%), whereas the accuracy of classification modelling was higher for the clinical rating scales combined with the neuropsychological test (70.440%) than for the former two parameters alone. The three parameters with the highest weight values were learning problem (0.731), hyperactivity/impulsivity (0.676) and activity capacity (0.569). The three parameters with the lowest weight values are integrated control force (0.028), visual attention (0.028) and integrated attention (0.034).
Conclusion: Our study findings indicate that the diagnosis of ADHD should be based on multidimensional assessment. For a more accurate diagnosis of ADHD, assessments and that more assessment parameters should be developed on the basis of different dimensions of physiology or psychology in the future to obtain a more accurate diagnosis of ADHD. Furthermore, the predictive model for ADHD may improve our understanding and help in optimisation of the treatment of such a condition.
Keywords: attention deficit hyperactivity disorder, LibSVM, parent rating scale, child behavior checklist, integrated visual and auditory continuous performance testing
Corrigendum for this paper has been published
Attention deficit hyperactivity disorder (ADHD) is one of the most common diseases in child psychiatric clinics, with a prevalence rate of 7.2% in school-age children.1,2 The pathogenesis of ADHD involves deficits in attention, working memory and executive inhibition. Furthermore, a recent research found that children with ADHD exhibit a deficit both in automatic and controlled cognitive processes.3 ADHD is involved in a variety of neuropsychological mechanisms. Therefore, the symptoms of ADHD are highly heterogeneous and are influenced by multiple factors, including physiological, psychological and social factors. At present, clinicians diagnose ADHD in children mainly based on the information provided by their parents, the behaviour of children at clinics and the assessments of clinical rating scales and neuropsychological tests. Conners' parent rating scale (PSQ) and Child Behavior Checklist (CBCL) are two of the most used clinical rating scales for evaluating ADHD.4 In recent years, increasing attention has been focused on the development of objective evaluation tools (such as various neuropsychological tests) for managing ADHD. Previous study has demonstrated that Integrated Visual and Auditory Continuous Performance Testing (IVA-CPT) could accurately discriminate children with ADHD.5 At present, there is no unified standard tool for analysing the results of various measurement tools used for ADHD. Therefore, clinicians analyse the results of clinical rating scales and neuropsychological tests entirely based on their clinical experience.
Machine learning provides an opportunity to parse the mechanisms and symptoms of mental disorders with multi-factorial and complex impact factors.6 Support vector machine (SVM), a machine learning method with high prediction accuracy, was first proposed by Vapnik et al.7 Machine learning involves means the establishment of a data classification model by through analysing a known data set with multi-dimensional parameters to predict the classification of new unknown data sets classified in accordance with unknown laws.8 Among various machine learning techniques, SVM is best suited to solve pattern recognition of small samples with high-dimensional parameters.9 SVM has been successfully applied in the diagnosis of clinical diseases and the prediction of treatment effect. In 2015, Kim et al used SVM to predict the response of ADHD children with ADHD to methylphenidate with 84.6% accuracy.10 Meanwhile, several studies have adopted SVM in the diagnosis of ADHD.
In the present study, two clinical rating scales (PSQ and CBCL) and one neuropsychological test (IVP-CPTin diagnosing ADHD) were adopted as measuring tools. SVM was used to analyse the results of these measurements and establish a classification prediction model. The present study aimed to offer suggestions for clinicians on how to analyse the results of clinical scales and neuropsychological tests for ADHD.
Methods and Subjects
From January 2017 to June 2018, school-aged children (aged 6–12 years) who were admitted to The Affiliated Hospital of Hangzhou Normal University for various learning problems were recruited. The diagnosis was conducted by an experienced psychiatrist in accordance with The Diagnostic and Statistical Manual of Mental Disorders (DSM-5) after a semi-structured interview from the ADHD module of Schedule for Affective Disorders and Schizophrenia in School-Age Children–Present and Lifetime Version (K-SADS-PL).11,12 The children diagnosed with ADHD were assigned to the ADHD group, whereas those who failed to meet any criteria for mental disorders were assigned to the control group. In this study, all children with brain damage, neurological disorders, genetic disorders, substance dependence, epilepsy, IQ of 70 or less or any other mental disorders were excluded. All the subjects volunteered to participate in the present study and the parents or legal guardians of these subjects provided written informed consent. All the protocols were approved by the Research Ethics Review Board of The Affiliated Hospital of Hangzhou Normal University. The present study was conducted in accordance with the Declaration of Helsinki. Finally, at the end of this study, 159 subjects (95 in the ADHD group and 64 in the control group) were recruited, and no significant difference was observed in the age and sex gender ratio between the two groups (Table 1).
Table 1 Demographic Characteristics of ADHD Group and Control Group
Measurements of Clinical Rating Scales
SQ consists of 48 questions of four grades and was completed by the parents. Six rating parameters (impulsion/hyperactivity, conduct problem, ADHD index, cognitive problems, psychosomatic problems and anxiety) were selected for analysis.13 CBCL consists of three parts and 128 questions, and it was completed by the parents. Fifteen parameters (social function, activity ability, social ability, learning ability, behaviour problem, internalisation, externalisation, delinquent behaviour, social problems, thought problems, anxiety/depression, somatic complaints, attention problems, withdrawn and aggressive behaviour) were selected for the analysis.13
Measurement by Neuropsychological Test
IVA-CPT was selected to assess the functions of attention and self-control ability to auditory and visual stimuli4. When the visual or auditory stimuli were presented to the subjects, four cognitive variables, including omission number, error response number, reaction time and stability, were monitored. A total of 29 parameters were obtained by analysing these cognitive variables.
The results of PSQ, CBCL and IVA-CPT were analysed using a two-sample t-test (false discovery rate corrected, FDR) to screen out the parameters with a significant difference between the two groups, which was shown by mean ± standard deviation. In case of significant effects, Cohen’s d was computed for the effect size.3,14 Based on the screened parameters, LibSVM was used to develop the classification model. The four kernel functions of LibSVM (linear, polynomial, radial basis function (RBF) and sigmoid) were used for classification modelling. The specific steps are as follows: 1) training of the four classification models using the four kernel functions; 2) using the four classification models to classify the subjects; 3) using leave-one-out method to evaluate the accuracy, sensitivity (true positive rate) and specificity (true negative rate) of the four classification models; 4) applying Plotroc.py in LibSVM toolbox to draw a receiver operating characteristic (ROC) curve to evaluate the classification accuracy (a high area under the ROC curve (AUC) value suggests a good classification accuracy; 5) calculation of the weight of evaluation parameters based on the classification model with the highest accuracy. A high weight value suggests an important classification prediction.
Comparative Analysis Between ADHD and Control Groups
The comparative analysis results showed significant differences in 16 parameters of PSQ and CBCL between the ADHD and control groups (p < 0.05, FDR corrected) (Table 2). Significant differences were observed in three parameters of IVA-CPT between the ADHD and control groups (p < 0.05, FDR corrected) (Table 3).
Table 2 Comparative Analysis of Clinical Rating Scales
Table 3 Comparative Analysis of Neuropsychological Test Results
LibSVM Classification Model
Firstly, 16 parameters of the clinical rating scales were selected for classification modelling. Among the four classification models using the four kernel functions, the prediction accuracy of linear classification model was the highest (61.635%), and its sensitivity and specificity were 71.579% and 46.875%, respectively (Table 4). The AUC of the linear classification model approximated 0.7839 (Figure 1). Secondly, three parameters obtained from IVA-CPT were selected for classification modelling. Among the four kernel functions, the classification prediction accuracy of the linear model was also the highest (59.748%), and its sensitivity and specificity were 100% and 0%, respectively (Table 4). The AUC of the linear model was about 0.6100 (Figure 1). Finally, 19 parameters (16 parameters of PSQ and CBCL and three parameters of IVA-CPT) were used in classification modelling. The classification prediction accuracy of the linear was the highest (70.44%), and its sensitivity and specificity were 74.737% and 64.063%, respectively (Table 4). The AUC amounted to 0.8222 (Figure 1).
Figure 1 ROC curve of LibSVM classification model.
Table 4 LibSVM Classification Model
In addition, the classification modelling analysis results of LibSVM showed the weight values of 19 parameters of the clinical rating scale and neuropsychological test for classification prediction (Table 5).
Table 5 Weight Values of the Parameters
To date, clinical rating scales have been used as the primary assessment tools for ADHD, and the information required in these tools was provided by the primary caregiver of children based on their subjective memory. The information required in the neuropsychological tests, such as IVA-CPT, which have been developed in recent years, was provided by children themselves. Neuropsychological tests are relatively objective. However, it still remains uncertain as to how the parameters obtained by these evaluation tools must be analysed and which parameters are the most reliable ones. In this study, t-test was conducted to compare the 50 parameters obtained from PSQ, CBCL and IVA-CPT between the ADHD and control groups, and 19 parameters with significant differences were screened out (4 parameters of PSQ, 12 parameters of CBCL and three parameters of IVA-CPT). Then, 16 clinical rating scale parameters and 3 neuropsychological test parameters were used for SVM classification modelling. The accuracy of the classification model using 16 parameters of clinical rating scales is higher than that of the classification model using 3 parameters of the neuropsychological test. This finding implies that the clinical rating scales might be more accurate than the neuropsychological tests in the diagnosis of ADHD. Finally, the accuracy of the classification model using all 19 parameters of the two kinds of assessment methods was higher than that of the models using the parameters obtained from clinical rating scales or neuropsychological test alone. This result suggests that a comprehensive reference of the results of various assessment methods could improve the accuracy of ADHD diagnosis. In 2011, the prediction accuracy of 62.52% on ADHD diagnosis was achieved by a research team from Johns Hopkins University using machine learning methods and based on the data of the brain functional and structural magnetic resonance imaging from the ADHD-200 competition.15,16 In 2019, Kaur et al used SVM method to predict ADHD in accordance with electroencephalography data, and the prediction accuracy reached 93.3%, which was higher than the accuracy obtained in this study.17 This finding suggests that in the future, the diagnosis of ADHD should be a comprehensive reference for a variety of assessment results.
The weights of 19 parameters in the classification model were also analysed. The top three parameters with the highest weight value were “learning problem” and “Hyperactivity/impulsivity parameters” of PSQ and the parameters “activity ability” of CBCL. The possible reason is that most children with ADHD were taken to the clinic by their parents only after they start schooling and caused serious concern because of their deficit in learning ability caused by attention deficit or hyperactivity symptoms. Therefore, the parents and clinicians are more concerned about “learning problem”, “hyperactivity/impulsivity parameters” and “activity ability”. Thus, these parameters play the most important roles in the diagnosis of clinical ADHD. Notably, the three parameters of the neuropsychological test showed the lowest weighted value. This result indicates that the main reference for the current clinical diagnosis of ADHD is still the results of the clinical rating scale and the parents’ description of children’s behaviour, academic performance and social adaptability. The current clinical diagnostic criteria for ADHD (such as DSM-5) are also closely related to the symptoms reflected by the parameters of these clinical rating scales.
In conclusion, the findings suggest that the diagnosis of ADHD should be based on multidimensional assessment. For a more accurate diagnosis of ADHD, more assessment parameters should be developed from different dimensions of physiology or psychology in the future. Furthermore, the predictive model for ADHD may lead to a better understanding and optimisation of the treatment of such a condition.
This work was supported by High-level Overseas Returnees (teams) Innovative and Entrepreneurship Project of Hangzhou (No 4045C5062000615) and Zhejiang Provincial Natural Science Foundation of China (No LQ14H090004) and the China Scholarship Council (201708330165).
The authors report no conflicts of interest in this work.
1. Thapar A, Cooper M. Attention deficit hyperactivity disorder. Lancet. 2016;387(10024):1240–1250. doi:10.1016/S0140-6736(15)00238-X
2. Wang T, Liu K, Li Z, et al. Prevalence of attention deficit/hyperactivity disorder among children and adolescents in China: a systematic review and meta-analysis. BMC Psychiatry. 2017;17(1):32. doi:10.1186/s12888-016-1187-9
3. Capri T, Santoddi E, Fabio RA. Multi-source interference task paradigm to enhance automatic and controlled processes in ADHD. Res Dev Disabil. 2020;97:103542. doi:10.1016/j.ridd.2019.103542
4. Wolraich ML, Hagan JF
5. Tinius TP. The integrated visual and auditory continuous performance test as a neuropsychological measure. Arch Clin Neuropsychol. 2003;18(5):439–454. doi:10.1093/arclin/18.5.439
6. Tai AMY, Albuquerque A, Carmona NE, et al. Machine learning and big data: implications for disease modeling and therapeutic discovery in psychiatry. Artif Intell Med. 2019;99:101704. doi:10.1016/j.artmed.2019.101704
7. Vapnik V. The Nature of Statistical Learning Theory. New York: Springer;1995.
8. Noble WS. What is a support vector machine? Nat Biotechnol. 2006;24(12):1565–1567. doi:10.1038/nbt1206-1565
9. Pirooznia M, Deng Y. SVM classifier - a comprehensive Java interface for support vector machine classification of microarray data. BMC Bioinformatics. 2006;7(Suppl 4):S25. doi:10.1186/1471-2105-7-S4-S25
10. Kim JW, Sharma V, Ryan ND. Predicting methylphenidate response in ADHD using machine learning approaches. Int J Neuropsychopharmacol. 2015;18(11):pyv052. doi:10.1093/ijnp/pyv052
11. Austerman J. ADHD and behavioral disorders: assessment, management, and an update from DSM-5. Cleve Clin J Med. 2015;82(11 Suppl 1):S2–S7. doi:10.3949/ccjm.82.s1.01
12. Chen YL, Shen LJ, Gau SS. The Mandarin version of the kiddie-schedule for affective disorders and schizophrenia-epidemiological version for DSM-5 - a psychometric study. J Formos Med Assoc. 2017;116(9):671–678. doi:10.1016/j.jfma.2017.06.013
13. Zhang MY, Zhu CM, Wu WY, Chen YF. Handbook of the psychiatric rating scale. 2nd ed.Changsha: Hunan Science & Techology Press, 1998.
14. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum; 1988.
15. Itani S, Rossignol M, Lecron F, Fortemps P. Towards interpretable machine learning models for diagnosis aid: a case study on attention deficit/hyperactivity disorder. PLoS One. 2019;14(4):e0215720. doi:10.1371/journal.pone.0215720
16. Brown MR, Sidhu GS, Greiner R, et al. ADHD-200 global competition: diagnosing ADHD using personal characteristic data can outperform resting state fMRI measurements. Front Syst Neurosci. 2012;6:69. doi:10.3389/fnsys.2012.00069
17. Kaur S, Singh S. Phase space reconstruction of EEG signals for classification of ADHD and control adults. Clin EEG Neurosci. 2019;1550059419876525.
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.