A virtual reality application for assessment for attention deficit hyperactivity disorder in school-aged children
Received 25 February 2019
Accepted for publication 8 April 2019
Published 4 June 2019 Volume 2019:15 Pages 1517—1523
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Yuping Ning
Yantong Fang,1 Dai Han,1–3 Hong Luo1
1Children and Adolescents Mental Health Joint Clinic, The Affiliated Hospital of Hangzhou Normal University, Hangzhou, Zhejiang, People’s Republic of China; 2Institutes of Psychological Sciences, Hangzhou Normal University, Hangzhou, Zhejiang, People’s Republic of China; 3Zhejiang Key Laboratory for Research in Assessment of Cognitive Impairments, Hangzhou, Zhejiang, People’s Republic of China
Background and objective: The development of objective assessment tools for attention deficit hyperactivity disorder (ADHD) has become a hot research topic in recent years. This study was conducted to explore the feasibility and availability of virtual reality (VR) for evaluating symptoms of ADHD.
Methods: School-aged children were recruited. The children with ADHD or without ADHD were assigned into the ADHD group or Control group, respectively. They were all evaluated using the Conners’ Parent Rating Scale (CPRS), Child Behavior Checklist (CBCL), Integrated Visual and Auditory Continuous Performance Test (IVA-CPT), and a VR test.
Results: The correct items, incorrect items, and the accuracy rate of the VR test of the children with ADHD were significantly different with those of the children in the Control group. The correct items, incorrect items, total time, and accuracy of the VR test were significantly correlated with the scores of IVA-CPT (auditory attention and visual attention), CPRS (impulsion/hyperactivity and ADHD index), and CBCL (attention problems and social problems), respectively.
Discussion: The results supported the discriminant validity of the VR test for evaluating ADHD in school-age children suffering from learning problems. The VR test results are associated with the commonly used clinical measurements results. A VR test is interesting for children and therefore it attracts them to complete the test; whilst at the same time, it can also effectively evaluate ADHD symptoms.
Keywords: attention deficit hyperactivity disorder, ADHD, virtual reality, VR, Conners’ Parent Rating Scale, CPRS, Child Behavior Checklist, CBCL, Integrated Visual and Auditory Continuous Performance Test, IVA-CPT
Attention deficit hyperactivity disorder (ADHD) is one of the most common childhood and adolescent psychiatric disorders, with prevalence rates ranging from 5–7%.1,2 In addition to the three core symptoms (inattention, hyperactivity, and impulsivity), this disorder is characterized by some secondary symptoms of impairments of cognition, emotion regulation, and other secondary symptoms.3 These primary and secondary symptoms create significant difficulties in academic attainment, executive functioning, and social interactions.4 To date, the diagnosis of ADHD mainly relies on clinical interviews, self/other reports, and rating scales. However, the validity and reliability of the diagnosis of ADHD is limited to subjective assessments. The inconsistency between the results of objective measurements and subjective measurements have been reported in many studies on ADHD.5–8 It has been suggested that using objective measures may enhance assay sensitivity in clinical trials on ADHD6 The study of Konrad et al9 indicated that the teacher–behavior ratings of ADHD symptoms seemed to be predominantly influenced by changes of motor activity, and did not cohere with the measurement results of computerized attention tasks.
In recent years, increasing attention was paid to the development of objective evaluation tools. The Continuous Performance Test (CPT) is a type of test, which requires participants to select a response to a particular auditory/visual stimulus that is delivered at regular intervals. Various kinds of CPTs were developed to evaluate children with ADHD objectively, such as OPATUS CPTA assessment (https://opatus.se/). Previous studies have demonstrated that the integrated visual and auditory continuous performance testing (IVA-CPT) could accurately discriminate children with ADHD from control children.10 Quinn11 reported that malingerers could fake ADHD on the subjective rating scale, but could not on IVA-CPT. This difference demonstrated the superiority of objective measurements in the evaluation of ADHD. However, it is notable that CPT is monotonous and repetitive, and requires an individual to focus attention for a long time. Children frequently become bored during this kind of task. Only children with good self-control can successfully complete it. Frequently, it is difficult for children with ADHD to complete all items of the CPT in the prescribed time. Consequently, both the reliability and validity of IVA-CPT might be influenced. It is necessary to assess ADHD by a more interesting test.
Virtual reality (VR) is a relatively new technology that simulates an imagined or real environment, thereby enabling users to immerse themselves in a virtual 3-dimensional environment. This technology is being increasingly adopted in educational, entertainment, military, and medical areas. Many studies have suggested that a neuropsychological assessment based on VR technology is more enjoyable and can gather better behavioral data of distractibility and attentional deficits, compared with traditional neuropsychological tests.12,13 Negut et al14 reported the superior efficiency of VR-based CPT in children with ADHD, compared with traditional CPT. The studies on the quality of experience in real environments and VR showed that virtual experience is associated with Flow, a state of consciousness characterized by narrowed focus of attention, deep concentration.12 Therefore, an increased number of studies have tried to introduce VR technology into neuropsychological assessment of ADHD in many countries.15–17 The present study was conducted to explore the feasibility of VR-based assessment in school-aged children with ADHD, who suffered with learning problems. The validity of the VR test was systematically investigated by comparing outcomes of the VR test of children with ADHD with those of control children. Also, the relationship between outcomes of the VR test and other frequently-used measurements of ADHD was analyzed.
All participants were recruited from the school-aged (6–18 years) children attending the Children and Adolescents Mental Health Joint Clinic of The Affiliated Hospital of Hangzhou Normal University from January 2017 to September 2018. These children attended the clinic for bad academic achievements, and were selected depending on their and their parents’ willingness to participate in the study after they received a detailed explanation about it. The children who were firstly diagnosed with ADHD based on the Diagnostic Interview Schedule for Children Version V (DSM-V) were assigned into the ADHD group (see Table 1). The children who did not meet the criteria for any mental disorders were assigned into the Control group (see Table 1). The children with brain damage, neurological disorders, genetic disorders, substance dependence, epilepsy, an IQ of 70 or lower, or any other mental disorder reported during the personal history and anamnesis were excluded. All participants’ parents provided written informed consent to the content and ethics of this study. All protocols were conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Review Board of Hangzhou Normal University.
Table 1 The demography and VR test in the ADHD and control groups
The VR test is performed using the Virtual Reality Medical Center system (VRMC, Hangzhou Xinjing Technology Co., Ltd, Zhejiang, China). The total run-time performance is set by a computer and a high-end VR headset with 1,080×1,200 pixels per eye resolution, a 90 Hz refresh rate, and a 110 degrees field of view (see Figure 1). With this VR device, a high immersive interaction can be achieved due to its game-like interface system, various sensors, location trackers, and a data back-end system for data collection and statistical feedback. The VRMC system for the ADHD test is composed of three measurement scenes for ADHD, including Position Tracking (30 items), Stroop (41 items), and Object Recognition (60 items). The whole test should be completed in 20 minutes. The results of this test are presented using four indexes: correct items (the number of correct items), incorrect items (the number of incorrect items), accuracy (the percentage of correct items to total itemsx100), and total time (total testing time).
Figure 1 Examples of ADHD evaluation using a VR test.
The IVA-CPT was used to evaluate the ability to sustain attention and control responses on various auditory and vi sual stimuli (ages 6 years onwards). In recent years, it has frequently been used to assist the diagnosis of ADHD. The IVA-CPT contains many scales, of which the scales of auditory control, auditory attention, visual control, and visual attention were used in the present study.
Conventional questionnaire psychological test
ADHD-related behaviors were evaluated by the Conners Parent Rating Scale (CPRS), which contains 48 items on a 4-point scale. The six sub-scales of CPRS (impulsion/hyperactivity, conduct problems, cognitive problems, psychosomatic problems, anxiety, and ADHD index) were adapted to evaluate symptoms of ADHD. The Children’s Behavioral Checklist (CBCL, ages 6–18) was used to assess psychiatric symptoms according to parent ratings. It is a 113-item scale, in which eight sub-scales (attention problems, social problems, cognitive problems, anxiety/depression, somatic complaints, withdrawn, aggressive behavior, delinquent behavior) were adapted in the present study to present the symptoms of ADHD.
All data analyses were conducted using SPSS statistical software version 19.0 (SPSS, Inc., Chicago, IL, USA) and Microsoft Excel. Data normality was assessed by the Kolmogorov-Smirnov test. Data showing a normal distribution was summarized and is expressed as mean±SD. The index of VR test was analyzed by a Student's t-test for pairwise comparison between the two groups, and the relationship between the VR test and each other measurement was analyzed by Pearson correlation analysis.
The demography and the VR test in the ADHD group and control group
The comparative analysis results indicated that there is no significant difference on the age and sex ratios between the ADHD group and Control group. The accuracy and correct items in children with ADHD was significantly lower than that in children in the Control group, and the total time for children with ADHD was significantly higher than that for children in the Control group (P<0.05) (presented in Table 1).
Correlation analysis between VR test and IVA-CPT
Table 2 shows that incorrect items are negatively associated with scores of visual attention of IVA-CPT; the total time of the VR test is negatively associated with scores of both auditory and visual attention of IVA-CPT.
Table 2 The correlation between VR test and IVA-CPT
Correlation analysis between the VR test and Conners/CBCL
Table 3 indicates that incorrect items of the VR test were positively associated with the Hyperactivity index of CPRS. For the VR test and CBCL, Table 3 shows that correct items were negatively associated with the social problems index of the CBCL; incorrect items were positively associated with the scores of attention problems, social problems, cognitive problems, and aggressive behavior of CBCL.
Table 3 The correlation between VR test and CPRS/CBCL
At present, the application of virtual reality technology to clinical practice is still in it's infancy. ADHD is a neurodevelopmental disorder, mainly characterized by high levels of inattention, hyperactivity, and impulsivity, which were objectively evaluated using a VR test in the present study. The correct items, incorrect items, and the accuracy rate of the VR test of the children with ADHD were significantly different to those of the children in the Control group. This suggested that the VR test could distinguish children with ADHD from children with learning problems. Similarly, several previous researches have also demonstrated the discriminant validity of VR for attention assessment in children with ADHD.18,19
Moreover, the present study investigated the relationship between VR test results and IVA-CPT, which is frequently used as a measurement of attention and impulse control in children with ADHD. The level of visual attention on the IVA-CPT is associated with the correct items and the total time of the VR test. However, the level of auditory attention on the IVA-CP was only associated with the total time of the VR test. This suggests that the VR test adapted in the present study was sensitive to the attention function on visual information.
School-aged children with ADHD frequently suffer impairments in social, cognitive, academic, behavioral, and familial functioning, which are caused by inattention, hyperactivity, and impulsivity. These impairments have commonly been evaluated by altered scores of many sub-scales of the CBCL and CPRS.20,21 In the present study, relationships between the results of a VR test and the conventional questionnaire psychological tests (CPRS and CBCL) were analyzed. The results indicated that the correct items and incorrect items of the VR test were associated with the levels of impulsion/hyperactivity and ADHD index found on CPRS. Gilboa et al22 reported a significant association between VR assessment variables (virtual classroom, VC) and the results of CPRS-R (Conners’ Parent Rating Scales-Revised: Short) among children with acquired brain injury. These authors suggested that VR assessment appears to be both a sensitive and ecologically valid assessment tool for use in the diagnosis of attention deficits. Furthermore, the relationships were analyzed between the results of the VR test and CBCL, which is commonly used in clinical assessment of ADHD symptoms. The score of attention problems on the CBCL was associated with the correct items, incorrect items, and accuracy of the VR test. This indicates that the indicators of the VR test can sensitively reflect the level of attention deficit, the core symptom of ADHD. It is notable that the correct items, the incorrect items, and the total time of the VR test were significantly associated with a score of social problems on the CBCL. For school-age children with ADHD, attention disorders, impulsive, and hyperactive behaviors also often lead to problems in social function, cognitive ability, learning ability, behavior, and family function.23–26 In particular, ADHD children tend to be impulsive and hyperactive, and, therefore, often show inappropriate behavior which leads to a lack of social skills such as sharing and cooperation. The sub-scales of social problems, cognitive problems, aggressive behavior, and delinquent behavior of the CBCL can reflect the secondary symptoms of ADHD.20,21 However, this study found no significant correlation between the results of the VR test and the CBCL’s sub-scale scores on cognitive problems, aggressive behavior, and delinquent behavior. Further research is needed to understand why.
This study is a preliminary investigation on the feasibility of applying VR technology in the auxiliary analysis diagnosis of ADHD. The VR test results in this study only include correct answer items, wrong answer items, and total answer time, and the indexes of the VR test are more closely related to visual attention. The content and indexes of the VR test still need to be expanded and improved. In addition, the sample size of the study also needs to be further expanded.
In conclusion, the present study investigated the feasibility and availability of the VR method for evaluating symptoms of ADHD or potentially for diagnosing ADHD in school-aged Chinese children. The results of the present study indicated that ADHD children can be identified from children with learning problems according to a VR test. It must be pointed out that the study is flawed. The VR test indicators adopted in the present study are relatively single, and mainly reflect visual attention. However, multiple VR test indicators should be used to evaluate the multidimensional symptoms of ADHD. This study is only a preliminary exploration on the feasibility of a VR test in the assessment of ADHD symptoms. Relevant VR testing software is still in the initial stage of development. Future studies are necessary to improve the VR test for ADHD.
This work was supported by grants from the Zhejiang Provincial Medical Health Science and Technology Project (2019PY062) and the Zhejiang Provincial Natural Science Foundation of China (LQ14H090004). The authors thank Dr. Jing Huang (Technique Centre, Xinjing Technology Company Limited, Hangzhou, People’s Republic of China) for installing and debugging the VR instruments and assisting in operating the VR instruments and collecting research data.
All of the authors contributed to conception and design, acquisition or analysis and interpretation of data, drafting the article or revising it critically for important intellectual content, approved the final manuscript, and agree to be accountable for all aspects of the work.
The authors report no conflicts of interest in this work.
1. Thomas R, Sanders S, Doust J, Beller E, Glasziou P. Prevalence of attention-deficit/hyperactivity disorder: a systematic review and meta-analysis. Pediatrics. 2015;135(4):e994–e1001. doi:10.1542/peds.2014-3482
2. Wang T, Liu K, Li Z, et al. Prevalence of attention deficit/hyperactivity disorder among children and adolescents in China: a systematic review and meta-analysis. BMC Psychiatry. 2017;17(1):32. doi:10.1186/s12888-017-1489-6
4. Powell L, Parker J, Harpin V. What is the level of evidence for the use of currently available technologies in facilitating the self-management of difficulties associated with ADHD in children and young people? A systematic review. Eur Child Adolesc Psychiatry. 2018;27(11):1391–1412. doi:10.1007/s00787-017-1092-x
5. Manor I, Meidad S, Zalsman G, Zemishlany Z, Tyano S, Weizman A. Objective versus subjective assessment of methylphenidate response. Child Psychiatry Hum Dev. 2008;39(3):273–282. doi:10.1007/s10578-007-0087-0
6. Sumner CR, Haynes VS, Teicher MH, Newcorn JH. Does placebo response differ between objective and subjective measures in children with attention-deficit/hyperactivity disorder? Postgrad Med. 2010;122(5):52–61. doi:10.3810/pgm.2010.09.2201
7. Choi J, Yoon IY, Kim HW, Chung S, Yoo HJ. Differences between objective and subjective sleep measures in children with attention deficit hyperactivity disorder. J Clin Sleep Med. 2010;6(6):589–595.
8. Corkum P, Tannock R, Moldofsky H, Hogg-Johnson S, Humphries T. Actigraphy and parental ratings of sleep in children with attention-deficit/hyperactivity disorder (ADHD). Sleep. 2001;24(3):303–312.
9. Konrad K, Gunther T, Heinzel-Gutenbrunner M, Herpertz-Dahlmann B. Clinical evaluation of subjective and objective changes in motor activity and attention in children with attention-deficit/hyperactivity disorder in a double-blind methylphenidate trial. J Child Adolesc Psychopharmacol. 2005;15(2):180–190. doi:10.1089/cap.2005.15.180
10. Kim J, Lee Y, Han D, Min K, Kim D, Lee C. The utility of quantitative electroencephalography and integrated visual and auditory continuous performance test as auxiliary tools for the attention deficit hyperactivity disorder diagnosis. Clin Neurophysiol. 2015;126(3):532–540. doi:10.1016/j.clinph.2014.06.034
11. Quinn CA. Detection of malingering in assessment of adult ADHD. Arch Clin Neuropsychol. 2003;18(4):379–395.
12. Gaggioli A. Quality of experience in real and virtual environments: some suggestions for the development of positive technologies. Stud Health Technol Inform. 2012;181:177–181.
13. Parsey CM, Schmitter-Edgecombe M. Applications of technology in neuropsychological assessment. Clin Neuropsychol. 2013;27(8):1328–1361. doi:10.1080/13854046.2013.834971
14. Negut A, Jurma AM, David D. Virtual-reality-based attention assessment of ADHD: clinicaVR: classroom-CPT versus a traditional continuous performance test. Child Neuropsychol. 2017;23(6):692–712. doi:10.1080/09297049.2016.1186617
15. Parsons TD, Bowerly T, Buckwalter JG, Rizzo AA. A controlled clinical comparison of attention performance in children with ADHD in a virtual reality classroom compared to standard neuropsychological methods. Child Neuropsychol. 2007;13(4):363–381. doi:10.1080/13825580600943473
16. Clancy TA, Rucklidge JJ, Owen D. Road-crossing safety in virtual reality: a comparison of adolescents with and without ADHD. J Clin Child Adolesc Psychol. 2006;35(2):203–215. doi:10.1207/s15374424jccp3502_4
17. Pollak Y, Shomaly HB, Weiss PL, Rizzo AA, Gross-Tsur V. Methylphenidate effect in children with ADHD can be measured by an ecologically valid continuous performance test embedded in virtual reality. CNS Spectr. 2010;15(2):125–130.
18. Nolin P, Stipanicic A, Henry M, Lachapelle Y, Lussier-Desrochers D, Allain P. ClinicaVR: classroom-CPT: a virtual reality tool for assessing attention and inhibition in children and adolescents. Comput Human Behav. 2016;59:327–333. doi:10.1016/j.chb.2016.02.023
19. Diaz-Orueta U, Garcia-Lopez C, Crespo-Eguilaz N, Sanchez-Carpintero R, Climent G, Narbona J. AULA virtual reality test as an attention measure: convergent validity with Conners‘ Continuous Performance Test. Child Neuropsychol. 2014;20(3):328–342. doi:10.1080/09297049.2013.792332
20. Schmidt M, Reh V, Hirsch O, Rief W, Christiansen H. Assessment of ADHD symptoms and the issue of cultural variation: are conners 3 rating scales applicable to children and parents with migration background? J Atten Disord. 2017;21(7):587–599. doi:10.1177/1087054713493319
21. Fumeaux P, Mercier C, Roche S, Iwaz J, Stephan P, Revol O. Validation of the French version of conners‘ parent rating scale-revised, short form in adhd-diagnosed children and comparison with control children. J Atten Disord. . Epub 2018 Mar 18.
22. Gilboa Y, Kerrouche B, Longaud-Vales A, et al. Describing the attention profile of children and adolescents with acquired brain injury using the virtual classroom. Brain Inj. 2015;29(13–14):1691–1700. doi:10.3109/02699052.2015.1075148
23. Bora E, Pantelis C. Meta-analysis of social cognition in attention-deficit/hyperactivity disorder (ADHD): comparison with healthy controls and autistic spectrum disorder. Psychol Med. 2016;46(4):699–716. doi:10.1017/S0033291715002573
24. Oades RD, Lasky-Su J, Christiansen H, et al. The influence of serotonin- and other genes on impulsive behavioral aggression and cognitive impulsivity in children with attention-deficit/hyperactivity disorder (ADHD): findings from a family-based association test (FBAT) analysis. Behav Brain Funct. 2008;4:48. doi:10.1186/1744-9081-4-48
25. Thomaidis L, Choleva A, Janikian M, et al. Attention Deficit/Hyperactivity Disorder (ADHD) symptoms and cognitive skills of preschool children. Psychiatrike. 2017;28(1):28–36. doi:10.22365/jpsych.2017.281.28
26. Torrente F, Lopez P, Alvarez Prado D, et al. Dysfunctional cognitions and their emotional, behavioral, and functional correlates in adults with attention deficit hyperactivity disorder (ADHD): is the cognitive-behavioral model valid? J Atten Disord. 2014;18(5):412–424. doi:10.1177/1087054712443153
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.