A virtual reality application for assessment for attention deficit hyperactivity disorder in school-aged children

Introduction

Attention deficit hyperactivity disorder (ADHD) is one of the most common childhood and adolescent psychiatric disorders, with prevalence rates ranging from 5–7%.^1,2 In addition to the three core symptoms (inattention, hyperactivity, and impulsivity), this disorder is characterized by some secondary symptoms of impairments of cognition, emotion regulation, and other secondary symptoms.³ These primary and secondary symptoms create significant difficulties in academic attainment, executive functioning, and social interactions.⁴ To date, the diagnosis of ADHD mainly relies on clinical interviews, self/other reports, and rating scales. However, the validity and reliability of the diagnosis of ADHD is limited to subjective assessments. The inconsistency between the results of objective measurements and subjective measurements have been reported in many studies on ADHD.^5–8 It has been suggested that using objective measures may enhance assay sensitivity in clinical trials on ADHD⁶ The study of Konrad et al⁹ indicated that the teacher–behavior ratings of ADHD symptoms seemed to be predominantly influenced by changes of motor activity, and did not cohere with the measurement results of computerized attention tasks.

In recent years, increasing attention was paid to the development of objective evaluation tools. The Continuous Performance Test (CPT) is a type of test, which requires participants to select a response to a particular auditory/visual stimulus that is delivered at regular intervals. Various kinds of CPTs were developed to evaluate children with ADHD objectively, such as OPATUS CPTA assessment (https://opatus.se/). Previous studies have demonstrated that the integrated visual and auditory continuous performance testing (IVA-CPT) could accurately discriminate children with ADHD from control children.¹⁰ Quinn¹¹ reported that malingerers could fake ADHD on the subjective rating scale, but could not on IVA-CPT. This difference demonstrated the superiority of objective measurements in the evaluation of ADHD. However, it is notable that CPT is monotonous and repetitive, and requires an individual to focus attention for a long time. Children frequently become bored during this kind of task. Only children with good self-control can successfully complete it. Frequently, it is difficult for children with ADHD to complete all items of the CPT in the prescribed time. Consequently, both the reliability and validity of IVA-CPT might be influenced. It is necessary to assess ADHD by a more interesting test.

Virtual reality (VR) is a relatively new technology that simulates an imagined or real environment, thereby enabling users to immerse themselves in a virtual 3-dimensional environment. This technology is being increasingly adopted in educational, entertainment, military, and medical areas. Many studies have suggested that a neuropsychological assessment based on VR technology is more enjoyable and can gather better behavioral data of distractibility and attentional deficits, compared with traditional neuropsychological tests.^12,13 Negut et al¹⁴ reported the superior efficiency of VR-based CPT in children with ADHD, compared with traditional CPT. The studies on the quality of experience in real environments and VR showed that virtual experience is associated with Flow, a state of consciousness characterized by narrowed focus of attention, deep concentration.¹² Therefore, an increased number of studies have tried to introduce VR technology into neuropsychological assessment of ADHD in many countries.^15–17 The present study was conducted to explore the feasibility of VR-based assessment in school-aged children with ADHD, who suffered with learning problems. The validity of the VR test was systematically investigated by comparing outcomes of the VR test of children with ADHD with those of control children. Also, the relationship between outcomes of the VR test and other frequently-used measurements of ADHD was analyzed.

Methods

Participants

All participants were recruited from the school-aged (6–18 years) children attending the Children and Adolescents Mental Health Joint Clinic of The Affiliated Hospital of Hangzhou Normal University from January 2017 to September 2018. These children attended the clinic for bad academic achievements, and were selected depending on their and their parents’ willingness to participate in the study after they received a detailed explanation about it. The children who were firstly diagnosed with ADHD based on the Diagnostic Interview Schedule for Children Version V (DSM-V) were assigned into the ADHD group (see Table 1). The children who did not meet the criteria for any mental disorders were assigned into the Control group (see Table 1). The children with brain damage, neurological disorders, genetic disorders, substance dependence, epilepsy, an IQ of 70 or lower, or any other mental disorder reported during the personal history and anamnesis were excluded. All participants’ parents provided written informed consent to the content and ethics of this study. All protocols were conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Review Board of Hangzhou Normal University.

Table 1 The demography and VR test in the ADHD and control groups

VR test

The VR test is performed using the Virtual Reality Medical Center system (VRMC, Hangzhou Xinjing Technology Co., Ltd, Zhejiang, China). The total run-time performance is set by a computer and a high-end VR headset with 1,080×1,200 pixels per eye resolution, a 90 Hz refresh rate, and a 110 degrees field of view (see Figure 1). With this VR device, a high immersive interaction can be achieved due to its game-like interface system, various sensors, location trackers, and a data back-end system for data collection and statistical feedback. The VRMC system for the ADHD test is composed of three measurement scenes for ADHD, including Position Tracking (30 items), Stroop (41 items), and Object Recognition (60 items). The whole test should be completed in 20 minutes. The results of this test are presented using four indexes: correct items (the number of correct items), incorrect items (the number of incorrect items), accuracy (the percentage of correct items to total itemsx100), and total time (total testing time).

Figure 1 Examples of ADHD evaluation using a VR test.

IVA-CPT

The IVA-CPT was used to evaluate the ability to sustain attention and control responses on various auditory and vi sual stimuli (ages 6 years onwards). In recent years, it has frequently been used to assist the diagnosis of ADHD. The IVA-CPT contains many scales, of which the scales of auditory control, auditory attention, visual control, and visual attention were used in the present study.

Conventional questionnaire psychological test

ADHD-related behaviors were evaluated by the Conners Parent Rating Scale (CPRS), which contains 48 items on a 4-point scale. The six sub-scales of CPRS (impulsion/hyperactivity, conduct problems, cognitive problems, psychosomatic problems, anxiety, and ADHD index) were adapted to evaluate symptoms of ADHD. The Children’s Behavioral Checklist (CBCL, ages 6–18) was used to assess psychiatric symptoms according to parent ratings. It is a 113-item scale, in which eight sub-scales (attention problems, social problems, cognitive problems, anxiety/depression, somatic complaints, withdrawn, aggressive behavior, delinquent behavior) were adapted in the present study to present the symptoms of ADHD.

Statistics

All data analyses were conducted using SPSS statistical software version 19.0 (SPSS, Inc., Chicago, IL, USA) and Microsoft Excel. Data normality was assessed by the Kolmogorov-Smirnov test. Data showing a normal distribution was summarized and is expressed as mean±SD. The index of VR test was analyzed by a Student's t-test for pairwise comparison between the two groups, and the relationship between the VR test and each other measurement was analyzed by Pearson correlation analysis.

Results

The demography and the VR test in the ADHD group and control group

The comparative analysis results indicated that there is no significant difference on the age and sex ratios between the ADHD group and Control group. The accuracy and correct items in children with ADHD was significantly lower than that in children in the Control group, and the total time for children with ADHD was significantly higher than that for children in the Control group (P<0.05) (presented in Table 1).

Correlation analysis between VR test and IVA-CPT

Table 2 shows that incorrect items are negatively associated with scores of visual attention of IVA-CPT; the total time of the VR test is negatively associated with scores of both auditory and visual attention of IVA-CPT.

Table 2 The correlation between VR test and IVA-CPT

Correlation analysis between the VR test and Conners/CBCL

Table 3 indicates that incorrect items of the VR test were positively associated with the Hyperactivity index of CPRS. For the VR test and CBCL, Table 3 shows that correct items were negatively associated with the social problems index of the CBCL; incorrect items were positively associated with the scores of attention problems, social problems, cognitive problems, and aggressive behavior of CBCL.

Table 3 The correlation between VR test and CPRS/CBCL

Discussion

At present, the application of virtual reality technology to clinical practice is still in it's infancy. ADHD is a neurodevelopmental disorder, mainly characterized by high levels of inattention, hyperactivity, and impulsivity, which were objectively evaluated using a VR test in the present study. The correct items, incorrect items, and the accuracy rate of the VR test of the children with ADHD were significantly different to those of the children in the Control group. This suggested that the VR test could distinguish children with ADHD from children with learning problems. Similarly, several previous researches have also demonstrated the discriminant validity of VR for attention assessment in children with ADHD.^18,19

Moreover, the present study investigated the relationship between VR test results and IVA-CPT, which is frequently used as a measurement of attention and impulse control in children with ADHD. The level of visual attention on the IVA-CPT is associated with the correct items and the total time of the VR test. However, the level of auditory attention on the IVA-CP was only associated with the total time of the VR test. This suggests that the VR test adapted in the present study was sensitive to the attention function on visual information.

School-aged children with ADHD frequently suffer impairments in social, cognitive, academic, behavioral, and familial functioning, which are caused by inattention, hyperactivity, and impulsivity. These impairments have commonly been evaluated by altered scores of many sub-scales of the CBCL and CPRS.^20,21 In the present study, relationships between the results of a VR test and the conventional questionnaire psychological tests (CPRS and CBCL) were analyzed. The results indicated that the correct items and incorrect items of the VR test were associated with the levels of impulsion/hyperactivity and ADHD index found on CPRS. Gilboa et al²² reported a significant association between VR assessment variables (virtual classroom, VC) and the results of CPRS-R (Conners’ Parent Rating Scales-Revised: Short) among children with acquired brain injury. These authors suggested that VR assessment appears to be both a sensitive and ecologically valid assessment tool for use in the diagnosis of attention deficits. Furthermore, the relationships were analyzed between the results of the VR test and CBCL, which is commonly used in clinical assessment of ADHD symptoms. The score of attention problems on the CBCL was associated with the correct items, incorrect items, and accuracy of the VR test. This indicates that the indicators of the VR test can sensitively reflect the level of attention deficit, the core symptom of ADHD. It is notable that the correct items, the incorrect items, and the total time of the VR test were significantly associated with a score of social problems on the CBCL. For school-age children with ADHD, attention disorders, impulsive, and hyperactive behaviors also often lead to problems in social function, cognitive ability, learning ability, behavior, and family function.^23–26 In particular, ADHD children tend to be impulsive and hyperactive, and, therefore, often show inappropriate behavior which leads to a lack of social skills such as sharing and cooperation. The sub-scales of social problems, cognitive problems, aggressive behavior, and delinquent behavior of the CBCL can reflect the secondary symptoms of ADHD.^20,21 However, this study found no significant correlation between the results of the VR test and the CBCL’s sub-scale scores on cognitive problems, aggressive behavior, and delinquent behavior. Further research is needed to understand why.

This study is a preliminary investigation on the feasibility of applying VR technology in the auxiliary analysis diagnosis of ADHD. The VR test results in this study only include correct answer items, wrong answer items, and total answer time, and the indexes of the VR test are more closely related to visual attention. The content and indexes of the VR test still need to be expanded and improved. In addition, the sample size of the study also needs to be further expanded.

Conclusion

In conclusion, the present study investigated the feasibility and availability of the VR method for evaluating symptoms of ADHD or potentially for diagnosing ADHD in school-aged Chinese children. The results of the present study indicated that ADHD children can be identified from children with learning problems according to a VR test. It must be pointed out that the study is flawed. The VR test indicators adopted in the present study are relatively single, and mainly reflect visual attention. However, multiple VR test indicators should be used to evaluate the multidimensional symptoms of ADHD. This study is only a preliminary exploration on the feasibility of a VR test in the assessment of ADHD symptoms. Relevant VR testing software is still in the initial stage of development. Future studies are necessary to improve the VR test for ADHD.