How Accurately Does the Information on Motor Development Collected During Health Checkups for Infants Predict the Diagnosis of Neurodevelopmental Disorders? – A Bayesian Network Model-Based Study

Background

Neurodevelopmental disorders (NDDs) affect various aspects of development, such as cognition, motor ability, language, sociability, imagination, emotion, self-regulation, and learning, leading to various learning, behavioral and emotional problems. Gillberg¹ proposed the concept of Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations (ESSENCE) and suggested that concerns (parental or specialist) persisting for at least a few months in early childhood in areas of (1) general development, (2) motor development, (3) responses to sensory stimuli, (4) communication/language, (5) activity/impulsiveness, (6) attention, (7) social interaction, (8) behavior, (9) mood, (10) sleep, (11) feeding, and (12) history of consciousness disturbance should always be taken “seriously” as they might likely be early signs of NDD. ESSENCE comprises autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), tic syndrome (TS), speech and language disorder (SLD), developmental coordination disorder (DCD), intellectual developmental disorder (IDD), and borderline intellectual functioning (BIF). Previous studies have shown that if there are obvious concerns in two or more of the ESSENCE areas referred to above, a comprehensive neurodevelopmental assessment and follow-up should be undertaken.^2,3 Gillberg also emphasized that clinicians and researchers should always consider the coexistence/comorbidity of NDDs (if there is one, there is likely to be at least, one more) and the sharing of early signs (such that those signs might signal, eg, ADHD, ASD, DCD – one, two or three).^1,4

NDDs that are not detected at an early stage and are left without proper management are likely to have various psychiatric and mental health consequences.⁵ In addition, NDDs have been reported to be associated with physical problems, such as obesity, chronic pain, accidents, and premature death.^6–8 The detection of NDD signs as early as possible, even before a specific diagnosis is confirmed, is a significant public/community health issue.¹

Motor developmental problems are considered early signs of NDD,^9–16 and are assessed on one item in the ESSENCE-Questionnaire (ESSENCE-Q), which is used as a screening tool for ESSENCE.^2,17,18

In Japan, child health checkups provide meaningful NDD screening opportunities. Relevant studies and reports have been published for health checkups of children who are older than 18 months and younger than 24 months (the “18-month-checkup”) and for children who are older than three years and younger than four years (the “36-month-checkup”).^19,20 Most municipal governments in Japan offer health checkups for infants aged approximately 3–4 months (the “4-month-checkup”) and those aged about 9–10 months (the “10-month-checkup”). However, no studies on NDD screening have been conducted based on the data from these earlier health checkups. Motor developmental problems are relatively easy to recognize in these early health checkups for infants. If it is suggested that this is an early sign of NDD, it would be possible to initiate an appropriate intervention for NDD at an early stage, leading to an improved prognosis.

In the present study, we investigated to what extent early motor development problems predict a future diagnosis of NDDs using a Bayesian network model (BN). BN is a probabilistic graph representing a set of variables and their conditional dependencies using a directed acyclic graph (DAG). The nodes in the graph represent random variables. If an edge connects two nodes, it relates the probability of dependence from one node to the other. Each node is also assigned a Conditional Probability Table (CBT), recording its probability distribution. Using BN, the relationship between disease/disorder and clinical symptoms/signs can be expressed probabilistically.²¹ For example, if the probability of a diagnosis of NDD increases when information is given about an abnormality on a motor development item in the 4-month-checkup (posterior probability) compared to the probability when such information is not given (prior probability), it can be assumed that the item contributes to the diagnosis.

Methods

Subjects and “Explanatory Variables”

This is a general population-based retrospective cohort study of 501 children (256 boys and 245 girls) and their parents who participated in the 18- and 36-month checkups in two cities (Kami and Aki) in Kochi Prefecture between April 2014 and March 2015. In Kami City, 143 (75 boys and 68 girls) of 152 eligible children (79 boys and 73 girls) (participation rate, 94%) and 150 (74 boys and 76 girls) of 158 eligible children (77 boys and 81 girls) (95%) participated in the 18- and 36 month-checkup, respectively. In Aki City, 100 (54 boys and 46 girls) of 112 eligible children (63 boys and 49 girls) (89%) and 108 (53 boys and 55 girls) of 116 eligible children (54 boys and 62 girls) (93%) in the 18- and 36-month-checkup participated, respectively. Data on children’s motor development were collected with the parents’ written informed consent from child development records at the 4-, 10- and 18-month-checkups. The child development records included items based on interviews with mothers by public health nurses and records of direct observation by pediatricians. Published data on typical development, along with the Japanese version of the Denver Developmental Screening Test, were used, at both sites, as references to develop the items for interviews with mothers.

“Motor development abnormalities” were identified from (1) the nurse interview items about developmental milestones that the children had not achieved at the time of the interview with their mothers, and (2) items that the pediatricians, who performed neurological examinations during the health checkups, considered abnormal. Sex, birth weight, gestational age at birth and newborn status (the Apgar score one minute and five minutes after birth) are known to be associated with a diagnosis of NDD,²² hence they were also included as explanatory variables in the analysis.

All explanatory variables are listed in Table 1. Note that the codes in parentheses after an item distinguish the health checkups at different months (C means checkup, and the numbers are age in months) and items (M for items based on interviews with mothers and P for items based on observation by pediatricians, followed by item number; item numbers are in the order listed in the records). The dependent variable was the diagnosis of NDD (see below).

Table 1 Basic Attributes and Motor Development Items Used as Explanatory Variables

Diagnosis of NDD

Pediatricians with extensive experience in neurodevelopmental assessment and NDDs examined each child at the final step of the 18- and 36-month-checkup. Furthermore, they detailed all records on child development since birth and reviewed the ESSENCE-Q records by the mothers, public health nurses, and psychologists. They evaluated these data to determine if more detailed neurodevelopmental examinations would be considered indicated. If deemed necessary, children underwent a secondary health checkup by a child psychiatrist with extensive clinical experience with young children. At the secondary checkup, the psychiatrist reviewed all records in detail, interviewed the mothers, and observed the children’s behaviors according to the ESSENCE-Q structure. Based on the assessment results, children suspected of NDD were referred to a neurodevelopmental clinic.

All developmental areas were assessed at the neurodevelopmental clinic to confirm the diagnosis. The mothers completed the ESSENCE-Q again, and specialists completed the ESSENCE-Q based on the observation of behaviors of the children and interviews with the mothers; in addition to the information recorded at the health checkups, all developmental areas under the ESSENCE umbrella, ie, general development, motor development, reaction to sensory stimuli, communication/language, activity/impulsiveness, social interrelatedness, attention, behavior, mood, sleep, eating and history of consciousness disturbance, were evaluated.^2,3 The Kyoto Scale of Psychological Development 2001 (KSPD2001),²³ the psychometric property comparable to Bayley Scales of Infant Development second edition (BSID-II), was used to evaluate cognitive function. Diagnostic Interview for Social and Communication Disorders (DISCO-10)²⁴ was used to assess the development of sociability and communication. The Japanese version of the Strengths and Difficulties Questionnaire (SDQ) for parents,²⁵ a tool for evaluating the mental health and development in children, such as behavior, emotion, and interpersonal relationships, in terms of the effects and duration of symptoms, distress in children, disabilities in various settings, burdens on others and other aspects, was also used. Assessments with the ESSENCE-Q, KSPD2001, and SDQ were repeated every six months. In addition, ADHD-RS²⁶ and ASSQ²⁷ were used as regular assessment tools for children with hyperactivity, impulsivity, and inattention problems and those suspected to have ASD, respectively. To diagnose DCD, occupational therapists and the child psychiatrist assessed children’s motor-perceptual performance based on clinical observations of behaviors such as standing, walking, throwing, and retrieving a ball. After the initial evaluation, the examinations were repeated every 1–3 months. Specialists continued unstructured clinical observations consistently over the study period. In addition, interviews with and reports from parents and nursery school/kindergarten teachers were conducted in almost every examination. The diagnosis was reviewed every six months; during every review, all the information obtained up to that point was checked and re-evaluated. The regular assessments outlined above were repeated until the child was six. After these procedures were completed, the diagnoses described were based on the diagnostic criteria for pediatric mental and neurodevelopmental disorders in the International Statistical Classification of Diseases and Related Health Problems-10/Diagnostic and Statistical Manual of Mental Disorders-IV. The diagnoses included in the analysis were ASD, ADHD, DCD, SLD, BIF/IDD, and others (including tics, reactive attachment disorder, and social anxiety disorder).

Analysis

All children were included except those with no data on motor development at all. The number of children with data available for any of the checkups was 467. Table 2 shows the numbers of eligible children, participants, and participants for whom health checkup data were available. The analysis was performed according to the steps shown in Figure 1.

Table 2 Numbers of Eligible Children, Participants, and Participants for Whom Health Checkup Data Were Available

Figure 1 Flow chart showing the analysis process. Abbreviations: ROC, receiver operating characteristic; PPV, positive predictive value; NPV, negative predictive value; UI, utility index.

Before a BN was constructed, a test of independence (Fisher’s exact test) was conducted to analyze the relationships of the diagnosis of NDD with sex, birth weight, gestational age at birth, newborn status (the Apgar score one minute and five minutes after birth), and motor development-related items. The test’s purpose was to compare items found to have significant relationships with the diagnosis and items included in the BN and ensure that the network included the necessary and sufficient items. The significance level was set at p< 0.05. Items for abnormalities found in no more than one child were excluded even if p < 0.05 because the statistical power was considered insufficient.

Sex and all developmental items rated by mothers regarding motor development, NDD diagnosis, and diagnosis type (BIF/IDD, ASD, ADHD, DCD, SLD, others) were taken as candidate nodes for the BN. The bnlearn package in R statistical software was used to construct the DAG structure and derive the CPT. Bnlearn was configured so that only explanatory variable nodes that were found to be associated with any diagnosis type by chi-square test (p < 0.05) were included in the DAG.

Evaluation of the Accuracy of NDD Diagnosis Prediction

The predictive accuracy of the derived BN for the diagnosis of NDD was evaluated as follows;

1. The posterior probabilities of being positive for NDD diagnosis were determined using the BN.
2. These posterior probabilities were compared with the actual diagnosis results, and the sensitivity and specificity obtained when a certain posterior probability was used as a cut-off value were shown graphically as a receiver operating characteristic (ROC) curve.
3. The accuracy of prediction of NDD diagnosis based on posterior probabilities determined using the BN was evaluated using the area under the ROC curve (AUC). As a screening test, AUC > 0.7 and AUC > 0.9 indicate acceptable and very high levels of accuracy, respectively.^28–30
4. The posterior probability (the optimal cut-off value) yielding the maximum Youden Index (sensitivity + specificity – 1) is determined with the ROC curve, and the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and utility index (UI) were computed. The UI is a measure of the clinical usefulness of screening; positive UI (UI+) calculated as the sensitivity × PPV and negative UI (UI-) calculated as the specificity × NPV are used to evaluate the “rule-in accuracy” and the “rule-out accuracy”, respectively. The evaluation criteria are as follows: poor, UI± < 0.2; fair, 0.2 ≤ UI± < 0.4; moderate, 0.4 ≤ UI± < 0.6; good, 0.6 ≤ UI± < 0.8 and very good, UI± ≥ 0.8.^3,31
5. We searched for a combination of motor development abnormalities for which the posterior probability exceeded the optimal cut-off value described above.

Ethical Considerations

The study was conducted after approval from the Institutional Review Board of the Kochi Prefectural Medical and Welfare Centre for Handicapped (No. 24–473). Written informed consent was obtained from the subjects’ parents/caregivers. This study was conducted according to the principles of the Declaration of Helsinki.

Results

Table 3 shows percentage of those with normal and abnormal values for each variable. Considering the age in months at the time of the checkups, the results were largely consistent with those of previous studies.^32,33

Table 3 Percentage of Those with Normal and Abnormal Values for Each Variable

Diagnoses

The overall prevalence of diagnosed NDD rates in the 18-month-and 36-month-old populations was 9.1% (9.1% in Kami City and 9.0% in Aki City) and 13.6% (11.0% and 16.7%), respectively. The male: female ratio was 3.4:1 in the 18-month-old population and 2.2:1 in the 3-year-old population.

Table 4 shows the diagnoses of the children with NDD.

Table 4 The Diagnoses of the Children with NDD

Results from the BN

Figure 2 shows the DAG constructed using sex, health checkup results, diagnostic data, and diagnosed NDD of the subjects. Among the 10-month-checkup items, C10_M1(“Does the child crawl?”) was connected by arcs to C10_M2(“Does the child grasp things to pull him/herself up to stand independently?”), C10_M3(“Does the child cruise?”), C10_M5(“Does the child walk when you hold his/her hands to guide him?”), and C10_P1(“pulling to stand”) because they had associations, and C10_P1 was connected to BIF/IDD because the former showed the strongest association with the latter. Among 18-month-checkup items, C18_M1(“Can the child walk well?”) was connected by arcs to C18_M2(“Can the child crouch down to pick up objects?”) and C18_P1(“walking”) to represent their associations, and C18_P1 and C18_M2 were connected to BIF/IDD and DCD, respectively. Associations were also found among the NDD diagnosed, and arcs connected DCD and ADHD to others and ASD, respectively. The same data were used to derive the CPT of the BN. As an example, the CPT of the BIF/IDD node is shown in Table 5. This table shows the probabilities of being negative or positive for BIF/IDD when C10_P1 and C18_P1 values are given. To evaluate the accuracy of NDD prediction by this BN, we computed the probabilities of NDD predicted from sex and health checkup results of the subjects using the BN and prepared a ROC curve, as shown in Figure 3. Notably, 98 children (53 boys and 45 girls) who had two or more missing data among the health checkup items included in the BN were excluded from the accuracy evaluation to prepare the ROC curve. This curve had an AUC of 0.735 (95% CI: 0.648–0.821). The posterior probability with the maximal Youden Index (optimal cut-off value) was 0.138; at this value, the sensitivity, specificity, PPV, NPV, UI+, and UI- were 0.619 (0.472–0.766), 0.761 (0.715–0.808), 0.250 (0.167–0.333), 0.940 (0.911–0.968), 0.155 (0.015–0.294) and 0.715 (0.679–0.752), respectively. All combinations of sex and health checkup results with posterior probabilities exceeding this cut-off value are listed in Table 6.

Figure 2 The constructed Bayesian network (BN). Node names starting with C denote the following health checkup items (M, items based on interviews with mothers; P, neurological findings of examinations by pediatricians). Abbreviations: ADHD, attention-deficit/hyperactivity disorder; ASD, autism spectrum disorder; BIF, borderline intellectual functioning; DCD, developmental coordination disorder; IDD, intellectual developmental disorder; NDD, neurodevelopmental disorders; SLD, speech and language disorder; C4_M3, 4-month-old-checkup “Does the child play with his/her hands close together in front of him/her?”; C10_M1, 10-month-checkup “Does the child crawl?”; C10_M2, 10-month-checkup “Does the child grasp things to pull him/herself up to stand independently?”; C10_M3, 10-month-checkup “Does the child cruise?”; C10_M5, 10-month-checkup “Does the child walk when you hold his/her hands to guide him?”; C10_P1, 10-month-checkup “Pulling to stand”; C18_M1, 18-month-checkup “Can the child walk well?”; C18_M2, 18-month-checkup “Can the child crouch down to pick up objects?”; C18_P1, 18-month-checkup “Walking”. Note: *As an example, the conditional probability table (CPT) of the BIF/IDD node corresponding to dependencies in an area enclosed by dashed lines is shown in Table 5.

Figure 3 Receiver operating characteristic curve when the constructed Bayesian network is applied to the analysis set (N = 369). The value on the curve is the cutoff value (specificity, sensitivity) at the maximal Youden Index.

Table 5 Conditional Probability Table for Borderline Intellectual Functioning/Intellectual Developmental Disorder

Discussion

The current study showed that the presence of motor development problems early in development increases the probability of a future diagnosis of any NDD, which supports the need for an ESSENCE perspective when looking at the course of delayed motor development from a different angle to previous studies.

Based on the BN results, the items associated with DCD (“Can the child walk well?” (C18_M1) and “Can the child crouch down to pick up objects?” (C18_M2)) may suggest that posture transition from walking and standing, and the development of balance function underlie the development of coordinated movement and motor dexterity. Endorsed items that might increase the posterior probability of being diagnosed with IDD/BIF were C18_M1, C18_P1 (“walking”), and all items interviewed from mothers in the 10-month-checkup except for C10_M4 (“Does the child stand on his/her own?”). This is in line with previous research suggesting an association between IDD and delayed motor development before the age of one.³⁴ In summary, the results could indicate that general intellectual development assessments should be considered necessary whenever motor development abnormalities are suspected at the 10-month-checkup.

The item associated with SLD (“Does the child play with his/her hands close together in front of him/her?” (C4_M3)) may suggest an association between motor control development and language development, as has been demonstrated in previous research. However, the diagnosis of ASD was found to have no association with delayed motor development. In a systematic review, Lim et al³⁵ reported a result of a meta-analysis showing a significant association between the diagnosis of ASD and abnormal motor development in the first 18 months of life. However, there were variations among the different studies. In most studies included in their review, structured scales with established validity and reliability were used to evaluate motor development. The motor development-related items used in the health checkups here have not been standardized, which may have influenced the outcome. The later diagnosis of ADHD was also found to have no association with delayed motor development.

Further studies will be necessary to establish a clearer picture. Of the 11 cases diagnosed with SLD, three had SLD only, seven had SLD and ADHD, and one had SLD and ASD. Speculatively, in these cases of SLD and other NDD comorbidities, the association between motor developmental problems and ADHD or ASD may be mediated by language delay.³⁶

The ROC curve had an AUC of 0.735, which means that NDD screening based on problems in motor development had acceptable, albeit much less than perfect, accuracy. However, the sensitivity with the optimum cut-off value was 0.619, which is not sufficiently high for a screening tool. The UI- value was 0.715, and the rule-out accuracy was also inadequate. This indicates that, while an abnormality among the motor development-related items in a health checkup suggests that the child would be at risk of an NDD and requires careful follow-up, the absence of abnormalities among the motor development-related items does not necessarily mean that the child is free of the risk of NDD and still may require examinations in all areas of development included under the ESSENCE umbrella. Given the paucity of data and a relatively high frequency of missing values, combinations giving the posterior probabilities exceeding the cut-off value shown in Table 6 should be carefully interpreted. Nevertheless, the following finding that when motor development is compromised under the age of one, careful follow-up on various developmental aspects, including non-motor aspects, would still be considered valid.

Table 6 Combinations Where the Posterior Probability of Neurodevelopmental Disorders is Above the Cut-Off Value

The novelty and strength of this study is the use of BNs to quantify the effects of motor development problems on the diagnosis of NDD in the form of the posterior probability. However, a considerable number of missing values in the data used in this study may have had a major impact on the results. Given that the data used were originally not recorded for research, it is possible that no values were recorded just because no abnormalities were found in these items. Nonetheless, we had to treat them as missing values because no data was recorded.

Conclusion

The study showed that motor development problems early in development increased the probability of a future diagnosis of any of the NDDs, but was not sufficiently sensitive as a screening tool for NDDs. The need for a broad, holistic ESSENCE perspective when looking at the course of motor development problems was emphasized.