Preferences in the Design and Delivery of Neurodevelopmental Follow-Up Care for Children: A Systematic Review of Discrete Choice Experiments

Introduction

Neurodevelopmental disorders encompass a range of conditions that affect early development and functioning of the brain. Difficulties may arise in social, cognitive, or emotional domains and include diagnoses such as attention-deficit/hyperactive disorder (ADHD) and autism spectrum disorder (ASD).¹ Consequently, neurodevelopmental disorders can cause significant morbidity in children with subsequent impact on families and incur substantial costs to healthcare and education systems.² A survey from the United States reported that 15% of children aged 3 to 17 years were affected by neurodevelopmental disorders.³ Along with genetic and environmental factors, critical illnesses early in life, including congenital heart disease and prematurity, are also a known risk factor for neurodevelopmental delays.^4–8 Early detection of neurodevelopmental delay is important for timely diagnosis and intervention, and it is therefore important to understand the preferences of parents and clinicians for engaging with neurodevelopmental services and follow-up care.^8–10 The use of the term “neurodevelopmental follow-up care” in this manuscript refers to the care provided to children and their families over time after diagnosing the neurodevelopmental disorder or delay in order to monitor the medical and developmental outcomes and therefore provide suitable support.^11,12 Knowing what parents and clinicians prefer, in terms of neurodevelopmental follow-up care, is likely to aid in the design of services to enhance uptake and acceptability.

There are different methods of understanding patient preferences, both qualitative and quantitative. Some studies have used surveys, interviews, and focus groups to identify preferences in neurodevelopmental follow-up care;^13–17 however, the experimental design and analysis methods in these studies were not robust for quantifying preferences in this context. Regarding care for children with neurodevelopmental disorders, there may be many important attributes and the weighted preferences for each of these cannot be readily captured and disentangled using these methods for the purpose of informing health service design. Consequently, quantitative choice methods may be suitable to use in this context as they may be more reliable at predicting authentic behavior^18,19 and can provide a robust method to capture and quantify the most relevant preferences.

Discrete choice experiment (DCE) is a quantitative choice method, where individuals are presented several hypothetical health scenarios (choice sets), each containing several alternatives with different attributes for the individuals to choose from.²⁰ A DCE can be designed using the following key steps: identifying whether an experiment is labelled or unlabelled; identifying and finalising attributes; identifying the attribute levels; identifying the number of choice sets in experimental design; selecting a suitable experimental design strategy; conducting a pilot study; conducting the main study; and analysing the results using a suitable analysis method.^21,22 DCE is considered a robust technique that uses the ‘framework of rational choice’, which infers that people are likely to select the option that will yield maximum benefit or utility when given a set of choices. In this context, for an individual i, making a choice j, utility (U) is described as a function of characteristics of the choices (Zj) and the characteristics of the people making the choice (Xi) and an error term denoting unobserved attributes of choices and individuals (eij).²³

In this equation, F is assumed to be a linear function (hence this is also known as linear random utility model) and can be represented as:

where β and γ are the utilities (also known as parameter estimates) associated with the choice features and interaction of choice features and individual features, respectively.

DCE, being a stated preference technique, is useful in determining weighted preferences of various stakeholders, including patients, parents and clinicians in a healthcare decision-making situation, which is not possible via other common methods such as interviews, focus groups, and surveys.²⁴ Additionally, they estimate monetary and non-monetary values, for example, willingness to pay (WTP), willingness to accept (WTA), and probability scores,^21,25 which can be beneficial to include in a neurodevelopmental follow-up care as it may help in better understanding the approximate expenditure for future follow-up care.

Although the application of DCEs in fields of healthcare is relatively new, the use of DCEs in studies of neurodevelopmental follow-up care has recently increased. However, methodological limitations in experimental design and analysis methods, as well as sub-optimal reporting in publications have been observed in this field.^26–29 In order to advance the quality and strength of conclusions that can be drawn from DCEs in this field, it is important to understand the methodological features that have been used, as well as identify potential neurodevelopmental care attributes appropriate for use in future preference studies. This systematic review aimed to examine how DCEs have been designed and delivered in studies for neurodevelopmental follow-up care of children, and to identify preferred attributes of these models of care for the purpose of informing neurodevelopmental follow-up care health service design.

Methods

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses report (PRISMA) guidelines.³⁰ A protocol for this review was registered with PROSPERO (Record ID = CRD42022325685).

Data Sources, Search Strategy, and Screening

On 08 February 2022, we searched PubMed, Embase, CINAHL, and Scopus databases. The search strategy was developed using existing literature and expert assistance from researchers experienced with systematic reviews in this field. A combination of Medical Subject Heading (MeSH) terms and keywords were used for groups of terms related to “children”, “patient preference” and “neurodevelopmental follow-up care”. Search syntax and count of returned titles for each string for all databases are provided in Online Resource 1 (Tables S1–S6). An advanced Google search was also conducted with the first 20 pages screened; however, no additional relevant papers were identified.

All records obtained through the searches were uploaded to EndNote version 20³¹ and deduplicated prior to screening. These records were uploaded to Rayyan,³² a web-based systematic review management platform, for screening. Two authors independently completed the title and abstract screening of all the records identified from the databases (PS and UH). Any disagreements were resolved by consulting a third author (SK). After the title and abstract screening, full-text publications were independently screened by another pair of reviewers (PS and SS) and disagreements were resolved by consulting SK.

Study Selection

Original research studies were included if (1) the population was children, infants or adolescents, (2) a DCE methodology was used, and (3) the context was neurodevelopmental follow-up care or programs for neurodevelopmental disorders (for example, ADHD, ASD, dyslexia, and cerebral palsy). Studies using any other preference methods including surveys and interviews were excluded. Publications were excluded if they were reviews, meta-analysis, case studies, case reports, conference abstracts, letters to editor, guidelines, and commentaries. Non-human studies were also excluded. No limits were applied on publication date or language.

Data Extraction

Two authors were involved in data extraction for included studies, one extracted data (PS) and the other checked the extracted data (SS) for all studies. Any unresolved disagreements were resolved by consulting a third author (SK). Missing data were reported for the final analysis and interpretation. All information was extracted into a Microsoft Excel Workbook.³³

The following data were extracted: publication details (author name, year, country, title, and journal name); study details (aims/objectives, study design, dates of recruitment, primary and secondary outcome); neurodevelopmental disorder and children’s details (type of neurodevelopmental disorder/delay, inclusion criteria for children in the study, age, and sample size); respondent details (respondent type, age, sample size, response rate, and recruitment strategy); DCE methodological details (for example, attributes included for a model of care, experimental design, analysis method, and software used), results, conclusions, limitations, and the advantages and disadvantages of the studies.

Risk of Bias Assessment

We assessed risk of bias in the included studies with a widely used checklist initially created by Lancsar and Louviere in 2008,²⁴ and modified by Mandeville in 2014.³⁴ Domains include choice task design, experimental design, conduct, and analysis. Two reviewers (PS and LV) independently applied the checklist to each study and recorded judgements for each item. Any unresolved disagreements were resolved by consulting a third author (SK).

Data Analysis

The data were summarised descriptively and tabulated for each DCE study included in the review. A narrative synthesis was completed for demographic characteristics, attribute selection process, experimental design, analysis method, and the identified attributes.

Results

A total of 6618 papers were identified; after removing duplicates, 6361 remained. During screening, most studies were excluded as they were not performed in the context of neurodevelopment follow-up or were not focussed on children. From the 49 publications that progressed to full-text screening, 16 studies were included in the review.^{26–29,35–46} The screening and selection process is reported in PRISMA diagram in Figure 1 Preferred Reporting Items for Systematic Reviews and Meta-Analyses report.

Figure 1 Preferred Reporting Items for Systematic Reviews and Meta-Analyses report.

Characteristics of Included Studies

Table 1 summarises the important study characteristics of the included DCE publications. Studies were conducted from the years 2008 to 2022 mostly in the United States of America (n=5)^{28,29,36,41,42} and Canada (n=3)^26,27,40 or both (n=2),^44,45 followed by European countries (n=5),^{37–39,43,46} and Australia (n=1).³⁵ ADHD was the most frequently studied neurodevelopmental disorder (n=7),^{37–39,42,44–46} followed by children’s mental health (CMH) (n=2)^26,27 and ASD (n=1).²⁹ Parents (n=9)^{27,37–40,42,44–46} or caregivers (n=4)^28,35,36,41 were the main DCE respondent type. However, two studies also reported professionals (including child and youth worker, social workers, psychologist, psychiatrist, early childhood education providers, nurses)^26,43 and one study reported teachers as respondents.²⁹ The number of participants in studies ranged from 38 to 1194 with the median sample size being 241.

Table 1 Demographic Characteristics of the Included DCE Studies (n=16)

Methodological Characteristics of Included DCE Studies

Table 2 describes the attribute selection, choice set generation, and experimental designs used in each included DCE study.

Table 2 Characteristics of the Attribute Selection Process, Choice Set Generation, and Experimental Designs of the Included DCE Studies (n=16)

Attribute Selection Process

Authors used a variety of methods to select attributes including focus group discussions (n=8) and literature reviews (n=6) and sometimes a combination of these and other methods. Studies included two (n=8), three (n=7), or five (n=1) alternatives. The number of attributes included in studies ranged from 3 to 20 and the maximum levels included were 5. Attributes can either be quantitative (numerical) or qualitative (categorical). Usually, categorical variables need to be coded to use in utility functions.⁴⁷ Most studies in this review lacked reporting on the method used to code categorical variables (n=11). Of those that did report this, two used dummy coding^40,42 and two used effects coding.^28,36 Only two studies reported utility specifications, which stated using alternative specific constant.^35,43

Generating Choice Sets

In this review, there were more unlabelled (n=13) than labelled choice sets (n=3). Most studies used a forced choice approach (n=14), where respondents have no choice but to answer that choice set. Only one study used an opt-out approach,⁴⁰ and one study did not report this information.³⁸ The total number of choice sets in included studies ranged from 8 to 609 but the number of choice sets presented per participant ranged from 3 to 30. None of the studies explicitly stated whether they used a homogenous (respondents are shown the same choice sets) or heterogeneous (respondents are shown only a subset of choice sets by dividing into blocks or groups) design matrix. However, most studies did not block their designs (n=11); therefore, we assumed that they used a homogenous design matrix.

Experimental Design Strategy

In this review, all studies used fractional factorial design except one that did not state the type of experimental design.²⁹ Orthogonal design (n=5) was a common design: one study performed an orthogonal array³⁹ and one an orthogonal balance.³⁶ Efficient designs were also identified (n=5), of which three were D-efficient designs and two were Bayesian D-efficient. Of the Bayesian D-efficient designs, one study used a Modified Federov algorithm and 500 Halton draws³⁵ and the other used a Hierarchical Bayesian algorithm with 2000 draws (type of draws was not stated).⁴⁰ Six studies did not report which fractional factorial category they used.

Study Type and DCE Administration

In this review, half of the studies conducted a pilot study, and the other half did not. One of the studies included in the review was itself a pilot study.³⁶ Only two studies used priors in this review,^35,41 while the others did not state using or not using them. Arora et al obtained prior values of +0.1 and −0.1 via existing literature³⁵ and Tsai et al used zero valued priors.⁴¹ Arora et al also reported that their sample was obtained from a broader clinical trial while the remaining studies did not mention any information on sample size. Most DCE surveys were administered online or via email (n=9), one was paper based, few were both (n=5), and one was via both telephone and email.

Analysis Methods in Included DCE Studies

The analysis methods used in each DCE are presented in Table 3. Various models were used in the DCE studies (Figure 2 Types of analysis methods identified in the included studies [n = 16]). Most of the studies used a single model (n=13) and the rest used multiple models. Of those that used one model, mixed logit was common (n=6), followed by multinomial logit (MNL) (n=3), and random-effects logit (n=2). Studies with multiple models used mixed logit and latent class analysis (LCA) (n=2) and MNL and LCA (n=1). Two of the mixed logit model studies extended their model specification to generalised MNL (GMNL)³⁵ and heteroskedastic MNL,⁴¹ which makes different assumptions about parameter distributions. For model estimation, only five studies have stated using Hierarchical Bayes estimation method.^{26,27,42,44,45} While it can be assumed that others have used maximum likelihood estimation, this was not reported explicitly. Of those studies that used LCA, classes were selected using log-likelihood, Akaike information criterion (AIC) or Bayesian information criterion (BIC). Preference heterogeneity was reported in 10 studies. Studies accounted for explained, unexplained, and scale heterogeneity. Only three studies performed monetary evaluations and reported willingness to pay (WTP) (n=2)^37,40 and willingness to accept (WTA) (n=1).³⁵

Table 3 Characteristics of the Analysis Methods and Most Preferred Attributes Identified in the Included DCE Studies (n=16)

Figure 2 Types of analysis methods identified in the included studies [n = 16]. Abbreviations: MNL, multinomial logit; LCA, latent class analysis.

Outcomes of Included DCE Studies: Preferred Attributes

In DCE studies, the attributes preferred by parents or caregivers and clinicians differed from each other. For parents and caregivers, the social, behavioral, and emotional situation of the child were important. Effectiveness and side effects of medication, receiving a genetic diagnosis, lesser waiting time, and spending less income were also identified as important attributes by parents. Individual and group parent training was highlighted as important by some parents. One study indicated that parents preferred treatments with behavior therapy and the desire to avoid medication.⁴² For clinicians, a child’s controlling behavior, ability to self-care, decision-making, and future communication and expression skills were important attributes. A study also found that professionals prefer active learning materials with parenting groups and therapist coaching calls.²⁶ Table 3 highlights only the most important attributes (as a result of model outcome) in DCE studies. The range of attributes initially included by authors in all of the studies is presented in Online Resource 2 (Table S7).

Risk of Bias and Quality of Included Studies

An assessment of the reporting quality and risk of bias of included studies is provided in Table 4. There were high risks in the choice task design specially in the inclusion of an opt-out or status quo option or justification of forced choice. Not justifying why a forced choice was included in the choice set may obligate the participants to choose something they are not interested in. Moderate to high risks were also identified in the experimental design of the study which may hinder with the neurodevelopment follow-up care outcome as the choice tasks may be inadequately designed. The conduct of discrete choice experiments was typically of low risk with an exception in the ‘response rate sufficient to minimise response bias’ section. Studies either had a low response rate or did not report it at all. A high response rate may ensure better generalisability of outcomes. The analysis of discrete choice experiments was also typically of low risk. Econometric models selected were particularly appropriate for choice task design.

Table 4 Quality Assessment of the Included DCE Studies (n=16)

Discussion

The primary aim of this systematic review was to synthesize the methods and findings of DCE studies conducted in the context of neurodevelopmental follow-up for children. Our review identified that most studies had been conducted over the past decade. This aligns with a literature review in 2019 reporting a 20% increase in DCE use over the preceding 10 years.²⁵ Most included studies focussed on ADHD and ASD which are the two most common neurodevelopmental disorders observed in childhood.^1,48 Overall, the findings of this review suggested variability in the methods used by the studies and highlighted some important attributes from the perspective of parents and carers. Most importantly, the design of a DCE is crucial to the conduct and quality of the study, and it includes several steps that require attention to detail. Here, we discuss the key shortcomings of the DCE studies included in this review and suggest opportunities for improvement.

Most studies identified in this review were unlabelled. Even though the research question may dictate whether an experiment is labelled or unlabelled, evidence suggests that a participant’s choice may sometimes be influenced by labels when they are present.^49,50 The utility function (described as the measurement of consumers’ preferences) for an unlabelled experiment is identical or generic, but for a labelled experiment the utility function should be specified.⁵¹ Two out of three labelled studies in this review used label-specific constants. The importance of using utility functions accurately has been highlighted in previous research as it is crucial to identify the relevant importance of an attribute over other, and in turn, better understand respondents’ behavioral responses and satisfaction with the overall model.^21,51,52

This review reported four studies conducting a partial profile,^26,27,44,45 which means they showed only a subset of attributes in each choice task due to large number of attributes. Evidence suggests that there is significant impact of the number of alternatives, attributes and levels shown to the respondents in a DCE on their behavioral responses.^53–55 A smaller number of attributes reduce cognitive burden on respondents, although quantity may not be as important as the relevance of the attribute.⁵⁶ Consequently, partial profiling may be beneficial in studies comprising a large number of attributes and is an important methodological consideration for future studies in this field.

Most studies did not report blocking their experimental design and are likely to have used a homogenous design. Prior evidence indicates that heterogeneous designs are advantageous because they provide more information and allow blocking that may reduce cognitive burden for respondents.⁵⁷ However, a homogeneous design may be appropriate if few parameters are to be estimated.⁵⁸ Seven studies in this review used a large number of choice sets.^{26,27,35,39,42,44,45} Although the number of choice sets is typically determined by the total number of parameters to be estimated in the choice model, there has been inconsistency in prior recommendations regarding the number needed. Hensher et al suggested using 4 to 16 choice tasks.⁵⁹ This is in contrast to studies that have demonstrated the number of choice sets may have minimal impact on findings.^60,61 Nonetheless, regardless of the methodological nuance required to address specific research questions, potential fatigue caused to the respondents remains a key consideration when designing studies in this field.

Design strategy was typically not clearly reported in the included studies. This limited our ability to draw strong conclusions regarding the most appropriate approach for conducting neurodevelopmental follow-up DCEs. On the one hand, efficient designs may be better in that they capture maximum information, calculate reliable parameter estimates with smaller sample sizes, and can be used if any attribute levels are unusual (unrealistic or impossible) and dominant.^62–64 On the other hand, orthogonal designs may be better at encompassing the attribute space, and are less technically demanding to conduct.⁶⁵ However, in the studies included in this review, capturing maximum and reliable information seemed to be prioritised in this context of investigating preferences for neurodevelopmental follow-up care.

Half of the studies identified in the review conducted pilot studies and only two studies reported information on priors. For the remaining studies, this is a notable limitation as there is no indication of accuracy regarding how the parameter values were obtained. Previous studies suggest that conducting a pilot study is important for generating feedback about the DCE and to obtain parameter priors so that a better design and impression of likely parameter estimates can be obtained for the main study.^66,67 Moreover, if informative priors are estimated, this will assist with minimum required sample size calculations.^66,68

The analysis methods used by DCEs in this review each have strengths and limitations. Mixed logit model was a common analysis method among studies in this review, followed by MNL. MNL (also known as conditional logit) has been used and modified by many researchers since its introduction in 1973.⁶⁹ However, due to advancement in software packages and increasing demand, more variations and complexities were added in estimating models which likely contributed to mixed logit and latent class analysis models receiving more attention in the literature.⁷⁰ Furthermore, MNL accounts for explained preference heterogeneity; and mixed logit, random-effects logit, and latent class models account for unexplained preference heterogeneity. Preference heterogeneity refers to different respondents having different preferences in the same choice scenario⁷¹ which a recent study suggests is important in health-related DCEs.⁷¹ Preference heterogeneity was reported in 10 studies in this review, two of which also reported scale heterogeneity (observed in scale parameters).⁷² This indicates that the authors acknowledged different behaviors may influence neurodevelopmental follow-up care.

Implications for Future Research

The nature of the research question as well as the quality and quantity of data are influential and interdependent considerations when determining the most appropriate experimental DCE design.²¹ This review has highlighted opportunity for methodological improvements at all stages of the design and analysis process to improve robustness of DCEs in this field. During the initial design stage of a DCE, careful consideration of the attribute selection process, generation of choice sets, and strategising a suitable experimental design among future studies in the field is likely to aid in optimising the accuracy and informativeness of findings. Similarly, appropriate prior selection and sample size calculation are key methodological features that should be purposefully devised. It is also important for future studies to consider response behaviors that may be related to unexplained heterogeneity that can be accounted for through appropriate analyses including mixed logit or latent class models. More monetary value calculations could help in building better health service models and offer stakeholders an understanding about an estimate expenditure for follow-up care.

Strengths and Limitations of the Review

Although this review considered a broad scope of literature with no date or language limitations, which may be considered a strength, the included publications required an appropriate level of methodological description to achieve the aim of this review. This meant that some publication types, including conference abstracts, case studies, and letters to editor, were considered unsuitable and excluded from the review and consequently findings from this review are not inclusive of information reported in other publication forms. While this was appropriate for addressing the intended study aims, it is noteworthy that among the included journal manuscripts unclear reporting still often contributed to uncertainty in our understanding of some important components of DCE methodology used. Due to the limited number of studies and heterogenous sample characteristics across diagnostic sub-groups, it was also not possible for separate analyses to be completed across sub-groups of neurodevelopmental disorders. Rather, the scope of the present review was limited to providing an overview of preferences for informing a healthcare service design perspective, that is not necessarily neurodevelopment diagnosis specific, regarding overall approaches to neurodevelopment-related follow-up care. Due to the nature of neurodevelopmental disorders and individual circumstances, differences in preferences may exist both among consumers within the same diagnosis category and between diagnostic groups. However, health system design internationally has not typically involved the establishment of different health systems and services for individual diagnostic groups, but rather services that provide neurodevelopmental follow-up are typically designed to provide services for a range of at-risk children with or without specific diagnoses at the time of entering the service. While we have tried to be inclusive of a range of diagnosis and disorders in the scope of searches for this study, the available literature was not inclusive of all diagnostic groups that may access services for neurodevelopmental follow-up care. Consequently, while findings addressed the study aims regarding the design and delivery of DCEs examining neurodevelopmental care of children and identified the preferred attributes that have been reported, the findings arising from studies included in this review should not be interpreted as representing the preferences of all consumers and families who may access neurodevelopmental follow-up care.

Conclusion

In summary, several aspects of DCEs were assessed in neurodevelopmental follow-up care. DCE is an appropriate preference method as it allows the estimation of weighted preferences, uncertainties, preference heterogeneity, the relevance of an attribute over another, and offers significant preference information beneficial to patient-centred care. The attributes identified from studies in this review may increase awareness of important components of future follow-up programs. Furthermore, when discussing management options, findings from this review may contribute to a broader understanding of potential areas for focussed discussions that may be important from the perspective of children who require neurodevelopmental support, along with the needs of their families.