Prescreening clinical trial volunteers using an online personality questionnaire

Plain language summary

In order to test a new anti-anxiety drug, we needed to recruit 24 highly anxious people who were not undergoing treatment. To accelerate our search, we created an online personality questionnaire that allowed us to automatically screen thousands of people for high scores on neuroticism (the trait tendency to experience worry) and for the study’s basic requirements (in this case magnetic resonance imaging [MRI] compatible, living in or near London, right-handed, nonsmokers, aged between 18 and 50 years). We still needed to do telephone screening and onsite medical checks, but our online questionnaire made these steps quicker and cheaper than usual because we were applying them to a population already screened for basic inclusion criteria in a quick and noninvasive manner.

Introduction

Recruitment is a crucial aspect of clinical trials.^1,2 However, both underrecruitment and protracted recruitment periods are a persistent issue.^1,3 Failure to meet recruitment targets is one of the most frequent reasons for early trial termination.^4,5 Discontinuation or extension of trials is not only costly^5,6 but can also impact the validity and reliability of study results.^3,7 Improving recruitment is vital to maintaining evidence-based medicine and gold-standard research.

Problems identifying eligible candidates and obstacles to recruitment through clinical services are a particular concern,⁸ especially within vulnerable populations.^7,9 Recruitment through clinical services is a traditional approach, but a host of limiting factors have been highlighted.⁸ This route typically requires referrals from gatekeepers; however, the time pressure on gatekeepers^1,5,8 and concerns around referral to research impacting the therapeutic relationship^1,9 limit the efficacy of this approach. Secondary and tertiary services tend to see more complex cases with greater comorbidity of diagnoses,¹⁰ meaning that trial volunteers who are treatment-free, treatment-naïve, or highly prototypical are less likely to be identified through these means.⁸ This can severely limit recruitment to early stage disease-mechanism trials and initial phase clinical trials, which often require treatment-free participants without secondary diagnoses.

A number of potential alternatives have been explored. Email and online media distribution have proven effective in maximizing interest.^11,12 Online advertisement has been particularly successful, outperforming recruitment through clinical services.⁸ Improved broadcasting of research opportunities is progressive, as not only the range of candidates accessed is widened, but candidate’s autonomy is improved and, crucially, gatekeeper time investment is reduced. However, screening the individuals responding to online advertisements still requires considerable time investment from researchers,⁸ often for little recruitment pay-out. Considering the time-sensitivity of clinical trials that have often fixed end dates, researcher time investment is a key concern. E-screening methods have been proposed, using clinical database searches within clinical facilities; although these are efficient in removing ineligible candidates,¹³ they are dependent on participants having had previous contact with clinical services. Mail-based prescreening drives, where participants complete and return postal questionnaires, have been successful in specific patient populations.¹⁴ Combining a prescreening stage with online recruitment could be a huge boon to research.

A potential new method of recruitment was developed for use in the Anxiety-Related Personality Traits (APT) trial, which was a functional magnetic resonance imaging (MRI) trial of a novel anxiolytic drug. The APT trial was a Phase IIa randomized controlled trial (RCT), recruiting treatment-free individuals meeting the Mini International Neuropsychiatric Interview (MINI¹⁵) criteria for generalized anxiety disorder (GAD) without comorbid clinical depression. The study criteria were predicted to be difficult to fulfil through traditional routes, as GAD typically has high rates of comorbidity¹⁶ and underrepresentation of treatment-free individuals in care services.¹⁰ This was compounded by the relatively tight time frame for recruitment. As online prescreening is remote and does not necessarily lead to enrolment in research, care should be taken to ensure that the process is minimally invasive for the participant population, while collecting suitable breadth of information. Therefore, in order to find highly anxious individuals, a non-clinical personality questionnaire was chosen. The Trait Self-Description Inventory (TSDI¹⁷) is a 172-item self-report questionnaire measuring the Big-5 personality traits of openness (to experience), conscientiousness, extraversion, agreeableness, and neuroticism. The literature shows strong associations between high scores on the personality trait of neuroticism and experience of and diagnosis of anxiety disorders,^18,19 with departures of only 1 SD from the average score associated with increased risk.¹⁸ Accordingly, we used the TSDI as a prescreening measure to identify individuals who possessed high neuroticism scores (1 SD ≥ mean) and were thus more likely to be suitable for the APT trial.

Aim of the study

This study aimed to explore the use of an online prescreening tool for recruitment for clinical trials. Previously determined average scores and SD for trait neuroticism in a 50-item version of the TSDI (reduced through factor analysis prior to this project) were used as cutoff scores in the online screening tool, to identify those most likely to meet eligibility criteria for the APT trial (data averages were derived from www.researchgate.net/publication/320197485, where the present study data are available online). Individuals responding to the APT trial advert completed the online screening tool as a first step in the recruitment process.

Methods

Potential candidates for the APT trial advert accessed the online prescreening tool through a web portal created by Psyal.co.uk (Psyal, London, UK; www.measureyourpersonality.com). The tool determined initial eligibility through trait neuroticism scores (determined as 58 and 64 for male and female candidates, respectively), MRI scanner compatibility (ie, no metal in the body/eyes, no self-reported claustrophobia), location (restricted to Greater London accessible), and basic trial requirements (ie, nonsmoking, aged 18–50 years).

The APT trial consisted of repeated measures within-subjects design with four MRI dosing visits separated by a washout period and had strict inclusion criteria. It required 24 participants aged 18–50 years who met the criteria for, though not necessarily diagnosed with, GAD as outlined by the MINI¹⁵ and were not currently under any treatment program (pharmacological or psychological) for their anxiety. Individuals were not eligible if symptoms of depression were present (ie, required a score of <15 on the Montgomery-Asberg Depression Rating Scale²⁰) and needed to be able to undergo MRI procedures. Other psychiatric disorders (with the exception of social phobia), any significant cardiovascular, gastrointestinal, hepatic, renal, respiratory, endocrine, immunologic or hematological disease, current use of any prescription or over-the-counter medication (with the exception of contraceptive medication), or smoking (current or within last 3 months) were exclusion criteria. If recruited to the APT trial, participants would be paid for their time. Participants were recruited from the general public and university staff or students across the Greater London area between June 2015 and June 2016.

Ethics was granted by NRES Committee Chelsea, London (14/LO/2127); all participants provided written informed consent to the online prescreening tool at the web portal, and written informed consent in person was obtained for all medical screening visits and enrolment.

Design

As traditional recruitment through clinical services was not expected to be highly effective, the prescreening questionnaire was instead advertised in the general population. After completion of the online prescreening session, the portal generated an individually tailored personality profile that was emailed to all participants automatically by the website algorithm. The individualized feedback was designed to be engaging and to promote participation in this stage of the project, without resorting to typical (and costly) methods of enticement (eg, voucher/prize raffles). The questionnaire was disseminated through online advertising across both public (Gumtree, Call for Participants, Facebook) and institutional (King’s College London and partner website and email circulars) platforms.

If the respondents met the predefined criteria (ie, neuroticism score ≥64 for females, ≥58 for males, MRI compatible, etc), an addition to the automated email was inserted, which included the APT trial information sheet with a message inviting them to contact the research team. The individuals who then initiated contact were telephone screened by the research team to check eligibility in more detail. Those eligible at the telephone screening stage were invited to attend onsite medical screening. This method aimed to not only minimize researcher time spent on participant identification and prescreening but also minimize unsuccessful medical screening visits. Contact was in the control of the candidates to minimize unnecessary impact on candidates. The researchers received a copy of the emails sent to eligible individuals, which allowed them to track the number of completions.

Analysis

The analysis comprised descriptive statistics and was performed using SPSS (IBM Corporation, Armonk, NY, USA). A CONSORT flowchart tracked participant progress.

Results

A total of 6,293 individuals completed the online personality questionnaire (after removal of duplicates). The prescreening algorithm identified 862 eligible candidates who received the additional study information and contact details. Of these, 266 contacted the research team and 173 participated in telephone screening. Telephone screening established that 53 of these candidates were eligible for medical screening at the clinical site; the other 120 were excluded at this stage on the basis of factors such as availability, current medications schedules, and physical health concerns. At the onsite medical screening, study physicians excluded 25 of the 53 potential participants due to complex exclusion criteria that could not be assessed during the telephone screening session eg, blood test abnormalities; not meeting criteria for GAD), resulting in 28 enrolments. Four candidates later withdrew (due to scanner breakdown, n=1; time constraints, n=2; nonadherence to study guidelines, n=1), resulting in the APT trial completing on time and on budget (Figure 1). Recruitment was paused when 24 candidate positions were filled, to prevent unnecessary medical checks for individuals who may not end up being fully enrolled. Table 1 indicates the gender and age data at each stage of the recruitment process. Table 2 shows average score and internal consistency for each personality trait.

Figure 1 CONSORT diagram of participant progress through study. Abbreviations: TSDI, Trait Self-Description Inventory; MINI, Mini International Neuropsychiatric Interview; GAD, generalized anxiety disorder; MRI, magnetic resonance imaging.

Table 1 Average age and gender of sample at different recruitment stages Note: a14 missing data items.

Table 2 Average (mean) scores and reliability alpha scores for personality traits

Discussion

Recruitment using our online prescreening method generated a large response rate, permitting successful recruitment of a full cohort of hard-to-identify participants within a restricted time frame. Considering only 31% of RCT studies achieve recruitment success and 54% require grant extensions in order to fulfil targets,³ the recruitment method that we have developed appears promising. This method combines the efficiency and economic benefits of online advertisement (eg, Wise et al⁸) with online prescreening to accelerate recruitment and reduce time demands on study personnel. Research has suggested word-of-mouth is key to nonfinancially incentivized recruitment,²¹ indicating the importance of an interesting tool such as a personality measure in promoting engagement. Recent initiatives recommend the use of digital technology to generate research data and study populations,²² suggesting development of an online prescreening method is timely.

Of the volunteers invited to medical screening onsite, only 9.3% did not meet the main psychiatric eligibility criteria (presence of GAD), indicating the accuracy of our prescreening method. In addition to the considerable issues in recruitment of treatment-naïve participants generally, recruitment of GAD patients without depression can be especially difficult due to the high levels of comorbidity between these conditions.¹⁶ Timely recruitment of an adherent sample as shown here indicates the success of this method in identifying a minority population. Considering the cost of medical screening (eg, psychiatrist time, blood sample processing, transport expenses, etc) and the time sensitivity of clinical trials, our method has helped greatly reduce unnecessary onsite medical screening visits. Furthermore, this methodology aimed to reduce pressure on gatekeepers elicited by traditional routes of recruitment through clinical services and could also increase patient autonomy in research involvement,⁹ widening research participation.

Research has highlighted transdiagnostic factors within mental health pathology, positing common underlying traits which contribute to the development and maintenance of mood and anxiety disorders.^19,23–25 Variations in trait strength or the combination of underlying personality constructs may influence the symptom profile of different disorders.¹⁹ The high level of comorbidity,^24,25 broad treatment response, and common neurobiological function¹⁹ in mood and anxiety disorders are supportive of a more dimensional approach. Trait neuroticism is a key candidate facet within this model.^19,23,26 Recruitment through constructs such as neuroticism is therefore a logical step; work has already suggested diagnostic frameworks in a similar vein to the recruitment paradigm outlined here.²⁴

Though using neuroticism scores to identify those likely to experience anxiety is in line with the literature,¹⁸ further work is required to fully refine this process and ensure broad recruitment to trials is maintained. As with any online tool, care must be taken to ensure a wide range of demographics are represented and access to Internet is not limited.⁸ A large proportion of candidates completing the online prescreening were female, although as anxiety disorders are more common in women than men,²⁷ this skew may be specific to the target disorder.

In light of these points, our methodology is proposed as a supplement to traditional modes of recruitment, with particular utility as a means of recruiting treatment-free prototypical participants who are not in touch with clinical services. Neuroticism has been linked to other mental and physical health disorders,^18,28 suggesting our method could be used to identify candidate participants with a broad range of conditions. Moreover, other personality traits have been tied to psychopathology, such as high and low levels of extraversion being linked to conduct disorders and mood disorders, respectively.²⁸ This indicates potential for tailoring of the personality criteria of the online platform to enrich recruitment populations in a multi-dimensional manner by, for example, selecting individuals who score high on neuroticism but low on extraversion.

This project was exploratory, and though the findings are promising, future work should compare this recruitment plan with traditional clinical services routes for full assessment of efficacy and analysis of patient groups recruited through each channel. Development of a wider screening process using this methodology to recruit to a range of trials alongside clinical service-based recruitment is currently underway. Future projects should ensure the ethical considerations behind using trait characteristics as identifiers in health are given due weight and would benefit from developing community engagement relationships with target populations to ensure relevance and diverse representation is achieved.²⁹

In conclusion, the online prescreening personality questionnaire created for the APT study allowed automatic screening of thousands of people for high scores on neuroticism, plus the study’s basic screening requirements (in this case MRI safe, living in or near London, right-handed, nonsmokers, aged between 18 and 50 years). Though our new method does not obviate the need for telephone screening or onsite medical checks, it does make these steps quicker and cheaper than usual and aims to minimize impact on candidate participants. Thus, the online prescreening personality questionnaire approach is a useful tool for recruiting clinical trial participants, as it is not only cost-saving but also time-saving for both candidates and trial staff, while aiming to be of interest to the candidate population.

Acknowledgment

The authors would like to acknowledge and thank Bionomics Ltd. Without their grant, the study could not have been resourced.

Disclosure

Allan H Young and Adam M Perkins were the named principal investigators who were awarded the grant from Bionomics Ltd, which supported this work. Allan H Young, Steven C R Williams, and Adam M Perkins were supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at the South London and Maudsley NHS Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the National Health Service (NHS), National Institute for Health Research (NIHR), Medical Research Council (MRC), or Department of Health. Fiona Patrick has been supported throughout her PhD by departmental funding within the Centre for Affective Disorders, Department of Psychological Medicine at the Institute of Psychiatry, Psychology and Neuroscience, King’s College London. The authors report no other conflicts of interest in this work.

---

References

1.		Borschmann R, Patterson S, Poovendran D, Wilson D, Weaver T. Influences on recruitment to randomised controlled trials in mental health settings in England: a national cross-sectional survey of researchers working for the Mental Health Research Network. BMC Med Res Methodol. 2014;14(1):23.
2.		Rojavin MA. Recruitment index as a measure of patient recruitment activity in clinical trials. Contemp Clin Trials. 2005;26(5):552–556.
3.		McDonald AM, Knight RC, Campbell MK, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7(1):9.
4.		Kasenda B, Von Elm E, You J, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014;311(10):1045–1051.
5.		Sullivan J. Subject recruitment and retention: barriers to success. Applied Clinical Trials Online. Available from: http://www.appliedclinicaltrialsonline.com/subject-recruitment-and-retention-barriers-success. Accessed March 25, 2018.
6.		Kopcke F, Prokosch H-U. Employing computers for the recruitment into clinical trials: a comprehensive systematic review. J Med Internet Res. 2014;16(7):e161.
7.		Patterson S, Kramo K, Soteriou T, Crawford MJ. The great divide: a qualitative investigation of factors influencing researcher access to potential randomised controlled trial participants in mental health settings. J Ment Heal. 2010;19(6):532–541.
8.		Wise T, Arnone D, Marwood L, Zahn R, Lythe KE, Young AH. Recruiting for research studies using online public advertisements: examples from research in affective disorders. Neuropsychiatr Dis Treat. 2016;12:279–285.
9.		Howard L, de Salis I, Tomlin Z, Thornicroft G, Donovan J. Why is recruitment to trials difficult? An investigation into recruitment difficulties in an RCT of supported employment in patients with severe mental illness. Contemp Clin Trials. 2009;30(1):40–46.
10.		Keown P, Holloway F, Kuipers E. The prevalence of personality disorders, psychotic disorders and affective disorders amongst the patients seen by a community mental health team in London. Soc Psychiatry Psychiatr Epidemiol. 2002;37(5):225–229.
11.		Kirkby HM, Wilson S, Calvert M, Draper H. Using e-mail recruitment and an online questionnaire to establish effect size: a worked example. BMC Med Res Methodol. 2011;11(1):89.
12.		Montag C, Reuter M, Jurkiewicz M, Markett S, Panksepp J. Erratum: imaging the structure of the human anxious brain: a review of findings from neuroscientific personality psychology (Rev Neurosci. 2013;24:167–190. 10.1515/revneuro-2012-0085). Rev Neurosci. 2014;25(2):307.
13.		Thadani SR, Weng C, Bigger JT, Ennever JF, Wajngurt D. Electronic screening improves efficiency in clinical trial recruitment. J Am Med Informatics Assoc. 2009;16(6):869–873.
14.		Andersen F, Engstad TA, Straume B, et al. Recruitment methods in Alzheimer’s disease research: general practice versus population based screening by mail. BMC Med Res Methodol. 2010;10:35.
15.		Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22–33.
16.		Takahashi Y, Roberts BW, Yamagata S, Kijima N. Personality traits show differential relationships with anxiety and depression. Psychologia. 2015;58:15–26.
17.		Collis JM, Elshaw CC. The development of the trait self description inventory: results of US/UK collaboration. TTCP/HUM/98-001; 1998. Technical Panel HUM-TP3, Military Human Resources Issues, The Technical Cooperation Program. Available from: http://www.iamps.org/IAMPS_2002_Syed-iamps2002.pdf. Accessed March 25, 2018.
18.		Lahey BB. Public health significance of neuroticism. Am Psychol. 2009;64(4):241–256.
19.		Barlow DH, Sauer-Zavala S, Carl JR, Bullis JR, Ellard KK. The nature, diagnosis, and treatment of neuroticism. Clin Psychol Sci. 2014;2(3):344–365.
20.		Montgomery SA, Asberg M. New depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134(Apr):382–389.
21.		Luzurier Q, Damm C, Lion F, Daniel C, Pellerin L, Tavolacci MP. Strategy for recruitment and factors associated with motivation and satisfaction in a randomized trial with 210 healthy volunteers without financial compensation. BMC Med Res Methodol. 2015;15(1):1–9.
22.		Department of Health and Social Care. A Framework for Mental Health Research. London, UK; 2017. Available from: https://www.gov.uk/government/publications/a-framework-for-mental-health-research. Accessed March 25, 2018.
23.		Griffith JW, Zinbarg RE, Craske MG, et al. Neuroticism as a common dimension in the internalizing disorders. Psychol Med. 2010;40(7):1125–1136.
24.		Rodriguez-Seijas C, Eaton NR, Krueger RF. How transdiagnostic factors of personality and psychopathology can inform clinical assessment and intervention. J Pers Assess. 2015;97(5):425–435.
25.		Krueger RF, Chentsova-Dutton YE, Markon KE, Goldberg D, Ormel J. A cross-cultural study of the structure of comorbidity among common psychopathological syndromes in the general health care setting. J Abnorm Psychol. 2003;112(3):437–447.
26.		Gray JA. The neuropsychology of anxiety – an inquiry into the functions of the septo-hippocampal system. Behav Brain Sci. 1982;5(1982):469–534.
27.		Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:593–602.
28.		Malouff JM, Thorsteinsson EB, Schutte NS. The relationship between the five-factor model of personality and symptoms of clinical disorders: a meta-analysis. J Psychopathol Behav Assess. 2005;27(2):101–114.
29.		Hood NE, Brewer T, Jackson R, Wewers ME. Survey of community engagement in NIH-funded research. Clin Transl Sci. 2010;3(1):19–22.