Prescreening clinical trial volunteers using an online personality questionnaire
Received 28 March 2018
Accepted for publication 23 May 2018
Published 5 September 2018 Volume 2018:14 Pages 2297—2303
Checked for plagiarism Yes
Review by Single-blind
Peer reviewers approved by Dr Colin Mak
Peer reviewer comments 2
Editor who approved publication: Dr Roger Pinder
Fiona Patrick,1 Allan H Young1,2 Steven CR Williams,2,3 Adam M Perkins1,2
1Department of Psychological Medicine, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK; 2National Institute for Health Research Biomedical Research Centre, South London and Maudsley NHS Foundation Trust, London, UK; 3Department of Neuroimaging, Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK
Background: The cost of a clinical trial is affected by the efficiency of participant recruitment. It would be desirable to create a prescreening method that identifies appropriate candidates for full screening, in order to prevent inconvenience for both trial and volunteers. This study presents an online prescreening tool for this purpose.
Methods: In order to facilitate recruitment of 24 individuals meeting the criteria for generalized anxiety disorder to a pharmacological functional magnetic resonance imaging trial, we created an online personality questionnaire that generated a personality profile for each respondent and screened for the trial’s basic criteria.
Results: Our online platform screened 6,293 people for anxious personality traits in 1 year. A total of 862 eligible individuals were identified through this route, each of whom automatically received an email invitation to contact the study team for further telephone screening, if interested. Of those, 266 individuals contacted the team and 173 were telephone screened, with 53 attending the study site for medical checks. Twenty-eight individuals were fully eligible, and 24 completed the trial. This permitted completion on time and on budget.
Conclusion: Our online prescreening personality questionnaire platform did not remove the need for telephone screening or onsite medical checks, but increased the efficiency of recruitment through noninvasive identification of those meeting key requirements. Thus, our platform is a useful recruitment technique for clinical trials and is time-saving for both the trial and potential participants.
Keywords: recruitment, prescreening, personality, mental health, clinical trials
Plain language summary
In order to test a new anti-anxiety drug, we needed to recruit 24 highly anxious people who were not undergoing treatment. To accelerate our search, we created an online personality questionnaire that allowed us to automatically screen thousands of people for high scores on neuroticism (the trait tendency to experience worry) and for the study’s basic requirements (in this case magnetic resonance imaging [MRI] compatible, living in or near London, right-handed, nonsmokers, aged between 18 and 50 years). We still needed to do telephone screening and onsite medical checks, but our online questionnaire made these steps quicker and cheaper than usual because we were applying them to a population already screened for basic inclusion criteria in a quick and noninvasive manner.
Recruitment is a crucial aspect of clinical trials.1,2 However, both underrecruitment and protracted recruitment periods are a persistent issue.1,3 Failure to meet recruitment targets is one of the most frequent reasons for early trial termination.4,5 Discontinuation or extension of trials is not only costly5,6 but can also impact the validity and reliability of study results.3,7 Improving recruitment is vital to maintaining evidence-based medicine and gold-standard research.
Problems identifying eligible candidates and obstacles to recruitment through clinical services are a particular concern,8 especially within vulnerable populations.7,9 Recruitment through clinical services is a traditional approach, but a host of limiting factors have been highlighted.8 This route typically requires referrals from gatekeepers; however, the time pressure on gatekeepers1,5,8 and concerns around referral to research impacting the therapeutic relationship1,9 limit the efficacy of this approach. Secondary and tertiary services tend to see more complex cases with greater comorbidity of diagnoses,10 meaning that trial volunteers who are treatment-free, treatment-naïve, or highly prototypical are less likely to be identified through these means.8 This can severely limit recruitment to early stage disease-mechanism trials and initial phase clinical trials, which often require treatment-free participants without secondary diagnoses.
A number of potential alternatives have been explored. Email and online media distribution have proven effective in maximizing interest.11,12 Online advertisement has been particularly successful, outperforming recruitment through clinical services.8 Improved broadcasting of research opportunities is progressive, as not only the range of candidates accessed is widened, but candidate’s autonomy is improved and, crucially, gatekeeper time investment is reduced. However, screening the individuals responding to online advertisements still requires considerable time investment from researchers,8 often for little recruitment pay-out. Considering the time-sensitivity of clinical trials that have often fixed end dates, researcher time investment is a key concern. E-screening methods have been proposed, using clinical database searches within clinical facilities; although these are efficient in removing ineligible candidates,13 they are dependent on participants having had previous contact with clinical services. Mail-based prescreening drives, where participants complete and return postal questionnaires, have been successful in specific patient populations.14 Combining a prescreening stage with online recruitment could be a huge boon to research.
A potential new method of recruitment was developed for use in the Anxiety-Related Personality Traits (APT) trial, which was a functional magnetic resonance imaging (MRI) trial of a novel anxiolytic drug. The APT trial was a Phase IIa randomized controlled trial (RCT), recruiting treatment-free individuals meeting the Mini International Neuropsychiatric Interview (MINI15) criteria for generalized anxiety disorder (GAD) without comorbid clinical depression. The study criteria were predicted to be difficult to fulfil through traditional routes, as GAD typically has high rates of comorbidity16 and underrepresentation of treatment-free individuals in care services.10 This was compounded by the relatively tight time frame for recruitment. As online prescreening is remote and does not necessarily lead to enrolment in research, care should be taken to ensure that the process is minimally invasive for the participant population, while collecting suitable breadth of information. Therefore, in order to find highly anxious individuals, a non-clinical personality questionnaire was chosen. The Trait Self-Description Inventory (TSDI17) is a 172-item self-report questionnaire measuring the Big-5 personality traits of openness (to experience), conscientiousness, extraversion, agreeableness, and neuroticism. The literature shows strong associations between high scores on the personality trait of neuroticism and experience of and diagnosis of anxiety disorders,18,19 with departures of only 1 SD from the average score associated with increased risk.18 Accordingly, we used the TSDI as a prescreening measure to identify individuals who possessed high neuroticism scores (1 SD ≥ mean) and were thus more likely to be suitable for the APT trial.
Aim of the study
This study aimed to explore the use of an online prescreening tool for recruitment for clinical trials. Previously determined average scores and SD for trait neuroticism in a 50-item version of the TSDI (reduced through factor analysis prior to this project) were used as cutoff scores in the online screening tool, to identify those most likely to meet eligibility criteria for the APT trial (data averages were derived from www.researchgate.net/publication/320197485, where the present study data are available online). Individuals responding to the APT trial advert completed the online screening tool as a first step in the recruitment process.
Potential candidates for the APT trial advert accessed the online prescreening tool through a web portal created by Psyal.co.uk (Psyal, London, UK; www.measureyourpersonality.com). The tool determined initial eligibility through trait neuroticism scores (determined as 58 and 64 for male and female candidates, respectively), MRI scanner compatibility (ie, no metal in the body/eyes, no self-reported claustrophobia), location (restricted to Greater London accessible), and basic trial requirements (ie, nonsmoking, aged 18–50 years).
The APT trial consisted of repeated measures within-subjects design with four MRI dosing visits separated by a washout period and had strict inclusion criteria. It required 24 participants aged 18–50 years who met the criteria for, though not necessarily diagnosed with, GAD as outlined by the MINI15 and were not currently under any treatment program (pharmacological or psychological) for their anxiety. Individuals were not eligible if symptoms of depression were present (ie, required a score of <15 on the Montgomery-Asberg Depression Rating Scale20) and needed to be able to undergo MRI procedures. Other psychiatric disorders (with the exception of social phobia), any significant cardiovascular, gastrointestinal, hepatic, renal, respiratory, endocrine, immunologic or hematological disease, current use of any prescription or over-the-counter medication (with the exception of contraceptive medication), or smoking (current or within last 3 months) were exclusion criteria. If recruited to the APT trial, participants would be paid for their time. Participants were recruited from the general public and university staff or students across the Greater London area between June 2015 and June 2016.
Ethics was granted by NRES Committee Chelsea, London (14/LO/2127); all participants provided written informed consent to the online prescreening tool at the web portal, and written informed consent in person was obtained for all medical screening visits and enrolment.
As traditional recruitment through clinical services was not expected to be highly effective, the prescreening questionnaire was instead advertised in the general population. After completion of the online prescreening session, the portal generated an individually tailored personality profile that was emailed to all participants automatically by the website algorithm. The individualized feedback was designed to be engaging and to promote participation in this stage of the project, without resorting to typical (and costly) methods of enticement (eg, voucher/prize raffles). The questionnaire was disseminated through online advertising across both public (Gumtree, Call for Participants, Facebook) and institutional (King’s College London and partner website and email circulars) platforms.
If the respondents met the predefined criteria (ie, neuroticism score ≥64 for females, ≥58 for males, MRI compatible, etc), an addition to the automated email was inserted, which included the APT trial information sheet with a message inviting them to contact the research team. The individuals who then initiated contact were telephone screened by the research team to check eligibility in more detail. Those eligible at the telephone screening stage were invited to attend onsite medical screening. This method aimed to not only minimize researcher time spent on participant identification and prescreening but also minimize unsuccessful medical screening visits. Contact was in the control of the candidates to minimize unnecessary impact on candidates. The researchers received a copy of the emails sent to eligible individuals, which allowed them to track the number of completions.
The analysis comprised descriptive statistics and was performed using SPSS (IBM Corporation, Armonk, NY, USA). A CONSORT flowchart tracked participant progress.
A total of 6,293 individuals completed the online personality questionnaire (after removal of duplicates). The prescreening algorithm identified 862 eligible candidates who received the additional study information and contact details. Of these, 266 contacted the research team and 173 participated in telephone screening. Telephone screening established that 53 of these candidates were eligible for medical screening at the clinical site; the other 120 were excluded at this stage on the basis of factors such as availability, current medications schedules, and physical health concerns. At the onsite medical screening, study physicians excluded 25 of the 53 potential participants due to complex exclusion criteria that could not be assessed during the telephone screening session eg, blood test abnormalities; not meeting criteria for GAD), resulting in 28 enrolments. Four candidates later withdrew (due to scanner breakdown, n=1; time constraints, n=2; nonadherence to study guidelines, n=1), resulting in the APT trial completing on time and on budget (Figure 1). Recruitment was paused when 24 candidate positions were filled, to prevent unnecessary medical checks for individuals who may not end up being fully enrolled. Table 1 indicates the gender and age data at each stage of the recruitment process. Table 2 shows average score and internal consistency for each personality trait.
Figure 1 CONSORT diagram of participant progress through study.
Table 1 Average age and gender of sample at different recruitment stages
Table 2 Average (mean) scores and reliability alpha scores for personality traits
Recruitment using our online prescreening method generated a large response rate, permitting successful recruitment of a full cohort of hard-to-identify participants within a restricted time frame. Considering only 31% of RCT studies achieve recruitment success and 54% require grant extensions in order to fulfil targets,3 the recruitment method that we have developed appears promising. This method combines the efficiency and economic benefits of online advertisement (eg, Wise et al8) with online prescreening to accelerate recruitment and reduce time demands on study personnel. Research has suggested word-of-mouth is key to nonfinancially incentivized recruitment,21 indicating the importance of an interesting tool such as a personality measure in promoting engagement. Recent initiatives recommend the use of digital technology to generate research data and study populations,22 suggesting development of an online prescreening method is timely.
Of the volunteers invited to medical screening onsite, only 9.3% did not meet the main psychiatric eligibility criteria (presence of GAD), indicating the accuracy of our prescreening method. In addition to the considerable issues in recruitment of treatment-naïve participants generally, recruitment of GAD patients without depression can be especially difficult due to the high levels of comorbidity between these conditions.16 Timely recruitment of an adherent sample as shown here indicates the success of this method in identifying a minority population. Considering the cost of medical screening (eg, psychiatrist time, blood sample processing, transport expenses, etc) and the time sensitivity of clinical trials, our method has helped greatly reduce unnecessary onsite medical screening visits. Furthermore, this methodology aimed to reduce pressure on gatekeepers elicited by traditional routes of recruitment through clinical services and could also increase patient autonomy in research involvement,9 widening research participation.
Research has highlighted transdiagnostic factors within mental health pathology, positing common underlying traits which contribute to the development and maintenance of mood and anxiety disorders.19,23–25 Variations in trait strength or the combination of underlying personality constructs may influence the symptom profile of different disorders.19 The high level of comorbidity,24,25 broad treatment response, and common neurobiological function19 in mood and anxiety disorders are supportive of a more dimensional approach. Trait neuroticism is a key candidate facet within this model.19,23,26 Recruitment through constructs such as neuroticism is therefore a logical step; work has already suggested diagnostic frameworks in a similar vein to the recruitment paradigm outlined here.24
Though using neuroticism scores to identify those likely to experience anxiety is in line with the literature,18 further work is required to fully refine this process and ensure broad recruitment to trials is maintained. As with any online tool, care must be taken to ensure a wide range of demographics are represented and access to Internet is not limited.8 A large proportion of candidates completing the online prescreening were female, although as anxiety disorders are more common in women than men,27 this skew may be specific to the target disorder.
In light of these points, our methodology is proposed as a supplement to traditional modes of recruitment, with particular utility as a means of recruiting treatment-free prototypical participants who are not in touch with clinical services. Neuroticism has been linked to other mental and physical health disorders,18,28 suggesting our method could be used to identify candidate participants with a broad range of conditions. Moreover, other personality traits have been tied to psychopathology, such as high and low levels of extraversion being linked to conduct disorders and mood disorders, respectively.28 This indicates potential for tailoring of the personality criteria of the online platform to enrich recruitment populations in a multi-dimensional manner by, for example, selecting individuals who score high on neuroticism but low on extraversion.
This project was exploratory, and though the findings are promising, future work should compare this recruitment plan with traditional clinical services routes for full assessment of efficacy and analysis of patient groups recruited through each channel. Development of a wider screening process using this methodology to recruit to a range of trials alongside clinical service-based recruitment is currently underway. Future projects should ensure the ethical considerations behind using trait characteristics as identifiers in health are given due weight and would benefit from developing community engagement relationships with target populations to ensure relevance and diverse representation is achieved.29
In conclusion, the online prescreening personality questionnaire created for the APT study allowed automatic screening of thousands of people for high scores on neuroticism, plus the study’s basic screening requirements (in this case MRI safe, living in or near London, right-handed, nonsmokers, aged between 18 and 50 years). Though our new method does not obviate the need for telephone screening or onsite medical checks, it does make these steps quicker and cheaper than usual and aims to minimize impact on candidate participants. Thus, the online prescreening personality questionnaire approach is a useful tool for recruiting clinical trial participants, as it is not only cost-saving but also time-saving for both candidates and trial staff, while aiming to be of interest to the candidate population.
The authors would like to acknowledge and thank Bionomics Ltd. Without their grant, the study could not have been resourced.
Allan H Young and Adam M Perkins were the named principal investigators who were awarded the grant from Bionomics Ltd, which supported this work. Allan H Young, Steven C R Williams, and Adam M Perkins were supported by the National Institute for Health Research (NIHR) Biomedical Research Centre at the South London and Maudsley NHS Foundation Trust and King’s College London. The views expressed are those of the authors and not necessarily those of the National Health Service (NHS), National Institute for Health Research (NIHR), Medical Research Council (MRC), or Department of Health. Fiona Patrick has been supported throughout her PhD by departmental funding within the Centre for Affective Disorders, Department of Psychological Medicine at the Institute of Psychiatry, Psychology and Neuroscience, King’s College London. The authors report no other conflicts of interest in this work.
Borschmann R, Patterson S, Poovendran D, Wilson D, Weaver T. Influences on recruitment to randomised controlled trials in mental health settings in England: a national cross-sectional survey of researchers working for the Mental Health Research Network. BMC Med Res Methodol. 2014;14(1):23.
Rojavin MA. Recruitment index as a measure of patient recruitment activity in clinical trials. Contemp Clin Trials. 2005;26(5):552–556.
McDonald AM, Knight RC, Campbell MK, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7(1):9.
Kasenda B, Von Elm E, You J, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014;311(10):1045–1051.
Sullivan J. Subject recruitment and retention: barriers to success. Applied Clinical Trials Online. Available from: http://www.appliedclinicaltrialsonline.com/subject-recruitment-and-retention-barriers-success. Accessed March 25, 2018.
Kopcke F, Prokosch H-U. Employing computers for the recruitment into clinical trials: a comprehensive systematic review. J Med Internet Res. 2014;16(7):e161.
Patterson S, Kramo K, Soteriou T, Crawford MJ. The great divide: a qualitative investigation of factors influencing researcher access to potential randomised controlled trial participants in mental health settings. J Ment Heal. 2010;19(6):532–541.
Wise T, Arnone D, Marwood L, Zahn R, Lythe KE, Young AH. Recruiting for research studies using online public advertisements: examples from research in affective disorders. Neuropsychiatr Dis Treat. 2016;12:279–285.
Howard L, de Salis I, Tomlin Z, Thornicroft G, Donovan J. Why is recruitment to trials difficult? An investigation into recruitment difficulties in an RCT of supported employment in patients with severe mental illness. Contemp Clin Trials. 2009;30(1):40–46.
Keown P, Holloway F, Kuipers E. The prevalence of personality disorders, psychotic disorders and affective disorders amongst the patients seen by a community mental health team in London. Soc Psychiatry Psychiatr Epidemiol. 2002;37(5):225–229.
Kirkby HM, Wilson S, Calvert M, Draper H. Using e-mail recruitment and an online questionnaire to establish effect size: a worked example. BMC Med Res Methodol. 2011;11(1):89.
Montag C, Reuter M, Jurkiewicz M, Markett S, Panksepp J. Erratum: imaging the structure of the human anxious brain: a review of findings from neuroscientific personality psychology (Rev Neurosci. 2013;24:167–190. 10.1515/revneuro-2012-0085). Rev Neurosci. 2014;25(2):307.
Thadani SR, Weng C, Bigger JT, Ennever JF, Wajngurt D. Electronic screening improves efficiency in clinical trial recruitment. J Am Med Informatics Assoc. 2009;16(6):869–873.
Andersen F, Engstad TA, Straume B, et al. Recruitment methods in Alzheimer’s disease research: general practice versus population based screening by mail. BMC Med Res Methodol. 2010;10:35.
Sheehan DV, Lecrubier Y, Sheehan KH, et al. The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998;59(Suppl 20):22–33.
Takahashi Y, Roberts BW, Yamagata S, Kijima N. Personality traits show differential relationships with anxiety and depression. Psychologia. 2015;58:15–26.
Collis JM, Elshaw CC. The development of the trait self description inventory: results of US/UK collaboration. TTCP/HUM/98-001; 1998. Technical Panel HUM-TP3, Military Human Resources Issues, The Technical Cooperation Program. Available from: http://www.iamps.org/IAMPS_2002_Syed-iamps2002.pdf. Accessed March 25, 2018.
Lahey BB. Public health significance of neuroticism. Am Psychol. 2009;64(4):241–256.
Barlow DH, Sauer-Zavala S, Carl JR, Bullis JR, Ellard KK. The nature, diagnosis, and treatment of neuroticism. Clin Psychol Sci. 2014;2(3):344–365.
Montgomery SA, Asberg M. New depression scale designed to be sensitive to change. Br J Psychiatry. 1979;134(Apr):382–389.
Luzurier Q, Damm C, Lion F, Daniel C, Pellerin L, Tavolacci MP. Strategy for recruitment and factors associated with motivation and satisfaction in a randomized trial with 210 healthy volunteers without financial compensation. BMC Med Res Methodol. 2015;15(1):1–9.
Department of Health and Social Care. A Framework for Mental Health Research. London, UK; 2017. Available from: https://www.gov.uk/government/publications/a-framework-for-mental-health-research. Accessed March 25, 2018.
Griffith JW, Zinbarg RE, Craske MG, et al. Neuroticism as a common dimension in the internalizing disorders. Psychol Med. 2010;40(7):1125–1136.
Rodriguez-Seijas C, Eaton NR, Krueger RF. How transdiagnostic factors of personality and psychopathology can inform clinical assessment and intervention. J Pers Assess. 2015;97(5):425–435.
Krueger RF, Chentsova-Dutton YE, Markon KE, Goldberg D, Ormel J. A cross-cultural study of the structure of comorbidity among common psychopathological syndromes in the general health care setting. J Abnorm Psychol. 2003;112(3):437–447.
Gray JA. The neuropsychology of anxiety – an inquiry into the functions of the septo-hippocampal system. Behav Brain Sci. 1982;5(1982):469–534.
Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:593–602.
Malouff JM, Thorsteinsson EB, Schutte NS. The relationship between the five-factor model of personality and symptoms of clinical disorders: a meta-analysis. J Psychopathol Behav Assess. 2005;27(2):101–114.
Hood NE, Brewer T, Jackson R, Wewers ME. Survey of community engagement in NIH-funded research. Clin Transl Sci. 2010;3(1):19–22.
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.Download Article [PDF]