Recruiting for research studies using online public advertisements examples from research in affective disorders

Introduction

One of the most crucial elements of any clinical research study is recruitment. This can be challenging, especially if the research focuses on understanding disease mechanisms by recruiting treatment-naïve patients with little comorbidity and/or complex histories.

Patients with mood disorders in countries such as the UK commonly present to primary care services where they receive diagnostic assessments and first-line treatments.^1,2 Although primary care theoretically constitutes the ideal location for identifying research participants, time pressure on clinicians and a primarily clinical focus commonly impinge on effective recruitment. Patients treated in secondary care usually suffer from more complex, frequently refractory forms of illness, and often have comorbid disorders,³ which can make them less suitable for nonnaturalistic research projects. Lack of research focus may also complicate access to research in secondary care.⁴ Furthermore, the stigma attached to mental health conditions may be a limiting factor to engaging patients in research within psychiatric services.^5,6 This is further complicated when recruiting for basic research studies of disease mechanisms, as these studies often require patients who are untreated. These individuals are often unlikely to be in contact with health care services, making recruitment through clinical routes especially challenging.

Most published evidence regarding recruitment emerges from clinical trials. Difficulties experienced in clinical trial recruitment can lead to overall reduced sample sizes and in some instances inadequately powered studies.⁷ Common problems include low numbers of eligible participants, limited numbers of volunteers consenting to participate, and delays in making arrangements with clinical services.⁷

The most common effect is a delay in the rate of recruitment that extends the length of the study and causes negative effects on workload, costs, and morale of research staff.⁸ In some cases, unacceptable recruitment rates can lead to studies being prematurely terminated.⁹ A solution often adopted by clinical trials testing is to recruit over multiple sites.¹⁰ Although this approach offers a pragmatic solution by expanding the number of participants recruited simultaneously, it often adds complexity to the research protocol, may increase experimental error and significantly increases costs.

An alternative strategy is to maximize recruitment from a confined geographical area. In this respect, one solution is to recruit directly from the general public by using public advertisements. This is a particularly attractive option when seeking unmedicated patients who are not expected to be in contact with health care services. Online advertising through search engines and social media has become more accessible in the past decade and may present an effective way to facilitate recruitment. Despite evidence highlighting the potentials of this method for recruiting into clinical trials in psychiatric research,^11–13 to our knowledge there has been no research focused on assessing the viability of online recruiting for basic research studies on disease mechanisms, which often require relatively specific and difficult to recruit populations. Moreover, little work has explored the advantages of recruiting participants with affective disorders generally with these methods. To determine the viability of this method of recruiting, we evaluated the effectiveness of online recruitment for three research studies focused on understanding the mechanisms of disease in individuals with affective disorders in the UK. Our primary aim was to establish whether these methods are indeed effective in recruiting to these types of study.

Methods

General methods

We used online advertisements as primary recruitment method for three neuroimaging studies in affective disorders in association with other approaches, which included university-wide and newspaper advertisements. Collaborative links were also established with primary and secondary care services to identify suitable participants. The results presented here are secondary analyses of studies looking at the neurobiological basis of affective disorders.

Study A assessed the use of neuroimaging in predicting recurrence of depressive symptoms. Individuals who had previously experienced major depression, were in clinical remission, and were medication-free at the time of enrolment were identified and followed up over a period of time. This is a difficult group to identify through clinical services as it is unlikely that these patients would be in contact with clinicians any longer and may receive pharmacological treatment as maintenance.

Study B and Study C were two neuroimaging studies comparing individuals with unipolar and bipolar depression versus healthy controls and unipolar depression versus healthy controls, respectively. Both these studies required a sample of medication-free, currently depressed patients. The majority of symptomatic individuals receive pharmacological treatment, making the recruitment of medication-free individuals particularly challenging. In addition, there is evidence that bipolarity occurs more often than previously believed in individuals presenting with depressive symptoms.¹⁴

As online advertisements reach a wide audience they produce a large number of responses. However, many of those responding do not meet inclusion criteria. Hence, to increase specificity, researchers involved in these studies asked a brief set of questions to potential participants on the telephone with the intention to exclude ineligible individuals from more lengthy face-to-face interviews. Questions focused on specific exclusion criteria that prevented participation in the research such as magnetic resonance imaging contraindications, disease-specific symptoms, and severity.

Study A was approved by the South Manchester National Health Service (NHS) research ethics committees and study B and C was approved by Bromley NHS research ethics committee. Research approvals included capacity for advertising online and all participants provided written, informed consent to take part in the research. Participants in all studies were paid for their participation, and details of financial compensation were provided on the advertisements related to the research as described here.

Search engine advertisements

In Study A, advertisements were targeted at users in the Greater Manchester area in the UK performing online searches using the search engine Google. Advertisements featured a small amount of text and a link to a page within the University of Manchester website which sponsored the research and contained detailed information about the study. The advertisements were regionally restricted and designed by using relevant keywords such as “depression” aiming at a selected but sufficiently inclusive local sample.

Social media advertisements

For Study B, advertisements were placed on the social media website Facebook. In order to maximize visibility, advertisements were placed on the sidebar of Facebook pages and in the main newsfeed where other users’ own posts normally appear. These were restricted to the London area and users aged between 18 and 65 years. Advertisements gave a brief description of the study and the primary inclusion criteria, and provided a link to a study webpage, hosted on the King’s College London website, which included detailed information about the study, eligibility criteria, and contact details for the study team.

Online classified advertisements

For Studies B and C, advertisements were also placed on Gumtree, a popular website for classified advertisements in the UK. The advert was posted frequently in the section for volunteers and charity work. For these studies, we elected to use only the basic free service, although paid services are available that increase the visibility of advertisements. The postings contained a full advert for the study, including the main inclusion criteria, along with an option to send an email to the researchers.

In summary, Study A used search engine advertisements on Google, Study B used social media advertisements on Facebook as well as online classified advertisements on Gumtree, and Study C used classified advertisements on Gumtree alone. A number of other recruitment strategies were used for Study A (Figure 1), but here we focus on the results of advertising using Google advertisements as our aim here is to assess the viability of online advertisements, and we had limited data regarding the effectiveness and cost of the other methods. All three studies also used traditional clinical recruitment methods, and while we did not collect data on the costs of these methods, we performed an exploratory comparison of success rates between online and traditional recruitment methods to provide an indication of their utility in this form of research.

Figure 1 Results of advertisements for Study A. Abbreviation: GP, general practitioner.

Results

Search engine advertisements

During the time that Study A was open to recruitment, advertisements were shown 21,905,245 times resulting in 29,682 clicks to the website. A total of £10,582.62 was spent in total and the average cost per click was £0.36. A total of 281 individuals who replied to the advert were screened, and 40 of these potential participants were recruited into the study. The total cost per recruited patient was £264.57.

Figure 1 shows how Google advertisements performed compared with a number of other recruitment methods used for Study A during the study time period. Google advertisements were the most successful strategy in terms of number of participants included in the research equating to 35.2% of the total recruited sample.

Social media

During the 7-month recruitment period for Study B, social media advertisements reached 933,567 unique users, resulting in 22,333 clicks. Having excluded hundreds of unsuitable participants after their initial contact, 180 were telephone screened. Seventeen were subsequently included in the study following a full assessment of suitability. Thirteen of these were experiencing an episode of unipolar depression and four experienced bipolar depression. The total spent on these advertisements was £3,486, which equates to £0.15 per unique click and £205.06 per patient recruited. However, while the advertisements were targeted at people with these conditions, a number of healthy volunteers were also recruited through these advertisements, which are not counted in these figures, contributing to reducing the overall cost. Facebook advertisements were also briefly used in recruitment for Study A; however, these were soon discontinued as it was judged at the time that Google advertisements were providing more responses and as such there are insufficient data to discuss their effectiveness here.

Online classified advertisements

For Study B, 372 people emailed to enquire about participation after seeing online classified advertisements and 33 potential participants were phone screened. Only one of these individuals was recruited into the study. For Study C, 284 people who had seen online classified advertisements enquired about participation over an 8-month period. Of these potential participants, 92 were telephone screened and 12 were subsequently included in the study after a full assessment of suitability. As we elected to use the free service, there were no costs associated with posting these advertisements.

Because of the large discrepancy between the number of initial contacts and the number of participants recruited into the studies, the reasons for exclusion were explored. As shown in Figure 2, the most common reason was lack of engagement following the initial email sent by the researcher to provide information about the study. Further common reasons for exclusion included not being symptomatic or currently receiving treatment.

Figure 2 Reasons for exclusion of participants from classified advertisements. Abbreviation: MRI, magnetic resonance imaging.

Comparison with traditional recruitment methods

All studies also attempted to recruit through primary care services. For Study A, posters were placed in General Practitioner practices in the Manchester area but no participants were recruited through this method. Study B and C both recruited through Improving Access to Psychological Therapies services in South London. Improving Access to Psychological Therapies is a service that bridges primary and secondary care providing access to psychological therapies via direct referrals but also accepting self-referrals directly from clients. Caseloads were regularly reviewed by researchers to identify potentially suitable service users. Four and three participants were included in Study B and C, respectively, through this route. No patients were recruited via secondary care services.

In terms of direct costs, these methods are certainly less expensive than online advertisements. However, it should be noted that traveling to GP surgeries to display posters and regularly reviewing clinical services’ caseloads did require a significant time commitment. These methods were therefore not cost-free due to the need to pay a researcher to complete these tasks, although exact time or financial costs were not systematically recorded.

Discussion

We used three methods of online public advertisement to recruit participants with affective disorders into neuroimaging research studies. The main finding is that online public advertisement strategies are viable methods of recruitment and might be of superior efficacy to traditional methods when medication-free participants with relatively less complex affective disorders are sought. Notably, in all three studies online public advertisements proved more effective than recruitment through clinical services. This indicates that advertisements provide a valuable alternative to recruiting from mental health services, which make them especially attractive in affective disorders due to the difficulties in finding participants through these services, in particular when researchers are seeking populations who may not be in contact with clinical services such as those who are untreated at the time of recruitment. Our experiences are in line with previous studies, which found these methods to be effective in recruiting participants for clinical trials.^11,13

While there are benefits to using these methods, they do have important limitations. First, some services can be expensive, and as such funders will need to be sympathetic to the recruitment needs of the study and allow for funding to be allocated for recruitment purposes if these methods are to be adopted. In contrast, advertisements on online classified advert websites are typically free. However, only a minority of respondents to these advertisements were included in our research. Despite this, the negligible cost and prompt exclusion of ineligible individuals make this method worth adopting in addition to other methods. Of interest, these advertisements were far more successful for Study C than for Study B, which may be due to the more lenient inclusion criteria of Study C. This study did not exclude participants with comorbid anxiety disorders, which are common among people with depression¹⁵ and used a lower severity threshold. Therefore, depending on the inclusion criteria of the study, this may be an effective strategy for recruitment.

In addition, we were able to recruit a number of healthy volunteers through these advertisements, despite them being targeted at patient populations. Although we did not record the numbers of healthy volunteers who made contact, and therefore cannot evaluate this in more detail, both Study B and C were able to recruit a sufficient number of healthy participants through this route alone. The majority of studies of disease mechanisms in affective disorders require healthy comparison groups, making this an important added benefit of online public advertisements.

Another limitation is the time investment required to address a potentially large number of responses. As mentioned earlier, this is not necessarily a complicated task and can be managed by researchers with no formal clinical training. Also, although not discussed here, web-based screening questionnaires can be utilized to perform initial screening, further reducing time spent on this task. Furthermore, many email responses can be automated to greatly reduce the time spent responding to the large number of contacts generated by advertisements. Previous findings from clinical trials have shown that web-based advertisements tend to be cost-effective even when accounting for the time spent by researchers on recruitment,¹³ indicating that the time commitment required to use these methods is not prohibitively greater than that required for traditional recruitment methods. It should also be noted that these methods do not place any additional pressure on clinicians, and instead move workload from clinical staff to researchers.

The method of recruitment may have some effect on the characteristics of the resulting sample. For example, it would be expected that community samples have less severe forms of the disorders under investigation, resulting in reduced contact with mental health services. Different methods of public advertisement may also target patient groups with distinct demographic characteristics, such as age,¹¹ and this should be considered when planning recruitment strategies. It follows that more severe forms of illness and mental states and/or forms of disease such as manic or severely depressed states might not be accessible through advertisements. Finally, it is important to note that this method of recruitment may not be effective across geographical and cultural contexts, where access to online services may differ.

Finally, there are limitations to the methodology used here that restrict the conclusions that can be drawn from our results. In particular, our three studies differed in many ways other than the recruitment methods used. Any comparisons between recruitment methods used between studies should be interpreted with caution, as differences in success rates may be due to factors such as inclusion criteria or geographical location. Furthermore, although we did not have sufficient data to fully evaluate the effectiveness of other methods such as newspaper advertisements, it should be noted that these methods may also be useful. Nevertheless, these advertisements do have downsides, such as poor control over the flow of responses compared with online advertisements. These methods may also target different demographics, and this may be worth considering in future research.

In addition, we did not record the time spent on screening participants, which would inevitably have had extra costs associated with it and would therefore have increased the costs of these recruitment methods, it follows that the figures provided are based on the upfront costs of advertising. Finally, it is important to mention that one previous study of recruitment methods to a clinical trial found that online methods of advertising were still cost-effective even when accounting for researcher time.¹³

Conclusion

Given the difficulties associated with recruiting patients for research in affective disorders, particularly when unmedicated patients are required, public advertisements are an option that should be given consideration. Potential limitations of this method of recruitment include cost and time demands associated with screening potential participants. Milder forms of affective disorders are more likely to be recruited through these methods meaning that they are not suitable for all research. However, for the majority of studies in affective disorders, online advertisements can be a viable and effective method of recruitment especially if interest focuses on relatively less severe forms of disease and medication-free samples.

Acknowledgments

Study A was funded by an Medical Research Council Clinician Scientist Fellowship to RZ (G0902304). Study B was funded by Academy of Medical Sciences grant AMS-SGCL8 to DA and a National Institute for Health Research PhD studentship to TW. Study C was funded by a Medical Research Council PhD studentship to LM.

This paper represents independent research part funded by the National Institute for Health Research Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King’s College London. DA’s research is funded by the Academy of Medical Sciences (AMS-SGCL8). The views expressed are those of the authors and not necessarily those of the NHS, the National Institute for Health Research, Academy of Medical Sciences, or the Department of Health. The funding organizations had no role in the study design, data collection and analysis, or manuscript approval. The authors would like to thank the staff of the NIHR/Wellcome Trust Clinical Research Facility at King’s College Hospital, Dean Broadhurst, and Michael Kelly in Mental Health Research Network for their support in the conduct of Study B.

Disclosure

AHY is employed by King’s College London and South London and Maudsley NHS Foundation Trust; has given paid lectures and sits on advisory boards for all major pharmaceutical companies with drugs used in affective and related disorders; is lead Investigator for the Embolden Study (AstraZeneca), Brain-Computer Interface Neuroplasticity Study, Aripiprazole Mania Study, and investigator initiated studies from AZ, Eli Lilly, Lundbeck, Wyeth; has received grant funding from National Institute of Mental Health (USA); Canadian Institutes of Health Research (Canada); National Association for Research on Schizophrenia and Affective Disorders (USA); Stanley Medical Research Institute (USA); MRC (UK); Wellcome Trust (UK); Royal College of Physicians (Edin); British Medical Association (UK); University of British Columbia – Vancouver General Hospital Foundation (Canada); Western Economic Diversification Canada (Canada); CCS Depression Research Fund (Canada); Michael Smith Foundation for Health Research (Canada); NIHR (UK). DA has received travel grants from Janssen-Cilag and Servier. TW, LM, RZ, and KEL have no conflicts of interest to declare.

---

References

1.		Harman JS, Veazie PJ, Lyness JM. Primary care physician office visits for depression by older Americans. J Gen Intern Med. 2006;21(9):926–930. doi:10.1111/j.1525-1497.2006.00497.x.
2.		Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet. 2009;374(9690):609–619. doi:10.1016/S0140-6736(09)60879-5.
3.		Keown P, Holloway F, Kuipers E. The prevalence of personality disorders, psychotic disorders and affective disorders amongst the patients seen by a community mental health team in London. Soc Psychiatry Psychiatr Epidemiol. 2002;37(5):225–229. doi:10.1007/s00127-002-0533-z.
4.		Borschmann R, Patterson S, Poovendran D, Wilson D, Weaver T. Influences on recruitment to randomised controlled trials in mental health settings in England: a national cross-sectional survey of researchers working for the Mental Health Research Network. BMC Med Res Methodol. 2014;14(1):23. doi:10.1186/1471-2288-14-23.
5.		Barney LJ, Griffiths KM, Jorm AF, Christensen H. Stigma about depression and its impact on help-seeking intentions. Aust N Z J Psychiatry. 2006;40(1):51–54. doi:10.1111/j.1440-1614.2006.01741.x.
6.		Mojtabai R. Mental illness stigma and willingness to seek mental health care in the European Union. Soc Psychiatry Psychiatr Epidemiol. 2010;45(7):705–712. doi:10.1007/s00127-009-0109-2.
7.		McDonald AM, Knight RC, Campbell MK, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials. 2006;7(1):9. doi:10.1186/1745-6215-7-9.
8.		Hunninghake DB, Darby CA, Probstfield JL. Recruitment experience in clinical trials: literature summary and annotated bibliography. Control Clin Trials. 1987;8(4 Suppl 1):6–30. doi:10.1016/0197-2456(87)90004-3.
9.		Kasenda B, von Elm E, You J, et al. Prevalence, characteristics, and publication of discontinued randomized trials. JAMA. 2014;311(10):1045–1051. doi:10.1001/jama.2014.1361.
10.		Flynn L. The benefits and challenges of multisite studies: lessons learned. AACN Adv Crit Care. 2009;20(4):388–391. doi:10.1097/NCI.0b013e3181ac228a.
11.		Carmi L, Zohar J. A comparison between print vs. internet methods for a clinical trial recruitment – A pan European OCD study. Eur Neuropsychopharmacol. 2014;24(6):874–878. doi:10.1016/j.euroneuro.2014.01.021.
12.		Domingues I, Alderman T, Cadenhead KS. Strategies for effective recruitment of individuals at risk for developing psychosis. Early Interv Psychiatry. 2011;5(3):233–241. doi:10.1111/j.1751-7893.2011.00278.x.
13.		Krusche A, Rohr IR von, Muse K, Duggan D, Crane C, Williams JMG. An evaluation of the effectiveness of recruitment methods: the staying well after depression randomized controlled trial. Clin Trials. 2014;11(2):141–149. doi:10.1177/1740774514521905.
14.		Angst J, Azorin J-M, Bowden CL, et al. Prevalence and characteristics of undiagnosed bipolar disorders in patients with a major depressive episode: the BRIDGE study. Arch Gen Psychiatry. 2011;68(8):791–798. doi:10.1001/archgenpsychiatry.2011.87.
15.		Kessler RC, Berglund P, Demler O, et al. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). JAMA. 2003;289(23):3095–3105. doi:10.1001/jama.289.23.3095.