Patient involvement in clinical research: why, when, and how

Introduction

Patients are gaining a more active role in health care systems. The move toward patient-centered medicine aims to provide the best health care for each individual patient, taking his or her goals, preferences, and values into account.¹ Doctor–patient relationships are changing, and concepts such as shared decision making and patient empowerment² are becoming a reality.

Patient advocacy groups now claim that their opinions must have greater influence on the decisions that affect them, which is reflected in the phrase “nothing about me without me”. The development of new health care management models where patients become clients and the enormous expansion of information technology are additional factors that contribute to accelerate this change.³ Patient-centered medicine cannot be practiced without patients participating in their own health care decisions and in the research that informs such decisions.⁴

Although this cultural shift is beginning to change the way we understand health care, it is not having the same impact on the research process.⁵ This may be because society does not see patient responsibility to participate in research as obvious as the responsibility to participate in their own medical care.⁴ According to the predominant culture, research is performed on patients, not with patients.⁶ Thus, patients continue to be regarded as a source of data and not as the true protagonists in the process. Initiatives have been developed in recent years to change this situation, but quite often these end up being merely tokenistic.⁷ This paper analyzes the reasons why patients should be more actively involved in research and designates the phases in which they could participate (Figure 1). Moreover, there are specific examples of how patients can achieve the level of true partnership in the research process.

Figure 1 How to engage patients in clinical research?

Identifying research priorities

The fact that the questions of clinical research are scientifically relevant does not necessarily mean that they are relevant from the patient’s perspective.⁵ Although patients should play an active role in setting research priorities, such participation continues to be the exception rather than the rule.^8,9 Patient involvement is essential for achieving true translational research.¹⁰ Nevertheless, most research questions are posed from a medical or regulatory perspective, and they are often based on what has been called the “culture of the laboratory”, which is excessively focused on basic science and often removed from the true needs of patients.^11,12

The Patient-Centered Outcomes Research Institute¹³ in the USA and the James Lind Alliance (www.lindalliance.org)¹⁴ in the UK have outlined the differences in research priorities between doctors and patients and instituted measures to ensure that patients participate in the process. The Patient-Centered Outcomes Research Institute has started to include patients in the review and prioritization of its research projects,¹³ and the aims of the James Lind Alliance include the creation of a database of the principal uncertainties surrounding medical treatments (UK DUETs -database of uncertainties about the effects of treatments). These uncertainties are classified as known–unknown, in order to establish where new evidence would need to be generated, and which uncertainties already lend themselves to a systematic review of current evidence.¹⁵ The Alliance studies have been based on systematic prioritization devised jointly by patients, caregivers, and clinicians. The results have made clear that aspects related to real-life management of disease matter most to patients. For example, in a study performed on patients with asthma, the top priorities were management of the adverse effects associated with bronchodilator treatment and management of asthma associated with other health conditions.¹² The identification and proper handling of patient needs in clinical research will yield advantages in terms of clinical and economic benefits. It has been estimated that up to 80% of the expense involved in gathering evidence in research could be avoided.¹⁶ Considering patients’ priorities could contribute to more efficient clinical research.

Leading and designing research

Although it is not yet common practice, some patient associations have begun to lead research projects.¹⁷ The increasing development of health social networks and crowd-sourced studies has enabled organizations such as PatientsLikeMe and 23andMe to promote research projects led by patient groups.¹⁸ There are actually several examples of studies that use this method.¹⁹ The proliferation of these networks, based on the altruistic transfer of personal data by patients, requires proper handling in terms of privacy and data protection.

Patient involvement is crucial for identifying the questions to ask and the outcomes to assess,⁴ therefore it is increasingly common to involve patients or patient advocacy groups on study design.^20–22 This involvement could help to achieve more relevant results, while at the same time improving patients’ recruitment and retention. Conducting comparative effectiveness research²⁰ based on real-life data, long-term follow-up, comparisons with best current treatments, greater emphasis on individualized treatment, and, most of all, selection of variables relevant to patients or patient-reported outcomes (PROs)²¹ should become a priority for patients engaged in study design.²²

It has been stated that >75% of patients with diabetes would prefer that clinical trials include variables that measure the impact of the disease on their quality of life, such as the onset of kidney failure and dialysis, or blindness, rather than surrogate variables such as HbA1c.²³ In oncology, there are examples that demonstrate the advantages of including patient-experience end points and not only the traditional survival-based end points.²⁴ There are similar examples for other diseases.^23,25–27 Worthy of special mention is the OMERACT (Outcome Measures in Rheumatoid Arthritis Clinical Trials) project, which has demonstrated the benefits of increasing involvement of patients with rheumatologic diseases in the design of clinical trials, particularly pertaining to identification of variables that matter to them.²⁷ It is encouraging that the regulatory agencies have already taken some steps to ensure the participation of patients in study design on the assumption that without this perspective the risk-benefit trade-off analysis for new pharmaceuticals would be incomplete.

Although an increasing number of clinical studies include them, the use of PROs, defined as “any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else”²⁸ is still limited.²⁹ Only 18% of clinical trials in the area of diabetes³⁰ and 16% of trials in the cardiovascular field³¹ include PROs. Likewise, fewer than 30% of drug data sheets include PROs.^32,33 The development of guidelines by regulatory agencies on the inclusion of PROs in the evaluation of new drugs³⁴ will probably increase their use in the coming years.

To be useful in clinical research, PROs must be relevant and adequately validated.³⁵ Unfortunately, this has not been the case with many of those used in research^36,37 and the scales continue putting too much emphasis on symptoms. Simple and easily measurable PROs based on objective metrics³⁸ that go beyond the mere symptoms of the disease must be developed and utilized. These must evaluate aspects such as quality of life and functional status, using global ratings provided by the patients themselves.³² The development of computerized health applications (apps) can facilitate the use of PROs. However, it must be ensured that such applications actually measure what they are intended to measure³⁹ and that they are adequately regulated.⁴⁰

Improving access to clinical trials

If participation in clinical research is a societal obligation,⁴¹ then the ability to do so should be a patient right. More than 70% of the general population believes that patients need more opportunities to participate in clinical trials.⁴² In spite of this, only a small portion of patients who could participate in a clinical trial are able to do so. Participation in clinical trials typically does not exceed 5% of patients with cancer.⁴² This is because only those patients whose doctors are also investigators participate in clinical trials. The final aim of everyone involved in the development of clinical trials and of health care systems that seek excellence should be to enable all patients who meet the selection criteria to participate in research, if they desire to do so.

Access for more patients, and for patients with different perspectives, would speed up obtaining relevant knowledge and results that are more applicable to heterogeneous groups of patients. Access to research is influenced by socioeconomic, racial, and ethnic factors and by the type of health insurance.^43,44 Programs to prevent such disparities should be set up to ensure that all patients who wish to participate in clinical trials can do so. It would be very beneficial for improving patients’ access to clinical research to enhance and systematize information about ongoing studies and about the sites participating in them. Patient associations should receive this information so they can pass it along with their associates. In addition, opportunities offered by new technologies should not be neglected. In this regard, clinical trial databases should include information about the characteristics of ongoing studies as well as details about the participating sites. There are already several drug agencies (eg, www.clinicaltrialsregister.eu in the European Union) that include such information. Recent initiatives, such as Trials4Me (http://trials4me.lillycoi.com/) which uses Google Maps to locate sites participating in clinical trials from the Clinicaltrials.gov database, will greatly facilitate patient searches. Just as there is a unified reference database for clinical trials (ClinicalTrials.gov), a platform should be created with unified and standardized information directed to patients.

The growing development of learning health care systems should give rise to “point of care research”, where clinical research is embedded into clinical practice.^45,46 A large number of patients could have access to comparative effectiveness studies in which health interventions used in normal clinical practice are evaluated. This would contribute not only to the more efficient generation of new and relevant knowledge for patients but also to a speedier application of the results. The enormous development of electronic health records and information technologies could help in carrying out randomized database studies⁴⁷ (or randomized registry trials), a method recently termed “the next disruptive technology” in clinical research.⁴⁸ This method proposes the integration of research modules into usual clinical practice in order to combine the main strengths of randomized controlled trials (ie, initial randomization) and registries (ie, naturalistic follow-up). Obviously, in order for the development of big data to achieve its full potential, high-quality data with the end purpose of improving health care results in individual patients must be collected in medical practice and in research.⁴⁹

Adequate information about the study

It must be ensured that patients receive all the information they need to make a free and informed choice before consenting to participate in a study. The informed consent should constitute the means by which the principle of autonomy is applied. For the process to work properly, the information provided to patients must be complete, relevant, and easy to understand. Frequently, both the informed consent document (ICD) and the patient information sheet are excessively long documents that are difficult to read and understand.^50,51 Because of this, ~30% of participants are unaware of basic aspects of the research, such as the existence of random assignment, the possibility that they will be assigned to placebo, the fact that adverse effects could occur during the study, and even the availability of an insurance policy.^52,53

The ICD needs to be restored to its basic function, which is to ensure that patients have suitable information, not only about the overall aims of clinical research but also about the aims and characteristics of the study, its potential risks and benefits, and the option of withdrawing from it if they so desire. For this to happen, the informed consent process should no longer be viewed as an administrative requirement, which is intended to protect the researchers more than the patients themselves.⁵⁴ According to the new Regulation of the European Parliament on Clinical Trials on Medical Products for Human Use, the written information provided to the trial subjects or their legal representatives to obtain informed consent “shall be kept comprehensive, concise, clear, relevant, and understandable to a layperson”.⁵⁵

It is essential for participants to understand that the purpose of clinical research is to generate useful information for future patients and not necessarily to achieve a therapeutic benefit, since this cannot be guaranteed. Nevertheless, many patients participate in clinical trials because of the trust that they have in their doctors and the assumption that their participation will give them access to better treatments. This therapeutic misconception⁵⁶ leads many patients (as well as many researchers) to confuse research with medical care and to tend to overestimate the potential benefits of participating in clinical trials. Although there are times when participation in a clinical trial is the only way in which a patient who has failed with other alternatives can access a new, potentially more effective medication,⁵⁷ systematic reviews offer no evidence that clinical trial participants achieve better outcomes than nonparticipants.⁵⁸

There are few examples of studies where the opinion of health care system users has been considered to improve the design and readability of the patient information sheets included in clinical trials.⁵⁹ It would be highly advisable for patients to participate in ethical discussions related to the studies to make sure that the information provided is clear and accessible. As pointed out earlier, this participation could be achieved not only by having patients involved in study design but also through the participation of patient representatives in the Institutional Review Boards (IRBs). Although IRB members support the participation of patients in their membership lists,⁶⁰ very few IRBs actually put this into practice. Often, for practical reasons, the non-health care professionals who belong to IRBs are members of the hospital administration.⁶⁰ In this regard, the new European Regulation for clinical trials represents a positive step forward, as it⁵⁵ explicitly establishes that “Member States should ensure the involvement of laypersons, in particular patients or patients’ organizations”. Although in theory any citizen could represent the interests of patients, the fact that one is or has been a patient could be an additional advantage, provided of course that the individuals in question are highly motivated and have a sufficient educational level.

It has been suggested recently that, for comparative effectiveness studies on interventions in normal clinical practice, a simplified ICD could be used, similar to the one for medical care, or that the ICD could even be dispensed with.⁶¹ In the case of these “low-intervention clinical trials”, the European Regulation recognizes the need to adapt the ICD,⁵⁵ so patients could play an important role in the review and evaluation of these simplified documents. They could also participate in the design of decision aids that would help to ensure that the consent granted by the patients is truly informed.^62,63

Assessing patients’ experience

Carrying out a clinical trial is an enormously complex process. There are often problems with recruiting or retaining patients, which can make the process even more drawn out and expensive. Once the study is complete, it would be of great value to know about the opinions and experiences of the participants, so future studies could be designed better and achieve greater acceptance by doctors and patients.^64,65

There have been several studies evaluating the experiences of patients who have participated in clinical trials. In a survey covering ~5,000 patients who had taken part in clinical trials at 15 research sites supported by the National Institute of Health, 73% of the participants rated their experience with a score of 9 or 10 on a 10-point scale. In addition, 97% stated that they would definitely or probably recommend a clinical trial to a friend or relative.⁶⁶ These results are similar to those from another study of patients with depression, in which 87% of those interviewed were glad that they had participated in the clinical trial and 75% stated that they would be willing to participate again in the future.⁶⁴ Approximately 30% of the patients in the latter study stated that their participation had increased their trust in their doctors. These and other surveys indicated that one of the main motivations of patients in participating in clinical trials is their willingness to help future patients (with percentages higher than 90%).^64,67 This should be a powerful argument for overcoming the therapeutic misconception and the underlying hidden paternalism.

Despite these positive results, there are several areas that need improvement. In the previously mentioned study,⁶⁶ those surveyed indicated that the ICD did not contain enough details about the study (20%) or about the risks (19%). They said that it had been difficult for them to understand the content (22%), that they had not had enough time to think over their decision before signing the ICD (21%), or that they had received some form of pressure from the research staff to participate (6%). In another study, participants mentioned problems with describing their experiences in the questionnaires that were included in the clinical trial.⁶⁸ Most likely, the patient experiences with respect to participation in clinical trials may vary with factors such as educational level, age, the trial phase, or the disease under study. It would therefore be advisable to collect these experiences in a systematic way to identify the problems and propose solutions adapted to each circumstance.

Informing participants about study results

According to some surveys, ~95% of patients⁶⁹ and members of IRBs⁷⁰ believe that patients should be informed of the results of the research study. Nevertheless, this is a fairly uncommon practice, and the usual situation is that after participating in a study, patients are not notified of the results.⁷¹

The reasons offered for not informing patients have to do with their supposed desire to not know the results and with the possible anxiety that these results could cause. Then there is the difficulty, in practice, of communicating the results or ethical conflicts that a negative result could involve for the doctor in his dual role as physician and investigator.⁷¹ However, it would seem to be an ethical standard, dictated by respect for the participants, to thank them for their participation and to provide them with the aggregate study results before they are made public to the scientific community.⁷² Although informing patients of individual results is a matter of controversy, it could be justified by the same favorable arguments used to support disclosing aggregate results.⁷³ Fortunately, the 2013 update to the Declaration of Helsinki states, for the first time, that patients should have the option to receive information about the overall results of the study.⁷⁴

From a logistics standpoint, giving the participants information about the results is a relatively easy requirement to implement. It would involve merely including this option on the information sheet given to subjects when they are invited to participate in the study, and at the same time establishing how the results will be communicated at the end of the study to those participants who want them.^72,73 Patients would have to be offered a summary of the aggregate results in understandable language. This would detail the most important results, their implications, the possible long-term effects that are foreseen, and how the results will be communicated to the scientific community. Whether the results are positive or negative, it is important for patients to know if the study yielded new information and was useful for the advancement of knowledge.

Several studies have evaluated the satisfaction of patients with different ways of communicating results. In one of these, most of the patients (80%) chose to receive the information in a letter.⁷⁵ Positive experiences have also been reported with presentations made by the investigator to a group of patients.⁷⁶ It would seem that patients prefer to receive the information in a more personalized manner, for example, in a telephone call, rather than impersonally via the sponsor’s press release.⁷⁷ These differences suggest that, ideally, an attempt should be made to communicate the results in a way that fits the preferences of the participants.

Disseminating and applying research findings

The study results should be communicated not only to the study participants but also to the wider public. This obligation is stated in the previously mentioned European Regulations on clinical trials. In 2013, the European Agency upgraded the EudraCT database to enable sponsors to enter result-related information from clinical trials, in line with guidance from the European Commission. Since 2014, with the launch of a final iteration of EudraCT, it is mandatory for sponsors to make publicly available the details of the study, including a summary of the results. Such information should use the language understandable by the general public and be presented in simple and accessible formats. It would be highly advisable to have patients involved in the preparation of these documents.

There should also be mention of the initiatives by some of the scientific journals, such as the Annals of Internal Medicine and the Journal of the American Medical Association, which publish “summaries for patients” and brief and approachable reviews of common diseases and their treatments. An additional step has been taken by the British Medical Journal, which is heading a major campaign to promote and accelerate patient participation in the various areas of the health care field. For example, they have included patients in the peer review process for research articles.⁷⁸

Although the purpose of research is to generate new knowledge that is able to improve outcomes for future patients, many of the results of research are never put into practice or are not implemented for many years. It requires an estimated average of 17 years for only 14% of new discoveries to enter into daily clinical practice.⁷⁹ It is of little use, and indeed would be unethical, to involve patients in research and ask them about their experiences if these are not used to improve care.⁸⁰ The experiences of patients (eg, through PROs) should be included not only in regulatory documents (eg, drug data sheets) but also in clinical practice guidelines. Patient preferences are an essential component in the health care personalization process. There are already guidelines in which recommendations are established on the basis of such preferences.⁸¹ Patient participation with the groups of experts who prepare these clinical guides would ensure that patient perspectives and preferences are considered.

To achieve greater transparency in the research process, some initiatives have been launched aimed at improving the dissemination of research results. In spite of the potential risks involved,⁸² there are unquestionable advantages to the development of an open scientific approach in which “data sharing” is the norm.⁸³ From the standpoint of society, such an approach could generate greater confidence in the results of research, improving clinical trial participation and funding.⁸⁴ The Institute of Medicine has established recommendations for “guide sharing of clinical trial data”.⁸⁵ In addition, the European Medicines Agency⁸⁶ has published standards for promoting transparency and the dissemination of research results and allows different interest groups to request data from studies. Surprisingly, in the past 4 years, only 5.5%, 1.5%, and 0.5% of requests for access to European Medicines Agency documents were submitted by the general public, patient organizations, and nonprofit organizations, respectively.⁸⁶ It would not be surprising if the growing interest in research by patient organizations, many of which already have their own scientific advisors, were to generate an increase in requests for access to patient data.

Information and education: the expert patient

The earlier sections discussed the why, when, and how of active patient participation in research. However, there are two requirements to be met for all of these proposals to be implemented: 1) society must be much better informed on the aims and processes of clinical research and 2) the concept of the “expert patient” must be developed. Regarding the first requirement, there is a need to normalize society’s image of research. The general public’s scant familiarity with the basic aspects of research, and the fact that such familiarity tends to focus on the most negative aspects of research, contributes to it being perceived with a certain degree of suspicion and even fear.

Health care professionals, regulators, communication media, patient associations, and pharmaceutical companies need to collaborate to provide proper information about the purposes of research and the mechanisms available to protect participants. Society must realize that there is no progress without research, and that without patient participation in current studies, there can be no new knowledge to benefit future patients. For the same reason, we owe our current advantages of many medical advances to patients who participated in clinical trials in the past. The development and dissemination of campaigns in the media and educational materials can be of great help. The web pages of regulatory agencies such as the US Food and Drug Administration (http://www.fda.gov/ForPatients/ClinicalTrials/default.htm), national health institutes, such as the NIH in the USA (http://www.nih.gov/health/clinicaltrials/index.htm), and scientific associations such as the American Society of Clinical Oncology (http://www.cancer.net/) include patient-directed information on the basic principles of clinical research. Similarly, some very interesting initiatives have emerged, such as Healthtalkonline (http://healthtalkonline.org/), supported by Oxford University, that include videos in which actual patients relate their experiences with clinical trials and clarify certain basic aspects of the research process.

Second, many of the proposals made in this paper will not be feasible unless they include expert patients.⁸⁷ This concept, based on the idea that implementation of educational and self-care programs could help patients with chronic diseases who take an active role in managing their own conditions,⁸⁸ applies to the field of clinical research. Preparing patients to be experts in research will require an investment in their education. Of particular note here are the training courses offered by the European Patients’ Academy on Therapeutic Innovation (EUPATI), a consortium of 30 European organizations dedicated to providing information and resources on medical research to individuals affected by various diseases. The expert course has been designed to teach patients or their family members all about the clinical trial process, from preclinical research all the way through regulatory approval. The course consists of six training modules: 1) discovery of medicines and planning of medicine development; 2) nonclinical testing and pharmaceutical development; 3) exploratory and confirmatory clinical development; 4) clinical trials; 5) regulatory affairs, medicinal product safety, pharmacovigilance, and pharmacoepidemiology; and 6) health technology assessment principles and practice. The purpose is to empower these patients to participate actively in the research process, by direct collaboration with the industry, with regulatory agencies, or with patient associations.⁸⁹ The education of these expert patients could help to change the current situation, in which “experiential knowledge is often seen as less valuable than scientific knowledge”.⁹⁰

Conclusion

There are many opportunities for patients to participate more actively in the entire research process. As has been noted throughout this paper, some of the proposals are relatively easy to put into practice (Table 1). The lack of implementation suggests that there are cultural barriers holding back the change process. The most urgent need is probably to change the relative emphases assigned to the ethical principles guiding the physician/investigator–patient relationship. The classic relational model, based on the principle of beneficence, which in turn is based on the authority of the physician, may be responsible for the fact that research is performed for patients, but not with them. The therapeutic misconception, the absence of patients on IRBs, the poor quality of information provided to participants, and the low levels of patient participation in establishing research priorities and study design are all elements indicating that the benevolence principle predominates in the research field.

Table 1 Specific initiatives to engage patients in the research process Abbreviations: EUPATI, European Patients’ Academy on Therapeutic Innovation; FDA, Food and Drug Administration; NIH, National Institute of Health; PCORI, Patient-Centered Outcomes Research Institute; OMERACT, outcome measures in rheumatoid arthritis clinical trials; ASCO, American Society of Clinical Oncology.

For the changes described to become realities, physician/investigator–patient relationships should be based on the principles of autonomy and non-maleficence. Moreover, there can be no autonomy as long as patients lack the information they need. Providing the population with more information about the aims of research will help to normalize the process, generating trust and promoting participation in research. There must be improvement in the quality of information provided to participants, and promotion of the emergence of the expert patient: one who can become actively involved in all of the activities described in this paper and who is capable of properly representing the interests of other patients.

Disclosure

JAS, RK and AF are employees of Lilly Spain. The opinions and ideas expressed in this article are personal and do not necessarily represent those of the company. The authors report no other conflicts of interest in this work.

---

References

1.		Laine C, Davidoff F. Patient-centered medicine. A professional evolution. JAMA. 1996;275:152–156.
2.		Lipkin M. Shared decision making. JAMA Intern Med. 2013;173:1204–1205.
3.		Porter ME. What is value in health care? N Engl J Med. 2010;363:2477–2481.
4.		Tinetti ME, Basch E. Patients’ responsibility to participate in decision making and research. JAMA. 2013;309:2331–2332.
5.		Lloyd K, White J. Democratizing clinical research. Nature. 2011;474:277–278.
6.		Thornton S. Beyond rhetoric: we need a strategy for patient involvement in the health service. BMJ. 2014;348:g4072.
7.		Domecq JP, Prutsky G, Elraiyah T, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.
8.		Caron-Flinterman JF, Broerse JE, Bunders JF. The experiential knowledge of patients: a new resource for biomedical research? Soc Sci Med. 2005;60:2575–2584.
9.		Hanley B, Truesdale A, King A, Elbourne D, Chalmers I. Involving consumers in designing, conducting, and interpreting randomised controlled trials: questionnaire survey. BMJ. 2001;322:519–523.
10.		van der Scheer L, Garcia E, van der Laan AL, van der Burg S, Boenink M. The benefits of patient involvement for translational research. Health Care Anal. Epub 2014 Dec 24.
11.		Jenks S. The public applauds cancer research but not how research priorities are set. J Natl Cancer Inst. 1997;89:350–351.
12.		Elwyn G, Crowe S, Fenton M, et al. Identifying and prioritizing uncertainties: patient and clinician engagement in the identification of research questions. J Eval Clin Pract. 2010;16:627–631.
13.		Fleurence RL, Forsythe LP, Lauer M, et al. Engaging patients and stakeholders in research proposal review: the patient-centered outcomes research institute. Ann Intern Med. 2014;161:122–130.
14.		Chalmers I, Atkinson P, Fenton M, Firkins L, Crowe S, Cowan K. Tackling treatment uncertainties together: the evolution of the James Lind Initiative, 2003–2013. J R Soc Med. 2013;106:482–491.
15.		Petit-Zeman S, Firkins L, Scadding JW. The James Lind Alliance: tackling research mismatches. Lancet. 2010;376:667–669.
16.		Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374:86–89.
17.		Vayena E. The next step in the patient revolution: patients initiating and leading research. BMJ. 2014;349:g4318.
18.		Kaye J, Curren L, Anderson N, et al. From patients to partners: participant-centric initiatives in biomedical research. Nat Rev Genet. 2012;13:371–376.
19.		Swan M. Crowdsourced health research studies: an important emerging complement to clinical trials in the public health research ecosystem. J Med Internet Res. 2012;14(2):e46.
20.		Sox HC, Greenfield S. Comparative effectiveness research: a report from the Institute of Medicine. Ann Intern Med. 2009;151:203–205.
21.		Barr PJ, Elwyn G. Measurement challenges in shared decision making: putting the ‘patient’ in patient-reported measures. Health Expect. Epub 2015 Jun 25.
22.		Mullins CD, Vandigo J, Zheng Z, Wicks P. Patient-centeredness in the design of clinical trials. Value Health. 2014;17:471–475.
23.		Murad MH, Shah ND, Van Houten HK, et al. Individuals with diabetes preferred that future trials use patient-important outcomes and provide pragmatic inferences. J Clin Epidemiol. 2011;64:743–748.
24.		Basch E. Toward patient-centered drug development in oncology. N Engl J Med. 2013;369:397–400.
25.		Lindenmeyer A, Hearnshaw H, Sturt J, Ormerod R, Aitchison G. Assessment of the benefits of user involvement in health research from the Warwick Diabetes Care Research User Group: a qualitative case study. Health Expect. 2007;10:268–277.
26.		Ali K, Roffe C, Crome P. What patients want: consumer involvement in the design of a randomized controlled trial of routine oxygen supplementation after acute stroke. Stroke. 2006;37:865–871.
27.		de Wit M, Abma T, Koelewijn-van Loon M, Collins S, Kirwan J. Involving patient research partners has a significant impact on outcomes research: a responsive evaluation of the international OMERACT conferences. BMJ Open. 2013;3(5):e002241.
28.		U.S. Department of Health and Human Services Food and Drug Administration. Guidance for Industry. Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring, MD: U.S. Department of Health and Human Services Food and Drug Administration; 2009.
29.		Dinan MA, Compton KL, Dhillon JK, et al. Use of patient-reported outcomes in randomized, double-blind, placebo-controlled clinical trials. Med Care. 2011;49:415–419.
30.		Gandhi GY, Murad MH, Fujiyoshi A, et al. Patient-important outcomes in registered diabetes trials. JAMA. 2008;299:2543–2549.
31.		Rahimi K, Malhotra A, Banning AP, Jenkinson C. Outcome selection and role of patient reported outcomes in contemporary cardiovascular trials: systematic review. BMJ. 2010;341:c5707.
32.		Gnanasakthy A, Mordin M, Clark M, DeMuro C, Fehnel S, Copley-Merriman C. A review of patient-reported outcome labels in the United States: 2006 to 2010. Value Health. 2012;15:437–442.
33.		Willke RJ, Burke LB, Erickson P. Measuring treatment impact: a review of patient-reported outcomes and other efficacy endpoints in approved product labels. Control Clin Trials. 2004;25:535–552.
34.		Coons SJ, Kothari S, Monz BU, Burke LB. The patient-reported outcome (PRO) consortium: filling measurement gaps for PRO end points to support labeling claims. Clin Pharmacol Ther. 2011;90:743–748.
35.		Garratt A. Patient reported outcome measures in trials. BMJ. 2009;338:a2597.
36.		Vieta A, Badia X, Sacristan JA. A systematic review of patient-reported and economic outcomes: value to stakeholders in the decision-making process in patients with type 2 diabetes mellitus. Clin Ther. 2011;33:1225–1245.
37.		Vieta A, Badia X, Álvarez E, Sacristán JA. Which nontraditional outcomes should be measured in healthcare decision-making in schizophrenia? A systematic review. Perspect Psychiatr Care. 2012;48:198–207.
38.		Wolpert M, Ford T, Trustam E, et al. Patient-reported outcomes in child and adolescent mental health services (CAMHS): use of idiographic and standardized measures. J Ment Health. 2012;21:165–173.
39.		McCartney M. How do we know whether medical apps work? BMJ. 2013;346:f1811.
40.		Kamerow D. Regulating medical apps: which ones and how much? BMJ. 2013;347:f6009.
41.		Harris J. Scientific research is a moral duty. J Med Ethics. 2005;31:242–248.
42.		The Patient Access to Cancer Care Excellence (PACE) [homepage on the Internet]. Available from: https://pacenetwork.com/. Accessed September 28, 2015.
43.		Gifford AL, Cunningham WE, Heslin KC, et al; HIV Cost and Services Utilization Study Consortium. Participation in research and access to experimental treatments by HIV-infected patients. N Engl J Med. 2002;346:1373–1382.
44.		Unger JM, Hershman DL, Albain KS, et al. Patient income level and cancer clinical trial participation. J Clin Oncol. 2013;31:536–542.
45.		Gelijns AC, Gabriel SE. Looking beyond translation – integrating clinical research with medical practice. N Engl J Med. 2012;366:1659–1661.
46.		Sacristan JA. Clinical research and medical care: towards effective and complete integration. BMC Med Res Methodol. 2015;15:4.
47.		Sacristán JA, Soto J, Galende I, Hylan TR. Randomized database studies: a new method to assess drugs’ effectiveness? J Clin Epidemiol. 1998;51:713–715.
48.		Lauer MS, D’Agostino RB Sr. The randomized registry trial – the next disruptive technology in clinical research? N Engl J Med. 2013;369:1579–1581.
49.		Sacristan JA, Dilla T. No big data without small data: learning health care systems begin and end with the individual patient. J Eval Clin Pract. 2015;21(6):1014–1017.
50.		Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med. 2003;348:721–726.
51.		Taylor HE, Bramley DE. An analysis of the readability of patient information and consent forms used in research studies in anaesthesia in Australia and New Zealand. Anaesth Intensive Care. 2012;40:995–998.
52.		Hereu P, Pérez E, Fuentes I, Vidal X, Suñé P, Arnau JM. Consent in clinical trials: what do patients know? Contemp Clin Trials. 2010;31:443–446.
53.		Tam NT, Huy NT, Thoa le TB, et al. Participants’ understanding of informed consent in clinical trials over three decades: systematic review and meta-analysis. Bull World Health Organ. 2015;93:186H–198H.
54.		Dreyfuss D. Is it better to consent to an RCT or to care? Muetadeltaepsilonnu alphagammaalphanu (“nothing in excess”). Intensive Care Med. 2005;31:345–355.
55.		Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/EC.
56.		Henderson GE, Churchill LR, Davis AM, et al. Clinical trials and medical care: defining the therapeutic misconception. PLoS Med. 2007;4:e324.
57.		Agrawal M, Grady C, Fairclough DL, Meropol NJ, Maynard K, Emanuel EJ. Patients’ decision-making process regarding participation in phase I oncology research. J Clin Oncol. 2006;24:4479–4484.
58.		Vist GE, Bryant D, Somerville L, Birminghem T, Oxman AD. Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. Cochrane Database Syst Rev. 2008;(3):MR000009.
59.		Koops L, Lindley RI. Thrombolysis for acute ischaemic stroke: consumer involvement in design of new randomised controlled trial. BMJ. 2002;325:415.
60.		Villen N, Redondo S, Font R, Quintana S. Perfil de los miembros no sanitarios de los comités éticos de investigación clínica y comités de ética asistencial de Cataluña [Profile of lay members of ethics committees in Catalonia (Spain)]. Med Clin. 2012;138:296–299. Spanish.
61.		Pletcher MJ, Lo B, Grady D. Informed consent in randomized quality improvement trials: a critical barrier for learning health systems. JAMA Intern Med. 2014;174:668–670.
62.		Brehaut JC, Fergusson DA, Kimmelman J, Shojania KG, Saginur R, Elwyn G. Using decision aids may improve informed consent for research. Contemp Clin Trials. 2010;31:218–220.
63.		Gillies K, Skea ZC, Campbell MK. Decision aids for randomised controlled trials: a qualitative exploration of stakeholders’ views. BMJ Open. 2014;4:e005734.
64.		Tallon D, Mulligan J, Wiles N, et al. Involving patients with depression in research: survey of patients’ attitudes to participation. Br J Gen Pract. 2011;61:134–141.
65.		Kost RG, Lee LM, Yessis J, Coller BS, Henderson DK; Research Participant Perception Survey Focus Group Subcommittee. Assessing research participants’ perceptions of their clinical research experiences. Clin Trans Sci. 2011;4:403–413.
66.		Kost RG, Lee LM, Yessis J, Wesley RA, Henderson DK, Coller BS. Assessing participant-centered outcomes to improve clinical research. N Engl J Med. 2013;369:2179–2181.
67.		Wendler D, Krohmal B, Emanuel EJ, Grady C; ESPRIT Group. Why patients continue to participate in clinical research. Arch Intern Med. 2008;168:1294–1299.
68.		Holmberg C, Karner JJ, Rappenecker J, Witt CM. Clinical trial participants’ experiences of completing questionnaires: a qualitative study. BMJ Open. 2014;4:e004363.
69.		Partridge AH, Burstein HJ, Gelman RS, Marcom PK, Winer EP. Do patients participating in clinical trials want to know study results? J Natl Cancer Inst. 2003;95:491–492.
70.		MacNeil SD, Fernandez CV. Attitudes of research ethics board chairs towards disclosure of research results to participants: results of a national survey. J Med Ethics. 2007;33:549–553.
71.		Partridge AH, Winer EP. Informing clinical trial participants about study results. JAMA. 2002;288:363–365.
72.		Dal-Re R, Costa J. Información de los resultados a los participantes de los ensayos clínicos [Informing clinical trials participants about the results]. Med Clin. 2007;128:375–378. Spanish.
73.		Shalowitz DI, Miller FG. Disclosing individual results of clinical research: implications of respect for participants. JAMA. 2005;294:737–740.
74.		World Medical Association. World Medical Association Declaration of Helsinki: ethical principles for medical research involving human subjects. JAMA. 2013;310:2191–2194.
75.		Dalal H, Wingham J, Pritchard C, et al. Communicating the results of research: how do participants of a cardiac rehabilitation RCT prefer to be informed? Health Expect. 2010;13:323–330.
76.		Avins AL, Bent S, Padula A, Staccone S, Badua E, Goldberg H. Initial experience with a group presentation of study results to research participants. Trials. 2008;9:16.
77.		Dorsey ER, Beck CA, Adams M, et al; Huntington Study Group TREND-HD Investigators. Communicating clinical trial results to research participants. Arch Neurol. 2008;65:1590–1595.
78.		Richards T, Montori VM, Godlee F, Lapsley P, Paul D. Let the patient revolution begin. BMJ. 2013;346:f2614.
79.		Balas EA, Boren SA. Yearbook of Medical Informatics: Managing Clinical Knowledge for Health Care Improvement. Germany: Schattauer Verlagsgesellscaft mbH; 2000;65–70.
80.		Coulter A, Locock L, Ziebland S, Calabrese J. Collecting data on patient experience is not enough: they must be used to improve care. BMJ. 2014;348:g2225.
81.		Thompson IM Jr, Leach RJ, Ankerst DP. Focusing PSA testing on detection of high-risk prostate cancers by incorporating patient preferences into decision making. JAMA. 2014;312:995–996.
82.		Rosenblatt M, Jain SH, Cahill M. Sharing of clinical trial data: benefits, risks, and uniform principles. Ann Intern Med. 2015;162:306–307.
83.		Ross JS, Krumholz HM. Ushering in a new era of open science through data sharing: the wall must come down. JAMA. 2013;309:1355–1356.
84.		Goodman SN. Clinical trial data sharing: what do we do now? Ann Intern Med. 2015;162:308–309.
85.		Lo B. Sharing clinical trial data: maximizing benefits, minimizing risk. JAMA. 2015;313:793–794.
86.		Bonini S, Eichler HG, Wathion N, Rasi G. Transparency and the European Medicines Agency – sharing of clinical trial data. N Engl J Med. 2014;371:2452–2455.
87.		Elberse JE, Caron-Flinterman JF, Broerse JE. Patient-expert partnerships in research: how to stimulate inclusion of patient perspectives. Health Expect. 2011;14:225–239.
88.		Cordier JF. The expert patient: towards a novel definition. Eur Respir J. 2014;44:853–857.
89.		Chakradhar S. Training on trials: patients taught the language of drug development. Nat Med. 2015;21:209–210.
90.		Elberse JE, Pittens CA, de Cock Buning T, Broerse JE. Patient involvement in a scientific advisory process: setting the research agenda for medical products. Health Policy. 2012;107:231–242.