Placebo-only-controlled versus active-controlled trials of new drugs for nine common life-threatening diseases

Introduction

Previous clinical trials have generated controversy for withholding active interventions from human subjects in control groups.^1,2 While disagreement remains, there is general consensus that widely used treatments proven to be beneficial should not be withheld when doing so would expose human subjects to increased risk of death or irreversible morbidity.^3,4

The Food and Drug Administration (FDA) advises pharmaceutical companies and investigators on appropriate clinical trial design, including choice of control groups, and reviews data from clinical trials prior to approving new drugs for marketing.⁵ While for some trials the FDA encourages companies and investigators to provide active interventions to control group subjects,^6–8 in other contexts the agency advises that placebo-controlled designs are permissible, provided effort is made to minimize risk to human subjects. For example, guidance developed by the International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use and adopted by the FDA states that a “short term” placebo-controlled trial in mild uncomplicated hypertension “might be considered generally acceptable, while a longer trial, or one that included sicker patients, probably would not be.”⁴ Other FDA guidance documents endorse placebo-controlled trials for chronic obstructive pulmonary disease (COPD)⁹ and diabetes, including withdrawing medication in diabetic subjects previously effectively controlled on low-dose antidiabetic monotherapy for up to 6 months.¹⁰ Even in the absence of a public guidance,¹¹ placebo-controlled trials may be effectively required by the FDA in practice, as is the case with placebo-controlled trials for new psychiatric drugs.^12,13

Applications of such FDA advice under new and varied circumstances may result in potential widespread risk to control-group subjects should effective treatments be withheld inappropriately. Yet to our knowledge, no previous research has systematically evaluated the withholding of active interventions in control groups in clinical trials submitted to the FDA. Published studies assessing ethical considerations for placebo-controlled trials have focused on only a small number of trials or diseases.^14–23 Other studies assessed the scientific and methodological variations in the research design in active- and placebo-controlled trials, rather than assessing the proportion of clinical trials for which interventions were withheld from control subjects. These studies generally failed to quantify the number of placebo-only-controlled trials, distinguish the presence of an additional placebo control group in a multi-armed trial, or assess whether subjects in the placebo group were required to be given concomitant or background therapy.^24–30

To our knowledge, the extent to which placebo-only versus active interventions are provided to control group subjects in clinical trials has not been examined across multiple diseases. This study attempts to fill this gap by systematically assessing the distribution and characteristics of placebo-only versus active-controlled clinical trials supporting FDA approvals of new drugs and biologics for nine common life-threatening diseases.

Materials and methods

Primary data source

The FDA’s database of approved drug products (Drugs@FDA) was our primary data source.³¹ This publicly available online database comprises information on all drugs and some biological products approved under a new drug application (NDA) or biologic license application (BLA), respectively.³² Each NDA or BLA includes data from all clinical trials cited by the product manufacturer which demonstrate the relevant safety and effectiveness of the product. The FDA’s review and discussion of these trials are available in medical review documents posted at Drugs@FDA.

Search strategy

Step 1: Identification of NDAs/BLAs

In step 1, we identified all original NDAs and BLAs approved between January 1, 2006, and December 31, 2011, by using the FDA’s Drugs@FDA database. We excluded supplemental approvals, abbreviated approvals, tentative approvals, approvals not based on new clinical trial data, and duplicates (Supplementary methods, Supplementary materials). We also excluded approvals for infectious diseases and cancer (for which the FDA advises against placebo use^7,8), depression (the treatment of which increases, rather than decreases, the risk of suicidality for patients under the age of 25 years), and diagnostic agents. We then reviewed the FDA approval documents and further excluded approvals that did not involve an indication for treating a life-threatening disease, defined as a disease for which long-term mortality risk is elevated, including risk attributable to secondary complications (eg, elevated mortality following fracture in osteoporosis).³³ We then selected nine life-threatening diseases associated with the largest number of approvals: diabetes mellitus (type 1 or 2), hypertension, asthma, schizophrenia, COPD, partial seizures, osteoporosis, Parkinson’s disease, and bipolar disorder. Evidence of elevated mortality risk for each disease is documented in Table S1 in the Supplementary materials.

Step 2: Identification of clinical trials

In step 2, we identified all clinical trials described in the FDA medical review documents for approvals selected in step 1. If the medical review lacked sufficient data, we searched for needed information from clinicaltrials.gov, the peer-reviewed literature, and other public sources (Supplementary Methods, Supplementary materials). The sources used for the final clinical trials included in the study are available in Table S2 in the Supplementary materials.

Clinical trials were excluded if they enrolled only healthy subjects; lacked a control group; enrolled only subjects not requiring treatment for one of the selected diseases; were Phase I, pharmacokinetic, pharmacodynamic, bioavailability, or bioequivalence studies; lacked data required for analysis of the type of intervention provided to control group subjects; or involved a one-time administration of the intervention (Supplementary methods, Supplementary materials). We also verified that no trial expressly required that all subjects had failed all currently available therapies (because they were either ineffective or not tolerated) prior to randomization.

If a trial included multiple phases in which subjects were randomly reassigned to a new intervention, we considered each phase to be a separate trial.

Data abstraction

For each included trial, we abstracted information on the enrolled population, the type of intervention provided to control group subjects, and trial characteristics. Information on the enrolled population included diagnosis and level of disease control on any prior treatment, if available. Information on the type of intervention included drug name, drug dosage, and any reported lifestyle intervention. Trial characteristics included duration of the intervention and, where available, size, start year, location(s), and disease severity of enrolled subjects. SS performed the database search and data abstraction, and a second reviewer, MAC or SA, checked the accuracy of the extracted information against the source documents.

Measures

Outcome

We classified each trial into one of two mutually exclusive categories: active-controlled and placebo-only-controlled, based on the type of intervention provided to control group subjects at the point of randomization. We considered active interventions to include pharmacological as well as dietary or lifestyle interventions (eg, vitamin D and calcium supplements in osteoporosis and diet and exercise counseling in diabetes). Trials were classified as active-controlled if only all the subjects in all control groups received an active intervention. This category included cases where all subjects received concomitant or background therapy, and experimental or placebo intervention was added on at randomization. For example, if all the subjects enrolled in an asthma trial were required to be receiving inhaled corticosteroids (ICS) at study entry, and at randomization, one group received background ICS plus an experimental new agent, and the other received background ICS plus placebo, then that trial was classified as an active-controlled trial. Trials were classified as placebo-only-controlled if subjects in at least one control group were not required to receive any active intervention. This included trials with both active and placebo control arms, provided that the placebo control group was not required to receive any concomitant or background therapy.

SS initially classified the outcome variable for each trial, and a second reviewer, either MAC or SA, performed an independent classification. Disagreements between the reviewers were resolved in conference among SS, SA, MAC, and SMW.

Trial characteristics

We examined trial duration, size, start year, location, and, where applicable, disease severity. Trial duration was measured as the number of weeks from randomization to the end of the intervention period. Trial size was measured as the total actual enrollment (or, where unavailable, projected enrollment) including all experimental and control groups. We also recorded the number of subjects enrolled in each control group, if available. Trial location was measured as developed country, non-developed country, both, or unknown, according to the country or countries where the trial was carried out. Developed countries were defined as “advanced economies” as classified by the International Monetary Fund 2011 World Economic Outlook Report.³⁴ All other countries were defined as non-developed countries.

We recorded the disease severity of subjects at the time of enrollment in trials for asthma, COPD, hypertension, and diabetes only, as consistent criteria enabling classification of severity for other diseases were not available.

For asthma trials, we recorded disease severity as moderate to severe persistent asthma if subjects’ forced expiratory volume in 1 second (FEV₁) measurements were <80% predicted, if asthma symptoms were reported daily, or if the source described “moderate” or “severe” asthma, and as mild persistent asthma if subjects’ FEV₁ measurements were ≥80% predicted, if asthma symptoms were reported ≥2 days a week but not daily, or if the source described “mild” asthma.³⁵ For COPD trials, disease severity was recorded as severe if subjects’ FEV₁ measurements were <50% predicted or the source described “severe” or “very severe” COPD, and as moderate if FEV₁ measurements were 50%–79% predicted or the source described “moderate” COPD.³⁶ For hypertension trials, disease severity was recorded as severe if subjects’ systolic blood pressure (SBP) measurements were ≥180 mmHg or diastolic blood pressure (DBP) measurements were ≥110 mmHg, moderate if subjects’ SBP measurements were 160–179 mmHg or DBP measurements were 100–109 mmHg, and mild if subjects’ SBP measurements were 140–159 mmHg or DBP measurements were 90–99 mmHg.³⁷ For diabetes trials, disease severity was recorded as severely uncontrolled if the subjects’ HbA1c measurements were >10% and moderately uncontrolled if the subjects’ HbA1c measurements were ≥7% and ≤10%.³⁸

Where a trial protocol permitted enrollment of subjects across a range of severities, we recorded the severity for that trial based on the subjects with the highest severity permitted to enroll. For example, a trial enrolling subjects with mild or moderate hypertension was classified as a “moderate” hypertension trial.

Statistical analyses

Descriptive statistics (frequencies and proportions) were calculated for categorical variables: duration, size, start year, country location, and, where applicable, disease severity. Median, interquartile range (IQR), and range were also calculated for trial duration and size. We used the Pearson’s chi-square or Fisher’s exact tests to examine bivariate associations between the outcome variable and trial characteristics. A multivariable logistic regression model was fitted for the overall trial sample to predict the odds of each trial being placebo-only-controlled versus active-controlled. A significance level of 0.05 was used for all comparisons. All statistical analyses were conducted using SAS version 9.3.

Results

Our initial search of Drugs@FDA generated a total of 12,129 NDA and BLA approvals during the study period. After exclusions, we identified 73 approvals related to the nine life-threatening diseases with the most approvals and 1,412 clinical trials associated with these approvals. Of these, 904 trials were excluded, resulting in a final sample of 508 trials (Figure 1). Five of the nine diseases – diabetes, hypertension, asthma, schizophrenia, and COPD – had a total of 452 trials, accounting for 89.0% of all included trials.

Figure 1 Attrition chart describing how the sample of included clinical trials was selected and the basis for excluding Food and Drug Administration (FDA)-approved new drug applications (NDAs) and clinical trials for the included NDA approvals.

Type of intervention provided

Of all 508 trials, 201 (39.6%) were placebo-only-controlled and 307 (60.4%) were active-controlled (Table 1). The total recorded enrollments in control groups provided with placebos only, and in control groups provided with active interventions, were 19,361 and 93,093 subjects, respectively. Some of the subjects provided with active interventions were enrolled in placebo-controlled trials that included one or more active control groups in addition to the placebo group.

Table 1 Classification of trials by type of control group (N = 508) Note: aThe P-value tests the association between the type of intervention and all the nine disease categories. Abbreviation: COPD, chronic obstructive pulmonary disease.

The proportion of placebo-only-controlled trials ranged from 75.5% for COPD to 0.0% for partial seizures and osteoporosis. These differences in the proportions of trials that were placebo-only-controlled among the nine diseases were statistically significant (P<0.001).

Trial characteristics and study outcome

Overall, trial duration ranged from 0.1 to 260 weeks (median [IQR] 12 [8–26] weeks). Trial size ranged from 6 to 8,606 subjects (median [IQR] 417.5 [187–685] subjects). Most of the trials, 305 (60.0%), began enrollment in 2002 or later, and most of them were conducted in developed countries (200 [39.4%]), or both developed and non-developed countries (220 [43.3%]) (Table 2). The majority of hypertension, asthma, and COPD trials enrolled subjects with moderate or severe disease: 86 (73.5%), 80 (90.9%), and 53 (100%), respectively. Approximately a third of diabetes trials enrolled subjects with severely uncontrolled diabetes: 45 (36.9%). Only eight diabetes trials (6.6%) enrolled subjects with type 1 diabetes (numbers not shown in tables).

Table 2 Characteristics of trials Notes: aPercentage represents percent of total trials (508) except for disease severity, where it represents percent within disease category. bDisease severity was assessed only for diabetes, hypertension, asthma, and COPD. Abbreviation: COPD, chronic obstructive pulmonary disease.

Table 3 presents results for the distribution of trial characteristics by the study outcome. Of 37 trials lasting longer than 52 weeks, only one was placebo-only-controlled. By contrast, of 267 trials lasting ≤12 weeks, 154 (57.7%) were placebo-only-controlled. The differences in the proportions of trials that were placebo-only-controlled by trial duration were statistically significant (P<0.001).

Table 3 Trial type by trial characteristics Notes: aPercentages represent percent of total within each category. bThe P-values test the association between the trial type and each trial characteristic. cDisease severity was assessed only for diabetes, hypertension, asthma, and COPD. Abbreviation: COPD, chronic obstructive pulmonary disease.

Placebo-only controls were used in 98 (49.0%), 77 (35.0%), and 1 (33.3%) trials conducted in developed countries, both developed and non-developed countries, and non-developed countries, respectively (P=0.003). However, trial location was unknown for 85 trials (16.7%), and only three trials (0.6%) were conducted exclusively in non-developed countries.

Many trials enrolling the most severely ill subjects were placebo-only-controlled. We identified 40 (76.9%) placebo-only-controlled trials for severe COPD, 22 (56.4%) for severe hypertension, and 35 (43.8%) for moderate or severe persistent asthma, but only three (6.7%) for severely uncontrolled diabetes.

A lower proportion of trials enrolling subjects with severely uncontrolled diabetes than those enrolling subjects with moderately uncontrolled diabetes were placebo-only-controlled (6.7% versus 25.8%, respectively, P=0.005) (Table 3). When diabetes trials with unknown severity were removed from the analysis, this association remained significant (P=0.01). No significant differences in our study outcome were found between disease severity categories for hypertension, asthma, and COPD.

None of the type 1 diabetes trials were placebo-only-controlled. Among the 19 type 2 diabetes trials that were placebo-only-controlled, 15 withdrew diabetes medications from subjects previously taking one (12 trials) or up to two (3 trials) oral agents.

Logistic regression results

Our logistic regression model predicting the odds of a trial being placebo-only-controlled versus active-controlled, as a function of disease type and available trial characteristics for all nine diseases (trial duration, size, start year, and location), showed that both trial duration and disease type were associated with our study outcome (Table 4). Longer trials were associated with lower odds of being placebo-only-controlled compared with shorter trials in 10-week increments (odds ratio [OR] =0.59, 95% confidence interval [CI] 0.50–0.70). COPD, schizophrenia, asthma, and hypertension trials were associated with higher odds of being placebo-only-controlled than diabetes trials (OR =16.68, 95% CI 6.79-40.96; OR =6.61, 95% CI 3.00-14.55; OR =2.18, 95% CI 1.05-4.53; and OR =1.99, 95% CI 1.02-3.90, respectively).

Table 4 Logistic regression for the odds of trial being placebo-only-controlled versus active-controlleda (n=508) Note: aC-statistic = 0.85. Abbreviations: OR, odds ratio; CI, confidence interval; LL, lower limit; UL, upper limit.

Discussion

In this first systematic characterization of the use of placebo-only versus active control groups in clinical trials supporting FDA drug or biological product approvals for nine common life-threatening diseases, we determined that over a third of trials included placebo-only controls, potentially placing over 19,000 human subjects, who were randomly provided with placebos only, at increased risk of harm. The extent of this practice varied according to disease type, disease severity, and trial duration. Some of these placebo uses were predictable, given that the FDA explicitly has endorsed placebo-controlled trials where risk to subjects is relatively low, including short-term trials for mild hypertension and easily controlled diabetes. Yet, we also found placebo-only-controlled trials enrolling subjects with moderate or severe disease, including some trials enrolling subjects with severe hypertension and, less frequently, severely uncontrolled diabetes (HbA1c >10%), conditions for which immediate treatment has been commonly recommended in clinical practice.^37,38

Trials of longer duration were significantly less likely to be placebo-only-controlled, and use of placebo-only controls among trials lasting longer than 52 weeks was rare, a finding that may reflect intention on the part of clinical investigators and sponsors to minimize risks to human subjects in placebo-only-controlled trials by limiting duration. Yet, we identified such placebo-only use in 16.1% of trials lasting >26 and ≤52 weeks, a duration that raises potential ethical concerns.

Use of placebo-only controls varied significantly by disease type. Some of these variations may be due to varying degrees of confirmed risk to human subjects. For example, placebo-only controls may be more common in trials for COPD because treatment has not been established to prolong survival, meaning failure to provide active pharmacological treatment may not increase risk of mortality for subjects (although treatment does reduce the risk of symptom exacerbations, which can lead to life-threatening complications).³⁹ Also, new anti-seizure drugs are typically assessed by adding the new drug to an existing medication, as denying effective seizure treatment can have serious consequences for subjects.⁴⁰ In other cases, placebo use may be driven by methodological considerations. For example, available active treatments for schizophrenia, while shown to reduce mortality,²⁷ have had variable performance in past placebo-controlled superiority trials.¹³ This failure to demonstrate consistent effectiveness has led FDA officials to reject an active-controlled non-inferiority design as unreliable for establishing the effectiveness of new drugs for this disease.^12,13,43 Further exploration of the risks posed to human subjects in such trials is warranted, as is consideration of alternative trial designs that could avoid risk to human subjects while producing scientifically valid results (eg, by enrolling only subjects who have failed to achieve therapeutic goals using all available effective treatments, or establishing non-inferiority or superiority compared with the current standard of care).

Regulatory standards likely influenced the choice of controls in the trials examined in this study. For example, FDA guidance advises that it is ethically appropriate to withdraw therapy from diabetic subjects previously controlled with one low-dose oral agent.¹⁰ We identified this practice in trials involving similar (or slightly sicker) subjects: for 15 trials, investigators withdrew therapy from diabetic subjects previously taking at least one oral agent, with three involving subjects previously taking up to two oral agents. Withdrawing such care and substituting placebos present ethical concerns, even in cases where some or all enrolled subjects may have failed to achieve adequate control using the withdrawn medication, as subjects may be partially responsive to prior treatments and may experience harm from treatment withdrawal.

This study has several limitations. First, we did not examine whether reasonable justifications, such as possible scientific or methodological reasons for withholding active treatment from control groups, were mentioned by investigators in the protocols for these clinical trials. We also did not assess whether these trials included stopping rules or other safety measures to ensure that the design of these studies decreased the possibility of harm to placebo-only-treated subjects. Although our study assumed a potential for risk exposure among placebo-only subjects based on trial design at randomization, it was not designed to assess the degree of risk, including the risk of mortality, or the actual harm, if any, to these subjects post-randomization.

Other general limitations inherent in our data sources need to be described. These include the fact that the publicly available summaries of information available in FDA medical reviews and other sources varied in comprehensiveness, raising the possibility that important information related to the study intervention was not reported. Importantly, for this analysis, we assessed only whether an active intervention was withheld or provided, not whether it fell below the standard of care. We also did not assess whether trials appropriately excluded subjects with comorbidities, utilized appropriate withdrawal criteria, or satisfied the requirements of informed consent. Finally, our findings cannot be generalized to diseases beyond the nine we assessed, particularly to diseases where treatment has a greater impact on mortality (eg, HIV infection).

Conclusion

This study shows that the practice of providing only placebos to control groups is common in trials involving common life-threatening diseases. Because these diseases require ongoing treatment, this practice potentially entails varying degrees of risk to human subjects. Future research is warranted to assess how the use of placebo interventions in trials supporting FDA approval of drugs and biologics adversely impacts placebo-group subjects. Trials involving severe disease or long duration in particular require increased scrutiny from investigators, clinical trial sponsors, institutional review boards, and regulators, particularly the FDA.

Acknowledgments

Funding for this study was provided by Public Citizen. SS’s current affiliation is Deputy Director of Regulatory Affairs, Center for Science in the Public Interest, Washington, DC, USA. Jennifer Rubio, former Public Citizen’s Health Research Group employee, helped in editing this manuscript.

Author contributions

MAC and SMW conceived this project. SS, MAC, SMW, and SA contributed to the study design, data collection, and data interpretation. AA performed the statistical analysis and data interpretation. MAC supervised the study. All authors contributed toward drafting and revising the paper and agree to be accountable for all aspects of the work.

Disclosure

The authors report no conflicts of interest in this work.

References

1.		Lurie P, Wolfe S. Unethical trials of interventions to reduce perinatal transmission of the human immunodeficiency virus in developing countries. N Engl J Med. 1997;337(12):853–856.
2.		Carlson RV, Boyd KM, Webb DJ. The revision of the Declaration of Helsinki: past, present and future. Br J Clin Pharmacol. 2004;57(6):695–713.
3.		World Medical Association General Assembly. Declaration of Helsinki: ethical principles for medical research involving human subjects. Available from: https://www.wma.net/policies-post/wma-declaration-of-helsinki-ethical-principles-for-medical-research-involving-human-subjects/. A cc ess ed March 19, 2018.
4.		International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH Harmonized Tripartite Guideline: Choice of Control Group and Related Issues in Clinical Trials: E10; 2000. Available from: https://www.ich.org/fileadmin/Public_Web_Site/ICH_Products/Guidelines/Efficacy/E10/Step4/E10_Guideline.pdf. Accessed March 19, 2018.
5.		Food and Drug Administration. Notice: International Conference on Harmonisation; choice of control group and related issues in clinical trials; availability. 66 Federal Register 24390-24391. May 14, 2001. Available from: https://www.gpo.gov/fdsys/pkg/FR-2001-05-14/pdf/01-12026.pdf. Accessed March 20, 2018.
6.		Temple R, Ellenberg SS. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 1: ethical and scientific issues. Ann Intern Med. 2000;133(6):455–463.
7.		Food and Drug Administration Center for Drug Evaluation and Research. Guidance for Industry, Antibacterial Drug Products: Use of Noninferiority Trials to Support Approval. Rockville, MD; 2010. Available from: https://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm070951.pdf. Accessed March 19, 2018.
8.		Food and Drug Administration. Guidance for Industry Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics Guidance for Industry Clinical Trial Endpoints for the Approval of Cancer Drugs and Biologics. Rockville, MD; 2007. Available from: https://www.fda.gov/downloads/Drugs/Guidances/ucm071590.pdf. March 19, 2018.
9.		Food and Drug Administration Center for Drug Evaluation and Research. Guidance for Industry, Chronic Obstructive Pulmonary Disease: Developing Drugs for Treatment. Rockville, MD; 2007. Available from: https://qibawiki.rsna.org/images/4/41/FDA-COPDGuidance-20100201.pdf. Accessed March 19, 2018.
10.		Food and Drug Administration Center for Drug Evaluation and Research. Guidance for Industry, Diabetes Mellitus: Developing Drugs and Therapeutic Biologics for Treatment and Prevention. Rockville, MD; 2008. Available from: https://www.fda.gov/downloads/Drugs/.../Guidances/ucm071624.pdf. Accessed March 19, 2018.
11.		Food and Drug Administration. Search for FDA Guidance Documents. Available from: https://www.fda.gov/RegulatoryInformation/Guidances/. Accessed March 19, 2018.
12.		Feifel D. The use of placebo-controlled clinical trials for the approval of psychiatric drugs: part I-statistics and the case for the “greater good”. Psychiatry (Edgmont). 2008;6(3):41–43.
13.		Laughren TP. The scientific and ethical basis for placebo-controlled trials in depression and schizophrenia: an FDA perspective. Eur Psychiatry. 2001;16(7):418–423.
14.		Rothman KJ, Michels KB. The continuing unethical use of placebo controls. N Engl J Med. 1994;331(6):394–398.
15.		Stein CM, Ray WA. The ethics of placebo in studies with fracture end points in osteoporosis. N Engl J Med. 2010;363(14):1367–1370.
16.		Tramèr MR, Reynolds DJM, Moore RA, McQuay HJ. When placebo controlled trials are essential and equivalence trials are inadequate. BMJ. 1998;317:875–880.
17.		Garattini S, Bertelé V, Banzi R. Placebo? no thanks, it might be bad for me! Eur J Clin Pharmacol. 2013;69(3):711–714.
18.		Miller FG. Unnecessary use of placebo controls: the case of asthma clinical trials. Arch Intern Med. 2002;162(15):1673–1677.
19.		Estellat C, Ravaud P. Lack of head-to-head trials and fair control arms. Arch Intern Med. 2012;172(3):237–244.
20.		Emsley R, Fleischhacker WW. Is the ongoing use of placebo in relapse-prevention clinical trials in schizophrenia justified? Schizophr Res. 2013;150(2–3):427–433.
21.		Tashiro S, Yamada MM, Matsui K. Ethical issues of placebo-controlled studies in depression and a randomized withdrawal trial in Japan: case study in the ethics of mental health research. J Nerv Ment Dis. 2012;200(3):255–259.
22.		Kaufman KR. Comparative bioethics in bipolar and epilepsy research. Seizure. 2002;11(1):51–56.
23.		Miller FG, Joffe S. Equipoise and the dilemma of randomized clinical trials. N Engl J Med. 2011;364(5):476–480.
24.		Brunoni AR, Tadini L, Fregni F. Changes in clinical trials methodology over time: a systematic review of six decades of research in psychopharmacology. PLoS One. 2010;5(3):1–16.
25.		Kesselheim AS, Myers JA, Avorn J. Characteristics of clinical trials to support approval of orphan vs nonorphan drugs for cancer. JAMA. 2011;305(22):2320–2326.
26.		Manalo IF, Gilbert KE, Wu JJ. An updated survey for the 2007-2013 period of randomized controlled trials for psoriasis: treatment modalities, study designs, comparators, outcome measures and sponsorship. J Eur Acad Dermatology Venereol. 2015;29(10):1945–1950.
27.		Olfson M, Marcus SC. Decline in placebo-controlled trial results suggests new directions for comparative effectiveness research. Health Aff (Millwood). 2013;32(6):1116–1125.
28.		Thomas RH, Freeman MK, Hughes PJ. Preapproval and postapproval availability of published comparative efficacy research on biological agents. Am J Health Syst Pharm. 2013;70(14):1250–1255.
29.		Wang B, Kesselheim AS. Characteristics of efficacy evidence supporting approval of supplemental indications for prescription drugs in United States, 2005-14: systematic review. BMJ. 2015;351:h4679.
30.		Schwartz LM, Woloshin S, Zheng E, Tse T, Zarin DA. ClinicalTrials.gov and Drugs@FDA: a comparison of results reporting for new drug approval trials. Ann Intern Med. 2016;165(6):421–430.
31.		Food and Drug Administration. Drugs@FDA: FDA approved drug products. Available from: http://www.accessdata.fda.gov/scripts/cder/drugsatfda/. Accessed March 19, 2018.
32.		Food and Drug Administration. Drugs@FDA Frequently Asked Questions. Available from: http://www.fda.gov/Drugs/InformationOnDrugs/ucm075234.htm. Accessed March 19, 2018.
33.		Cummings SR, Melton LJ. Epidemiology and outcomes of osteoporotic fractures. Lancet. 2002;359(9319):1761–1767.
34.		International Monetary Fund. World Economic Outlook: Tensions from the Two-Speed Recovery. Washington, DC; 2011. Available from: https://www.imf.org/external/pubs/ft/weo/2011/01/. Accessed March 19, 2018.
35.		National Asthma Education and Prevention Program. Expert Panel Report 3: Guidelines for the Diagnosis and Management of Asthma Full Report 2007; 2007. Available from: https://www.nhlbi.nih.gov/files/docs/guidelines/asthgdln.pdf. Accessed March 19, 2018.
36.		Global Initiative for Chronic Obstructive Lung Disease. Global Strategy for the Diagnosis, Management, and Prevention of Chronic Obstructive Pulmonary Disease; 2010.
37.		National High Blood Pressure Education Program. The Sixth Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure; 1997. Available from: http://www.sld.cu/galerias/pdf/servicios/hta/6to._reporte_del_jnc_usa.pdf. Accessed March 19, 2018.
38.		Nathan DM, Buse JB, Davidson MB, et al. Management of hyperglycemia in type 2 diabetes: a consensus algorithm for the initiation and adjustment of therapy. Diabetes Care. 2006;29(8):1963–1972.
39.		Calverley PMA, Anderson JA, Celli B, et al. Salmeterol and fluticasone propionate and survival in chronic obstructive pulmonary disease. N Engl J Med. 2007;356(8):775–789.
40.		Perucca E. Designing clinical trials to assess antiepileptic drugs as monotherapy: difficulties and solutions. CNS Drugs. 2008;22(11):917–938.
41.		Veterans Administration Cooperative Study Group on Antihypertensive Agents. Effects of treatment on morbidity in hypertension. JAMA. 1970;213(7):1143–1152.
42.		Khan A, Schwartz K, Stern C, et al. Mortality risk in patients with schizophrenia participating in premarketing atypical antipsychotic clinical trials. J Clin Psychiatry. 2007;68(12):1828–1833.
43.		Temple R, Ellenberg SS. Placebo-controlled trials and active-controlled trials in the evaluation of new treatments. Part 1: ethical and scientific issues. Ann Intern Med. 2000;133(6):455–463.