Hospitalization Endpoint in Clinical Trials of Outpatient Settings: using Anti-SARS-COV-2 Therapy as an Example

Introduction

SARS-COV-2 or COVID-19 is a global health-threatening pandemic that was initially detected in Wuhan in Hubei Province, China, in December 2019.¹ The virus causes acute atypical respiratory disease, and the World Health Organization (WHO) Emergency Committee declared it a global health emergency in January 2020.² More than 80% of infections were mild and approximately 14% were severe.³ Studies have shown that 75% of patients with severe disease progress to pneumonia possibly requiring hospitalization and critical care admissions.² Disease severity is strongly related to patients’ age, with adults aged over 65 accounting for 80% of hospitalisations. Furthermore, the severity of COVID-19 is strongly linked to health issues such as hypertension, diabetes, obesity, cardiovascular disease, and respiratory conditions.⁴

Many clinical trials on interventions have been conducted over the past three years to tackle the pandemic. For instance, as of 15 December 2022, there have been over 4, 800 clinical trials of COVID-19 interventions on ClinicalTrials.gov. In response to this pandemic and the growing number of COVID-19 trials, a WHO-led working group developed a minimum set of outcome measures for COVID-19 trials.⁵ This set included three elements: measure of viral burden, measure of patient survival and measure of patient progression through the healthcare system, using the WHO Clinical Progression Scale.⁵ However, these core outcomes are more likely to be applicable to trials conducted on hospitalized patients, and there is a need to set standards for outcomes that capture different settings of care, such as non-hospitalized patients.⁶

One of the options that has been widely used in COVID-19 trials in non-hospitalized patients with mild-to-moderate disease is the hospital admission rate.^7–9 However, evaluating the overall impact of hospitalization is not feasible and may have certain limitations. Therefore, this study aimed to review the reported endpoints in clinical trials conducted on non-hospitalized patients with mild-to-moderate COVID-19 in terms of measuring intervention efficacy. Additionally, the study discussed in particular the use of hospitalization as a primary endpoint, using clinical trials for anti-SARS-CoV-2 therapies as an example, and its potential pitfalls.

Materials and Methods

Search and Study Selection

In this narrative review, we searched MEDLINE and ClinicalTrials. gov from January 2020 to December 2022 for Phase III randomised controlled trials (RCTs) in outpatients with mild-to-moderate COVID-19, using either anti-SARS-COV-2 monoclonal antibodies or antiviral agents.

The terms used for searching the literature included those related to the disease of interest (COVID-19, Severe Acute Respiratory Syndrome [SARS]), population of interest (non-hospitalised COVID-19 patients, outpatients with COVID-19, mild or moderate COVID-19 patients, high-risk COVID-19 patients, ambulatory COVID-19 patients), and management of interest (anti-COVID19 monoclonal antibodies (MAB), anti-SARS-COV-2 antibodies products, and antiviral treatment for COVID-19) (see Appendix A for the definitions of the used search terms). The present review conducted base on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guideline.

We included all phase III experimental studies (RCTs) if they met our PICO-D criteria: (1) that related to the population of interest: outpatient subjects with a mild to moderate COVID-19; (2) intervention of interest: anti-SARS-COV-2 monoclonal antibodies and antiviral for COVID-19; (3) comparison of interest: placebo or standard of care; (4) outcomes of interest: treatment efficacy of COVID-19. Only English language publications were included, as non-English papers could not be feasibly translated into English. For research using ClinicalTrials. gov, only completed studies and those with available results were included. The exclusion criteria included Phase I, and Phase II experimental studies (RCTs); studies not having a comparison group; patients were additionally excluded if they had previously been hospitalized for Covid-19; and patients who had previously received a SARS-CoV-2 vaccine.

Screening and Extraction of Data

The titles and abstracts of potentially suitable articles were reviewed. These underwent full-text screening to assess eligibility. The following data were extracted from each eligible study: (1) type of intervention (2) comparator (placebo or standard care); (3) primary objective; (4) primary endpoints; and (5) the use of estimands in the trial.

Results

Summary of Trials on Outpatient Interventions for COVID-19

The search strategy identified 12 Phase III trials (Figure 1) that investigated the management of mild-to-moderate COVID-19 in outpatient settings using monoclonal antibodies or antiviral interventions.^7–18

Figure 1 Flowchart Identifying Trials Included in the Study. Adapted from Moher D, Liberati A, Tetzlaff J, Altman DG: The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6(6): e1000097. Creative Commons.

Outcomes in Included Trials Using Monoclonal Antibodies

Three Phase III trials evaluated the efficacy of anti-SARS COV-2 monoclonal antibodies.^7–9 All the trials used hospitalization and death from any cause as primary endpoints in high-risk patients. Of these, two used COVID-19-related hospitalizations,^7,9 while one used hospitalization for acute illness management.⁸ A summary of which is presented in Table 1.

Table 1 Summary of the Current Clinical Trials Evaluating Monoclonal Antibodies (MAB) in COVID-19 Outpatients

Outcomes in Included Trials Using Antiviral Agents

Nine Phase III studies assessed the efficacy of antiviral agents for the treatment of mild or moderate COVID-19,^10–18 of which, four used hospitalization and death as the main endpoints.^10,12,13,17 All these studies assessed hospitalization in relation to COVID-19, except for one that measured the incidence of hospitalization for any cause. In contrast, two studies used the Reverse Transcription-Polymerase Chain Reaction (RT-PCR) as the primary measure.^14,15 One study assessed the efficacy of umifenovir in patients with mild and moderate COVID-19, and different measures were used based on the severity of the patients.¹¹ For instance, for asymptotic mild patients, the time taken for a nasopharyngeal swab RT-PCR test to be negative was the outcome measure used. For patients with moderate COVID-19, the average change in the ordinal scale from baseline scores on the eight-point WHO ordinal scale was used.¹¹ In addition, one trial used symptom alleviation by Day 7 as a primary efficacy endpoint, while another used clinical response and time to clinical response measures by defining the clinical response as follows: one-order decline in disease category on the five-category ordinal scale. The categories are death, mechanical ventilation, non-invasive ventilation, oxygen mask or nasal cannula and discharge.^16,18 A summary of the trials are presented in Table 2.

Table 2 Summary of the Current Clinical Trials Evaluating Antiviral Treatment in COVID-19 Outpatients

The Estimands

Recently, the estimand framework has offered a crucial method for identifying the treatment effect that must be assessed in a trial. The estimands were not specified and evaluated in all identified studies (both those trials using monoclonal antibodies and antiviral agents). However, the procedures advised by the Cochrane Group or the CONSORT declaration and the current requirement for reporting clinical trials in medical journals still need to consider the importance of estimands to clinical studies.^19–21

Discussion

In this study, we reviewed the outcomes reported in outpatient settings in clinical trials related to COVID-19 therapies. The most common outcome reported was hospitalization. Below, we discuss the potential pitfalls of hospitalization as a primary endpoint in RCTs, as well as possible ways to improve reporting of such outcomes.

The Pitfalls of Using Hospitalization as a Primary Endpoint in RCTs

Hospitalization and emergency room (ER)/hospital admissions as primary endpoints with different hospital utilization patterns and investigator decisions could potentially bias the results of clinical trials, particularly if no specific criteria or definitions of these outcomes are considered.

Furthermore, the rate of hospital admission can be significantly affected by socioeconomic status and national resources. For instance, free universal healthcare and the availability of medical insurance can affect healthcare accessibility, thereby influencing outcome measures. This issue may be more pronounced in trials conducted in multiple countries with different healthcare resources.

The Use of Hospitalization as a Primary Endpoint in COVID-19 Trials

The sponsors of clinical trials for COVID-19 treatments have used various primary endpoints, depending on the trial population, for example, mild disease among outpatients vs severe disease in hospitalized patients. As the outcome measures depend on the infectivity and clinical features of the disease, choosing an appropriate endpoint to measure the efficacy of intervention in high-risk mild-to-moderate COVID19 is challenging. However, in 2020, Marshall et al retrospectively reviewed clinical trials and observational studies conducted on COVID19 patients (regardless of severity).⁵ The results are based on extracted data from WHO registered studies up to April 2020. Most of the primary endpoints measured were the viral load using a quantitative PCR (148 studies), mortality (118 studies), duration of hospitalization (32 studies) and progression or improvement of the clinical symptoms of the patient (175 studies). In addition, respiratory function measured by oxygen saturation using mechanical ventilation or extracorporeal membrane oxygenation (ECMO) were used as primary endpoints in 101 studies based on the WHO COVID-19 infection group review.⁵

A virological endpoint may be helpful for proof-of-concept studies and is generally accepted. However, the primary endpoint of Phase III clinical trials should reflect the primary objectives of the study. The effect of the treatment on mortality was clinically significant. However, it has been acknowledged that death as the primary outcome is not appropriate in an outpatient setting. Although a reduction in mortality would offer undeniable proof of the therapeutic importance of treatment, deaths are uncommon, and large sample sizes are needed for evaluation, which may be unfeasible.

As this study focused on non-hospitalized patients, most of the included Phase III randomized clinical trials for COVID-19 treatment used multiple endpoints. The most common primary endpoint was COVID-19-related hospitalizations and deaths for at least 28 days. Therefore, it is essential to discuss and agree on acceptable endpoints. Ideally, the primary endpoint should be clinically meaningful (ie capturing patient condition and survival). In addition, they must be measurable and sufficiently sensitive to allow for realistic sample sizes. Despite the ease with which relevant clinical outcomes of COVID-19 can be examined and made available within days or weeks, the therapeutic benefits that manifest early but disappear later may not be clinically significant. Additionally, a later treatment effect may go unnoticed because of the early examination. For a novel disease with significant heterogeneity, the endpoint timeframe or timing of evaluation is critical. Although there are clinically significant endpoints, their utilization can be difficult due to social factors and practicalities. For instance, some endpoints may lose their significance when the number of patients exceeds a hospital’s capacity.²²

Improving the Use of Hospitalization as a Primary Endpoint in COVID-19 Trials

Depending on the primary objective of the study, the proportion of patients who were not hospitalized at a predetermined time point and the rate of progression to severe disease may be appropriate for outpatients with mild COVID-19. Therefore, the inclusion of objective criteria for illness progression in the definition of endpoints may be justified. However, endpoints such as hospitalization rate may not be sufficiently objective for acceptance. When states are influenced by external elements such as center and hospital resources, reproducibility and meaningfulness may be distorted. For instance, this can occur if the number of patients exceeds a hospital’s capacity. Additionally, the outcomes of multicenter trials may be affected by regional variations in standards of care (including different guideline recommendations). Ideally, clinical studies should follow uniform protocols; however, dogmatic limits can reduce enrolment.²³ Assessing the need for oxygen therapy (a more precise COVID-19 metric) can account for regional variations in practice and situations where patients would typically be hospitalized but not due to a shortage of hospital space. However, evaluating daily SpO2/FiO2 until discharge, death, or 28 days is an effective indicator of reproducibility.

The inability to capture multiple disease states is another limitation of hospitalization. Patient-reported results, which capture aspects important to patients, can address this issue.²³ Additionally, the use of hospitalization to estimate the treatment effect could be confounded by events that occur after initiation of treatment, such as the use of alternative medications or discontinuation of the assigned treatment. Such events have been termed intercurrent events. These events may have an impact on the measurements themselves or their interpretation.²⁴ The crucial problems of concurrent events and summary measurements are addressed using an estimands framework, and with the help of these components, the treatment impact is correctly specified. A specific definition of the estimand enables a more transparent evaluation of whether the estimating method effectively answers the relevant clinical question.¹⁹

Strength and Limitations

The review article demonstrated several strengths in its approach. The comprehensive search strategy utilized reputable databases like MEDLINE and ClinicalTrials.gov, ensuring a thorough exploration of relevant literature, while focusing on Phase III RCTs prioritized high-quality evidence and enhanced the reliability of findings. Clear selection criteria based on PICO-D elements and adherence to PRISMA guidelines further contributed to methodological rigor. On the other hand, the article provided valuable insights into the advantages and pitfalls of using hospitalization as a primary endpoint in COVID-19 trials, this clear presentation facilitated understanding and evaluation of the arguments presented. Moreover, the relevance of the discussion to the latest COVID-19 pandemic underscored its importance to clinicians, researchers, and policymakers. Additionally, the inclusion of a discussion on the estimands framework added depth to the methodological considerations, highlighting the importance of specifying treatment effects in clinical trials.

However, the article also exhibited certain limitations. Language bias, stemming from the exclusion of non-English publications, might have led to the omission of valuable evidence published in other languages. Similarly, the focus on Phase III RCTs might have overlooked insights from earlier-phase trials or observational studies. The potential for publication bias, as well as the absence of risk of bias assessment in included studies, could impact the robustness of the evidence synthesis. Lastly, the absence of external peer review might have impacted the validation and rigor of the analysis presented.

Conclusion

In a clinical development program of anti-SARS COV-2 monoclonal antibodies and antiviral treatment, the evaluation of the therapeutic efficacy in non-hospitalised patients with mild-to-moderate COVID-19, at high risk for progression to severe COVID-19 and choosing an appropriate primary endpoint is challenging. The most commonly used primary efficacy endpoint for Phase III was the “proportion of participants who experience COVID-19-related hospitalisation or death within at least 28 days”. This measure can predict the extent of clinical benefit of COVID-19 treatment using either monoclonal antibodies or antiviral agents. However, hospitalisation has poor performance with different characteristics, such as dependence on resources. Therefore, variables that could affect this measure should be assessed by the investigator using specific criteria. In addition, these factors could affect the endpoint through reproducibility and the ability to capture multiple clinical states. Several ideas for minimising these issues have been discussed based on recent evidence. In conclusion, regardless of the primary endpoint selected, collecting fundamental outcome measures will ensure comparability between studies and will be crucial for later attempts to synthesise data from various trials.