Risk Management in Clinical Trials: Assessment of Current Practices at Portuguese Clinical Trial Sites

Introduction

With the clinical trials becoming more complex and demanding, it is necessary to review current practices and optimize processes to ensure that quality is not compromised. Aware of this need, the International Conference on Harmonisation (ICH) revised the first version of Good Clinical Practice (GCP) guidelines to encourage implementation of improved and more efficient approaches to clinical trial conduct. In result, risk management is pointed out as an opportunity to accomplish this objective and it has been identified as the sole responsibility of the sponsors.^1–4

Risk management, like any methodology for ensuring the quality of clinical trials, is a shared responsibility of all stakeholders.⁵ In fact, over the last two decades, there has been a growing concern with quality and efficiency in this field. Sponsors, especially industry sponsors, have intensified the risk culture implementation, for example, through the implementation of risk-based monitoring methodologies.^6–8 Regulatory authorities have also taken some steps in the mindset change from a conservative approach, with the objective of ensuring zero defects, to a risk-based approach by which areas of greatest risk are identified and prioritized.^2,3,9 Several guidance and consultation documents aligned with this new way of thinking have been issued by the health regulatory authorities in the beginning of the 2010s.^3,10,11 On the other hand, clinical trial sites seem to be the stakeholder with lesser work developed in this field, showing lack of autonomy in the identification of the potential risks related with their specific processes, the establishment of related corrective and preventive plans, and the maintenance of the acquired knowledge in for future projects.^5,12

At the same time, through their collaboration with different sponsors and projects, clinical trial sites are expected to be well positioned to implement systematic risk management strategies.^13,14 Apart from the advantages to the clinical trial itself and the patients involved, the implementation of a risk management culture by sites can also help them to drive their own performance, strengthening decision-making process, and promoting the more sustainable and efficient use of their resources.¹⁵ Over the last decade several attempts have been made in the UK to demonstrate the usefulness of risk management techniques in healthcare units, mainly to improve patient safety in current medical procedures.^16–18 However, no evidence has been published regarding the usefulness of using risk management tools to improve clinical trials’ success at the clinical site level.

Despite the different tools and approaches to manage clinical trials’ risks at the sponsor or protocol level,¹⁹ these are insufficient as some activities are intrinsically dependent on the sites’ management. Therefore, sites can complement sponsor efforts to achieve the desired results with fewer resources.

However, there is a research gap on this field denoted by the lack of previous studies approaching clinical trial sites to assess the use of risk management procedures and respective guidelines/tools and measure its impact on trial quality and performance indicators.

As it is undoubted the vital role that clinical trial sites play in the success of a clinical trial, it is important to understand their current contribution to the quality management of clinical trials, identify the needs to enhance it, and reflect on the development and practical application of supporting guidelines and tools for sites. Aligned with these purposes, we surveyed clinical trial sites in Portugal as a case study to answer the following questions:

(1) Do clinical trial teams identify and discuss risks of clinical trials at the time of the trial’s feasibility process?

(2) What are the reasons behind the decision of clinical trial teams to perform, or not to perform, a risk assessment?

(3) What are the most common risks identified by the clinical trial teams?

(4) What are the most valuable features that clinical research teams expect from a site-tailored risk assessment tool?

Methods

Identification of Respondents

Our recruitment strategy focused on searching on public registers for the identification of healthcare institutions which conduct clinical trials in Portugal. First, we used the National Registry for Clinical Studies (RNEC) which is the electronic platform for mandatory submission of all clinical trials with medicinal products, medical devices and cosmetic products conducted in Portugal since 01 Jan 2017. The list of the Portuguese clinical trial sites participating in clinical trials submitted through RNEC is publicly available, and it was obtained by filter by “Clinical Study Site” option. The list was then analyzed based on the following criteria:

1. Healthcare institutions will be contacted instead of individual clinical departments;
2. Private hospital groups that aggregate several institutions will be contacted centrally;
3. Only institutions with available contact details will be contacted.

As RNEC does not allow to capture sites conducting clinical trials with different types of intervention besides medicinal products, medical devices or cosmetic products, the Portuguese Clinical Research Infrastructure Network (PtCRIN) was also contacted. By contacting PtCRIN’s members and other entities identified by PtCRIN, we aimed to identify additional clinical trial sites that perform clinical trials with nutritional or behavioral interventions, for example.

The contact details were collected from public sources, such as the entities’ websites, publications, and public databases. Because of the recruitment design, the identity of the respondent clinical trial site was known to the authors. However, the data collected was aggregated and anonymized for analysis and publishing

Development of the Survey

We conducted a cross-sectional study, with data being collected through a survey. The complete survey is available in Additional File 1.

The survey consisted of an original non-validated questionnaire, based on the instrument developed by Hurley et al.²⁰ Prior to its dissemination, the survey was reviewed by eight clinical trial professionals from different clinical trial sites regarding the time to completion, questions’ construction and readability, and suitability of the content to the defined objectives. The feedback received was incorporated in the final version of the survey.

We requested clinical trial sites to answer quantitative and qualitative questions, including multiple choice questions, Likert scales and short-answer questions concerning the risk management methodology in the scope of their clinical trial activities.

The survey consisted of a background information section followed by three main sections. In the first section, respondents were asked to answer four questions with the objective to characterize the clinical trial site, including the institution identification, role of the respondent, number of trials conducted in the last two years, and type of intervention of clinical trials conducted in the last 5 years. In the second section, the respondents’ own perception about the current risk management practices were addressed by questioning the use of a risk management tool or procedure to identify and discuss risks of clinical trials at the time of the trial feasibility process, the reasons behind the decision of clinical trial sites to perform, or not to perform, a risk assessment, and the description of the most common risks identified by the clinical trial sites. Finally, in the last section, respondents’ opinion on the applicability of a risk management tool was requested, and a three-point scale (less important, important, very important) was used for the identification of the most valuable features that clinical trial sites expect from a tailored risk assessment tool. To avoid missing data, all questions were mandatory to answer.

This study was reviewed and approved by the Ethics Committee of the NOVA Medical School on 23 April 2020 (n°26/2020/CEFCM).

Collection of the Answers

After analyzing the RNEC database and removing duplicates, we identified a total of 60 clinical trial sites for contact. By the contact with PtCRIN, eight additional clinical trial sites were identified. Therefore, the total number of institutions identified was 68. We collected the contact details from public sources, such as the entities’ websites, publications, or public databases.

The survey coded to a web application for online survey creation named EUSurvey, developed and maintained by the Directorate-General for Informatics of the European Commission. The survey was accessible through a web link between 12 May 2020 and 30 June 2020.

The survey link was sent by e-mail to all the pre-identified clinical trial sites along with an explanation of the research scope and main objectives. Whenever possible, we directed the survey to the person responsible for the clinical trial activity at the site, namely managers or collaborators of the Clinical Research Units (CRU). Afterwards, weekly reminders were sent by e-mail for the sites with no answer submitted by that date. For non-respondent institutions whose phone contacts were publicly available, reminders by phone call were also performed. After a total of five contacts, the institutions were identified as non-respondents and no further reminders were performed.

Only one answer by the institution was considered. In case more than one completed survey was received for the same clinical trial site, only the last one received was considered for analysis.

Statistical Analysis

We analyzed the data using absolute and relative frequencies for categorical variables. Median values and interquartile range were reported for quantitative variables. Associations between variables were evaluated by applying the Fisher’s exact test. Data were analyzed using SPSS version 25. A p value threshold of ≤0.05 was considered statistically significant. There were no missing data as all responses were complete. Given the descriptive design and a finite number of respondents, we did not formally estimate a required sample size. The raw data analyzed are available for consultation in Additional File 2.

Results

Demographics

From the 60 clinical trial sites identified via RNEC platform, we received 43 answers – response rate of 71.7%. In the same way, from the eight additional sites identified through PtCRIN, we received three answers – response rate of 37.5%. Overall, from the 68 clinical trial sites contacted, 46 were receptive to collaborate in this survey – overall response rate of 67.6%. Among the respondents' institutions were hospital centers, local health units, hospitals (public, private, or public–private partnerships), specialized clinics, academic research networks, and other health institutions involved in the conduct of clinical trials. The complete list of respondents is available in Additional File 3.

Socio-demographic characteristics of the participating institutions are depicted in Table 1. Most surveys were completed by Clinical Research Coordinators (CRC) (46.0%) – who are professionals directly involved in the execution of clinical trial processes at the site – or Clinical Research Units’ (CRU) managers (33.0%) – who are accountable for defining and controlling those processes’ execution.

Table 1 Characteristics of the Participating Institutions by Role of the Respondent, Number of Trials Initiated in the Last 2 Years, and Type of Trial’s Intervention

In what regards clinical trials’ risks, 70.0% of respondent sites highlighted the lack of time of investigators and investigational teams. More than one-fourth of the respondent sites identified the negotiation and financial aspect delays (28.3%), the lack of specialized, professionalized and/or trained staff (28.3%) and the recruitment below expected (26.1%) as one of the three most common risks across their trials (Table 2). Apart from the risks presented in the question’s options, other additional risks were identified by sites, such as the failure to retain participants in the study by non-compliances related to the protocol-related assessments or the inefficacy of articulation with third-party vendors contracted by sites, for example, those provided support on imaging assessments.

Table 2 Most Common Risks Identified by the Clinical Trial Teams in the Conduct of Clinical Trials at Their Clinical Trial Site

Risk Management Practices

Concerning the use of risk management practices in clinical research, most sites (56.5%) reported the use of a systematic tool or procedure to support the identification and evaluation of the clinical trials’ risks at the site level. When requested to describe the strategy used, these sites identified mainly the non-systematic analysis of the protocol by the Principal Investigator (PI) and/or CRU (32.6%) mainly to assess recruitment target and major logistical aspects, the use of checklists or questionnaires to support risk identification (13.0%), and the use of systematic risk analysis tool (6.5%) (Table 3).

Table 3 Current Use of Risk Management Methodologies in the Clinical Trial-Related Operations, and Description of the Corresponding Risk Management Tools or Procedures Identified by the Clinical Trial Teams at Their Sites

The use of risk management tools across the different roles of the respondents was explored, but no significant differences were found (p = 0.89). Similarly, we observed that the number of clinical trials initiated in the last two years is not associated with the use of a standard tool (p = 0.40).

The reasons why clinical research sites use or not a risk management tool were questioned. Regarding the sites that use a standard tool, most reported the anticipation of possible difficulties (88.5%), the guarantee of patient safety (69.2%) and the allocation of the required staff (53.8%). Among the sites that do not use a tool, the most common reason pointed was the lack of experience in performing a risk analysis (80.0%). The tool’s anticipated complexity (45.0%) and the fact that sites’ risk assessment is not an ICH-GCP requirement (45.0%) also showed to motivate the lack of a standardized procedure (Table 4).

Table 4 Most Common Reasons Identified by Clinical Trial Teams for or Against the Use of a Risk Management Tool or Procedure in Their Trial-Related Operations

Willingness to Use a Risk Management Tool and Its Characteristics

When asked about the willingness to use a tool specifically developed for clinical research sites to facilitate the risk assessment and analysis, 87.0% of the sites believe it would be useful, while only 4.4% refusing the use of the tool and 8.7% answering “Maybe”. From this last group, sites argued that they are willing to use the tool if it demonstrates practical utility and applicability to the daily tasks.

Concerning the characteristics and features that clinical research sites would value most in a tool to support risk management at the site level, sites were requested to rank their preferences according to a 3-point scale from “Low relevance” to “Very Relevant” (Figure 1). “Very Relevant” was the most common classification for all the pre-defined characteristics/features, except for the availability of a paper-based version that was considered as “Low Relevance” by most sites (67.0%).

Figure 1 Most valuable characteristics and features identified by respondent clinical trial sites for the development of a risk management tool or procedure. Abbreviations: ICH, International Conference on Harmonisation; GCP, Good Clinical Practice; RNEC, National Registry for Clinical Studies; PtCRIN, Portuguese Clinical Research Infrastructure Network; CRU, Clinical Research Units; CRC, Clinical Research Coordinators; PI, Principal Investigator.

Discussion

In this work, our focus was the assessment of risk management practices by clinical trial teams at healthcare units.

From the answers to this survey, it was observed that the majority handles mainly clinical trials with medicinal products. The number of trials with medical devices or other interventions such as nutrients, cosmetics, or behaviors seems to be very low, unveiling a great potential to develop these types of research in the country. These results are consistent with the data publicly available at clinicaltrials.gov and other publications.^21,22 By the role of the respondent, it was also noted that most respondent sites have professionals fully dedicated to clinical research activities, and at least one-third have a CRU for its coordination and management. These results are in line with previous studies that discuss the improvement in the development of structures to support clinical research activities at the Portuguese healthcare institutions.^12,23

According to the respondents, the lack of time of the investigation teams to dedicate to clinical research is the main risk that impacts the site performance in the conduct of clinical research. Any other risk was much less pointed out by the respondents, corroborating the findings described in recent publications on this issue in several regions.^24–26 The staff’s availability to perform the trial-related tasks will undoubtedly impact the quality of those activities, increasing the probability of almost every identified risk to occur. It is possible that the staff is wasting too much time with repeated and administrative tasks which can be harmonized and centralized in the CRU. Furthermore, clinical trial teams can also be spending unnecessary time solving and managing issues that could be prevented in the first place by anticipating them.

Even though risk management is having considerable attention from the stakeholders in the last years, it needed a more in-depth assessment of the risks that may arise at the site level and the strategies that the clinical trial sites can implement to mitigate them upstream. When asked about their willingness to use a risk management tool, sites highlighted as the most important feature the clear identification of risks and mitigation strategies. It also valued its easy adaptation to different therapeutic areas and types of intervention and its quick application and maintenance. A risk management tool specifically tailored to the sites’ trial-related activities is recognized as beneficial for sites to work on their inefficiencies and time constraints, decreasing the time spent on pointless tasks. In addition, by directing their efforts and time towards activities that will effectively bring the most valuable results, sites and their staff can strengthen their communication and understanding of projects, contributing to a collaborative mindset within the research team.²⁷ Based on the data and information collected, a risk management tool was developed to fit this purpose (Additional File 4). The development and validation of this tool will be carried out in the future work.

Currently, most sites do not focus on the harmonization of their clinical research activities. The identification of trials’ risks is not systematically documented preventing reproducibility. Furthermore, the knowledge is limited to the PI or CRU staff involved in each trial initial assessment, and then knowledge is not strictly translating into better implementation procedures to manage transversal risks. It is evident that optimization is needed to avoid fully restart of the process for each trial and wasting of the staff’s time and efforts.

Through the critical analysis of these findings, it is perceived that we must work on a change in the institutional mindset towards clinical research. The sites reported that that the most relevant risks and the actions to prevent them are already identified and implemented by the time the trial is received at the site. This thought could lay on the argument that sponsors, ethics committees and regulatory bodies have already assessed and addressed the trials’ risks before the clinical trial site is activated. Actually, although these parties indeed assess and manage the trial risks, they have no accountability for the sites’ processes. Therefore, sites need to ensure they control risks at the site level, by taking ownership of their activities. This is a dangerous assumption as it can prevent sites from taking actions in advance. On the other hand, this dependence on the sponsors’ recommendations to handle risks also leads sites to have completely different approaches to the same risk, what in the end creates inefficiencies within the site’s teams.

As the survey was answered by one single person at each site, we should consider the views of the respondent when reading the results. On the other hand, the high response rate could help to minimize this effect. It is also important to refer that the survey was available for answering during the beginning of the Covid-19 pandemic in Portugal, which could have impacted the results. During this time, the healthcare institutions’ constraints with staff and time increased and clinical research was not a priority.²⁸

In the future, it would be interesting to conduct similar studies in different countries to understand the sites concerns about their own performance and the implementation of risk management methodologies that differ across countries or regions. A comparative analysis could be performed to explore differentiating factors in the sites’ mindset and risk culture. As the sites’ experience in risk management methodologies grows, an integration of the procedures to manage the risks of clinical trial activities into a broader quality management system within the healthcare institution can foresee. The measurement of key performance indicators would also facilitate the evaluation of the trial plans implemented and potentialize improvements. CRU can play an important role in fostering the development and implementation of risk management processes and sensitizing clinical research teams to the relevance of their contribution to the improvement of the research’s overall quality.¹²

This work highlights the need for changing the mindset of clinical investigation teams to be aligned with a risk management culture. This change will enhance the overall performance and improve the institutions’ reputation among clinical research stakeholders. In the end, these achievements may attract more investment for healthcare institutions, resulting in a higher number of trials and subjects recruited. This success will lastly be translated into a benefit for the patients and society.

Conclusion

This work provides an overview of the current risk management practices in Portuguese clinical trial sites. The major conclusion is that most of the sites do not have a standardized tool or process to systematically analyze and document the risks related to their clinical trials’ operations. However, sites have generally recognized the importance of risk management methodologies as an opportunity to anticipate difficulties in the trial conduct and optimize the use of sites’ resources. Hopefully, these findings will raise attention to the importance of empowering sites to proactively contributing to the successful conduct of clinical trials and encourage enhanced collaboration to support this involvement. The development of a risk management tool specifically tailored to the sites’ needs could be a starting point for improving the current situation.