Recruitment and inclusion procedures as “pain killers” in clinical trials?

Introduction

Recruitment describes the selection process of participants/patients into a clinical trial, from the first communication to their enrolment, and is one of the most essential components in clinical research. A recruitment process can be complicated and time consuming, depending on the nature of the disease, the character of the inclusion/exclusion criteria, the type of intervention (single subject or group therapy), and the necessary procedures to check for those.

Complicated and time-consuming recruitment procedures are possibly acceptable for patients suffering from chronic diseases with a relatively stable time course of symptoms, but challenging for a clinical study recruiting patients suffering from pain syndromes with fluctuating symptom patterns such as musculoskeletal pain disorders. In diseases and syndromes with fluctuating pain, one might observe changes in pain ratings from inclusion to baseline which may obscure a potential treatment effect and contribute to high or early drop-out rates.

Generally, patients with fluctuating symptom patterns are usually contacted for participating in a potential study at times when the symptoms are particularly severe. If inclusion requires a time-consuming recruitment process, patients may experience reduced symptom severity at the time of enrolment, which will obscure the potential effect of the intervention investigated (“little or no pain – little or no gain”). In worst case, they may not fit the inclusion criteria anymore by the time the intervention begins.

Thus, the natural course of a disorder or a disease may play a significant role in clinical trials, even though it may be difficult to estimate its impact.^1–3 Only a few clinical trials provide a consistent report of the time period from the first communication with the study participant, baseline assessment, and details of the time course of the symptoms.

Performing a feasibility pilot study as a first step can help to identify and avoid threats to the implementation of a larger trial.^4,5 In a feasibility study on neck pain, our study group encountered a substantial decrease of pain intensity, between the telephone interview, as a first contact and the enrolment into the study. This change in pain ratings due to study routines and time schedule is relevant and can impose a threat to the main study itself.^6–12

Therefore, the aim of this paper is to analyze the phenomenon of a clinically significant pain reduction after inclusion but before the beginning of the intervention and to investigate the contributing factors in order to avoid this phenomenon in larger trials on chronic non-specific neck pain.

Methods

General remarks

The data presented here are a part of a feasibility study, “Chronic, non-specific neck pain. Quantitative Sensory Testing (QST) as a tool for the investigation of massage and relaxation as interventions - a feasibility study” which aimed to explore study routines to prepare a clinical trial comparing clinical effects of massage and meditation as treatment interventions for chronic, non-specific neck pain including quantitative sensory testing as a biomarker. Although some details of the feasibility study are presented, our analyses will focus on the change in pain ratings during recruitment or the time period from the first contact to baseline assessment, just prior to the intervention.

Study participants

Study participants were recruited through local newspaper advertisements. During a telephone interview performed by one single person, they were screened for inclusion and exclusion criteria before referral to physical examination by a physician (A.J.N.).

Inclusion criteria were age between 21 and 75 years and non-specific neck pain persisting at least 3 months with a mean pain intensity of greater than or equal to 40 on a 100-point numerical rating scale (NRS, 0= “no pain at all”, 100= “worst pain imaginable”). The patients were excluded if they suffered from a congenital deformation of the spine or had neurological symptoms, neuropathic pain, spine surgery less than 12 months prior to screening, or received transcutaneous electrical nerve stimulation (TENS), acupuncture, osteopathy, chiropractic maneuver, or infiltration in the area within 4 weeks prior to inclusion. With regard to medications, patients were excluded if they needed to take blood-thinning medicine, steroids, or strong pain medications, such as opioids. Patients were allowed to continue to use non-steroidal anti-inflammatory drugs (NSAID), if that was part of their current pain management. The use of medications was documented in a pain diary throughout the study.

The study was approved by the regional ethics committee (Regional committees for medical and health research ethics, REC North 2014/1105). Participants were informed about the purpose of the study, risk/benefit profile of the interventions and the study itself. All participants were free to withdraw from participation at any point if they wished to. All participants provided written informed consent prior to participation. The study was performed in accordance with the declaration of Helsinki.

Design

Figure 1 shows the study design of the feasibility study. After verification of eligibility of the study participants in telephone interview (T₀) and screening by study doctor, participants were included in the study. The study had a parallel design with two intervention arms; massage and relaxation. After baseline assessment (T₁), patients received five treatment sessions of either massage or relaxation, followed by post-intervention assessment (T₂), and follow-up assessment (T₃) 4 weeks later.

Figure 1 Study design of feasibility study across all assessments (T0, T1, T2, and T3). The analysis in the current study focuses on the time interval between the telephone interview (T0) and the baseline assessment (T1) presenting the gray structured square. The randomization into the groups (massage and relaxation) was performed after the baseline assessment (T1). Abbreviations: T0, telephone interview; T1, baseline assessment; T2, post-intervention assessment; T3, follow-up assessment.

The analysis presented here focuses on the change of pain intensity indicated by study participants from the telephone interview (T₀) to the baseline assessment (T₁), before randomization into two study groups.

Primary outcome was the change of pain intensity. Pain intensity was recorded at all 4 assessments (T₀, T₁, T₂, and T₃) using the numerical rating scale (NRS, 0–100) at T₀ and the visual analog scale (VAS, 0–100) at T₁, T₂, and T₃. As secondary outcome, we performed a Quantitative Sensory Testing (QST) according to the standardized protocol establishing by the German Research Network on Neuropathic Pain (DFNS) at T₁, T₂, and T₃.¹³

Outcome measures

Pain

The analyses of the data presented in this paper focus on the differences of pain ratings given by the study participants between the telephone interview (T₀) and the baseline assessment (T₁). Pain intensity assessed during the telephone interview (T₀) was recorded on a numerical rating scale (NRS) graded from 0 to 100 (0= “no pain at all”, 100= “worst pain imaginable”). Pain intensity assessed at baseline assessment prior to randomization (T₁) was recorded on a visual analog scale (VAS) graded from 0 mm to 100 mm (0 mm = “no pain at all”, 100 mm = “worst pain imaginable”). Both, NRS and VAS scores were transferred in a common 0–100 pain rating scale to compare the pain ratings at both time points.

Expectation

Within the feasibility study, expectations regarding the anticipated treatment effects were measured at baseline assessment (T₁). The patients self-rated their expectations about the intervention on a VAS ranging from 0 mm to 100 mm (0 mm = “not effective at all”, 100 mm = “highly effective”) at the baseline assessment (T₁). Since expectations are seen as a part of a potential placebo effect within an interventional study, they are likewise relevant for the analyses presented in this paper.^14–16

Statistical methods

Data for age, weight, height, BMI, pain intensity, and number of days between telephone interview (T₀) and baseline assessment (T₁) are presented as means with standard deviation (SD) and 95% confidence interval (95% CI of mean). We used Student T-test for paired samples to calculate the difference in pain intensity at T₀ and T₁. Normal distribution was tested using the Kolmogorov–Smirnov test.

For the statistical analyses of the change in pain intensity over time, the whole sample was further divided into two subgroups with regard to waiting time, ie, days between T₀ and T₁; group A: 1–14 days and group B: >14 days, based on a common waiting time after inclusion of about two weeks in clinical trials. The differences of pain ratings were analyzed with analysis of variance (ANOVA) using the differences of pain ratings between T₀ and T₁ as the within-subject factor and the grouping in two groups according to the days between the T₀ and T₁ as the between-subject factors. The pain rating (NRS 0–100) at T₀ was set as covariate in this model. No post hoc test was performed, since only two groups were compared.

Finally, three correlation analyses were performed to investigate a) the relationship between number of days between T₀ and T₁ and change in pain ratings, with the aim to identify a possible dependence on waiting time potentially related to the natural course of the disease, and b) the relationship between change in pain ratings between T₀ and T₁ and pain rating level at T₀, to identify regression to the mean, and c) the relationship between the difference of pain ratings between T₀ and T₁ and the expectation (0–100) about treatment effects at T₁, in order to investigate the impact of patient’s expectations which may play a role as part of a placebo effect. Correlation analyses were conducted using Pearson’s product–moment correlation coefficient for normally distributed parameters, and Kendall’s Tau correlation coefficient for non-normally distributed parameters. Correlation coefficients were interpreted as follows: |r| ≤0.29=negligible, 0.30–0.49=low, 0.50–0.69=moderate, 0.70–0.89 high, and >0.90=very high.

A p-value of <0.05 was considered statistically significant. Statistical analyses were performed using SPSS Statistics 22 (IBM Germany GmbH, Ehningen, Germany).

Results

Flow chart

A total of 85 potential patients were screened for eligibility by telephone interview (T₀), of these 51 were invited to the screening by the study physician (A.J.N.). A total of 44 patients fulfilled the inclusion criteria and agreed to participate in the study. The reasons for exclusion were pain intensity below 40 on a 0–100 pain rating scale at the time point of the clinical investigation (N=11), the localization of pain apart from the neck (N=6), and other ongoing, non-pharmacological treatments (N=2).

All patients underwent baseline assessment at (T₁). Two patients were considered as outlier according to waiting time between T₀ and T₁ and excluded from the statistical analyses. The outliers were defined as the values that were more than 1.5 x interquartile range beyond the 25th and 75th percentiles.¹⁷

Figure 2 shows a flow chart of patient flow throughout the feasibility study. The time points analyzed to determine possible changes in pain ratings during recruitment and inclusion are marked in black.

Figure 2 Flow chart of recruitment and progress of patients through study (the whole flow chart, gray). The analysis of the pain ratings during recruitment is based on pain intensity from the screening interview by telephone (T0) and baseline assessment (T1) (marked black in the flow chart). Abbreviations: T0, telephone interview; T1, baseline assessment.

Sample characteristics

A total of 42 neck pain patients (45.3±13.2 years, mean±SD), 6 males and 36 females, were included in the statistical analyses. Average pain intensity and SD at T₀ was 60.3±13.3 and at T₁ 38.1±21.7. The average time between T₀ and T₁ was 18.7±11.1 days. The expectation of the patients regarding the anticipated effectiveness of the treatment was 69.2±19.5 on a 0–100 VAS. Table 1 shows the demographic and clinical characteristics for the total sample.

Table 1 Demographic and clinical characteristics of the total sample being analyzed

Analyses of the pain ratings during recruitment and inclusion

Time course

Of the whole sample (N=42), 34 participants showed a decrease in pain rating from T₀ to T_1. This was the case for all participants (N=20) who waited 1–14 days and for 14 out of 22 with a waiting time >14 days (N=22). Only 6 patients reported an increased pain intensity, while 2 patients reported similar values at T₀ and T₁ (see Figure 3). Figure 3 reveals that 24 participants exhibited a pain score of <40 at T₁ and were thus not fulfilling the main inclusion criterion to the feasibility study.

Figure 3 Course of pain ratings (0–100) between the telephone interview (T0) and baseline assessment (T1). For all patients (N=42), both pain ratings were plotted before randomization. For the statistical analyses, the patients were classified into two groups depending on the waiting time between T0 and T1: group A patients with 1–14 days between T0 and T1 (N=20), group B patients with more than 14 days between T0 and T1 (N=22). The horizontal dashed line indicates the pain intensity criterion for inclusion into the feasibility study (greater than or equal to 40 on a 0–100 rating scale). Abbreviations: T0, telephone interview; T1, baseline assessment.

The average pain intensity at T₀ was 60.3±13.3 (0–100, mean±SD) and at T₁ 38.1±21.7 (0–100, mean±SD). The difference in pain intensity between T₀ and T₁ corresponded 36.8% of the value at T₀.

The analysis of variance (ANOVA) for the pain ratings on T₀ and T₁ showed a significant main effect of the factor; days between T₀ and T₁ (F(1,39)=6.17; p=0.017; ε=0.14). Furthermore, we found a significant effect for the covariate pain rating (NRS, 0–100) at T₀ (F(1,39)=14.14; p=0.001; ε=0.27), indicating that the decrease in pain ratings over time was partly dependent on the initial pain rating at T₀.

Correlation of the change in pain scores and the initial pain scores at T₀

In order to further investigate the covariate pain rating at T₀, we conducted an additional correlation analysis including the change in pain ratings from T₀ to T₁ and the pain level at T₀. The analysis revealed a significant, although moderate positive correlation (r=0.52; p<0.001) supporting the findings from the regression analysis above that the pain reduction from T₀ to T₁ was in part dependent on the initial pain rating at T₀ (see Figure 4).

Figure 4 The scatter plot shows the relationship between pain changes from telephone interview (T0) to baseline assessment (T1) and the pain rating at T0. For each patient (N=42), the change in pain rating (x-axis) is plotted against the pain level at T0 (y-axis). The vertical dashed line represents a difference in pain rating of 0. Data points on the right illustrate patients with a decreasing pain rating to T1, while data points on the left illustrate patients with an increasing pain rating to T1.Abbreviations: T0, telephone interview; T1, baseline assessment; NRS, numerical rating scale.

Correlation of the change in pain scores with absolute waiting time

If the decrease in pain ratings is dependent on waiting time, then the difference in pain ratings between T₀ and T₁ should correlate with the absolute waiting time. The correlation analysis revealed a statistically significant negative but negligible correlation (r = −0.25; p=0.020) between number of days from T₀ to T₁ and the corresponding difference in pain ratings. This indicates that the decrease in pain ratings was not strongly dependent on waiting time (see Figure 5).

Figure 5 The scatter plot shows the relationship between waiting time until baseline assessment and differences in the pain intensity between the two measurement points. For each patient (N=42), the number of days between the telephone interview (T0) and baseline assessment (T1) (x-axis) are plotted against the differences in pain ratings from the telephone interview (T0) and baseline assessment (T1) (y-axis). The horizontal dashed line represents a change in pain rating (T0 - T1) of 0. Data points above the dashed line illustrate patients with a decreasing pain rating to T1, while data points below the dashed line illustrate patients with an increasing pain rating to T1. Abbreviations: T0, telephone interview; T1, baseline assessment.

Correlation of the change in pain scores with expected treatment effects (expectation) at T₁

There was no significant correlation (r=−0.01; p=0.948) between the difference of pain ratings between T₀ and T₁, and the participant’ expectation (0–100) about anticipated treatment effect (see Figure 6). Therefore, the decrease in pain ratings was not related to expectations about effectiveness of the interventions.

Figure 6 The scatter plot depicts the correlation between the differences in pain from the telephone interview (T0) to baseline assessment (T1), and the expectations of the study participants about the study outcome. For each patient (N=42), the differences (T0-T1) for the pain ratings (x-axis) are plotted against and the expectations at T1 (y-axis). The vertical dashed line represents the differences of the pain ratings (T0-T1). The data points on the right illustrate patients with a decreasing pain rating up to T1, data points on the left illustrate patients with an increasing pain rating up to T1. Abbreviations: T0, telephone interview; T1, baseline assessment; VAS, visual analog scale.

Discussion

The mean decrease in pain intensity from the first contact (T₀) until randomization (T₁) in our study was 36.8%. This is by definition above the cut off of 30% the minimal clinically important difference (MCID) and equivalent to a moderate pain reduction.^6,10,12, Moreover, the effect was rather consistent, as 34 of the total sample (N=42) revealed a pain reduction during the waiting period.

Natural course of the disease

Neck pain fluctuates over time, and patients are probably most willing to participate in a study when their pain is temporarily more intense. Later, when the patient is to be randomized the pain levels may be lower due its natural course.^2,3

The natural history of a disease describes its time course during the absence of an intervention.¹ This effect should have a similar influence on all study groups, if they are equally handled with regard to study procedures, and blinding and randomization are successful. However, the natural course of the disease may affect study outcomes substantially across groups, depending on the time course of the symptoms and the time interval between inclusion and intervention, and thus imposes a threat to the internal validity of a study.¹

The significant correlation between waiting time and change in pain rating supports at first glance the assumption that a natural course of neck pain contributed to the effect. Nonetheless, the effect is not entirely consistent over time, as we also see less pain reduction with increased waiting time. A closer look at Figure 4 reveals negative values for six patients (left of the dotted vertical line). These were the patients who waited exceptionally long for T₁ (ranged from 15 to 43 days from T₀ to T₁). Thus, it is likely that these patients waited so long that they experienced an increase in pain ratings over time. Thus, the curve may in fact be more u-shaped as it appears in the correlational analysis, with an initial decrease in pain, followed by an increase over time.

Nonetheless, symptom fluctuation did not seem to be the only reason for the moderate pain reduction during the enrolment phase. The correlation between the change in pain rating and the initial pain rating at T₀ suggests that the change in pain rating over time is also dependent on initial pain intensity at T₀. A fact that could support the hypothesis that participants are more likely to enter a study when their pain is particularly intense.

Regression to the mean

Another highly relevant phenomenon with regard to fluctuating symptom patterns is the so-called “regression to the mean”.^18–20 Regression to the mean is a purely statistical phenomenon, describing the general tendency for extreme values to converge towards a middle level.²¹ The more extreme the initial value is, the higher the potential for such regression will be.²²

For fluctuating pain syndromes, this implies that a high pain intensity prior to examination will to some extent, in itself, predict the following decrease. Taken together, the fact that it is necessary to define a reasonably high initial value as inclusion criterion in pain trials implies that regression to the mean is likely to occur. Regression to the mean can affect any investigation where the response to treatment is classified relative to initial values.²³

In conclusion, both the natural course of the pain syndrome and regression to the mean may have influenced the decrease in pain intensity in our study.^1,24 A predefined inclusion criterion with high pain intensity will lead to a data structure, where regression to the mean is likely to occur.

Possible placebo effects and expectation

Expectation-dependent placebo effects are an inherent part of all clinical interventions and must be taken into account. It is therefore a standard baseline procedure in clinical trials to include questions about expectations about the treatment effects. This was therefore done at T₁ in the planned feasibility study. Although we did not assess expectation at T₀, we assumed that by the fact that the study participants showed up at T₁ to enter the interventional part of the study that their expectations about the effectiveness of the treatment were at least as high at T₁. Therefore, even though the correlation between the participant’s expectation at T₁ and the difference in pain ratings between T₀ and T₁ represents a “backward analysis”, it still provides information on how expectation influenced the difference in pain ratings.^14–16

The correlation analysis revealed that there was no influence of treatment effect related expectations on the difference in pain ratings. Figure 6 shows the reason for that: treatment effect expectations were uniformly high (69.2±19.5, Table 1) at T₁ so that there was not enough variation in expectations to correlate with the change in pain ratings. In conclusion, even though we should consider expectation-dependent placebo effects as part of the treatment effect in the feasibility study (not presented here), we find it unlikely, that the expectation-dependent placebo effects played a role for the reduction in pain ratings within the waiting period.

The Hawthorne effect

Another, critical issue with regard to fluctuations in outcome measures may be the so-called Hawthorne effect. This effect describes the phenomenon that a person may change her or his behavior, experiences, emotions, etc., when becoming a study participant.^25–28 The phenomenon is interpreted as a type of reactivity to the situation, where a person is being systematically investigated and “observed”. A possible reason for this effect may be increased attention to factors that are related to the study outcomes.

Thus, independent of the natural course of the disease, the inclusion procedure and the enrolment into a study may, in itself, have a major impact on the main study outcomes. Even though this influence can be expected to be equally distributed among the study groups,^29–32 the Hawthorne effect is difficult to control in experimental as well as, placebo and non-treatment control groups.

Limitations

The data presented are part of the feasibility study on how to integrate the standardized protocol of DFNS QST protocol into a clinical trial on the effects of massage and meditation for chronic, non-specific neck pain. However, the study was not planned to directly assess recruitment and enrolment-related problems. Thus, a more differentiated design, directly aiming at possible threats to the internal validity of outcomes in pain trials during the recruitment and enrolment phase, would have been more appropriate. Nonetheless, we assume that the challenges to a clinical study on pain presented here are of relevance to other pain researchers and have to be regarded in further clinical trials.

The use of two different pain rating scales in this study is another limitation. During the screening (telephone interview, T₀), patients assessed pain intensity with a numeric rating scale (NRS, 0–100) while a visual analog scale (VAS, 0–100) was used at baseline assessment (T₁). The reason for this incongruence is that the NRS is easier to handle in a telephone interview situation.

Although it has been shown that NRS and VAS ratings correlate significantly, VAS scores have a tendency to reveal lower ratings compared to NRS scores. Thus, it is not unproblematic to use these two scales interchangeably when assessing self-reported pain.^33,34 In our study, the fact that NRS scores seem to reveal higher pain ratings than VAS scores means that the pain scores measured at T₀ possibly overestimate pain compared to the VAS pain measures at T₁. Ultimately, this overestimation would contribute to higher pain values at the first measurement point (T₀) and could thus lead to an overestimation of the decrease in pain ratings from T₀ to T₁. In conclusion, it would be important to estimate whether the decrease in pain ratings from T₀ to T₁ is within or outside the range of the difference in pain ratings between the NRS and VAS ratings.

According to the literature, the differences between NRS and VAS means were |0.52| cm for the initial assessment and |0.86| cm across all assessments in a study conducted among teenagers (mean age 14.7±3.1 years). Moreover, in this study, the differences between NRS and VAS ratings decreased with increasing pain severity across all assessments.³⁴ In another study conducted with adults, the differences between NRS and VAS ranged between |2.30| cm and |1.30| cm.³⁴

In our study, the difference between the telephone interview (T₀) and the baseline assessment (T₁) was 60.3±13.3 and 38.1±21.7 representing a change of 36.8% and a distance of |22.2| mm or |2.22| cm between the NRS at T₀ and the VAS at T₁. According to the data on children provided by Myrvik (2015) this difference of 2.22 cm would be outside the range of differences between the two scales, but within the range according to the data provided Holdgate (2003), even though at the outer limit of the interval.

In conclusion, the fact that the pain ratings measured with the NRS tend to be higher than those measured with the VAS and the fact that the NRS was used at T₀ might have contributed to systematically higher pain ratings at T₀ in this study. It may have thus contributed to larger decreases in pain ratings from T₀ to T₁.³⁴ However, since we expected regression to the mean to play a role in this analysis, our statistical approach already controls for the dependency of the differences between T₀ and T₁ on the initial values at T₀. In our ANOVA model, the initial values were used as covariate, and thus, the influence of the initial values on the total effect is extracted. Indeed, there is a significant effect of the covariate (pain rating NRS at T₀), nonetheless, the factor days between T₀ and T₁ remained significant and thus non-neglectible. It is impossible to estimate how much of the difference between NRS and VAS contributed to the effect of the increased initial values (pain ratings at T₀); however, the ANCOVA reveals a significant effect of waiting time beyond a significant effect of the covariate. Thus, even though the initial values at T₀ played an important role for the total effect, there still remains substantial support for a decrease in pain ratings during the waiting time between recruitment and inclusion.

Further directions and recommendations

The Hawthorne effect predicts that a certain reduction in pain has to be expected simply due to the fact that patients participate in a study. Consequently, inclusion criteria should possibly be strict, eg, pain intensity >60. There are several challenges related to this: i) The total available study population will be significantly smaller and consequently, recruitment may be more difficult; ii) Such a high pain intensity may not reflect the majority of the patients and will thus restrict the generalization of the study results (external validity); iii) A high pain rating as inclusion criterion will also make the main outcome more vulnerable to distribution-related phenomena, such as regression to the mean.

While the Hawthorne effect is uncontrollable, the impact of the natural course of the disease could be controlled by keeping the time frame as tight as possible. In our study on neck pain, the main reduction in pain intensity occurred within 2 weeks and thus a time frame of a few days would be more advisable.

However, another methodological approach to the natural course of the disease and regression to the mean could be to wait more than 14 days in order to avoid the initial fluctuations in pain ratings (see Figures 3 and 4 where six study participants already show an increase in pain ratings over time). The benefits of such a design would be that it is easier to pick up a defined treatment effect while the disadvantages are that the study population may not represent typical pain patients.

Moreover, the main outcome measure of a study, in this case the pain intensity, should be recorded systematically after the first contact (eg, telephone interview) throughout the recruitment and enrolment. This allows for better documentation of symptom changes over time. Another possibility would be to conduct an observational pilot study before an interventional trial to observe the natural course of the disease in the target group. Such data would allow for a more systematic recruitment approach since a population-derived inclusion criterion, information about the time course of the symptom, and the recruitable patient population would be available.

Conclusion

Three main potential influences on a clinically relevant decrease in pain scores during the enrolment phase of a study were identified; the natural course of the disease, regression to the mean, and the Hawthorne effect. They can all impose a threat to a clinical trial of diseases with a fluctuating symptom pattern because they are difficult to control. Clinical trial methodology, by defining a primary inclusion criterion related to the primary outcome measure, comprises a risk of a regression to the mean in spontaneously fluctuating diseases. Awareness of this phenomenon can contribute to better routines for clinical studies accounting for a change in the main outcome measure during enrolment and recruitment.