Not Two Sides of the Same Coin: A Qualitative Comparative Analysis of Post-Treatment Abstinence and Relapse

Introduction

Substance use disorder (SUD) is a multifaceted public health problem that impacts individuals and communities worldwide.¹ High relapse rates pose a significant challenge in the recovery process from SUD, with one-year rates ranging between 40 and 60%.^2,3 Relapse can have severe negative health, psychosocial, legal, and economic consequences,^4,5 whereas abstinence can facilitate recovery.^6,7

Numerous studies have explored biological, genetic, psychological, and environmental factors that may predict post-treatment abstinence or relapse,^5,8–16 showing inconclusive results.¹⁰ For instance, while most studies link psychiatric co-morbidity to an increased relapse risk,^17,18 some studies suggest a reduced risk.^10,19 This highlights the inherent complexity of post-treatment substance use outcomes that arise from a dynamic interplay of multiple factors over time.^9,20,21

While conventional statistical methods are valuable for assessing the impact of interaction effects on abstinence and relapse, their primary emphasis on average effects presents challenges in comprehending the intricate and synergistic nature of the factors that lead to these outcomes on an individual level.^22–24 Qualitative comparative analysis (QCA) is a multi-method set-theoretic approach that differs from conventional statistical methods, as it aims to identify and explain the specific combinations of factors that contribute to particular outcomes.²⁵ QCA examines how multiple conditions interact to produce an outcome, similar to a “causal recipe” (conjunctural causation), and whether different conditions or combinations of conditions can explain the same outcome (equifinality). Further, with QCA, it is possible to determine whether and how conditions operate differently in causing an outcome across different cases. Lastly, QCA enables the identification of causal asymmetry, meaning that the conditions leading to an outcome, like abstinence, are not merely the opposite of the conditions leading to the non-occurrence of an outcome, like relapse. In a service innovation study by Ordanini et al,²⁶ QCA has already demonstrated its ability to identify intricate configuration effects that could not be captured fully by using conventional interaction effects. Through the application of QCA, we aim to explore the (combinations of) conditions that result in abstinence or relapse. By adopting this comprehensive approach, we aim to contribute to a deeper understanding of the complex and interrelated factors influencing post-treatment substance use outcomes.

Methods

Setting and Sample

From September 2020 to October 2022, we collected qualitative and quantitative data at a Salvation Army SUD treatment facility in The Netherlands that tailors to individuals with SUD and comorbidities. Participants were enrolled in an inpatient treatment program based on the Community Reinforcement Approach (CRA), of either 12 or 28 weeks. Intended length of treatment was determined by the availability of financing from the Social Support Act (WMO), not on problem severity.

All patients entering treatment were considered eligible to participate in this study. However, for the purpose of conducting the QCA analysis, only patients with a complete dataset, comprising both baseline and follow-up measurements, were included (see Figure 1). QCA was designed for medium-sized samples,²⁷ and our inclusion process continued until we reached a sufficiently large sample (n = 58) that exhibited variability in both the conditions and outcome measures. This study received approval from the regional ethical board under the reference number 2020–6837.

Figure 1 Inclusion process.

Outcomes and Conditions

Our outcomes of interest were abstinence and relapse. We defined abstinence by the number of days without (self-reported) substance use during the 30-day period leading up to the three-month follow-up evaluation. A value of 3 out of 30 days with substance use denoted the boundary between a slip (ie, a temporary setback after which abstinence is resumed) and full relapse.

To identify theoretically relevant conditions for our study, we followed the methodology of Goicolea et al²⁸ by conducting a literature review. This resulted in a list of 48 potential conditions, out of which 40 were considered feasible and relevant for our study. Within QCA, it is advised to avoid models with more than seven causal conditions, as these become highly challenging to interpret given the large number of parameters involved.²⁹ Therefore, we aimed to limit our conditions to a maximum of seven, which would also comfortably meet the criteria for sample size. This criterion is linked to the number of conditions, and seven conditions require at least 30 cases to establish a reliable model.³⁰

To refine our selection, we analyzed the data collected from the first ten cases in our study. This allowed us to assess the relevance of each condition in this population. Conditions that were not deemed relevant to our cases were omitted. Through this iterative process, we reduced the number of conditions to seven, balancing individual and contextual factors: psychiatric co-morbidity, mild to borderline intellectual disability (total IQ 50–85, MBID), treatment duration, post-treatment formal (eg, treatment, outpatient support) and informal (social network) support, and engagement in daily activities. We refer to Supplementary Tables 1 and 2 for a more detailed description of the selection process and to Supplementary Table 3 for definitions and measurements of these conditions.

Procedure

All patients entering treatment were approached by the researcher and fully informed. Those willing to participate signed informed consent to (1) complete questionnaires, (2) access their electronic patient files, and (3) participate in a semi-structured interview by phone 3 months post-treatment. Baseline questionnaires were administered to measure psychiatric comorbidity, MBID, and social disintegration (see Supplementary Table 3 for questionnaires). Treatment length was derived from the electronic patient files. Three-month post-treatment, researchers assessed post-treatment informal support, formal support, and daily activities with a questionnaire (see Supplementary Table 3), complemented by a semi-structured interview covering these aspects and other relevant post-treatment experiences, along with an evaluation of substance use frequency over the past 30 days.

Analysis

Included and excluded individuals were compared on key variables (age, gender, MBID, psychiatric co-morbidity, time in treatment, treatment completion). Subsequently, QCA analyses were performed with the software program fsQCA 3.0.³¹

Calibration

All cases (ie, patients) were assigned scores ranging from 0 to 1, indicating the degree to which they were associated with each of the conditions and the outcome. This process, called calibration, resulted in a data matrix containing the calibrated scores of all cases on all conditions and the outcome. Supplementary Table 3 gives a detailed account how calibration scores were assigned, and Supplementary Table 4 displays the full calibrated data matrix.

Truth Table

In the calibrated data matrix, each row corresponds to an individual case. To shift our focus from diversity among cases to identifying commonalities, we constructed a truth table (see Supplementary Tables 5 and 6). This truth table lists all possible combinations of conditions (ie, configurations), which cases are represented by each configuration, and how consistently the configurations co-occur with the outcome. Unlike the data matrix, which contains fuzzy scores ranging from 0 to 1, the truth table simplifies each condition and the outcome as either present (coded as 1) or absent (coded as 0). This simplification streamlines the evaluation of a finite number of potential condition combinations. The more detailed fuzzy calibration scores are retained for the development and assessment of the minimal formula (ie, the resulting model). To construct the truth table, the software matched cases with rows that best represented their blend of present and absent conditions. It proceeded to calculate a consistency value for each row, indicating the portion of cases that exhibit both the configuration and the outcome.

Logical Minimization

The truth table was reviewed, and redundant conditions were removed through logical minimization (Quine-McCluskey algorithm). Redundancy occurs when a condition can be both present and absent with the outcome. For example, if informal support can be both present and absent in relation to the outcome abstinence, this condition may be considered redundant.

To make our results more concise, we incorporated theoretically informed directional expectations (similar to hypotheses) into the logical minimization process. This approach considers combinations of conditions that might not be observed in the dataset but are logically and theoretically plausible. Specifically, we expected that the absence of psychiatric co-morbidity, MBID, and/or social disintegration, along with the presence of formal and informal support and/or engagement in daily activities, would promote abstinence. In the case of predicting relapse, we reversed these expectations. We maintained a neutral stance on the condition “long treatment” because we hypothesized that the ideal duration of treatment depends on the severity or complexity of the problem.³² As such, it may contribute to abstinence or relapse in both present and absent form.

Assessing the Solutions

Before tending to the results of logical minimization, we first assessed whether conditions or combinations of conditions were necessary for the outcome (see Figure 2). If a case does not have a necessary condition, it cannot have the outcome. The lower threshold for necessity was set at 0.9 (a score of 1 would indicate presence of the condition at all times when the outcome is present).

Figure 2 Necessity (left) and sufficiency (right).

We then proceeded to the results from our logical minimization. Logical minimization produces three models – or in QCA terms, solutions: the complex, parsimonious and intermediate solution. In the current paper, we focused on the intermediate solution, as it balances complexity and conciseness.^33,34 The solutions resulting from the performed logical minimization indicate which conditions or combinations of conditions are sufficient for (ie, lead to) the outcome abstinence or, respectively, relapse (see Figure 2).

Our solution was further evaluated with two parameters of fit, solution consistency and solution coverage (see Figure 3). Consistency indicates the extent to which the solution or parts of the solution are a consistent subset of the outcome (ie, co-occur with the outcome). As such, it indicates explanatory power. The lower threshold for consistency was set at 0.8. Solution coverage indicates the extent to which the entire solution and its components explain the outcome of interest. As such, it indicates the level of empirical relevance. There is no established lower threshold for coverage, as solutions with low coverage can still be relevant.

Figure 3 Consistency and coverage. (A) Low consistency, low coverage. (B) High consistency, low coverage. (C). High consistency, high coverage.

The different parts of the solution (solution terms) were also evaluated on their level of consistency, and two types of coverage: raw and unique coverage. Raw coverage indicates the relative importance of a solution term (which proportion of the outcome is covered by the solution term?), and unique coverage controls for overlap (how much of the outcome is uniquely explained by the solution term?).

Results

Participants (n = 58) were on average 44 years old, with 91.4% being male, and the majority having psychiatric comorbidities, MBID and experiencing social disintegration, see Table 1. Included participants were slightly older than excluded participants (t(120) = −3.044, p < 0.003), see Supplementary Table 7.

Table 1 Baseline Characteristics

Necessity

For both abstinence and relapse, we could not identify single necessary conditions (see Table 2). Nevertheless, we observed near-necessary consistency scores of 0.89 for the presence of informal support and 0.87 for engagement in activities, indicating their importance in achieving abstinence.

Table 2 Results necessity analysis (Single Conditions)

Additionally, we evaluated the occurrence of necessary combinations of conditions, which resulted in several sets of necessary OR-combinations (see Supplementary Table 8 for an overview). In an OR-combination, the outcome depends on at least one condition; not all conditions need to occur simultaneously. For abstinence, we found necessary OR-combinations consisting of (post)treatment conditions, with the most consistent combination being activities+informal support (consistency 0.956, coverage 0.740). For relapse, we identified combinations involving at least 3 conditions, all consisting of at least two baseline conditions. The most consistent necessary combination included MBID+Psychiatric comorbidity+~Activities (consistency 0.951, coverage 0.429).

Sufficiency

Two combinations of conditions were sufficient for the outcome abstinence, and five for the outcome relapse (Table 3).

Table 3 Intermediate Solution for Abstinence and Relapse

Abstinence

The intermediate solution for abstinence was a highly consistent subset of the outcome (0.90), indicating that it was firmly supported by the empirical data. The solution accounted for 59% of this outcome (coverage 0.59). The first solution term (A) – presence of baseline social disintegration, post-treatment support (formal and informal), and daytime activities – was the most prevalent (raw coverage 0.51, n = 20), highly consistent (0.90), and showed a unique coverage of 0.30. The second solution (B) – the absence of psychiatric comorbidity and social disintegration at baseline, absence of long treatment, and informal support and activities post treatment – was less prevalent (raw coverage 0.28, n = 4), but still highly consistent (0.87). However, the unique coverage was low (0.08), suggesting it explains a smaller portion of the outcome independently.

The most prevalent solution term for abstinence (A) was best exemplified by case 17 and 28. Both males had SUD for over 30 years, experienced traumatic childhoods, social disintegration and periods of homelessness. Neither had MBID, and both underwent SUD treatment multiple times in the past. Only case 17 scored above the clinical threshold for psychiatric comorbidity. Case 17 spent 36.5 weeks in treatment, and case 28 only 8. After treatment, both men lived in sheltered housing, received intensive professional support, and participated in daily activities. Case 28 expressed a desire for more work opportunities, and case 17 a need for additional formal support from a psychologist. Following treatment, both men received ample informal support, mostly from their families. Case 17, who previously had a (self-proclaimed) “useless” social life, was able to reconnect with his family during treatment. Conversely, case 28 already had a social network and did not require further support in this regard. Both reported that contemplating the consequences of relapse for themselves and their loved ones was enough to avoid substance use.

Relapse

The intermediate solution for relapse was slightly less fitting, compared to the solution for abstinence. Consistency was high (0.87), and the solution explained 45% of the outcome relapse (coverage 0.45). The solution consisted of five solution terms (raw coverage 0.25–0.32, n = 2–5), which showed high levels of overlap, as evidenced by multiple cases being represented in more than one solution term and indicated by low levels of unique coverage (0.01–0.04).

The first two solution terms (A, B) identified that the presence of either MBID or psychiatric comorbidity, combined with the absence of long treatment, support (formal and informal), and activities, was sufficient for relapse. The third solution term (C), which was the most prevalent, indicated that a triple diagnosis (SUD, MBID, psychiatric comorbidity) in conjunction with the absence of long treatment, formal support, and activities, was sufficient for relapse. The fourth solution term (D) showed that the presence of social disintegration and psychiatric comorbidity, along with the absence of post-treatment support (formal and informal) and activities, was sufficient for relapse. Lastly, solution term (E) stated that the presence of social disintegration, MBID, and long treatment, coupled with the absence of post-treatment formal support and activities, was sufficient for relapse.

The solution for relapse is best illustrated by case 12, who is covered by all solution terms. Case 12 had a SUD, spanning over three decades, a history of homelessness, and he scored well above the threshold for psychiatric comorbidity and social disintegration. He began a 28-week treatment program, which was discontinued by the clinic after seven weeks. Relapse occurred two weeks after leaving treatment and soon escalated into daily use. After a few temporary living arrangements, he finally settled into a rented room.

Post-treatment, case 12 reported receiving no formal support, and he was on a waiting list for SUD treatment. He had no daytime activities, often felt bored and had a very small informal social network. Although his sister visited him once a week, he reported feeling lonely most of the time.

Discussion

This study used qualitative comparative analysis (QCA) to explore the baseline, treatment, and post-treatment conditions leading to post-treatment abstinence and relapse in patients with SUD. Necessity analysis revealed no single necessary condition for either abstinence or relapse. However, the presence of informal support and engagement in activities were identified as important (near-necessary) conditions for abstinence. Additional necessity analyses found several necessary OR-combinations. For abstinence, these primarily involved (post-)treatment conditions, with the most consistent combination being activities+informal support. Relapse required at least three conditions, including two or more baseline conditions, with the most consistent combination involving MBID+psychiatric comorbidity+~activities.

Sufficiency analyses revealed seven distinct causal pathways, providing insight into the “causal recipes” leading to either abstinence or relapse. These pathways revealed that socially disintegrated individuals relied on comprehensive post-treatment conditions (formal support, informal support, engagement in activities) to remain abstinent, while those without baseline social disintegration and psychiatric comorbidity (without differentiating between different disorders) benefited from short treatment combined with less comprehensive aftercare (informal support, engagement in activities). For relapse, the pathways consistently showed an absence of formal support and activities, frequently coupled with absent informal support. However, there were considerable variations in baseline and treatment conditions across the different solution terms for relapse.

While previous studies often linked abstinence to the absence of baseline conditions related to relapse,^36–40 we only observed this pattern in the least prevalent configuration for abstinence—specifically, among patients without social disintegration and psychiatric comorbidity. Surprisingly, our study found that the most prevalent configuration for achieving abstinence included the presence of social disintegration. This suggests that socially disintegrated individuals can achieve and maintain abstinence if post-treatment conditions are beneficial.

Treatment duration was counterintuitively related to abstinence and relapse. While short treatment duration featured in both the abstinence and relapse solution, long treatment was exclusively related to relapse. Although longer treatment duration predicted sustained abstinence in several other studies,^41–45 our study may reflect the interplay between baseline conditions and treatment duration: individuals with less severe baseline issues tended to benefit from shorter treatment durations, whereas those with pre-existing baseline conditions were more prone to relapse after short treatment. This finding aligns with scholars advocating “matched care”, ie, tailoring treatment intensity and duration to the individuals’ problem severity and specific needs.^32,46 Nonetheless, it remains interesting that the final solution term (E) for relapse demonstrated that patients with MBID and social disintegration experienced relapse after long treatment duration, particularly in the absence of formal support and engagement in activities. This may underscore the significance of effective aftercare for this particular group, as even an extended treatment duration cannot compensate for its absence. Further research is needed to clarify this.

When evaluating post-treatment conditions (formal support, informal support, and engagement in activities) individually, our findings are in line with previous studies, associating their presence with abstinence and their absence with relapse.^8,10,47–50 For individuals with baseline social disintegration, all post-treatment conditions appear to synergistically facilitate abstinence, suggesting that comprehensive aftercare, encompassing both formal and informal support, along with engagement in activities, holds promise as an effective intervention. In contrast, individuals with milder baseline issues no longer rely on formal post-treatment support as a critical factor for achieving abstinence. This implies that they may already derive sufficient benefit from informal support and engagement in activities. Conversely, relapse configurations typically involved one or more baseline conditions, in conjunction with the absence of post-treatment formal support and engagement in activities. This further highlights the significance of individualized and comprehensive care pathways that extend beyond inpatient treatment.

The different causal pathways for abstinence and relapse found in this study illustrate that these outcomes are not two sides of the same coin. This may provide an explanation for the inconclusive results observed in previous studies,¹⁰ as this “inconclusiveness’ may arise from causal asymmetry, where conditions cannot be simply reversed to reverse the outcome. Future studies should heed Glaesser”s⁵¹ advise and analyze abstinence and relapse outcomes separately. Furthermore, they should consider the intricate interplay of baseline, treatment, and post-treatment conditions, rather than exclusively concentrating on individual conditions. From a clinical perspective, it is crucial to recognize that fostering conditions for abstinence can diverge from preventing factors that trigger relapse. In light of this, McKay⁵² advises enhancing the appeal and effectiveness of the “recovery journey” by prioritizing abstinence-centered support and incentives. The Community Reinforcement Approach (CRA) is one example of a strategy that emphasizes post-treatment support and engagement in activities to promote sustained abstinence.⁵³

Strengths and Limitations

To our knowledge, this study is the first using QCA methodology to explore post-treatment abstinence and relapse dynamics. QCA, designed to unravel complex causality in medium-sized samples,³⁵ allowed us to shed light on these dynamics with a sample size smaller than what is typically needed for conventional statistical procedures.

It is important to note that data collection occurred amidst the Covid-19 pandemic, which may have impacted our research. Despite the fact that inpatient treatment at the Salvation Army clinic remained largely unaffected by the pandemic, the post-treatment phase for patients might have been more complicated due to restricted social interactions and interruptions in daily activities. Additional limitations include the sample size and the specific (predominantly male) population of the Salvation Army, which limits the generalizability of our findings.

Maintaining a follow-up period of only three months could also be considered a limitation, although the literature does indicate that most relapses occur within the first three months after treatment.⁵⁴ Moreover, akin to more conventional modes of analysis, incorporating all potentially relevant conditions or distinguishing within our conditions was unfeasible. However, the conditions not included, such as type of psychiatric comorbidity or substances used, could also impact the causal pathways to abstinence or relapse,^15,16,55 rendering any selection a partial representation of the myriad interactions among factors. Addressing this challenge may be a focus for future research. Finally, while only including participants who completed the follow-up assessment may have introduced selection bias, analysis of key variables between included and excluded participants did not reveal significant differences other than age, suggesting limited bias.

Conclusion

In our sample, post-treatment conditions were important for achieving abstinence, whereas individual baseline conditions combined with (post-)treatment conditions were important factors for relapse. As such, abstinence and relapse may not be two sides of the same coin. Each may have its own underlying unique dynamics, which can differ across contexts. To better understand these dynamics, abstinence and relapse should be studied further as separate outcomes. In clinical practice, it may be relevant to both promote abstinence alongside managing relapse.