A New Drug Safety Signal Detection and Triage System Integrating Sequence Symmetry Analysis and Tree-Based Scan Statistics with Longitudinal Data

Introduction

Drug safety signal detection is the very first step of drug safety signal management. In most countries, quantitative signal detection relies on data from spontaneous reporting systems (SRS).¹ However, SRS comes with some limitations.^2–4 Although individual case safety reports from SRS provide valuable information for clinical review, its inherent under-reporting, biased reporting and lack of drug-exposure data make it hard to estimate real-world incidence of potential ADRs. Furthermore, due to the limited number of cases documented in SRS, signal detection is prone to insufficient power to detect ADRs with low incidence, and is subject to high risk of false-positive findings.⁵

In the recent decade of advancements in pharmacovigilance, longitudinal health care data has been proposed as an important data source besides SRS to detect potential safety signals of medications in real world where patients’ characteristics were more complicated than in clinical trials.^6,7 Compared to SRS, long considered the cornerstone for drug safety signal detection, longitudinal datasets, especially those of population-sized scale, have the advantage of containing data on population-level exposure to drugs, ensuring greater accuracy in risk estimates of adverse drug reactions (ADRs). Furthermore, given that they allow patients to be followed over time, longitudinal health care data can provide more comprehensive information, thus making the exploration of associations between drug exposures and specific outcomes more convincing.⁶ To detect drug safety signals from longitudinal health care data, data-mining approaches have been utilized to perform proactive screening.⁸ In addition to conventional disproportionality analysis, novel tools have been under development. Among these, sequence symmetry analysis (SSA) and tree-based scan statistics (TreeScan) are two data-mining techniques⁸ that are relatively well-developed tools which have been used in multiple FDA Sentinel projects^9,10 and have been increasingly applied to drug safety signal detection, given their relative ease of comprehension and high efficiency when generating potential drug-event pairs.^11–17

Combining the advantages of both longitudinal data and novel data-mining approaches is likely to enhance the efficiency of drug safety surveillance. Although some studies^13,18 have proven the strengths of the aforementioned signal detection methods, only limited studies have explored how to integrate these tools into an existing signal triage algorithm. Once data-mining methods are applied, it is anticipated that a very large number of signals will be generated, leading to a challenge in prioritizing the signals. Therefore, it is important to evaluate the applicability of data-mining tools in signal identification and to establish a triage mechanism for signal prioritization. In this study, we aimed to delineate a new triage system integrating SSA and TreeScan for drug safety signal generation and prioritization. We also conducted an exploratory study in which we chose one class of drug used in chronic disease (ie, sodium-glucose co-transporter-2 inhibitors (SGLT2i)) and one used as episodic treatment (ie, non-fluorinated quinolones (NFQ)) in order to evaluate the adaptability of incorporating these methods into the new system.

Methods

Data Source

Taiwan’s National Health Insurance Database (NHID), under the custody of the Health and Welfare Data Science Centre was used as the major data source to conduct the signal detection analyses of our proposed triage system. The NHID is a longitudinal nationwide database containing over 99% of all citizens’ health insurance claims.¹⁹ Medical diagnoses and prescriptions from outpatient visits and inpatient hospitalizations between 2010 and 2018 were retrieved from the NHID. Patient identification numbers, which were encrypted to protect individual data privacy, were used to link individuals between data subsets. This study was approved by the Research Ethics Committee Board of National Taiwan University Hospital (201803030RINC).

Construction of the Triage System

This study covered the aspect of signal detection and refinement under the broader scope of signal management. Signal detection concerned the application of data-mining tools to identify potential safety signals of the drug of interest, while signal refinement concerned an algorithm to classify and prioritize the detected signals. The goal of constructing the triage system was to improve the proactiveness of the current drug safety surveillance system in Taiwan. Post-marketing drug safety should ideally be monitored by both passive and active surveillance methods. In addition to the existing SRS which has been seen as the cornerstone of drug safety surveillance, we proposed a triage algorithm utilizing longitudinal data for active surveillance. We incorporated guidance from the Council for International Organizations of Medical Sciences (CIOMS) Working Group VIII²⁰ and the current structural framework used in Taiwan’s national pharmacovigilance system²¹ to construct the new system, with integrated data-mining approaches to aid in signal generation and with a classification algorithm to categorize and prioritize safety signals.

Data-Mining Methods

We applied data-mining approaches in the step of signal identification, using SSA and TreeScan as the major tools in the NHID. Both tools are well-established methods that have been applied in various studies for generating potential safety signals.^{11–13,15,16,22} Validation studies have shown that both methods have moderate-to-high sensitivity and specificity.^23–25 SSA examines the relationship between two drugs by comparing the sequence of introduction of the drug of interest (ie, the index drug) and other drugs during a pre-specified period for all subjects. If the frequency of the prescription sequence of index drug followed by marker drug is greater than the frequency of the inverse sequence, this may imply that it is the use of the index drug that increases the use of the marker drug, the latter possibly being used to treat an adverse event (AE) induced by the index drug. By contrast, TreeScan scans for excess risk of potential ADRs in a hierarchical level and simultaneously adjusts for the multiple testing inherent in the large number of overlapping groups of data evaluated. In pharmacovigilance, it could be used to evaluate hundreds or thousands of potential AEs at the same time to see if any event or group of events occur with a higher probability than expected. Accumulating evidence has shown the applicability of using SSA and TreeScan to generate potential safety signals. A more detailed description of SSA and TreeScan is provided in the Supplement. Considering the simplicity and efficiency of SSA in estimating risk of ADRs and also the benefits of TreeScan in hierarchical screening without the need for prior definition of certain events and in controlling type 1 error rate across multiple testing, we believe that the two methods could complement each other in the interpretation of identified signals. In this study, TreeScan analysis and SSA were conducted using TreeScan™ (https://www.treescan.org/) and SAS v9.4 software.

Exploratory Analysis

We conducted an exploratory analysis using SGLT2i and NFQ as test examples to examine the applicability of the proposed system. SGLT2is represent a newer class of drugs indicated for diabetes, a chronic disease with high prevalence, while NFQs represent an old class of medications only applied in short-term therapy for certain infections. NFQs are no longer available in many countries because newer fluoroquinolones (FQs) appear to have broader antibacterial activity and have been widely used in clinical practice;²⁶ however, NFQs are still available and frequently prescribed in some countries, especially in Asia.²⁷ In Taiwan, NFQs accounted for 26 to 46% of prescribed, defined daily doses of all quinolone antibiotics between 2008 and 2009.²⁸ Therefore, it is important to assure both effectiveness and safety of NFQs.

We applied both SSA and TreeScan as safety signal detection approaches. We used medication prescriptions and diagnostic codes for the SSA and TreeScan analyses, respectively, as the mining data. To avoid potential selection bias, we included only new users of the index drugs (ie, SGLT2i or NFQ). New users were defined as those without exposure to the study drugs during the year prior to the index date, which was the first prescription date of each study drug during the study period.

For the SSA analysis, we extracted the first prescriptions of the index drug and all other drugs within a pre-specified time window for each individual. We then assessed whether the frequency of the prescription sequence of index drug followed by each of the other drugs was greater than the inverse sequence thereof, and calculated a sequence ratio (SR) for each possible drug-event pair. The SRs were further adjusted using Tsiropoulos’ method²⁹ to account for potential bias resulting from prescribing trends of specific drugs.

For TreeScan, we prespecified a diagnosis tree with 5 levels of hierarchy coded by the International Classification of Diseases, Ninth or Tenth Revision, Clinical Modification (ICD-9-CM or ICD-10-CM). TreeScan can be conducted in a cohort- or self-controlled design. If an appropriate comparator candidate exists for the index drug, a cohort design can be applied, and the index drug’s risk of a pre-specified event can be assessed by comparing the observed number with the expected number (as calculated from the comparator drug). If, however, the index drug lacks an adequate comparator, then a self-controlled TreeScan design comparing the risk of a pre-specified event during exposure- versus non-exposure period can be applied. Schemata of different TreeScan designs are provided in Supplement Figure 1. For this analysis, we carried out a cohort-design TreeScan comparing new use of the index drug with the comparator drug to minimize confounding by indication. We calculated propensity score (PS) which is the probability of a patient being treated with the index drug and performed 1:1 matching to create two groups with balanced characteristics. The incident event was defined as any diagnosis that had not appeared before the index date. Considering that different physicians might code the same clinical condition using slightly different ICD codes, we judged incident events by the first three digits of the ICD codes. A patient might encounter multiple incident events in the follow-up period, with each of these events being tested by TreeScan individually. Considering the probability of being a case would be 0.5 under the 1 to 1 matching design, unconditional Bernoulli model was applied. In this study, we defined the detected drugs from SSA and the detected events from TreeScan as alerts with the significance level at 0.05.

Investigation of SGLT2i

For the SSA, we included new users of SGLT2i from May 2016 to December 2018. We collected every first prescription of drugs other than SGLT2i (potential marker drugs) within 12 months before and after the index date. To qualify as a first prescription of each drug, we applied a washout period of one year. For every potential marker drug, we calculated the adjusted SR based on the prescribing order of the index drug and the marker drug. We selected dipeptidyl peptidase-4 inhibitors (DPP4i) as the comparator drug because it was considered the same line of therapy in diabetes treatment and we conducted SSA for DPP4i as well.

For TreeScan, we included new users of SGLT2i during the same cohort entry period as for SSA. New users of DPP4i were also identified during the same inclusion period as active comparators. Variables considered in the generation of PS included age, sex, Charlson Comorbidity Index (CCI), health resource utilization and disease- and medication history collected at the index date or during the baseline (within 1 year before the index date). We followed the patients from the index date to outcome occurrence, drug switching, drug discontinuation, death or 365 days after the index date, whichever came first.

Investigation of NFQ

For SSA analysis of NFQs, we included new users from January 2016 to December 2016. Considering the short-term clinical use of NFQs, we set the time window for SSA analysis at 90 days before and after the index date. Adjusted SR was calculated for every possible pair of index drug and marker drug. We selected FQ as the comparator drug and conducted the same SSA.

For the TreeScan analysis of NFQs, we conducted a cohort study design with PS matching, similar to the analysis of SGLT2i. We identified new users of NFQs and FQs between 2011 and 2014, whereby the FQ users were included to serve as the comparator group. Considering that quinolone antibiotics are used for a diverse set of infection indications, we collected general covariates that included baseline demographics, CCI and health resource utilization for the calculation of PS. Consistent with the SSA of NFQs, we set the end of observation 90 days after the index date. Patients were followed up until occurrence of an incident event, drug switching, drug discontinuation, death or 90 days after the index date.

Results

The Proposed Triage System

The proposed triage system includes both prescription SSA and TreeScan as quantitative signal detection tools for safety signal detection in longitudinal databases (Figure 1). The system starts with the requirement for detection of safety signals for a (class of) target medication (drug X). Once drug X to be analysed is determined, we assess if there is an appropriate (class of) comparator drug Y to be compared with, as a reference drug. If an appropriate comparator exists, a parallel prescription SSA of both drugs and a TreeScan analysis with cohort design and propensity score-adjustment are conducted. If drug X lacks an appropriate comparator, a single SSA of drug X and a self-controlled TreeScan are performed instead. Next, all alerts detected as significant are collected. In the parallel SSA, alerts detected in both drug X and Y are considered as potential ADRs within the same class of drug (in the situation where drug Y is of the same drug class as drug X), or due to bias resulting from mutual indications with common use of medications among the study population. We exclude these alerts at this stage because these alerts are not considered to be specific to the use of drug X. The final alerts are then classified further.

Figure 1 Proposed signal detection and triage algorithm. Abbreviations: ADRr, ADR-related event; CCDS, company core data sheets; CH, Cohort-related event; SRS, spontaneous reporting system; Ir, Indication-related event; PT, Protopathic bias.

Classification and Triage of Detected Signals

After the signal detection step, we rank all detected alerts by their statistical significance (ie, adjusted SRs from SSA or p-values followed by log likelihood ratio from TreeScan) and clinical pharmacists experienced in pharmacovigilance from TDRF (a designated foundation which operates the Taiwan National ADR reporting system) further classify these alerts. First, an alert is assessed as to whether it is a known ADR or a clinical indication. If neither, the pharmacists review other information from the risk profile, literature and case reports of the drug from SRS, or consult clinical experts to ultimately categorize the detected alert as an event related to a known ADR, an event related to indication, an event related to characteristics of patients, potential bias, or unknown. The risk profile (see Supplement Tables 4 and 5) contains safety information from drug labels, company core data sheets, and risks labelled as important identified risks and important potential risks from the Periodic Benefit-Risk Evaluation Report (PBRER) and Risk Management Plan (RMP) document of the drug. All detected alerts are classified into one of the following categories.

ADR – The event is a known ADR listed in the ‘adverse reactions’, ‘warnings’ or ‘precautions’ sections of drug labels or company core data sheets.

ADR-related event (ADRr) – The event is not specifically listed in the known risk profile, but is clinically considered to be related to known ADRs.

Indication (I) – The event is listed as an indication in the drug labelling.

Indication-related event (Ir) – The event is not specifically listed in the labelling as an approved indication, but is considered to be related to a clinical indication or represents an off-label use.

Cohort-related event (CH) – After ruling out known ADRs and indications, the event cannot be explained by the pharmacological effects of the drug and is judged to be related to the characteristics of the patient group.

Protopathic bias (PT) – After ruling out known ADRs and indications, the event is considered a disease or a symptom that the target drug is often prescribed to treat.

Unknown (U) – If the event cannot be classified into any of the above, then it is considered an unknown ADR and is identified as a signal.

Examples of different categories of alerts are listed in Supplement Table 6. Besides adverse events detected by TreeScan, this classification also applied to drug alerts that were detected by SSA. For SSA, events described in all classification were refer to events that are indications of the detected drugs.

After classification of the alerts, those identified as unknown (U) are considered as safety signals of the target drug that require further prioritization. Criteria for prioritizing a signal include clinical and statistical significance. Clinical significance encompasses the seriousness of the event, and incidence or prevalence of the event. Statistical significance includes the effect size of the association between drug X and the detected signal, and whether the signal was identified by both SSA and TreeScan. The specific criteria and weights can be designed depending on the needs under different circumstances. Signals or drug-event pairs with higher priority are then further evaluated using pharmaco-epidemiological studies with more specific design tailored to each signal.

Safety Signals of SGLT2i

We identified 216,393 new users of SGLT2i for SSA and found an overall total of 46 alerts (Figure 2 and Table 1). Among them, 14 were medications used in the treatment of known ADRs, generally urinary tract infections, 28 were related to the characteristics of patients and 4 alerts were classified as signals that included the prescription of fluorouracil, calcium folinate, levodopa and decarboxylase inhibitor, and epinastine. Of the 46 alerts, 21 were also identified in the SSA of DPP4i.

Table 1 Detected Alerts of SGLT2 Inhibitor: Results from Prescription Sequence Symmetry Analysis

Figure 2 Classification of detected alerts. Numbers shown are numbers of detected alerts for each type. Abbreviations: ADR, known adverse drug reaction; ADRr, events related to known adverse drug reaction; CH, cohort-related; I, indication; Ir, indication-related; PT, protopathic bias; U, unknown alerts identified as potential signals.

For the TreeScan, we included 118,308 matched pairs of SGLT2i users and DPP4i users. The distributions of demographics, comorbidities and concurrent medications were similar for the groups after PS matching (Supplement Table 1). The results from TreeScan generated 38 alerts, all of which were known ADRs, ADR-related events or related clinical diagnoses of the patient cohort (Figure 2 and Table 2).

Table 2 Detected Alerts of SGLT2 Inhibitor: Results from Propensity Score-Matched TreeScan

Safety Signals of NFQ

We found a total of 82 alerts from SSA of NFQs. Among these, 17 were medications used to treat known ADRs or related events, 9 were medications used in managing indication-related problems, 52 were related to cohort characteristics and the remaining 4 were considered as protopathic bias (Figure 2 and Supplement Table 2). Of the 82 alerts, 44 were also identified in the SSA of FQs.

In the TreeScan analysis, a total of 394 alerts were detected and no signals were identified. Of these alerts, 74 were known ADRs or related events, 149 were indications or related clinical diagnoses, 111 were related to patient characteristics and 60 were assessed to be protopathic bias (Figure 2). The individual alerts are summarized in Supplement Table 3.

Discussion

Overall Findings

Our study delineated a new triage system integrating SSA and TreeScan for drug safety signal generation and prioritization using population-based longitudinal data. One of the greatest strengths of the system is that it utilizes quantitative data-mining tools to perform automatic and mass screening for potential drug-event pairs, and to properly classify all detected safety alerts into different categories following a triage algorithm consistent with clinical evaluation. Based on an exploratory analysis, we selected two classes of drugs with different characteristics for scanning and found that SSA and TreeScan performed well in this system. For SGLT2i as the newer class of drugs used in a highly prevalent disease, most signals were known ADRs or clinical events related to the characteristics of the treated patients; while for NFQs, an older class of drugs indicated for short-term infection, no new safety signals were found.

Discussion of the Triage System

Complementary Benefit of Integrating SSA and TreeScan for Signal Detection

In the proposed system, we utilize prescription records for SSA and disease diagnoses for TreeScan as the data to be mined. Leveraging different types of data increases the sensitivity when identifying drug safety signals. In situations where patients suffer from adverse events and are treated by medications but these are not necessarily being coded as a specific disease or clinical event, using prescription data enables us to capture more relevant information. Additionally, a drug or a class of drug could be prescribed for multiple indications, and therefore using diagnosis data helps us to distinguish different clinical events. Hence, SSA and TreeScan complement each other, allowing a more comprehensive detection of potential safety signals.

Design of a Comparator for the Index Drug

The proposed triage system is designed to incorporate clinical consideration of the drug to be scanned. For each (class of) drug X (illustrated in Figure 1), we first determine if there is any appropriate comparator (drug Y), which could be a specific drug or a class of drugs. This is representative of most clinical care conditions where multiple-drug choices within or across classes are available, and a physician often selects the one that might provide the most effectiveness with the least risk, or is most convenient to the specific patient. The benefit of having a comparator is that it enables us to identify additional safety concerns that might be associated with drug X;²⁰ hence we can focus on potential new AEs instead of those that have already been recognized with the comparator drugs. Another advantage of having a comparator drug is that it helps to remove possible AEs that are related to patient characteristics. Because the comparator is often another active ingredient within the same therapeutic category, the underlying conditions between groups can be considered similar, thus mitigating confounding by indication.³⁰ Another strength of the triage system is the mechanism of safety signal classification. We categorize all detected alerts into different groups, either as known ADRs, indications, following other related events, bias, or unknown AEs based on various sources of evidence. These sources cover a broad range, including drug labels, regulatory information, literature reviews and safety reports generated from domestic SRS, and are thus unlikely to miss any possible known AEs. Overall, the proposed triage system functions as a filter to screen out potential drug-event pairs that require further investigation both statistically and clinically. Therefore, the new system can be expected to aid in prioritization of drug safety signals and to improve the efficiency of current drug safety surveillance.

Discussion on the Exploratory Analysis

Overall Performance of SSA and TreeScan in 2 Different Drug Classes

Comparing the results of prescription SSA and TreeScan, it was noted that fewer alerts were identified for SGLT2i relative to NFQ. This might be due to different lengths of time on the market and differences in the patient groups. SGLT2i is a newer class of oral antidiabetic drugs for which evidence regarding its safety profile is still accumulating, while NFQ is an old group of antibiotics which has long been used in treating a wide range of infections. Therefore, the probability of detecting multiple AEs in the latter might be higher, due to its longer time on the market and wider-ranging patient scope. Furthermore, it was noted that 39% and 50% of alerts detected by SSA for SGLT2i and NFQ, respectively, were also identified in TreeScan. By contrast, 47% and 8% of alerts detected by TreeScan for SGLT2i and NFQ, respectively, were also found in SSA. The results show the complementary benefit of utilizing both types of data-mining approaches in order to maximize the sensitivity of signal detection.

Further examination of the distribution of alert types revealed a predominant proportion of patient characteristics-related events from the results of SSA. Approximately 61% and 63% of the alerts detected by the SGLT2i and NFQ SSAs, respectively, were related to patient characteristics, while these proportions decreased to 34% and 23%, respectively, for SGLT2i and NFQ in the TreeScan analyses, which included a comparator group. Similarly, when we compared the results from parallel SSAs of SGLT2i vs DPP4i and NFQ vs FQ, we noticed that approximately half of the detected alerts were also found in the comparator drug groups. This finding emphasized the importance of the comparator drug in controlling confounding caused by the disease or indications shared among patient groups.

Interpretation of SGLT2i Alerts

The results of the SGLT2i analyses showed that both SSA and TreeScan found alerts related to genital mycotic or urinary tract infections, which are well-known ADRs of SGLT2is (Tables 1 and 2). In addition, TreeScan identified the shoulder lesion as an alert, which included inflammation or pain of the shoulder. This might correspond to arthralgia in the current risk profile. Both approaches found alerts related to patient characteristics as well. These included medications and diagnoses related to dental problems and postmenopausal women. Studies have found that diabetes patients have a high prevalence of periodontal disease.³¹ Poor glycaemic control in diabetes patients is associated with more severe periodontal disease.³² Furthermore, more than half of the SGLT2i new users were over 50 years old (data not shown), indicating that alerts related to menopausal disorders might be due to the characteristics of these patients. The SSA results showed there were alerts related to medications used in cancer treatment and during hospitalization such as electrolytes, fluids and broad-spectrum antibiotics. The SSA results for DPP4i yielded most of these alerts as well, implying that these medications were related to cohort characteristics of diabetes patients, rather than real ADRs. Nevertheless, the SSA yielded 4 alerts assessed to be potential signals: fluorouracil, calcium folinate, levodopa and decarboxylase inhibitor, and epinastine. The former two are used in multiple cancer treatment, including invasive bladder cancer and breast cancer,³³ and are also listed as important potential risks in the PBRER of the pharmaceutical company (Supplement Table 4). Levodopa and epinastine are used to treat Parkinson’s disease and allergic conjunctivitis, respectively. However, we have found no evidence supporting any relationship between SGLT2i and the use of these medications as yet. Further analyses are required for clarification.

Interpretation of NFQ Alerts

Neither SSA nor TreeScan analysis of NFQ found new safety signals, and all alerts were classified as known ADRs, indications, cohort-related characteristics or protopathic bias (Figure 2). The SSA identified known ADRs including medications used in treating peptic ulcer, anaemia, dyslipidaemia, tissue inflammations and psychiatric disorders (Supplement Table 2). TreeScan found alerts consistent with disorders of the musculoskeletal and nervous system, such as synovitis, arthralgia and neuritis. In addition, TreeScan found more significant alerts related to the eyes, ears, and gastrointestinal system (Supplement Table 3). Since the design of the TreeScan analysis included a comparator drug group, which was the FQs, the results could be interpreted as additional safety concerns that NFQs might carry, when compared with FQs. However, the detected known ADR alerts of NFQs were mostly diagnoses with unspecific or less severe symptoms. This might have been due to the imbalance in NFQ- and FQ prescription rates between hospitals and clinics. A post hoc analysis (data not shown) supported this observation, finding that nearly 70% of NFQs were prescribed at local clinics, suggesting that the diagnoses behind NFQ prescriptions mostly involved less severe symptoms of diseases or disorders. However, although we did not find new signals for NFQ, we did identify many alerts related to diagnoses of pregnancy and childbirth, which we classified as cohort-related characteristics rather than true signals. This might imply that many NFQs were prescribed at obstetrics and gynaecology clinics, raising concerns that many pregnant women were being prescribed NFQs, which, as a class of antibiotics, are not recommended for first-line treatment during pregnancy due to a lack of safety evidence.

Strengths and Challenges of the Proposed Triage System

Through this exploratory study, we uncovered several strengths of applying the proposed system to enhance the current signal detection and triage algorithm. First, the utilization of data-mining methods on longitudinal data provides an additional tool for generating potential signals besides the conventional approach. Safety signals can be drawn from multiple sources, including SRS, active surveillance system, literature review, other regulatory bodies, etc.³⁴ Implementation of the proposed system could aid in increasing the proactiveness of the active surveillance system. Second, data-mining approaches are primarily automatic and conducted through computerized programs. This can quickly narrow down the scope of potential signals within the ocean of all possible adverse reactions while reducing manpower costs and increasing signal detection efficiency. Third, longitudinal data usually involves larger sample sizes compared to SRS data, and therefore could enhance the detecting power of safety signals, which is especially critical for the identification of ADRs with lower incidence. Even more so, if population-based data can be acquired, then the incidence of different AEs can be more accurately estimated. Fourth, consideration of an appropriate comparator drug in the proposed system allows the investigators to filter out the background noise of confounding by patient characteristics or mutual indication, or by clinical events that occurred following the disease progression.

Some challenges remain for integrating the novel approaches into the current signal detection procedure. Many important considerations must go into the establishment of such an integrated system. What criteria should be met to trigger the triage system for a certain drug? How do we build the database for storing and maintaining the data? How often should we screen or end the mining process for a drug to update the safety information in a real-time manner? With regard to the data-mining step, it is difficult to set an appropriate statistical threshold to define a “significant” signal. Compared to results with a significance level of 0.01, those with a significance level of 0.05 would definitely yield more signals, which would potentially increase the sensitivity but compromise the specificity of signal detection. Caution should be exercised because the level of significance is directly related to the probability of false positive findings, to be considered as noise in signal detection and possibly a concern under multiple testing. SSA is likely to suffer from this problem while the statistics of TreeScan can spontaneously control the false positive rate introduced by multiple testing. Another challenge of this system is that a valid method to combine signals from different data sources has yet to be formulated. Aggregating signals detected from multiple databases is important for further prioritization and evaluation of potential drug-event pairs. So far, several models^35–37 have been proposed to integrate signals, but it will require tests in real-world situations to demonstrate their practicality and efficiency.

Conclusion

This study proposed and tested a new triage system integrating prescription SSA and TreeScan for drug safety signal generation and prioritization in Taiwan. The exploratory analyses of SGLT2i and NFQ demonstrated the system’s utility in generating potential safety signals and categorizing different detected alerts, and thus could improve the efficiency of the drug safety signal detection and triage process.