Back to Journals » Clinical Interventions in Aging » Volume 17
Development and Validation of ‘Prediction of Adverse Drug Reactions in Older Inpatients (PADROI)’ Risk Assessment Tool
Authors Yadesa TM , Kitutu FE, Tamukong R, Alele PE
Received 10 December 2021
Accepted for publication 11 February 2022
Published 25 February 2022 Volume 2022:17 Pages 195—210
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 3
Editor who approved publication: Prof. Dr. Nandu Goswami
Tadele Mekuriya Yadesa,1– 3 Freddy Eric Kitutu,4,5 Robert Tamukong,1,3 Paul E Alele6
1Department of Pharmacy, Mbarara University of Science and Technology, Mbarara, Uganda; 2Department of Pharmacy, Ambo University, Ambo, Ethiopia; 3Pharmacy Biotechnology and Traditional Medicine Center, Mbarara University of Science and Technology, Mbarara, Uganda; 4Department of Pharmacy, School of Health Sciences, Makerere University, Kampala, Uganda; 5Sustainable Pharmaceutical Systems (SPS) Unit, School of Health Sciences, Makerere University, Kampala, Uganda; 6Department of Pharmacology and Therapeutics, Mbarara University of Science and Technology, Mbarara, Uganda
Correspondence: Tadele Mekuriya Yadesa, Department of Pharmacy, Mbarara University of Science and Technology, P.O.Box 1410, Mbarara, Uganda, Tel +256753312571, Email [email protected]
Background: Adverse drug reactions (ADR) detection and prediction methods in hospitalized older adults remain imprecise. The identification of the risk factors for ADRs in this group of patients is crucial to develop plausible prediction models.
Objective: This study aimed at developing and validating a “Prediction of ADR in Older Inpatients (PADROI)” risk assessment tool in hospitalized older adults.
Methods and Materials: We had previously conducted a derivational study that aimed to determine the risk factors of ADRs in hospitalized older adults. We developed the PADROI model as a potential ADR risk assessment tool incorporating 8 predictors each given a score by rounding off the respective adjusted odds ratios (AORs) to the nearest whole number. Subsequently, we conducted another prospective cohort among adults aged 60 years and older admitted to Gynecology and Obstetrics, Medical, Oncology, Surgery, and Psychiatry wards at Mbarara Regional Referral Hospital (MRRH) from July 5 to September 17, 2021.
Results: A total of 124 participants, 70 females and 54 males aged 60– 95 years, were included in this validation cohort; 62 of them experienced 90 ADRs. When applied to the derivational cohort, the area under receiver operating characteristic curve (AUROC) for the PADROI model was shown to be 0.896 (0.869– 0.923; at 95% CI). In the validation study, AUROC of PADROI was 0.917 (0.864– 0.971 at 95% CI; p < 0.001). Overall, PADROI correctly predicted 91.7% of those who experienced an ADR.
Conclusion: Using the adjusted odds ratios from our derivational cohort, we developed an ADR prediction tool (PADROI) that achieved an excellent AUROC (0.917), high sensitivity (87.1%) and specificity (90.3%). The current model demonstrated a high potential for clinical applicability which can be strengthened if similar results are reproduced in larger and multi-centered studies.
Keywords: development, validation, prediction, adverse drug reaction, inpatients, older adults
The occurrence of adverse drug reactions (ADRs) continues to substantially contribute to the morbidity, mortality and high health care cost in the older population.1–4 The prevalence of ADRs among hospitalized older adults ranged from 6–46% in high income countries (HICs) and from 10.7–64.0% in low- and middle-income countries (LMICs). The majority (60%) of all ADRs in this patient population are potentially preventable.5
Current recommended practice for detecting and predicting ADRs in the elderly comprises thorough documentation and consistent evaluation of prescribed and over-the-counter medications through standardized medication reconciliation.6 ADR detection and prediction methods in hospitalized older adults remain imprecise. Focusing on high-risk medications and patients with multi-morbidity may advance prediction of adverse drug reaction.7
Obtaining a good ADR risk-prediction model usually occurs over four stages; first, identification of predictors of the phenomenon; second, validation that involves testing the potential model performance; third, evaluation of impact and usefulness in routine clinical practice; and lastly, implementation to assess acceptability and performance in real-life clinical practice.8
Area Under the Receiver Operating Characteristic Curves (AUROC) is the average value of the sensitivity for a test over all possible values of specificity or vice versa.9 In screening tests, sensitivity, the true positive rate, refers to the proportion of people that are correctly predicted to have a condition out of all people who actually have it. A sensitivity of 100% means the ability to correctly predict all people with the condition. Specificity, the true negative rate, refers to the proportion of those who were correctly predicted not to have a condition among those that actually do not experience it.10−13 Prediction models are categorized based on their AUROC curve values as: “excellent” (AUROC curve ≥0.900), “very good” (AUROC curve 0.80–0.89), “acceptable” (AUROC curve 0.70–0.79) and ”poor” (AUROC curve <0.7).14–16
Previously developed ADR risk-prediction models in older inpatients had achieved AUROC of 0.7017 to 0.7410 during their validation stages and all of them showed low specificity (<65%) and some of them used retrospective studies to develop their models.17,18 The existing models need further work to enable the development of a robust ADR risk-prediction model that is externally validated, with practical design and good performance.19 However, to the best of our knowledge, there is no study published on the validation of an ADR risk-prediction model for older inpatients in LMICs.
These ADR-risk prediction tools help health professionals to accurately predict patients that are going to incur an ADR during their hospital stay, thus, highlighting the need for close monitoring, avoidance of some medications or combination of drugs and to implement other relevant interventions and, ultimately, to mitigate the burden of ADRs in clinical as well as economic aspects.20,21 This study, thus, aimed at developing and validating ‘Prediction of ADR in Older Inpatients’ (PADROI), an ADR-risk prediction tool, by employing two separate prospective cohort studies in older adults in Uganda, a low income country.
Development and Scoring of the PADROI Model
The authors had previously conducted a systematic review on risk factors of ADRs among hospitalized older adults. It identified 15 independent risk factors reported by previous studies. The details of the systematic review were published elsewhere.5 Then we included the 15 previously reported risk factors and four more relevant independent variables, as independent variables in our derivational study conducted at MRRH from November 9, 2020 to May 7, 2021. This study, particularly, aimed at determining the risk factors of hospital-acquired ADRs in this group of patients. Details of the results were published elsewhere.22 In the current validation study, we aimed at developing a potential ADR-risk assessment tool from the data set of the derivational study, and then to validate it in a separate follow-on prospective observational study. The data on the 19 independent variables were extracted and were assessed for assumptions of logistic regressionand subjected to multivariable logistic regression. We first verified that there was no significant multicollinearity between any of the 13 independent variables. Moreover, both forward and backward conditional logistic regression similarly showed 8 out of the 13 independent variables to be significantly associated with the occurrence of hospital-acquired ADRs. Eight independent variables that were statistically significant and were retained in the final model included: age 60–75 (AOR = 1.97, 95% CI: 1.14–3.41; p = 0.015) compared with >75 years, previous ADR in 1 year (AOR = 2.43, 95% CI: 1.42–4.17; p = 0.001), PIM (AOR = 4.56, 95% CI: 2.70–7.70; p <0.001), polypharmacy (AOR = 3.29, 95% CI: 1.98–5.46; p <0.001), having CCI ≥6 (AOR = 8.47, 95% CI: 4.85–14.99; p <0.001), having heart failure (AOR = 2.83, 95% CI: 1.34–6.02; p = 0.007) or kidney disease (AOR = 1.95, 95% CI: 1.05–3.61; p = 0.034) and a hospital stay >10 days (AOR = 3.53, 95% CI: 1.89–6.61; p <0.001) compared with <5 days.
Study Setting and Period
This validation cohort was done at Mbarara Regional Referral Hospital (MRRH); the largest referral hospital in south-western Uganda. The hospital consists of Emergency, Inpatient and Outpatient services for a population of about four million people in its catchment areas including the districts of Mbarara, Bushenyi, Ntungamo, Kiruhura, Ibanda, Buhweju, Rubirizi, Mitooma and Isingiro. The hospital also receives patients from farther districts of Kabale, Masaka, Fort Portal and neighboring countries such as Rwanda and Tanzania. The current study was conducted in Gynecology and Obstetrics, Medical, Oncology, Surgery, and Psychiatry wards of the hospital from July 5 to September 17, 2021.
A prospective cohort study was conducted to determine the prediction ability of the PADROI ADR prediction tool.
We included all inpatients 60 years and older who were admitted to Gynecology and Obstetrics, Medical, Oncology, Surgery, and Psychiatry wards of MRRH during the study period and gave their informed consent. We excluded patients in coma or any level of unconsciousness as well as unstable psychiatric patients with mood disorders, schizophrenia and dementia who were on acute treatment. We also excluded patients who died or were discharged in less than 48 hours.
Sample Size Determination
The sample size for this validation study was calculated using MedCalc® Software Version 19.2 (©1993–2020), that employed the standard formula for sample size determination for ROC studies.23 Taking a power of 90% and Type I error of 0.01, null hypothesis AUROC (the best available AUROC in older inpatients) of 0.74,10 and a target AUROC of 0.896 obtained from derivational cohort of the current project,22 48 positive cases (patients with ADR) and 48 negative cases (patients without ADR) are required for this ROC study. By adding 25% for potential drop-outs, 60 positive cases and 60 negative cases, totaling 120 patients, who are 60 years and older were required.
We employed a consecutive sampling technique involving all adults 60 years and older admitted at Gynecology and Obstetrics, Medical, Oncology, Surgery, and Psychiatry wards. Accordingly, all older patients who passed the inclusion and exclusion criteria were consecutively recruited until the target sample was achieved for both the cases and controls.
Three research assistants were involved in the data collection process. The principal investigator (senior clinical pharmacist) together with two physicians formed a team of experts. Firstly, an interviewer-based structured questionnaire was used to collect the data on participants’ characteristics, history of medical, surgical, gynecological and psychiatric conditions and previous drug uses, drug allergies, use of non-prescription and alternative medicines. Secondly, we reviewed patients’ medical files for diagnosis, comorbid conditions, previous adverse drug events, and diagnostic test results as soon as possible but within 48 hours. Thirdly, every day during their hospital stay except on Sundays, we updated the participants’ information after we interviewed them, conducted targeted physical assessments, and reviewed their medical files.
Fourthly, we used the British National Formulary (BNP)24 and UpToDate25 to identify the known ADR profile of the drugs used. We employed the Beers Criteria26 to identify PIMs. Polypharmacy was defined as concurrent use of five or more drugs (active pharmaceutical ingredients).18,27–31 Lexicomp® was used to detect clinically significant drug-drug interactions. The medications suspected for ADRs were classified according to the WHO-Anatomical Therapeutic Chemical (ATC) classification.32 Charlson Comorbidity Index was used to rate the complexity and prognosis of the participants’ conditions.33
All adverse events suspected by the principal investigator and the physician were considered for ADR causality rating and discussion by the team. The body systems affected by ADRs were categorized using the International Statistical Classification of Diseases for Mortality and Morbidity Statistics (ICD-11 MMS).34
Identification of ADRs
We used Edwards and Aronson’s definition of ADR: ‘an appreciably harmful or unpleasant reaction, resulting from an intervention related to the use of a medicinal product, which predicts hazard from future administration and warrants prevention or specific treatment, or alteration of the dosage regimen, or withdrawal of the product.’35 ADRs were first suspected when there was a relationship between the time of drug administration and the onset and course of the adverse reaction while excluding other potential causes.36 Every day except on Sundays, all ADRs suspected by the principal investigator were discussed with the team of experts consisting of the principal investigator (senior clinical pharmacist) and two senior physicians to establish the drug causality of the suspected ADRs. The principal investigator then used the Naranjo ADR assessment scale37 to rate the causal relationship of an ADR and the suspected medication. ADRs were classified as definite (9–12 points), probable (5–8 points), possible (1–4 points), or doubtful (0 points). We excluded all doubtful ADRs whereas we endorsed those rated as possible, probable or definite as’ possible ADR’s once consensus was reached by the team of experts. When consensus was not reached, a majority decision of the three members of the team was applied.
Data Analysis and Interpretation
The data were entered and cleaned by EpiInfo version 126.96.36.199 and then transferred to and analyzed by IBM Statistical Package for the Social Sciences (SPSS version 23.0 Inc., Chicago, Illinois). Descriptive statistics was used to determine the frequencies and percentages of the ADR occurrences and categories, drugs associated with the ADRs, and distribution of the predictors among patients with at least one hospital-acquired ADR and those without one.
For both the derivational and validation studies, we determined the predictive ability of the PADROI model by fitting Receiver Operating Characteristic curves and calculating the area under the curve and sensitivities and specificities at different cut-off points using ROC analysis using SPSS. A p value of <0.05 was considered statistically significant in all analyses.
Data Management and Quality Assurance
The research assistants were trained by the principal investigator. Then a pre-test was conducted involving two patients at each ward. Data collection procedure was revised based on the experiences from the pilot test. The collected data were checked daily for completeness and consistency by the principal investigator. Confirmation and causality rating of ADRs were discussed among the team of experts to reach a consensus. Data were double-entered, cross-checked, and password-protected.
This study was conducted according to the Declaration of Helsinki.38 The current study was approved by Institutional Review Board of Mbarara University of Science and Technology (Reference No. MUREC 1/7) and Uganda National Council for Science and Technology (Reference No. HS992ES).
Development and Scoring of the PADROI Risk Assessment Tool
A score was assigned to each of the 8 included predictors by rounding off the respective Adjusted Odds Ratios (AORs) to the nearest whole number. The score for hospital stay was considered only when patients were hospitalized for 11 or more days. For patients who experienced an ADR, the hospital stay referred to the number of days before the first ADR occurred instead of the total duration of hospital stay. Likewise, only polypharmacy and PIM incidents that were experienced before the occurrence of an ADR were used in scoring of PADROI. Charlson Comorbidity Index (CCI) calculated before the first incident of ADR was considered. Previous ADRs included any possible suspected ADR that occurred within one year preceding the current hospitalization. Kidney disease includes any structural or functional renal problem diagnosed by a doctor as confirmed by laboratory tests, diagnostic tools or biopsy. The weight of the risk associated with each independent predictor was obtained by rounding off the respective AORs to the nearest whole number. The sum of weights of the eight-risk factors of the PADROI model totaled 29. We also developed an alternative model by assigning points to each predictor using the adjusted β coefficients rounded off to one decimal and then by multiplying each score by 10 (Table 1).
Table 1 PADROI ADR Risk Prediction Score Form Derived from a Cohort of Ugandan Older Adults from July to September 2021
ROC Curve for the Derivational Study
The AUROC curve for the PADROI model that was developed using adjusted odds ratios was shown to be 0.896 (0.869–0.923; at 95% CI; p <0.001) for the derivational study. This AUC is classified as’ very good or 0.800–0.899.’ The ROC curve was positioned at the top left for sensitivity which revealed its high prediction ability for ADRs (Figure 1). A cut-off point for PADROI score of 11 and above showed an optimal ADR risk prediction ability with a balanced sensitivity and specificity. At this point PADROI showed a sensitivity of 79.3% (true positive) and specificity of 86.1% (true negative). The mean PADROI scores were observed to be 15.5±5.8 and 5.9±4.6 for patients with ADR and those without respectively.
Figure 1 ROC curve of PADROI tool that was developed using adjusted odds ratios for the derivational study.
The AUROC curve was 0.897 (0.870–0.924 at 95% CI; p <0.001) for the PADROI tool that was developed using the adjusted β coefficients. Thus, the ADR risk prediction ability of the two tools was shown to be the same. Because of its simplicity (total scores of 29 compared with 94) and a smoother AUROC curve, we selected PADROI tool that was developed from the adjusted odds ratios for our validation study (Figures 2 and 3).
Figure 2 ROC curve of PADROI tool that was developed using adjusted β coefficients for the derivational study.
Figure 3 The comparison of ROC curves of PADROI tools that were developed using adjusted odds ratios and using β coefficients for the derivational study.
The Participant Characteristics
For this validation cohort, we studied 124 patients consisting of 70 females and 54 males; 62 patients with ADRs and 62 patients without ADRs (Table 1). The median age of the patients was 67 (62–75) ranging between 60 and 95 years. Fifty-one of the 124 patients were admitted at the Medical ward followed by 30 at Surgery wards (Figure 4).
Figure 4 The gender and ward distribution of older patients hospitalized at MRRH from July to September 2021.
ADR Occurrence and the Implicated Drugs
Sixty-two patients experienced a total of 90 ADRs during the current hospital stay. Applying the Naranjo ADR causality scale, 68 (75.6%), 19 (21.1%), and 3 (3.3%) ADRs were rated as probable, possible, and definite ADRs, respectively. ADRs affecting the nervous (38, 42.2%), gastrointestinal (28, 31.1%), and cardiovascular systems (13, 14.5%) were the three most frequently experienced ADRs during the current hospitalization. Metronidazole (11/90), ceftriaxone (7/90), furosemide (6/90), tramadol (6/90) and morphine (6/90) were observed to be the five drugs most commonly suspected as the cause of the ADRs (Table 2).
Table 2 The Types and Categories of ADRs Detected and the Implicated Drugs Among Older Patients Hospitalized at MRRH from July to September 2021
The Distribution of ADRs Among Patients with Different Risk Factors
Ninety-three (75.0%) of the patients were 60–75 years old; out of which 48 (51.6%) experienced at least one ADR. Among 98 (79.0%) patients who stayed in the hospital for 11 and more days, 52 (53.1%) incurred ADR. Similarly, among the 53 (42.7%) patients who had experienced an ADR in the previous one year, 33 (62.3%) experienced ADR during the current admission. Twenty-six (21.0%) of them had been diagnosed with a renal disease, 21 (16.9%) patients had heart failure and 37 (29.8%) had a CCI≥6. Another 45 (36.3%) patients took at least one PIM and 76 (61.3%) of the patients were on polypharmacy; out of which 37 (82.2%) and 49 (64.5%) experienced ADR respectively (Table 3).
Table 3 The Distribution of ADRs Among Hospitalized Elderly Patients with Different Independent Predictors and at MRRH, Uganda from July to September, 2021
Receiver Operating Characteristic (ROC) Curve of the Validation Cohort
The ROC curve for this validation study is positioned at top-left side showing a good prediction ability of the tool for hospital-acquired ADRs in hospitalized older adults. The current AUROC, which is the average value of the sensitivity for a test over all possible values of specificity or vice versa,9 of 0.917 (0.864–0.971 at 95% CI; p <0.001) is categorized as excellent (AUC >0.900). Overall, the PADROI tool has correctly predicted 91.7% of those who experienced an ADR but falsely classified 8.3% of the patients to be at no risk of incurring an ADR (Figure 5). The mean PADROI scores for participants with ADR and without ADR was shown to be 15.4±5.3 and 5.9±3.6, respectively.
Figure 5 The ROC curve of the validation study among older adults hospitalized at MRRH from July to September, 2021, Mbarara, Uganda.
Sensitivity and Specificity of the ROC at Different Cut-off Values for PADROI
As the scores of the PADROI increased from 0 to 10, the AUROC increased from 0.516 to 0.887 and its sensitivity declined from 100% to 87.1% and the specificity increased from 3.2% to 90.3%. Beyond the PADROI scores of 10, however, the AUROC did not show increase and then began to decline as the sensitivity was declining. Thus, a cut-off point for PADROI scores of ≥10 points showed an optimal prediction ability where it correctly predicted 54 (PADROI ≥10) out of the 62 patients who actually experienced an ADR (sensitivity of 87.1%) whereas 56 (PADROI <10) out of the 62 patients without ADR were correctly classified as not at risk of ADR (specificity = 90.3%) (Table 4).
Table 4 The Sensitivities and Specificities at Different Cut-off Points of PADROI Scores in Predicting ADR Among Older Adults Hospitalized at MRRH from July to September, 2021, Mbarara, Uganda
Receiver Operating Characteristic (ROC) Curve of the Combined Cohort
The AUROC for the combined cohort was observed to be 0.900 (0.876–0.924 at 95% CI; p <0.001). It showed a sensitivity of 78.3% and a specificity of 89.4% at a cut-off PADROI point of 11 and above (Figure 6).
Figure 6 The ROC curve of the combined cohort of older adults hospitalized at MRRH from July to September, 2021, Mbarara, Uganda.
Identification of the predictors of ADR occurrence in hospitalized older patients is crucial to develop a strong ADR-risk prediction tool.20,21 A systematic review of previously reported risk factors and a subsequent prospective derivational study were conducted to determine the risk factors of hospital-acquired ADRs in older adults. In the current study, we developed an ADR-risk prediction tool for older inpatients using adjusted odds ratios in the final model of multivariable logistic regression, and an alternative tool using the adjusted β coefficients of the same model. The two potential ADR-prediction tools showed the same prediction ability; AUROC curves of 0.896 and 0.897, respectively. However, because of its simplicity (total scores out of 29 compared with out of 94) and a smoother AUROC curve, we selected the PADROI risk assessment tool that was developed from the adjusted odds ratios for our validation study.
The PADROI’s AUROC curve of 0.896 of the derivational cohort is classified as’ very good or 0.800–0.899.’ The ROC curve was positioned at the top left for sensitivity which revealed its high prediction ability for ADRs. The current AUC is considerably higher than those reported during previous derivational studies including 0.623 in Irish older adults,18 0.74 in UK older inpatients,10 0.71 in Italian older patients,17 and 0.627 in Japanese older patients.39
While applying PADROI to the derivational cohort, a cut-off score of 11 and above out of 29 showed an optimal prediction ability for hospital-acquired ADRs in older adults. At this point, a sensitivity of 79.3% (true positive) and specificity of 86.1% (true negative) were observed. A cutoff point with a higher specificity compared with sensitivity (86.1% vs 79.3%) was preferred though falsely ruling out ADR risk has more deleterious effects on the patient. However, a sensitivity of about 80% is also quite adequate to minimize the number of cases falsely predicted to be at no risk. Higher specificity, in turn, enables to avoid unnecessary intensive monitoring for ADRs and needless avoidance of some medications though the ADR is not actually incurred. Unlike the previous similar ADR prediction studies that were shown to be unsuitable for a wide clinical use because of their AUROC values of less than 0.80,7 PADROI score showed a very good AUROC and was potentially applicable for wider clinical use. Thus, we carried out a separate validation cohort for external validation.
In the validation study, 62 patients experienced a total of 90 possible ADRs during the current hospital stay. ADRs affecting the nervous system (38/90), gastrointestinal (28/90) and cardiovascular (13/90) systems were the three most frequently experienced ADRs during the current hospitalization. The proportion of ADRs affecting the nervous system, however, is considerably higher than that of the derivational study. This can be explained by the inclusion of psychiatry wards in the validation cohort. On the other hand, these results are comparable with findings from previous studies that had shown ADRs affecting gastrointestinal tract,39–43 nervous system30,39,43 and cardiovascular system30,41 to be the commonest types in this population. Patients in psychiatry wards experienced more ADRs affecting the nervous system. Metronidazole (11/90), ceftriaxone (7/90), furosemide (6/90), tramadol (6/90) and morphine (6/90) were observed to be the five drugs most commonly suspected as the cause of the ADRs. These results are comparable with the derivational cohort and other previous studies that showed antimicrobials and drugs acting on the nervous system and cardiovascular drugs to be the most frequent culprit medications in hospital-acquired ADRs in older adults.40,43–47
Consistent with previous studies, the proportion of hospital-acquired ADR was higher in patients aged 60–75 years compared with those older than 75;41,42,48,49 among those who were hospitalized for 11 and more days, in patients with a history of an ADR in the previous one year,17,50 in those with a renal disease,18,22,30 in patients with heart failure17,18,51 and those with a CCI ≥6,17,18,48 in older patients that took at least one PIM18,30 and those on polypharmacy10,22,45,50 compared with their controls.
The ROC curve for the current validation study is also positioned at the top-left side showing high prediction ability of the tool for hospital-acquired ADRs in hospitalized older adults. AUROC measures the probability that’ a patient with an ADR had a higher predicted probability than a patient without an ADR.’ An AUROC of 1 represents a perfect model whilst 0.5 is random concordance.10−12
The current AUROC of 0.917 (0.864–0.971 at 95% CI; p <0.001) is categorized as excellent (AUROC >0.900). AUROC is the average value of the sensitivity for a test over all possible values of specificity or vice versa.9 Overall, the PADROI tool has correctly predicted 91.7% of those who experienced an ADR but falsely classified 8.3% of the patients to be at no risk of incurring an ADR.
Sensitivity is the ability of a screening tool to correctly identify people who have a condition whereas specificity is its ability to correctly identify people who do not have the condition.13 During the validation cohort PADROI scores of 10 and above showed the best-balanced sensitivity and specificity where it correctly predicted 54 (PADROI ≥10) out of the 62 patients who actually experienced an ADR to be at risk (sensitivity of 87.1%) and 56 (PADROI <10) out of the 62 patients without ADR were correctly classified to be at no risk of ADR (specificity = 90.3%). Thus only 12.9% and 9.7% of the patients were wrongly classified to be at no risk of an ADR and at a risk of ADR respectively. The current sensitivity and specificity of 87.1% and 90.3% are considerably higher than sensitivities and specificities of 80% and 55% in BADRI model from a UK study,10 68% and 65% in GerontoNet ADR Risk Score from an Italian cohort.17
The AUROC of the PADROI’s validation study is significantly higher than 0.592 from an Irish validation cohort,18 0.61 from a multi-centered observational study in Europe,52 0.623 in Ireland,30 0.70 in an Italian study,17 and 0.73 from another validation study of European patients.10 Our combined cohort, similarly, showed an excellent AUROC of 0.900 (0.876–0.924 at 95% CI; p <0.001) and a sensitivity of 78.3% and a specificity of 89.4% at a cut-off PADROI score of 11/29 and above. This is significantly higher compared with an AUROC of 0.566 reported by the combined cohort from an Irish study that applied the GerontoNet ADR risk scale to its combined cohorts.18
The possible explanations for the current higher AUROC include: higher number of variables studied in derivational cohort and thus more comprehensive final model as compared with the previous studies that consisted of 4–6 variables10,17,30 and the use of two separate prospective cohorts for derivational and validation studies as compared with ADRROP18 and GerontoNet17 models which were retrospectively developed from patient databases. Moreover, the lower AUROC curve from previous studies might be attributed to their inclusion of other study settings or different countries10,17,30 during the validation study unlike the PADROI model that was developed and validated in the same setting. Moreover, our study included all possible, probable and definite ADRs unlike O’Mahony et al. that excluded possible ADRs.30 This might explain the lower sensitivity (62%) reported by the latter.
The current validation study’s strengths include its power of 90%, consistency in ADR definition, ADR identification procedures, employing validated standard data collection tools, and engaging a multidisciplinary team consisting of doctors and pharmacists in an attempt to improve the accuracy of ADR identification and characterization. However, the study has some important limitations to be taken into consideration including: the model being driven from a single-centered study in a low income setting, external validation involving a single health facility that was the same as one involved in the derivational study and a short study period of the validation cohort. Accordingly, the types of medications used, the common medical conditions, and the health professional and health facility factors might differ from the conditions in other health facilities in the region and elsewhere. This might potentially limit its application in wider clinical uses and thus, we recommend larger and multi-centered cohorts of older adults to be undertaken applying this greatly promising model in low- and middle-income countries as well as high-income countries.
A prospective derivational study was conducted to develop an ADR-risk prediction tool named as PADROI among older inpatients. The PADROI model achieved a very good AUROC in derivational cohort, and excellent AUROC curve in validation cohort as well as in combined cohort. The PADROI model demonstrated a specificity of 90.3% and a sensitivity of 87.1%. The current AUROC curve, sensitivity and specificity were all considerably higher than those achieved by previous studies. We recommend a multi-centered validation study to evaluate the performance and impact the PADROI tool in clinical practice.
ACE, African Center of Excellence; ADR, Adverse Drug Reaction; AOR, Adjusted Odds Ratio; AUROC, Area Under the Receiver Operating Characteristic Curve; ATC, Anatomical Therapeutic; BUN, Blood Urea Nitrogen; CCI, Charlson Comorbidity Index; CI, Confidence Interval; eGFR, Estimated Glomerular Filtration Rate; MRRH, Mbarara Regional Referral Hospital; MUST, Mbarara University of Science and Technology; PADROI, Prediction of Adverse Drug Reactions in Older Inpatients Model; PIM, Potentially Inappropriate Medications; SPSS, Statistical Package for the Social Sciences; WHO, World Health Organization.
Data Sharing Statement
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
Ethics Approval and Consent to Participate
This study was conducted in accordance with the Declaration of Helsinki.39 The study protocol was approved by the Research and Ethics Committee of Mbarara University of Science and Technology. We also obtained clearance from MRRH and informed consent from each participant.
Consent for Publication
All authors agreed to the submission of this manuscript for publication in addition to the consent to publish which was included in the informed consent form which attained ethical and participant approval.
The authors extend their sincere gratitude towards PHARMBIOTRAC-MUST for funding this study. We would also like to thank the research assistants and physicians: Dr. Joshua Kiptoo, Dr. Kushemererwa Oliver, Dr. Bonny Luzze, Dr. Mohamed Mukhtar Abdirahman, Dr. Jacinta Ambaru Ojia, and the workers and management of MRRH for their assistance and collaboration throughout the study. We also thank the study participants and their caregivers for their collaboration.
All authors contributed to data analysis, drafting or revising the article, have agreed on the journal to which the article will be submitted, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.
This study was funded by Pharmacy Biotechnology and Traditional Medicine Center at Mbarara University of Science and Technology.
The authors declare that they have no competing interests in this work.
1. Pirmohamed M, James S, Meakin S, et al. Adverse drug reactions as cause of admission to hospital: prospective analysis of 18 820 patients. BMJ (Clinical Research Ed.) 2004;329(7456):15–19. doi:10.1136/bmj.329.7456.15
2. UMC-WHO. Safety Monitoring of Medicinal Products: Guidelines for Setting Up and Running a Pharmacovigilance Centre. WHO Collaborating Centre for International Drug Monitoring; 2000: 28.
3. Seddon C. Reporting adverse drug reactions A guide for healthcare professionals. BMA 2006.
4. Rabiu A, Simbak N, Haque M. A Systematic Review of Knowledge, Attitude and Practice on Adverse Drug Reactions and Pharmacovigilance Among Doctors. J Appl Pharm Sci. 2014; 4(11): 117-127.
5. Yadesa TM, Kitutu FE, Deyno S, Ogwang PE, Tamukong R, Alele PE. Prevalence, characteristics and predicting risk factors of adverse drug reactions among hospitalized older adults: a systematic review and meta-analysis. SAGE Open Med. 2021;9:20503121211039099. doi:10.1177/20503121211039099
6. Lavan AH, Gallagher P. Predicting risk of adverse drug reactions in older adults. Ther Advan Drug Saf. 2016;7(1):11–22. doi:10.1177/2042098615615472
7. Jennings E, Gallagher P, O’Mahony D. Detection and prevention of adverse drug reactions in multi-morbid older patients. Age Ageing. 2019;48(1):10–13. doi:10.1093/ageing/afy157
8. Toll DB, Janssen KJ, Vergouwe Y, Moons KG. Validation, updating and impact of clinical prediction rules: a review. J Clin Epidemiol. 2008;61(11):1085–1094. doi:10.1016/j.jclinepi.2008.04.008
9. Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thor Oncol. 2010;5(9):1315–1316. doi:10.1097/JTO.0b013e3181ec173d
10. Tangiisuran B, Scutt G, Stevenson J, et al. Development and validation of a risk model for predicting adverse drug reactions in older people during hospital stay: Brighton Adverse Drug Reactions Risk (BADRI) model. PLoS One. 2014;9(10):e111254.
11. Pencina MJ, D’Agostino RB
12. Fawcett T. An introduction to ROC analysis. Pattern Recognit Lett. 2006;27(8):861–874. doi:10.1016/j.patrec.2005.10.010
13. Trevethan R. Sensitivity, specificity, and predictive values: foundations, pliabilities, and pitfalls in research and practice. Front Public Health. 2017;5:307. doi:10.3389/fpubh.2017.00307
14. Fischer JE, Bachmann LM, Jaeschke R. A readers’ guide to the interpretation of diagnostic test properties: clinical example of sepsis. Intensive Care Med. 2003;29(7):1043–1051. doi:10.1007/s00134-003-1761-8
15. Hosmer DW
16. Akobeng AK. Understanding diagnostic tests 3: receiver operating characteristic curves. Acta paediatrica. 2007;96(5):644–647. doi:10.1111/j.1651-2227.2006.00178.x
17. Onder G, Petrovic M, Tangiisuran B, et al. Development and validation of a score to assess risk of adverse drug reactions among in-hospital patients 65 years or older: the GerontoNet ADR risk score. JAMA Intern Med. 2010;170(13):1142–1148.
18. O’Mahony D, O’Connor MN, Eustace J, Byrne S, Petrovic M, Gallagher P. The adverse drug reaction risk in older persons (ADRROP) prediction scale: derivation and prospective validation of an ADR risk assessment tool in older multimorbid patients. Eur Geriatr Med. 2018;9:191–199. doi:10.1007/s41999-018-0030-x
19. Stevenson JM, Williams JL, Burnham TG, et al. Predicting adverse drug reactions in older adults; a systematic review of the risk prediction models. Clin Interv Aging. 2014;9:1581. doi:10.2147/CIA.S65475
20. Ogoina D, Obiako RO, Muktar HM, et al. Morbidity and mortality patterns of hospitalised adult HIV/AIDS patients in the era of highly active antiretroviral therapy: a 4-year retrospective review from Zaria, Northern Nigeria. AIDS Res Treat. 2012;2012:940580. doi:10.1155/2012/940580
21. Namme Luma H, Doualla M-S, Choukem S-P, et al. Adverse drug reactions of Highly Active Antiretroviral Therapy (HAART) in HIV infected patients at the General Hospital, Douala, Cameroon: a cross sectional study. Pan Afr Med J. 2012;12:87.
22. Yadesa TM, Kitutu FE, Tamukong R, Alele PE. Prevalence, incidence, and characteristics of adverse drug reactions among older adults hospitalized at mbarara regional referral Hospital, Uganda: a prospective cohort study. Clin Interv Aging. 2021;16:1705–1721. doi:10.2147/CIA.S332251
23. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. doi:10.1148/radiology.143.1.7063747
24. Committee Joint Formulary. British National Formulary; 2019.
25. UpToDate Contributors. UpToDate 2021;Version 23; 2021.
26. American Geriatrics Society. Updated AGS beers criteria® for potentially inappropriate medication use in older adults. J Am Geriatr Soc. 2019;67(4):674–694. doi:10.1111/jgs.15767
27. Bergman-Evans B, Adams S, Titler MG. Improving medication management for older adult clients. J Gerontol Nurs. 2006;32(7):6–14.
28. Jones BA. Decreasing polypharmacy in clients most at risk. AACN Adv Crit Care. 1997;8(4):627–634.
29. Preskorn SH. Multiple medication use in patients seen in the veterans affairs healthcare system: so what? J Psychiatric Pract. 2005;11(1):46–50. doi:10.1097/00131746-200501000-00006
30. O’Mahony D, Gallagher P, O’Connor MN, Byrne S. Adverse drug reactions in older patients during hospitalisation: are they predictable? Age Ageing. 2012;41(6):771–776. doi:10.1093/ageing/afs046
31. Bushardt RL, Massey EB, Simpson TW, Ariail JC, Simpson KN. Polypharmacy: misleading, but manageable. Clin Interv Aging. 2008;3(2):383. doi:10.2147/CIA.S2468
32. World Health Organisation. WHO Collaborating Centre for Drug Statistics Methodology. Available from: https://www.whocc.no/atc_ddd_index/.
33. Meng J, Zhang JX, Pu Y. Quantification of the independent prognostic value of Charlson Comorbidity Index among patients with non-small cell lung cancer. J Clin Oncol. 2018;36(15_suppl):e21164–e21164. doi:10.1200/JCO.2018.36.15_suppl.e21164
34. World Health Organisation. ICD-11 for mortality and morbidity statistics. Available from: https://icd.who.int/browse11/l-m/en.
35. Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. The Lancet. 2000;356(9237):1255–1259. doi:10.1016/S0140-6736(00)02799-9
36. Handler SM, Hanlon JT, Perera S, Roumani YF, Nace DA. Consensus list of signals to detect potential adverse drug reactions in nursing homes. J Am Geriatr Soc. 2008;56:808–815. doi:10.1111/j.1532-5415.2008.01665.x
37. Naranjo CA, Busto U, Sellers EM, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther. 1981;30(2):239–245. doi:10.1038/clpt.1981.154
38. Association WM. World Medical Association Declaration of Helsinki. Ethical principles for medical research involving human subjects. Bull World Health Organ. 2001;79(4):373.
39. Kojima T, Matsui T, Suzuki Y, et al. Risk factors for adverse drug reactions in older inpatients of geriatric wards at admission: multicenter study. Geriatr Gerontol Int. 2019;20(2):144.
40. Conforti A, Costantini D, Zanetti F, Moretti U, Grezzana M, Leone R. Adverse drug reactions in older patients: an Italian observational prospective hospital study. Drug Healthc Patient Saf. 2012;4:75–80. doi:10.2147/DHPS.S29287
41. Rawat RS. Evaluation of potentially inappropriate medication use and risk of adverse drug reactions in hospitalized older adults: an observational study in a tertiary care hospital. Int J Pharm Res. 2018;11(2):79–85.
42. Sneha SG, Simhadri K, Subeesh VK, Sneha SV. Predictors of adverse drug reactions in geriatric patients: an exploratory study among cancer patients. South Asian j Cancer. 2019;8(2):130. doi:10.4103/sajc.sajc_218_18
43. Harugeri APG, Ramesh M, Guido S, Basavanagowdappa H. Frequency and nature of adverse drug reactions in elderly in-patients of two Indian medical college hospitals. J Postgrad Med [serial online]. 2011;57(3):189–195.
44. Ahmed B, Nanji K, Mujeeb R, Patel MJ. Effects of polypharmacy on adverse drug reactions among geriatric outpatients at a tertiary care hospital in Karachi: a prospective cohort study. PLoS One. 2014;9(11):e112133. doi:10.1371/journal.pone.0112133
45. Tangiisuran B, Davies J, Wright J, Rajkumar C. Adverse drug reactions in a population of hospitalized very elderly patients. Drugs Aging. 2012;29:669–679.
46. Liao P-J, Mao C-T, Chen T-L, Deng S-T, Hsu K-H. Factors associated with adverse drug reaction occurrence and prognosis, and their economic impacts in older inpatients in Taiwan: a nested case–control study. BMJ Open. 2019;9(5):e026771. doi:10.1136/bmjopen-2018-026771
47. Parekh N, Ali K, Stevenson JM, et al. Incidence and cost of medication harm in older adults following hospital discharge: a multicentre prospective study in the UK. Br J Clin Pharmacol. 2018;84(8):1789–1797. doi:10.1111/bcp.13613
48. Najjar M, Abd. Aziz N, Hassan Y, et al. Predictors of polypharmacy and adverse drug reactions among geriatric in patients at Malaysian hospital. Healthmed. 2010;4:273.
49. Harugeri A, Parthasarathi G, Madhan R, Guido S, Basavanagowdappa H. Frequency and nature of adverse drug reactions in elderly in-patients of two Indian medical college hospitals. J Postgrad Med. 2011;57:189–195. doi:10.4103/0022-3859.85201
50. Alhawassi TM, Krass I, Pont LG. Prevalence and risk factors for adverse drug reactions in older adults in the acute care setting. Pharmacoepidemiol Drug Saf. 2015;24(Suppl. 1):91.
51. Sikdar KC, Dowden J, Alaghehbandan R, MacDonald D, Peter P, Gadag V. Adverse drug reactions in elderly hospitalized patients: a 12-year population-based retrospective cohort study. Ann Pharmacother. 2012;46(7–8):960–971. doi:10.1345/aph.1Q529
52. Lavan A, Eustace J, Dahly D, et al. Incident adverse drug reactions in geriatric inpatients: a multicentred observational study. Ther Advan Drug Saf. 2018;9(1):13–23. doi:10.1177/2042098617736191
© 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.