Back to Journals » Infection and Drug Resistance » Volume 14

Nomogram Model for Predicting the Risk of Multidrug-Resistant Bacteria Infection in Diabetic Foot Patients

Authors Ma Y, Zhang L, Hu Y, Shi T

Received 28 October 2020

Accepted for publication 13 January 2021

Published 18 February 2021 Volume 2021:14 Pages 627—637

DOI https://doi.org/10.2147/IDR.S287852

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3

Editor who approved publication: Professor Suresh Antony


Yi-ni Ma,1,* Li-xiang Zhang,2,* Yuan-yuan Hu,3 Tian-lu Shi1

1Department of Pharmacy, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230036, People’s Republic of China; 2Department of Cardiology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230036, People’s Republic of China; 3Department of Endocrinology, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230036, People’s Republic of China

*These authors contributed equally to this work

Correspondence: Tian-lu Shi
Department of Pharmacy, The First Affiliated Hospital of USTC, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, 230036, People’s Republic of China
Tel +86-18900518553
Email ahslyy4224@163.com

Objective: This study established an individualized nomogram for predicting the risk of multidrug-resistant bacterial (MDRB) infection in patients with the diabetic foot (DF), and providing a reference for clinical prevention and treatment.
Methods: A total of 199 DF patients admitted to the hospital from July 2015 to December 2018 were included in this study. The pathogenic bacteria at the site of infection were detected and the factors affecting the occurrence of MDRB infection in DF patients summarized. The R software was used to draw the nomogram, and the Bootstrap Method used to internally verify the model. The calibration curve and the Harrell’s Concordance Index (C-index) were used to evaluate the predictive effect of the nomogram model.
Results: Logistic regression analysis showed that age, course of diabetes, previous use of antibacterial drugs, types of antibacterial drugs, and osteoporosis were risk factors for multidrug-resistant infections in DF (P< 0.05). The area under the receiver operating characteristic curve (AUC, Area Under Curve) of the nomogram model after internal verification was 0.773 (95% CI: 0.704– 0.830). The mean absolute error between the predicted probability of infection in the nomogram and the actual occurrence of MDRB was 0.032, indicating that the nomogram model had good forecasting efficiency and stability.
Conclusion: The risk factors for multidrug-resistant infections in DF are age, course of diabetes, previous use of antibacterial drugs, types of antibacterial drugs used, and osteoporosis. The nomogram model drawn on these risk factors has good predictive accuracy and can assist medical staff in formulating targeted infection prevention strategies for patients.

Keywords: diabetic foot, multidrug-resistant bacteria, infection, risk-factors, nomogram

Introduction

Diabetic foot (DF) is one of the serious complications that cause disability and death in diabetic patients in our country. Studies have shown that about 18% of elderly diabetic patients can develop DF.1 DF includes neurological, ischemic, and mixed lesions, which can manifest as ulcers, infections, and gangrene.2 In all non-traumatic low amputations, diabetic patients accounted for 40% to 60%. Among diabetes-related low distal amputations, 85% occurred after foot ulcers.3 A multi-center survey confirmed that about 40% to 80% of DF ulcers were co-infected.4,5 With the increase of multidrug-resistant bacteria (MDRB) in recent years, it is difficult to treat DF infections. Therefore, clarifying the factors influencing MDRB infection in patients with DF could improve the prognosis of patients and reduce the economic burden.6 This study integrated risk factors through the predictive model confirmed to have good discrimination, calibration, and clinical applicability, with an aim of providing evidence and reference for the clinical prevention and treatment of DF.

Patients and Methods

Patients Characteristics

One hundred and ninety-nine DF patients were from the First Affiliated Hospital of the University of Science and Technology of China sampled from July 2015 to December 2018. The inclusion criteria as follows: (1) Met the 1999 WHO Diabetes Diagnostic Criteria; (2) Met the diagnostic criteria of DF in “Guidelines for the Prevention and Control of Type 2 Diabetes in China (2017 Edition)”7 and “Chinese Guideline on Prevention and Management of DF (2019 Edition)”.8–11 The patients with incomplete clinical data; malnutrition circulatory ulcers of the lower limbs caused by varicose veins; and foot ulcers caused by other diseases were excluded from the study. This research protocol complied with the ethical standards in the Declaration of Helsinki and was approved by the ethics committee of the China Clinical Trial Registration Center. Since this study was a retrospective observational study, the research data were collected and analyzed under anonymity, thus informed consent was exempted.

Study Methods

Clinical data that included gender, age, length of stay in the hospital, course of diabetes, past medical history, diabetes complications and comorbidities, treatment plan, type and number of days of antibacterial drugs, and auxiliary examinations were collected through the electronic medical record of the hospital information system. “Type of antibacterial drugs” in this study included antibiotics and chemically synthesized antimicrobials. The antibiotics mainly include β-lactams antibiotics, aminoglycosides, tetracyclines, chloramphenicol, macrolides, Lincosamides, glycopeptides, antifungal antibiotics, and anti-tumor antibiotics. Chemically synthesized antimicrobials mainly include sulfonamides, quinolones, trimethoprim, nitrofurans, nitroimidazoles, and oxazolones. The sample size calculation of our study was based on our primary objective (binary outcome). In prediction studies, the number of outcome events dictated the effective sample size. Based on some empirical investigations, the sample was defined to have at least 10 outcome events per variable (EPV) in multivariate regression analysis, or more precisely per estimated parameter. Our sample and the number of events exceeded that determined by the EPV approach for determining sample sizes and thus, was expected to provide robust estimates.

Statistical Analysis

The original data were analyzed by statistical software R (version 3.6.1). Normally distributed measurement data were represented by mean ± standard deviation (±s), Categorical data were represented by a number of cases, rate, and percentage, and grade data was represented by a number of cases and percentages. Univariate logistic analysis and multivariate logistic regression analysis were used to screen for the independent influencing factors. Based on the logistic regression analysis results, the “regplot” package was used to draw the nomogram, the “pROC” package was used to draw the ROC curve, and the area under the ROC curve (Area Under Curve, AUC) was used to evaluate the distinction of the nomogram model. The “rms” package was used to draw the calibration curve, and the calibration of the nomogram model was evaluated using the mean absolute error (MAE) of the calibration curve. The “rmda” package was used to draw the Decision Curve Analysis (DCA) to evaluate the clinical applicability of the nomogram model. To prevent overfitting, the nomogram model was verified internally using the Bootstrap Method. P<0.05 was considered statistically significant.

Results

Description of the Microbes Including MDRB

Among the DF patients, 101 (50.75%) were infected with MDRB. A total of 212 strains of pathogenic bacteria were isolated. One hundred and fifteen strains were MDRB with 66 (57.39%) and 49 (42.61%) strains being gram-positive and gram-negative bacteria, respectively. Specific distribution is presented in Table 1.

Table 1 Constituent Ratios of the Isolated MDRB (%)

Patients’ Characteristics

The median age at diagnosis was 62.29 ± 9.16 years, and the male to female ratio was 117:82. Other general information is given in Table 2.

Table 2 Description of Baseline Data of Subjects

Single-Factor Analysis Results

The univariate logistic analysis results indicated that age, diabetes course, use of antibacterial drugs in the past, types of antibacterial drugs used, presence of blood system diseases (mainly including anemia, polycythemia, hematopoietic stem cell diseases, various lymphomas, bleeding and thrombosis Diseases, etc.) and presence of osteoporosis were the six possible risk factors for MDRB infection in patients with DF (P<0.05) (Table 3).

Table 3 Single-Factor Logistic Analysis of Risk Factors for Infection in Patients with DF

Multivariate Regression Analysis Results

The incidence of MDRB infection in DF was used as the dependent variable, and 6 variables with statistical significance in the univariate logistic analysis were used as the independent variables to perform multivariate Logistic regression analysis. The logistic model screened out 5 independent risk factors for the occurrence of infections. Age, course of diabetes, whether antibacterial drugs have been used in the past, types of antibacterial drugs used, and presence of osteoporosis were the 5 independent risk factors for MDRB infection in patients with DF (P< 0.05) (Table 4).

Table 4 Multivariate Logistic Analysis of Risk Factors for Infection in Patients with DF

Establishment of Nomogram

Based on the logistic regression analysis results, the nomogram model was established using the “regplot” package with the 5 independent risk factors as the test variables, and the occurrence of MDRB infection in the patient as the state variable (Figure 1). The specific calculation formula for predicting the risk of infection with DF by nomogram was Logit(P)=−5.395+1.523*(osteoporosis=Yes)+0.051*age+0.930*(Course of disease≥10 years)+0.894*(Previous use of antibacterial drugs=Yes)+1.242*(Types of antibacterial drugs≥2).

Figure 1 The nomogram model for predicting the risk of MDRB infection in patients with DF. **P<0.01.

Abbreviations: MDR, multidrug resistant; MDRB, multidrug-resistant bacteria; DF, diabetic foot.

The score of each variable in the nomogram was obtained by drawing a vertical line up to the score scale line whereas the total score of the patient was obtained by adding the scores of all variables. To get the predicted probability of the patient’s MDRB infection, a vertical line was drawn going down through the bottom total scoreline to the predicted probability line.

For example, if a patient with DF was 55 years old, had used antibacterial drugs in the past, currently using more than two kinds of antibacterial drugs, has a course of diabetes of more than 10 years, and but without osteoporosis, the total score of this patient would be 278 (38+43+60+65+72=278) points, the corresponding MDRB infection risk would be 0.618 (Figure 2).

Figure 2 Application example of a nomogram for predicting the risk of MDRB infection in patients with DF. **P<0.01.

Abbreviations: MDR, multidrug resistant; MDRB, multidrug-resistant bacteria; DF, diabetic foot.

Evaluation of Clinical Applicability of Nomogram

The DCA curve of the nomogram model was drawn through the “rmda” package (Figure 3). When the predicted probability was between 0.1 and 0.95, the nomogram model had good clinical applicability.

Figure 3 DCA curve analysis of nomogram model.

Abbreviations: DCA, decision curve analysis; MDRB, multidrug-resistant bacteria.

Notes: The X-axis measures the predicted probability threshold of the MDRB infection in a patient by the nomogram model, and Y-axis measures Standardized net benefit. Nomogram showed the standardized highest net benefit and clinical usefulness in the DCA curve.

Verification of Nomogram

The ROC curve of the nomogram was drawn by the “pROC” package. The ROC curve showed that the AUC of the nomogram was 0.766 (95% CI: 0.701–0.831) and the AUC of the nomogram was better than the AUC of any single risk, as shown in Figure 4. The bootstrap method (re-sampling times=500) was used to internally verify the nomogram to prevent overfitting. After internal verification, the AUC of the nomogram model was 0.773 (95% CI: 0.704–0.830), indicating that the model had good and relatively stable discrimination (Figure 5). After internal verification, the “rms” program package was used to draw the calibration curve. The mean absolute error between the prediction probability of the nomogram and the actual risk of MDRB infection was 0.032, revealing a good prediction consistency and calibration (Figure 6).

Figure 4 ROC curve analysis of different indicators.

Abbreviations: ROC, receiver operating characteristic; AUC, area under curve; MDRB, multidrug-resistant bacteria.

Notes: The Y-axis indicates the sensitivity of the nomogram while the X-axis indicates the specificity of the nomogram. The AUC of the nomogram in predicting the probability of the MDRB infection in a patient was higher than any single indicator.

Figure 5 ROC curve analysis of the nomogram model after internal verification.

Abbreviations: ROC, receiver operating characteristic; AUC, area under curve.

Note: Y-axis indicates the sensitivity of the nomogram while the X-axis indicates the specificity of the nomogram.

Figure 6 Calibration curve analysis of nomogram model after internal verification.

Abbreviation: MDRB, multidrug-resistant bacteria.

Notes: The Y-axis indicates the actual probability of the patient’s MDRB infection and the X-axis indicates the predicted probability of the patient’s MDRB infection. The 45-degree black dotted line represents the ideal prediction; the solid black line surrounding the 45-degree black dotted line represents the bias-corrected prediction; the black dotted line surrounding the 45-degree black dotted line represents the apparent prediction.

Discussion

Clinical studies have shown that the concurrent infection of DF is a significant cause of amputation,12 and that the prognosis of patients with MDRB infection was worse.13 Among the 199 patients in this study, 101 were infected with MDRB. The MDRB infection rate was 50.75%, which was higher than the results of related foreign studies.14,15 Among all chronic complications of diabetes, DF disease is relatively easy to identify and its complications can be effectively prevented. Domestic and foreign studies have pointed out16,17 that the key to preventing DF disease is to regularly check whether patients have risk factors for DF disease, identify these risk factors as soon as possible, and remove or correct the factors that easily cause ulcers. Logistic regression analysis showed that age, diabetes course, previous use of antibacterial drugs, types of antibacterial drugs used, and osteoporosis were the risk factors for MDRB infection of DF (P<0.05). Similar findings were reported in the literature.12

Patients with DF infections do not effectively respond to antimicrobial treatment or they frequently replace antimicrobial drugs. This results in an increased probability of drug-resistant bacteria and causes multidrug resistance. Therefore, before choosing the effective antibiotics to control infection, bacterial culture, and drug sensitivity tests on ulcer wounds should be carried out. The decision to replace antibiotics should be made based on the effect of clinical treatment. If the treatment effect is obvious, the patient should continue using the antibiotic even if the drug susceptibility test shows that the bacteria is resistant to the antibiotic. If the treatment effect is not obvious or ineffective, the antibiotics should be replaced based on the results of drug susceptibility tests, and the type and course of antibiotics used strictly controlled.

Studies have shown that compared with non-diabetic patients, diabetic patients, especially the elderly, were twice as likely to develop the vascular disease in the lower extremities.18 The average age of the patients in this study was (62.29 ± 9.16) years old, and the average glycosylated hemoglobin was (9.74 ± 2.62) %. As the course of diabetes prolonged, the proportion of chronic complications of diabetes increased leading to the occurrence of DF. Therefore, blood sugar control should be routinely carried out in patients with DF,19 and the occurrence of hypoglycemia minimized to reduce the incidence of foot ulcers and infections. This could help in reducing the rate of amputation and medical expenses.20

Even though the baseline level of the number of male and female cases was biased, we found that the cases and risk of infection in female patients with DF was lower than that in male patients (Table 3). However, the difference was statistically insignificant and could be due to the small sample size of our study. Further large sample-based studies are needed to establish the effect of gender on infection in patients with DF.

As a chart tool, the nomogram can visually display the statistical model and obtain the numerical probability of the target event, which is more accurate in quantifying the risk.21 Studies have reported21–25 that the nomogram model could effectively predict the incidence of nosocomial infection, the postoperative survival rate of patients with non-small cell lung cancer, and the risk of postoperative delirium in elderly patients with hip fractures. Nonetheless, no prediction model of diabetes report of MDRB infection among foot patients exists in China. The nomogram model constructed in this study included five indicators: age, diabetes course, past application of antibacterial drugs, types of antibacterial drugs, and presence of osteoporosis. These five indicators can be quickly collected during the patient’s hospitalization to facilitate early diagnosis and risk assessment. The internally verified AUC of the nomogram model was 0.773 (95% CI: 0.704–0.830), and the mean absolute error between the predicted probability and the actual risk of MDRB infection was 0.032, showing a good prediction accuracy.

We used the DCA curve to evaluate the clinical applicability of the nomogram model. The DCA curve estimates the standardized net benefit by the probability threshold and is used to categorize observations as “high risk.” The standardized net benefit means the benefit of the intervention to the patient minus the harm of the non-patient treatment and the loss of the patient’s untreated condition. In Figure 3, X-axis measures the predicted probability threshold of the patient’s MDRB infection by the nomogram model, Y-axis measures standardized net benefit. The black horizontal straight line and the gray slash line were the corresponding standardized net benefit when the intervention was all or none. The nomogram model is only valuable if the DCA curve is plotted above both lines. The standardized net benefit of the nomogram was highest when the predicted probability was between 0.1 and 0.95, which proved that the nomogram model had good clinical applicability.

Limitations

Despite the significant findings of our study, there were some limitations. First, the samples used in this study were from a tertiary hospital, and the sample size was small. Our results were from a cohort and a single-center retrospective study, and could not fully represent all Chinese patients with DF. Thus, a large sample-based study is needed to externally verify the specific clinical predictive value of the nomogram. We plan to carry out multi-center, large-scale, prospective cohort studies to explore the application significance of this nomogram for predicting the risk of infection in DF patients. Second, limited risk factors were included in the construction of prediction models in this study. This limited the clinical application value of the nomogram model to some extent. Antibiotics are only a part of the management and treatment of the DF. We neglected some other important factors in the process of data collection and analysis such as surgical debridement, local hemostasis, negative pressure wound treatment, wound size and severity, osteomyelitis, and healing outcome. All of these factors need to be considered in the prediction and treatment of infection in patients with DF. We could also compare the healing outcomes of the MDRB and non-MDRB groups as well as any other association with different treatment strategies to facilitate the selection of the most effective treatment for the patient. Third, although the robustness of our nomogram was examined with internal validation using bootstrap testing, external validation could not be conducted and the generalizability is uncertain for other populations in other regions and countries. As a result, this monogram needs to be externally evaluated in wider populations. Fourth, DF infections often involve multiple bacterial species with biofilm structures and are protected by a self-produced extracellular polymer matrix. In biofilm mode, even non-MDRB become much more resistant to antibiotics. Therefore, therapeutic interventions for DF patients with non-MDRB infections also require further consideration.

Conclusion

The risk of amputation in patients with DF combined with MDRB infection is much higher than in non-infected patients with DF. Moreover, patients with diabetes are likely to be invaded by pathogenic bacteria in the case of neurological ischemia; thus, DF can induce bacterial infection and becomes difficult to treat. The emergence of MDRB has resulted in the inevitable combination of drugs, making it difficult to control the infection. Therefore, the prevention and control of infection have become the main focus of DF research. The application of nomograms in clinical work can accurately quantify and individually predict the risk of infection by MDRB in DF patients, thereby enabling the screening of the patients with a high risk of infection as soon as possible. With an estimated individual risk, clinicians and patients can take more necessary measures on life-style monitoring and medical interventions.

In this study, DF patients have a higher risk of MDRB infection. Age, diabetes course, previous use of antibacterial drugs, types of antibacterial drugs used, and the presence of osteoporosis are the risk factors for MDRB infection of DF. Our nomogram prediction model has good clinical applicability, and a good prediction accuracy thus can be used in formulating effective prevention and treatment measures, reasonably using antibacterial drugs, preventing ischemic ulcer and gangrene, reducing the mortality of patients with diabetes, and improving the prognosis of patients.

Ethics Approval

The protocol of this study was approved by the Chinese Ethics Committee of Registering Clinical Trial, and informed consent was waived because of the retrospective and observational nature of the study. This study was conducted in accordance with the standards set by the “Helsinki Declaration” and the patient information was anonymized and de-identified prior to analysis.

The reason why ethical approval of this research protocol was not obtained from our own institution’s ethics committee was that after consulting the members of the Ethics Committee of our hospital, we learned that they needed a long time of about six to eight months to review the ethics of the observational research protocol. Considering the timeliness of carrying out this research, we conducted an ethical review through the ethics committee of the China Clinical Trial Registration Center, which took one month. In the process of ethical review, due to the retrospective and observational nature of this research, the informed consent of the patients was exempted. The China Clinical Trial Registration Center registers clinical research all over the country, and the ethics committee of this institution will also conduct ethical review and approval for some time-sensitive research to ensure that the research can be published in time. Studies approved by the ethics committee of the China Clinical Trial Registration Center have been accepted by some journals. For specific cases, please refer to “https://doi.org/10.1159/000494443”, and “https://doi.org/10.1186/s12889-020-8323-7”.

Funding

This study was not supported by any funding.

Disclosure

The authors declare no conflicts of interest.

References

1. Bondor CI, Veresiu IA, Florea B, et al. Epidemiology of diabetic foot ulcers and amputations in Romania: results of a cross-sectional quality of life questionnaire based survey. J Diabetes Res. 2016;21(12):543–549.

2. Kim TG, Moon SY, Park MS, et al. Factors affecting length of hospital stay and mortality in infected diabetic foot ulcers undergoing surgical drainage without major amputation. J Korean Med Sci. 2016;31(1):120–124. doi:10.3346/jkms.2016.31.1.120

3. Diabetic foot Branch of China International Exchange and Promotion Association for Medical and HealthCare. Chinese guideline on diagnosis and management of diabetic foot. Chin Med J. 2017;97(4):251–258.

4. Singh N, Armstrong DG, Lipsky BA. Preventing foot ulcers in patients with diabetes. JAMA. 2005;293:217–228. doi:10.1001/jama.293.2.217

5. Lipsky BA, Berendt AR, Cornia PB, et al. 2012 Infectious Diseases Society of America clinical practice guideline for the diagnosis and treatment of diabetic foot infections. Clin Infect Dis. 2012;54(12):e132–e173.

6. Agbi KE, Carvalho M, Phan H, et al. Case report: diabetic foot ulcer infection treated with topical compounded medications. Int J Pharm Compd. 2017;21(1):22–27.

7. Chinese Diabetes Society. Guidelines for the prevention and control of type 2 diabetes in China (2017 Edition). Chin J Diabetes. 2018;10(1):4–67.

8. Chinese Diabetes Society, Chinese Society of Infectious Diseases, Chinese Society for Tissue Repair and Regeneration. Chinese guideline on prevention and management of diabetic foot (2019 edition)(I). Chin J Diabetes. 2019;11(2):92–108.

9. Chinese Diabetes Society, Chinese Society of Infectious Diseases, Chinese Society for Tissue Repair and Regeneration. Chinese guideline on prevention and management of diabetic foot (2019 edition)(II). Chin J Diabetes. 2019;11(3):161–189.

10. Chinese Diabetes Society, Chinese Society of Infectious Diseases, Chinese Society for Tissue Repair and Regeneration. Chinese guideline on prevention and management of diabetic foot (2019 edition)(III). Chin J Diabetes. 2019;11(4):238–247.

11. Chinese Diabetes Society, Chinese Society of Infectious Diseases, Chinese Society for Tissue Repair and Regeneration. Chinese guideline on prevention and management of diabetic foot (2019 edition)(IV). Chin J Diabetes. 2019;11(5):316–327.

12. Jiang Y, Wang X, Xia L, et al. A cohort study of diabetic patients and diabetic foot ulceration patients in China. Wound Repair Regen. 2015;23(2):222–230. doi:10.1111/wrr.12263

13. Ambrose T, Sharkey LM, Louis-Auguste J, et al. Cytomegalovirus infection and rates of antiviral resistance following intestinal and multivisceral transplantation. Transplant Proc. 2016;48(2):492–496. doi:10.1016/j.transproceed.2015.09.070

14. Semedo-Lemsaddek T, Mottola C, Alves-Barroco C, et al. Characterization of multidrug-resistant diabetic foot ulcer Enterococci. Enferm Infecc Microbiol Clin. 2016;34(2):114–116. doi:10.1016/j.eimc.2015.01.007

15. Hardjo Lugito NP, Nawangsih C, Moksidy JC, et al. Diabetic foot gangrene patient with multi-drug resistant Pseudomonas putida infection in Karawaci District, Indonesia. J Glob Infect Dis. 2015;7(1):37–39. doi:10.4103/0974-777X.146378

16. McInnes A, Jeffcoate W, Vileikyte L, et al. Foot care education in patients with diabetes at low risk of complications: a consensus statement. Diabet Med. 2011;28(2):162–167. doi:10.1111/j.1464-5491.2010.03206.x

17. Netten JJ, Price PE, Lavery LA, et al. Prevention of foot ulcers in the at-risk patient with diabetes: a systematic review. Diabetes Metab Res Rev. 2016;32(Suppl 1):84–98. doi:10.1002/dmrr.2701

18. Guan H, Li YJ, Xu ZR, et al. Prevalence and risk factors of peripheral arterial disease in diabetic patients over 50 years old in China. Chin Med Sci J. 2007;22(2):83–88.

19. Jiang Y, Ran X, Jia L, et al. Epidemiology of type 2 diabetic foot problems and predictive factors for amputation in China. Int J Low Extrem Wounds. 2015;14(1):19–27. doi:10.1177/1534734614564867

20. Hingorani A, LaMuraglia GM, Henke P, et al. The management of diabetic foot: aclinical practice guideline by the Society for Vascular Surgery in collaboration with the American Podiatric Medical Association and the Society for Vascular Medicine. J Vasc Surg. 2016;63(2Suppl):3S–21S. doi:10.1016/j.jvs.2015.10.003

21. Marcelo GF, Emerson PG, Augusto AM, et al. Independent validation of a predictive nomogram for risk of reinfection in women with recurrent non-complicated urinary tract infections. Ther Adv Urol. 2020;12:1–9.

22. Christian G, David HJ, Christian D, et al. Development and external validation of nomograms in oropharyngeal cancer patients with known HPV-DNA status: a European Multicentre Study (OroGrams). Br J Cancer. 2018;118(12):1672–1681. doi:10.1038/s41416-018-0107-9

23. Monteiro-Soares M, Dinis-Ribeiro M. A new diabetic foot risk assessment tool: DIAFORA. Diabetes Metab Res Rev. 2016;32(4):429–435. doi:10.1002/dmrr.2785

24. Yang Y, Zhao X, Dong T, et al. Risk factors for postoperative delirium following hip fracture repair in elderly patients: a systematic review and meta-analysis. Aging Clin Exp Res. 2017;29(2):115–126. doi:10.1007/s40520-016-0541-6

25. Liang W, Zhang L, Jiang G, et al. Development and validation of a nomogram for predicting survival in patients with resected non-small-cell lung cancer. J Clin Oncol. 2015;33(8):861–869. doi:10.1200/JCO.2014.56.6661

Creative Commons License This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.

Download Article [PDF]