Nomogram and Web Calculator Based on Lasso-Logistic Regression for Predicting Persistent Organ Failure in Acute Pancreatitis Patients

Introduction

Acute Pancreatitis (AP) is a commonly encountered gastrointestinal emergency in the clinical setting. It refers to an acute abdominal condition caused by the abnormal activation of pancreatic enzymes, leading to self-digestion of the pancreas. Its primary feature is a localized inflammatory response within the pancreas. In severe cases, it can result in the dysfunction of other organs. Its hallmark clinical symptom is a sudden onset of persistent upper abdominal pain, which may radiate to the back.¹ With the rise in living standards in recent years, the incidence of AP has been steadily increasing. Globally, the incidence of AP ranges from 4.9 to 73.4 per 100,000,¹ and it’s rising at an annual rate of 3.07%.² AP is progressively becoming a significant disease threatening human health and deserves attention. Studies have shown that approximately 20% of AP patients can develop Organ Failure (OF), with respiratory, renal, and cardiac dysfunctions being most common.³ OF is a critical determinant of the prognosis of AP and, to a large extent, dictates the outcome for AP patients. OF lasting ≤48 hours typically carries a lower risk of complications and mortality. However, patients with Persistent Organ Failure (POF) (lasting >48 hours) face a mortality rate as high as 50%.⁴ Therefore, early prediction and assessment of the potential development of POF in AP patients and providing timely treatment are crucial in reducing mortality from AP.

In 1974, Ranson proposed the Ranson scoring system after studying 100 patients with AP who were admitted to hospital for 48 hours.⁵ The scoring system included 5 clinical indicators on admission and 1 point for each of the 6 indicators for 48 hours, totaling 11 points, and the score of 3 and above was considered as severe pancreatitis, and the morbidity and mortality rate of less than 3 points was 0.9%, 3–4 points was 16%, 5–6 points was 40%, and 6 points or more points is 100%, and its scoring system is considered a milestone in the estimation of the severity of AP. However, this scoring system is limited in its clinical application for predicting POF, with non-ICU patients not routinely collecting all data points at the time of data collection and calculating a score that takes at least 48h with an accuracy of approximately 75%.⁶ The APACHE II score is a patient prognostic prediction score proposed by Knaus et al.⁷ This score is not an exclusive scoring system for AP and it can be applied to almost all ICU patients, where a score of ≥8 on the APACHE II scale indicates the possible presence of severe pancreatitis. The APACHE score is highly sensitive, but many of its parameters require the ICU setting to be measured and the scoring system is too examinative, cumbersome, and expensive, and is also time-delayed, with deficiencies in the prediction of POF.⁸ The CTSI is also a useful scoring system for accurately diagnosing the severity pancreatitis. However, CT may underestimate or misclassify the severity of the disease if performed within 72 hours of symptom onset.⁹ All of these scores are either delayed for POF or lack sufficient research evidence and are inadequate. A recent systematic review and meta-analysis concluded that current early predictors of POF, infected pancreatic necrosis and mortality are not sufficiently accurate for individualized patient prediction, and that ideal predictors should be applied to patients on admission or within 24 hours of symptom onset, with an accuracy of between 95% - 100%.¹⁰

In recent years, there has been a keen interest in identifying predictive factors for diseases to recognize clinical indicators that threaten human health as early as possible. Statisticians are focused on developing and refining modern statistical methods to ensure the selection of rigorous methods for estimating the impact of predictive factors on outcomes. However, most variable selection processes rely on more traditional statistical methods, such as univariate and multivariate regression analyses. These methods sometimes yield contradictory hazard ratios between univariate and multivariate COX regressions. This contradiction arises from multicollinearity among variables, leading to biased results.¹¹ In 1996, Robert Tibshirani introduced a novel method for estimating linear models, known as the Least Absolute Shrinkage and Selection Operator (LASSO).¹² This approach accommodates a plethora of covariates in the model and addresses overfitting by constructing a penalty function.¹³ It also resolves issues of multicollinearity, yielding more relevant predictive factors, thus compensating for the shortcomings of traditional methods.¹⁴ However, the application of LASSO regression in the realm of AP remains limited. In this paper, we employ LASSO regression to select predictive factors and integrate it with multivariate logistic regression to mitigate the interference of confounding factors, ensuring the acquisition of precise predictive variables. The results are visualized in the form of a nomogram.

The aim of this study is to establish a prediction model based on LASSO-logistic regression for predicting POF in AP patients. This will enable early identification of high-risk individuals for POF using patient indicators within the first 24 hours of admission, guiding clinicians to tailor timely and individualized therapeutic measures for high-risk patients, thereby enhancing their prognosis.

Materials and Methods

Patients

We collected patients with a first diagnosis of AP from January 1, 2017, to January 1, 2022, at Shanxi Bethune Hospital. Patients from 1/1/2017 to 1/1/2021 were used to build the model and patients from 1/1/2021 to 1/1/2022 were used to validate the model.

The diagnostic criteria for AP include the followings: ① continuous upper abdominal pain; ② serum amylase and/or lipase levels more than 3 times the upper limit of normal; ③ abdominal imaging findings consistent with acute pancreatitis. A diagnosis of AP can be made if any two of these three criteria are met.¹⁵

The diagnostic criteria for OF are based on the modified Marshall scoring system,¹⁶ where a score of ≥2 for any organ indicates the presence of OF. OF that persists beyond 48 hours is defined as POF.

Inclusion and Exclusion Criteria

Inclusion criteria: ① meeting the diagnostic criteria for AP; ② aged ≥18 years; ③ admitted within 48 hours of symptom onset; ④ hospitalized at Shanxi Bethune Hospital and initially diagnosed with AP.

Exclusion criteria: ① incomplete clinical data or missing medical records; ② chronic pancreatitis, trauma, or pregnancy-associated pancreatitis; ③ patients with diagnosed chronic diseases of the heart, lungs, kidneys, etc.; ④ patients with prior immunological disorders or hematological diseases.

Data Collection

General Information includes: sex, age, height (cm), weight (kg), body mass index (BMI), etiology, triggering factors, smoking history, medical history, and presence or absence of POF. Vital signs upon admission, as well as laboratory indices within 24 hours of admission (covering liver and kidney function tests, pancreatic function, blood cell analysis, serum electrolytes, coagulation function) are also recorded. Data were initially selected for 100 variables, and after inclusion and exclusion, 46 variables were used for further analysis.

Data Processing and Statistical Analysis

Data cleaning was conducted using STATA (Version 17). Variables with more than 20% missing values were excluded. Variables with missing values between 5% and 20% were imputed using multiple imputations to select the optimal dataset to fill in missing values. Variables with less than 5% missing values were replaced by their mean. Outliers were treated using the winsorize method.¹⁷

In the clinic, AP patients with POF represent a small percentage of all AP patients, which leads to unbalanced data. In order to solve this problem, simple random sampling was applied and this study was conducted according to 1:3. Simple random sampling using SPSS (Version 26.0). Bar charts are drawn using GraphPad prism 9.

Predictive model was conducted using R (Version 4.2.0). Baseline data analysis was performed using the “compare groups” package. A complete predictive model requires three steps: variable selection, model establishment, and model evaluation.①Variable selection: “glmnet” package for LASSO regression to screen predictive variables; ② Model establishment: To further control confounding factors, the results of the LASSO regression were subjected to multivariate logistic regression analysis using the “glm” package; ③ Model evaluation: The “pROC” package was used to draw ROC curves to evaluate the model’s ability to distinguish different outcome events, the “rms” package for calibration plots to assess the discrepancy between predicted and actual outcomes, and the “rmda” package for DCA to evaluate the model’s clinical utility. P-value < 0.05 was considered statistically significant. Web calculator using “DynNom” package.

Results

Characteristics of Patients

Between January 1, 2017, and January 1, 2022, a total of 1489 patients were diagnosed with AP at Shanxi Bethune Hospital. Out of these, 1166 patients met the inclusion criteria (Figure 1). A total of 933 patients were enrolled from January 1, 2017 to January 1, 2021, according to the modified Marshall scoring criteria, 145 patients had concomitant POF, and 788 patients had no POF. We sampled 435 patients without POF in a ratio of 1:3, which was used as a modeling group together with the patients with POF.

Figure 1 Flowchart of patient enrollment.

Table 1 provides a summary of the baseline characteristics of the AP patients. Among them, 377 were males and 208 were females, with an average age of 48 years. The mean age of patients with POF was 56 years, which was significantly higher than that of the group without POF.

Table 1 Differences Between the Two Groups in AP Patients with or Without POF

Bar charts of outcome events in the two groups were plotted to understand whether POF predicted patients’ poor outcomes. As shown in the Figure 2, compared with patients without POF, patients with POF stayed in the hospital longer, had higher rates of bacterial infection, developed walled-off necrosis in pancreas, accumulated pancreatic fluid collections, were more likely to be admitted to the ICU, and were even more likely to die in the hospital. Thus, screen high-risk groups and give timely intervention is essential. Establishing a predictive model for early diagnosis of POF in AP patients addresses this need.

Figure 2 Comparison of clinical outcomes based on whether there are complicated with POF. Length of stay. For non-normal distributed data, Mann–Whitney test was used. P<0.001. ****P<0.001. (B) ICU stay. (C) Bacterial Infection. (D) Death. (E) Walled-off necrosis. (F) Pancreatic fluid collections. (B and D) Fisher’s exact test. P<0.001. (C, E and F) Chi-square test. P <0.001.

Building Predictive Models

The model was established using training set, we used univariate logistic regression and LASSO regression to select predictors. Then, independent predictors were screened by excluding confounding factors using multifactor logistic regression.

Supplementary Table 1 describes the results of univariate and multivariate logistic regression screening for predictors in the training set. Ultimately, seven variables were selected, which were age, heart rate, direct bilirubin, serum creatinine, serum calcium, lymphocytes count and d-dimer.

The results of the LASSO regression are shown in Figure 3, when the model error is the smallest, 14 independent variables are screened out, and when the model error is one standard error, 8 independent variables are selected. In order to be more convenient for clinical application, we select the independent variables screened out when log(λ) is one standard error, namely Age, Heart Rate, Albumin, Urea, Serum Creatinine, Ca (Calcium), Lymphocytes and D-dimer. To further eliminate the effects of confounding factors, the factors screened out by LASSO regression were incorporated into a multivariate logistic regression. This resulted in the identification of five independent predictive factors. The results of multivariate logistic regression are shown in Table 2, Age, Heart Rate, Serum Creatinine, Ca (Calcium), Lymphocytes and D-dimer are independent predictors of POF.

Table 2 Multivariate Logistic Regression Results

Figure 3 Selection of predictive variables using LASSO regression. (A) Characteristics of variable coefficient changes. The horizontal coordinates represent the value of the parameter log(λ), ordinate represents the coefficient of the independent variable. Finally, the coefficients of all independent variables are compressed to 0, and the later the independent variable becomes 0, the greater the contribution to the model. The dashed line on the left indicates the value of the parameter log(λ) when the model error is minimal, the dashed line on the right represents the value of the parameter log(λ) when the model error is amplified by one standard error. (B) Ten-fold cross-validation for adjusting parameters. The horizontal coordinates represent the value of the parameter log(λ), ordinate represents the mean squared error of the model.

Model Visualization

To provide a more intuitive display of the model and to facilitate its clinical application, we constructed nomograms and calculated scores for each observational indicator. Figure 4 illustrates the nomogram that predicts whether or not a patient with AP will develop POF. Figure 4A shows the nomogram created by univariate logistic regression and Figure 4B shows the nomogram created by LASSO regression. Each index on the graph corresponds to its test result, and one can determine the corresponding prediction score (as denoted by the scale at the top) based on the test result. By summing up the scores for each index, the predicted total score can be determined, and the value corresponding to this total score provides the probability of POF.

Figure 4 Nomogram for predicting whether patients with AP will develop POF. The nomogram created by univariate logistic regression (A). The nomogram created by LASSO (B).

Evaluation of Model Predictive Performance

The validation dataset was utilized to access the prediction performance of the model. As depicted in Figure 5A and B (in Figure 5, Mod A is the nomogram built by univariate logistic regression and model B is the nomogram built by LASSO), the ROC curves for both the training and validation cohorts demonstrate the model’s capacity to differentiate outcome events: the AUC were 0.867 and 0.864 for the training cohort, 0.838 and 0.835 for the validation group. Further supporting the model’s efficacy, Figure 5C and D show that the predicted outcome events closely align with the actual ones.

Figure 5 Nomogram performance. ROC curves of the model for predicting POF probabilities in the training cohort (A) and validation cohort (B). Calibration plots for predicting POF probabilities in the training cohort (C) and validation cohort (D). The gray slash represents the ideal reference line, and the black curve represents the predictive performance of the model. The closer the black curve is to the gray slash, the closer the predicted outcome event is to the actual event. Decision curves for the training cohort (E) and validation cohort (F). The abscissa is the threshold probability, and the ordinate is the net benefit rate, when the black dashed line is above the two solid lines, it indicates that the model provides a net benefit.

To further elucidate the model’s clinical utility, we plotted the Decision Curve Analysis (DCA) in Figure 5E and F. This analysis reveals that our model provides significant net clinical benefits. When compared to other metrics, our nomogram consistently delivers greater net benefits.

For a comprehensive validation, we internally evaluated our model on three fronts: discriminative ability, calibration, and clinical utility. The outcomes confirm that our model is proficient at predicting whether AP patients will develop POF and underscores its positive implications for clinical application.

By comparison, the predictive value of the predictive models created by the two statistical methods was similar. The model of LASSO regression was more streamlined and more conducive to rapid clinical triage.

Web Calculator

While the nomogram is convenient and low cost, it cannot provide exact values in its calculation. Therefore, we developed a web calculator based on nomogram to simplify the calculation process and yield more precise prediction values (https://acute-pancreatitis-with-pof.shinyapps.io/Gx-AP-POF-Dymnomo/). Drag the numerical strip on the left to input the test result, and then click “Predict” to obtain the precise predicted value (Figure 6).

Figure 6 Dynamic web calculator to predict whether patients with AP will develop POF. The left band inputs variable values, Graphical Summary shows the probability and confidence interval of POF occurrence in the form of pictures.

Discussion

POF is a complex physiological process, influenced by various factors. Early identification of risk factors associated with POF in patients is vital for stratified management of AP. In this study, 46 clinical data of the patients were collected, with the laboratory test indicators being comprehensive. Importantly, these indicators were obtained from the initial examination of patients within the first 24 hours after admission, minimizing the interference from subsequent clinical treatments. From the statistical point of view, we established 2 kinds of prediction models, the two models differ by one indicator, but the predictive value is similar, the statistical model established by LASSO regression seems to be more streamlined, in addition, LASSO regression can effectively solve the problem of data multicollinearity. Furthermore, we present two visualization models for clinicians to choose from: the Nomogram and a web calculator. While the Nomogram offers simplicity and clarity, the web calculator boasts convenience and accuracy. Upon validation, our model demonstrated great predictive capabilities, holding potential benefits for clinical application.

This research indicates that an increase in age is one of the predictor for POF. This could be attributed to the relative decline in physiological functions in the elderly, rendering them less resilient to diseases. Previous studies have demonstrated that age is not only closely associated with the incidence of AP but also serves as an independent risk factor for AP mortality.^18,19

The present study considered vital signs of patients upon admission and identified heart rate as an independent predictor of POF in AP patients, which is consistent with the findings of Pan et al.²⁰ Rapid heart rate is primarily regulated by the sympathetic nerve.²¹ Some studies suggest that the inflammatory response and lung injury in rats with AP are significantly reduced when the drugs inhibiting the sympathetic nerve are administered.²² This offers a novel perspective for the clinical treatment of AP.

Serum creatinine is a well-recognized indicator of renal function and is incorporated into several scoring systems, such as the modified Marshall score and the APACHE II score. Our study suggests that serum creatinine can independently predict the onset of POF, aligning with the findings of many other studies.^23,24 Inflammatory events in the body lead to systemic fluid redistribution allowing a surge of toxins, cytokines, and inflammatory mediators into the circulation, which subsequently causes renal injury and an associated rise in creatinine.²⁵ Studies indicate that acute kidney injury often follows organ failure in AP patients, and the mortality rate for patients with acute kidney injury combined with AP is over 25%.²⁶

The progression of pancreatitis requires the activation of pancreatic enzymes by calcium ions, which constantly initiates their self-digestion.²⁷ On the other hand, exudation of pancreatic fluid degrades pancreatic adipose tissue, and free fatty acids combine with calcium ions to form calcium saponification, leading to a decrease in blood calcium level.²⁸ In this study, we observed that the lower the blood calcium level, the higher the probability of POF. Serum calcium is a pivotal predictor for POF in AP patients.

D-dimer is commonly used to assess thrombosis and is also a risk factor for predicting both AP severity and thrombosis.^29–31 Coagulation and inflammation are known to influence each other reciprocally: the onset of inflammation can activate coagulation, and in turn, coagulation can stimulate inflammation.³² An increasing body of studies confirms the correlation between D-dimer levels at the time of admission and AP severity. For instance, in a study by Zhang et al, it was observed that patients with elevated D-dimer levels were more likely to experience severe AP.^33,34

AP is an inflammatory disease. There is an important relationship between the role of inflammatory cells and the severity of AP. The activation of T lymphocytes and B lymphocytes is significantly related to the pathogenesis of AP, and may be an important factor leading to the progression of AP.³⁵ Our study is consistent with existing findings that lymphocytes can independently predict whether or not AP patients develop POF. Nowadays, researchers are no longer limited to the analysis of the cellular level, and the predictive value of cytokines is widely used. Some studies have shown that CD4+T lymphocytes and CD19+B lymphocytes can predict the occurrence of organ failure.³⁶

Most of the existing AP prediction models are based on univariate logistic regression, and a few researchers have applied machine learning to predict the severity of AP. Some researchers developed an EASY-APP based on machine learning to predict the severity of AP. The study used patients from multiple countries to build a prediction model. Respiratory rate, abdominal guarding, axillary body temperature, serum amylase, gender and serum glucose level are more related to the severity of AP. However, the data were highly imbalanced, with only 70 of 1184 patients enrolled having severe pancreatitis (AUC score was 0.803 ± 0.010).³⁷ The study by Mikolasevic et al pointed out that the presence of nonalcoholic fatty liver disease was associated with severe acute pancreatitis, but this result was not obtained in our study.³⁸ Hong et al studied 700 patients with acute pancreatitis, biliary etiology, chronic concomitant diseases, hematocrit, blood urea nitrogen, and the serum albumin level were independently associated with POF.³⁹

In the prediction model established by univariate logistic regression, direct bilirubin was the pair predictor of POF in AP patients, which was different from the model established by LASSO regression. According to previous statistics, nearly half of AP patients were diagnosed with biliary pancreatitis or cholecystitis and gallstones, although we have tried to adjust the imbalance of the data.

In this study, the six selected predictors were combined to create both visual Nomogram and an online calculator. These tools are designed to be more conducive to clinical application and to provide valuable guidance for clinical decision-making. However, there are some limitations to consider. This study originates from a single center and employs a retrospective approach, which may introduce biases. In addition, of the total sample, only 145 AP patients exhibited complications with POF, representing just 25% of all patients. This distribution may reduce our models sensitivity. In the future, validation from multiple centers with larger datasets will be essential to corroborate our findings.

Conclusion

Heart Rate, Serum Creatinine, Ca (Calcium), Lymphocytes and D-dimer are independent predictors of POFwere independently associated with persistent organ failure in acute pancreatitis. Using 6 clinical risk indicators, we developed nomogram and web calculator.