Development and Internal Validation of a Multivariable Prehospital Prediction Score for Critical Illness Progression and 30-Day Mortality in Non-Traumatic EMS Patients

Introduction

Several early warning scoring (EWS) systems have been developed to detect clinical deterioration and predict adverse outcomes in hospitalized and emergency department patients, including the National Early Warning Score (NEWS), Modified Early Warning Score (MEWS), Worthing Physiological Scoring (WPS), and Rapid Emergency Medicine Score (REMS).^1–5 These tools primarily rely on physiological parameters to identify patients at risk of illness progression, intensive care unit (ICU) admission, or mortality.^6–9

However, most existing scoring systems were designed and validated in in-hospital or emergency department settings, where patient monitoring, diagnostic resources, and physician availability differ substantially from those in prehospital environments. In many EMS systems worldwide, particularly in low- and middle-income countries, prehospital emergency care is commonly delivered by paramedics and emergency nurse practitioners (ENPs) rather than emergency physicians. In contrast, some high-income countries utilize physician-led EMS systems with advanced prehospital interventions and broader diagnostic capabilities. EMS providers are therefore required to make rapid triage and transport decisions under time constraints and with varying levels of personnel, medical equipment, transport duration, and digital infrastructure depending on the operational setting.

Although several early warning systems have demonstrated acceptable predictive performance in hospital-based studies, their applicability in emergency medical service (EMS) systems remains limited, particularly for non-traumatic patients managed before hospital arrival.^10,11 Some existing tools may be operationally complex, require multiple physiological variables, repeated assessments, or computational support that may not be consistently available during field operations. In addition, implementation of prehospital decision-support systems may be influenced by training requirements, operational workload, and limitations in electronic data systems, particularly in resource-constrained EMS environments. Furthermore, differences in EMS structures, staffing models, transport times, and resource availability across healthcare systems may limit the generalisability and real-time applicability of existing prediction models in prehospital settings. These limitations highlight the need for a simple and operationally feasible multivariable prehospital prediction score that can support rapid risk stratification and decision-making across diverse EMS environments.

Beyond individual patient assessment, prehospital triage represents a critical leadership function within EMS systems. Effective EMS leadership requires rapid decision-making, prioritization of limited resources, and coordination across healthcare networks. Inaccurate or inconsistent triage decisions may lead to inappropriate destination selection, inefficient resource utilization, and delays in definitive care, ultimately affecting patient outcomes and system performance. Therefore, decision-support tools that enhance leadership capacity in real-time, high-pressure environments are essential. In addition to supporting operational efficiency, early identification of high-risk patients may improve patient-centered outcomes through earlier escalation of care, timely referral to hospitals with critical care capability, and more appropriate allocation of emergency resources.

Accordingly, this study aimed to develop and internally validate a multivariable prehospital prediction score for critical illness progression and 30-day mortality among red-severity, non-traumatic adult patients transported by the Vajira Emergency Medical Service (V-EMS). Illness progression was defined as the requirement for mechanical ventilator support, vasopressor and/or inotrope administration, ICU admission during hospitalization, or death within 30 days after emergency department arrival.

Methods

Study Design and Settings

This retrospective single-center cohort study was conducted in V-EMS which is comprehensive life support (CLS), Faculty of Medicine Vajira Hospital, Navamindradhiraj University, Bangkok, Thailand. V-EMS is the zone leader of EMS unit area 1 out of 11 areas in the area division of the EMS system in the Bangkok area, dispatched from Erawan Center, Bangkok, including networking operational hospitals with both public and private hospitals of 6 hospitals in all responsible areas.^12,13 Each EMS team of the V-EMS includes at least three staff, including paramedics or ENPs as operation team leaders and emergency medical technicians. During every operation, paramedics or ENPs would function using an off- and on-line medical protocol under the supervision of EPs. The Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement was applied in reporting this study.¹⁴

Participants

We collected data of consecutive adult patients with non-traumatic illnesses from EMS reports coded using the Thailand Emergency Medical Triage Protocol and Criteria-Based Dispatch Code (TEMTP-CBDC), specifically symptom groups 1–20, which correspond to medical (non-trauma) emergencies.

TEMTP-CBDC overview and implementation: TEMTP-CBDC is Thailand’s national pre-hospital triage and dispatch system developed by the National Institute for Emergency Medicine. During an emergency call, trained dispatchers classify the patient into one of 26 symptom groups (1–20 = medical/non-trauma, 21–25 = trauma, 26 = emerging infectious diseases such as COVID-19) and assign a preliminary color-coded severity level—red, yellow, green, or white—according to standardized physiologic and symptom-based criteria. Upon EMS arrival, paramedics or emergency nurse practitioners confirm or update the symptom group and severity code based on the patient’s actual presentation. A red severity code designates a life-threatening or potentially life-threatening condition requiring immediate intervention and rapid transport to a definitive-care hospital.

In this study, only patients aged ≥18 years who were assigned a red severity code within symptom groups 1–20 (non-traumatic illnesses) and transported by the Vajira Emergency Medical Service (V-EMS) to Vajira Hospital were included. The on-scene severity code recorded by EMS personnel represented the final triage status used for study inclusion. Data were consecutively collected from 1 January 2019 to 31 December 2024 to reduce selection bias and improve representativeness of the study population.

Eligibility Criteria

Eligible patients were adults aged ≥18 years with non-traumatic medical conditions who were transported by the V-EMS team and delivered to the emergency department of Vajira Hospital. Patients were required to be coded within TEMTP-CBDC symptom groups 1–20 and assigned a red severity code at the scene according to the TEMTP-CBDC criteria, indicating life-threatening or potentially life-threatening conditions requiring urgent prehospital management and transport.

Exclusion Criteria

Patients were excluded if they experienced cardiac arrest before EMS arrival or during transport, had end-stage disease receiving palliative treatment, were pregnant, underwent inter-facility transport, refused treatment or hospitalization, died before EMS arrival, or had incomplete or missing data in key study variables or outcome measures.

Data Collection

Data of patients were obtained from two primary sources. The first source was the EMS patient care report from the Bangkok EMS Center (Erawan Center), which followed a standardized national template. This form included information on EMS unit identification, patient demographics, clinical presentation, prehospital vital signs, and treatments provided by EMS teams. All data were recorded contemporaneously by trained dispatchers and paramedics or emergency nurse practitioners (ENPs) during field operations and were reviewed for completeness before submission to the Erawan database.

The second source was the electronic medical record (EMR) system of Vajira Hospital, which provided in-hospital data, including definitive interventions such as mechanical ventilator support, vasopressor and/or inotrope administration, ICU admission, and 30-day mortality outcomes. Patients were followed until discharge or death, and outcomes were verified up to 30 days after EMS arrival at the hospital.

All collected data were extracted and consolidated in Microsoft Excel. Variables included age, gender, comorbidity, systolic (SBP) and diastolic (DBP) blood pressures, mean arterial pressure (MAP), heart rate, oxygen saturation (SpO₂), respiratory rate (RR), body temperature, capillary blood glucose (CBG) level, shock index (SI), Glasgow Coma Scale (GCS), response time, scene time, transfer time, airway management, oxygen supplementation, intravenous fluid administration, and hospital outcomes (ICU admission and 30-day mortality).

Comorbidity was defined as the presence of at least one chronic medical condition documented in the EMS or hospital EMR, such as hypertension, diabetes mellitus, dyslipidemia, coronary artery disease, chronic kidney disease, stroke, respiratory disease, or other chronic illnesses. For analytical purposes, comorbidity was recoded into a binary variable (“Yes” or “No”) to improve interpretability and applicability of the model in prehospital settings.

Data Completeness and Missing Data Management

All records were cross-checked between the EMS database and hospital EMR to ensure completeness. Cases with missing or implausible values in key variables (eg., vital signs or outcome measures) were excluded from the final analysis. No data imputation was performed to preserve data integrity.

Measurement Standardization

All prehospital physiological measurements were obtained using standardized EMS devices calibrated according to the Vajira Hospital operational guidelines. EMS personnel followed uniform measurement protocols for vital signs, ensuring consistency across all cases.

Bias Control

Potential information bias was minimized by using prospectively collected EMS forms and EMR data recorded at the point of care. To reduce selection bias, all eligible cases meeting inclusion criteria during the study period were included consecutively without sampling.

Definitions

1. Patients with critical illness progression were defined as EMS patients transported to the hospital who experienced at least one of the following outcomes within 30 days after emergency department arrival: mechanical ventilator (MV) support, vasopressor and/or inotrope administration, ICU admission, or death. Thirty-day mortality was defined as death from any cause occurring within 30 days after hospital arrival by EMS. Patients who were alive at 30 days, transferred to another hospital, or lost to follow-up were treated as censored observations in survival analysis. Response time was defined as the duration from emergency call receipt to ambulance arrival at the scene.¹⁵
2. Endotracheal intubation (ETI) performed in the prehospital setting was analyzed as a prehospital severity indicator rather than an in-hospital outcome variable, whereas the requirement for MV during hospitalization was included as part of the composite definition of critical illness progression. ETI reflects EMS providers’ real-time recognition of severe airway compromise and physiological instability at the scene, while MV represents sustained ventilatory support following hospital admission. Although clinically related, these variables represent different phases of patient management and were therefore analyzed separately.
3. Cutoff thresholds for physiological variables were determined a priori from prior studies and established clinical guidelines. A respiratory rate ≥25 breaths/min was adopted from Veldhuis et al (10), while a shock index (SI) ≥0.9 was based on El-Menyar et al¹⁶ Additional physiological thresholds were selected according to established emergency and prehospital triage frameworks, including NEWS, qSOFA, and national EMS triage standards.
4. The TEMTP-CBDC was used to determine symptom classification and severity code at the scene. The protocol consists of 26 symptom groups, including symptom groups 1–20 for non-trauma emergencies, symptom groups 21–25 for trauma-related emergencies,¹⁷ and symptom group 26 for emerging infectious diseases, including COVID-19.¹⁸

Development of the Prehospital Severity Stratification Score

The multivariable prehospital prediction score was developed using a structured multistep approach designed for practical application in EMS settings. Candidate predictor variables were selected a priori based on clinical relevance, prior literature, availability during prehospital assessment, and feasibility for rapid evaluation by EMS personnel in time-sensitive environments.

Initially, clinically relevant variables obtained during prehospital care, including physiological parameters and prehospital interventions, were evaluated using univariable logistic regression analysis. Variables with p-values <0.20 in univariable analysis and those considered clinically important were subsequently entered into multivariable logistic regression using backward elimination to identify independent predictors of critical illness progression.

The final multivariable model incorporated routinely collected physiological parameters and prehospital management indicators that reflected EMS providers’ real-time recognition of patient severity. To improve operational feasibility and consistency with real-world EMS documentation, continuous variables were dichotomized according to clinically established thresholds derived from prior studies and emergency care guidelines.

The final score was constructed as a simple additive point-based system using regression coefficients from the final multivariable model. Higher total scores indicated greater severity and a higher likelihood of critical illness progression and 30-day mortality. The score was designed to support rapid prehospital risk stratification, triage prioritization, and destination decision-making rather than to function as a purely explanatory statistical model.

Sample Size Determination

Multivariable analysis was performed using multiple logistic regression. Sample size adequacy was evaluated based on the number of outcome events per candidate predictor (events-per-variable, EPV), which is a commonly recommended approach in prediction model development. Traditionally, a minimum of 10 outcome events per parameter has been suggested to reduce the risk of model overfitting and improve model stability.

In the present study, the initial candidate variable list included eight independent predictors. Therefore, at least 80 outcome events were considered necessary for model development. A previous study by Veldhuis et al reported an illness progression rate of approximately 12.2%, which was used as an initial reference for estimating the expected sample size. Based on this assumption, a minimum total sample size of approximately 656 patients was calculated. An additional 5% margin was incorporated to account for potential incomplete or missing data, resulting in a minimum estimated sample size of approximately 691 patients.

However, the final study cohort consisted of 700 consecutive eligible patients, with 272 events of illness progression, yielding an EPV that exceeded commonly recommended thresholds. This indicates that the study was adequately powered for model development and internal validation. The observed illness progression rate (38.9%) was higher than that reported in previous studies, likely because our study population was restricted to red-severity, non-traumatic adult patients transported within an urban EMS network to a tertiary referral center. Differences in case-mix, triage criteria, and composite outcome definitions may also have contributed to this discrepancy.

Statistical Analysis

Descriptive analysis was performed to examine the variable distribution. Continuous variables are presented as mean ± standard deviation or median and interquartile range, and categorical variables are presented as frequencies and proportions. Two-group comparisons were evaluated using the independent t-test or the Mann–Whitney U-test for numeric variables and the Chi-square test or Fisher’s exact test for categorical variables.

Candidate variables for univariable analysis were selected a priori based on clinical plausibility and prior evidence linking physiological instability or prehospital management factors to critical illness progression in emergency settings.

Multicollinearity among candidate variables was assessed using correlation analysis and variance inflation factors. Variables demonstrating strong collinearity were evaluated and removed during the model selection process to improve model stability, interpretability, and clinical applicability.

The analysis of pre-hospital predictive factors that influenced illness progression used univariable and multivariable analyses with multiple logistic regression, and the results were reported using regression coefficient, odds ratio (OR), and 95% CI. Model development utilized backward elimination. Significant factors associated with illness progression based on the univariable analysis (p-value < 0.2) were included in the model, while non-associated independent factors were removed one by one based on Wald test results. The model fit was evaluated using R² and the Hosmer–Lemeshow goodness-of-fit test, and its discrimination ability was assessed with the receiver operating characteristic (ROC) curve, reported as the area under the ROC (AuROC) with 95% CI. The regression coefficients from the final model were used to construct the prehospital severity stratification score associated with illness progression.

Internal validation of the prehospital prediction score was conducted to evaluate model discrimination, calibration, and potential optimism. Discrimination was assessed using AuROC and 95% confidence intervals. Based on Youden’s index, an optimal cutoff value was identified to classify patients into high- and low-risk groups. Diagnostic performance metrics, including sensitivity, specificity, positive predictive value, positive likelihood ratio (LR+), and negative likelihood ratio with 95% confidence intervals, were subsequently calculated.

Calibration and internal model stability were evaluated using bootstrap resampling, which is a recommended and robust approach for prediction model validation, particularly in moderate sample sizes. This method enables efficient use of the entire dataset and provides less biased estimates of model performance compared with split-sample or simple cross-validation approaches. The calibration performance was illustrated using a calibration plot.

Nevertheless, we acknowledge that internal validation cannot fully address model transportability or generalizability. Therefore, external validation in independent EMS populations and diverse healthcare settings will be essential to confirm the robustness, clinical utility, and implementation potential of the proposed prediction score. Future prospective and multicenter validation studies are warranted prior to widespread clinical adoption.

To analyze the ability of the pre-hospital score in predicting mortality within 30 days, survival analysis depicting survival time data with Kaplan–Meier curves was employed, and associations were analyzed using parametric survival models with the Weibull regression model. Results were reported as hazard ratio (HR) and 95% CI. P-values < 0.05 were considered significant. Stata version 18.0 (StataCorp College Station, TX, USA) was used for all analyses.

Results

A total of 700 patients met the eligibility criteria, comprising 272 (38.9%) with critical illness and 428 (61.1%) with non-critical illness (Figure 1). Baseline characteristics of both groups are summarized in Table 1. Patients with critical illness were more frequently male and had a higher prevalence of respiratory disease compared with those without critical illness. They also exhibited a higher incidence of hypotension (SBP < 90 mmHg), hypoxia (SpO₂ < 95%), tachypnea (RR ≥ 25 bpm), elevated body temperature (≥ 37.5°C), increased shock index (≥ 0.9), and reduced Glasgow Coma Scale (≤ 9). Airway interventions, oxygen supplementation, and suctioning were significantly more common among patients with critical illness (all p < 0.001).

Table 1 Demographic and Clinical Characteristics of Adult Non-Traumatic Patients According to Critical Illness. (n = 700)

Figure 1 Flow chart.

Overall, 38.9% of patients experienced illness progression after EMS transport. Among these, 23.1% required mechanical ventilation, 16.4% received vasopressors or inotropes, 13.6% were admitted to the ICU, and 14.7% died within 30 days (Table 2).

Table 2 Critical Illness in Adult Non-Traumatic Patients Transported by Emergency Medical Services System

In univariable analysis, several clinical and physiological variables were associated with illness progression, including male sex, respiratory disease, hypotension, hypoxia, tachypnea, hyperthermia, elevated shock index, decreased GCS, endotracheal intubation, and oxygen supplementation (all p < 0.05).

In multivariable logistic regression, respiratory rate ≥ 25 bpm, shock index ≥ 0.9, GCS ≤ 9, and oxygen supplementation remained significant predictors of illness progression (Table 3). These four parameters were therefore incorporated into the final pre-hospital prediction score for critical illness progression. No significant multicollinearity was observed among variables retained in the final model.

Table 3 Univariable Analyses and Multivariable Analyses for Pre-Hospital Predictors for Predicted Critical Illness in Adult Non-Traumatic Patients Transported by Emergency Medical Services System

The Hosmer–Lemeshow test revealed a result of (χ² = 2.46; df = 6; p-value = 0.873). Variables in the analysis did not have multicollinearity and had an R square of 0.1869, meaning that all five independent variables could explain the variability of predicting illness progression of 18.69% and could be expressed as prediction equation in Logit transformation as follows:

which X₁ respiratory rate ≥25

X₂ SI ≥0.9

X₃ Glasgow coma score ≤9

X₄ ETI

X₅ oxygen supplementation

ROC and AuROC revealed that the regression equation could predict illness progression (p-value < 0.001), with an AuROC of 0.765 (95% CI: 0.728–0.802) (Figure 2a). To simplify the equation for clinical application, we developed score transformation using logistic regression coefficient and determining scoring criteria in each variable demonstrated in (Table 4) and the score from calculating from this equation was called the pre-hospital prediction score. Patients delivered by EMS systems had a mean pre-hospital prediction score of 4.98 ± 5.20 (range, 0–21). Notably, patients with and without illness progression had a mean pre-hospital prediction score of 8.10 ± 5.99 and 2.99 ± 3.38, respectively; the difference was statistically significantly (p-value < 0.001).

Table 4 Significant Predictors of Critical Illness in Adult Non-Traumatic Patients Transported by Emergency Medical Services System and Assigned Item Score

Figure 2 Comparison of the area under the receiver operating characteristic curve (AuROC) between the multivariable prehospital prediction score and the prediction model for critical illness progression in adults with non-traumatic illnesses transported by emergency medical services (A). Receiver operating characteristic curve (ROC) analysis of the multivariable prehospital prediction score for predicting critical illness progression in adults with non-traumatic illnesses transported by emergency medical services (B).

Validation of the Pre-Hospital Prediction Score

The AuROC of the pre-hospital prediction score was 0.765 (95% CI: 0.728–0.801) (Figure 2a), and the post-estimation ROC for predicting death within 24 hours had an AuROC of 0.771 (95% CI: 0.733–0.809) (Figure 2b). Using a calibration plot (Figure 3a), internal validation revealed that the pre-hospital prediction score and illness progression had good agreement, with a slope of 1.000. Additionally, the calibration plot with bootstrapping reproduced 500 bootstrap resamples (Figure 3b) and revealed that the pre-hospital prediction score and illness progression maintained good agreement, with slope of 1.002. The ratio between the predicted and observed illness progression (E/O ratio) was 0.997, with a C-statistic of 0.764.

Figure 3 Calibration plot comparing the predicted probability of critical illness progression with the observed probability of critical illness progression among adults with non-traumatic illnesses transported by emergency medical services (A). The red diagonal line represents the ideal calibration line between predicted and observed outcomes (0 = no event; 1 = event). E:O indicates the expected-to-observed ratio, CITL indicates calibration-in-the-large, and AUC indicates the area under the receiver operating characteristic curve. Internal validation of the multivariable prehospital prediction score using 500 bootstrap resamples (B).

Risk curve analysis results (Figure 4a) demonstrated that patients with a higher pre-hospital prediction score had an increased risk of illness progression, and decision curve analysis (Figure 4b) showed that using the pre-hospital prediction score had a high net benefit when compared with not using it. Using the maximal value of 0.43 based on Youden’s index, a cut-off value of 7 allowed for the stratification of patients into low- and high-risk groups. A pre-hospital prediction score of 7 had a sensitivity of 63.2% (95% CI: 57.2–69), specificity of 79.7% (95% CI: 75.5–83.4), positive predictive value of 66.4% (95% CI: 60.3–72.1), and negative predictive value of 77.3% (95% CI: 73.1–81.2). Overall, 66.4% of patients with a score of 7 experienced illness progression (Table 5).

Table 5 Pre-Hospital Prediction Score Performance Probabilities of Critical Illness in Adult Non-Traumatic Patients Transported by Emergency Medical Services System at Different Cut-off Points

Figure 4 Risk curve analysis showing the observed risk of critical illness progression (hollow circles) and predicted risk of critical illness progression according to the multivariable prehospital prediction score (solid line). Circle size represents the relative number of patients within each prediction score category (A). The vertical dotted line indicates the proposed cutoff value of the prehospital prediction score for identifying patients at increased risk of critical illness progression Decision-curve analysis demonstrating the net benefit across threshold probabilities for predicting critical illness progression among adults with non-traumatic illnesses transported by emergency medical services (B).

Ability of the Pre-Hospital Prediction Score in Predicting Mortality Within 30 days

Based on the pre-hospital prediction score, the high-risk group had a significantly higher 30-day mortality rate than the low-risk group (25.5% vs. 8.4%, p < 0.001). Survival analysis demonstrated that survival at 30 days was 75.3% (95% CI: 69.6–80.1) in the high-risk group and 91.8% (95% CI: 88.9–94.0) in the low-risk group (Figure 5). Patients in the high-risk group had a 3.44-fold higher hazard of death within 30 days compared with those in the low-risk group (HR = 3.44, 95% CI: 2.29–5.17, p < 0.001).

Figure 5 Survival curve for 30-day mortality among adults with non-traumatic illnesses transported by emergency medical services according to risk groups stratified by the multivariable prehospital prediction score.

Discussion

First, this study developed and internally validated a multivariable prehospital prediction score for critical illness progression and 30-day mortality among non-traumatic EMS patients in an urban middle-income setting. The final model incorporated routinely available physiological parameters, including respiratory rate, SI, and GCS, together with prehospital management indicators such as ETI and oxygen supplementation. These variables reflect both physiological deterioration and EMS providers’ real-time recognition of severe illness in time-sensitive prehospital environments.

Importantly, the model was designed as a practical prehospital severity stratification and decision-support tool rather than a purely explanatory prediction model for specific in-hospital interventions. In real-world EMS practice, interventions such as ETI and oxygen administration often represent early recognition of physiological instability before definitive diagnosis or stabilization. Therefore, these variables were included to capture dynamic clinical judgment and escalation of care during prehospital management.

Continuous variables were dichotomized using clinically established thresholds to improve operational simplicity and facilitate rapid bedside application in EMS settings. This pragmatic approach prioritized usability and rapid risk identification under real-world field conditions. In the present study, respiratory rate ≥25 breaths/min was independently associated with illness progression, with patients demonstrating a 2.45-fold higher risk compared with those with lower respiratory rates. Similar findings have been reported in previous prehospital and emergency care studies.^10,19,20 Increased respiratory rate is a well-recognized physiological marker of respiratory compromise and systemic deterioration (21). Previous scoring systems, including NEWS, MEWS, and qSOFA, have consistently demonstrated associations between tachypnea and increased mortality, ICU admission, vasopressor requirement, and intubation.^21–23These findings support respiratory rate as a rapid and practical predictor of deterioration in EMS settings.^24–26

Similarly, SI ≥0.9 was identified as a significant predictor of illness progression, consistent with previous reports linking elevated SI with hemodynamic instability, ICU admission, and mortality.^27–31SI was selected because it integrates heart rate and systolic blood pressure into a single hemodynamic indicator, thereby improving clinical interpretability and reducing redundancy among correlated physiological variables.

In addition, patients with GCS ≤9 demonstrated a 1.72-fold increased risk of illness progression compared with those with higher GCS values. Previous studies have similarly demonstrated that impaired consciousness and severe neurological compromise are associated with adverse outcomes, ICU admission, and mortality.^32–34 ETI and oxygen supplementation were also associated with increased risk of illness progression in the present study. Although these interventions are not purely physiological variables, they likely reflect EMS providers’ recognition of respiratory failure, hypoxia, or severe physiological deterioration at the scene. Previous studies have similarly demonstrated associations between prehospital hypoxia, intubation, vasopressor use, and increased mortality.^19,30,35 Furthermore, oxygenation-related parameters are incorporated into established scoring systems such as REMS and NEWS.^1,5

Second, the prediction score demonstrated acceptable predictive performance for identifying patients at risk of critical illness progression during the prehospital phase. The model demonstrated acceptable discriminative performance, with an AuROC of 0.765 (95% CI: 0.728–0.802), good calibration performance, and stable internal validation results after bootstrap resampling. In addition, patients classified as high-risk according to the prediction score had a significantly increased hazard of 30-day mortality (HR = 3.44, 95% CI: 2.29–5.17, p < 0.001), supporting the prognostic utility of the model for prehospital risk stratification.

The predictive performance of the present model was comparable to that of previously published prehospital risk prediction models. Veldhuis et al reported an AuROC of 0.740 for predicting critical illness using prehospital physiological variables (10), while external validation studies from the United States demonstrated AuROC values ranging from 0.73–0.77 for prehospital critical illness prediction models.³⁶ In the present study, the selected cutoff score of 7 yielded a sensitivity of 63.2% and specificity of 79.7%, suggesting a clinically acceptable balance between early risk identification and false-positive classification in EMS settings. Compared with the out-of-hospital clinical predictor of critical illness in the Washington EMS system (AuROC = 0.78, sensitivity = 22.0%),²⁰ the present model demonstrated substantially higher sensitivity, which may be advantageous in EMS systems where early recognition and timely escalation of care are critical.

Studies in Europe have similarly demonstrated the prognostic utility of NEWS and NEWS2 in prehospital settings. Hoikka et al reported that higher NEWS categories were strongly associated with increased 30-day mortality¹¹), while Jouffroy et al demonstrated that prehospital NEWS2 ≥7 was associated with increased mortality among septic shock patients.³⁷ However, unlike NEWS or NEWS2, which rely primarily on physiological variables, the present model integrates both physiological indicators and prehospital management responses, thereby capturing both objective physiological derangement and EMS providers’ real-time clinical assessment. Moreover, some existing early warning scores may require multiple calculation steps or additional tools, potentially limiting feasibility in resource-constrained EMS settings.^38,39 Differences in EMS structures, staffing models, transport duration, digital infrastructure, and operational resources between physician-led and paramedic-led EMS systems may also influence implementation feasibility and model performance across healthcare settings.

The model also demonstrated favorable calibration performance, with a calibration slope of 1.000 and bootstrap validation slope of 1.002, indicating consistency between predicted and observed outcomes. Furthermore, decision-curve analysis demonstrated a positive net clinical benefit across a broad range of threshold probabilities, supporting the potential utility of the score in real-world EMS decision-making and resource allocation. Early identification of high-risk patients may facilitate more timely referral to hospitals with ICU capability, improve triage prioritization, and potentially enhance patient-centered outcomes through earlier escalation of care.

Notably, the illness progression rate in the present cohort was relatively high (38.9%), likely reflecting the inclusion of red-severity, non-traumatic patients transported to a tertiary referral center within an urban EMS network. Differences in case-mix, triage criteria, and healthcare system structure may therefore influence predictive performance across EMS systems, emphasizing the importance of contextual validation.

Collectively, these findings suggest that the proposed multivariable prehospital prediction score may support earlier identification of high-risk non-traumatic EMS patients, improve prehospital triage prioritization, and facilitate more appropriate destination planning before hospital arrival. Future multicenter external validation studies are warranted to confirm generalizability, transportability, and real-world clinical effectiveness across diverse EMS systems.

Limitations

This study has several important limitations. First, the primary aim of this study was not to predict the future need for mechanical ventilation alone, but rather to develop a prehospital severity stratification and decision-support tool to identify patients at high risk of clinical deterioration and resource-intensive care during hospitalization. The composite outcome included sustained organ support, ICU admission, and mortality. Accordingly, prehospital airway and ventilatory interventions, including endotracheal intubation and oxygen supplementation, were conceptualized as indicators of early clinical recognition of physiological instability and escalation of care rather than direct surrogates for in-hospital outcomes. However, we acknowledge that these variables are closely related to respiratory support and should be interpreted cautiously.

Second, this was a retrospective single-center study conducted within an urban EMS network. Although the V-EMS system operates as a comprehensive life support network covering both public and private hospitals in a large metropolitan region, the findings may not be directly generalizable to rural or differently structured EMS systems. EMS systems vary considerably across countries in terms of staffing models, prehospital interventions, transport duration, and resource availability. Therefore, the performance and operational applicability of this multivariable prehospital prediction score may differ in rural EMS systems, physician-based EMS models, or healthcare settings with different triage structures and resource capacities. Therefore, external validation in diverse geographical and operational settings is necessary to confirm robustness and generalizability.

Third, the study period overlapped with the coronavirus disease 2019 (COVID-19) pandemic, which substantially affected EMS utilization, triage, and hospital admission patterns. Although the proportion of high-acuity cases remained relatively stable, pandemic-related factors such as infection-control protocols, altered dispatch priorities, and delayed hospital transfers may have influenced prehospital times and outcomes. These contextual effects should be considered when interpreting the results.

Fourth, residual confounding may have been present. Important factors such as disease severity, treatment limitation status, in-hospital management, and dynamic clinical responses were not available in the dataset. The relatively modest explanatory power of the model (R² = 18.69%) suggests that additional clinical and biological factors may contribute to illness progression. Prospective studies incorporating more detailed physiological and treatment data are warranted.

Fifth, internal validation was performed using bootstrap resampling, which is an accepted and robust method for prediction model development in moderate sample sizes. However, only internal validation was conducted in the present study. Although this approach reduces overfitting and provides more reliable performance estimates than simple data splitting, it cannot fully address model transportability. Therefore, external validation in independent EMS populations and diverse healthcare settings is essential to confirm generalizability, robustness, and clinical utility before widespread clinical implementation. Future prospective multicenter studies are warranted to evaluate performance across different operational environments.

Sixth, the inclusion of prehospital airway and ventilatory interventions, such as endotracheal intubation and assisted ventilation, may introduce partial incorporation bias and target leakage, potentially leading to overestimation of predictive performance. Although these variables were conceptualized as markers of severity recognition and real-world clinical decision-making, their inclusion may inflate discrimination metrics. In addition, previously published early warning scores may have limited applicability in prehospital settings because some models require repeated physiological assessments, operationally complex calculations, or variables that may not be consistently available during field operations. Future research should therefore evaluate alternative model configurations excluding these interventions and perform sensitivity analyses to assess model robustness.

Seventh, outcome data from patients transported to other hospitals were not available, which may introduce outcome misclassification and selection bias. Although Vajira Hospital is a tertiary referral center, some patients may have received definitive care elsewhere.

Eighth, several potentially relevant predictors, including detailed comorbidity severity, socioeconomic factors, and treatment-specific variables, were not available in the EMS dataset. These factors may influence clinical outcomes and should be incorporated in future prospective studies.

Finally, continuous physiological variables were categorized according to national triage thresholds to enhance clinical usability and alignment with real-world EMS documentation. Although this pragmatic approach improves interpretability and operational feasibility, it may reduce predictive precision. Future work should explore data-driven or continuous modeling approaches to optimize discrimination.

Conclusion

The multivariable prehospital prediction score demonstrated acceptable discriminative performance for predicting critical illness progression and 30-day mortality among non-traumatic EMS patients, with good calibration and stable internal validation performance. The score was also able to stratify patients into clinically distinct high- and low-risk groups, with high-risk patients demonstrating significantly increased 30-day mortality. These findings support the potential utility of the model as a practical and operationally feasible tool for prehospital risk stratification, triage prioritization, and destination planning in EMS, particularly in resource-constrained and high-acuity settings. In addition, implementation of the score may support more efficient resource allocation and earlier referral of high-risk patients to hospitals with appropriate critical care capability. However, further prospective external validation in independent populations and diverse EMS systems is required before broader clinical implementation and routine integration into prehospital practice.