Prediction of Late Hospital Arrival in Patients with Mild and Rapidly Improving Acute Ischemic Stroke in a Rural Area of China

Introduction

The prevalence and incidence of ischemic stroke in China have increased over the past 20 years,^1,2 which has substantially increased the economic burden on families and society.³ Among all ischemic stroke patients, more than half are mild and rapidly improving acute ischemic stroke (MaRAIS) patients.^4–6 Although there is no specific criteria defining MaRAIS (patients with 18 years or older with mild stroke symptoms or rapidly improving acute ischemic stroke defined clinically), most of these patients present with a National Institutes of Health Stroke Scale (NIHSS) score ≤ 5.^7,8 Most MaRAIS patients are considered to have a benign prognosis, even if untreated. Nevertheless, research has shown that 23–38% of patients who initially present with MaRAIS subsequently develop functional disability and poor clinical outcomes compared with other types of stroke.^9,10

Previous studies indicate that thrombolytic therapy with recombinant tissue plasminogen activator can significantly improve outcomes for MaRAIS patients.⁷ Additionally, various studies have shown that early treatment, including the combination of clopidogrel and aspirin, could improve prognosis in mild stroke patients who are treated within 24 hours of symptom onset.^11,12 Consequently, early intervention is vital for MaRAIS patients. We defined late hospital arrival in patients with MaRAIS as more than 24 hours from onset to emergency visit. Indeed, many MaRAIS patients do not recognize the disease early, and thus they delay access to the treatment that would be most effective if provided earlier. This is especially true in rural areas.¹³ In China, over half of the population lives in rural areas. The prevalence of stroke in rural areas of China has increased substantially.¹ However, the early emergency treatment rate of stroke remains low.²

Late hospital arrival is affected by multiple determinants.^14,15 Although there are many risk factors, developing tools for the accurate prediction of late hospital arrival and early intervention may be the most effective actions to improve the unsatisfactory onset-to-door time. The purpose of our study is to develop a valid and easy-to-use prediction model to determine factors associated with late hospital arrival in MaRAIS patients in our rural area and provide evidence for efficient stroke interventions in the future.

Materials and Methods

Patients

A total of 173 patients were enrolled from the South Campus of Renji Hospital affiliated with the Shanghai Jiao Tong University School of Medicine between September 9, 2019 and May 13, 2020. The inclusion criteria was that patients presented with stroke symptoms within 14 days from symptom onset and had a brain computed tomography scan. All patients were diagnosed as having MaRAIS, defined by an NIHSS score of 0–5. The exclusion criteria was that patients with non-ischemic strokes, or NIHSS score>6 ischemic strokes. All participants signed the informed consent and completed a brief survey questionnaire. Information collected included demographics (ie, age, sex, education level, weight and height, medical insurance, and occupation), clinical characteristics, lifestyle factors (eg, smoking and alcohol use), vascular risk factors (eg, hypertension, diabetes, hyperlipemia, coronary disease, and atrial fibrillation), family history of stroke, medication history (eg, aspirin, clopidogrel, statins, or warfarin or other anticoagulants), personal disease history (eg, ischemic stroke, hemorrhage stroke), arrival characteristics (eg, transportation mode [ambulance or other]), knowledge of stroke symptoms and thrombolytic therapy, onset-to-door time (time from stroke symptom onset to the time of arrival at the hospital), and NIHSS score (Supplementary Table 1). Body mass index (BMI) was calculated using weight (kg) and height (m) based on BMI (kg/m²) = weight/height². Research approval was obtained from the Ethics Committee at Renji Hospital, Shanghai Jiao Tong University School of Medicine (2018–182-SR-01) and this study complies with the Declaration of Helsinki.

Statistical Analysis

Using onset-to-door time, the 173 patients in our study were divided into early (<24 hours) and late (≥24 hours) arrival groups. All data in this study were analyzed with R software (version 3.6.3; https://www.r-project.org).

First, we screened the risk factors for the two groups with the least absolute contraction selection operator (LASSO). This method is suitable for high dimensional data compression and can be used to analyze the optimal predicted risk factors of the disease.¹⁶ Features with nonzero coefficients in the LASSO regression model were selected.¹⁷ Then, multivariable logistic regression analysis was used to build a prediction model by incorporating the features selected in the LASSO regression model. The features were assessed with odds ratios (ORs) and 95% confidence intervals (CIs), and the corresponding p-values were then obtained. The statistical significance levels were all two-sided. All potential predictors were applied to develop a prediction model for the risk of late hospital arrival by using the cohort.^18,19

Calibration curves were plotted to assess the calibration of the late hospital arrival nomogram. To quantify the discrimination performance of the late hospital arrival nomogram, Harrell’s C-index was measured. To further verify the stability of this prediction model, the entire dataset was used for internal verification and the late hospital arrival nomogram was subjected to bootstrapping validation (1000 bootstrap resamples) to calculate a relatively corrected C-index.²⁰ Decision curve analysis was conducted to determine the clinical usefulness of the late hospital arrival nomogram by quantifying the net benefits at different threshold probabilities in the MaRAIS cohort.²¹ The net benefit was calculated by subtracting the proportion of all patients who were false positive from the proportion of the patients who were true positive, and then by weighing the relative harm of forgoing interventions compared with the negative consequences of an unnecessary intervention.²²

Results

Demographic Characteristics of the Patients

In the present study, 173 patients were included in our final analysis. All patients were divided into early (n = 97) and late (n = 76) arrival groups (128 men and 45 women; median age 66 years [range 33–91 years]). The specific demographic indicators and clinical characteristics of the two groups are shown in Table 1.

Table 1 Demographic and Clinical Characteristics of Patients in the Early and Late Groups

Factor Selection

Of all demographic and clinical characteristics, 31 factors were reduced to four potential predictors on the basis of the data from the 173 patients in the cohort (~8:1 ratio; Figure 1A and B), and those with nonzero coefficients were incorporated into the LASSO regression model. Those factors included transportation mode, history of diabetes, knowledge of stroke symptoms, and knowledge of thrombolytic therapy (Table 2).

Table 2 Screening Out Risk Factors for Late Hospital Arrival with LASSO Regression

Figure 1 Demographic and clinical factor selection using the LASSO binary logistic regression model. (A) Optimal parameter (lambda) selection in the LASSO model used fivefold cross-validation based on minimum criteria. The partial likelihood deviance (binomial deviance) curve was plotted against the log (lambda) sequence. Each curve represents the trajectory of each of the independent variable coefficients. Dotted vertical lines were drawn at the optimal values by using the minimum criteria and 1 SE of the minimum criteria (the 1-SE criteria). (B) LASSO coefficient profiles of the 31 features. A coefficient profile plot was produced against the log (lambda) sequence. A vertical line was drawn at the value selected using fivefold cross-validation, where optimal lambda resulted in four features with nonzero coefficients. Abbreviations: LASSO, least absolute shrinkage and selection operator; SE, standard error.

Development of an Individualized Prediction Model

The results of the logistic regression analysis for transportation mode, history of diabetes, knowledge of stroke performance, and knowledge of thrombolytic therapy are shown in Table 3. The model that incorporated the above independent variables was developed and presented as the nomogram (Figure 2).

Table 3 Screening Out Risk Factors for Late Hospital Arrival with Logistic Regression

Figure 2 Late hospital arrival nomogram. The late hospital arrival nomogram was developed in the cohort, using transportation, diabetes, KOSP, and KOTT. Abbreviations: Trans, transportation mode; KOSP, Knowledge of stroke symptoms; KOTT, Knowledge of thrombolytic therapy.

Apparent Performance of the Late Hospital Arrival Nomogram in the Cohort

The calibration curve of the late hospital arrival nomogram for the prediction of late hospital arrival risk in MaRAIS patients demonstrated moderate agreement in this cohort (Figure 3). The C-index for the prediction nomogram was 0.709 (95% CI: 0.636–0.783) for the cohort. This was confirmed to be 0.692 through bootstrapping validation, which suggested that the model had satisfactory prediction accuracy. To summarize the results from the above verification, the nomogram of the model had moderate predictive ability.

Figure 3 Calibration curves of the late hospital arrival nomogram prediction in the cohort. The x-axis represents the risk of late hospital arrival. The y-axis represents the risk of actual seeking treatment. The diagonal dotted line represents a perfect prediction by an ideal model. The solid line represents the performance of the nomogram, of which a closer fit to the diagonal dotted line represents a better prediction.

Clinical Use

The decision curve analysis for the late hospital arrival nomogram is presented in Figure 4. The decision curve showed that if the threshold probabilities of a patient and doctor were 3% and 71%, respectively, using this nomogram to predict the late hospital arrival risk adds more benefit than the intervention-all-patients scheme or the intervention-none scheme. Within this range, the net benefit was comparable, with several overlaps, on the basis of the late hospital arrival risk nomogram.

Figure 4 Decision curve analysis for the late hospital arrival nomogram. The y-axis measures the net benefit. The blue line represents the risk nomogram of late hospital arrival. The thin solid line represents the assumption that all patients are seeking treatment early. The thick solid line represents the assumption that no patients are seeking treatment early. The decision curve showed that if the threshold probabilities of a patient and a doctor are 3% and 71%, respectively, using this late hospital arrival nomogram to predict late hospital arrival risk adds more benefit than the all-patient-intervention scheme or the intervention-none scheme.

Discussion

In this study, the nomogram, which included transportation mode, history of diabetes, KOSP, and KOTT, had a moderate ability with a C-index of 0.709 to predict the individual late hospital arrival risk prediction among MaRAIS patients in a rural area of Shanghai, China. And this nomogram could be applied in clinical practice. Nomograms are currently widely used as a novel predictive tool. A nomogram is considered to be a user-friendly digital interface with high accuracy and an easily understood prognosis, which facilitates better clinical decision-making.²³ However, nomograms are rarely applied to cerebral ischemic diseases.

Regarding MaRAIS patients, earlier hospital arrival after symptom onset was associated with better outcomes than late arrival.^4,9 Accurate risk assessment would assist physicians in distinguishing those at a high risk of seeking later treatment and provide timely interventions to increase the probability of a favorable net benefit. Indeed, predicting late-seeking treatment of individual patients was previously quite difficult. Here, we developed and validated a novel prediction tool for the risk of late hospital arrival among MaRAIS patients using four easily assessed variables. Incorporating demographic and clinical characteristic factors into an easy-to-use nomogram facilitates the MaRAIS individualized prediction of late hospital arrival. For example, a MaRAIS patient with diabetes who does not take an ambulance to the hospital and lacks knowledge about stroke symptoms and thrombolytic therapy has an estimated 90% probability of arriving late to the hospital.

Prior studies have shown that 29% of patients with mild or rapidly improving symptoms not receiving thrombolysis had a poor outcome.²⁴ Thus, recurrence and progression of stroke events in patients with an initial mild stroke or transient ischemic attack (TIA) are frustrating.²⁵ Previous nomogram studies focused more on prognostic factors for acute stroke.²⁶ Our study is the first to establish a nomogram model for predicting late hospital arrival in MaRAIS patients in rural China, which could assist clinicians with identification of patients at high risk of late hospital arrival.

In the present study, approximately 44% of patients arrived late at hospital. Consistent with previous studies,^14,15 MaRAIS patients with diabetes were more likely to delay their hospital arrival when they had a stroke compared with people without diabetes. It has been speculated that patients with diabetes may mistake the symptoms of stroke as hypoglycemia and that self-management may delay stroke treatment.²⁷ Therefore, it is necessary to improve the knowledge of strokes in the community, endocrine ward, and outpatient diabetes clinics.

Additionally, we found that the rate of ambulance use was only 16.8%. For patients with ischemic stroke, early transportation to the hospital is associated with effective stroke treatment.²⁸ Despite this, many patients in China with MaRAIS failed to immediately call ambulances.^28,29 Because the rural emergency medical service (EMS) system is inefficient and because rural residents have less knowledge about stroke and lower incomes, this phenomenon is rather common in rural areas.^30,31 It is necessary to improve the ambulance service system and community education in rural areas, so that residents will call an ambulance immediately when a stroke occurs.

The incidence of stroke in rural areas of China has dramatically increased; however, little is known about stroke symptoms in patients in rural areas compared with urban areas.³² According to our research, the rate of stroke symptom awareness was 39.3%. Knowledge of stroke symptoms alone may not lead to patients going to the hospital sooner,^30,33 but patients may act quicker when they know the time window for stroke treatments, such as intravenous thrombolysis. However, only 8.7% of the MaRAIS patients in the present study had knowledge of recombinant tissue plasminogen activator treatment. Thus, future public education campaigns should emphasize that accessing treatment sooner maximizes treatment effects. This education may reduce the treatment seeking time.

No significant differences in arrival time were noted for various other factors, including sex, medical insurance, and neurological deficits. Different from other studies,^15,34 a history of chronic diseases, like hypertension, atrial fibrillation, coronary disease, or hyperlipidemia, did not increase the risk of late hospital arrival. Surprisingly, educational level was not associated with late treatment-seeking behavior. This suggests that stroke education should focus on stroke information, including early symptoms and treatment, regardless of educational level.³³

In conclusion, our prediction nomogram could be used for a wide range of individuals. Our prediction tool may serve as a guide for further rural community intervention. It also indicates that the government should improve public stroke education and increase awareness of stroke and the importance of timely treatment, including using ambulances for transportation, particularly among people with diabetes. Targeted education may be more efficient.

Our study has some limitations. First, this observational study was based on hospitals’ voluntary enrollment and did not have a well-designed sampling frame. Our sample size was relatively small. Additionally, we did not analyze the distance between the patient’s residence and the hospital because our hospital was the only tertiary hospital in our rural town and almost all patients lived in the vicinity of this hospital. In subsequent studies, it will be crucial to consider additional factors such as distance to hospital, patient marital status, whether patients live alone, and whether they have communication difficulties at stroke symptoms onset. Finally, we should verify the applicability and validity of this model in a larger sample and in multiple centers.

Conclusions

This study establishes a simple nomogram that can guide clinicians in developing a medical education program to target MaRAIS patients who may arrive late to the hospital. This is especially true for diabetic patients. The government should implement related stroke education for MaRAIS patients according to the risk factors identified here. This is urgent and salient in rural areas. This nomogram needs more extensive clinical external verification and further research is needed to determine whether interventions in MaRAIS patients on the basis of this nomogram would encourage them to go to hospital as early as possible.