The Validity of Clinical Scoring in the Diagnosis of Stroke Subtype: Validation Study

Introduction

Stroke is defined by WHO as a rapidly developing clinical syndrome of focal or global disturbance of cerebral function lasting more than 24 hours or leading to death with no apparent cause other than vascular origin.¹ It can be either hemorrhagic or ischemic. The actual proportions of stroke types vary depending on the population but from all stroke cases ischemic stroke approximately account for 80%, while hemorrhagic strokes account for 20%.² Because the therapy differs, it is necessary to distinguish between the subtypes for optimal management.³ Ischemic stroke necessitates the administration of thrombolytic and/or antiplatelet therapy, whereas hemorrhagic stroke may necessitate the administration of haemostatic therapy.⁴ In differentiating of stroke subtypes non-contrast CT is gold standard and cheaper than MRI, but in most resource-constrained settings it’s still inaccessible and unaffordable.⁵ Clinically, a definite diagnosis of hemorrhagic or ischemic stroke may not be made in all patients,⁶ and clinical stroke diagnosis limitations have been established in various locations.⁷

To overcome these challenges and improve clinical bedside diagnosis, clinical stroke scores for stroke types have been developed and are recommended to be used in the absence of appropriate imaging modalities.⁸ The Bensson score, the Guy’s hospital score, the Siriraj stroke score, and the Greek stroke score are the most commonly used.^9,10 In practice, the score could be used as a low-cost screening test or as an alternative to CT as a diagnostic test. A screening test must have a high specificity and a high negative predictive value in order to be considered reliable. A diagnostic test, on the other hand, must have high sensitivity and positive predictive value.⁹

Methods and Subjects

The study was conducted in Mizan-Tepi University Teaching Hospital. Cross-sectional study were conducted on 140 (31 females, 140 males) acute stroke patients. Ethical clearance was obtained from Mizan-Tepi University college of medicine and health science Research Ethics Review committee (ref. no. CRERC/01/13, 01/02/2021). Written informed consent was secured from all study participants.

The data were collected by checklist adapted from WHO manual (see Appendix 1, Table S1). Data were entered into Epi-data version 4.1, and exported into SPSS version 25 for analysis.

Study Population

All acute stroke patients from who were admitted to Mizan-Tepi university teaching hospital emergency outpatient department for stroke diagnosis during the study period.

Eligibility Criteria

Inclusion Criteria

• This study included all acute stroke patients whose CT scan showed infarction or hemorrhagic stroke
• Patients with neurological deficit lasting for more than 24 hours

Exclusion Criteria

Patients with stroke caused by other causes such as tumors, or trauma, TB as well as patients with subarachnoid hemorrhage and patients, whose neurologic deficit was resolved within 24 hours, were excluded from the study.

Data Collection Tools

The study tools were designed in accordance with the WHO manual. The tool incorporates two parts: socio-demographic characteristics and clinical variables. Expensive training was provided for data collectors and supervisors on data collection procedures and the ethical issues. The radiologists were required to report only the CT scan results as having hemorrhagic or infarction. They were kept blind to the clinical scores of the patients. In the diagnosis of the disease progression bias, 15 days between the onset of the stroke and the administration of the reference standard was considered appropriate. This time period helps to avoid misinterpreting resolving hemorrhage as infarcts and compensates for the CT scans lack of sensitivity in early ischemic stroke. The equivocal results of each of clinical score findings and those with normal CT scan diagnosis were excluded from the final analysis of specificity and sensitivity.

Clinical variables (level of consciousness, headache within 2 hours, diastolic blood pressure, atheroma markers, apoplectic response, planter response, history of hypertension, history of heart disease, transient ischemic attack, WBC >12,000, neurological deterioration within 3hrs, history of alcohol intake, history of hyperlipidimia, diabetes mellitus, atrial fibrillation at admission, angina, intermittent claudication) were documented at first arrival, after 3hr and after 24hr at bedside and the CT scan was done in all the patients.

Data Analysis

Socio-demographic characteristics of the study participants were analyzed by descriptive statistics. The scores were calculated independently for each patient using each stroke score calculation, while the definition and guidelines were followed as per the original scores (Table 1).

Table 1 Clinical Variables and Calculation Formula Needed for Clinical Score

The Siriraj Stroke Score is calculated as (2.5*level of consciousness) + (2*vomiting) + (2*headache) + (0.1 *diastolic B.P)-(3 *atheroma markers) – 12. A score of less than −1 was considered as infarction and a score of more than +1 as hemorrhage. Scores between −1 and +1 were considered as equivocal.

Guys Hospital scoring was calculated as Apoplectic onset + level of consciousness + plantar response + (diastolic blood pressure (24 hours after admission) * (0.17) + atheroma markers + history of hypertension + previous event (transient ischemic attack) + heart disease + constant (−12). A score of less than 4 indicates infarction, a score between 4 and 24 were considered as equivocal and a score above 24 were considered as hemorrhage.

Greek stroke score was calculated as 6* (neurological deterioration within three hours from admission) +4* (vomiting) + 4*(white blood cells >12,000) + 3*(decreased level of consciousness). A score of <3 indicates infarction, a score of >11 was considered as hemorrhage and a score between 3 and 11 were considered as equivocal.

Bensson score was calculated as (2*alcohol consumption) + (1.5*plantar response both extensor) + (3*headache) + (3*history of hypertension) – (5*history of transient ischemic attack)-(2*peripheral arterial disease) – (1.5*history of hyperlipidemia) – (2.5*atrial fibrillation on admission). A score of <1 indicates infarction and a score >1 indicates hemorrhage.

After the clinical score had been analyzed, specificity, sensitivity, positive predictive value and negative predictive value of each of clinical diagnosis were calculated.

Results

Socio-Demographic Characteristics

This study included 140 patients, 109 (77.9%) of whom were males and 31 (22.1%) of whom were females. The ages of the participants ranged from 18 to 100, with a mean of 54.66 (16.86) years.

Distribution of Acute Stroke

Based on the CT scan reading, the study showed that 70 (50%) of the patients were hemorrhagic, 68 (48.6%) were ischemic, and 2 (1.4%) were normal on computed tomography (Figure 1).

Figure 1 Distributions of acute strokes by CT scan.

Frequency of Stroke Type’s by Different Stroke Score Values

From a total of 140 acute stroke patients, 64 (45.7%) were hemorrhagic, 45 (42.1%) were ischemic, and 31 (22.1%) were equivocal, according to Siriraj stroke score value; 15 (10.7%) were hemorrhagic, 49 (35%) were ischemic, and 78 (54.3%) were equivocal, according to Guys stroke score value; 24 (17.1%) were hemorrhagic, 38 (27%) were ischemic, and 78 (55.7%) were equivocal, according to Greek stroke score value; 108 (77.1%) were hemorrhagic and 32 (22.9%) were ischemic stroke type according to Bensson stroke score value (Figure 2).

Figure 2 Frequency of stroke type’s by different stroke score value.

Comparison results of Different Stroke Score and CT Scan of Brain

Out of 64 patients identified as hemorrhagic stroke types by Siriraj stroke score value, 46 were diagnosed as hemorrhagic stroke, 17 as ischemic stroke, and 1 as normal by computed tomography scan of brain (Figure 3D). A computed tomography scan of the brain revealed that 35 of the 49 ischemic patients identified by Guys stroke score were ischemic, 12 were hemorrhagic, and two were normal (Figure 3B). By computed tomography scan of the brain, 38 and 40 patients were diagnosed as hemorrhagic and ischemic stroke types, respectively, out of a total of 78 equivocal results by Greek stroke score (Figure 3A).

Figure 3 Axial section sample patients’ CT images representing substantial heterogeneity in terms of type and time from the onset of stroke. (A) Left frontoparietal sub-acute infarct (MCA territory); (B) Left Acute ischemic infarct with age-related cortical atrophy (MCA territory); (C) Left parietal lobe intraparenchymal hemorrhage with left frontal and parietal subdural hematoma; (D) Left Basal Ganglia and Thalamic Acute hemorrhage. Abbreviations: A, Anterior; P, posterior.

Out of 108 patients identified as hemorrhagic stroke types by Bensson stroke score value, 62 were diagnosed as hemorrhagic stroke, 44 as ischemic stroke, and 2 as normal by computed tomography scan of the brain. A computed tomography scan of the brain revealed that 24 of the 32 ischemic patients identified by the Bensson stroke score were ischemic and 8 were hemorrhagic (Table 2, Figure 3C).

Table 2 Comparison of Siriraj, Guys, Greek and Besson Stroke Score Value and CT Scan of Brain

Validation Study of Siriraj, Guys, Greek and Bensson Stroke Score Value

The analysis for determining the validation of the Siriraj stroke score was performed on 107 cases after excluding 31 equivocal categories from the Siriraj score, indicating that it was not certain whether the cases were hemorrhagic or ischemic stroke, and while 2 were normal categories based on CT scan results. The sensitivity, specificity, positive predictive value, and negative predictive value of the Siriraj stroke score for distinguishing hemorrhage from ischemic stroke were 83.9%, 68.6%, 74.6%, and 79.5%, respectively, and the overall accuracy was 82% (Table 3).

Table 3 Validation Study of Siriraj, Guys, Greek and Besson Stroke Score Value

The analysis for determining the validation of the Guys stroke score was performed on 62 cases after excluding a total of 76 equivocal categories, and 2 normal CT scan results. Guys clinical score’s sensitivity, specificity, positive predictive value, and negative predictive value for differentiating hemorrhage from ischemic stroke were 47.8%, 89.7%, 73.3%, and 74.5%, respectively, and Guys score’s overall accuracy was 74.5% (Table 3).

The analysis for determining the validation of the Greek stroke score was performed on 60 cases after excluding a total of 78 equivocal categories, and 2 normal CT scan results. The sensitivity, specificity, positive predictive value, and negative predictive value of the Greek stroke score for distinguishing hemorrhage from ischemic stroke were 58.8%, 88.5%, 87%, and 62.2%, respectively, with an overall accuracy of 71.6% (Table 3).

After excluding two normal CT scan results, the analysis for determining the validation of the Bensson stroke score was performed on 138 cases. The sensitivity, specificity, positive predictive value, and negative predictive value of the Bensson clinical score for differentiating ischemic stroke from hemorrhagic stroke were 35.3%, 88.6%, 75%, and 58.5%, respectively, and the overall accuracy of the Bensson score was 62.3% in this study (Table 3).

Discussion

According to the investigators, this was the first prospective study in Ethiopia. This study’s sample size was larger than the two previous retrospective validation studies in Ethiopia (Siriraj score and Greek score). This study’s proportion of hemorrhagic and ischemic strokes was (70 vs.68). In this study, the proportion of hemorrhagic stroke was slightly higher than the proportion of ischemic stroke (70 vs 68), which is consistent with the original Siriraj score study (142 vs 64)¹¹ as well as the study conducted in Ethiopia for validation of Siriraj and Greek stroke scores (29 vs 20)¹² and (49 vs 42).¹³ However, it contradicts the original validation studies of Greek and guys studies, in which the ischemic proportion was higher than in this study (31 vs.137)⁹ and (186 vs.42).¹⁴

The sensitivity of Siriraj score in this study for the diagnosis of hemorrhagic stroke was 83.9%, which is comparable to the study conducted in India 84%¹⁵ and slightly comparable to the original validation study of the Siriraj score in Thailand 89.3%,¹¹ but lower than the studies conducted in North-East Nigeria and South-East Nigeria, which were 94.4%¹⁶ and 94%¹⁷ respectively for differentiation of hemorrhagic stroke from ischemic stroke. It was, however, higher than in Pakistan 71.4%,¹⁸ South Africa 60%,¹⁹ Tunisia 60%,²⁰ and Ethiopia 77%.¹² The specificity of this study was 68.6%, which is slightly similar to the study conducted in India, which showed 71% specificity for hemorrhagic stroke,¹⁸ but it is much lower than the studies conducted in different parts of the world, such as India, Pakistan, South Africa, Tunisia 85%,²⁰ and South-East Nigeria, which were 89%,¹⁵ 81%,¹⁸ 88%,¹⁹ and 92%.¹⁷ The overall accuracy of this validation study was 82%, which is slightly higher than the studies area of North Nigeria 84.6%¹⁶ and Tunisia 81%.²⁰ However, it is higher than the study conducted at Tikur Anbesa Teaching Hospital in Ethiopia 69.2%¹² and the study conducted in Pakistan (76.3%),¹⁸ but lower than the original validation study in Thailand 90.3%¹¹ and the study conducted in South-East Nigeria 93%.¹⁷ In this study, Siriraj stroke score showed 22% equivocal category, which is slightly comparable to studies, conducted in Tunisia 17%,²⁰ Thailand 20%,¹¹ and North-East Nigeria 22%.¹⁶

The discrepancy in study results could be explained by differences in setting, sample size variation, patient trance cultural differences, and the prevalence of hemorrhagic stroke, as well as methodological differences of various studies. The ideal stroke type prediction score should strike a balance between validity, simplicity, and utility.²⁰ Because the Siriraj score is simple to collect and calculate, and it is also available immediately at the bedside, it is used to reduce health-care costs in low-income countries such as Ethiopia. As a result, in situations where CT scan brain is not available, it is preferable to use the Siriraj scoring system for clinical and bedside diagnosis of hemorrhagic stroke especially in rural areas like Mizan-Tepi teaching hospital, Ethiopia.

According to the investigators, this Allen score validation study was the first prospective study in Ethiopia. This study’s sensitivity of Guys score for differentiating hemorrhagic stroke from ischemic stroke was 47.8%, which is higher than the study conducted in South Africa (34%) among populations with high hemorrhagic stroke prevalence.¹⁹ However, it is much lower than the studies conducted in Oxford (81%) and London (88%),¹⁴ as well as Guys Hospital (93%),²¹ and it was lower than the studies conducted in different parts of India (63%),¹⁵ 81%,²² and 88.23% for hemorrhage.²³ The specificity of this study was 89.7%, which is higher than the 76% found in an Indian study of 130 stroke patients.²² However, this is lower than the study conducted in other parts of India among 200 stroke patients 95%¹⁵ and study in South Africa 95%.¹⁹

This study’s overall predictive accuracy of Guys score was 74.5%, which is slightly comparable to studies from Oxford with an overall predictive accuracy of 78%¹⁴ and London with an overall predictive accuracy of 82%.¹⁴ But lower than the Guys Hospital study’s 90%²¹ and the South India study’s 91%.²³ The overall accuracy of different studies may differ due to the exclusion or inclusion of equivocal results from the analysis, as well as the prevalence of different stroke types in different settings. The cost of achieving this accuracy in hemorrhage diagnosis was to misdiagnose 22%, 22%, and 18% of patients with infarction in Oxford, London, and Guys hospitals, respectively,¹⁴ which was comparable to the misdiagnosis of 26.7% of infarction as hemorrhagic stroke in this study. However, this score was uncertain for 76 (54.3%) patients in this study for identifying either hemorrhagic or ischemic stroke, for acute stroke management some variables are difficult to use at the bedside like consciousness level and diastolic BP, so this tool is insufficient for recommending as a diagnostic screening tool at bedside in this study area even if it has good specificity for hemorrhagic stroke.

This study’s sensitivity of Greek score was 58.8%, which is lower than the original validation study from Athens, Greece of 97%,⁹ west Bengal, India 80%,¹⁵ Cairo, Egypt 87.5%,²⁴ and Addis Ababa, Ethiopia 77.8%,²⁵ but higher than the study conducted in tertiary hospital in India 42%.²⁶ This study’s specification was 88.5%, which is consistent with the studies conducted in Cairo, Egypt 89.3%²⁴ and Addis Ababa, Ethiopia 89.3%,²⁵ but it is significantly lower than the original validation study from Athens, Greece 99%⁹ and West Bengal, India 99%.¹⁵ This study’s overall accuracy of Greek score was 71.6%, which is slightly comparable to a study from a single Tertiary care Hospital in India 79.5%,²⁶ but lower than the study from Addis Ababa, Ethiopia 83.6%.²⁵ The percentage of equivocal categories of Greek stroke score in this study was 55.7%, which is comparable to the study from India (51%), but much higher than the study from Addis Ababa, Ethiopia (39.5%) and the original validation study of Greek 7.8%.⁹ The validation study of the Greek stroke score revealed insufficient accuracy, with 71.6% identifying hemorrhage from infarct. Even though the Greek stroke score had good specificity in diagnosing hemorrhage stroke, the high proportion of uncertain cases (78) was the main limitation of this stroke score to be used to guide the physician in stroke management (88.5%).

The original Bensson score was developed to distinguish non-hemorrhagic stroke from hemorrhagic stroke with a high positive predictive value (100%), which means that all patients with a score less than 1 had non-hemorrhagic stroke,¹⁰ which was significantly higher than the study conducted in different countries and in particulars of this study, which was 75%. The positive predictive value of this study is also lower than the positive predictive value of ischemic stroke in India, which was 98.8%¹⁵ and Nepal, which was 95.2%.¹⁷ However, this study is slightly comparable to the study conducted in Massachusetts, USA, 82%,²⁷ and the study conducted in Athens, Greece, 82% of positive predictive value for the score less than 1.⁹

The sensitivity of the Bensson stroke score for distinguishing non-hemorrhagic stroke from hemorrhagic stroke in this study was 35.3%, which is comparable to studies conducted in Massachusetts, United States 38%,²⁷ Nepal 45.45%,¹⁷ but lower than studies conducted in India 65%¹⁵ and Athens, Greece 82%.⁹ In this study, the specificity of the Bensson score was 88.9%, which was lower than the studies conducted in Nepal 96.3%,¹⁷ India 98%,¹⁵ and Athens, Greece 96%.⁹ The variation in the results could be explained by differences in sample size, subject selection bias, and the prevalence of stroke subtypes. For example, the prevalence of ischemic stroke was 68% in Athens, Greece,⁹ and 58% in Nepal,¹⁷ both of which are higher than the 49.3% in this study. Even though the Bensson score is simple to calculate at the bedside and does not require laboratory testing, this study found that it had a low positive predictive value (75%) and very low sensitivity for ischemic stroke (35.3%), and a large proportion of ischemic stroke (52.9%) were not identified as ischemic by using this tool in this study, so it is too difficult to take a risk of starting ischemic stroke management immediately during admission.

The main limitations of this study were the lack of comparison between the four clinical score methods and checking their combination effect, which could improve evaluation performance if used in combination, and the use of a small sample size, which occurred due to the exclusion of uncertain patients from the analysis. Another issue was the availability of a CT scan, which was located 250 kilometers away from the study area, which hampered the overall data collection procedure. To overcome this challenge, all necessary measurements were made with good coordination among the radiologist, clinical data collectors, and supervisors, as well as with the principal investigators.

Conclusion

According to this validation study none of the clinical score methods were adequate to identify hemorrhagic stroke from ischemic stroke. However, the Siriraj stroke score demonstrated good sensitivity and fair overall accuracy for hemorrhagic stroke even though had poor specificity, implying that the Siriraj score could be used for hemorrhagic stroke diagnosis in areas where CT scan is not accessible, such as in rural poor settings, but it should be supported by physician decision by considering the disease’s health risk if treatment is not started immediately and the patients’ health service cost for CT scan. The Siriraj stroke score could be used in academic and epidemiologic studies of different types of strokes.