Emerging Validation for the Adapted Chinese Version of Quick Aphasia Battery

Introduction

Stroke, characterized as a neurological deficit of cerebrovascular cause, is the second leading cause of death and disability worldwide.¹ It is the fifth leading cause of death in the United States, with an overall prevalence of 2.6% in those over 20 years of age between 2009 and 2012.² Leading the first cause of death and disability in China, there are 201 per 100,000 stroke attacks in 2019, and the standardized prevalence rate was 2.58% in people older than 40 years old.³ Approximately 21–38% of post-stroke patients would be diagnosed with aphasia as one of the common complications,⁴ leading to language disorders,⁵ including aspects of phonological, morphological, semantic, syntactic, pragmatic, motor speech et al. Post-stroke aphasia (PSA) is kind of a consequence of damage in a widely distributed and complex language network in the dominant hemisphere (typically left).⁶ These deficits are attributed to damage in higher cognitive areas involved in language processing rather than to areas involved in motor control of articulatory structures.⁷ While, even the mildest degree of PSA brings detrimental effects on patients, like loss of employment, social isolation, depression, or lower quality of life.⁸

Recovery from aphasia depends on neuroplasticity,⁹ which implies neural changes supporting learning and relearning of language elements and processes. The structural and physiological changes of neuroplasticity occur at the synaptic, cellular, and macrostructural levels. Yet, exploration of the role of neuroplasticity in aphasia treatment is nascent. It is clear that the location, degree of damaged brain structures, and impact of the damage on functional systems will place limits on the neuroplasticity for effective aphasia therapy currently. The clinical management and rehabilitation of aphasia will rely on brain damage measurement and residual connectivity, to predict long-term outcomes and personalize treatment selection.¹⁰

Typically, language impairments will be most severe in the acute stage of stroke, and the greatest period of recovery intervenes within the first 3 months.¹¹ However, many patients cannot catch the valuable time due to insufficient attention in the early stage, absence of suitable assessment tools, lack of a speech therapist (ST), or financial issues. Therefore, it is necessary to use a simple tool to identify post-stroke patients’ language disorders in the early stage, provide possibility to catch the golden time to arrange individual language therapy, then achieve better recovery outcomes.

While existing tools in China cut both ways, some comprehensive batteries, such as Aphasia Battery of Chinese (ABC),¹² China Rehabilitation Research Center Aphasia Examination (CRRCAE),¹³ Western Aphasia Battery (WAB),¹⁴ Boston Diagnostic Aphasia Examination (BDAE)¹⁵ usually takes 1–2 hours to complete series tasks. While it is almost impossible for a weak patient who just underwent a stroke to spend such a long time and energy to complete all tasks. On the other side, some other quick screening tests, like Language Screening Test (LAST),¹⁶ which takes 5–10min, but can only distinguish whether aphasia happened or not simply, cannot give a further diagnosis to guide rehab, so that not really be used since adapted.

After relevant literature study, we found a newly developed tool, quick aphasia battery (QAB), designed by Wilson.SM from Vanderbilt University¹⁷ to support research into the neuroplasticity of language networks after brain regions damaged. QAB includes 8 subtests of the level of consciousness, connected speech, word comprehension, sentence comprehension, picture naming, repetition, reading, and motor speech. It had been verified with excellent psychometric properties and can be used to identify patients’ aphasia condition and its severity within 15 minutes by general medical staff after simple training. Wilson administered QAB in 334 PSA patients on the 5th day, 1 month, 3 months, and 1 year after stroke onset, to explore the relationship between stroke area and rehab process evidenced by QAB score and radiology results.¹⁸ Already adapted into Arabic, Catalan, Danish, French, Korean, Romanian, and Spanish versions, but lacked in Chinese.¹⁹ The study conducts translation and cross-cultural adaption of QAB, aiming to provide an effective, multidimensional, reliable tool for identifying PSA in China.

Materials and Methods

Material

QAB was designed to identify the patient’s aphasia and its severity, including 8 subjects mentioned above, each subtest includes 2–12 items that score from 0 to 4, with clear guidelines and a scoresheet. All measures showed good or excellent test–retest reliability (intraclass correlation coefficient (ICC)=0.98) and excellent inter-rater reliability (ICC=0.99). Sensitivity and specificity for the diagnosis of aphasia were 0.91 and 0.95, respectively. All QAB measures were highly correlated with corresponding WAB measures where available.

Methods

Following the process of cross-cultural adaption proposed by the World Health Organization (WHO), comprising steps of forward translation, expert panel, back-translation, pre-test, and interview, then final version.²⁰

Forward Translation

We invited two Chinese native scholars who had worked overseas and know well about English, they translated the original QAB into Chinese version, respectively. The first step was to find conceptually equivalent wordings and phrases.

Expert Panel

We compared and integrated the forward two versions into one after discussion. Then 6 related experienced experts were invited, including 1 professor from the Department of Speech Rehabilitation at “985 Project” university, 1 expert who is familiar with research design and cross-cultural adaption, 2 neurological physicians from tertiary hospitals, and 2 speech therapists from rehabilitation hospitals. They evaluate all items, discuss all inconsistencies, and give revised suggestions based on their knowledge. The discussion is throughout the whole adaption process until all experts and research group members agree with all the items.

Back-Translation

2 bilinguals good at Chinese and English were invited, one of them is an English native and graduated from a Chinese major. They back-translate the integrated Chinese version into English again individually. We compared and discussed those two versions with the original QAB to find out inconsistencies. Discuss controversial expressions with Dr. Wilson by E-mail also.

Pre-Test and Interview

After the preliminary version developed, a pre-test is necessary to verify the feasibility of elementary, to see whether the instruction and guideline are comprehensible to target public.

The pilot study was conducted in a stroke unit of tertiary hospital in Shanghai. 5 stroke patients (blind to the battery) were tested by 5 in charge nurses (be quickly trained by ST before testing) with the preliminary version. Besides, 20 healthy subjects were also recruited to talk with ST about the preliminary version, setting as empt. All participants were chosen by non-probability sampling method, who were willing to be involved and conditions allowed involved, agreement consent obtained. The time taken, perceived difficulty, and confusion with the picture or verbal statements were collected.

Final Version

After revision based on all feedback from experts and pre-test, we developed the final Chinese version of the quick aphasia battery (CQAB) with all members involved recognized. Then, the validation study conducted to test the psychometric properties of the CQAB.

Validation Study

The process lasted from August 2022 to January 2023, stroke patients (acute/chronic phase:87/41) from 3 general and rehab hospitals in Shanghai were recruited by purpose sampling and informed consent obtained previously. The inclusion criteria were as follows: ≥18 years old, ability to speak and read Chinese words, undergone stroke evidenced by MRI or CT scan. Patients who underwent stroke within 2 weeks of onset were divided into the acute group, and beyond 2 weeks were divided into the chronic group.⁶ Patients who were dementia, blind, deaf, dumb, or had speech disorders history were excluded. Involved patients’ approach condition was assessed by care providers, then tests were conducted bedside or in private room individually, videotaping for further review and timekeeping.

First, 2 professional STs conduct tests on chronic phase stroke patients (1st ST uses WAB, which was known as the “golden standard” to identify patients’ aphasia, then the 2nd conducts CQAB on the same patient after 15min of break). Record patients’ reactions and time spent. Then scores are given individually without discussion. Second, we conduct a study on acute phase stroke patients,1 ST and 1 general medical staff (quickly trained the 1 ST) test patients according to CQAB items alternatively, and score individually.

Data Analysis

Original QAB has supported a set of EXCEL formats to calculate individual scores automatically. Then we gather all individual scores and import them to SPSS, statistics 25.0 for the following analysis. The demographic characteristics were described by frequencies (n, percentage) for categorical variables and means (mean±SD) for continuous variables. Internal consistency was evaluated using Cronbach’s alpha coefficient, split-half reliability, test–retest reliability, and inter-rater reliability. The content validity index (CVI) was used to interpret content recognition. Structural validity was evaluated by exploratory factor analysis (EFA), and the Kaiser Meyer Olkin (KMO) test was used to evaluate whether data were appropriate. The discriminant validity was calculated by Pearson chi-squared tests, and criterion validity was examined by two samples of T-tests. Besides, receiver operating characteristic (ROC) curve analysis was performed to reflect diagnostic accuracy and efficiency.²¹ The optimal frailty cutoff point was determined by calculating the maximum Youden’s index (YI, YI= sensitivity+ specificity–1). Kappa coefficients were used to describe the strength of the agreement between the CQAB and WAB. The results were considered statistically significant at p < 0.05.

Results

Results of Cross-Cultural Adaption

We strive to adapt CQAB closer to Chinese language expression and cultural customs. Following the expert panel and pre-test, we made 28 adjustments mainly.

Suggestions from the expert panel included word concept conversion, word order adjustment, and collocation replacement. For example: “cut the grass”. It is a common scenario in America but rare to have grass in common residences in China. So that we adapted it into Chinese means “weed the field”, which is familiar to farming culture. We translated some terminologies into the same forepassed expression, such as “Close/Open Class Word”, and “Telegraphic Speech”.²²

Results of pre-test shows the time taken was 8 minutes 12 seconds-13 minutes 45 seconds, which is respected to original QAB “15min administrated” setting. Final total score ranges from 8.75 to 9.93, only 1 of them classified to mild aphasia as QAB original setting, matched with the result of WAB test conducted by ST previous which consumed 1 hour and 47 minutes. It indicates CQAB may play similar effects as complicated WAB. Patients and medical staff’s responses also imply us to adjust expression to avoid contextualization misleading. In healthy subjects, they responded to CQAB quickly and completed it within 5–7 minutes with non-aphasia showed. They did not ask for supplementary instructions or explanations but pointed out some pictures with low recognition, then we substituted them until all pictures can be identified easily.

Results of Psychometric Properties

Demography Statistic

This study recruited 134 patients in total, while 6 of them dropped out (2 low cooperation, 1 slur version, 1 speech impairment history, 2 consciousness disturbance), so a scale of 128 participants included finally, of which the majority were male (62.4%), a median of age 63.2 years old, most of them have one or more comorbidity (78.4%), almost all patients were retired (92.8%), most of them are right-handedness (84.4%). Regarding the results of MRI or CT, the most recurrent being related to ischemic stroke (86.4%), hemorrhagic stroke (13.6%) followed. The mean administration time in aphasia patients was 13.2 ± 4.7 minutes, and in non-aphasia patients was 11.2 ± 3.5, with unhurried administration. More details are shown in Table 1.

Table 1 Demography Statistic

Reliability Verification

Test-Retest Reliability

We conducted three versions of CQAB on the same patient, the intraclass correlation coefficient (ICC) within different versions is 0.849, with a 95% confidence interval (CI):0.461–0.955, p<0.001 (ICC > 0.70 indicates adequate test–retest reliability usually²³). Evidence that the three versions of the CQAB are equivalent with good test–retest consistency, each version could be used to assess reproducibly with the same effect.

Reliability of Internal Consistency

Contributing to the equivalent of three versions, we calculate the internal consistency with all data acquired from all those 3 versions. The overall Cronbach’s alpha coefficient is 0.962, overall Guttman coefficient is 0.960. Generally, Cronbach’s α coefficient>0.9 indicates good internal reliability and Guttman’s coefficient>0.8 can be considered good split-half reliability.²⁴ Those results show that the internal consistency and stability of the CQAB are good.

Inter-Rater Reliability

41 acute-phase patients were evaluated by two different combination examiners. According to results, the ICC between different examined teams is 0.998, showing a high correlation, which means that results are consistent regardless of who the second examiners were, indicating that CQAB has good inter-rater reliability with the intraobserver agreement. With the further maturity and continuous improvement of CQAB, non-rehabilitation medical staff can also use it to evaluate aphasia conditions after a quick training. Expected to improve the efficiency of early assessment for speech impairment in future clinical work.

Validity Verification

Content Validity

Eight senior experts in the fields of stroke and rehabilitation were invited to conduct expert consultations. They evaluated the CQAB and scored from 1 to 4 on each item, higher score means higher recognition. The item of content validity index (I-CVI) ranges from 0.875 to 1, and the average scale of content validity index (S-CVI) is 1 for the 1st version, 0.875 for the 2nd and 3rd versions, overall CVI is 0.917. I-CVI higher than 0.78 and S-CVI higher than 0.9 indicates good content validity usually.²⁵

Structural Validity

KMO value of CQAB is 0.861 (>0.8) and the χ2 value of Bartlett’s spherical test is 831.138 (p<0.001), which means the study is suitable for exploratory factor analysis (EFA).²⁶ Then 2 factors were extracted after EFA, and the cumulative variance contribution rate is 91.588% (>50%). The factor loading values of each entry on the corresponding factor were 0.798–0.919, all >0.4.²⁷ Only “word finding” and “grammatical construction” double loaded but both less than 0.4, without meaning. Therefore, two factors are retained, “word comprehension” and “sentence comprehension” which are mainly related to factor 2, named “Language Understanding”; “word finding”, “grammatical construction”, “speech motor programming”, “repetition”, and “reading” are sorted into factor 1, named as “language application”. Shown in Table 2.

Table 2 Results of Exploratory Factor Analysis

Discriminant Validity & Criterion Validity

WAB is used as the golden standard, which is recognized by the ROMA consensus statement, and widely used in clinical studies with good reliability and validity.²⁸ Chronic-stage patients were classified into aphasia and non-aphasia by WAB. Then CQAB was used to assess the 87 subjects individually. The results of two-sample t-test between two tools’ scores showed that the total score of the aphasia group was lower than that of the non-aphasia group, with a significant difference (p<0.001), indicating that CQAB has good discriminatory validity. Results of Pearson correlation analysis show that the correlation coefficient between the total score of CQAB and WAB is 0.977 (p<0.001), in “word comprehension”, “sentence comprehension”, “word finding”, “grammatical construction”, “speech motor programming”, “repetition”, and “reading” are 0.834, 0.759, 0.953, 0.935, 0.914, 0.900, and 0.908, respectively (p<0.001). The value meets 0.6–0.8 means a strong correlation,²⁸ our results indicate that the criterion validity of CQAB reaches standard also.

Above all, data on reliability and validity are summarized and shown in Table 3.

Table 3 Sum Up of Reliability and Validity Results of CQAB

ROC Curve

Results show that the area under receiver operating characteristic curve (AUC) is 0.99 (95% CI:0.919–0.963), statistically significant (p<0.001), evidence excellent diagnostic accuracy and efficiency (The criteria of AUC:0.50 to 0.60 = fail; 0.60–0.80 = fair; 0.80–0.90 = good; 0.90–1.0 = excellent²⁹). The ROC curve shown in Figure 1.

Figure 1 ROC curve. Abbreviation: ROC curve, receiver operator characteristic curve.

And optimal frailty cutoff point is 8.86, which means when CQAB overall score is 8.86, the value of diagnostic to identify aphasia is greatest, with a sensitivity of 0.977 and a specificity of 0.932. The cutoff point of the original QAB is 8.9, sensitivity 0.91, and specificity 0.95. CQAB matches with the original setting, and even with higher sensitivity. The cutoff point and YI shown in Table 4.

Table 4 Distribution of Sensitivity and Specificity Indicators of the CQAB

Concurrent Validity with Respect to the WAB

Same to the original design, subtests in CQAB derived from WAB are highly correlated also. CQAB “word comprehension” correspond to WAB “auditory word recognition”; CQAB “sentence comprehension” corresponds to WAB “yes/no questions”; CQAB “word finding” corresponds to WAB “naming”; CQAB “grammatical construction” corresponds to WAB “fluency”; CQAB “repetition” corresponds to WAB “repetition”; and CQAB “overall” corresponds to the WAB “aphasia quotient”. CQAB “reading” corresponds to “reading” in supplementary parts of WAB, while not usually administered. Results of the Kappa coefficient between the two tools show that correlation coefficients within the above subtests are 0.882, 0.837, 0.918, 0.913, 0.96, and 0.955 in overall score (0.81–1.00 means almost perfect³⁰), shown in Figure 2. As our study, there are high correlated. All CQAB measures were highly respected to WAB measures where available, indicating that CQAB can play a similar role to identify aphasia as WAB while taking less time and energy.

Figure 2 Correlation analysis between WAB& CQAB. Abbreviations: WAB, Western Aphasia Battery; CQAB, Chinese version of quick aphasia battery.

Discussion

Stroke has high morbidity and may even be higher in further, the population of PSA will grow larger and larger, impacts more to patient’s daily life and mental health.^31,32 It pushes us to pay more attention and medical resource to PSA’s earlier identification and treatment.

Following the guideline for cross-cultural adaption from WHO, we develop the CQAB adapted to environment and customs in China. The cross-cultural adaption process was performed following standardized literature guideline. The pre-test showed that CQAB could be clearly understood by the healthy population, post-stroke patients, and general medical staff. In addition, the experts state that CQAB has clear semantic meaning and language expression, which aligns with Chinese cultural background and expression habits. The content validity of overall CQAB is 0.917, showing a high recognition by experts and increasing credibility of the battery. Ensuring respondents can understand questions as designed.

Then the psychometric properties of CQAB were evaluated by validation tests. Cronbach’α of the whole battery is 0.962, split-half reliability is 0.96, demonstrating excellent internal consistency. Exploratory factor analysis (EFA) was conducted to verify the construct validity of CQAB as the KMO value was 0.861, results show that the loadings on each entry ranged from 0.798 to 0.919, all greater than 0.4, two factors (named “language understanding” and “language implication”) structure were verified, explaining 91.58% of the total variance, showing the adapted CQAB has a stable construct validity.

The test–retest reliability is 0.849, which shows that the battery is repeatable on the same and/or different populations. And the inter-rater reliability is 0.998 higher than the standard, indicating that general medical staff can also manage the CQAB. Nurses can play a crucial role in screening for speech impairments, giving speech therapy, and supporting emotional support as reported.³³ Hopes to alleviate the shortage of professional speech therapists, increase handleability for early assessment, promote feasibility for early rehabilitation, and improve the relationship between nurses and patients.

Furthermore, all CQAB measures were highly respected to WAB measures related, as the correlation coefficients within the overall battery and 5 individual subtests show highly correlated. WAB was set as the golden standard, the result of Pearson correlation analysis evidenced the great discriminant validity and Criterion validity of CQAB in distinguishing aphasia and non-aphasia conditions, indicating that CQAB can play a similar role in assessing speech impairment as WAB while taking less time and energy. Expected to provide an effective tool to improve the possibility of early recognition for PSA patients and to fill the gap between comprehensive assessment tools and screening scales in the Chinese clinic.³⁴

Last but not least, the sensitivity of CQAB was 0.977 and the specificity was 0.932 when the cutoff point was 8.86 shown in the ROC curve. In other words, for patients with CQAB overall score below 8.86, the true positive rate for diagnosis as aphasia was 97.7%; the true negative rate for non-aphasia per clinical impression is 93.2%, with CQAB overall score higher than 8.86. There is still a small probability for misdiagnosed, and the reason may be related to the impact of stroke on executive attention, disorder emotion, or impaired memory, but not only speech impairment.³⁵

Above all, CQAB shows great reliability and validity and seems feasible to be used in assessing stroke patients’ aphasia condition by general medical staff bedsides. Works to evaluate patients’ language impairment, and clarify patients’ remaining ability of speech. Drawing up plans for subsequent rehabilitation treatment based on multidimensional results, achieving better efficacy as early initiate rehabilitation.

Additionally, QAB was designed to diagnose aphasia triggered by not only stroke but also some other diseases, like brain trauma, tumor, or cerebrovascular malformation.^36,37 The equivalent 3 versions could be used to evaluate during subsequent rehab treatment termly. To avoid memory learning, we adapted three equivalent versions to make CQAB more appropriate for longitudinal studies. Those ideas provide more possibility for wider use of CQAB in China.

Limitation

However, the study still has some limitations. First, due to the epidemic of Covid-19 in China, we collected only 128 stroke patients in hospitals in Shanghai, where the population may have a higher level of education and culture to understand all questions and pictures in our battery.³⁸ Second, the sensitivity and specificity were conducted in stroke patients who underwent the chronic phase, rather than in the acute phase, due to acute phase patients cannot tolerate tests of professional WAB plus 3 versions of CQAB together. Thirdly, we only conducted tests on stroke patients, not including other diseases, which may impact patients’ language and speech. However, all those limitations are the directions for our further following research.

Conclusion

This is the first study to translate, cross-culturally adapt, and validate the Chinese version of the quick aphasia battery among post-stroke aphasia patients in China. The CQAB respected the original version in semantic and idiomatic. The validation test demonstrates good internal consistency, satisfactory inter-rater reliability, and test–retest reliability; acceptable content validity, structural validity, discriminant validity, criterion validity and concurrent validity; Sensitivity and specificity higher than the original version. Providing an effective, multidimensional, convenient tool for assessing PSA in early stage, and evaluating rehab effect in future clinics in China hopefully.