Translation and Cross-Cultural Adaptation of Post-Stroke Checklist into Mandarin: A Cognitive Interview Study

Introduction

With more than two million new cases annually, stroke has the highest disability-adjusted life-years loss of any disease in China.^1,2 The burden is even expected to increase further as a result of the epidemiological transition from infections to non-communicable diseases, which may pose huge challenges to the healthcare system.³ However, there is no standardized process for long-term post-stroke care, though our health systems have been struggling to deliver effective interventions.

The Post-stroke Checklist (PSC) was developed by the global stroke community advisory panel based on the evidence to help medical personnel in standardizing the process of identifying the need for long-term care after stroke and facilitating referrals.⁴ It has been proven to be feasible and useful and has gained international recognition as endorsed by the World Stroke Organization to improve post-stroke care. Also, the PSC has been translated into different languages and has shown acceptable feasibility and usability in some settings, such as Sweden,⁵ Italy,⁶ the UK,⁷ Singapore⁷ and Germany.⁸ However, it has not yet been translated into Mandarin despite been validated for use in the Chinese post-stroke victims.

In order to use the instrument across cultures, the items need not only to be translated linguistically but also be adapted culturally to maintain equivalence between the source and target versions; this usually involves processes such as forward translation, reconciliation, back translation, expert committee review, cognitive interviewing and proofreading.^9–12 The role of an expert committee is to review all the translations, make critical decisions, reach a consensus on any discrepancy, consolidate all versions of the survey questionnaire, develop a pre-final translated version in targeted language for field testing, and validate its content validity.^9,11 Cognitive interviews can help researchers identify the manner in which targeted subjects understand, mentally process, and respond to the material, with a special emphasis on the unrecognized problems to minimize the bias and other classic questionnaire errors before field testing.^13–15 It is viewed as an excellent method of conducting an in-depth evaluation from the patient perspective to pretest a survey questionnaire in cross-cultural adaptation.¹⁶ Therefore, the purpose of this study was three-fold: 1) to translate the PSC into Mandarin, 2) to modify the Mandarin version of the PSC (M-PSC) and validate its content by means of expert review, and 3) to culturally adapt the M-PSC according to the findings of cognitive interviews among stroke survivors.

Materials and Methods

This is a descriptive study that used qualitative methods. The study followed published consensus guidelines for translation and cross-cultural adaptation (Figure 1).¹² Consolidated criteria for reporting qualitative research (COREQ) were followed to ensure standardized reporting.¹⁷

Figure 1 The translation and cross-cultural adaptation process.

This study was approved by the Ethics Committee of Nanfang Hospital (No. NFEC-2018-054), proceeded only after all relevant clearances have been obtained from all the settings involved and conducted in accordance with the Declaration of Helsinki. Further, anonymity and confidentiality were ensured and written informed consent was obtained from all the participants.

Measures

The PSC⁴ is an 11-item checklist administered by health care professionals to post-stroke individuals at 6 and 12 months and annually thereafter, which focuses on areas where interventions have the largest impact on stroke survivors’ quality of life. There is a dichotomous “yes”/ “no” response scale of each item and a referral recommendation for each problem identified.

Translation

1) Preparation: Permission to translate the PSC was obtained from the copyright owner through email, 2) Forward translation: One researcher (MJ) majoring in nursing and another translator without a medical background independently translated the PSC into Mandarin. They are both native Mandarin speakers and have a good level of English proficiency, 3) Reconciliation: The two versions were merged into a single forward translation upon consensus, 4) Back translation: Two native English speakers were blinded to the original measure and individually developed a back-translated version in English based on the forward translation, both of whom have a good command of Chinese and can communicate fluently in Chinese, 5) Back translation review: Then, the back-translated versions were reviewed against the original measure to identify any discrepancies.

Expert Review

A three-step expert review, involving six bilingual experts (a neurologist, MD, PhD; a deputy director of nursing engaged in stroke, PhD; a rehabilitation physician, MD, PhD; a psychology professor; a professor in medical English; and a chief superintendent nurse working in community care) and four translators who participated the translation process, was conducted. First, the original measure, translated versions, and notes related to challenging phrases, uncertainties, and rationales for choices were organized into an encrypted word document and sent to the experts through WeChat messaging app. The experts individually reviewed and feedback to one of the researchers (MJ) to sort out. Then, an expert committee review was held with all the experts and translators by video conferencing. They reviewed all the translations, reached a consensus on any discrepancy, and developed a pre-final version of the M-PSC. Further, a 4-point scale from 1 (not relevant) to 4 (highly relevant) was sent to the experts to assess the content validity index for each item (I-CVI) and their mean (scale-content validity index/average [S-CVI/AVE]). A measure is considered to have excellent content validity as the minimum of I-CVI exceeds 0.78 with an S-CVI/Ave of greater than 0.90.¹⁸

Cognitive Interviewing

Participants

Participants were recruited in Guangzhou and enrolled from 1) a stroke rehabilitation outpatient department of a tertiary hospital in Yuexiu District, 2) a rehabilitation department using traditional Chinese medicine of a secondary hospital in Baiyun District, and 3) a community in Baiyun District. Utilizing purposive sampling, a resident living in the community and two clinical nurses working in the hospital were responsible for the recruitment. The information of the participants was rechecked to ensure they met the eligibility criteria, as the community recruiter lacked medical knowledge. Only the recruitment of community participants and the review of their admission was done through WeChat, everything else was done face-to-face, including the signing of informed consent forms. We aimed to obtain a sample that is as diverse as possible regarding education level, gender, age, occupation, and marital status. Participants recruited from outpatients and communities in the same district might have similar characteristics. To offer sufficient information power,¹⁹ we chose an outpatient from a different district with the community. Individuals were eligible to participate in this study if they had experienced a cerebral infarction or cerebral hemorrhage and had been more than six months since the most recent stroke. Those with cognitive impairment (Montreal Cognitive Assessment score <26), dysphasia, and those inability to speak fluently Mandarin were excluded. A set of nine practice rounds, where the number of rounds of iteration was determined based on the saturation of information, was conducted. A total of 29 individuals with stroke were invited to take part in this study. Two of them declined participation; one did not admit to having the disease and the other did not agree to recordings or notes during the interview, resulting in a study sample of 27 participants. All participants were interviewed once during the period November 2020 to February 2021.

Data Collection

Cognitive interviews with participants recruited from outpatient and inpatient departments took place in an undisturbed office in the hospital, while interviews with community participants were conducted through WeChat messaging app. An MSN candidate (MJ), who had received training in qualitative research and cognitive interviews, and previous interview experience, performed the cognitive interviews. Participants were provided with a background as to why the study was of interest as well as an overview of the ethics of the research. Apart from the interviewer’s name and affiliation, participants did not know about the interviewer’s characteristics; no prior relationship existed between them. Once the participants had agreed to participate, the recruiter set a mutually suitable time for the patient and the interviewer to conduct the interview. During the formal interview, the interviewer gave the patients a brief introduction of the study, confirmed that they had signed an informed consent form, and started the interview with a classic think-aloud training exercise: “How many windows are there in this room?” and “Please tell me the details about how you came up with this answer”, to help them become familiar with the process of think aloud.¹³ Then, a semi-structured interview, using concurrent probing and immediate retrospective probing, was followed. The interviewer read each item of the M-PSC aloud, then the participants answered and vocalized all of their thoughts as they respond to the question. The scripted probes were presented after each question. For Q2b, we also used a card sort task to instruct the participants to point out what was difficult for them to do. Anticipated probes were developed using the question appraisal system that aimed at identifying potential problems in survey questionnaires.¹³ Emergent probes were generated as necessary during the interviews. The cognitive testing protocol was developed based on a guiding book,¹³ and was pretested in an interview cognitive simulation. Scripted probes for each item could be found in Supplemental Appendix 1. Interviews were audio-recorded, and no one else was present besides the participant and the interviewer. Field notes were taken by the interviewer during and after the interview. Additionally, the first author (MJ) conducted a maximum of three interviews a day to keep energy.¹³

Analysis

Transcription and analysis were carried out immediately after each interview. Two researchers (JJL and SMH) transcribed the audio-recording of each interview verbatim. Transcripts were then rechecked against the recording for accuracy by another researcher (HQL). Data were analyzed deductively based on the four-stage cognitive model: comprehension, information retrieval, judgment, and response process.²⁰ Codes were assigned independently by two data analysts (HQL and JJL). The item of the M-PSC was modified if five participants found the question to be difficult to understand. Codes and recommendations for changes to each item were discussed until consensus was reached. Then, the recommendations were sent to a focus group for further discussion. The focus group consisted of seven participants: one facilitator, one researcher of this study (HQL), two researchers of a parallel study,²¹ two clinical nurses, and one observer. The three researchers discussed the recommended changes under the guidance of the facilitator, and once the suggested changes were confirmed, the corresponding probes for that item in the protocol would be modified as well. A simulated cognitive interview according to the protocol was then conducted with the first author (MJ) as the interviewer, who was also responsible for all interviews in this study, and one researcher (HQL) pretending to be a stroke victim. The interviewer’s performance and the content of the protocol were discussed to determine if improvements were needed. Findings from earlier interviews informed adaptation for the M-PSC in further interviews. When five consecutive participants reported no problems or five of nine individuals did not report any issues, the item was modified appropriately.

Trustworthiness

Several strategies were used to ensure trustworthiness.²² Data triangulation was used in recruiting participants from a community, a secondary hospital and a tertiary hospital respectively, while investigator triangulation was applied by involving two researchers coding and analyzing independently and a focus group to confirm suggested changes. A detailed description was provided on the context of the research, setting, sample, sample size, sample strategy, demographic and clinical characteristics, inclusion and exclusion criteria, interview procedure, and so on. In addition, reflexive notes were conducted during the interviews and the analysis.

Results

Translation and Expert Review

We successfully developed an initial Mandarin version of the PSC, back-translated it into English, reviewed two back-translated versions against original measure regarding discrepancies, and performed a three-step expert review. During a 65-minute video conference, the expert committee provided comments on all items, mainly involving discrepancies resulting from cultural variation. A case in point was the term “observe progress”, which was too concise in the Chinese context and was thus replaced with “Please follow up the rehabilitation status of post-stroke individuals”. Concerning the term “primary care team”, there was no equivalent term in Chinese, so we changed this term to “staff in primary health care institutions”. In the Chinese health care context, the management of data of post-stroke individuals is performed by the “community chronic disease management team”, not by a particular “community stroke team”. Similarly, “OT or PT” was changed to “rehabilitation physician”, and “community continence adviser or equivalent” was replaced by “neurologist, rehabilitation physician or urologist”. Then, the term “stroke association” was changed to “stroke club”. The experts noted that stiffness was always relieved after rehabilitation and suggested reversing item 4, which was viewed as a large change; thus, the original item was kept for further evaluation in cognitive interviews. In addition, it was proposed that it was irrational for the ability in activities of daily living to be self-reported with a yes-or-no question; hence, the subitem “Do you have difficulty doing something” was changed to a compulsory question. Additionally, the term “clinician with an interest” was modified to “specialist”. For the content validity, the minimum of I-CVI exceeded 0.78 (0.83–1) and the AVE was 0.98 with an S-CVI greater than 0.90.

Cognitive Interviewing

Twenty-seven interviews were conducted in three iterations with stroke survivors from two hospitals and a community. Participants’ characteristics are shown in Table 1. Interviews lasted between 16 and 47 mins. Each item of the M-PSC was modified. One item was added, and one item was changed in terms of pronoun. For more details, refer to Supplemental Appendix 2. Overall, the M-PSC was regarded as comprehensive and easy to use. Problems emerging from the cognitive interviews were mainly related to comprehension and some to judgment and retrieval of information. No problems were identified regarding the response process.

Table 1 Characteristics of Participants (n=27)

Comprehension

Comprehension problems necessitated revisions. For some items, problems were related to the wording of the Mandarin translation and were solved by adapting the wording (eg, “arms,” “participation,” “medication for preventing another stroke,” “hot drinks,” “Advice on health-related lifestyle changes, such as diet and exercise,” “Item 10: since the stroke, are you finding things important to you more difficult to carry out, such as leisure activities, hobbies, work and relationships with loved ones?”) The word “arms” was first adapted to the expression and then to a clear illustration to show where arms were located.

Interviewer: what do the arms (the first expression in Mandarin: 手臂) mean to you?

P14: Back of hand. Left, back of the left hand is numb. (after the stroke)

Interviewer: Do you think this belongs to the “arms” (the second expression in Mandarin: 胳膊) too? (pointing to the shoulder)

P22: Yes, the shoulders, they belong to arms too.

In other items, clarifying explanations (eg, “lifestyle changes,” “rehabilitation therapy,” “depression,” “things important to you”) were added to improve understanding. As a special note, we initially removed all the examples from Item 10 to make it concise, but after that, the participants indicated that they were not sure what was “things important to you”. We then added two relatively appropriate explanations from the original deleted examples.

Interviewer: Can you repeat the question (Item 10) I just asked in your own words?

P10: This question is too long for me to remember … I don’t know how to understand it.

Interviewer: What do the words “things important” mean to you, in this question?

P16: Hahaha, I do not know what things are important to me. To my understanding, it means that you ask me to do some things I cannot do. It means can I do those things or not. That making money is the hardest thing to do, and I cannot earn money now. The most important thing in life is to make money. The only way to eat and have fun is to have money. What can you do to survive without money?

A few participants requested clarification for some words (eg, “medication suggestions.” “activity of daily living,” “daily living,” “activity,” “anxiety”). However, they were not modified as they did not meet the revision requirement.

Retrieval of Information

Problems with retrieving information were not pronounced but sometimes participants had difficulty answering questions that contained inappropriate assumptions (eg, “last assumption”, “loved ones,” “stiffness”). We removed the first two inappropriate assumptions. For the third, we modified it into two follow-up questions to make it more logical.

Interviewer: Is this question (Item 10) clear?

P9: A little (unclear) … because my wife passed away.

Interviewer: Since the stroke, do you have increasing stiffness in your arms, hands, and/or legs?

P13: It is improving … getting better. It is not getting stiffer. It becomes better now. eh … you are always asking for “worse and worse”.

Judgment

Judgment problems were generally related to double-barreled items (eg, Item 1, 3, 4, 6, 8, 9) and the sensitive topic (“relationship with your family”). Item splitting was used to deal with the double-barreled items. The sensitive topic was recommended to be managed in accordance with the proximity of the relationship between the medical staff and the stroke survivor’s family. If you have a good relationship with the family and can help improve it, we highly recommended that you ask the question and act on it. If not, please skip this question, as what we do in the third iteration.

Interviewer: Since your stroke or last assessment, are you having more of a problem controlling your bladder or bowels?

P3: Problems with bowel movements … but no problem with urination.

Interviewer: Does this question make you feel uncomfortable?

P1: Well, sort of, when it comes to personal space that is.

Discussion

This study described the process of translation and cross-cultural adaptation of the M-PSC by use of published consensus guidelines.¹² Adaptations according to expert review and cognitive interviews were made to support the implementation of the M-PSC in clinical practice. The content validity of the M-PSC was excellent, and overall, it was well accepted by the experts and the stroke survivors. The final M-PSC contains 11 questions, one of which is asked or not depending on the medical staff.

Experts are familiar with the targeted subjects under study and have an important role in deciding whether the pre-final translation can achieve equivalence with the source.^9,11 Many published studies have engaged the expert committee review to develop the pre-final version of survey questionnaire and establish content validity for field testing.^23–25 The PSC is unique in that it contains two sections: the specific items and the referral recommendations. In this study, the experts made important changes to the overall layout and referral recommendations of the M-PSC, and confirm its content validity. Indeed, the composition of an expert committee also requires careful consideration, as it is crucial to the achievement of cross-cultural equivalence.^9,11 It should comprise methodologists, health professionals, language professionals, and translators. Our study fulfilled this criterion with a group of four stroke clinical experts in different professions, a methodologist, a language professional, and the translators. Some minor problems were identified and the items were lightly modified at this stage.

The primary revisions of items were conducted in cognitive interviews. Every item was identified with problems, and the four-stage model covered all of the problems reported. Most problems were within comprehension, followed by others. These results match those observed in previous studies.^26,27 In many instances, participants were not aware of their misunderstandings, which were only identified through verbalization of their thought processes or by probing. The cognitive interviewing methodology could understand how targeted subjects comprehend and generate their answers to the question, aiming to reveal problems concerning comprehension, information retrieval, judgment, and response process.¹³ In this study, we refined the Mandarin translation to eliminate inappropriate wording and assumptions, unintelligible terms, and double-barreled items and proposed a revised version of the M-PSC informed by the results from cognitive interviews. The sensitive topic was retained and skipped depending on the relationship between the medical staff and the stroke survivor’s family. However, previous studies noted that some logical or structural problems could not be covered by the four-stage model.^27–29 We did not identify the same findings in the current study. It might be due to the fact that the related problems were already addressed in the expert review.

Our experience in this study suggests that the combination of forward translation, reconciliation, back translation, expert review and cognitive interviewing will be a productive line of research that may produce acceptable cross-cultural equivalence. Indeed, cognitive interviewing has been criticized for being too subjective, so future research is also needed to explore how to combine qualitative and quantitative methods in cognitive interviewing for translation and cross-cultural adaptation. We encountered an interesting problem in cognitive interviews. It is when all the items reached saturation except for one alternative subitem, and four consecutive participants reported no problems with it (criteria for saturation is that five consecutive participants not reporting problems). There is no doubt that more informants are required to reach saturation for alternative subitems. In such a situation, is it necessary to run another iteration or should the iteration be stopped to avoid waste? A further discussion on the saturation for items and alternative subitems in cognitive interviewing is needed.

Limitation of the Study

A limitation of this study is that the participants were not specifically trained for cognitive interviewing, and it was nearly impossible for them to perform a perfect “think aloud”. We used “window questions” to help them vocalize their thoughts as they responded to the question, and developed scripted probes to facilitate this process. Secondly, in-home participants were interviewed using video chat, while inpatients and outpatients’ participants were interviewed face to face. Video interviews and face-to-face interviews are both valuable methods for data collection. We chose video chat to conduct interviews with home-bound survivors to reduce their exposure during the COVID-19 pandemic. Thirdly, stroke severity was not obtained. We did consider including participants with different levels of stroke severity. However, our primary inclusion criterion was individuals who had been more than six months since the most recent stroke. For inpatients and outpatients, stroke severity as measured by National Institute of Health Stroke Scale was not available in the medical records unless participants’ most recent stroke and current visit were to the same hospital. For community participants, they were unable to provide their stroke severity through discharge summaries or other materials. Also, the percentage of individuals with cerebral hemorrhage is relatively low. On the one hand, the prevalence of cerebral hemorrhage is lower than that of cerebral ischemia. On the other hand, individuals with cerebral hemorrhage experience more neurological deficits than those with cerebral ischemia, and they are often home-bound rather than in the hospital. They prefer not to discuss the experience with others so that they can hide their stroke history, which might be due to the stigma associated with stroke. Finally, the M-PSC needs to be validated with additional research.

Conclusion

The PSC was translated into Mandarin and content validated based on expert review and cognitive interviewing. Adaptations were made to support that the M-PSC could be used in long-term post-stroke care. Before implementing the M-PSC in clinical practice, however, more validation efforts would be helpful and further studies could be performed to investigate the feasibility as well as the usability of the M-PSC in China.