Development and Initial Validity of the Patients’ Literacy Scale Among Outpatients in Hangzhou City, China

Introduction

It is almost universally acknowledged that the doctor–patient relationship (DPR) is tense in the world^1–3 and receiving increasing attention from governments and scholars.^4,5 Although the governments have legislated to prohibit violence against medical staff, the incidence of disputes between patients and medical workers is still rising.^6,7 Existing research has manifested that the current tension between doctors and patients is not only caused by doctors⁸ but also by patients.^9–11 There are few studies on the intervention of the DPR from patients’ perspective. This is because, there is a lack of appropriate measurement instruments for the effect of patient intervention. In addition, previous studies have clearly illustrated the responsibility and norms related to the role of doctors,^12,13 while the responsibility and norms of patients were still undefined. Parsons first defined the sick role (or patients’ role) as a social role, in which patients were not only the sick individuals but also people with social expectations.¹² A series of systems and social norms would strengthen these social expectations which lead patients to become more active to obtain health.¹² However, the definition of patients’ role needs to be further improved for the theory of Parsons may not be applicable to changes in the current society.¹⁴

The initiative participation of patients in the doctor–patient-related matters has been regarded as a beneficial attempt for promoting DPR,^15–17 which means that patients should be activated.¹⁸ Patient activation refers to patients with motivation, education, skills and confidence to make effective decisions to be responsible for their health.¹⁹ Patient activation requires the patients to believe the vital role in their health management and to actively obtain medical-related information and enhance their health literacy (HL). Therefore, patient activation would benefit for patients to seek medical treatment positively. It is worrying that patients’ performance cannot be determined. Relevant research has found that public adherence to control measures of some diseases was affected mainly by their knowledge, attitudes, and practices (KAP).^20,21 People with great KAP may take positive efforts to avert the spread of disease^20,22 and can also facilitate their course of diseases, especially those with chronic disease.^23,24 The status of patients’ KAP is related to response to disease prevention and control, medical treatment, and personal health,^25,26 which might be applicable in our study.

Nowadays, the threshold of seeing a doctor is higher for patients than before. On the one hand, the use of advanced medical technology has increased the information asymmetry between patients and doctors; on the other hand, social media (eg self-media, official account) has spread too much indiscriminate medical knowledge, which increased patients’ difficulties in making decisions on the contrary.²⁷ All these new changes are urging patients to improve their comprehensive consulting abilities, such as the skills of communication, information acquisition and utilization, etc. To sum up, these abilities were collectively referred to as patients’ literacy (PL) in this study. In accordance with the theory of patients’ role, patient activation, and patients’ KAP, PL is defined as the ability to initiatively obtain, read, and analyze health-related materials, make appropriate decisions, reduce risks of health-related problems. To our knowledge, there are no specific scales for PL measurement. Most researchers use scales that measure HL to assess PL.^28,29 They mainly test the ability of writing and reading or understand the prevalence among disease-specific HL of patients,^29–31 rather than test patients’ comprehensive abilities. Moreover, these scales also exists several limitations: 1) focusing on only one dimension,³² 2) focusing on only one specific disease,^29,30 3) focusing on the awareness rate of medical knowledge,^31,32 and 4) too many items to use conveniently.^29–31

In China, the DPR is also facing the serious challenges. There were average 27 violence against medical workers per year recorded in each hospital in China in 2013.³³ The Chinese government even has legislated to prohibit violence against medical staff in order to appeal to an end to infringement of the right of medical workers.⁶ However, laws and regulations, such as Regulation on the Prevention and Handling of Medical Disputes (RPHMD), only illustrate the ambiguous explanation regarding the rights of patients and lack the guidance of their responsibilities.³⁴ RPHMD requires doctors to obey all laws and regulations to avoid risk, while it only requires patients to respect doctors and offer disease information honestly which is not enough. Moreover, research on DPR in China has found that the DPR in tertiary hospitals were worse than primary hospitals,^35,36 and it is closely related to the lack of KAP among patients under the establishment of the National Hierarchical Medical System.³⁷ Many Chinese scholars have introduced and developed several scales to measure HL, for example, Chinese version of health literacy survey tool European health literacy questionnaire (HLS-Asia-Q) for general Chinese population,³⁸ cancer health literacy – 30/6 (CHLT-30/6-Chinese) for Chinese cancer patient population,³⁹ short test of functional health literacy in adults (S-TOFHLA) for Chinese to assess functional health literacy,⁴⁰ revised dental beliefs survey (DBS-R) for Chinese patients’ perception to dentist,⁴¹ etc. However, literature measuring the literacy (including KAP, activation) of patients is limited, especially outpatients. Therefore, this study combines the theory of KAP and patient activation, aiming to construct the indicator system of PL, develop the instrument of measuring PL, and prepare for future intervention studies from patients’ perspective. The specific objectives are to assess 1) internal consistency of the patients’ literacy scale (PLS), 2) item-total correlation, and 3) exploratory and confirmatory factor analysis.

Materials and Methods

Participants and Procedure

This study was conducted in two stages. The first stage was item building, which adopted the interview method; the second stage was item refinement, namely, primary evaluation of measurement properties, which included validity and reliability assessments.

Item Building

Interview

Using purposive sampling, participants (both patients and doctors) were selected in September 2019 from a comprehensive hospital in Hangzhou city of Zhejiang province in China. We organized 67 face-to-face interviews to obtain the initial items of PL, including 2 focus groups and 65 semi-structured interviews.²⁷

Each focus group lasted about 1 hour and each semi-structured interview lasted about 30 minutes. The interviews mainly focus on these questions: 1) What should the patient prepare before seeing the doctor? 2) What can patients do to make the medical procedure more efficient? 3) What does the patient do that will hinder the doctor? 4) What can a patient do to be considered as a good patient? 5) What are the aspects of the health literacy of patients?

All interviews were recorded and transcribed into manuscripts. The texts of manuscripts were analyzed by selective coding. We adopted the thematic framework analysis method, including five stages (familiarization, identifying themes, indexing, charting and summarizing, interpretation), to process the text data in excel by two researchers. The text data was marked and summarized by citing the typical expressions of the interviewees.

Item Refinement

Based on the extracted codes, categories, and subcategories of the item building stage, 21 items were prepared from the item pool. The content of items was reviewed several times, and some contents of items were replaced and modified. Finally, the initial scale for PL has been finished. The scale had 20 items and a five-point Likert scale (1= completely disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree, 5 = completely agree) was used in response options. Notably, item 2, 14, 17 on the first-version scale were designed for reverse scoring in order to screen out respondents who did not answer carefully.

Face Validity

The face validity refers to the transparency or relevance of the test from the perspective of test participants,⁴² which determines the ease of understanding of the item contents and the response rate. In this study, 20 patients of different genders (9 males vs 11 females), ages (22.90 ± 2.11 years old) and educational levels were inquired to check whether the expressions of the initial scale are easy to understand and comply with language habits. After the phase of the inquiry, researchers reworded several questions that participants raised (eg, item 1 “I have a high level of basic knowledge of the common disease” modified as “I know more about the basic knowledge of the common diseases.”). The process of face validity was repeated by those same patients, and no questions were raised.

Content Validity

Two-round assessments were carried out to confirm the content validity of the PLS. During the first round, 9 scholars, including 2 experts in the social medicine field, 2 senior physicians and 5 PhD candidates in Public Administration, were requested to evaluate the content validity of the initial version scale, such as the item grammar, wording and allocation. The second-round assessment was conducted after construct validity measurement, and we invited 7 experts (2 scholars in social medicine field, 2 linguists and 3 postgraduates in Public Administration) to check and modify the contents of the second version of PLS by conducting a focus group.

During this process, four items have been changed (items 1, 3, 6, 13). For example, item 3 “I know what the doctors who diagnose me are good at” was changed into “I will find out the specialty of my doctor in advance”. The detail is displayed in Table 1.

Table 1 The Contents of the Patients’ Literacy Scale

Construct Validity

This study used factor analysis to measure the construct validity of the scale. Bartlett’s sphericity test was performed to ascertain whether the data were suitable for exploratory factor analysis (EFA), and the Kaiser–Meyer–Olkin (KMO) index was used to test whether the sample is adequate for factor analysis (p < 0.05, KMO > 0.50).⁴³ EFA was conducted to determine the underlying structure of variables. Principal component analysis and Promax rotation were used for factor extraction and rotation matrix, respectively. We adopted two criteria, Kaiser–Guttman’s rule and the Cattell’s scree test, to confirm the number of factors that would be retained. The eigenvalue >1 was considered as the main component. Items were reserved if the factor loadings were ≥0.40.⁴⁴ After a pilot study completed by 306 patients in a comprehensive hospital in November 2020 in Hangzhou, China, item 2, 6, 7, 8, 18 was deleted (factor loadings <0.30). Therefore, the fourth version of the PLS has 15 items, and none of the items was set for the reverse scoring.

We checked and established the factor structure obtained from EFA by performing confirmatory factor analysis (CFA). The fit indices used in this study included the following: minimum discrepancy function divided by degrees of freedom (CMIN/DF, <3 good); root mean square error of approximation (RMSEA, <0.05 good; <0.10 acceptable); root mean square residual (RMR, <0.05 good); goodness-of-fit index (GFI, >0.90 acceptable; >0.95 good); comparative fit index (CFI, >0.90 good); Tucker–Lewis index (TLI, >0.90 acceptable).⁴⁵

Convergent and Discriminant Validity

Convergent validity refers to using different variables to measure the degree of similar constructs.⁴⁶ Average variance extracted (AVE) was used to assess the convergent validity, because it can explain the degree of items sharing between the construct in structural equation modeling (SEM).⁴⁷ Generally, the ideal value of AVE should be >0.50, while the acceptable value of AVE should >0.36.⁴⁸

Discriminant validity was used to test whether concepts or measurements should not be too high correlated.⁴⁹ Discriminant validity could be established by correlating one latent construct to another and when the correlation value of each construct is <0.85, the discriminant validity exists.⁴⁷

Known-Group Validity

Known-group validity is a form of construct validity in which hypotheses are pre-specified and then tested to reflect whether the tool can distinguish differences in a priori expectations.⁵⁰ If a statistical discrepancy is found, it supports the effectiveness of the instrument and if the discrepancy is not significant, the tool/item is defective, the hypothesis is defective, or the power is insufficient.⁵⁰ The one-way ANOVA with Tukey and Bonferroni post hoc analysis was conducted to evaluate the PLS among patients with different ages, genders, educational levels, and patients who went to the hospital with different levels, which included first-level, secondary and tertiary hospitals.

Reliability

The Cronbach’s alpha coefficient and McDonald’s ω were used to determine the reliability of the PLS. The value of Cronbach’s alpha >0.70 represents that scale meets the standard reliability,⁴⁵ while >0.60 represent the acceptable outcome.⁵¹ Further, we examined split-half reliability by using the Spearman-Brown formula. Our study also tested composite reliability (CR). CR refers to the combination reliability of all latent constructs, which is similar to the Cronbach’s alpha. The higher the CR values represent the higher internal consistency.⁴⁷ The ideal value of CR is >0.70, and the acceptable value of CR is >0.60.^48,52

Sample Size and Setting

We assumed a 95% confidence interval, a 0.80 power and calculated the sample size of 211 with the Power Analysis & Sample Size (PASS) free trial (NCSS, LLC, USA) software. We conducted the study in three hospitals with different levels in December 2020. The participants were drawn by accidental sampling, and questionnaires were distributed face-to-face. The inclusion criteria of our study population were that 1) outpatients themselves; 2) individuals who were willing to be involved in the study, while the exclusion were that 1) patients aged below 16 years and over 80 years; 2) accompanying family members; 3) patients with audiology communication, vision, or cognitive disorder. All questionnaires were saved at an online dataset by Wenjuanxing, a platform providing functions equivalent to Amazon Mechanical Turk.

For the completeness of the questionnaire and the convenience of patients seeking medical treatment, this questionnaire is set with a breakpoint to continue answering. Our survey received 426 questionnaires. We conducted a logic error-check inference to ensure accuracy on all submitted questionnaires. There were 123 questionnaires excluded, including 44 who failed to complete within less than 60 seconds; 79 who failed to pass the attention test, with an eligibility rate of 71.13%. Given each item should include at least 10 samples,⁵³ at least 150 respondents (15 items * 10) were considered for EFA and CFA. Finally, 152 samples were for EFA and 151 samples were for CFA by random allocation.

Ethical Statement

This study was conducted based on the Declaration of Helsinki. The Ethics Committee of Zhejiang University School of Public Health approved the protocol of this study (2019–066). All individuals voluntarily participated in our research, and each participant also signed informed consent before the interview and survey. Furthermore, the participants were informed that their names would be replaced by codes in the interview texts and questionnaire.

Statistical Analysis

Continuous variables are presented as mean and standard deviation (SD). Categorical variables are presented as numbers and frequency. Psychometric properties of the scale are reflected by validity and reliability. The random allocation and EFA were analyzed using Statistical Package for the Social Sciences (SPSS) version 22.0 (SPSS Inc., Chicago, IL, USA), with a significance level of 0.05. CFA was evaluated using Analysis of Moment Structure software (AMOS) version 21. McDonald’s ω was calculated by Jamovi software version 1.6.23.

Results

General Demographic Characteristics of Participants

Table 2 shows the samples’ demographic characteristics. For the samples by EFA, the average age was 27.21 years and 76.32% were females. Most of them received university/college education. Over half individuals (54.60%) have more than 50,000 Yuan after-tax annual income. There were 54.61% respondents reporting first visit to the hospitals, and 32.89% and 26.32% of them visit the department of internal medicine and surgery, respectively. For the samples by CFA, 68.87% were females, averaging 27.58 years old. Most participants (49.67%) were undergraduate. The after-tax annual income of less than 30,000 yuan (39.07%) and over 90,000 yuan (31.13%) accounted for the majority of the sample. Nearly half of respondents (49.01%) reported first visiting the hospital, with 27.15% and 27.81% visiting the department of internal medicine and surgery, respectively.

Table 2 Demographic Characteristics of Participants (N = 303)

Exploratory Factor Analysis

Table 3 shows the exploratory factor loadings of the PLS. The KMO measurement of the samples’ adequacy was 0.787, which was higher than the commonly recommended value of 0.600. The Bartlett’s test of sphericity was significantly lower than 0.001, indicating that the data were suitable for factor analysis, and the variables were correlated. The four factors explained 61.266% of the total variance. Factor 1 loaded five items. Factor 2 and 4 both clustered three items. Factor 3 had included four items. All factor loadings of each item were above the 0.50.

Table 3 Exploratory Factor Loadings of Patients’ Literacy Scale (n=152)

Internal Consistency Reliability

Table 4 displays the results of the item analysis of PLS. The mean scores of all items were between 2.58 and 4.36. The corrected item-total correlation ranged from 0.342 to 0.633. The test to Cronbach’s alpha and McDonald’s ω if an item deleted did not significantly improve in the overall Cronbach’s alpha and McDonald’s ω of the scale.

Table 4 Item Analysis for Patients’ Literacy Scale (n=152)

Table 5 illustrates the reliability of the overall scale and its four factors. The overall Cronbach’s Alpha and McDonald’s ω of the 15-item scale was 0.815 and 0.838, respectively. The coefficient of split-half reliability was 0.720.

Table 5 Reliability Test for Four Factors of Patients’ Literacy Scale (n=152)

Confirmatory Factor Analysis

Table 6 demonstrates the convergent and discriminant validity indices of the PLS. All of the factors could meet the criteria of CR (>0.600) and AVE (>0.360). Factor 1 was estimated the highest CR value (0.749). While factor 3 was tested the lowest AVE value (0.365).

Table 6 The Indices of the Convergent and Discriminant Validity of Patients’ Literacy Scale

Table 7 shows the model fit of five models (including four factor models and one full model) and the correlations among the factors. F1 was worse than the other models (RMSEA = 0.096, CMIN/DF = 2.389). Each item within the factors had a certain degree of correlation. The adjusted full model exhibited enough fit statistics (RMSEA = 0.065, CMIN/DF = 1.635, GFI = 0.900, CFI = 0.905, TLI = 0.909), after adjusting the crude model. The correlation value of all four factors was lower than 0.85, representing the discriminant validity existing.

Table 7 Model Fit and Corrections Among Constructs of Patients’ Literacy Scale

The final model had good known-group discriminant validity (Table 8). The total score had significant differences among patients of different ages, hospital levels and educational levels (p< 0.05). The PL was positively related to patients’ age. The gender difference between men and women is only reflected in F1. While patients who received high school or below education had higher PL than those who were undergraduate or graduate and over. Our study also found that the mean total score of PLS was positively correlated to patients’ age. Patients diagnosed in community hospitals or tertiary hospitals had higher patient literacy than those in secondary hospitals.

Table 8 The Known-Group Validity of Patients’ Literacy Scale

Discussion

This study first discussed the concept of PL which was different from HL, because the trend of patients’ role is changing from passive recipients to active participants currently.⁵⁴ The health system also places more and more emphasis on patients in order to design and offer better health services.⁵⁵ Thus, the concept of PL should stress the initiative and importance of patients. Based on previous studies and the theory of patients’ role, we finally concluded that PL should include the individual’s ability to use healthcare resources wisely, choose appropriate and proportionate healthcare services, respect healthcare professionals, and beliefs and confidence for self-managing health, which refers broadly to the patient performance that deserves recognition. The PLS was developed based on the concept.

The PLS strictly followed the scale development and construction process. The final version of the scale with 15 items was confirmed by experts’ reviews. Moreover, the PLS is relatively easy to administer in 5 minutes, for it concerns all real situations that patients would encounter. The PLS can serve as a quick evaluation instrument to identify PL. To ensure the PLS’ content validity, we invited two-round experts to verify and employed two-round patients to identify the face validity. KMO and Bartlett’s test of sphericity demonstrated satisfactory sampling adequacy and suitability of data for EFA. Four factors with the eigenvalue higher than one were produced by EFA, with identified factors explained over 60% of the variance among all items in PLS. This finding demonstrates that PLS contains sufficient items to satisfactorily measure essential factors associated with PL.⁴³ Thus, this scale has good validity and reliability.

According to EFA, the PLS included four dimensions, and we defined F1 to F4 as “patient activation”, “knowledge”, “attitudes”, “practice” by the theory of KAP and patient activation. The “patient activation” was identified by the theory of patient activation,¹⁸ which can reflect the proactive behavior, self-management intention and the self-care ability of patients before and in the process of seeing a doctor, affecting the patient’s experience of seeing a doctor. It has been proven that describing the condition accurately and empathizing with doctors are skills that patients can learn to help smooth doctor–patient communication.^56,57 The “knowledge” was concluded by the KAP model, which reflects the knowledge level of participants, and can directly affect their attitudes to deal with medical practices. Proper knowledge, especially certain legal knowledge related to health and medical practice may help reduce the medical dispute, which is worth popularizing to patients. The “attitudes” was obtained by the KAP model, which reflects the degree of trust and understanding of patients to doctors. Trust is a fundamental component of the DPR and is related to increased satisfaction, adherence to treatment, and continuity of care.⁵⁸ Therefore, it is essential to know and increase the trust of the patients in doctors. In addition, as the modern healthcare system shifts to digital platforms and channels, patients’ trust in the official health information from these sources is also crucial.⁵⁹ The “practice” also concluded from the KAP model, reflecting the positive or negative attitudes of patients before seeking medical treatment. Previous studies have found that practice, like patients prepared before seeing the doctors can help doctors make better decisions and guidance.^60,61 All these dimensions can not only measure PL but also evaluate the efficiency of doctor–patient communication.

In this study, the indicators of goodness-of-fit by CFA showed that the model fit of the structural equation model of PLS is appropriate. The known-group validity was supported among patients of different ages, who went to hospitals with different levels and patients with different educational levels. Understandably, PL becomes higher as the age increases, because people will become more experienced, mature, and accountable in coping with health-related problems as they get older. We found that there was no gender difference in PL. It is interesting that patients with higher education have lower PL. We speculated that it might result from psychological entitlement, which refers to a stable and universal subjective belief or perception that one feels entitled to preferential treatment and is exempt from social responsibility.⁶² People with higher psychological entitlement may be more likely to do negative actions.⁶³ For example, people with higher education had a higher level of psychological entitlement.⁶⁴ Therefore, higher educated people are more likely to think that they should receive preferential treatment compared to lower educated ones in the process of seeing a doctor, resulting in low scores in the “patient activation” and “practice” dimension of PL. They may also have higher expectation of medical outcomes, which means they can better understand the medical information obtained from doctors but ignore other information like medical entanglement, resulting in low scores in the PL dimension of “knowledge”. Moreover, patients with higher educational levels which are closer to the doctors’ educational levels, which shorten the entitlement distance with the doctor, and may lose the respect to doctors, resulting in low score in the “attitudes” dimension of PL. More studies should be conducted to confirm this speculation in the future. In this study, patients who go to the first-level hospital or the third-level hospital have higher PL than those who go to secondary hospitals, because most patients who go to the first-level hospitals are mildly ill, and they may consult for some medicines, while patients who go to the third-level hospitals usually have more severe illnesses, so they may know more about their diseases and prepare in advance.

The purpose of this study is not just developing the scale of PL but also intended to increase the PL of all kinds of patients. However, increasing PL is a form of mindset shifting, which expresses in many dimensions from patient activation to KAP. Mindset shifting relies on subjective information filtering (choosing what to accept or reject), indicating that it may be more beneficial for researchers to use the PLS supported by an information processing approach.⁶⁵ To increase PL, governmental and public supports also can not be neglected.⁶⁶ Currently, the PL of patients in China is worrying. Therefore, we should take the cost of future applications of PLS into consideration, in order to further improvement and promotion of PL among patients. The limitations of this study are as follows. First, the representation of samples was limited, because samples were collected in outpatient halls in three hospitals with different levels in Hangzhou. Thus, the scale still needs to verify in other regions. The scale’s validity and reliability also need more evidence in other countries with different cultures. Second, we did not measure the test–retest reliability of the scale, for it was hard to find the same respondents again. Third, our study used a self-report questionnaire with certain recall bias. Forth, we correlated the error terms to obtain the better model fit in the PLS, which is not recommended during model construction. We will collect more data samples to improve the model fit in the future. Fifth, we did not evaluate the concurrent validity, because there is no similar type of or related scale for comparison.

Conclusion

In conclusion, the PLS is the first to comprehensively measure the literacy of the patients. It is an instrument with acceptable validity and reliability. Future research could validate among large-scale samples, and further apply in clinical research and practice for better care. The 15-item PLS could not only benefit for researchers but also for both patients and doctors. On the one hand, researchers can evaluate the effects in design, implementation and interventions of patient-related studies by using PLS. On the other hand, it also can be used as a standard and advocacy target for patients’ behavioral model which can further improve the DPR.