Instruments for Patient Education: Psychometric Evaluation of the Expected Knowledge (EKhp) and the Received Knowledge of Hospital Patients (RKhp)

Introduction

The importance of empowering patient education and patient’s own active engagement has been identified internationally at a strategic level.^1–4 In the future, its importance will be even greater due to changes in health-care organizations, shorter contacts with professionals, and increasing options for patients to select and influence their care and treatment. Patient and public involvement in health care, such as patient-centeredness, individualized services and participation in decision-making, is emphasized.^2–4 Patient education is a key factor in the care of many health problems,² such as with orthopedic patients as their hospital stays are reduced and their number is increasing.⁵ Thus, evaluation of patients’ knowledge processed in empowering patient education is well founded, and there is an international need for instruments for this purpose in health care.

Patient education is one of the main nursing actions, and it has been connected with positive health outcomes^6–10 and cost-effectiveness.² Patient education is reciprocal, reflective action.^11,12 With patient empowerment being the main goal, the evaluation of actions is mainly based on patients’ preferences.^13–15 To support empowerment, multidimensional knowledge is needed.^13,16–19

Studies have demonstrated an increase in patients’ knowledge levels through educational sessions.^20–25 However, health-care professionals’ skills in empowering patient education still need improvement.^{5,14,15,26,27} More comprehensive, multidimensional instruments for evaluation of the knowledge processed in empowering patient education are needed.¹⁴ For measurement purposes, we have developed parallel instruments to measure the expected knowledge^28–30 as well as received knowledge of patients^16,31,32 to identify barriers to patient empowerment by indicating the unfulfilled knowledge expectations. We use the concept of knowledge to emphasize the role of patient education as comprising not only distribution of information to patients but also providing information that can be understood and integrated as part of their cognition and actions.^15,16,24

Development of the Instruments Used in the Study

The theoretical background of the instruments Expected Knowledge of Hospital Patients (EKhp, ©Leino-Kilpi, Salanterä, Hölttä 2003) and Received Knowledge of Hospital Patients (RKhp, ©Leino-Kilpi, Salanterä, Hölttä 2003) lies in the concept of empowerment as the main goal of patient education. Empowerment can be defined as the experience of control to know, influence and decide over one’s own life – in this case, a healthy life in particular. It enables self-management, such as problem-solving and decision-making, creating an experience of independence and power.^17,33–36

Empowerment has several characteristics that are necessary to specify for measurement purposes. Leino-Kilpi et al³⁷ have introduced the elements of empowerment. Based on literature and a questionnaire with open-ended questions (n=226 patients with chronic disease) discovering the meaning of illness, control over one’s own health and means of support, there are six elements of empowerment:^{13,16,25,37–39} bio-physiological, functional, experiential, ethical, social, and financial element. The bio-physiological element is concerned with aspects such as illness, symptoms and treatment. The cognitive element is about knowledge of one’s own health and health problems, and the ability to gain, use and evaluate that knowledge. The functional element can be defined as functions of one’s own mind and body, eg, mobility, rest, nutrition, and having the strength and ability to act in relation to the health problem. The experiential element is based on earlier experiences of power and management as well as the emotions connected to them. The ethical element is defined as experience of being valued and respected as an individual, and the feeling of the motive behind the care is to ensure one’s safety. The social element is about power of social interaction with families, other patients and patient unions, for example. The financial element includes the costs and benefits of health or illness and their connection with self-management.^14,33,37,40 The elements of empowerment have been tested and used in empowering patient education in numerous nursing contexts.^41–53 The structure and items of EKhp and RKhp are based on these elements (40 items divided into six elements of empowerment).^33,39 The instruments were created to evaluate the knowledge expectations and received knowledge of patients in the context of empowering patient education. The theoretical hypothesis is that received knowledge corresponding to expected knowledge supports and gives potential for the empowerment of the patient.⁵⁴

For structuring the instruments, in each subscale, items of expected knowledge or received knowledge were formulated, and content validity was established by a variety of subject matter experts (4 nurses, 4 nurse leaders, 4 nurse researchers, and 4 patients). The instruments were originally developed and piloted in surgical hospital care in collaboration with nurses in clinical practice.^16,29,55 EKhp is aimed at evaluating the level of expected knowledge on admission to hospital (in wards or outpatient clinics), and RKhp at evaluating the knowledge received by patients, measured at the end or after their hospital stay.^29,30 The instruments are completed by patients, significant others, or in some cases, nurses. Response options are on a 4-point Likert scale (1=fully disagree, 2=disagree to some extent, 3=agree to some extent, 4=fully agree), with higher score indicating higher knowledge expectations (EKhp) or fulfillment of received knowledge (RKhp). The option “Not applicable in my case” is also included. In some studies, the Likert scale is inverted (Table 1).^16,29,56

Table 1 Reported Findings of the EKhp and RKhp Instruments

In 2003, the EKhp and RKhp instruments were piloted and tested for the first time with 237 adult surgical patients.^16,29 After that, the instruments have been used jointly in Finland for orthopedic²⁸ and cancer patients.³⁰ In studies utilizing the instruments separately, EKhp has been used for heart failure patients scheduled for cardiac resynchronization therapy implantation in Iceland⁵⁷ and for patients with cancer,⁵⁸ knee osteoarthritis,⁵⁹ and prostate cancer in Finland.⁶⁰ RKhp has been used in Finland for patients in ambulatory surgical nursing,³¹ total joint arthroplasty,³² surgical nursing,^55,61,62 medical wards,⁶³ and for family members and nurses in surgical nursing.⁶¹ Two parallel instruments to EKhp and RKhp have been used to discover expected knowledge (EKso) and received knowledge of significant others (RKso) (Table 1).^64–68

The respondents in the national studies (n=53–480, Table 2) were adult surgical patients, representing both genders, with a mean age of 44–72 years (SD 9–17), mostly employed (12–77%) or retired (17–85%).^{16,28–32,57–60,62,63} In all of the studies, validity and reliability of EKhp and RKhp have been evaluated, content validity has been confirmed by experts, and internal consistency of the instruments has been sufficient. Internal consistency of the EKhp instrument has been reported in several studies with Cronbach’s alpha ranging from 0.80 to 0.98 and subscales between 0.77 and 0.96,^{28–30,58–60} while that of the RKhp instrument has been reported to range from 0.80 to 0.99, and subscales between 0.76 and 0.98 (Table 1).^{16,28–32,62,63}

Table 2 Characteristics of Respondents in Studies Using the EKhp and RKhp Instruments

The instruments have also been used for multiple purposes. For example, EKhp has been utilized in the development of an inter-professional screening instrument for cancer patients’ education process⁶⁹ and in a project evaluating patients’ expected knowledge by hospital patients themselves (n=547) and by nurses (n=155) in several different medical wards.⁷⁰ Moreover, several modified instruments have been made and used based on EKhp and RKhp,^71–76 and an instrument assessing the content of provided patient education (EPNURSE) includes parts of EKhp and RKhp.^66,68,77 The instruments and their variations have also been used in academic master’s theses in various nursing contexts^78–88 and in numerous bachelor’s theses.

For international purposes, the instruments were translated from Finnish to English by native language experts using a standard double-checking translation procedure.^89,90 The English version was confirmed by the research team and was then used as original version for international use and for translations in participating countries. The translation procedures in every participating country were also conducted according to standard forward-back translation procedure and were evaluated and confirmed by local researchers in each country. Possible differences and problems were discussed and any amendments accepted by the international research team partners.

In this study, we analyze the psychometrics of the EKhp and RKhp instruments based on international data collected in seven countries. The ultimate goal is to develop the instruments to support international researchers in the field of patient education to find tested, valid instruments for their purposes, and finally, to support the availability of tailored education for patients in different countries.

Patients and Methods

Data Collection

Data for the project Empowering Surgical Orthopedic Patients Through Education (ESOPTE) were collected in countries from the southern (Cyprus, Greece, Spain), northern (Finland, Iceland, Sweden) and central (Lithuania) parts of Europe. Participating countries were chosen to represent a large geographical area and various types of health-care systems.⁶⁶ The data were collected in one to five hospitals in each country. The inclusion criterion for hospitals was general or university hospital performing surgical orthopedic knee or hip replacement procedures. Health-care professionals provided patient education according to the educational model of each country consisting of similar content consisting of a) the upcoming surgery, b) patients’ self-management and symptom management, c) recovery, and d) health-related quality of life.

The instruments, with cover letters to respondents as well as informed consent forms, were piloted in each country in a corresponding group of patients (30 patients per country). The partners agreed to collect the data from orthopedic patients by using the EKhp and RKhp according to the shared protocol. Designated contact persons in every hospital supported the data collection, ie, distributed the instruments and returned them. EKhp was given to patients prior to the patient’s admission, and RKhp at the end of hospital care. Completed questionnaires were returned to the mailboxes placed on the wards or sent back by mail in a prepaid envelope. Data collection was performed during the years 2009–2012 and results have been reported in 15 international studies (Table 1).^{54,56,61,64–68,91–97} In this paper, the combined results of all countries form the basis of the analysis.

Sample

The required international sample size was at least 1,540 patients from all participating countries together, based on power calculation of the instruments with a power level of 0.9 and 0.8 differences of mean scores with 0.95 standard deviation within groups at the significance level 0.01.⁹³ Minimum was 220 patients per country. Inclusion criteria for patients attending orthopedic knee or hip replacement surgery for their osteoarthritis disease were 1) ability to understand Finnish/Greek/Icelandic/Lithuanian/Spanish/Swedish, 2) ability to complete the questionnaires independently (or with help of significant others), 3) 18 years of age or older, 4) no diagnosed cognitive disorders, and 5) volunteer to participate in the study.

Data Analysis

Data were analyzed using SPSS 22 (IBM Corporation Chicago, IL, USA) statistical software. Descriptive statistics (frequencies, percentages, mean, standard deviation, range) were calculated to describe the sample characteristics and main variables. Sum-variables were formulated based on the six theoretical elements of empowerment in the instruments. Internal homogeneity of the scales was evaluated using Cronbach’s alpha coefficients (with the criterion of α ≥ 0.70).

Furthermore, data were also imported to Mplus version 7.11⁹⁸ for modeling the expected knowledge and received knowledge scale associations using structural equation analysis. First, to determine whether the data fit the hypothetical model, values of the Chi-Square Test of Model Fit, with degrees of freedom (df) and p-value, were calculated. This statistic assesses the model by comparing the Χ² value of the model to the Χ² of the null model.⁹⁹ Chi-square “assesses the magnitude of discrepancy between the sample and fitted covariances matrices”,¹⁰⁰ and there should be an essence of a statistical significance.⁹⁹ As this parameter is known to be influenced by large sample size,⁹⁹ alternative incremental fit indices were also used for the evaluation of the model fit: comparative fit index (CFI, with the criterion of CFI ≥ 0.95),¹⁰⁰ Tucker–Lewis index (TLI, with the criterion of ≥0.95), root-mean-square error of the approximation (RMSEA, with the criterion of <0.06,¹⁰⁰ or a stringent upper limit of 0.07)¹⁰¹ and the standardized root-mean-square residual (SRMR; criterion <0.08). Akaike’s Information Criterion (AIC), used to assess the suitability of the model if several models are estimated with the same data, indicates to the researcher which of the models is the most parsimonious.¹⁰²

Results

Patient Characteristics

A total of 1,595 (76%) hospital patients completed EKhp and 1,343 (64%) completed RKhp. The respondents comprised Icelandic (n=273, 17%), Swedish (n=268, 17%), Spanish (n=260, 16%), Finnish (n=253, 16%), Greek (n=207, 13%), Lithuanian (n=170, 11%) and Cypriot patients (n=164, 10%). The mean age of the respondents was 67 years (SD 10.6, range 25–91). Most of the respondents were female (62%) with vocational education (51%) in addition to primary school as basic education (51%). Over half of the respondents were retired (57%) and nearly a third were employed (27%), with 16% having employment history in health care or social services. Most respondents were admitted to hospital for knee arthroplasty (knee 63%, hip 37%) and had a history of treatment in the same hospital (73%). Forty-six percent of the respondents had a chronic illness, and a minority had had previous arthroplasty or arthroplasties (22%).

Content Validity

The content validity of the EKhp and RKhp instruments was judged with structured criteria including assessment of the instruments’ and subscales’ sufficiency of concreteness for measurement, focus on the expected and received knowledge, similarity to other subscales, items belonging to the subscale, importance in the expected or received knowledge, relevance to the expected or received knowledge, and clarity, coverage, and uniqueness of the instruments.¹⁰³ Content validity was assessed in every country by expert panels set up by the countries themselves. EKhp and RKhp were evaluated as decent and no changes to the instruments were proposed.

Structural Validity

Confirmatory Factor Analysis in Countries

Data from every participating country were used to test the structural validity of the instruments with confirmatory factor analysis. One latent variable model was built for each country to find out the R-square of each subscale (Table 3). For EKhp, the lowest R-squares were achieved for bio-physiological and financial subscales while the highest were achieved for ethical, functional and social subscales. For RKhp, the lowest R-squares were achieved for bio-physiological, functional and financial subscales, and the highest for ethical subscale.

Table 3 R-Squares of Subscales of the EKhp and RKhp Instruments

Confirmatory Factor Analysis for Seven Countries Together

The confirmatory factor analysis of EKhp was performed by first fitting the hypothetical model in the sample of 1,595 patients. The hypothetical model proposed for this study did not demonstrate a good fit with data derived from the sample of patients in seven European countries. Chi-square was 372.006 (df 9) and was statistically significant (p <0.001) not reaching the requirement.⁹⁹ RMSEA was 0.159 (p <0.001), and CFI of 0.950 and TLI of 0.916 were found. Thus, all of the fit indices demonstrated a need for improvement of the model.

A revised model of EKhp included error covariance between bio-physiological and functional subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 135.66 (df 8) and was significant (p <0.001) not reaching the criteria.⁹⁹ CFI of 0.982 and TLI of 0.967 indicated a good fit of the data to the slightly revised model. RMSEA was 0.100 (p <0.0001), not indicating not a good fit (stringent upper limit of 0.07).¹⁰¹

Table 4 Modeling of the EKhp and RKhp Instruments

The second revised model of EKhp included error covariance between bio-physiological and functional subscales as well as between experiential and functional subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 51.73 (df 7) and was significant (p <0.001) not reaching the criteria.⁹⁹ CFI of 0.994 and TLI of 0.987 indicated a good fit of the data to the slightly revised model. RMSEA was 0.063 (p =0.08) indicating a good fit (stringent upper limit of 0.07).¹⁰¹

The third revised model of EKhp (Figure 1) included error covariance between bio-physiological and functional subscales, between experiential and functional subscales, as well as between ethical and financial subscales. The model fit improved markedly and fitted well (Table 4). The Chi-Square Test of Model Fit was 22.78 (df 6) and was significant (p <0.001) not reaching the criteria.⁹⁹ CFI of 0.998 and TLI of 0.994 indicated a good fit of the data to the slightly revised model. RMSEA was 0.042 (p =0.739) indicating a very good fit (stringent upper limit of 0.07).¹⁰¹

Figure 1 The third revised model of EKhp.

Confirmatory factor analysis of RKhp was performed by first fitting the hypothetical model in the sample of 1,343 patients. The hypothetical model proposed for this study did not demonstrate a good fit with data derived from the sample of patients in seven European countries. Chi-square was 650.00 (df 9) and was statistically significant (p <0.001) not reaching the requirement.⁹⁹ RMSEA was 0.228 (p <0.001), and CFI of 0.910 and TLI of 0.850 were found. Thus, all of the fit indices demonstrated a need for improvement of the model.

The revised model of RKhp included error covariance between bio-physiological and functional subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 190.23 (df 8) and was significant (p <0.001) not reaching the criteria.⁹⁹ CFI of 0.974 and TLI of 0.951 indicated a good fit of the data to the slightly revised model. RMSEA was 0.130 (p <0.0001) indicating not a good fit (stringent upper limit of 0.07).¹⁰¹

The second revised model of RKhp included error covariance between bio-physiological and functional subscales as well as between social and financial subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 91.120 (df 7) and was significant (p <0.001) not reaching the criteria.⁹⁹ CFI of 0.988 and TLI of 0.974 indicated a good fit of the data to the slightly revised model. RMSEA was 0.095 (p <0.0001) indicating not a good fit (stringent upper limit of 0.07).¹⁰¹

The third revised model of RKhp (Figure 2) included error covariance between bio-physiological and functional subscales, between financial and ethical, as well as between social and financial subscales. The model fit improved markedly and fitted well (Table 4). The Chi-Square Test of Model Fit was 34.393 (df 6) and was significant (p <0.001) not reaching the criteria.⁹⁹ CFI of 0.996 and TLI of 0.990 indicated a good fit of the data to the slightly revised model. RMSEA was 0.059 (p =0.187) indicating a good fit (stringent upper limit of 0.07).¹⁰¹

Figure 2 The third revised model of RKhp.

Reliability

Reliability of the instruments was demonstrated with internal consistency with Cronbach’s alpha values. In the total EKhp instrument, Cronbach’s alphas between the countries ranged from 0.87 (Iceland) to 0.99 (Lithuania). Subscales ranged between 0.8 and 0.98 in all countries, with the exception of the ethical subscale in Iceland, which was 0.57. Iceland had the lowest values in every subscale (0.57–0.83) except for the financial subscale (0.9).

Between the countries, the Cronbach’s alpha for the total RKhp ranged from 0.97 (Finland and Lithuania) to 0.99 (Greece). In subscales, the lowest values were seen in the functional subscale (in Finland, Greece, Iceland and Sweden, range in all countries 0.83–0.95) and the highest in the financial subscale, being the highest value in several countries: Finland, Greece, Spain and Sweden (range in all countries 0.87–0.99).

Discussion

The aim of this study was to analyze the psychometrics of the EKhp and RKhp instruments. This is the first parallel international use of EKhp and RKhp and the evaluation of their psychometrics. Our basic assumption is that instruments for assessment of patient education are necessary due to the current and increasing importance of empowering patient education on European level.^1,4 Meeting the knowledge expectations of patients in education is crucial for patient empowerment. Results of earlier studies using EKhp and RKhp suggest discrepancies between the expected and received knowledge, giving rise to concern about individually tailored empowering patient education.⁹⁴

EKhp and RKhp are parallel instruments: development of such types of instruments is rather complicated, and they are not very common in nursing. EKhp and RKhp were developed based on the theoretical assumption of correspondence between patients’ expectations and received knowledge giving more potential for patient education to be empowering for individual patients. This assumption has received support,^9,104 but the connection still needs further testing, especially the connection between expected and/or received knowledge and their relationship with nursing outcomes. This kind of testing would require both narrative designs analyzing patients’ experiences and randomized clinical trials in patient education. Furthermore, we also need more studies based on evidence analyzing the ethical element of empowering knowledge, especially the connection between patient’s right to know and empowering patient education.

Based on the results, EKhp and RKhp are proposed as international instruments with satisfactory psychometric qualities for the evaluation of empowering patient education. In confirmatory factor analyses, some error covariances were found. These relationships were also found in the Spanish versions.¹⁰⁵ However, R-square values of subscales were satisfactory, the instruments enabled data collection in all seven countries, and the instruments have been tested with numerous respondents before this international study. However, it is clear that we need to be aware of the constant changes in patients’ responsibilities for their care and treatment in order to enable support to the empowerment of patients, and the structure of the instruments must be modified accordingly. Furthermore, this study was the first one using EKhp and RKhp in most of the participating countries, and potential country-specific factors need further investigation for drawing country-specific conclusions. For instance, in Spain, the financial subscale in both instruments obtained good psychometric values; however, the healthcare context in Spain is mainly based on the public health-care system where patients are not aware of direct costs and therefore, patients do not expect to receive information on financial issues. Nevertheless, both instruments demonstrated ability to capture the expected and received knowledge in orthopedic patients regardless of any country-specific confounding factors. Therefore, our aim to develop and test the instruments for the common European health-care market was successful, and the instruments seem promising for that purpose.

In this study, the instruments demonstrated sufficient internal consistency measured by Cronbach’s alpha. Few alpha values of 0.99 were rather high, indicating that the instruments could be shortened in the future.¹⁰⁶ Based on the analysis, however, no specific item in the current form of the instruments could be identified. In the Icelandic data, the ethical subscale in EKhp received a lower alpha (0.57). The Iceland data set is rather small, and due to the possible cultural reasons behind this finding, testing needs to continue. The ethical subscale of EKhp demonstrated sufficient consistency in other Nordic countries (Finland and Sweden) which are culturally fairly similar to Iceland.¹⁰⁷

International comparison of empowering patient education presents many challenges. Patient education differs between countries and cultures due to numerous national factors. For example, in some countries, patient education is done with the use of rather static information material in the form of leaflets while in others, this training takes a more structured face-to-face format. Other differing factors might include the points in time when and to what extent the training is delivered. In this study, the assumption is that there is correspondence between countries in the care of orthopedic patients and their education. The earlier published results from this study data have demonstrated similarities internationally; for example, in patient education provided by nurses⁶⁶ and in differences between the expected and the received knowledge in patient education.⁹⁴ It is clear, however, that even though the items of the instruments are the same, the patients in different countries could interpret and understand them differently. Nevertheless, international research is important especially in the European region where patients travel across countries, health-care professionals are in the common labor market, and health strategies are shared.¹ Thus, it is beneficial to continue to produce information about patient education broadly, across national boundaries. Furthermore, the generation of high-quality cross-border information ensures that the level of training is kept at a high standard. In the future, concept analysis of empowering patient education could provide additional information on this matter. EKhp and RKhp can be advantageous premises for such research.

Instrument development is a long, systematic, and multi-phase process.^108,109 Psychometric evaluation of EKhp and RKhp on national and international levels has demonstrated validity and reliability of the instruments. However, further testing of the instruments is warranted due to the constant changes in health care. International research communities are invited to join in the testing of EKhp and RKhp to improve empowering patient education globally.

Limitations

This study has limitations in terms of generalization. Generalization of the results can be applied to hip and knee arthroplasty patients in Europe. The patients were from seven different countries and from many hospitals across the Europe; with caution, the results can thus be applied more generally to arthroplasty patients. Patient education in surgical care contexts is assumed to have many similarities;⁶⁶ the results may therefore be applicable to other surgical patient groups as well. Outside the context of surgical patients, the instruments have been used successfully in national studies; for example, with patients with cancer or cardiac disease. In the future, international testing of the instruments will need to be applied to other patient groups.

Conclusion

Based on the results of this study, EKhp and RKhp are promising instruments for international use by researchers and practitioners in the field of hospital care, especially in the field of surgical care. When the instruments are used jointly, they can produce unique information regarding the results of patient education from patients’ perspective: EKhp indicates expected knowledge, that is, the initial situation before patient education, while RKhp uncovers received knowledge. Information provided by the instruments gives potential to support the empowerment of patients. The instruments can be applied in multiple ways in nursing, health care, and research to discover, for example, the effectiveness, quality, or patient-centeredness of patient education.

Recommendations

For empowering patients with educational activities, validated instruments for measuring patients’ expectations (EKhp) and received (RKhp) knowledge are useful. They can be used internationally both in clinical work and research. Instruments are fitting particularly for hospital care, and testing within other contexts in future is needed.