Instruments for Patient Education: Psychometric Evaluation of the Expected Knowledge (EKhp) and the Received Knowledge of Hospital Patients (RKhp)
Received 10 July 2020
Accepted for publication 28 September 2020
Published 10 November 2020 Volume 2020:13 Pages 1481—1505
Checked for plagiarism Yes
Review by Single anonymous peer review
Peer reviewer comments 2
Editor who approved publication: Dr Scott Fraser
Helena Leino-Kilpi,1,2 Saija Inkeroinen,1 Esther Cabrera,3,4 Andreas Charalambous,1,5 Natalja Fatkulina,6,7 Jouko Katajisto,8 Árún K Sigurðardóttir,9,10 Panayota Sourtzi,11 Riitta Suhonen,1,2,12 Adelaida Zabalegui,13,14 Kirsi Valkeapää1,15
1Department of Nursing Science, University of Turku, Turku, Finland; 2Turku University Hospital, Turku, Finland; 3School of Health Sciences, TecnoCampus, University Pompeu Fabra, Barcelona, Spain; 4Department of Care Management and Social Work, Sechenov University, Moscow, Russia; 5Nursing Department, Cyprus University of Technology, Limassol, Cyprus; 6Department of Nursing, Klaipeda University, Klaipėda, Lithuania; 7Institute of Health Sciences, Faculty of Medicine, Vilnius University, Vilnius, Lithuania; 8Department of Mathematics and Statistics, University of Turku, Turku, Finland; 9School of Health Sciences, University of Akureyri, Akureyri, Iceland; 10Akureyri Hospital, Akureyri, Iceland; 11Faculty of Nursing, National and Kapodistrian University of Athens, Athens, Greece; 12Welfare Division, City of Turku, Turku, Finland; 13Hospital Clinic of Barcelona, Barcelona, Spain; 14Department of Nursing, Universitat de Barcelona, Barcelona, Spain; 15Human Performance Division, Finnish Defense Research Agency, Järvenpää, Finland
Correspondence: Saija Inkeroinen
Department of Nursing Science, University of Turku, Turku 20014, Finland
Tel +358 400760698
Email [email protected]
Purpose: In patient education, there is a need for valid and reliable instruments to assess and tailor empowering educational activities. In this study, we summarize the process of producing two parallel instruments for analyzing hospital patients’ expectations (Expected Knowledge of Hospital Patients, EKhp) and received knowledge (Received Knowledge of Hospital Patients, RKhp) and evaluate the psychometrics of the instruments based on international data. In the instruments, six elements of empowering knowledge are included (bio-physiological, functional, experiential, ethical, social, and financial).
Patients and Methods: The original Finnish versions of EKhp and RKhp were tested for the first time in 2003, after which they have been used in several national studies. For international purposes, the instruments were first translated into English, then to languages of the seven participating European countries, using double-checking procedure in each one, and subsequently evaluated and confirmed by local researchers and language experts. International data collection was performed in 2009– 2012 with a total sample of 1,595 orthopedic patients. Orthopedic patients were selected due to the increase in their numbers, and need for educational activities. Here we report the psychometrics of the instruments for potential international use and future development.
Results: Content validities were confirmed by each participating country. Confirmatory factor analyses supported the original theoretical, six-dimensional structure of the instruments. For some subscales, however, there is a need for further clarification. The summative factors, based on the dimensions, have a satisfactory internal consistency. The results support the use of the instruments in patient education in orthopedic nursing, and preferably also in other fields of surgical nursing care.
Conclusion: EKhp and RKhp have potential for international use in the evaluation of empowering patient education. In the future, testing of the structure is needed, and validation in other fields of clinical care besides surgical nursing is especially warranted.
Keywords: empowerment, nursing, patient education as topic, patient participation, patient-centered care, surveys and questionnaires
The importance of empowering patient education and patient’s own active engagement has been identified internationally at a strategic level.1–4 In the future, its importance will be even greater due to changes in health-care organizations, shorter contacts with professionals, and increasing options for patients to select and influence their care and treatment. Patient and public involvement in health care, such as patient-centeredness, individualized services and participation in decision-making, is emphasized.2–4 Patient education is a key factor in the care of many health problems,2 such as with orthopedic patients as their hospital stays are reduced and their number is increasing.5 Thus, evaluation of patients’ knowledge processed in empowering patient education is well founded, and there is an international need for instruments for this purpose in health care.
Patient education is one of the main nursing actions, and it has been connected with positive health outcomes6–10 and cost-effectiveness.2 Patient education is reciprocal, reflective action.11,12 With patient empowerment being the main goal, the evaluation of actions is mainly based on patients’ preferences.13–15 To support empowerment, multidimensional knowledge is needed.13,16–19
Studies have demonstrated an increase in patients’ knowledge levels through educational sessions.20–25 However, health-care professionals’ skills in empowering patient education still need improvement.5,14,15,26,27 More comprehensive, multidimensional instruments for evaluation of the knowledge processed in empowering patient education are needed.14 For measurement purposes, we have developed parallel instruments to measure the expected knowledge28–30 as well as received knowledge of patients16,31,32 to identify barriers to patient empowerment by indicating the unfulfilled knowledge expectations. We use the concept of knowledge to emphasize the role of patient education as comprising not only distribution of information to patients but also providing information that can be understood and integrated as part of their cognition and actions.15,16,24
Development of the Instruments Used in the Study
The theoretical background of the instruments Expected Knowledge of Hospital Patients (EKhp, ©Leino-Kilpi, Salanterä, Hölttä 2003) and Received Knowledge of Hospital Patients (RKhp, ©Leino-Kilpi, Salanterä, Hölttä 2003) lies in the concept of empowerment as the main goal of patient education. Empowerment can be defined as the experience of control to know, influence and decide over one’s own life – in this case, a healthy life in particular. It enables self-management, such as problem-solving and decision-making, creating an experience of independence and power.17,33–36
Empowerment has several characteristics that are necessary to specify for measurement purposes. Leino-Kilpi et al37 have introduced the elements of empowerment. Based on literature and a questionnaire with open-ended questions (n=226 patients with chronic disease) discovering the meaning of illness, control over one’s own health and means of support, there are six elements of empowerment:13,16,25,37–39 bio-physiological, functional, experiential, ethical, social, and financial element. The bio-physiological element is concerned with aspects such as illness, symptoms and treatment. The cognitive element is about knowledge of one’s own health and health problems, and the ability to gain, use and evaluate that knowledge. The functional element can be defined as functions of one’s own mind and body, eg, mobility, rest, nutrition, and having the strength and ability to act in relation to the health problem. The experiential element is based on earlier experiences of power and management as well as the emotions connected to them. The ethical element is defined as experience of being valued and respected as an individual, and the feeling of the motive behind the care is to ensure one’s safety. The social element is about power of social interaction with families, other patients and patient unions, for example. The financial element includes the costs and benefits of health or illness and their connection with self-management.14,33,37,40 The elements of empowerment have been tested and used in empowering patient education in numerous nursing contexts.41–53 The structure and items of EKhp and RKhp are based on these elements (40 items divided into six elements of empowerment).33,39 The instruments were created to evaluate the knowledge expectations and received knowledge of patients in the context of empowering patient education. The theoretical hypothesis is that received knowledge corresponding to expected knowledge supports and gives potential for the empowerment of the patient.54
For structuring the instruments, in each subscale, items of expected knowledge or received knowledge were formulated, and content validity was established by a variety of subject matter experts (4 nurses, 4 nurse leaders, 4 nurse researchers, and 4 patients). The instruments were originally developed and piloted in surgical hospital care in collaboration with nurses in clinical practice.16,29,55 EKhp is aimed at evaluating the level of expected knowledge on admission to hospital (in wards or outpatient clinics), and RKhp at evaluating the knowledge received by patients, measured at the end or after their hospital stay.29,30 The instruments are completed by patients, significant others, or in some cases, nurses. Response options are on a 4-point Likert scale (1=fully disagree, 2=disagree to some extent, 3=agree to some extent, 4=fully agree), with higher score indicating higher knowledge expectations (EKhp) or fulfillment of received knowledge (RKhp). The option “Not applicable in my case” is also included. In some studies, the Likert scale is inverted (Table 1).16,29,56
Table 1 Reported Findings of the EKhp and RKhp Instruments
In 2003, the EKhp and RKhp instruments were piloted and tested for the first time with 237 adult surgical patients.16,29 After that, the instruments have been used jointly in Finland for orthopedic28 and cancer patients.30 In studies utilizing the instruments separately, EKhp has been used for heart failure patients scheduled for cardiac resynchronization therapy implantation in Iceland57 and for patients with cancer,58 knee osteoarthritis,59 and prostate cancer in Finland.60 RKhp has been used in Finland for patients in ambulatory surgical nursing,31 total joint arthroplasty,32 surgical nursing,55,61,62 medical wards,63 and for family members and nurses in surgical nursing.61 Two parallel instruments to EKhp and RKhp have been used to discover expected knowledge (EKso) and received knowledge of significant others (RKso) (Table 1).64–68
The respondents in the national studies (n=53–480, Table 2) were adult surgical patients, representing both genders, with a mean age of 44–72 years (SD 9–17), mostly employed (12–77%) or retired (17–85%).16,28–32,57–60,62,63 In all of the studies, validity and reliability of EKhp and RKhp have been evaluated, content validity has been confirmed by experts, and internal consistency of the instruments has been sufficient. Internal consistency of the EKhp instrument has been reported in several studies with Cronbach’s alpha ranging from 0.80 to 0.98 and subscales between 0.77 and 0.96,28–30,58–60 while that of the RKhp instrument has been reported to range from 0.80 to 0.99, and subscales between 0.76 and 0.98 (Table 1).16,28–32,62,63
Table 2 Characteristics of Respondents in Studies Using the EKhp and RKhp Instruments
The instruments have also been used for multiple purposes. For example, EKhp has been utilized in the development of an inter-professional screening instrument for cancer patients’ education process69 and in a project evaluating patients’ expected knowledge by hospital patients themselves (n=547) and by nurses (n=155) in several different medical wards.70 Moreover, several modified instruments have been made and used based on EKhp and RKhp,71–76 and an instrument assessing the content of provided patient education (EPNURSE) includes parts of EKhp and RKhp.66,68,77 The instruments and their variations have also been used in academic master’s theses in various nursing contexts78–88 and in numerous bachelor’s theses.
For international purposes, the instruments were translated from Finnish to English by native language experts using a standard double-checking translation procedure.89,90 The English version was confirmed by the research team and was then used as original version for international use and for translations in participating countries. The translation procedures in every participating country were also conducted according to standard forward-back translation procedure and were evaluated and confirmed by local researchers in each country. Possible differences and problems were discussed and any amendments accepted by the international research team partners.
In this study, we analyze the psychometrics of the EKhp and RKhp instruments based on international data collected in seven countries. The ultimate goal is to develop the instruments to support international researchers in the field of patient education to find tested, valid instruments for their purposes, and finally, to support the availability of tailored education for patients in different countries.
Patients and Methods
Data for the project Empowering Surgical Orthopedic Patients Through Education (ESOPTE) were collected in countries from the southern (Cyprus, Greece, Spain), northern (Finland, Iceland, Sweden) and central (Lithuania) parts of Europe. Participating countries were chosen to represent a large geographical area and various types of health-care systems.66 The data were collected in one to five hospitals in each country. The inclusion criterion for hospitals was general or university hospital performing surgical orthopedic knee or hip replacement procedures. Health-care professionals provided patient education according to the educational model of each country consisting of similar content consisting of a) the upcoming surgery, b) patients’ self-management and symptom management, c) recovery, and d) health-related quality of life.
The instruments, with cover letters to respondents as well as informed consent forms, were piloted in each country in a corresponding group of patients (30 patients per country). The partners agreed to collect the data from orthopedic patients by using the EKhp and RKhp according to the shared protocol. Designated contact persons in every hospital supported the data collection, ie, distributed the instruments and returned them. EKhp was given to patients prior to the patient’s admission, and RKhp at the end of hospital care. Completed questionnaires were returned to the mailboxes placed on the wards or sent back by mail in a prepaid envelope. Data collection was performed during the years 2009–2012 and results have been reported in 15 international studies (Table 1).54,56,61,64–68,91–97 In this paper, the combined results of all countries form the basis of the analysis.
The required international sample size was at least 1,540 patients from all participating countries together, based on power calculation of the instruments with a power level of 0.9 and 0.8 differences of mean scores with 0.95 standard deviation within groups at the significance level 0.01.93 Minimum was 220 patients per country. Inclusion criteria for patients attending orthopedic knee or hip replacement surgery for their osteoarthritis disease were 1) ability to understand Finnish/Greek/Icelandic/Lithuanian/Spanish/Swedish, 2) ability to complete the questionnaires independently (or with help of significant others), 3) 18 years of age or older, 4) no diagnosed cognitive disorders, and 5) volunteer to participate in the study.
Data were analyzed using SPSS 22 (IBM Corporation Chicago, IL, USA) statistical software. Descriptive statistics (frequencies, percentages, mean, standard deviation, range) were calculated to describe the sample characteristics and main variables. Sum-variables were formulated based on the six theoretical elements of empowerment in the instruments. Internal homogeneity of the scales was evaluated using Cronbach’s alpha coefficients (with the criterion of α ≥ 0.70).
Furthermore, data were also imported to Mplus version 7.1198 for modeling the expected knowledge and received knowledge scale associations using structural equation analysis. First, to determine whether the data fit the hypothetical model, values of the Chi-Square Test of Model Fit, with degrees of freedom (df) and p-value, were calculated. This statistic assesses the model by comparing the Χ2 value of the model to the Χ2 of the null model.99 Chi-square “assesses the magnitude of discrepancy between the sample and fitted covariances matrices”,100 and there should be an essence of a statistical significance.99 As this parameter is known to be influenced by large sample size,99 alternative incremental fit indices were also used for the evaluation of the model fit: comparative fit index (CFI, with the criterion of CFI ≥ 0.95),100 Tucker–Lewis index (TLI, with the criterion of ≥0.95), root-mean-square error of the approximation (RMSEA, with the criterion of <0.06,100 or a stringent upper limit of 0.07)101 and the standardized root-mean-square residual (SRMR; criterion <0.08). Akaike’s Information Criterion (AIC), used to assess the suitability of the model if several models are estimated with the same data, indicates to the researcher which of the models is the most parsimonious.102
A total of 1,595 (76%) hospital patients completed EKhp and 1,343 (64%) completed RKhp. The respondents comprised Icelandic (n=273, 17%), Swedish (n=268, 17%), Spanish (n=260, 16%), Finnish (n=253, 16%), Greek (n=207, 13%), Lithuanian (n=170, 11%) and Cypriot patients (n=164, 10%). The mean age of the respondents was 67 years (SD 10.6, range 25–91). Most of the respondents were female (62%) with vocational education (51%) in addition to primary school as basic education (51%). Over half of the respondents were retired (57%) and nearly a third were employed (27%), with 16% having employment history in health care or social services. Most respondents were admitted to hospital for knee arthroplasty (knee 63%, hip 37%) and had a history of treatment in the same hospital (73%). Forty-six percent of the respondents had a chronic illness, and a minority had had previous arthroplasty or arthroplasties (22%).
The content validity of the EKhp and RKhp instruments was judged with structured criteria including assessment of the instruments’ and subscales’ sufficiency of concreteness for measurement, focus on the expected and received knowledge, similarity to other subscales, items belonging to the subscale, importance in the expected or received knowledge, relevance to the expected or received knowledge, and clarity, coverage, and uniqueness of the instruments.103 Content validity was assessed in every country by expert panels set up by the countries themselves. EKhp and RKhp were evaluated as decent and no changes to the instruments were proposed.
Confirmatory Factor Analysis in Countries
Data from every participating country were used to test the structural validity of the instruments with confirmatory factor analysis. One latent variable model was built for each country to find out the R-square of each subscale (Table 3). For EKhp, the lowest R-squares were achieved for bio-physiological and financial subscales while the highest were achieved for ethical, functional and social subscales. For RKhp, the lowest R-squares were achieved for bio-physiological, functional and financial subscales, and the highest for ethical subscale.
Table 3 R-Squares of Subscales of the EKhp and RKhp Instruments
Confirmatory Factor Analysis for Seven Countries Together
The confirmatory factor analysis of EKhp was performed by first fitting the hypothetical model in the sample of 1,595 patients. The hypothetical model proposed for this study did not demonstrate a good fit with data derived from the sample of patients in seven European countries. Chi-square was 372.006 (df 9) and was statistically significant (p <0.001) not reaching the requirement.99 RMSEA was 0.159 (p <0.001), and CFI of 0.950 and TLI of 0.916 were found. Thus, all of the fit indices demonstrated a need for improvement of the model.
A revised model of EKhp included error covariance between bio-physiological and functional subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 135.66 (df 8) and was significant (p <0.001) not reaching the criteria.99 CFI of 0.982 and TLI of 0.967 indicated a good fit of the data to the slightly revised model. RMSEA was 0.100 (p <0.0001), not indicating not a good fit (stringent upper limit of 0.07).101
Table 4 Modeling of the EKhp and RKhp Instruments
The second revised model of EKhp included error covariance between bio-physiological and functional subscales as well as between experiential and functional subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 51.73 (df 7) and was significant (p <0.001) not reaching the criteria.99 CFI of 0.994 and TLI of 0.987 indicated a good fit of the data to the slightly revised model. RMSEA was 0.063 (p =0.08) indicating a good fit (stringent upper limit of 0.07).101
The third revised model of EKhp (Figure 1) included error covariance between bio-physiological and functional subscales, between experiential and functional subscales, as well as between ethical and financial subscales. The model fit improved markedly and fitted well (Table 4). The Chi-Square Test of Model Fit was 22.78 (df 6) and was significant (p <0.001) not reaching the criteria.99 CFI of 0.998 and TLI of 0.994 indicated a good fit of the data to the slightly revised model. RMSEA was 0.042 (p =0.739) indicating a very good fit (stringent upper limit of 0.07).101
Figure 1 The third revised model of EKhp.
Confirmatory factor analysis of RKhp was performed by first fitting the hypothetical model in the sample of 1,343 patients. The hypothetical model proposed for this study did not demonstrate a good fit with data derived from the sample of patients in seven European countries. Chi-square was 650.00 (df 9) and was statistically significant (p <0.001) not reaching the requirement.99 RMSEA was 0.228 (p <0.001), and CFI of 0.910 and TLI of 0.850 were found. Thus, all of the fit indices demonstrated a need for improvement of the model.
The revised model of RKhp included error covariance between bio-physiological and functional subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 190.23 (df 8) and was significant (p <0.001) not reaching the criteria.99 CFI of 0.974 and TLI of 0.951 indicated a good fit of the data to the slightly revised model. RMSEA was 0.130 (p <0.0001) indicating not a good fit (stringent upper limit of 0.07).101
The second revised model of RKhp included error covariance between bio-physiological and functional subscales as well as between social and financial subscales. The model fit improved markedly and fitted fairly (Table 4). The Chi-Square Test of Model Fit was 91.120 (df 7) and was significant (p <0.001) not reaching the criteria.99 CFI of 0.988 and TLI of 0.974 indicated a good fit of the data to the slightly revised model. RMSEA was 0.095 (p <0.0001) indicating not a good fit (stringent upper limit of 0.07).101
The third revised model of RKhp (Figure 2) included error covariance between bio-physiological and functional subscales, between financial and ethical, as well as between social and financial subscales. The model fit improved markedly and fitted well (Table 4). The Chi-Square Test of Model Fit was 34.393 (df 6) and was significant (p <0.001) not reaching the criteria.99 CFI of 0.996 and TLI of 0.990 indicated a good fit of the data to the slightly revised model. RMSEA was 0.059 (p =0.187) indicating a good fit (stringent upper limit of 0.07).101
Figure 2 The third revised model of RKhp.
Reliability of the instruments was demonstrated with internal consistency with Cronbach’s alpha values. In the total EKhp instrument, Cronbach’s alphas between the countries ranged from 0.87 (Iceland) to 0.99 (Lithuania). Subscales ranged between 0.8 and 0.98 in all countries, with the exception of the ethical subscale in Iceland, which was 0.57. Iceland had the lowest values in every subscale (0.57–0.83) except for the financial subscale (0.9).
Between the countries, the Cronbach’s alpha for the total RKhp ranged from 0.97 (Finland and Lithuania) to 0.99 (Greece). In subscales, the lowest values were seen in the functional subscale (in Finland, Greece, Iceland and Sweden, range in all countries 0.83–0.95) and the highest in the financial subscale, being the highest value in several countries: Finland, Greece, Spain and Sweden (range in all countries 0.87–0.99).
The aim of this study was to analyze the psychometrics of the EKhp and RKhp instruments. This is the first parallel international use of EKhp and RKhp and the evaluation of their psychometrics. Our basic assumption is that instruments for assessment of patient education are necessary due to the current and increasing importance of empowering patient education on European level.1,4 Meeting the knowledge expectations of patients in education is crucial for patient empowerment. Results of earlier studies using EKhp and RKhp suggest discrepancies between the expected and received knowledge, giving rise to concern about individually tailored empowering patient education.94
EKhp and RKhp are parallel instruments: development of such types of instruments is rather complicated, and they are not very common in nursing. EKhp and RKhp were developed based on the theoretical assumption of correspondence between patients’ expectations and received knowledge giving more potential for patient education to be empowering for individual patients. This assumption has received support,9,104 but the connection still needs further testing, especially the connection between expected and/or received knowledge and their relationship with nursing outcomes. This kind of testing would require both narrative designs analyzing patients’ experiences and randomized clinical trials in patient education. Furthermore, we also need more studies based on evidence analyzing the ethical element of empowering knowledge, especially the connection between patient’s right to know and empowering patient education.
Based on the results, EKhp and RKhp are proposed as international instruments with satisfactory psychometric qualities for the evaluation of empowering patient education. In confirmatory factor analyses, some error covariances were found. These relationships were also found in the Spanish versions.105 However, R-square values of subscales were satisfactory, the instruments enabled data collection in all seven countries, and the instruments have been tested with numerous respondents before this international study. However, it is clear that we need to be aware of the constant changes in patients’ responsibilities for their care and treatment in order to enable support to the empowerment of patients, and the structure of the instruments must be modified accordingly. Furthermore, this study was the first one using EKhp and RKhp in most of the participating countries, and potential country-specific factors need further investigation for drawing country-specific conclusions. For instance, in Spain, the financial subscale in both instruments obtained good psychometric values; however, the healthcare context in Spain is mainly based on the public health-care system where patients are not aware of direct costs and therefore, patients do not expect to receive information on financial issues. Nevertheless, both instruments demonstrated ability to capture the expected and received knowledge in orthopedic patients regardless of any country-specific confounding factors. Therefore, our aim to develop and test the instruments for the common European health-care market was successful, and the instruments seem promising for that purpose.
In this study, the instruments demonstrated sufficient internal consistency measured by Cronbach’s alpha. Few alpha values of 0.99 were rather high, indicating that the instruments could be shortened in the future.106 Based on the analysis, however, no specific item in the current form of the instruments could be identified. In the Icelandic data, the ethical subscale in EKhp received a lower alpha (0.57). The Iceland data set is rather small, and due to the possible cultural reasons behind this finding, testing needs to continue. The ethical subscale of EKhp demonstrated sufficient consistency in other Nordic countries (Finland and Sweden) which are culturally fairly similar to Iceland.107
International comparison of empowering patient education presents many challenges. Patient education differs between countries and cultures due to numerous national factors. For example, in some countries, patient education is done with the use of rather static information material in the form of leaflets while in others, this training takes a more structured face-to-face format. Other differing factors might include the points in time when and to what extent the training is delivered. In this study, the assumption is that there is correspondence between countries in the care of orthopedic patients and their education. The earlier published results from this study data have demonstrated similarities internationally; for example, in patient education provided by nurses66 and in differences between the expected and the received knowledge in patient education.94 It is clear, however, that even though the items of the instruments are the same, the patients in different countries could interpret and understand them differently. Nevertheless, international research is important especially in the European region where patients travel across countries, health-care professionals are in the common labor market, and health strategies are shared.1 Thus, it is beneficial to continue to produce information about patient education broadly, across national boundaries. Furthermore, the generation of high-quality cross-border information ensures that the level of training is kept at a high standard. In the future, concept analysis of empowering patient education could provide additional information on this matter. EKhp and RKhp can be advantageous premises for such research.
Instrument development is a long, systematic, and multi-phase process.108,109 Psychometric evaluation of EKhp and RKhp on national and international levels has demonstrated validity and reliability of the instruments. However, further testing of the instruments is warranted due to the constant changes in health care. International research communities are invited to join in the testing of EKhp and RKhp to improve empowering patient education globally.
This study has limitations in terms of generalization. Generalization of the results can be applied to hip and knee arthroplasty patients in Europe. The patients were from seven different countries and from many hospitals across the Europe; with caution, the results can thus be applied more generally to arthroplasty patients. Patient education in surgical care contexts is assumed to have many similarities;66 the results may therefore be applicable to other surgical patient groups as well. Outside the context of surgical patients, the instruments have been used successfully in national studies; for example, with patients with cancer or cardiac disease. In the future, international testing of the instruments will need to be applied to other patient groups.
Based on the results of this study, EKhp and RKhp are promising instruments for international use by researchers and practitioners in the field of hospital care, especially in the field of surgical care. When the instruments are used jointly, they can produce unique information regarding the results of patient education from patients’ perspective: EKhp indicates expected knowledge, that is, the initial situation before patient education, while RKhp uncovers received knowledge. Information provided by the instruments gives potential to support the empowerment of patients. The instruments can be applied in multiple ways in nursing, health care, and research to discover, for example, the effectiveness, quality, or patient-centeredness of patient education.
For empowering patients with educational activities, validated instruments for measuring patients’ expectations (EKhp) and received (RKhp) knowledge are useful. They can be used internationally both in clinical work and research. Instruments are fitting particularly for hospital care, and testing within other contexts in future is needed.
EKhp, The Expected Knowledge of Hospital Patients; RKhp, The Received Knowledge of Hospital Patients; ESOPTE, Empowering Surgical Orthopedic Patients Through Education; CFI, comparative fit index; TLI, Tucker–Lewis index; SRMR, standardized root-mean-square residual; AIC, Akaike Information Criteria.
Data Sharing Statement
All data generated or analyzed during this study are included in this and other published articles [54,56,61,64-68,91-97] (Table 1).
Ethics Approval and Informed Consent
This study was conducted in accordance with the Declaration of Helsinki. In all phases of the development of the instruments, principles of research ethics have been followed, and ethical approvals and permissions for data collection have been obtained. In the international data reported in this article, the respondents were informed of the purpose of the study and principles of anonymity and confidentiality. Every participant gave a written informed consent. All data have been stored and coded, and identification of individual respondents in reports is not possible. Ethical approvals were given in each participating country according to their national standards by regional ethical authorities (Cyprus: Ministry of Health Research and Ethics Committee, Republic of Cyprus, Y.Y.184.108.40.206(2); Finland: Ethics Committee of Hospital District of Southwest Finland, ETMK: 102/180/2008; Greece: Ethics Committee of the Department of Nursing, National and Kapodistrian University of Athens, 3029/17.08.2010; Iceland: the National Bioethics Committee, 09-084-SI; Lithuania: Ethical Research Committee, Klaipeda University, Sv-14, 17/04/2009; Spain: Ethical and research committee of International University of Catalonia, 2010/5955, and Hospital Clínic de Barcelona, ref # 2010/5955 * 2010/5915; Sweden: Regional ethical committee of Linköping University, Dnr. M69-09), as well as permissions for data collection.
The abstract of this paper was presented at the STTI (Sigma) 5th Biennial European Conference as a poster presentation with interim findings. The poster’s abstract was published in Sigma repository: http://hdl.handle.net/10755/20777
Design of the study, data collection, analysis, interpretation of data and writing the manuscript were financially supported by the Academy of Finland. There were also other funding sources: Cyprus: the Cyprus University of Technology; Finland: University of Turku, the Finnish Association of Nursing Research, the Finnish Foundation of Nursing Education; Greece: Faculty of Nursing, National and Kapodistrian University of Athens; Iceland: the Landspitali University Hospital Research Fund, the Akureyri Hospital Science Fund, the University of Akureyri Science Fund, the KEA fund, Akureyri, the Icelandic Nurses’ Association Science Fund; Lithuania: University of Klaipeda; Spain: Colegio Oficial de Enfermeria de Barcelona; and Sweden: the Swedish Rheumatism Association, and the County Council of Östergötland.
The authors report no conflicts of interest in this work.
1. World Health Organization. Health 2020 - A European policy framework and strategy for the 21st century. World Health Organization, Regional office for Europe; 2013. Available from: http://www.euro.who.int/__data/assets/pdf_file/0011/199532/Health2020-Long.pdf.
2. Organisation for Economic Co-operation and Development. Caring for quality in health: lessons learnt from 15 reviews of health care quality. 2017. Available from: http://www.oecd.org/els/health-systems/Caring-for-Quality-in-Health-Final-report.pdf.
3. European Commission. Blocks: tools and methodologies to assess integrated care in Europe. Report by the expert group on health systems performance assessment. Luxembourg: Publications Office of the European Union; 2017.
4. European Commission. Sweeping changes to upgrade Europe’s health systems – EU support for health system efficiency. Consumers, Health, Agriculture and Food Executive Agency, European Commission; 2019. Available from: https://op.europa.eu/en/publication-detail/-/publication/e6c91cd7-e952-11e8-b690-01aa75ed71a1/language-en?WT.mc_id=Selectedpublications&WT.ria_c=19980&WT.ria_f=3171&WT.ria_ev=search.
5. Eloranta S, Katajisto J, Leino-Kilpi H. Orthopaedic patient education practice. Int J Orthop Trauma Nurs. 2016;21:39–48. doi:10.1016/j.ijotn.2015.08.002
6. Johansson K, Nuutila L, Virtanen H, Katajisto J, Salanterä S. Preoperative education for orthopaedic patients: systematic review. J Adv Nurs. 2005;50(2):212–223. doi:10.1111/j.1365-2648.2005.03381.x
7. Klemetti S, Kinnunen I, Suominen T, et al. Active preoperative nutrition is safely implemented by the parents in pediatric ambulatory tonsillectomy. Ambulatory Surg. 2010;16(3):75–79.
8. Klemetti S, Kinnunen I, Suominen T, et al. The effect of preoperative fasting on postoperative thirst, hunger and oral intake in paediatric ambulatory tonsillectomy. J Clin Nurs. 2010;19(3–4):341–350. doi:10.1111/j.1365-2702.2009.03051.x
9. Siekkinen M, Kesänen J, Vahlberg T, Pyrhönen S, Leino-Kilpi H. Randomized, controlled trial of the effect of e-feedback on knowledge about radiotherapy of breast cancer patients in Finland. Nurs Health Sci. 2015;17(1):97–104. doi:10.1111/nhs.12175
10. Zhao FF, Suhonen R, Koskinen S, Leino-Kilpi H. Theory-based self-management educational interventions on patients with type 2 diabetes: a systematic review and meta-analysis of randomized controlled trials. J Adv Nurs. 2017;73(4):812–833. doi:10.1111/jan.13163
11. Svavarsdóttir MH, Sigurðardóttir ÁK, Steinsbekk A. How to become an expert educator: a qualitative study on the view of health professionals with experience in patient education. BMC Med Educ. 2015;15:87. doi:10.1186/s12909-015-0370-x
12. Posma ER, van Weert JC, Jansen J, Bensing JM. Older cancer patients’ information and support needs surrounding treatment: an evaluation through the eyes of patients, relatives and professionals. BMC Nurs. 2009;8:1. doi:10.1186/1472-6955-8-1
13. Johansson K, Leino-Kilpi H, Salanterä S, et al. Need for change in patient education: a Finnish survey from the patient’s perspective. Patient Educ Couns. 2003;51(3):239–245. doi:10.1016/S0738-3991(02)00223-9
14. Vaartio-Rajalin H, Huumonen T, Iire L, et al. Patient education process in oncologic context: what, why, and by whom? Nurs Res. 2015;64(5):381–390. doi:10.1097/NNR.0000000000000114
15. Heli V-R, Helena L-K, Liisa I, Kimmo L, Heikki M. Oncologic Patientsʼ knowledge expectations and cognitive capacities during illness trajectory. Holist Nurs Pract. 2015;29(4):232–244. doi:10.1097/HNP.0000000000000093
16. Leino-Kilpi H, Johansson K, Heikkinen K, Kaljonen A, Virtanen H, Salanterä S. Patient education and health-related quality of life: surgical hospital patients as a case in point. J Nurs Care Qual. 2005;20(4):307–318. doi:10.1097/00001786-200510000-00005
17. Virtanen H, Leino-Kilpi H, Salanterä S. Empowering discourse in patient education. Patient Educ Couns. 2007;66(2):140–146. doi:10.1016/j.pec.2006.12.010
18. Koponen L, Rekola L, Ruotsalainen T, Lehto M, Leino-Kilpi H, Voipio-Pulkki LM. Patient knowledge of atrial fibrillation: 3-month follow-up after an emergency room visit. J Adv Nurs. 2008;61(1):51–61. doi:10.1111/j.1365-2648.2007.04465.x
19. Sakellari E, Athanasopoulou C, Kokkonen P, Sourander A, Leino-Kilpi H. Validation of the youth efficacy/empowerment scale - mental health Finnish version. Psychiatriki. 2019;30(3):235–244. doi:10.22365/jpsych.2019.303.235
20. Johansson K, Salanterä S, Katajisto J. Empowering orthopaedic patients through preadmission education: results from a clinical study. Patient Educ Couns. 2007;66(1):84–91. doi:10.1016/j.pec.2006.10.011
21. Klemetti S, Kinnunen I, Suominen T, et al. The effect of preoperative nutritional face-to-face counseling about child’s fasting on parental knowledge, preoperative need-for-information, and anxiety, in pediatric ambulatory tonsillectomy. Patient Educ Couns. 2010;80(1):64–70. doi:10.1016/j.pec.2009.10.003
22. Ryhänen AM, Siekkinen M, Rankinen S, Korvenranta H, Leino-Kilpi H. The effects of Internet or interactive computer-based patient education in the field of breast cancer: a systematic literature review. Patient Educ Couns. 2010;79(1):5–13. doi:10.1016/j.pec.2009.08.005
23. Salonen A, Ryhänen AM, Leino-Kilpi H. Educational benefits of internet and computer-based programmes for prostate cancer patients: a systematic review. Patient Educ Couns. 2014;94(1):10–19. doi:10.1016/j.pec.2013.08.022
24. Kesänen J, Leino-Kilpi H, Lund T, Montin L, Puukka P, Valkeapää K. The knowledge test feedback intervention (KTFI) increases knowledge level of spinal stenosis patients before operation-a randomized controlled follow-up trial. Patient Educ Couns. 2016;99(12):1984–1991. doi:10.1016/j.pec.2016.07.025
25. Kesänen J, Leino-Kilpi H, Lund T, Montin L, Puukka P, Valkeapää K. Increased preoperative knowledge reduces surgery-related anxiety: a randomised clinical trial in 100 spinal stenosis patients. Eur Spine J. 2017;26(10):2520–2528. doi:10.1007/s00586-017-4963-4
26. Rantanen M, Johansson K, Honkala E, Leino-Kilpi H, Saarinen M, Salanterä S. Dental patient education: a survey from the perspective of dental hygienists. Int J Dent Hyg. 2010;8(2):121–127. doi:10.1111/j.1601-5037.2009.00403.x
27. Virtanen H, Leino-Kilpi H, Salanterä S. Learning about a patient-empowering discourse: testing the use of a computer simulation with nursing students. J Nurs Educ Pract. 2015;5(6):15–24.
28. Heikkinen K, Leino-Kilpi H, Hiltunen A, et al. Ambulatory orthopaedic surgery patients’ knowledge expectations and perceptions of received knowledge. J Adv Nurs. 2007;60(3):270–278. doi:10.1111/j.1365-2648.2007.04408.x
29. Rankinen S, Salanterä S, Heikkinen K, et al. Expectations and received knowledge by surgical patients. Int J Qual Health Care. 2007;19(2):113–119. doi:10.1093/intqhc/mzl075
30. Ryhänen AM, Rankinen S, Siekkinen M, Saarinen M, Korvenranta H, Leino-Kilpi H. The impact of an empowering internet-based breast cancer patient pathway programme on breast cancer patients’ knowledge: a randomised control trial. Patient Educ Couns. 2012;88(2):224–231. doi:10.1016/j.pec.2012.02.013
31. Leino-Kilpi H, Heikkinen K, Hiltunen A, et al. Preference for information and behavioral control among adult ambulatory surgical patients. Appl Nurs Res. 2009;22(2):101–106. doi:10.1016/j.apnr.2007.05.003
32. Montin L, Johansson K, Kettunen J, Katajisto J, Leino-Kilpi H. Total joint arthroplasty patients’ perception of received knowledge of care. Orthop Nurs. 2010;29(4):246–253. doi:10.1097/NOR.0b013e3181e51868
33. Leino-Kilpi H, Luoto E, Katajisto J. Elements of empowerment and MS patients. J Neurosci Nurs. 1998;30(2):116–123. doi:10.1097/01376517-199804000-00005
34. Kuokkanen L, Leino-Kilpi H. Power and empowerment in nursing: three theoretical approaches. J Adv Nurs. 2000;31(1):235–241. doi:10.1046/j.1365-2648.2000.01241.x
35. Schoberer D, Leino-Kilpi H, Breimaier HE, Halfens RJ, Lohrmann C. Educational interventions to empower nursing home residents: a systematic literature review. Clin Interv Aging. 2016;11:1351–1363. doi:10.2147/CIA.S114068
36. Pekonen A, Eloranta S, Solt M, Virolainen P, Leino-Kilpi H. Measuring patient empowerment – a systematic review. Patient Educ Couns. 2020;103(4):777–787. doi:10.1016/j.pec.2019.10.019
37. Leino-Kilpi H, Mäenpää I, Katajisto J. Empowerment of Patients with Longterm Disease. Development of the Evaluation Basis for the Quality of Nursing Care.
38. Siekkinen M, Salanterä S, Rankinen S, Pyrhönen S, Leino-Kilpi H. Internet knowledge expectations by radiotherapy patients. Cancer Nurs. 2008;31(6):491–498. doi:10.1097/01.NCC.0000339245.53077.76
39. Kesänen J, Leino-Kilpi H, Lund T, Montin L, Puukka P, Valkeapää K. Spinal stenosis patients’ visual and verbal description of the comprehension of their surgery. Orthop Nurs. 2019;38(4):253–261. doi:10.1097/NOR.0000000000000572
40. Leino-Kilpi H, Mäenpää I, Katajisto J. Nursing study of the significance of rheumatoid arthritis as perceived by patients using the concept of empowerment. J Orthop Nurs. 1999;3(3):138–145. doi:10.1016/S1361-3111(99)80051-X
41. Johansson K, Salanterä S, Katajisto J, Leino-Kilpi H. Patient education in orthopaedic nursing. J Orthop Nurs. 2002;6(4):220–226. doi:10.1016/S1361-3111(02)00094-8
42. Johansson K, Salanterä S, Heikkinen K, Kuusisto A, Virtanen H, Leino-Kilpi H. Surgical patient education: assessing the interventions and exploring the outcomes from experimental and quasiexperimental studies from 1990 to 2003. Clin Eff Nurs. 2004;8(2):81–92. doi:10.1016/j.cein.2004.09.002
43. Johansson K, Salanterä S, Katajisto J, Leino-Kilpi H. Written orthopedic patient education materials from the point of view of empowerment by education. Patient Educ Couns. 2004;52(2):175–181. doi:10.1016/S0738-3991(03)00036-3
44. Ryhänen AM, Johansson K, Virtanen H, Salo S, Salanterä S, Leino-Kilpi H. Evaluation of written patient patient educational materials in the field of diagnostic imaging. Radiography. 2008;15(2):e1–e5. doi:10.1016/j.radi.2008.04.006
45. Heikkinen K, Suomi R, Jääskeläinen M, Kaljonen A, Leino-Kilpi H, Salanterä S. The creation and evaluation of an ambulatory orthopedic surgical patient education web site to support empowerment. Comput Inform Nurs. 2010;28(5):282–290. doi:10.1097/NCN.0b013e3181ec23e6
46. Heikkinen K, Salanterä S, Suomi R, Lindblom A, Leino-Kilpi H. Ambulatory orthopaedic surgery patient education and cost of care. Orthop Nurs. 2011;30(1):20–28. doi:10.1097/NOR.0b013e318205747f
47. Heikkinen K, Salanterä S, Leppänen T, Vahlberg T, Leino-Kilpi H. Ambulatory orthopaedic surgery patients’ emotions when using different patient education methods. J Perioper Pract. 2012;22(7):226–231. doi:10.1177/175045891202200703
48. Ryhänen AM, Rankinen S, Tulus K, Korvenranta H, Leino-Kilpi H. Internet based patient pathway as an educational tool for breast cancer patients. Int J Med Inform. 2012;81(4):270–278. doi:10.1016/j.ijmedinf.2012.01.010
49. Siekkinen M, Leino-Kilpi H. Developing a patient education method - the e-Knowledge Test with feedback. Stud Health Technol Inform. 2012;180:1096–1098.
50. Ryhänen AM, Rankinen S, Siekkinen M, Saarinen M, Korvenranta H, Leino-Kilpi H. The impact of an empowering Internet-based breast cancer patient pathway program on breast cancer patients’ clinical outcomes: a randomised controlled trial. J Clin Nurs. 2013;22(7–8):1016–1025. doi:10.1111/jocn.12007
51. Virtanen H, Leino-Kilpi H, Leinonen K, Puukka P, Wöntsö J, Salanterä S. Nursing student control over using a computer simulation program about empowering discourse. Comput Inform Nurs. 2013;31(10):512–522. doi:10.1097/01.NCN.0000432119.56581.24
52. Kesänen J, Leino-Kilpi H, Arifulla D, Siekkinen M, Valkeapää K. Knowledge tests in patient education: a systematic review. Nurs Health Sci. 2014;16(2):262–273. doi:10.1111/nhs.12097
53. Siekkinen M, Pyrhönen S, Ryhänen A, Vahlberg T, Leino-Kilpi H. Psychosocial outcomes of e-feedback of radiotherapy for breast cancer patients: a randomized controlled trial. Psychooncology. 2015;24(5):515–522. doi:10.1002/pon.3684
54. Cano-Plans S, Lacueva-Pérez L, Cabrera E, Zabalegui A. Knowledge expectations of orthopaedic patients. Int J Nurs Pract. 2018;24(3):e12639. doi:10.1111/ijn.12639
55. Gröndahl W, Muurinen H, Katajisto J, Suhonen R, Leino-Kilpi H. Perceived quality of nursing care and patient education: a cross-sectional study of hospitalised surgical patients in Finland. BMJ Open. 2019;9(4):e023108. doi:10.1136/bmjopen-2018-023108
56. Klemetti S, Leino-Kilpi H, Charalambous A, et al. Information and control preferences and their relationship with the knowledge received among European joint arthroplasty patients. Orthop Nurs. 2016;35(3):174–182. doi:10.1097/NOR.0000000000000246
57. Ingadottir B, Thylén I, Jaarsma T. Knowledge expectations, self-care, and health complaints of heart failure patients scheduled for cardiac resynchronization therapy implantation. Patient Prefer Adherence. 2015;9:913–921. doi:10.2147/PPA.S83069
58. Vaartio-Rajalin H, Leino-Kilpi H, Puukka P. Exploring patients’ knowledge expectations during the illness trajectory. Cancer Nurs Pract. 2014;13(4):30–38. doi:10.7748/cnp2014.05.13.4.30.e1054
59. Pellinen T, Villberg J, Raappana M, Leino-Kilpi H, Kettunen T. Knowledge expectations of recently diagnosed patients with knee osteoarthritis. J Adv Nurs. 2016;72(11):2857–2868. doi:10.1111/jan.13053
60. Salonen A, Meretoja R, Vahlberg T, Leino-Kilpi H. Prostate patients’ need of knowledge on the first hospital visit – a pilot study. Tutkiva Hoitotyö. 2017;15(3):3–11.
61. Eloranta S, Leino-Kilpi H, Katajisto J, Valkeapää K. Empowering education of orthopaedic patients, significant others and nurses. Tutkiva Hoitotyö. 2015;13(1):13–23.
62. Leino-Kilpi H, Gröndahl W, Katajisto J, Nurminen M, Suhonen R. Participation of family members and quality of patient care - the perspective of adult surgical patients. J Clin Nurs. 2016;25(15–16):2242–2250. doi:10.1111/jocn.13262
63. Leino-Kilpi H, Gröndahl W, Pekonen A, et al. Knowledge received by hospital patients–a factor connected with the patient-centred quality of nursing care. Int J Nurs Pract. 2015;21(6):689–698. doi:10.1111/ijn.12277
64. Sigurdardottir AK, Leino-Kilpi H, Charalambous A, et al. Fulfilment of knowledge expectations among family members of patients undergoing arthroplasty: a European perspective. Scand J Caring Sci. 2015;29(4):615–624. doi:10.1111/scs.12199
65. Stark AJ, Salanterä S, Sigurdardottir AK, Valkeapää K, Bachrach-Lindström M. Spouse-related factors associated with quality of recovery of patients after hip or knee replacement - a Nordic perspective. Int J Orthop Trauma Nurs. 2016;23:32–46. doi:10.1016/j.ijotn.2016.03.001
66. Charalambous A, Papastavrou E, Valkeapää K, et al. Content of orthopedic patient education provided by nurses in seven European countries. Clin Nurs Res. 2018;27(7):770–789. doi:10.1177/1054773817713178
67. Copanitsanou P, Sourtzi P, Cano S, et al. Empowering education of arthroplasty patients’ significant others in three Southern European countries. Int J Older People Nurs. 2018;13(3):e12193. doi:10.1111/opn.12193
68. Copanitsanou P, Sourtzi P, Valkeapää K, Lemonidou C. Comparison of patients, significant others, and nursing staff views about empowering education in Greece. Nurs Care Res. 2019;53:30–38.
69. Vaartio-Rajalin H, Huumonen T, Iire L, et al. Development of an inter-professional screening instrument for cancer patients’ education process. Appl Nurs Res. 2016;29:248–253. doi:10.1016/j.apnr.2015.07.002
70. Kummel M, Lundgren-Laine H, eds. Potilaan Polku Tulevaisuuden Sairaalassa. Tampere: Juvenes Print – Suomen Yliopistopaino Oy; 2016.
71. Rantanen M, Kallio T, Johansson K, Salanterä S, Virtanen H, Leino-Kilpi H. Knowledge expectations of patients on dialysis treatment. Nephrol Nurs J. 2008;35(3):249–256.
72. Lönnberg P, Koivunen M. Tahdistinpotilaan tiedontarve Satakunnan keskussairaalan Sydänyksikön poliklinikalla - Näyttöön perustuvan potilasohjauksen vahvistaminen (VeTePO)-hankkeen selvitysraportti. Satakunnan sairaanhoitopiiri; 2011.
73. Heikkinen K, Helena L-K, Taina N, Anne K, Sanna S. A comparison of two educational interventions for the cognitive empowerment of ambulatory orthopaedic surgery patients. Patient Educ Couns. 2008;73(2):272–279. doi:10.1016/j.pec.2008.06.015
74. Heikkinen K, Salanterä S, Leino-Kilpi H. Ambulatory orthopaedic surgery patients knowledge with internet-based education. Stud Health Technol Inform. 2010;160(Pt1):605–609.
75. Heikkinen K, Leino-Kilpi H, Salanterä S. Ambulatory orthopaedic surgery patients’ knowledge with internet-based education. Methods Inf Med. 2012;51(4):295–300. doi:10.3414/ME11-02-0004
76. Eloranta S, Gröndahl W, Engblom J, Leino-Kilpi H. Empowering education from the perspective of dialysis patients. Tutkiva Hoitotyö. 2014;12:4–11.
77. Klemetti S, Ingadottir B, Katajisto J, et al. Skills and practices of European orthopedic nurses in empowering patient education. Res Theory Nurs Pract. 2018;32(4):382–399. doi:10.1891/1541-65220.127.116.112
78. Kättö T. Gynaecology Laparoscopy Patient Education in Ambulatory Surgery [master’s thesis]. Turku: University of Turku; 2009.
79. Suotunen T. Expectations and Received Knowledge by Orthopaedic Emergency Patients [master’s thesis]. Turku: University of Turku; 2009.
80. Helajärvi-Tirri J. The Knowledge Expectations and Received Knowledge of Patients Who Have Had a Pacemaker Fitted [master’s thesis]. Turku: University of Turku; 2012.
81. Kallio T. Measuring Informational Needs of Dialysis Patient with Chronic Renal Insufficiency with Dialysis Patient Informational Needs Scale [master’s thesis]. Turku: University of Turku; 2005.
82. Uusitalo M. Stroke patients’ Information Needs [master’s thesis]. Turku: University of Turku; 2009.
83. Pesonen K. Oral Cancer Patients’ Knowledge Expectations at Different Stages of the DIsease [master’s thesis]. Turku: University of Turku; 2012.
84. Pellinen T. Knee Osteoarthritis Patient’s Information Needs in Patient Education [master’s thesis]. Jyväskylä: University of Jyväskylä; 2014.
85. Pekonen A. The Quality of Care in Department of Pulmonary Diseases [master’s thesis]. Turku: University of Turku; 2010.
86. Koski-Pärnä K. Received Knowledge of Hearing Aid Patient [master’s thesis]. Turku: University of Turku; 2012.
87. Oinas P. Access to Information by a Close Relative of a Stroke Patient During Hospital Treatment [master’s thesis]. Turku: University of Turku; 2013.
88. Lashgarizad K, Salehi T, Mansoureh Ashghali F, Hamid H. Investigating knowledge of patients undergoing total knee replacement. J Client Center Nurs Care. 2018;4(4):223–229. doi:10.32598/jccnc.4.4.223
89. Hilton A, Skrutkowski M. Translating instruments into other languages: development and testing processes. Cancer Nurs. 2002;25(1):1–7. doi:10.1097/00002820-200202000-00001
90. Sidani S, Guruge S, Miranda J, Ford-Gilboe M, Varcoe C. Cultural adaptation and translation of measures: an integrated method. Res Nurs Health. 2010;33(2):133–143. doi:10.1002/nur.20364
91. Ingadottir B, Johansson Stark A, Leino-Kilpi H, Sigurdardottir AK, Valkeapää K, Unosson M. The fulfilment of knowledge expectations during the perioperative period of patients undergoing knee arthroplasty – a Nordic perspective. J Clin Nurs. 2014;23(19–20):2896–2908. doi:10.1111/jocn.12552
92. Å JS, Ingadottir B, Salanterä S, et al. Fulfilment of knowledge expectations and emotional state among people undergoing hip replacement: a multi-national survey. Int J Nurs Stud. 2014;51(11):1491–1499. doi:10.1016/j.ijnurstu.2014.03.006
93. Valkeapää K, Klemetti S, Cabrera E, et al. Knowledge expectations of surgical orthopaedic patients: a European survey. Int J Nurs Pract. 2014;20(6):597–607. doi:10.1111/ijn.12189
94. Klemetti S, Leino-Kilpi H, Cabrera E, et al. Difference between received and expected knowledge of patients undergoing knee or hip replacement in seven European countries. Clin Nurs Res. 2015;24(6):624–643.
95. Å JS, Charalambous A, Istomina N, et al. The quality of recovery on discharge from hospital, a comparison between patients undergoing hip and knee replacement - a European study. J Clin Nurs. 2016;25(17–18):2489–2501. doi:10.1111/jocn.13278
96. Koekenbier K, Leino-Kilpi H, Cabrera E, et al. Empowering knowledge and its connection to health-related quality of life: a cross-cultural study. Appl Nurs Res. 2016;29:211–216. doi:10.1016/j.apnr.2015.05.004
97. Copanitsanou P, Valkeapää K, Cabrera E, et al. Total joint arthroplasty patients’ education on financial issues and its connection to reported out-of-pocket costs-a European study. Nurs Forum. 2017;52(2):97–106. doi:10.1111/nuf.12171
98. Muthén LK, Muthén B. Mplus User’s Guide.
99. Kline RB. Principles and Practice of Structural Equation Modeling.
100. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model. 1999;6(1):1–55. doi:10.1080/10705519909540118
101. Steiger JH. Understanding the limitations of global fit assessment in structural equation modeling. Pers Individ Differ. 2007;42(5):893–898. doi:10.1016/j.paid.2006.09.017
102. Diamantopoulos A, Siguaw JA. Introducing LISREL.
103. Perälä ML. The Quality Assessment of Patient Care: Validation of a Quality Measure (QUALPACS). Jyväskylä: Kirjapaino Gummerus Oy; 1995.
104. Ryhänen A. Internet-Based Breast Cancer Patient’s Pathway as an Empowering Patient Educational tool [dissertation]. Turku, Finland: University of Turku, 2012.
105. Cano-Plans S, Lacueva-Perez L, Cabrera-Torres E, Zabalegui A. Validación de la version española: expectativas y Conocimiento Recibido del Paciente Hospitalizado. Aquichan. 2017;17(3):292–304. doi:10.5294/aqui.2017.17.3.6
106. DeVellis RF. Scale Development: Theory and Application.
107. Grunfelder J, Rispling L, Norlén G, eds. State of the Nordic Region. Copenhagen: Nordic Council of Ministers; 2018.
108. Rattray J, Jones MC. Essential elements of questionnaire design and development. J Clin Nurs. 2007;16(2):234–243. doi:10.1111/j.1365-2702.2006.01573.x
109. Younas A, Porr C. A step-by-step approach to developing scales for survey research. Nurse Res. 2018;26(3):14–19. doi:10.7748/nr.2018.e1585
This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.