Back to Journals » Patient Preference and Adherence » Volume 11

Validation of the Italian version of the Patient Reported Experience Measures for intermediate care services

Authors Cinocca S, Rucci P, Randazzo C, Teale E, Pianori D, Ciotti E, Fantini MP 

Received 20 April 2017

Accepted for publication 20 June 2017

Published 27 September 2017 Volume 2017:11 Pages 1671—1676

DOI https://doi.org/10.2147/PPA.S140041

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Johnny Chen



Sergio Cinocca,1 Paola Rucci,1 Concetta Randazzo,1 Elizabeth Teale,2 Davide Pianori,1 Emanuele Ciotti,3 Maria Pia Fantini1

1Department of Biomedical and Neuromotor Sciences, Unit of Hygiene, Public Health and Biostatistics, Alma Mater Studiorum, University of Bologna, Bologna, Italy; 2Academic Unit of Elderly Care and Rehabilitation, Bradford Institute for Health Research, Bradford, UK; 3Bologna Local Health Authority, Bologna, Italy

Background: Intermediate care (IC) services are a key component of integrated care for elderly people, providing a link between hospital and home through provision of rehabilitation and health and social care. The Patient Reported Experience Measures (PREMs) are designed to measure user experience of care in IC settings.
Objective: To examine the feasibility and the scaling properties of the Italian version of PREMs questionnaires for use in IC services.
Methods: A cross-sectional survey was conducted on consecutive users of 1 home-based and 4 bed-based IC services in Emilia-Romagna (Italy). The main outcome measure was the PREMs questionnaire results. PREMs for each home- and bed-based IC services were translated, back-translated, and adapted through consensus among the members of the advisory board and pilot testing of face validity in 15 patients. A total of 199 questionnaires were returned from users of bed-based services and 185 were returned by mail from users of home-based services. The return rates and responses were examined. Mokken analysis was used to examine the scaling properties of the PREMs.
Results: Analysis performed on the bed-based PREMs (N=154) revealed that 13 items measured the same construct and formed a moderate-strength scale (Loevinger H=0.488) with good reliability (Cronbach’s alpha =0.843). Analysis of home-based PREMs (N=134 records) revealed that 15 items constituted a strong scale (Loevinger H=0.543) with good reliability (Cronbach’s alpha =0.875).
Conclusion: The Italian versions of the bed- and home-based IC-PREMs questionnaires proved to be valid and reliable tools to assess patients’ experience of care. Future plans include monitoring user experience over time in the same facilities and in other Italian IC settings for between-service benchmarking.

Keywords:
intermediate care, patient-reported experience measures, validation, elderly

Plain language summary

The number of elderly people with multiple chronic conditions is increasing, as a result of the aging of the Italian population. Health and social services provided by a coordinated multiprofessional staff in semi-residential facilities or at patient’s home (the so-called intermediate care [IC] services) are one of the organizational responses to the emerging needs of these people. This study was conducted to examine patients’ experience of care with IC services, using a questionnaire originally developed in English. The questionnaire was translated into Italian, revised by the working group to improve the wording of items, and then administered to a small group of patients to test its comprehensibility. The final form of the Patient Reported Experience Measures (PREMs) questionnaires, with incorporated changes, was then administered to patients treated at home and in community hospitals. The questionnaires were well understood by patients and were easy to administer by the health care staff. The large majority of patients reported a positive experience of care. After a statistical analysis, 2 items were omitted from the bed-based version of the PREMs. The final versions of the PREMs proved to be valid and reliable.

Introduction

The prevalence of long-term conditions such as heart disease, stroke, cancer, type 2 diabetes, obesity, and arthritis is increasing globally.1 Moreover, the presence of multimorbidity (the co-occurrence of at least two chronic conditions in one individual) has become common, especially in elderly people, ranging from 26% to >50% in patients aged 65–69 years and from 56.6% to 80% in patients aged ≥85 years.2,3 Although clinical guidelines and care pathways have been developed for the management of specific chronic conditions, the treatment of multimorbidity requires a holistic approach to address patients’ complex health care needs and to ensure continuity of care.47

IC is one of the organizational responses to emerging needs of elderly people with multiple chronic diseases. While in the UK IC services are more set up for rehabilitation/reablement, in Italy the term IC denotes health and social services provided by a coordinated multiprofessional staff in semi-residential facilities or at patient’s home. IC is aimed at preventing unnecessary hospitalizations and delayed discharge, facilitating discharge home, developing clinical pathways alternative to hospital and based on patients’ care needs, promoting access to rehabilitation services and physical therapy, and reducing long-term institutionalization.8,9

The UK National Audit of Intermediate Care (NAIC), assisted by the Plain English Campaign, defined IC as:

services [that] offer a link between places such as hospitals and people’s homes, and between different areas of the health and social care system – community services, hospitals, GPs and social care.10

In the UK, four types of IC have been defined: crisis response, bed based, home based, and reablement.10

In Italy, the National Health Plans 2011–2013 and 2014–2016 have redesigned the health care network by strengthening community care and integrating it with hospital care. This reorganization of health care services has included the development of IC services.11

IC bed-based services, the so-called community hospitals (CH), possess the following characteristics: 15 beds, managed by nurses with health care provided by a general practitioner (or other physicians). Emilia-Romagna has been a pioneer in this process, implementing 14 bed-based CH services since the 1990s.

Home-based services provide care in patients’ home, including rehabilitation, supportive and technical nursing care, domestic aid, and personal care (as well as respite care provided to informal caregivers).12 In the Emilia-Romagna region, home-based services have been implemented successfully for several decades.13

Fremont et al14 argued that the optimal management of chronic diseases relies on the degree of patient concordance with medications and therapeutic interventions inside and outside the hospital. Concordance is more likely if patients are aware of their condition and understand the treatment options available. Assessment of patients’ experience is thus considered a key element in this new model as it is recognized as a fundamental indicator of quality health provision.

Experience and satisfaction are different constructs. Experience measures whether specific processes of care did, or did not, occur. Several studies have shown that improving patients’ experience is often associated with increase in patient safety, clinical effectiveness, and health outcomes.15,16 However, patient satisfaction with care may not correlate with processes of care or clinical outcomes. Patients may be satisfied with poor quality care or dissatisfied in a highly effective service.17 Integration of measures of patient experience into evaluation of health care performance identifies problem areas (poorly delivered processes of care) that may be addressed to improve the quality of delivered care. The Patient Reported Experience Measures (PREMs) offer an objective and reliable measure of users experience and a practical way to explore person-centered, integrated care models.

There is an increased emphasis in Europe on measures of patient experience as health care quality indicators across different medical disciplines. PREMs have been widely used and integrated in the evaluation of quality of care for chronic heart failure,18 mental health,17 diabetes, asthma, and rheumatoid arthritis,19 in emergency care services,20 and to measure patients’ experience with medicine use.21 The NAIC in England and Wales has included PREMs designed specifically for IC since 2013.22,10

Creating and validating patient-reported indicators should be a priority, especially in areas where care is complex and demand is increasing, such as in case of chronic condition management. A wide and complete assessment is necessary to describe and improve care pathways.

In IC settings, positive personal experience represents a strong contribution to the process of recovery. Given the potential relevance of this measurement in IC, we have translated and adapted the English IC-PREMs23 to the Italian health care setting to evaluate patient experience in 4 bed-based and 1 home-based IC settings in the Emilia-Romagna region (Italy). The specific aim of this study was to examine the feasibility, acceptability, and scaling properties of the Italian version of IC-PREMs (hereafter IC-PREM-I) questionnaires (available on request from the corresponding author).

Methods

The original versions of the PREMs questionnaires developed by Teale et al23 and used in NAIC 2015 were translated by one of the authors and back-translated by a bilingual person of British origin.

Some questions relevant to both bed- and home-based services feature in both questionnaires (eg, patient involvement in defining treatment aims or sharing health information with patient’s family); other questions are service specific (eg, agreement about appointment times or how to contact staff in case of need in home-based services). Both questionnaires contain an open-ended narrative response question. A question regarding proxy completion was included with both questionnaires.

The study protocol was submitted to the Local Ethics Committees (Comitato Etico Interaziendale Bologna Imola and Comitato Provinciale di Modena) and the study qualified as exempt from approval because questionnaires were anonymous and research activities presented no psychological and/or physical risk to human subjects. A form explaining the study procedures was given to patients. All patients provided oral informed consent. Those not willing to participate did not fill the questionnaires.

Bed-based IC-PREM-I

The translated questionnaire was reviewed by the advisory board of Castelfranco Emilia and Castel San Pietro Terme health districts (2 of the 4 CHs involved in the validation project), and suggestions for possible modifications were discussed, reviewed, and incorporated into the final version. This final form was first administered in a pilot phase at Castel San Pietro’s CH, where professional nurses were instructed about the administration and then conducted a structured interview to 15 patients to evaluate feasibility and face validity. The bed-based version of PREMs questionnaire was subsequently administered to patients admitted to 4 CH of Emilia-Romagna region (Italy) voluntarily involved in the project (Castel San Pietro, Castelfranco, Forlimpopoli, and Fanano). Patient completion of the questionnaires occurred 1–2 days prior to discharge, supported by professional nurses. Data collection occurred between January 1st and September 30th of 2016.

Home-based IC-PREM-I

After the forward–backward translation of the questionnaire, a working group of Bologna Local Health Authority reviewed and modified some items to improve their comprehension. The home-based final version of PREMs questionnaire was administered to 500 patients receiving home care in the metropolitan area of Bologna, Emilia-Romagna (Italy), between April 1st and May 31st of 2016. The questionnaire was handed over to the patients with a prepaid envelope at the end of the service. Patients were asked to return the envelope by surface mail.

Statistical analysis

Item responses were recoded to a dichotomous format using the approach suggested by the Picker Europe24 and adopted by Teale et al in the IC-PREM validation paper.23 Specifically, responses indicating a problem were scored as “0” and responses indicating a good experience were scored as “1”.

Mokken scale analysis was used to analyze the construct validity of the PREMs. It is based on nonparametric item response theory (IRT) models and consists of an automated item selection procedure (AISP), which partitions items into one or more scales, possibly leaving some items unselected.25

The goodness-of-fit of the IRT model was assessed by computing the Loevinger H-coefficient (measuring scalability) and Cronbach’s alpha coefficient (measuring the reliability of the scale). Mokken proposed the following rules of thumb for H – a scale is considered weak if 0.3≤H<0.4, moderate if 0.4≤H<0.5, and strong if H>0.5.30 Cronbach’s coefficient is considered good if alpha is ≥0.8. Valid Mokken analysis rests on meeting a number of assumptions: unidimensionality, local independence, and latent monotonicity.

We planned to administer at least 150 questionnaires in each setting in order to ensure adequate item classification.26 R software (Mokken package) was used to conduct all the analyses.

Results

IC-PREM-I bed-based questionnaires results and Mokken analysis

Of the 199 bed-based questionnaires administered, 32.2% were filled in by patients, 42.2% by caregivers, and 10.1% by patients and caregivers. For 15.6% of the questionnaires, information on the type of respondent was not available. Overall, 154 questionnaires (77%) were returned complete and were used for statistical analysis. The item response frequency is provided in Table 1. Item responses were largely positive and ranged from 65% to 97%.

Table 1 Percentage of positive responses to the bed-based questionnaire

Mokken analysis of the 15-item questionnaire revealed that items Q2 (“I was given enough information”) and Q15 (“information about other services available”) had Loevinger H-coefficients <0.3, and the overall scalability was weak (H=0.379, standard error [SE] =0.051). Using AISP, one latent trait was identified. Items Q2 (“staff have sufficient information”) and Q14 (“ability in maintaining social contacts improved”) were discarded because they did not measure this trait. After removing these items, the Loevinger H-coefficient improved (H=0.448, SE =0.058) and Cronbach’s alpha was 0.843. The monotonicity assumption and local independence assumption were met; therefore, the final scale consisted of 13 items (Table 1).

IC-PREM-I home-based questionnaires and Mokken analysis

The postal return rate of IC-PREM-I home-based questionnaires was 37%. Of the 185 questionnaires received, 29.1% were filled in by patients and 68.7% by caregivers. For 2.2% of the questionnaires, information on the respondents who filled in the details was not available. Overall, 134 questionnaires (72%) were returned complete and used for statistical analysis. The item response frequency is provided in Table 2. Item responses were largely positive and ranged from 66% to 98%.

Table 2 Percentage of positive responses to the home-based questionnaire

Results of Mokken analysis indicated that all items had Loevinger H-coefficients >0.3 and the overall scalability was H=0.543 (SE =0.065). Again, using the AISP, only one latent variable was identified and all items measured this trait. The monotonicity assumption and local independence were met, and the final scale consisted of all 15 items (Table 2; Cronbach’s alpha =0.875).

Areas of improvement in IC in Italy

Response rates were largely positive for all items and across questionnaires. Using an arbitrary cutoff of 75% to define a moderate-to-good level of satisfaction with care, some key areas were identified for targeted improvement:

  • In the hospital setting: involvement in decisions about care, family or caregiver involvement in treatment, and maintenance of social contacts.
  • In the home setting: involvement in goal setting and clear information to patients about when the service would end.

Opinions, complaints, and positive comments derived from the open-ended question have been used to create word clouds to provide easy visual insight about patients and care experience (Figure S1).

Discussion

The IC-PREM-I bed- and home-based questionnaires are valid and reliable tools to assess patients’ experience in IC settings. The questionnaires were well understood by users and were easy to administer by the health care staff. Mokken analysis showed that the bed- and home-based final versions, consisting of 13 and 15 items, respectively, measure a single robust construct, allowing summation of scores to give an overall measure of experience.

The pilot administration of the questionnaires identified some areas for service improvement: better involvement of patients in decisions about their care and their ability to maintain social contacts in hospital facilities and need to improve information provision regarding the end of services. These aspects of IC require the attention of policy makers and IC staff to improve patients’ experience of care and to inform service development at both organizational and clinical levels.

As patient-reported measures are used to drive quality of care improvement, the Italian validation also offers the possibility to use the questionnaire for benchmarking, to compare different IC facilities, as emphasized by the recent recommendations about patient-reported indicators provided by the Organisation for Economic Co-operation and Development.27

To our knowledge, no other validated tools are available in Italy to assess patients’ experience of their care. We provided a new and useful tool for IC services to be integrated with other health care quality indicators to progress toward person-centered care.

The positive response rate to the items of the bed-based IC-PREM-I was very high in the present study compared with 2015 NAIC data.10 In contrast, patients reported a less positive experience of involvement in taking decisions regarding their care as measured by the home-based IC-PREM-I when compared with 2015 NAIC data. While these could be true phenomena, social desirability bias28,29 should be considered as a possible explanation; patients wish to be viewed positively by health care assistants and staff for their expectations to be met. Patients may be more willing to report a less favorable experience of care through a postal survey than they are when nursing staff supports completion.

This study has some limitations. The questionnaires have been administered in a region in Italy where IC facilities are long established (Emilia-Romagna). It is possible that administering PREMs in other Italian regions would have yielded different results. The postal questionnaires (home-based services) had a relatively low return rate (37%), though this is comparable with other studies using postal questionnaires15 and higher than the return rate in the original validation study (12.6%).23

Future plans are to continue the administration of PREMs in the facilities involved in the present study, to monitor user experience over time, and to determine whether changes at organizational level in IC facilities and services are responsive to patient need. We intend to involve further IC facilities in the periodical survey to allow regional comparison and benchmarking.

Acknowledgments

The authors wish to thank the colleagues of the working groups of Bologna Local Health Authority and of the community hospitals for their support in the conduction of the study: Mara Morini, Luca Barbieri, Laura Biagetti, Juri Coccia, Loretta Muraro, Manuela Petroni, Chiara Petrucci, Claudio Tacconi, Giulia Pieri, Vittorio Donini, Sabrina Gabrielli, Luigi Baldi, Loretta Vallicelli, Antonella Dallari, Stefano Boni, Andrea Spanò, and Alessandra Raggi.

Disclosure

The authors report no conflicts of interest in this work.


References

1.

Nugent R. Chronic diseases in developing countries: health and economic burdens. Ann N Y Acad Sci. 2008;1136:70–79.

2.

Lenzi J, Avaldi VM, Rucci P, Pieri G, Fantini MP. Burden of multimorbidity in relation to age, gender and immigrant status: a cross-sectional study based on administrative data. BMJ Open. 2016;6(12):e012812.

3.

Barnett K, Mercer SW, Norbury M, Watt G, Wyke S, Guthrie B. Epidemiology of multimorbidity and implications for health care, research, and medical education: a cross-sectional study. Lancet. 2012;380(9836):37–43.

4.

National Institute for Health and Care Excellence. Multimorbidity: clinical assessment and management; NIC guideline [NG56]. Published September 2016. Available from: https://www.nice.org.uk/guidance/ng56. Accessed August 4, 2017.

5.

van Weel C, Schellevis FG. Comorbidity and guidelines: conflicting interests. Lancet. 2006;367(9510):550–551.

6.

Smith SM, Soubhi H, Fortin M, Hudon C, O’Dowd T. Managing patients with multimorbidity: systematic review of interventions in primary care and community settings. BMJ. 2012;345:e5205.

7.

Salisbury C. Multimorbidity: redesigning health care for people who use it. Lancet. 2012;380(9836):7–9.

8.

Pearson M, Hunt H, Cooper C, Shepperd S, Pawson R, Anderson R. Intermediate Care: A Realist Review and Conceptual Framework. Final report. NIHR Service Delivery and Organisation Programme. Southampton: NIHR; 2013.

9.

Lenzi J, Mongardi M, Rucci P, et al. Sociodemographic, clinical and organisational factors associated with delayed hospital discharges: a cross-sectional study. BMC Health Serv Res. 2014;14:128.

10.

NHS (UK). National Audit of Intermediate Care Report; 2012–2015. Available from: https://www.nhsbenchmarking.nhs.uk/projects/naic. Accessed August 4, 2017.

11.

Conferenza delle regioni e delle province autonome Ministero della salute. Piano Sanitario Nazionale 2011–2013 [National Health Plan 2011–2013]. [November 2010]. Available from: http://www.socialesalute.it/res/download/181110_per_PSN.pdf. Accessed August 4, 2017. Italian.

12.

Genet N, Boerma WG, Kringos DS, et al. Home care in Europe: a systematic literature review. BMC Health Serv Res. 2011;11:207.

13.

Decreto Presidente Consiglio Ministri 29 novembre 2001. Definizione dei livelli essenziali di assistenza (in GU 8 febbraio 2002, n. 33). [Decree of the President of the Council of Ministers. 29 November 2001. Definition of Essential Care Levels. GU 8 February 2002, number 33]. Available from: http://www.gazzettaufficiale.it/eli/id/2002/02/08/02A00907/sg. Accessed August 4, 2017. Italian.

14.

Fremont AM, Cleary PD, Hargraves JL, Rowe RM, Jacobson NB, Ayanian JZ. Patient-centered processes of care and long-term outcomes of myocardial infarction. J Gen Intern Med. 2001;16(12):800–808.

15.

Etingen B, Miskevics S, LaVela SL. Assessing the associations of patient-reported perceptions of patient-centered care as supplemental measures of health care quality in VA. J Gen Intern Med. 2016;31(Suppl 1):10–20.

16.

Marrone G, Mellgren Åsa, Eriksson LE, Svedhem V. High concordance between self-reported adherence, treatment outcome and satisfaction with care using a nine-item health questionnaire in InfCareHIV. PLoS One. 2016;11(6):e0156916.

17.

Brown A, Ford T, Deighton J, Wolpert M. Satisfaction in child and adolescent mental health services: translating users’ feedback into measurement. Adm Policy Ment Health. 2014;41(4):434–446.

18.

Lagha E, Noble A, Smith A, Denvir MA, Leslie SJ. Patient reported experience measures (PREMs) in chronic heart failure. J R Coll Physicians Edinb. 2012;42(4):301–305.

19.

Bosworth A, Cox M, O’Brien A, et al. Development and validation of a Patient Reported Experience Measure (PREM) for patients with rheumatoid arthritis (RA) and other rheumatic conditions. Curr Rheumatol Rev. 2015;11:1–7.

20.

Male L, Noble A, Atkinson J, Marson T. Measuring patient experience: a systematic review to evaluate psychometric properties of patient reported experience measures (PREMs) for emergency care service provision. Int J Qual Health Care. 2017;29(3):314–326.

21.

Krska J, Katusiime B, Corlett SA. Validation of an instrument to measure patients’ experiences of medicine use: the Living with Medicines Questionnaire. Patient Prefer Adherence. 2017;11:671–679.

22.

Young J, Gladman JRF, Forsyth DR, Holditch C. The second national audit of intermediate care. Age Ageing. 2015;44(2):182–184.

23.

Teale EA, Young JB. A Patient Reported Experience Measure (PREM) for use by older people in community services. Age Ageing. 2015;44(4):667–672.

24.

Jenkinson C, Coulter A, Bruster S. The Picker Patient Experience Questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care. 2002;14(5):353–358.

25.

Andries van der Ark L. Mokken scale analysis in R. J Stat Softw. 2007;20(11):183–208.

26.

Hendrik Straat J, Andries van der Ark L, Sijtsma K. Minimum sample size requirements for Mokken scale analysis. Educ Psychol Meas. 2014;74(5):809–822.

27.

Recommendations to OECD ministers of health from the high level reflection group on the future of health statistics. Strengthening the international comparison of health system performance through patient-reported indicators. [January 2017]. Available from: https://www.oecd.org/els/health-systems/Recommendations-from-high-level-reflection-group-on-the-future-of-health-statistics.pdf. Accessed August 4, 2017.

28.

Althubaiti A. Information bias in health research: definition, pitfalls, and adjustment methods. J Multidiscip Healthc. 2016;9:211–2117.

29.

Kaminska O, Foulsham T. Understanding sources of social desirability bias in different modes: evidence from eye-tracking. Institute for Social & Economic Research Working Paper Series. University of Essex. No. 2013-04.

30.

Mokken RJ. A theory and procedure of scale analysis. The Hague/Berlin: Mouton/De Gruyter; 1971.

Creative Commons License © 2017 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.