A Scale for Measuring Electronic Patient Engagement Behaviors: Development and Validation

Introduction

Patient engagement is receiving increasing attention in both scientific literature and everyday healthcare practices.^1–3 It mainly refers to active participation of patients in their healthcare journey, demonstrating their active role in activities including making informed decisions and leveraging available resources to effectively manage their health conditions.^4,5 The role of patients in healthcare has evolved from a passive presence, where doctors made most decisions, to an active participation in their own care. This shift is part of a broader societal movement towards individual rights and autonomy, highlighting the growing recognition of the importance of involving patients in the decision-making processes regarding their health.⁶ The healthcare system now extends beyond merely treating diseases to also embracing the unique attributes, values, and experiences of each patient.^7–9 Engaging patients as partners in their care is clinically appealing for better health outcomes at lower costs.^10–12 Several studies have demonstrated that patient engagement has the potential of improving chronic disease self-management and healthcare quality.^13–16 Patients are expected to be more and more involved in the whole healthcare system and numerous efforts have been made to promote and encourage patient engagement in healthcare.¹⁷

The rise of electronic health (eHealth) technology has profoundly impacted patient engagement behaviors.^18–21 With the widespread adoption of digital health tools such as mobile health apps, telemedicine, and online healthcare communities, patients have greater access to healthcare information and resources to support their health and clinical activities and achieve more convenient and efficient communication with healthcare providers.^22–24 It is undeniable that the COVID-19 pandemic has further accelerated the integration of information technology in healthcare delivery.²⁵ As many services increasingly transition online, the provision of medical services is being transformed, enabling patients to rely on eHealth tools to fulfill their healthcare requirements.²⁶

In the context of the ongoing technological transformation within the health care sector, patient engagement has undergone significant evolution, integrating advanced electronic characteristics. This progression is manifested in the behaviors of patients, now identified as electronic patient engagement behaviors (EPEB). EPEB may be defined as a spectrum of proactive actions and practices patients undertake by leveraging electronic modalities and digital resources to facilitate enhanced access to health care benefits. With the growing importance of EPEB, it has become an increasingly significant aspect of the healthcare experience, particularly as patients are expected to use eHealth technologies when seeking healthcare services. This shift underscores the necessity for a comprehensive understanding of EPEB, aiming to establish a well-defined framework and evaluation dimensions. In response, the development of a scale specifically designed to measure patient engagement behaviors in an eHealth context is proposed. This scale would provide healthcare providers and researchers with critical insights into the extent of patient interaction with eHealth technologies and the impacts of these behaviors on health outcomes, thereby guiding strategies to optimize patient engagement in the digital era.

Several existing instruments including the Patient Activation Measure,²⁷ Patient Health Engagement Scale,²⁸ Patient Engagement Index,²⁹ and Patient Engagement in Health Care Questionnaire³⁰ have been developed to measure patient engagement. However, these instruments primarily may not adequately capture the multifaceted dimensions and items reflective of new behaviour traits, such as seeking health information online, attending virtual consultations, and tracking health data using wearable devices. This limitation potentially leads to inaccuracies in measurement and hinders our understanding of the impact and effectiveness of eHealth interventions on patient engagement outcomes. As eHealth technology becomes more deeply integrated into healthcare, it is essential to develop a new scale specifically designed to measure EPEB within this rapidly evolving landscape. Our scale is designed to bridge this gap, offering a more fitting and sensitive tool for measuring how patients engage with their health care in the eHealth context.

With the increasing integration of eHealth technology into healthcare, developing a new scale specifically designed to measure Electronic Patient Engagement Behaviors (EPEB) within this rapidly evolving landscape becomes crucial. Our scale is designed to bridge this gap, offering a more accurate and relevant tool for measuring how people engage with their healthcare in the context of eHealth.

Therefore, there are two research questions in this paper, (1) What are the key dimensions of the concepts of EPEB; (2) What are the psychometric properties of a newly developed scale to measure the EPEB?

Materials and Methods

Scale Development

The EPEB scale was designed as a generic instrument with components aimed at assessing electronic patient engagement behaviors in general outpatient or inpatient settings that leverage electronic health technologies and platforms. Based on the theoretical frameworks of engagement behavior framework¹ and engagement capacity framework,³¹ the researchers procured their items of EPEB scale through a systematic process, which involved the following steps (Figure 1): (a) performing a literature review regarding the concept of patient engagement and electronic health to identify and gather relevant dimensions and items from existing measurement instruments that had been used in previous studies, (b) conducting semi-structured interviews with healthcare professionals and patients for gathering qualitative data and insights regarding behaviors of patient engagement in healthcare, (c) synthesizing the information obtained from the literature review and semi-structured interviews in order to gain a comprehensive understanding of the dimensions and potential items for measuring EPEB, (d) determining the theoretical framework and components of EPEB and initial item pool for measuring EPEB (Figure 2), (e) engaging a panel consisting of a physician with extensive industry experience, a registered nurse, a hospital administrator with knowledge in hospital management, a healthcare researcher, and a patient to revise and assess the clarity and appropriateness of the measurement items. The physician and nurse provide clinical insights, the administrator offers operational viewpoints on integrating eHealth, the researcher ensures academic rigor and relevance, and the patient represents the end-user experience. (f) conducting a pilot testing phase to evaluate the effectiveness of the developed EPEB scale in practical settings. A convenience sample of patients was enrolled in the pilot survey to verify the availability of the initial scale and further reduce the item pool.

Figure 1 The process of developing the measured items for the EPEB scale.

Figure 2 The EPEB scale components.

The initial EPEB scale in this study consisted of 20 items divided into four subscales: (1) disease information search (four items), (2) doctor–patient interaction (nine items), (3) social interaction among patients (four items), and (4) disease self-monitoring (three items). A 5-point Likert rating scale was adopted to measure the responses due to its simplicity for respondents’ understanding, and suitability for statistical analysis, and alignment with common survey practices. Ranging from “strongly disagree” to “strongly agree”, this rating scale enabled participants to express their degree of agreement or disagreement with each item. The total score on the scale was calculated by summing the scores of all the items. Higher total scores indicate a higher level of electronic engagement in healthcare. This summation provides an overall measure of participants’ level of engagement, enabling researchers to compare and analyze responses across different individuals and groups.

Validation Framework

The validation framework consists of two studies to ensures a comprehensive validation process. In Study 1, a pilot survey was conducted to determine the dimensions of the EPEB using statistical methods and to further reduce the initial measuring items. The questionnaire is evaluated for ensuring understanding of questions, answer options, and length of items are evaluated. The methodologies employed included reliability testing to assess the consistency of the scale, exploratory factor analysis to uncover the underlying structure of the data, and Rasch analysis to evaluate the item responses for their alignment with the latent trait being measured. Subsequently, in Study 2 the final scale was administered to a larger sample of participants to assess its psychometric properties. The psychometric evaluation included item analysis, reliability tests, validity tests, and cross-sex measurement equivalence tests.

Participants and Data Collection

The data for the pilot survey were collected between April and May 2022 at a leading hospital in Wuhan, China. For the validation survey, data were collected from three first-class hospitals in Hubei Province, China between June and July 2022. Convenience sampling was employed in both studies to ensure efficient data collection. Participants in both studies met the following inclusion criteria: (1) aged 18 years and above, (2) clear-minded with no language communication barriers, and (3) willing to participate in the survey. Following Jackson’s recommendation of a minimum 1:10 sample-to-parameter ratio for maximum likelihood estimation of structural equation modeling,³² the sample size in this study should be no fewer than 200 participants. The pilot survey had a sample size of 312 participants, whereas the subsequent validation survey encompassed a larger sample size of 853 individuals.

Statistical Analysis

Data analysis involved two sample sets. The first sample set was used to conduct reliability tests and exploratory factor analysis (EFA) using IBM SPSS software version 25. To assess the reliability of the initial scale, two measures were employed: corrected item-total correlation (CITC) values and Cronbach’s α coefficient. The following steps were undertaken for the exploratory factor analysis: (1) KMO test and Bartlett’s sphericity test were conducted to determine the suitability of the data for factor analysis. (2) Principal component analysis was performed to extract factors based on the criterion of eigenvalues greater than one. Orthogonal rotation was applied using the maximum variance method.

Additionally, Rasch analysis was conducted as a complementary analysis to provide additional valuable information not fully addressed by EFA. This analysis involved examining item-fit statistics, including infit mean square (InfitMNSQ) and outfit mean square (OutfitMNSQ). By assessing these indices, this study aimed to determine the unidimensionality of the subscales and ensured that the selected items were appropriate in their content relevance. Winsteps software version 3.66.0 was used for Rasch measurement.

The second sample set was analyzed using IBM SPSS software version 25 and IBM SPSS AMOS software version 24. This included item analysis, reliability testing, validity testing, and cross-sex measurement equivalence testing. Item analysis comprised two approaches: the critical ratio method and the correlation coefficient method. These methods allow for a comprehensive examination of the individual items’ performance, highlighting any items that may require further scrutiny or potential removal from the scale. Reliability testing was performed to assess the consistency and stability of the scale. This involved examining both internal consistency, which measures the extent to which the items in each subscale are interrelated, and split-half reliability, which assesses the scale’s reliability by splitting it into two halves and comparing the results. Construct validity, specifically content and structural validity, was evaluated to enhance confidence in the proposed factors derived from the EFA. Content validity was assessed using the S-CVI (scale-level content validity index) and I-CVI (item-level content validity index). Confirmatory factor analysis (CFA) was then performed to test the structural validity and determine whether the data aligned well with the hypothesized factor structure. Finally, to ensure the scale’s applicability across different groups and populations, a multiple-group analysis was conducted using sex as the differentiating factor. This analysis involved both single-group CFA, examining male and female group independently, and multi-group CFA, comparing the factor structure across different sex groups.

Results

Demographic Characteristics

The majority of participants were female (63%), aged between 35 and 50 years (38%), married (74%), and residing in urban areas (83%). Most participants lived with more than three family members (61%) and held a bachelor’s degree (52%). Most participants reported a monthly family income in the range of 3500–5000 yuan (33%). The demographic profile of the participants in the validation survey closely resembled that of the participants in the pilot survey. The majority of participants were female (66%), aged between 35 and 50 years (36%), married (78%), and living in cities (77%). Most participants lived with more than three family members (64%) and held a bachelor’s degree (41%). Most participants had a monthly family income of 3500–5000 yuan (34%).

Pilot Survey

Reliability Test

First, we used a dataset collected from 312 patients (pilot survey). The study conducted a reliability test on the initial scale and purified the initial items according to the following two standards, including deleting items with a CITC not significant (P<0.05) or CITC value lower than 0.50 and removing items whose deleted reliability coefficient (Cronbach’s α) equal or higher than the overall reliability coefficient. The results of the reliability test are shown in Table 1. All items displayed CITC values greater than 0.50. However, after item 6 and item 17 were removed, the Cronbach’s α values of their respective dimensions increased. This item purification process ensured that all dimensions achieved Cronbach’s α values greater than 0.8, indicating a high level of internal consistency for the scale.

Table 1 Reliability Test, Exploratory Factor Analysis and Rasch Analysis of the Pilot Survey

Exploratory Factor Analysis

The results revealed a KMO value of 0.903 and a significant Bartlett’s test value of 4369.407 (P<0.001), confirming the suitability of the data for factor analysis. By conducting principal component analysis, 4 common factors with eigenvalues greater than one were extracted. These factors accounted for a cumulative variance contribution rate of 69.411%, indicating that they successfully explained a substantial portion of the data variability. The 4 extracted factors from the data could be reasonably conceptualized as 4 dimensions: disease information search, doctor–patient interaction, social interaction among patients, and disease self-monitoring.

Each item demonstrated a factor loading ranging from 0.443 to 0.859, indicating strong associations with their respective factors. Moreover, the commonalities of all items exceeded 0.4 (Table 1). However, it was observed that items 5, 6, and 17 displayed cross-loadings greater than 0.4 with differences less than 0.2; thus, these items should be considered for removal from the EPEB scale to ensure the robustness and clarity of the factor structure.

Rasch Analysis

The item fit measure, specifically the mean-square fit statistics (MNSQ infit/outfit), along with the point-measure correlation were examined, and the results are presented in Table 1. In Rasch analysis, items with MNSQ infit/outfit values falling within the range of 0.8–1.4 logits are considered acceptable Among the 20 items initially included in the scale, 17 met this criterion, indicating good fit with the underlying Rasch model. However, it was observed that three items, namely, items 8, 10, and 12, did not meet the acceptable fit criteria. Furthermore, the results of the point-measure correlation (PTMEA-CORR) were satisfactory, as they fell within the desired range of 0.4–0.85. This indicates a reasonable relationship between the items and latent traits being measured. However, further consideration is needed for the three items that do not meet the fit criteria and may require revision or removal from the EPEB scale.

Based on a comprehensive analysis of the reliability test, exploratory factor analysis, and Rasch analysis, the research team decided to eliminate items 5, 6, 10, 12, and 17 from the scale. Subsequently, an exploratory factor analysis was conducted again, and the results are presented in Table 2. The analysis revealed the presence of four distinct common factors, with each item demonstrating a factor loading exceeding 0.6. Furthermore, none of the items displayed cross-factor loadings exceeding 0.4, indicating that they were unambiguously associated with their respective factors.

Table 2 Exploratory Factor Analysis After Removing Unqualified Items

Validation Survey

Item Analysis

In total, 853 patients were included in the validation survey. First, item analysis was used to screen items, including the critical ratio and correlation coefficient methods. (1) Critical ratio method: The obtained samples were sorted by the total score of the EPEB scale from highest to lowest; the top 27% of the scale scores were included in the high group and the bottom 27% of the scale scores were included in the low group. Independent sample t-tests were conducted for the two groups, and the results showed that the mean scores of the items in the high group were higher than those in the low group, and the differences were statistically significant (P<0.001). (2) Correlation coefficient method: The Pearson correlation coefficient method was used to analyze the correlation between each item and the total score. The results showed that the correlation coefficients ranged from 0.632 to 0.802, and the P values were all less than 0.001. There were no items with coefficients less than 0.4 (Table 3).

Table 3 CR and Item-Total Correlation for Each Item of EPEB Scale

Reliability Test

We conducted a reliability test on the scale, including internal consistency and split half reliability: (1) Internal consistency: the Cronbach’s α coefficient of the EPEB scale was 0.932, and the Cronbach’s α coefficient of each dimension was 0.865, 0.904, 0.904, 0.900, respectively (Table 4). (2) Split-half reliability: the results showed that the Spearman-Brown split coefficient of the EPEB scale was 0.963.

Table 4 The Reliability and Convergence Validity of the EPEB Scale

Validity Test

Validity tests, including content validity and structural validity: (1) Content validity: the S-CVI of the EPEB scale was 0.911 and the I-CVI was between 0.833 and 1.000. (2) Structural validity: first- and second-order four-factor structural equation models of the EPEB scale were constructed using AMOS software, and confirmatory factor analysis was conducted. Simultaneously, the four-factor model was compared to single-factor, two-factor, and three-factor models. In the two-factor model, disease information search and doctor–patient interaction were combined as one factor, and social interaction among patients and disease self-monitoring were combined as the second factor. The three-factor model included disease information search as one factor, doctor-patient interaction and social interaction among patients as the second factor, and disease self-monitoring as the third factor.

The specific fitting values of each fitting indices are listed in Table 5, and the results show that the four-factor model had the best fit. Both first- and second-order models met the statistical requirements. A structural diagram of the scale is shown in Figure 3. In addition, the results of the CFA showed that the standardized factor loading of each item was above 0.6, the composite reliability was above 0.8, and the AVE was above 0.5, indicating that the scale had good convergence validity (Table 4). The results of the discriminant validity analysis showed that the AVE square root of each factor was greater than the correlation coefficient between the factor and other factors and that the internal correlation of the factor was greater than the external correlation, indicating that the scale had good discriminant validity.

Table 5 Fit Indices of Factor Structure of EPEB Scale

Figure 3 Path coefficient of four-factor EPEB scale.

Cross-Sex Measurement Equivalence Test

We conducted a cross-sex measurement equivalence test on the EPEB scale, including single- and multi-group confirmatory factor analyses. First, single-group confirmatory factor analysis was used to test the fitting effect of the four-factor model of the electronic patient engagement behavior scale in male and female patient populations. The results show that the fitting indices of both groups met the standards (Table 6). Second, a multi-group factor analysis was used to test the measurement equivalence between male and female patients. The results show that the fitting indices of the morphological equivalent model (M), weak equivalent model (M1), strong equivalent model (M2), and strict equivalent model (M3) met the measurement standards (Table 6).

Table 6 Cross-Sex Measurement Equivalence Test of EPEB Scale

Discussion

This paper proposed a definition of EPEB and provide a multidimensional conceptual model. To measure EPEB, the EPEB scale finally formed in this study contains 15 self-reported items in 4 dimensions which comprehensively reflected patients’ behavioral characteristics. Two patient samples from Hubei Province, China were used to test these measurement items. Exploratory factor analysis and confirmatory factor analysis were used to determine the final structure and content of the EPEB scale, and its reliability and validity were assessed.

Dimensions of EPEB Scale

The EPEB scale incorporates common components such as information acquisition, physician communication, social support, and self-management, which is consistent with previous research on patient engagement.^33–36 The first dimension of the scale, disease information search, reflects the behavior of patients utilizing various internet technologies to search for relevant information regarding diseases and treatments. This fundamental engagement behavior can help patients improve their understanding and awareness of their own diseases, enhance confidence in their ability to care for themselves.^37–39 The second dimension, doctor-patient interaction, refers to patients’ communication with their healthcare professionals more easily and efficiently in a combined online and offline manner, including request, consultation, feedback and evaluation.^40–43 The third dimension, social interaction among patients, involves patients using the internet or mobile devices to engage in socializing and sharing with others, including exchanging experiences, seeking advice and emotional support in online patient community. This behavior benefit patients’ mental well-being and even compliance with treatment through educational and emotional support from individuals facing similar health issues.^44–47 The fourth dimension, disease self-monitoring, highlights behaviors of recording and monitoring patient health status by smart health apps or wearable electronic devices that provide self-care support and improve healthy behaviors. The generated content such as everyday diet, blood pressure, blood sugar, and heart rate, can facilitate to identify abnormal conditions and reduce the risks of disease.^48–50 This behavior can help patients feel more empowered in managing their own health.^51,52 Overall, these dimensions represent key aspects of patient engagement in the eHealth era.

The EPEB scale also possesses unique features. Previous studies explored the measurement of patients’ active roles in care. For instance, the Patient Activation Measure (PAM) incorporates cognition, skills, actions, and beliefs to assess patient activity. Patient Health Engagement (PHE)⁵³ mainly focus on evaluating psychological effects on patients’ lives. While the Patient Engagement Index (PEI)²⁹ and Patient Engagement in Health Care Questionnaire³⁰ have made progress in revealing the behavioral characteristics of patient engagement, they have certain limitations when considering the influence and changes brought about by the eHealth environment. By contrast, the EPEB scale incorporates the impact of information technology, such as the use of online platforms to communicate with doctors or search for disease-related information, joining online communities for social sharing, and the use of wearing devices or smart apps to monitor personal health conditions.

The Psychometric Properties of the EPEB Scale

The findings demonstrate that the EPEB scale has good reliability and validity, indicating that it is a reliable and valid measurement tool for evaluating patient engagement behavior in an eHealth environment. The CFA results revealed that the four-factor model fit the data better than the single-, two-, and three-factor models, substantiating the multidimensional conceptual model of EPEB proposed in this study. The reliability analysis results showed that the Cronbach’s α coefficients of each subscale were above 0.8, and the total Cronbach’s α coefficient was 0.932, indicating good internal consistency reliability. The Spearman-Brown split-half coefficient was 0.963, indicating excellent split-half reliability. All standardized factor loadings were above 0.6, and composite reliability was above 0.8, indicating good convergent validity. Discriminant validity was established as the AVE for each factor surpassing the correlation coefficients between that factor and the other factors. The results of the measurement invariance test confirmed that the EPEB scale had configural, metric, and scalar invariance across sex, indicating that it is equally applicable to both male and female patients. These findings support the reliability and validity of the scale, and provide robust evidence that it is a valuable tool for measuring patient engagement behaviors in the eHealth context.

Research Implications

The development of the EPEB Scale has significant practical and theoretical implications. The EPEB conceptual model facilitates a comprehensive understanding of the characteristics and strategies of patients engaged in medical-related activities in an eHealth environment. Moreover, this study provides a reliable and effective measurement tool that assists healthcare professionals and managers in identifying and addressing the needs and expectations of individual patients in an eHealth environment. Furthermore, this tool has the potential to help patients understand and improve their status of engagement in health-related activities, thereby enhancing their health status and overall well-being. To achieve these objectives, healthcare providers can implement various interventions and recommendations based on different dimensions of the EPEB model. These include the provision of reliable online health information resources, enhancing online interactions between physicians and patients, establishing effective and friendly peer support networks, and developing personalized and intelligent self-management tools. Ultimately, the EPEB scale may contribute to the delivery of more effective and patient-centered healthcare services in eHealth.

Limitations and Future Work

Although this study developed an effective scale, it had several limitations. First, the concept of EPEB remains relatively underdeveloped and requires further refinement. In addition, rigorous testing for cultural compatibility is necessary to ensure the applicability of the scale across diverse populations. Moreover, the scarcity of relevant literature on EPEB limits the scope of this study and highlights the need for further research to better understand this construct. Second, the selection of hospital patients from Hubei, China, as the primary sample may have restricted the generalizability of the findings to broader patient populations. Furthermore, owing to the study’s exclusive focus on the structural dimensions of the EPEB, the important influencing factors and their interrelationships with other relevant variables were not explored. To address these limitations and advance the field, future studies should consider various approaches. First, a larger and more diverse sample should be employed to enhance the external validity of the scale. Additionally, more comprehensive data collection methods such as in-depth interviews and observational techniques can supplement quantitative measures to capture the motives, attitudes, and barriers to EPEB. Third, examining the predictive value of the scale by exploring its relationships with other patient-related variables such as satisfaction and compliance may strengthen its practical utility. Lastly, cross-cultural or cross-national comparative studies could offer valuable insights into similarities and differences in EPEB across different socio-cultural contexts.

Conclusion

In conclusion, the EPEB scale developed in this study is a reliable and comprehensive tool for measuring patient engagement behaviors in the eHealth context. The development and validation of the scale represents a notable contribution to the growing body of knowledge on patient engagement. It has potential value on developing tailored interventions aimed at promoting patient engagement in the digital era, and it also facilitates a deeper understanding of the influence of eHealth technologies on patient engagement for healthcare providers and researchers. Continued research efforts are crucial to refine and validate the findings, and ultimately translate them into actionable strategies for improving patient-provider relationships, shared decision-making, and overall healthcare quality.