Construction of Internet +home Care Quality Supervision Indicators in China Based on the Delphi Method

Introduction

With the development of the social economy and medical care in China, demands upon the health service are becoming increasingly diversified, and the provision and conceptualization of nursing are deepening and extending.¹ On February 12, 2019, the National Health Commission issued the Notice on Launching the “Internet +home care” Service Pilot Work in China.¹ Internet +home care refers to medical institutions using institution-registered nurses to provide door-to-door nursing services for discharged patients/patients suffering from diseases with limited mobility through the “online application, offline service” mode. In the same year, Internet +home care was piloted in some cities in China.² Initial results are now available, although the implementation is still in the exploratory stage. Relevant institutional measures are not yet standardized or perfect, and issues include high pricing and poor information connectivity, an unclear payment guarantee system, and an absence of a standardized nursing staff performance appraisal system.³ As such, constructing scientifically based Internet +home care quality supervision indicators (QSIs) is essential for safeguarding the benefits for patients and medical staff.

In the United States, Japan, and other countries, Internet +home care was implemented earlier. These countries have established practical approaches to evaluating the quality of home care, including the Outcome and Assessment Information Set,⁴ the Resident Assessment Instrument-Home Care,⁵ the Quick Investigation of Quality, and other home care assessment tools.⁶ There are differences in the operation mode of Internet +home care in different countries. Moreover, differences in domestic and foreign national conditions, cultures, and customs existed. Therefore, quality supervision indicators as used abroad cannot be directly applied in China.

According to the main body of the service, the model of Internet +home care can be divided into two categories. One is that medical institution nurses provide a door-to-door service, which is the service model adopted by countries such as China and Japan. The other is when company nurses provide services, such as Uber in the United States and Portea Medical in India. There are differences in the operation mode of Internet +home care in different countries.⁷ In this study, the constructed Internet +home care QSIs are applicable to the service operation model in China.

In evaluating nursing service quality, the commonly used three-dimensional quality structure theory, Quality Control Circle, and Plan-Do-Check-Act cycle theory constitute active evaluation by service providers, while feelings about recipients may not be sufficiently considered. The SERVQUAL (Service-Quality) evaluation method proposed by Parasuraman is a method by which to evaluate service quality from consumers’ perspectives.⁸ The service quality is evaluated according to the score differences of consumers’ “feelings” and “expectations” of service from the five perspectives of tangibility, reliability, guarantee, responsiveness, and empathy.⁹ Therefore, based on the SERVQUAL model, we constructed a series of Internet +home care QSIs centered on patient experience, suitable for Chinese national conditions. Importantly, these QSIs focus on the connotation of nursing services and provide an empirical basis for standardizing and guiding the healthy development of Internet +home care.

Materials and Methods

Participants

The semi-structured interview participants were medical managers and nursing managers at tertiary hospitals, patients, and family members. The focus discussion group was composed of multidisciplinary experts from the Ministry of Nursing, Medical Care, Education of China-Mobile Laboratory, and other relevant departments.

The inclusion criteria for the Delphi experts were: 1) appointed at deputy high school level or above; 2) holding a bachelor’s degree or above; 3) 10 years or above of hospital nursing management or clinical nursing work experience; 4) Familiar with SERVQUAL theory. Exclusion criteria included refusal to participate in the research or providing invalid questionnaire responses.

Methods

Literature Review

We searched PubMed, MEDLINE, Web of Science, China National Knowledge Infrastructure (CNKI), and Chinese Wanfang databases for eligible studies published between 2011 and 2021. The following keywords or medical terms were used: “Internet +home care, home care, quality supervision, SERVQUAL, nursing quality indicators, Delphi method.”

The inclusion criteria for articles were: (a) published in English or Chinese; (b) content related to this current study with the title, abstract, keywords, or text containing the search terms. The exclusion criteria were: (a) duplicate publication; (b) defects in the study design and/or poor quality. The Johns Hopkins body of evidence grade and quality assessment standards were used to evaluate the quality of the included literature, and the quality indicators related to Internet +home care involved in the literature were extracted and classified.

Semi-Structured Interviews

The research team developed an interview outline based on the literature review and clinical experience (see Figure 1). Meaningful items and phrases were marked in the interview, the same or similar materials were integrated, and the interview contents were refined and classified. The face-to-face semi-structured interviews were conducted with attendees wearing protective masks at a distance of 1 meter apart. Before the interview, the interviewee was introduced to the research purpose and method. After obtaining informed consent, the interview outline was followed in which participants were guided in describing their thoughts on the Internet +home care QSIs. The interview time was set to 30–45 minutes per interview, in which the interview quantity was information saturation and no new information appeared.

Figure 1 Semi-structured interview outline.

Focus Group Discussions

Based on the literature review results, semi-structured interviews, and the professional knowledge of the focus group, experts initially determined the Internet +home care QSIs based on five aspects: tangibility, reliability, guarantee, responsiveness, and empathy.⁷ All primary indicators were discussed and the connotation definition and qualitative measurement of each indicator were repeatedly demonstrated. The indicators were gradually refined and clarified at all levels to ensure the final QISs were more logical and scientifically based.

Delphi Consultations

The Delphi questionnaire consisted of four parts: (a) basic questionnaire introduction; (b) questionnaire regarding basic information about the experts; (c) evaluation of the importance and relevance of each indicator and suggestions from experts; (d) each experts’ familiarity with the content of the questionnaire and the basis for judgment. After completing the first round of consultation, the analysis, feedback, and revision were organized and the second round was formed and distributed. It was agreed that the consultation would be stopped if the expert opinions showed a relatively concentrated trend; otherwise, the third round of consultation would be carried out until the Internet +home care QSIs based on the SERVQUAL model was determined.

Experts completed the Delphi questionnaires by email. The principles for selecting and revising indicators were as follows: (a) if the importance of the indicator was less than 3.5 and the coefficient of variation was greater than 0.30, the indicator would be deleted; (b) if experts suggested adding or deleting some indicators, the focus group would discuss and review the literature to decide whether to adopt and modify the indicators; (c) for the revision suggestions or questions raised by experts on a specific indicator, the researchers would refer to the suggestions and give feedback to the experts. After the first round of consultation, if an expert noted that they were not sufficiently familiar with the field or if the returned questionnaire did not meet the filing requirements or was of low quality, the expert was not invited for the subsequent consultation.

Statistical Analysis

The general demographic data were described in frequency, percentage, mean and standard deviation (mean±SD). The expert positive coefficient refers to the recovery rate of the questionnaire, calculated as the ratio of experts number participating in the consultation relative to the total number of experts. Two factors determined the degree of authority (Cr): (Ca) the educational level of the expert and the basis for judgment (experience, theoretical analysis, reference to data at home and abroad, and intuitive feelings), and (Cs) the degree of the expert’s familiarity with the questions such that: Cr=(Ca+Cs)/2. The degree of coordination of expert opinions is represented by a coordination coefficient (Kendal’s concordance coefficient, W) and coefficient of variation. Factor analysis was used for weight analysis. The common factor variance of each indicator was obtained by principal component analysis. Moreover, the specific weight of each indicator was obtained by calculating the percentage of each common factor variance to the sum of the common factor variances. All statistical tests were two-sided, and analyses were performed using SPSS 24.0. P<0.05 determined statistical significance.

Results

Literature Review

A total of 208 articles were retrieved on the initial search, of which 6 articles were presented for review by the expert panel.^10–15 The flow chart of the literature search and retrieval is shown in Figure 2. We identified 5 first-level, 20 secondary, and 72 tertiary indicators.

Figure 2 Literature quality and grade assessment results: Level III (A) 2 articles, Level III (B) 3 articles, Level IV (A) 1 article.

Semi-Structured Interviews

After the literature review, 3 medical managers, 5 nursing managers, 5 patients, and 5 family members from tertiary hospitals were selected for semi-structured interviews. In total, 1 secondary and 2 tertiary indicators were eliminated, and 5 first-level, 19 secondary, and 70 tertiary indicators remained (Figure 3).

Figure 3 Selection of Internet +home care quality supervision indicators.

Focus Group Discussions

The multidisciplinary experts’ panel comprised three nursing staff, two medical staff, and three Chinese Ministry of Education-mobile laboratory staff. The panel discussed the significance and applicability of these indicators and finally reached a preliminary consensus. This process eliminated 1 secondary and 4 tertiary indicators, and facilitated the formation of the first round of the expert survey (Figure 3).

Delphi Consultations

Participants’ General Characteristics

Two rounds of expert consultations using the Delphi method were conducted with 20 and 17 expert members (in rounds 1 and 2, respectively). In both rounds, the majority of participants were female (90.0% and 88.2% in rounds 1 and 2, respectively), and most participants were aged between 41 and 50 years old (80.0% and 88.2% in rounds 1 and 2, respectively). Over 90.0% of the participants held senior professional titles (95.0% and 100.0% in rounds 1 and 2, respectively), and over half of the participants had master’s degrees (60.0% and 70.6% in rounds 1 and 2, respectively). Table 1 displays the general characteristics of the participants.

Table 1 Demographic Characteristics of the Participants [n(%)]

Reliability of the Delphi Method

The recovery rates for the two rounds of consultation were 100% and 85%, respectively, indicating that the experts were motivated and attached great importance to this topic. The Cr for the two rounds of expert inquiry were 0.810 and 0.833, respectively, and the average Cr was 0.822, indicating that the authoritativeness of the experts was high.

Kendall’s coefficient of concordance was calculated for each QSI. The coefficient of variation for the first round of consultation ranged from 0 to 0.128, and Kendall’s W was 0.189 (P<0.01). The second-round consultation coefficient of variation ranged from 0 to 0.080, and Kendall’s W was 0.125 (P<0.01), indicating high consistency in experts’ opinions.

Indicator System Revision and Weight Determination

The statistical results (means and standard deviations) and weights for the Internet +home care QSIs are presented in Table 2. Three participants suggested the naming and connotation of the secondary indicators. For example, they believed “timeliness” in responsiveness should be defined as “service response speed”. In addition, “establish a particular working group for Internet +home care” in tangibility and “carry out safe and reliable Internet +home care service” in reliability were suggested by four participants. Two participants suggested that some names and connotations for the QSIs should be modified. For tertiary indicators, participants recommended adding “there are clear job responsibilities and work procedures for Internet office managers” and “there are clear job responsibilities and work procedures for Internet platform staff” in tangibility, for example. Moreover, pairs of tertiary indicators were proposed to be combined separately, such as “standard attire, wearing a work badge” and “dignified appearance and decent behavior” were combined into “service staff should dress in standard attire, wear work badges, be dignified, and behave appropriately”. Based on discussions with the research team and after considering the experts’ comments, the following consensus was reached: add two secondary and eight tertiary indicators, delete three secondary and three tertiary indicators, combine three pairs of tertiary indicators separately, and adjust the connotations of some items.

Table 2 Internet +home Care Quality Supervision Indicators: Importance and Weight Determination

In the second round of consultation, one expert proposed adding one secondary indicator, “Internet +home care staff professional guarantee” to the guarantee. One secondary indicator was deleted and one was modified. In terms of the naming, connotation, deletion, and addition of tertiary indicators, three participants submitted their opinions. One tertiary indicator was added and two tertiary indicators were modified. Ultimately, the Internet +home care QSIs contained 5 first-level, 17 secondary, and 69 tertiary indicators.

Discussion

The Internet +home Care QSIs Based on the SERVQUAL Model is Highly Scientific and Reliable

Based on the SERVQUAL theoretical framework applied to the situation in China, the Internet +home care QSIs constructed by literature review, semi-structured interviews, focus group discussions, and Delphi expert consultations are scientifically based.¹⁶ In the construction of the initial indicator database, after several focus group discussions and expert interviews, all aspects of Internet +home care QSIs were comprehensively considered and set, ensuring the reliability of the indicators.

The experts’ qualifications, positive attitudes, authority, and consistency of opinions confirm the scientific basis and reliability of the research results. We selected multidisciplinary experts with abundant experience in clinical nursing, nursing management, and healthcare. The authority coefficients of the experts in the two rounds were above 0.8. Moreover, the experts’ professional titles and educational levels also showed that they had high authority and reliability. The effective recovery rates of the two rounds of Delphi consultation were more than 70%, reflecting the active participation of experts participating in this research. Kendall’s concordance coefficient for the two rounds indicated that the experts reached a consensus on each indicator’s importance, ensuring the indicators’ reliability. From the comprehensive data analysis, the Internet +home care QSIs achieved the expected goals in terms of scientificity and reliability. Overall, the indicators obtained in this study are reasonable and thus, can be used for Internet +home care quality supervision.

Content Analysis of Internet +home Care QSIs Based on the SERVQUAL Model

We established the Internet +home care QSIs, including 5 first-level, 17 secondary, and 69 tertiary indicators. Among the first-level indicators, the top two weights of indicators are reliability (29.49%) and guarantee (26.99%), which are consistent with the characteristics of the Internet +home care model. As a new service model, Internet +home care’s complete hardware, software, and serviceability guarantee is its necessary condition. Medical institution qualifications and hardware and software configurations are directly proportioned to the level of serviceability of providing door-to-door nursing staff. This reminds us that medical institutions that carry out Internet +home care should further refine the requirements and improve the qualifications and service capabilities, to provide better care for patients needing Internet +home care. Furthermore, tangibility, responsiveness, and empathy quality indicators cannot be ignored.

Tangibility refers to the elements directly perceived by patients, including software and hardware configurations, and human resources. Tangibility in this study includes 4 secondary and 16 tertiary indicators, whereby the top 3 weights of secondary indicators are “establish a particular working group for Internet +home care”, “there is an information platform for Internet +home care” and “there are institutions for Internet + home care”. The indicators reflect the importance of a fully staffed work team, a software information platform with supporting functions, and a medical institution with complete qualifications to implement nursing services. In cases the inadequate qualifications of staff and the resources are insufficient, the credibility of service level will be low, and the quality of service will be affected.¹⁷

Reliability refers to performing service safely, reliably, and accurately. Focusing on the service platform and service levels, the guarantee of door-to-door nursing safety is the core element of Internet +home care.¹⁸ The highest weight in secondary reliability indicators is “Data regarding the Internet +home care information platform is safe and reliable”. The big health data in Internet +home care is vast and contains much private and medical information. It is necessary to avoid network attacks, and at the same time, patients have the right to privacy, and medical staff must protect patients’ privacy.¹⁹ Among the tertiary indicators of reliability, the highest weight is “the information platform can realize a one-click alarm and synchronously connect to the public security system and the operation management backstage”. Internet +home care is currently in the trial operation stage in China, and safety issues cannot be avoided. It should be noted that users have concerns about the qualifications of nurses and the safety of medical care in Internet +home care.²⁰ Nurses’ concerns are concentrated on the safety of nursing operations, nurse-patient conflicts, and medical disputes,²¹ and so it is essential to improve security protection.

Guarantee refers to the standardization of management and the assurance of the professional skills of relevant service staff. The highest weight indicators among the secondary guarantee indicators are “Internet +home care staff professional guarantee” and “Internet +home care supervision system is complete”. Qualified Internet +home care staff provide sufficient guarantee for patients and interconnected systems should be established to guarantee the working years, professional titles, training, and assessment of nursing staff. Moreover, establishing complete and reasonable QSIs is the foundation of Internet +home care and an important quality and safety guarantee. Therefore, it is recommended that government and relevant management departments establish relevant policies and systems to provide platform users with better Internet +home care.²²

Responsiveness represents service staff responding quickly to patients demands and providing timely assistance. The secondary indicator with the highest weight in responsiveness is “service response speed”. The platform’s proper order dispatching strategy, timely order response, and active order confirmation will significantly reduce time consumption and improve efficiency. Among the tertiary indicators, the indicators with the highest weight are “platform functions can be optimized in a timely fashion according to patients’ needs and suggestions”, “Internet +home care staff receive orders and contact patients in time”, and “Internet +home care staff educate patients on health in a timely manner”. It is necessary to respond to patients’ orders in a timely fashion, pay attention to the requirements, evaluations and suggestions of patients, and take targeted improvement measures. It is recommended that nursing staff comprehensively evaluate the physical and psychological conditions of patients and provide appropriate health care education when providing nursing services to patients. In addition, such instruction should be personalized, easy to understand, and effective.

Empathy refers to concern about and caring for patients, being able to put oneself in the patients’ shoes, and providing individualized services for patients. After a two-round expert Delphi consultation about empathy, the secondary indicators are “provide humanized service” and “provide personalized service”. In nursing, being “people-oriented” should be implemented throughout the nursing process to maximize the nursing service experience for patients. Nursing staff must have a humanitarian spirit, and focus on patients’ physical and mental health, provide emotional and spiritual support, and ensure patients feel humanized. It is equally essential to provide personalized services to patients. Everyone is a unique individual; nursing staff should respect the individuality of patients, focus on the interests and demands of patients when providing services, and try their best to meet the psychological and physiological needs of patients as diverse social people. Therefore, nurses engaged in Internet +home care need to be able to identify patients’ needs, formulate personalized service contents, establish personalized electronic files for patients, and deliver personalized health education knowledge according to patients’ characteristics.²³

Limitations and Conclusions

The reliable and scientifically based Internet +home care QSIs identified here meet the needs of society, and the development of the Internet medical system has a guiding role in the improvement of Internet +home care. Cheng et al²⁴ found that Internet +home care is in high demand among Chinese residents and is highly accepted by nursing staff.²⁵ National policy vigorously promotes the development of Internet +home care, indicating a promising prospect in China. However, there is very little research on the quality supervision of Internet +home care in China. Therefore, based on the SERVQUAL theoretical system, we constructed the Internet +home care QSIs to standardize and guide the healthy development of Internet +home care, provide a reference for nursing managers and nursing educators in pilot hospitals, and provide a new direction for the standardized implementation of Internet +home care. Due to time constraints, we constructed the Internet +home care QSIs theoretically and did not conduct empirical research on Internet +home care. In future research, we will build an Internet +home care quality supervision platform based on the established Internet +home care QSIs, and continue to supervise and improve the quality of Internet +home care.