Using Multilevel Structural Equation Modeling (MSEM) to Identify the Predictors and Influencing Mechanism of Technology Use Among Chinese Physicians: An Example from Des-Gamma-Carboxy Prothrombin (DCP)

Introduction

It is well known that expanding the use of appropriate and effective health technologies would greatly benefit the early diagnosis and treatment of some major diseases, which would subsequently reduce the heavy burden of the corresponding diseases on individuals, families and society. For example, as one of the major diseases worldwide, liver cancer ranks the second leading cause of cancer-related deaths.¹ Especially in China, the number of new cases and deaths from liver cancer increases dramatically every year, accounting for more than half of the world’s cases.² Clinical practice shows that many patients with liver cancer are detected only after the onset of abdominal pain. By then, most cases are already in advanced stages, with extremely high treatment costs and low survival rates. Under the critical situation of liver cancer prevention and treatment, promoting the use of early diagnosis and treatment techniques for related diseases will be beneficial to improve the patients’ survival rate and quality of life, and enhance the service delivery capacity of the health system. In order to design targeted interventions that effectively promote the use of corresponding technologies, it is of great theoretical and practical significance to identify the determinants and elucidate their interactions and overall impact mechanisms.

Many theories and models have been used to explain the rationale mechanism for individual involvement in health behavior decisions about technology use and have showed great explanatory power in interpreting behaviors of healthcare professionals, such as the Theory of Planned Behavior (TPB), Innovation Diffusion Theory (IDT), Organizational Readiness for Change (ORC), and Technology-Organization-Environment framework (TOE). TPB was developed by Ajzen to understand the relationship between attitudes, subjective norms, perceived behavioral control, intentions, and behaviors.³ IDT summarizes five attributes of how an innovative product is introduced at the technological level, with compatibility and complexity frequently reported in studies of technology use.⁴ According to ORC, the changes of medical practitioners’ behavior should start from the organizational level, including atmosphere (eg, organizational culture) and practices (eg, systems and implementation activities).⁵ In addition to technical and organizational factors, TOE places particular emphasis on the effect of the external environment, such as industry pressure.⁶

To date, based on the above theories, a number of studies have identified and highlighted the importance of influencing factors influencing technology use from one or two of the five aspects: individual, technology, organizational atmosphere, organizational practice, and industry environment.^7–10 For instance, a study by Saigí-Rubió F remarked technology characteristics and environmental pressure as determinants of healthcare professionals’ intentions to use digital clinical consultations tools.⁷ Liu’s study focused on predictors that shape primary care physicians’ antibiotic prescribing behavior at the individual level.⁸ Besides, Randall CL and Knerr reported the significance of organizational factors to the successful implementation of certain health technologies.^9,10 And the association of these factors in different aspects has been sporadically documented in the previous literature. Specifically, several studies have revealed that certain technology characteristics influence physicians’ beliefs on cloud health information systems,¹¹ the introduction of electronic health records in medical centers,¹² and the industry environment of cloud computing.¹³ Beyond these associations, organizational practice has been found to be predicted by organizational atmosphere and industry environment, such as a study conducted in Canadian hospitals pointed out that culture served a role in program implementation¹⁴ and a cohort study confirmed that hospitals’ decisions to use surgical robots were related to regional competition.¹⁵ However, few have integrated these different perspectives and comprehensively explored the potential interactions between them. Additionally, most previous studies have focused on patients,^16–18 with less attention to healthcare providers (eg, physicians).

With regard to analytic strategies, consideration of predictors has recently expanded from individual-level factors to multilevel factors;¹⁹ in particular, structural factors of the organization or life context of the research subject are continuously being valued. The hospital is one of the most important arenas in the life of a physician. Physicians’ technology use may be influenced by structural characteristics among hospitals. Evidence demonstrates that the practice behavior of health care professionals is mediated to some extent by the context in which they work.²⁰ Failure to address interorganizational heterogeneity may cause biased results due to aggregation effects.²¹ To solve this issue while determining the relationship between the proposed factors, multilevel structural equation modeling (MSEM) is the best choice.²² Currently, MSEM has been used in the fields of public health policy^23–25 and adolescent health,^26,27 but rarely in health technology use.

Given that understanding of the mechanisms for the use of technology at multiple levels is still relatively limited, this study aims to examine the mechanisms linking the predictors and technology use in five aspects among physicians through the MSEM approach. Figure 1 displays the predictions we made in this study. To make the study more pertinent, we investigated the use of Des-gamma-Carboxy Prothrombin (DCP), a technology proven to be effective and suitable for early screening of liver cancer. The findings are promising for bridging the research gap about the mechanism of technology use among physicians.

Figure 1 The research model of the mechanism of physicians’ technology use.

Materials and Methods

Study Setting and Population

Since almost all health institutions at all levels are part of a set of regional medical consortiums (RMC, a model of integration of health services in China, where technical cooperation and assistance is one of its most prominent features²⁸), and most primary health institutions do not currently have the capacity to deploy DCP, this study was implemented in secondary and tertiary hospitals that have the capacity to deploy DCP. Physicians in charge of the direct use of DCP in the aforementioned hospitals were taken as the target population and were eligible to participate.

Sampling Strategy

From February to August 2019, a cross-sectional survey was conducted using a multistage random sampling. The concrete steps included: firstly, Fujian and Jiangxi provinces were selected from provinces with high and low incidence of liver cancer, respectively; Secondly, two RMCs were selected from each province, and hospitals with the ability to deploy DCP in the two selected RMCs in each province were included at a rate of 50%; Thirdly, physicians who have knowledge of DCP and work in liver disease-related departments (including hepatology, oncology, infection, gastroenterology, interventional medicine, radiotherapy, etc.) were invited to participate. It is expected to investigate 10~20 physicians per hospital and 5~8 hospitals per RMC. Totally, at least 200 physicians were approached, which meets the sample size requirement of at least five times the survey questions.²⁹

Study Variables

Dependent Variables

The dependent variable of this study is technology use, which derived from its related notions as follows. As described by Beyer and Estabrooks, there are three types of technology use: symbolic use (SU), conceptual use (CU) and instrumental use (IU).^30,31 SU indicates increased awareness through existing research findings; CU evinces general enlightenment through research findings; IU involves the use of research findings in a specific and direct manner.

Based on the above understanding, this study assessed SU by asking physicians about their initiatives and knowledge of DCP use. CU was captured by asking them about their actions in introducing DCP to patients, colleagues and department heads. IU was also asked regarding their use of DCP in clinical practice. More details on dependent variable are available in the Supplementary File. A six-point Likert scale was used for all items, with “never”, “rarely”, “less”, “general”, “more” and “a lot” recorded as 0, 1, 2, 3, 4 and 5, respectively. And the score for each type of technology use was measured by the mean score of all items under a certain technology use.

Physician Level Variables

Five aspects are included at the physician level. Value cognition (VC) is the perception and predisposition to DCP use. According to the TPB scale, this aspect is examined by three dimensions: attitudes, subjective norms and perceived behavioral control.³ Attitudes are defined as positive or negative evaluations of DCP use; subjective norms are perceived pressure to engage or not to engage in behavior from the influential people; perceived behavioral control reflects the physicians’ beliefs that DCP use is under his or her control.

The second aspect is technical assessment (TA). In the opinion of IDT, perceived ease of use and compatibility are the two most important factors in technology use.⁴ In this study, ease of use refers to the degree to which DCP can be easily used; compatibility refers to the consistency between the use of DCP in physicians and their previous experience, habits and existing practical processes.

ORC theory deems physicians’ perceptions of organizational factors are important prerequisites for changing their medical behaviors.⁵ Due to the different focal points, the organizational factors perceived by the physicians were divided into practice-related and atmosphere-related. Experienced organizational practice (EOP) refers to a series of practical actions that promote the DCP use, including mechanism establishment, staff allocation and post-implementation communication. It consists of two dimensions: perceived support mechanism and internal diffusion. As for perceived organizational atmosphere (POA), it is the preparedness and norms of the DCP use that are collectively shared by all members within the hospital, and it includes two dimensions of perceived organizational culture and technology sharing willingness.

Originating from the TOE framework,⁶ the last aspect of the perceived external environment (PEP) was developed. For hospitals, the external environment that affects their technology use is mainly manifested in the industry pressure. Such pressure may come from competition with other hospitals, from the preferences of business partners, or from compliance with industry standards.

All items within physician-level variables have the following response categories: (1) “Strongly disagree”, (2) “Disagree”, (3) “Neutral”, (4) “Agree”, and (5) “Strongly agree”. The score for each dimension was obtained by calculating the mean score of all the items in it.

Hospital Level Variables

Hospital-level variables refer to the structural characteristics of the hospital in the study, including hospital rank and hospital role. Hospital rank was classified into two groups: (1) “tertiary” and (2) “secondary”. Hospital role was defined according to the function of the hospital in the RMC. Four of the hospitals were categorized as “leading hospitals”, which are responsible for the overall organization of the RMC and coordinating the participation of other institutions in the various efforts. The other hospitals were classified as non-leading hospitals.

Statistical Analysis

After data checking, incomplete questionnaires were eliminated. Frequencies and percentages were used to present the participants’ demographic characteristics. Means, standard deviations and medians were used for demonstrating continuous characteristics of the variable scores.

Data collected for health services research regularly involve nested (or hierarchical) structures,³² such as physicians being nested within hospitals or patients being nested within wards. The characteristics of contextual factors may influence individual outcomes. Given that the data structure of this study was hierarchical, an MSEM was performed for outcome measures and mechanistic exploration, with physicians as level 1 and hospitals as level 2. The MSEM analysis consisted of three steps. Firstly, a null model (Model 1) was established without including any independent variables. Secondly, only physician-level variables were added in Model 2. Thirdly, both physician-level and hospital-level variables were contained in Model 3. We report the intraclass correlation (ICC) for each model to estimate the variance attributable to the hospital level. The ICC was considered high if it was greater than 0.20, moderate if 0.10 < ICC ≤ 0.20, and low if ≤0.10.^33,34 The following criteria assessed the model fit: the RMSEA (root mean square error of approximation) and SRMR (standardized root mean square residual) < 0.08, CFI (comparative fit index) and TLI (Tucker-Lewis index) > 0.90.³⁵ The AIC (Akaike information criterion) and BIC (Bayesian information criterion) were decreased compared to the previous models. Data analyses were conducted using Mplus Version 8.3. All tests were two-tailed with α = 0.05.

Results

Sample Demographics

The 229 participants were aged from 24 to 56 years old (Mean = 36.79; SD = 7.27), 66.4% (n = 152) were male, 91.7% (n = 210) had a bachelor’s degree or above, 79.9% (n = 183) had no administration position, 29.7% (n = 68) had 5~10 years in practice, 60.7% (n = 139) worked in tertiary hospitals, and 59.0% (n = 135) were affiliated with non-leading hospitals. The characteristics of the participants are described in Table 1.

Table 1 Characteristics of the Sample

Measurement of Outcomes

The scores of technology use were relatively low. SU, CU and IU scored 2.31 ± 1.21, 1.51 ± 1.61 and 1.61 ± 1.61, respectively. The proportions of scores above their mean scores was 52.40%, 41.00% and 41.50% for SU, CU and IU, respectively. The scores of the dimension were closer to the ones for the dimensions within the same aspect. Overall, the dimension score was in descending order: VC > TA > POA > PEP > EOP. Table 2 provides the outcomes of all measures.

Table 2 Measurement Scores of the Participants

Influencing Factors of Technology Use

The results from the MSEM analyses are presented in Table 3. Model 1 showed substantial variation at the hospital level, as the ICC values for SU, CU and IU were 0.111, 0.233 and 0.289, respectively. These suggested that the DCP use of physicians was clustered within the same hospital; the two-level hierarchical data structure was suitable for multilevel analysis. When adjusting for physician-level factors, VC and EOP had a positive influence on physicians’ DCP use (P < 0.05), POA had an inverse relationship with physicians’ DCP use (P < 0.001), and the effect of TA and PEP was insignificant. The addition of hospital-level factors did not alter test results. Model fit statistics indicated a reasonable fit of the final model: RMSEA = 0.077 < 0.08, CFI = 0.943 > 0.90, TLI = 0.922 > 0.90, SRMR for within = 0.046 < 0.08, SRMR for between = 0.037 < 0.08, and AIC and BIC were decreased by 91.75 and 64.28, respectively.

Table 3 Multilevel Structural Equation Modeling for the Technology Use of Physicians

Mediation Effects of Within-Level Factors

As further indicated by our MSEM analysis, VC had no indirect effect on physicians’ DCP use; its total effect was 0.447 (P < 0.01). The relationship between TA and physicians’ DCP use was mainly mediated by VC (Indirect effect = 0.311, P < 0.01). There was a significant positive relationship between TA and PEP. Besides, TA indirectly influenced EOP through PEP (Indirect effect = 0.309, P < 0.01). EOP was positively associated with physicians’ DCP use (B = 0.203, P < 0.05), but its impact on VC was not significant (B = −0.042, P > 0.05). POA has a negative direct effect on technology use (B = −0.237, P < 0.01). In addition, VC and EOP moderated the relation between POA and physicians’ DCP use (P < 0.05). POA has a significant influence on VC (B = 0.148, P < 0.01) and EOP (B = 0.410, P < 0.001). EOP was significantly influenced by PEP (B = 0.392, P < 0.001), and the relationship between PEP and physicians’ DCP use was mediated by EOP (B = 0.079, P < 0.05). Table 4 shows the details of the direct, mediated, and total effects estimated at the physician level.

Table 4 Direct, Indirect and Total Effect of Predictors on Technology Use

Discussion

Sufficient understanding of the mechanism contributes to a large extent to the effective implementation of technologies. To increase the corresponding knowledge, we developed the research model by integrating TPB, IDT, ORC and TOE using DCP as an example. The findings determined the factors associated with physicians’ technology use behavior from five aspects: VC, TA, EOP, POA and PEP. This study may provide insight into orientation strategies for advancing technology use in healthcare professionals.

The scores of DCP use in this study were low and were significantly lower than their attitudinal levels, suggested that although physicians hold positive attitudes and perceptions towards DCP use, problems such as lack of relevant knowledge and limited practical implementation still exist. This result was consistent with previous studies.^36–38 For instance, in Xu Y’s study, health care professionals had positive attitudes towards graduated compression stockings, while the application rate was low because a considerable part of them did not understand the essential knowledge, such as clinical implications and contraindications.³⁷

In this study, the significant direct impact of VC on DCP use by physicians revealed the combined function of attitudes, subjective norms and perceived behavioral control, which have been reported as important predictors of technology use in prior research.^39–41 Physicians will be more apt to adopt and use a technology in their clinical practice if they rate it highly and deem it can be implemented successfully. In addition, the direct influence of EOP on physicians’ DCP use was confirmed by both support mechanism and practical implementation. When hospitals provide financial, personnel, and information guarantees for technology use,⁴² as well as placing emphasis on pre-training, mid-term communication and post-term feedback,^43,44 it will facilitate effective interventions in physicians’ technology decision-making behavior.

Contrary to our expectations, POA had a negative effect on technology use. The plausible reason is that an outcome-oriented organizational culture has a significant negative impact on the behaviors of its members.⁴⁵ Hospital supervisors attach more importance to the achievement of performance goals than to the actual developmental needs of physicians. As Quinn once pointed out, overemphasis on a certain culture may divert members’ behavior from organizational objectives.⁴⁶ Also, POA indirectly affected physicians’ technology use through VC and EOP. On the one hand, most people adjust their behavior to align with the organization in general cases.⁴⁷ The “invisible norms” formed by the hospital’s long-term readiness to adopt new technologies can unknowingly affect physicians’ perceptions and even actual technology use behavior.⁴⁸ On the other hand, the implementation of technology use promotion often occurs when hospitals advocate self-innovation.

Notably, although no direct effects of TA and PEP on technology use were found, their mediation effects were observed through VC and EOP, respectively. The insignificance of TA reflected the limited influence of perceived ease of use and compatibility on physicians’ DCP use. A possible explanation was that such technical characteristics often have no significant effect in the early stages of technology use,⁴⁹ which was in line with previous findings.^50,51 To some extent, the perceived industry environment determines the macro-levels such as market orientation, and its influence on individual physicians needs to be realized through their organizations as “executive agencies”.

For nested data processing, MSEM is an effective approach to avoid clustered bias.²² Although the results showed the role of the hospitals in the RMC was insignificant (p-value of 0.057, very close to the significance level), their influence on physicians’ technology use cannot be completely ignored, which can be supported by the ICC estimates of the null model and final model. Obviously, there was a two-level structure in the study. That is, physicians’ technology use was aggregated across hospitals with different roles. Thus, the role of the hospital did contribute to the usage behavior of the physicians. As reported in Waddell JP’s study, some advanced medical devices and tests are only performed by specifically trained practitioners in certain hospitals.⁵² In China, the leading hospitals for the RMCs are generally local or inter-regional medical treatment centers with a strong medical services capacity,²⁹ where advanced technologies are first deployed. Logically and practically, DCP is first applied in the leading hospitals of RMCs since it is an innovative health technology. Physicians in leading hospitals have greater access to DCP and other new technologies, both physically available and as a result of patient demand.

Based on the above understandings, several strategies can be highlighted for improving the use of health technologies among health professionals. On the one hand, for the authorities, while urging leading hospitals to assume responsibility for technical support within the RMC, external incentives such as subsidies need to provided on an ongoing basis to enable them to sustain their role. On the other hand, hospital managers can advance the promotion of technology use by providing organizational management support. Firstly, provide physicians with high-quality and adequate training about technology use.⁵³ Secondly, mobilize authoritative physicians and managers to gradually adjust other physicians’ compliance with technology use.⁵⁴ Thirdly, cultivate organizational readiness for technology use through long-term implementation of bulletin boards and typical event learning. Finally, designate staff push timely industry information so that physicians can make more accurate technology decisions.⁵⁵

This study was strengthened by some features. Firstly, the integration of the TPB, IDT, OCR and TOE systematically expands the interpretations of the research basis from five aspects and also benefits a comprehensive identification of the influencing mechanism of technology use. Secondly, the subjects of this study were medical service providers (physicians, specifically), who have rarely been investigated in previous studies of technology use. Thirdly, the application of MSEM allows simultaneous analysis of physician-level and hospital-level factors, which makes the estimation of the model more accurate and comprehensive. Nevertheless, the study also has some limitations. First, the sample size was small due to funding constraints, and the accuracy and the generalizability of the findings could be improved by enrolling more participants in future research. Besides, the self-reported data might lead to social desirability bias that participants tend to give more positive responses. Finally, this study only uses DCP as an example, and the predictive power of the model needs to be tested by focusing on other technologies.

Conclusion

This study enriches the knowledge about physicians’ technology use and its associated influencing mechanism by indicating the combined impact of VC, TA, EOP, POA and PEP on technology use among physicians. The findings highlight the importance of taking some of the following pertinent measures. It is recommended that policy makers fully elaborate the driving effect of leading hospitals in RMCs on other hospitals. Additionally, several tailored interventions allow hospital managers to look at different ways to promote technology use, such as mobilizing core members, cultivating organizational readiness, implementing workable practice and providing relevant technology information.