Back to Journals » International Journal of General Medicine » Volume 14

Swedish Primary Care Physicians’ Intentions to Use Telemedicine: A Survey Using a New Questionnaire – Physician Attitudes and Intentions to Use Telemedicine (PAIT)

Authors Pikkemaat M, Thulesius H, Milos Nymberg V 

Received 10 May 2021

Accepted for publication 14 June 2021

Published 15 July 2021 Volume 2021:14 Pages 3445—3455

DOI https://doi.org/10.2147/IJGM.S319497

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3

Editor who approved publication: Dr Scott Fraser



Miriam Pikkemaat,1,2 Hans Thulesius,1,3 Veronica Milos Nymberg1,2

1Department of Clinical Sciences, Lund University, Malmö, Sweden; 2CPF, Centre for Primary Healthcare Research, Malmö, Sweden; 3Linnaeus University, Kalmar, Sweden

Correspondence: Veronica Milos Nymberg
Clinical Research Centre (CRC), Skåne University Hospital, Building 28, Floor 11, Jan Waldenströms Street 35, Malmö, 205 02, Sweden
Tel +46-767700240
Email [email protected]

Background: Research on intentions to use telemedicine in primary care is sparse. This survey study explored primary care physicians’ intentions to use telemedicine by using a newly developed questionnaire: Physician Attitudes and Intentions to use Telemedicine.
Methods: An anonymous web-survey with questions focusing on theory-based predictors of behavioral intentions such as Attitudes, Subjective norms and Perceived behavioral control was designed, validated, and sent to all primary care physicians at 160 primary health care centers in southern Sweden from May to August 2019. The questionnaire had 29 subject items (including 49 multiple-choice sub-items). Main outcome measures were intentions to use three domains of telemedicine and correlation between theory-based predictors and behavioral intentions for using telemedicine.
Results: The survey was validated by an expert group, amended, and then tested and retested. A majority of the 198 physicians who returned the web-surveys reported that they did not use e-mails (68%), nor video consultations (78%), chat (81%), or text messages (86%) in their everyday patient work. Yet, most physicians described a positive intention to use telemedicine in patient care for all three studied domains with Attitudes and Perceived behavioral control being significant predictors (p< 0.01) for Intentions to use digital contacts (R2 = 0.54), chronic disease monitoring with digital tools (R2 = 0.47) and artificial intelligence (R2 = 0.54). A structural validation of a preliminary instrument – Physician Attitudes and Intention to use Telemedicine (PAIT) – containing 28 sub-items was done by exploratory factor analysis with acceptable explanatory, reliability and sampling adequacy measures. Five factors emerged with Eigenvalues between 1.6 and 11.1 explaining 72% of the variance. Total Cronbach’s alpha was 0.91 and Kaiser-Meyer-Olkirk 0.79.
Conclusion: Before the covid-19 pandemic, Swedish primary care physicians reported a low use yet high behavioral intention to use telemedicine in a study where we developed the preliminary instrument Physician Attitudes and Intention to use Telemedicine. Perceived behavioral control had the largest predictive value of behavioral intention to use telemedicine. Thus, interventions aiming to increase the use of digital tools in primary care should possibly focus on empowering physicians’ self-efficacy towards using them.

Keywords: telemedicine, survey, primary care, theory of planned behavior

Introduction

The paradigm shift we are currently experiencing with fast development of telemedicine is remodeling health care worldwide.1 Telemedicine is a broad concept including digital remote access to health care (e-mail and video consultations in telemedicine), monitoring chronic diseases with digital tools (registration of vital parameters using mobile applications) and artificial intelligence (diagnostic algorithms, triage, support for treatment guidelines). In Sweden, internet-based health care services have been gradually introduced during the two last decades. Digital telemedicine is provided both by private, for-profit, and direct-to consumer firms, and by public health care centers using digital platforms alongside their regular in-office services.1 New digital environments are pressuring both patients and caregivers, craving continuous learning and adjusting to innovative e-tools.2,3 Meanwhile, expected benefits as increased availability of health care, potential cost savings1 but also increasing costs due to private telemedicine providers using the national tax-reimbursement system have motivated Swedish policymakers to increase e-health use within public health care.4 The use of digital tools for monitoring chronic diseases has showed evidence of improved health outcomes.5,6 In a systematic review, patients reported that quality of life and satisfaction with telemedicine was comparable to traditional care7 and were “double-empowered” by both health care availability and internet-based shared information and thus shared decision-making,8 yet older patients with chronic diseases may have different needs and preferences.9

Meanwhile, there is little research on physician’s attitudes and expectations of telemedicine. Ethical aspects with concerns about increased workload or the challenge of managing complex patients are described,10 as well as doubts about the safety of recommendations using digital tools for chronic disease management.11 Several theories have been used to study individuals’ adoption of new technology for example, the Technology Acceptance Model (TAM) (Davis, 1989), Theory of Diffusion of Innovations (Rogers, 1995), The Theory of Planned Behavior (Ajzen 1991),12 The Unified Theory of Acceptance and Use of Technology – UTAUT (Venkatesh, Morris, Davis and Davis 2003). Both organizational and human environments, as well as individual factors such as attitudes or self-efficacy have been identified as key determinants for successful adoption and use of technology among both patients and health care staff, according to a recent systematic review.13 Ajzen introduced The Theory of Planned behavior, stating that the strength of an individual’s beliefs determines the attitudes towards performing a certain behavior.12,14 A Subjective norm is a perceived social pressure to perform or not to perform a behavior defined by an individual’s perceptions of how people important to them (family, friends, co-workers) consider that behavior as good or bad.12 Perceived behavioral control is described as an individual’s trust in one’s own capability or self-efficacy to execute a behavior.12 Individuals’ Attitudes, Subjective norms, and Perceived behavioral control have been described to be strong predictors for behavioral intention, which in turn is correlated to performing a behavior (Figure 1).12 Therefore, the Theory of Planned Behavior might be useful in predicting behavioral intention and, consequently, behavior. Understanding the predictive utility of the Theory of Planned behavior not only enables the theory to be tested and behavior to be understood, but specific determinants of behavior can be identified. This can have important implications for interventions aimed to influence adoption of telemedicine, as interventions should modify or maintain underlying beliefs and norms, and interventions aiming to influence behavioral intention are expected to indeed influence behaviors.15

Figure 1 The theory of planned behavior, (Ajzen, 1991).

Research using the Theory of Planned Behavior has shown various attitudes and subjective norms towards e-health amongst physicians.16 There is evidence that successful implementation of new digital environments should comprise deeper understanding and involvement of the potential users of the tools.17,18 Therefore, the Theory of Planned Behavior is a promising theory to identify the latent constructs underlying the choices of adoption/non-adoption of a new technology.

The aim of this study was to explore the experiences and behavioral intentions of Swedish primary care physicians towards telemedicine in primary care. However, we found no suitable assessment tool for this task and therefore we set out to design and validate a questionnaire with a focus on behavioral predictors of using telemedicine derived from the Theory of Planned Behavior.

Materials and Methods

Design and Participants

Primary care in two south Swedish regions was at the time of the study provided by 90 public and 70 private primary health care centers (PHCCs) with an approximate total number of 820 physicians. An anonymous web-based survey was sent to all primary care physicians working at these 160 PHCCs. The survey ran from May to August 2019.

Questionnaire Design and Psychometric Testing

We created a survey using a manual based on the Theory of Planned Behavior.19 The survey had general questions about experiences of telemedicine, and questions focused on use of digital contacts, chronic disease monitoring with digital tools and artificial intelligence (see Supplementary file). Examples of questionnaire items that are targeting predictors of behavior derived from the Theory of Planned Behavior are presented in Table 1. Different items in the questionnaire assessed these variables on a 7-point Likert scale20 including the alternative “I don’t know”.

Table 1 Example of Questionnaire Items Derived from the Theoretical Constructs of the Theory of Planned Behavior (Ajzen 1991). All Items Had the Response Alternatives 1–7 and “I Don’t Know”

We validated and tested the survey questionnaire advised by Churchill's paradigm21 and COSMIN guidelines.22 We then followed the checklist for reporting results of internet e-surveys (CHERRIES)23 and in CHERRIES (Supplementary file) we report details of the questionnaire development by pilot-testing among 24 physician respondents. The survey was tested for construct validity and reliability. Internal validity was psychometrically tested by a group of five physicians with proficiency in telemedicine and digital tools, evaluating if the questionnaire was capturing the research question. They made written notes on paper questionnaires regarding leading or confusing questions. The comments of the expert group lead to minor changes in wording and formatting. Then, we performed a pilot study at two PHCCs by sending the questionnaire to 24 physicians. Internal consistency for items with several sub-items was tested with Cronbach’s α (CA). For items with low CA subitems were removed to improve CA. After modification, the remaining items had CA between 0.6 and 0.95.

Temporal stability of the questionnaire was assessed by resending the questionnaire to the same 24 respondents after two weeks. The physician respondents were asked to fill in their initials in “free answer” box to identify unique answers. Twelve physicians returned the questionnaire. The Pearson correlation coefficient was 0.63. This led us to change the structure of some questions to improve the final questionnaire.

After validating the questionnaire, we invited primary care physicians at 160 PHCCs to participate in the anonymous survey by e-mailing an electronic link asking the PHCC administrators to forward the link to the physicians using the electronic database manager REDCap.24 Data collection is described in detail in the supplementary file.

Outcome Measures

Primary outcome measures were scores for intentions to use telemedicine as predictors of behavior (Attitudes, Subjective norms, Perceived behavioral control, and Behavioral intention).

Secondary outcome measures were experiences of telemedicine and correlation between Behavioral intention and Attitudes, Subjective norms and Perceived behavioral control adjusted for years of experience, gender, age, and workplace. A psychometric outcome measure was the result of exploratory factor analysis of the questionnaire with the goal to develop a preliminary instrument for assessing attitudes and intention to use telemedicine.

Data Analysis

We used IBM SPSS statistics version 25 (IBM). Predictor variables were measured directly by asking the respondents about their attitudes, subjective norms, and perceived behavioral control. For each of the primary outcome variables we calculated the mean of the item scores to give an overall score. If more than half of the questions for each item were missing, the overall score was defined as missing. If the respondent chose the alternative “I don’t know”, the answer was considered missing. Free-text replies were not analyzed in this study.

Secondary outcome variables were described with mean and percentages. The correlation between predictors and intentions was assessed with multiple linear regression models.

Ethical Considerations

The Swedish Ethical Review Authority gave an advisory statement confirming that according to the Swedish Ethical Review Act (SFS 2003:460) ethical vetting was not required for this type of study using anonymous data and not asking participants about their own health or other sensitive topics. All participants that returned questionnaires provided an informed consent to participate in the study.

Results

A total number of 198 returned surveys from approximately 820 potential respondents yielded a crude response rate of 24%. The characteristics of the physician respondents are summarized in Table 2.

Table 2 Characteristics of the Physicians, Total N= 198

While the physicians reported high use of telephone and letters to communicate with patients 134, 154, 161 and 171 physicians reported no use of e-mail (68%), video consultations (78%), chat (81%), or SMS text messages (86%) in their everyday patient work (Table 3). Yet, most physicians described positive intentions to use telemedicine in patient care for all three studied domains: digital contacts, monitoring chronic disease and artificial intelligence. High mean scores were also attained for attitudes to have digital contacts with patients, using digital tools for chronic disease monitoring and use of artificial intelligence (Table 3).

Table 3 Descriptive Analysis of the Overall Scores for the Behaviour Predictors Emerged from the Theoretical Construct. Overall Score Was the Mean Value for Scores from Items Targeting the Same Predictor

Attitudes towards video consultations were assessed separately as less knowledge and use of video consultations was expected. The measurement of Subjective norms indicated high scores only for digital contacts. Perceived behavioral control and Behavioral intention scores were high in all measurements. There was a larger number of missing responses for questionnaire items that focused on the use of digital tools for monitoring chronic disease and artificial intelligence than for using digital contacts (Table 3).

We used a formative model to construct a preliminary instrument of attitudes, subjective norms and intentions to use telemedicine – Physician Attitudes and Intentions to use Telemedicine (PAIT) (Table 4). We excluded items 1 and 2 that reported general questions about experiences of telemedicine since they lacked a theoretical basis. We also excluded items 22–29 about artificial intelligence because of a too high number of missing responses. We did exploratory factor analysis with varimax rotation. Five-factor components emerged with Eigenvalues of 1.6–11 cumulatively explaining 72% of the variance and a total Cronbach’s alpha of 0.91; Kaiser-Meyer-Olkin value was 0.79.

Table 4 Exploratory Factor Analysis with Varimax Rotation for Physician Ratings of the Physician Attitudes and Intention to Telemedicine (PAIT) Questionnaire to 198 Swedish Primary Care Physicians. Extraction Sums of Squared Factor Loadings in % of Total Variance

Factor I is called: Attitudes to Chronic Disease Monitoring with Digital Tools. Factor II: Attitudes to Using Digital Tools. Factor III: Behavioral Intention to Using Digital Tools. Factor IV: Subjective Norms to Using Digital Tools. Factor V: Attitudes to Using Video.

We performed multiple regression analyses to examine the predictive value of the theoretical constructs: Attitudes, Subjective norms, and Perceived behavioral control (Table 5). Attitudes and Perceived behavioral control were found to be significant predictors (p<0.01) for intentions to use digital contacts (R2 = 0.54), chronic disease monitoring with digital tools (R2 = 0.47) and artificial intelligence (R2 = 0.54).

Table 5 Regression Analysis Showing Predictors for Behavioural Intentions to Use Digital Contacts, Digital Tools for Monitoring or AI. Model 1 (Crude Model)

Physicians reporting positive attitudes and high perceived behavioral control towards digital tools also had more favorable intentions to use them for monitoring chronic diseases. Behavioral intention to use artificial intelligence were also predicted by positive Attitudes and Perceived behavioral control but not by Subjective norms (Table 5). P-values remained significant after adjustment for age, gender, years of experience and type of practice (private or public) (Supplementary Tables 1–3).

Discussion

Main Findings

In this study, we designed a questionnaire to assess both the actual use of telemedicine in primary care and the intention to use telemedicine based on Attitudes, Subjective norms, and Perceived behavioral control resulting in the preliminary instrument Physician Attitudes and Intention to use Telemedicine (PAIT). Despite a low reported actual use of telemedicine among the Swedish primary care physicians, they described a positive intention to use telemedicine in patient care for all three studied domains: digital contacts, chronic disease monitoring and artificial intelligence. Attitudes, Subjective norms, and Perceived behavioral control were strong predictors for intentions to use digital contacts. Intentions to use digital tools for chronic disease monitoring and artificial intelligence were only associated with positive Attitudes and Perceived behavioral control but not with Subjective norms. Perceived behavioral control had the largest beta weight in all prediction models. This suggests that maintaining and reinforcing physicians’ self-efficacy towards use of telemedicine should have strong positive implications on adoption of such tools in patient care. The physicians reported lower Subjective norms to use digital tools and artificial intelligence which we interpret as lack of social pressure. A reasonable explanation could be that despite continuous development of different digital tools for disease monitoring as well as artificial intelligence, the use of these tools in primary care was at an early stage at the time of the study [4].

Strengths and Limitations

Our study is to our knowledge the first to assess the intention to use digital contacts and digital tools among medical professionals in the Nordic countries using an approach based on the Theory of Planned Behavior, and this is a strength of our study.

A major limitation of our study is the low response rate of 24%; possibly caused by the survey being forwarded by administrators and not sent to the physicians directly. Yet, the response rate aligns with results from recent published web-based surveys25,26 that have shown comparable or even better reliability than traditional paper surveys.25 However, the response rate indicates that selection bias cannot be ruled out. Another weakness of this study is that we did not use cookies or log-files to minimize the risk that one single responder could fill the survey more than once, a common problem in web-based surveys.23 Another limitation of the study is that the survey was based on self-report data, as recall bias may have influenced past behavior responses.

The predictive validity of model constructs is an original contribution from our results and by factor analysis emerged the preliminary assessment instrument PAIT with satisfactory explanatory values regarding variance and acceptable sampling adequacy according to the Kaiser-Meyer-Olkirk measure. Using an existing questionnaire might have saved us time and resources and increased generalizability. However, we did not find any existing questionnaire aimed to measure the constructs of our research interest. Therefore, we used the Theory of Planned Behavior manual for item construction and pilot tested and subsequently validated a questionnaire showing acceptable internal consistency (>0.6 in CA for all items) and construct validity. Further development of the questionnaire with removal of sub-items from items with CA>0.9 and testing the robustness of the preliminary instrument PAIT is planned for future studies to avoid redundancy and confirm (or rather modify and eventually reduce) the five PAIT factors.

The study did not assess future behavior (only past behavior), so it is not known whether reported behavioral intention predicted behavior. Although some communications (ie, video consultations, text messages) were not previously used frequently but intention to use was high, it would have been useful to know whether this was undertaken. The survey was sent prior to the covid-19 pandemic that started worldwide in the spring of 2020. Since then, health care, as well as other societal sectors, have adapted and adjusted towards digital platforms of communication.27 This process has undeniably influenced primary care staffs’ attitudes and perceptions of e-health and telemedicine. This study thus presents primary care physicians’ intentions to use e-health prior to the transformation caused by the covid-19 pandemic. Today, the survey results would probably be different, and the results of the present study will be interesting as a baseline to compare with the current situation.

Our findings have potential managerial implications. Perceived behavioral control had the largest predictive value of behavioral intention to use telemedicine, suggesting that interventions aiming to increase adoption of such tools in primary care should focus on empowering physicians’ self-efficacy towards use of digital tools. Future studies should explore if primary care staffs’ intentions to use telemedicine as well as experiences of digital communication tools have altered because of increased adoption of telemedicine during and after the pandemic.

Comparison with Other Studies

It has been advocated that the next generation of research on diffusion of health service innovations should be theory-driven, enabling the researchers to understand the mechanisms behind the success or failure of intervention programs.28 In the context of the rapidly spreading health care digitalization challenging the traditional health care norms, successful implementation of new interventions depends on how health care leaders set a culture and climate supporting and enabling change.28 Solutions to adoption barriers of telemedicine are unique to each country’s health care system and medical staff’s demands.28 Our findings suggest that Swedish primary care physicians already have a strong intention to use telemedicine, and the intention is predicted by positive attitudes and perceived behavioral control. Social norms were found to be a weak predictor of behavioral intentions in our study, and these findings are similar to results from a meta-analysis of previous research on the predictive value of constructs derived from the Theory of Planned Behavior.29 Previous research has found several factors that can restrain or facilitate health care staff to use telehealth in the clinical work, at the individual, organizational and contextual level.30 The results in our study focused on identifying theory-based predictors of behavior on an individual level, may be useful for stakeholders planning educational efforts and future implementation of digital technology and telemedicine in health care.

Conclusion

The questionnaire showed that Swedish primary care physicians reported a high behavioral intention to use digital contacts, digital tools, and AI despite low reported use of telemedicine prior to the covid-19 pandemic. Perceived behavioral control had the largest predictive value of behavioral intention to use telemedicine, suggesting that interventions aiming to increase adoption of such tools in primary care should focus on empowering physicians’ self-efficacy towards use of digital tools. Future studies should explore if increased adoption of telemedicine due to the covid-19 pandemic have altered primary care staffs’ intention to use telemedicine, and we look forward to testing of our preliminary instrument PAIT (Physician Attitudes and Intention to use Telemedicine) in times ahead.

Author Contributions

All authors contributed to data analysis, drafting, and revising the article, gave final approval of the version to be published, and agree to be accountable for all aspects of the work.

Funding

The authors received no financial support for the research or authorship of this article.

Disclosure

The authors declared no potential conflicts of interest.

References

1. Ekman B. Cost analysis of a digital health care model in Sweden. Pharmacoecon Open. 2018;2(3):347–354. doi:10.1007/s41669-017-0059-7

2. Bortom IT. om hälsa i en digital tid: institutet för framtidsstudier. Swedish. 2016. Available from: https://www.iffs.se/publikationer/if-rapporter/bortom-it-om-halsa-i-en-digital-tid/. Accessed June 2, 2021.

3. Nymberg VM, Bolmsjo BB, Wolff M, Calling S, Gerward S, Sandberg M. ‘Having to learn this so late in our lives … ’ Swedish elderly patients’ beliefs, experiences, attitudes and expectations of e-health in primary health care. Scand J Prim Health Care. 2019;37(1):41–52. doi:10.1080/02813432.2019.1570612

4. Landstingen tar upp kampen om digital vård. Swedish. 2018. Available from: http://www.lakartidningen.se/Aktuellt/Nyheter/2018/02/Landstingen-tar-upp-kampen-om-digital-vard/. Accessed June 2, 2021.

5. Pare G, Moqadem K, Pineau G, St-Hilaire C. Clinical effects of home telemonitoring in the context of diabetes, asthma, heart failure and hypertension: a systematic review. J Med Internet Res. 2010;12(2):e21. doi:10.2196/jmir.1357

6. Xiang R, Li L, Liu SX. Meta-analysis and meta-regression of telehealth programmes for patients with chronic heart failure. J Telemed Telecare. 2013;19(5):249–259. doi:10.1177/1357633X13495490

7. Frennert SA, Forsberg A, Östlund B. Elderly people’s perceptions of a telehealthcare system: relative advantage, compatibility, complexity and observability. J Technol Hum Serv. 2013;31(3):218–237. doi:10.1080/15228835.2013.814557

8. De Rosis S, Barsanti S. Patient satisfaction, e-health and the evolution of the patient-general practitioner relationship: evidence from an Italian survey. Health Policy (New York). 2016;120(11):1279–1292. doi:10.1016/j.healthpol.2016.09.012

9. Ware P, Bartlett SJ, Paré G, et al. Using eHealth technologies: interests, preferences, and concerns of older adults. Interact J Med Res. 2017;6(1):e3. doi:10.2196/ijmr.4447

10. Macdonald GG, Townsend AF, Adam P, et al. eHealth technologies, multimorbidity, and the office visit: qualitative Interview Study on the perspectives of physicians and nurses. J Med Internet Res. 2018;20(1):e31. doi:10.2196/jmir.8983

11. Boyle L, Grainger R, Hall RM, Krebs JD. Use of and beliefs about mobile phone apps for diabetes self-management: surveys of people in a hospital diabetes clinic and diabetes health professionals in New Zealand. JMIR mhealth uhealth. 2017;5(6):e85. doi:10.2196/mhealth.7263

12. Ajzen I. The theory of planned behavior. Organ Behav Hum Decis Process. 1991;50(2):179–211. doi:10.1016/0749-5978(91)90020-T

13. Gagnon MP, Ngangue P, Payne-Gagnon J, Desmartis M. m-Health adoption by healthcare professionals: a systematic review. J Am Med Inform Assoc. 2016;23(1):212–220. doi:10.1093/jamia/ocv052

14. Elliott MA, Armitage CJ, Baughan CJ. Drivers’ compliance with speed limits: an application of the theory of planned behavior. J Appl Psychol. 2003;88(5):964–972. doi:10.1037/0021-9010.88.5.964

15. Milos V, Jakobsson U, Westerlund T, Melander E, Molstad S, Midlov P. Theory-based interventions to reduce prescription of antibiotics--a randomized controlled trial in Sweden. Fam Pract. 2013;30(6):634–640. doi:10.1093/fampra/cmt043

16. Kuo KM, Talley PC, Lee CM, Yen YC. The influence of telemedicine experience on physicians’ perceptions regarding adoption. Telemed J E Health. 2015;21(5):388–394. doi:10.1089/tmj.2014.0091

17. Davis MM, Freeman M, Kaye J, Vuckovic N, Buckley DI. A systematic review of clinician and staff views on the acceptability of incorporating remote monitoring technology into primary care. Telemed J E Health. 2014;20(5):428–438. doi:10.1089/tmj.2013.0166

18. Hjelmager DM, Vinther LD, Poulsen SH, Petersen LS, Jensen MB, Riis A. Requirements for implementing online information material for patients with low back pain in general practice: an interview study. Scand J Prim Health Care. 2019;37(1):60–68. doi:10.1080/02813432.2019.1569413

19. Francis J. Constructing questionnaires based on the theory of planned behaviour: a manual for health services researchers. 2004. https://openaccess.city.ac.uk/id/eprint/1735/;. Accessed July 7, 2021.

20. Sullivan GM, Artino AR Jr. Analyzing and interpreting data from likert-type scales. J Grad Med Educ. 2013;5(4):541–542. doi:10.4300/JGME-5-4-18

21. Churchill GA. A paradigm for developing better measures of marketing constructs. J Mark Res. 1979;16(1):64–73. doi:10.1177/002224377901600110

22. Mokkink LB, Prinsen CAC, Patrick DL, et al. COSMIN Study Design Checklist for Patient-Reported Outcome Measurement Instruments. Amsterdam, The Netherlands: COSMIN; 2019. [updated July, 2019; cited September 23, 2019]. Available from: https://www.cosmin.nl/wp-content/uploads/COSMIN-study-designing-checklist_final.pdf#. Accessed July 7, 2021.

23. Eysenbach G. Improving the quality of web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res. 2004;6(3):e34. doi:10.2196/jmir.6.3.e34

24. Harvey LA. REDCap: web-based software for all types of data storage and collection. Spinal Cord. 2018;56(7):625. doi:10.1038/s41393-018-0169-9

25. Ritter P, Lorig K, Laurent D, Matthews K. Internet versus mailed questionnaires: a randomized comparison. J Med Internet Res. 2004;6(3):e29. doi:10.2196/jmir.6.3.e29

26. Leece P, Bhandari M, Sprague S, et al. Internet versus mailed questionnaires: a controlled comparison (2). J Med Internet Res. 2004;6(4):e39. doi:10.2196/jmir.6.4.e39

27. Matlock DD, Fukunaga MI, Tan A, et al. Enhancing success of medicare’s shared decision making mandates using implementation science: examples applying the Pragmatic Robust Implementation and Sustainability Model (PRISM). MDM Policy Pract. 2020;5(2):2381468320963070.

28. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. doi:10.1111/j.0887-378X.2004.00325.x

29. Armitage CJ, Conner M. Efficacy of the theory of planned behaviour: a meta-analytic review. Br J Soc Psychol. 2001;40(Pt 4):471–499. doi:10.1348/014466601164939

30. Byambasuren O, Beller E, Glasziou P. Current knowledge and adoption of mobile health apps among Australian general practitioners: survey Study. JMIR mHealth uhealth. 2019;7(6):e13199. doi:10.2196/13199

Creative Commons License © 2021 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.