Back to Journals » Patient Preference and Adherence » Volume 9

Patient perspectives of telemedicine quality

Authors LeRouge C, Garfield M, Hevner A

Received 10 May 2014

Accepted for publication 2 August 2014

Published 24 December 2014 Volume 2015:9 Pages 25—40

DOI https://doi.org/10.2147/PPA.S67506

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 3

Editor who approved publication: Dr Johnny Chen



Cynthia M LeRouge,1 Monica J Garfield,2 Alan R Hevner3

1Saint Louis University, St Louis, MO, 2Bentley University, Waltham, MA, 3University of South Florida, Tampa, FL, USA

Background: The purpose of this study was to explore the quality attributes required for effective telemedicine encounters from the perspective of the patient.
Methods: We used a multi-method (direct observation, focus groups, survey) field study to collect data from patients who had experienced telemedicine encounters. Multi-perspectives (researcher and provider) were used to interpret a rich set of data from both a research and practice perspective.
Results: The result of this field study is a taxonomy of quality attributes for telemedicine service encounters that prioritizes the attributes from the patient perspective. We identify opportunities to control the level of quality for each attribute (ie, who is responsible for control of each attribute and when control can be exerted in relation to the encounter process). This analysis reveals that many quality attributes are in the hands of various stakeholders, and all attributes can be addressed proactively to some degree before the encounter begins.
Conclusion: Identification of the quality attributes important to a telemedicine encounter from a patient perspective enables one to better design telemedicine encounters. This preliminary work not only identifies such attributes, but also ascertains who is best able to address quality issues prior to an encounter. For practitioners, explicit representation of the quality attributes of technology-based systems and processes and insight on controlling key attributes are essential to implementation, utilization, management, and common understanding.

Keywords: patient perspective, technology service encounters, health care operations, telemedicine, quality, field study, mixed methodology

Introduction

Telemedicine involves the use of modern information technology, especially two-way interactive audio/video communications, computers, and telemetry, to deliver health services to remote patients and to facilitate information exchange between primary care physicians and specialists at some distances from each other.1

As an integral component of telemedicine, high bandwidth video conferencing (also known as point-to-point connections) is used in medical education, peer consultation, and in direct patient care (see the American Telemedicine Association site at http://www.americantelemed.org for further details). This research limits its scope to telemedicine encounters that use medical video conferencing deployed for direct patient care.

There is widespread interest in utilizing this technology as an economical method to provide expert medical service to patients in remote and awkward locations and to address misdistribution of health care resources (ie, facilities and practitioners).13 However, utilization rates of installed telemedicine projects have not met expectations.4 While most previous work has found telemedicine to be medically effective, acceptance and use of telemedicine has not been well understood.5,6 Medical state licensure and cost reimbursement have been identified as barriers to utilization of installed systems, but it appears these impediments may not be long standing.7 Furthermore, mixed results have been reported in contexts where policy constraints were not a major impediment.813 These mixed results suggest that research must look more closely at telemedicine systems to understand how to improve utilization rates.

One reason for low utilization rates may be dissatisfaction with the telemedicine encounter experience. Early work looked at this question from the point of view of the physician,14 but few studies have looked at it from the patient perspective. Most prior patient satisfaction studies suggest that teleconsultation is acceptable to patients. Yet, this work has not provided adequate insights into what drives patient satisfaction levels with telemedicine;15 instead it is mostly technology-centered and focused on utilization rates. Studies on telemedicine quality to date tend to fall into two categories. The first category addresses quality in terms of net benefits, namely the quality of patient care or stakeholder satisfaction,1520 with limited insight regarding quality antecedents (attributes). The second category focuses on specific technology quality attributes, such as the quality of audio and/or video aspects of the technology.2123 In spite of increasing use and interest in telemedicine, generalized standards of quality that encompass the patient consultation experience have not emerged. Studying telemedicine quality from the patient perspective, as consumer and indirect user, is needed from a health care business perspective,24 as health care has become a competitive industry where patient opinions are shaping the marketplace and may be missed by an organization or provider if not directly studied.

The purpose of our research is to propose a patient-oriented taxonomy of telemedicine service quality attributes for direct patient care using high bandwidth video conferencing (hereafter referred to as medical video conferencing). We focus on the attributes related to process quality as a means to understand service operations rather than quality of results (ie, service reliability; see Harvey for further discussion of distinctions).25 To facilitate the usefulness of this taxonomy, we also address issues of relevance and control for each attribute identified. We use a multi-dimensional, sociotechnical approach to understanding quality using multiple forms of data collection and multiple perspectives of analyses.

We address the following research questions:

  • Question 1 What quality attributes from the patient perspective contribute to telemedicine encounter success?
  • Question 2 Which telemedicine encounter quality attributes from the perspective of patients are considered most relevant to encounter success?
  • Question 3 At what point in time in relation to the encounter (eg, before, introduction, core, closure, after) are identified attributes most controllable?
  • Question 4 What entities (eg, organization, equipment manufacturer, doctor, patient) have the most control over each identified attribute?

Materials and methods

We conducted an institutional review board-approved multi-method, inductive field study. Qualitative methods (eg, focus group and interview) were used to elicit, code, and analyze data from respondents. Quantitative methods (eg, survey) were used to validate the analyses. We also used numerous sources (telemedicine practitioners and academic researchers) to code and interpret the data and to address triangulation in analysis. The logic involved in the development of the telemedicine taxonomy may be traced both from the participant data to the quality attributes and from the quality attributes to the participant data.

Site

The Department of Veterans Affairs (VA) VISN 8 was the site of this work. VISN 8 is one of the most active VA districts in the area of telemedicine, with a geographic coverage including parts of Georgia, FL, USA, and the Caribbean. We collected data from ten sites in VISN 8 located across the VISN 8 footprint that were identified as particularly active telemedicine spoke sites (patient side for service). The VA’s VISN 8 health system currently provides a diverse spectrum of medical specialties through video conferencing, clinic histories (newly established sites and ongoing sites), types of facility (strip mall clinics to hospital settings), and provider telemedicine experience (new to telemedicine and seasoned). Diversity among these factors was important to the nature of the study. Although we felt it important to introduce various telemedicine encounter experiences into the study, we chose not to introduce organizational diversity to avoid organizational noise.

Establishing the DeLone and McLean model as a contextual fit for telemedicine

By reviewing quality, telemedicine, marketing, service-oriented management and information systems literature as well as unstructured interviews with domain researchers, we developed the initial quality attribute framework. The starting point of our framework was based on constructs in the DeLone and McLean Information Technology quality model (comprised of service quality, system quality, information quality, use, user satisfaction, individual impact, and organizational impact).26 As noted by DeLone and McLean, as well as other domain experts, to make this framework usable in a particular environment, the specific criteria for quality must emerge from investigating the context of its use and understanding of the concerns of the stakeholder groups.27 To contextualize the DeLone and McLean model for telemedicine encounter quality, the following methods of data collection were used:

  • Direct observation (40 hours) of medical video conferencing rooms, functional equipment, and segments of video conference sessions for representations of quality attributes and issues
  • Review of archived video and photographic images of telemedicine encounters and rooms
  • Open-ended patient survey of 84 telemedicine patients (see Figure S1)
  • Unstructured interviews with an originator of the model as well as telemedicine researchers inside and outside of the USA.

In assessing the results of the patient survey questions that compared medical video conferencing with an in-person examination (see Figure S1), it appears that patients within our sample were, by and large, satisfied with their telemedicine encounter.28 However, it became apparent that the use of telemedicine system terminology familiar to patients (eg, physical environment; way the examination was conducted) rather than imposing new constructs (eg, support quality or use quality) was necessary in further discussion with the patients. All dimensions of the research framework (information quality, use quality, technology quality, and service quality) were referenced in some form during direct observation, interviewing, and through the patient survey. For example, patient survey responses to the best and worst aspects of the telemedicine encounter indicated such things as the physical environment, scheduling support, personal attention, and technology were important. The following are the refined definitions of the dimensions of functional quality reflecting both the stakeholder concerns and assumptions that we use to inspire and organize inquiry into specific telemedicine encounter quality attributes.

  • System (technology) quality: the features of medical video conferencing equipment and telecommunication processes utilized for medical video conferencing encounters.
  • Information quality: the characteristics of information that allow participants to take appropriate action concerning patient care and facilitate diagnosis. Telemedicine information quality attributes should include attributes that facilitate simultaneously capturing appropriate input for collaborative communication (eg, aspects of the physical environment) as well as providing appropriate technology transmission output.
  • Support (service) quality: the support provided for use of a telemedicine system during the encounter. In a telemedicine system, it can be defined as the human infrastructure and physical environment provided by the organization that support user comfort and system use.
  • Use quality: the informed and effective communication and deployment of technology by direct users (medical staff) during the medical video conferencing encounter that facilitates desired outcomes.

We adapted the constructs in the information systems success model26 used to guide this study as follows to align with the quality focus of this work:

  • Information quality
  • System (technology) quality
  • Service (support) quality
  • Use (quality).

This framework provides only a precursory understanding of the quality construct. There is no universal set of quality attributes for any of the suggested dimensions because quality is both multi-faceted and domain-specific.25,29

Focus groups to identify framework attributes

Once the preliminary framework was contextualized, we began to collect patient data to better understand what domain-specific quality attributes patients perceived as important to a telemedicine encounter. Patient encounters may be sporadic or limited (eg, one to two occurrences) precluding the development of expertise. Thus, we chose to use focus groups to derive expertise for a collective patient assessment of quality in the telemedicine encounter process.30

To assemble these groups, all patients involved in telemedicine encounters in the previous 3 months at participating facilities were identified. Three months was selected as a maximum time lag from an encounter experience as significant time lags may deteriorate recall to the point where the patient may make little contribution to the group effort. We randomly selected individuals from the 3-month list. VA employees solicited focus group participation via telephone. Prospects on the list were contacted until all planned groups were booked with six to seven participants (to address no-shows). Reminder letters and telephone calls were directed to all potential participants. Of the 32 patients agreeing to participate, 30 actually participated in the focus group process (94%). Focus groups took place at the facility where the patient had his/her medical video conferencing encounter(s) to facilitate recall.

Six focus groups were conducted with an average of six participants per group. Group members were mostly male. We balanced the diversity of medical video conferencing experiences (eg, patients seen by various doctors, in various facilities, by various specialists, and for various conditions) among our focus group participants. All focus group sessions were audio recorded and later transcribed.

Two members of the research team used an open, hermeneutic process to identify and define quality attributes based on information gathered from the focus group.31 We used NVivo (a software package that supports qualitative coding and analysis) to perform this coding. Coding consisted of specifying slips,32 listing attributes, and providing definitions of each attribute. The relevant literature provided insight into identifying terms and labels that have been associated with quality to provide inspiration for coding and to juxtapose emerging knowledge from this study with existing theory and knowledge.31,33

Two telemedicine service providers also coded all of the focus group transcripts (in parallel to the research team efforts) using an open, hermeneutic process. The objective of this “provider” coding was to provide interpretive contextual enrichment to the analysis (enhance relevance) and to determine conceptual convergence with researcher coding (enhance validity).32 These coders did not have a predefined coding schema and were encouraged to annotate their own insights in the coding process. The providers worked as a team to identify attributes and develop definitions. Parallel form procedures were used to assess correspondence among meanings of the attributes identified by practitioners and researchers through open coding.

To assess the reliability of the resulting attributes and their definitions, a third researcher (not engaged in prior open coding) dual-coded a random 33% stratum from each focus group and interview transcript, using the previously established attributes and definitions provided as a coding schema (interrater reliability). The stability of codes over time (intrarater reliability) was assessed by having the same coder code the interviews at two different time periods (second time, four months after the first effort).

Framework validation

To validate the framework that emerged from the focus groups, focus group members were contacted and asked to complete a paper-based survey that included both quantitative and qualitative questions, meant to assess the participants’ perspectives of the correctness, completeness, and relevance of model attributes (see Figure S2 for open-ended questions). The participants also assessed the importance of the proposed attributes.

Assessing attribute control – who and when

Recognition of the relative importance among attributes is needed for research and practice to most effectively utilize the model and focus attention. Additionally, an understanding of the relative importance of the attributes can facilitate the creation and interpretation of formative measures for each dimension of telemedicine quality.

The usefulness of a model of telemedicine service encounter quality attributes to both practice and research is greatly enhanced when there is some understanding of when in the encounter certain attributes of quality can be manipulated and who can manipulate them. To further facilitate application and understanding of the field study findings, we assembled an expert panel of two providers and four researchers, possessing complementary expertise. The six-member panel independently completed a paper-based survey responding to the following two questions for each of the identified attributes:

  1. When is the quality attribute most controllable?
    • Before encounter
    • Beginning of encounter
    • Body of encounter
    • Encounter closure
    • After encounter
    • Not controllable
  2. Who has most control over the quality attribute?
    • Consulting physician
    • Telemedicine coordinator
    • Patient
    • Scheduler
    • Technical support
    • Equipment developer
    • Medical center management
    • External environment

The panel members were instructed to use their personal knowledge as well as their recall of the coded focus group transcripts to answer each question. In the spirit of comprehensive model development, we allowed participants to select multiple attributes in response to each question, as they deemed appropriate. The independent responses were aggregated in a table with differences highlighted and sent to each panel member for review. Panel members met to collectively discuss the differences noted in the aggregated table, acknowledging various insights and clarifying attribute implications. These discussions were followed by additional rounds of survey/review until consensus was reached.

Results

Contextualized framework of quality attributes

We present the resulting contextualized framework and quality attributes for the medical video conference derived from the focus group and validating procedures in Figure 1. Our framework uses four high-level categories of quality (system/technology quality; information [input/output] quality; service/support quality; and use [informed/effective] quality) supported by foundational work done to establish the DeLone and McLean model as a contextual fit for telemedicine (see section entitled “Establishing the DeLone and McLean model as a contextual fit for telemedicine”). To facilitate research rigor and appropriate representation in categorizing the identified attributes within framework constructs, three researchers familiar with this study, the coding process, and the DeLone and McLean model, participated in the mapping process. The researchers first performed an independent paper-based mapping, working from a list of attributes and definitions, and then participated in two rounds of meetings to collectively reach consensus. Aside from providing an appropriate representation of our research model, this process was employed to address any potential of either identified higher-order constructs being inappropriate to this domain or the need for additional higher-order quality constructs. No modification to the higher-order constructs was deemed necessary. However, the researchers subdivided information quality attributes into technology and physical environment aspects and subdivided support quality attributes into human and physical environment dimensions to enhance model precision. The identification of attributes across all quality dimensions indicates the propriety of the research framework as a foundation for the patient perspective.

Figure 1 Telemedicine service encounter quality model – patient perspective.

Regarding the identification of specific attributes under each category, several rounds of parallel form procedures were used to assess correspondence among meanings of the attributes identified by practitioners and researchers through open coding. Common themes were found throughout the process and labels for the attributes were derived through consensus (see Table S1 for final definitions). Final consensus resulted in six to eight specific quality attributes (eg, reliability, audio clarity, technical support) derived from patient focus groups for each of the four high-level categories of quality.

Results of dual coding by a third researcher using the attribute model and definitions previously reconciled indicated a high interrater reliability (consistently >90%). The “recoding over the passage of time” intrarater reliability measurement technique resulted in high intrarater reliability (consistently >95%). No patterns were noted in the intrarater or interrater differences that might reflect the need to modify the attribute listing.

The participants’ assessment of the importance of all proposed attributes in the framework had a mean in excess of three (anchor point, important) on a four-point scale (four anchor point, critical), indicating that the participants considered all attributes to be relevant. Given the opportunity, participating patients did not mention any missed attributes. Table 1 indicates the relevancy scores of each of the quality attributes. The presentation order of the constructs in Table 1 is an indication of the importance of each attribute (high numbers indicate most relevant attributes) based upon Likert-type scale measures from the survey administered to the focus group for validating purposes. The scores are ranked in order by highest mean, then lowest standard deviation.

Table 1 Results of patient validation and relevancy score
Abbreviation: SD, standard deviation.

For the most part, physical environment attributes (eg, facilitating décor and suitable temperature) seem to be on the lower end (under three points on a four-point scale). Scores above 3.5 on the four-point scale indicate that “basic” technical functionality (audio clarity, image resolution, and peripheral sophistication) coupled with aspects of the communication process (clear future directives, medical team coordination, telemedicine trained staff, professionalism, patient education/orientation, conveying access, and review of medical records) are most critical in this context.

Control of attributes – when and who

After two iterations of independent mapping and sharing results, the expert panel collectively reached consensus regarding when each quality attribute was most controllable and who has the most control over the quality attribute. When these quality attributes can be controlled is shown in Table 2 and who can control them is shown in Table 3. Table 2 illustrates that the majority of the attributes can be controlled in multiple time frames associated with the encounter, and almost all have the ability to be controlled prior to an encounter beginning. Table 3 indicates that most of the quality attributes are not under the control of the patient but rather under the control of either medical providers, those responsible for the telemedicine equipment used during the encounter, or various hospital administrators (including telemedicine coordinators and schedulers).

Table 2 When quality attribute is most controllable for telemedicine service encounters
Note: ⊕ indicates when each attribute is most in control for each medical video conferencing quality attribute.

Table 3 Role most in control for medical video conferencing quality attribute
Note: ⊕ indicates what role is most in control for each medical video conferencing quality attribute.

Discussion

Attributes of a quality telemedicine encounter

Organizations will only achieve advantage through quality when there is a match between the importance that consumers (patients) assign to individual quality attributes and the organization’s performance along those dimensions.34 While 89% of the patients felt the medical video conference examination was the same or better than an in-person examination, 42% felt that their experience could be improved (see Figure S1). The identification of attributes across all quality dimensions (use quality, service quality, system quality, and information quality) implies that both groups associate telemedicine quality with both social and technical factors. The intermingling of human (eg, technical support, scheduling support) and technical (eg, peripheral sophistication, audio clarity) attributes seems to exist throughout the ranking of relevance scores.

The quality attributes that the patients ranked higher (mean of greater than 3.5 as presented in Table 1) reflect what the patients perceive as critical components in their telemedicine service encounters. Namely, patients want instructions and guidance regarding their health provided via a means that addresses the challenges of distance and technology and that supports the information-sharing process. It is notable that peripheral sophistication, which addresses enhancing features, was ranked high (above 3.5, see Table 1) among the technology attributes list. Informal interviews with providers and direct observations of telemedicine examinations conducted during research groundwork indicate a provider may envision the ability to extend the capabilities of available equipment through some adaptation before peripherals are necessary. Patients may not have this vision and may feel more comfortable with equipment designated for a specific task. Given that physical environment attributes were among the lower-ranked attribute scores, it seems patients recognize comfort is desirable but may be willing to sacrifice some comfort to gain the conveniences of telemedicine.

Patients indicate that their medical provider should be able to focus on patient care, rather than figuring out technology. Technology should not get in the way of either the physician or the patient. To enable this focus, technical roles, organizational roles, and external factors must support provider efforts during windows of opportunity. Ideally, telemedicine will become another tool in the medical kit that can provide remote care where both the patient and provider walk away from the encounter with a feeling of fulfillment and success.

During a telemedicine encounter the consulting doctor is usually the person in charge. However, we found that many of the quality attributes are in the control of other actors and all of the attributes can (and should) be addressed in some way prior to an encounter. For an encounter to be successful, planning and preparation are essential. Technical, organizational, and medical provider roles must all work as a team and act before the encounter begins to maximize the potential for success.

The technical actions that appear to be most important to a successful telemedicine encounter are ones associated with the actual design of the equipment. Both technology quality attributes and information quality attributes are controlled by the equipment design. The equipment manufacturers need to ensure that the equipment is reliable, easy to maneuver (ergonomic), and can perform a range of medical tasks (peripheral sophistication).

The organizational actions that improve the successfulness of a telemedicine encounter can be largely managed prior to an encounter. These actions include managerial planning (in the form of staff training and patient information management) and building management (in the form of room design, ie, adequate privacy and lighting as well as room décor). Once the encounter begins, the provider and patient have a more active role in ensuring the quality level of the telemedicine encounter. With the help of some support roles, quality attributes, such as lighting, noise level, privacy, and technology performance, can be controlled. We also see communication-related attributes playing a strong role during the encounter. Style of communication (eg, professionalism-clinician in the room, “telepresence”) as well as communication content (eg, conveys access/review of patient records, clear future directives) are necessary. These attributes seem to have some element of technological influence. For instance, to control “telepresence”, both interpersonal skills and camera placement to create the image of virtual “eye contact” are needed.

Impact on practice

This research has led to a revision of training programs and telemedicine encounter protocols in the site locations that participated in the study. Additionally, patient telemedicine training and information brochures developed using data from this study are being used throughout the country by two major health care agencies. The model has also been used in the inception of new programs and as a tool for indoctrinating stakeholders in the telemedicine process. Furthermore, one telemedicine administration indicated the model served a need in the health care industry for a communal model that could be referenced by all telemedicine stakeholders. To this end, a telemedicine committee used the model during a planning session to provide a common language and framework from which to elicit current deficiencies in the existing telemedicine process. Results from the control and importance tasks in this study provided guidance to the committee regarding future action by identifying the process stage most affected by reported deficiencies and entities that could work toward remedies.

Limitations

This study took place within VISN 8 of the VA system in the USA. It is not clear if the findings are generalizable to private sector health facilities, or to other regions of the USA or within other countries. As the health and technological environment evolves, new attributes will likely arise and some attributes may become unworthy of mention. Furthermore, other health care delivery systems (telephone consults, mobile applications, in-home long-term care) may require adaptation of the quality attributes resulting from this study. Future research is necessary to understand the applicable boundaries of this framework.

Conclusion and future directions

The principal contributions of this paper are an organized taxonomy of quality attributes for a medical video conferencing system that contribute to encounter success from the patient perspective and indications of how these attributes can be controlled and by whom. Through this taxonomy, quality can then be assessed according to the presence or absence of such attributes and the fitness for their proposed use.35,36 The need to recognize both social and technical attributes as antecedents to success in the study of telemedicine service encounters is not a superficial one.

Attempting to define quality with enough precision to substantially enrich understanding and promote positive action for the sophisticated phenomenon of telemedicine service encounters is a complex task that begets our multi-method research. We employed an inductive field study as a means to define the elements of service quality that are vital to the success of a specific form of telemedicine encounter. We would direct researchers interested in studying quality to consider all relevant contextual attributes within the scope of the problem of interest (particularly those of critical importance).

We provide a second extension to understanding by identifying opportunities for control of each attribute. This effort enables researchers to better understand the encounter process from the patient perspective by associating attributes with the sequential phases of the encounter and with entities that may serve as points of control. Future research could focus on how the control of the quality of telemedicine service encounters is best achieved, given the multiple, disparate sources and points of control. Practices may use this model as a means to recognize and assess telemedicine quality standards, contemplate requirements while balancing constraints, and enact control directives. Retrofitting quality improvements into an existing system can be cost-prohibitive and technically challenging. Ideally, the design of a telemedicine system would begin with unambiguously stated quality requirements in conjunction with functional requirements. Our analysis of control issues indicates that there exist opportunities to economically and efficiently correct some attribute deficiencies beyond the planning stage (eg, patient orientation and adaptability). Telemedicine does introduce change, namely the utilization of technology service encounters, into the traditional process of health care delivery. The challenge to the service provider is to orchestrate the encounter so that all technical imperatives (and clinical needs, in this case) are met, while managing the social/communication process and customer expectations and perceptions. An understanding of the factors of quality and opportunities for control can help the change process as well as the ongoing management of telemedicine.

Disclosure

The authors report no conflicts of interest in this work.


References

1.

Darkins A, Carey M. Telemedicine and Telehealth. New York, NY, USA: Springer Publishing Company; 2000.

2.

Maheu MM, Whitten P, Allen A. E-Health, Telehealth, and Telemedicine: A Guide to Start-Up and Success. San Francisco, CA, USA: Jossey-Bass; 2001.

3.

Shea S, Kothari D, Teresi JA, et al. Social impact analysis of the effects of a telemedicine intervention to improve diabetes outcomes in an ethnically diverse, medically underserved population: findings from the IDEATel Study. Am J Public Health. 2012;103(10):1888–1894.

4.

Grigsby J. Telehealth: an assessment of growth and distribution. J Rural Health. 2002;18(2):348–358.

5.

Grigsby J, Sanders J. Telemedicine: where it is and where it is going. Ann Intern Med. 1998;129(2):123–127.

6.

Levin K, Madsen J, Petersen I, Wanscher C, Hangaard J. Telemedicine diabetes consultations are cost-effective, and effects on essential diabetes treatment parameters are similar to conventional treatment:7-year results from the Svendborg Telemedicine Diabetes Project. J Diabetes Sci Technol. 2013;7(3):587–595.

7.

Cram N. Telemedicine: a technology primer for clinical engineers. J Clin Eng. 2001;26(1):61–71.

8.

Linderoth HCJ. Fiery spirits and supporting programs of action – the keys to exploration and exploitation of open technologies. International Journal of Health Care Technology and Management. 2002;4(3/4):319–332.

9.

Shile PE, Kundel HI, Seshradi SB. Factors affecting the electronic communication of radiological information to an intensive-care unit. J Telemed Telecare. 1996;2(4):199–204.

10.

Mitchell BR, Mitchell JG, Disney AP. User adoption issues in renal telemedicine. J Telemed Telecare. 1996;2(2):81–86.

11.

Franken EA, Whitten P, Smith WL. Teleradiology services for a rural hospital: a case study. J Telemed Telecare. 1996;2(3):155–160.

12.

Armstrong AW, Kwong MW, Chase EP, Ledo L, Nesbitt TS, Shewry SL. Teledermatology operational considerations, challenges, and benefits: the referring providers’ perspective. Telemed J E Health. 2012;18(8):580–584.

13.

LeRouge C, Garfield MJ. Crossing the telemedicine chasm: have the U.S. barriers to widespread adoption of telemedicine been significantly reduced? Int J Environ Res Public Health. 2013;10(12):6472–6484.

14.

LeRouge C, Garfield M, Hevner A. Provider perspectives of telemedicine encounter quality. International Journal of Health Care Technology and Management. 2005;6(4/6):397–419.

15.

Mair F, Whitten P. Systematic review of studies of patient satisfaction with telemedicine. BMJ. 2000;320(7248):1517–1520.

16.

Brennan JA, Kealy JA, Gerardi LH, et al. Telemedicine in the emergency department: a randomized controlled trial. J Telemed Telecare. 1999;5(1):18–22.

17.

Bashshur RL. Telemedicine effects: cost, quality, and access. J Med Syst. 1995;19(2):81–91.

18.

Kairy D, Tousignant M, Leclerc N, Cote AM, Levasseur M, Researchers TT. The patient’s perspective of in-home telerehabilitation physiotherapy services following total knee arthroplasty. Int J Environ Res Public Health. 2013;10(9):3998–4011.

19.

Prescher S, Deckwart O, Winkler S, Koehler K, Honold M, Koehler F. Telemedical care: feasibility and perception of the patients and physicians: a survey-based acceptance analysis of the Telemedical Interventional Monitoring in Heart Failure (TIM-HF) Trial. Eur J Prev Cardiol. 2013;20(s2):18–24.

20.

von Wangenheim A, de Souza Nobre LF, Tognoli H, Nassar SM, Ho K. User satisfaction with asynchronous telemedicine: a study of users of Santa Catarina’s system of telemedicine and telehealth. Telemed J E Health. 2012;18(5):339–346.

21.

Crump WJ, Kumar R, Orsak G, Pfeil T. A field trial of two telemedicine camera systems in family practice. Arch Fam Med. 1998;7(2):174–176.

22.

Wootton R, Dorman J, Fisk NM, et al. The effect of transmission bandwidth on diagnostic accuracy in remote fetal ultrasound scanning. J Telemed Telecare. 1997;3(4):209–214.

23.

Hahm JS, Lee HL, Choi HS, Shimizu S. Telemedicine system using a high-speed network: past, present, and future. Gut Liver. 2009;3(4):247–251.

24.

Bensing J, Deveugele M, Moretti F, et al. How to make the medical consultation more successful from a patient’s perspective? Tips for doctors and patients from lay people in the United Kingdom, Italy, Belgium and The Netherlands. Patient Educ Couns. 2011;84(3):287–293.

25.

Harvey J. Service quality: a tutorial. Journal of Operations Management. 1998;16(2):583–597.

26.

DeLone WH, McLean ER. The DeLone and McLean model of information system success: a ten-year update. Journal of Management Information Systems. 2003;19(4):9–30.

27.

Stamoulis D, Kanellis P, Martakos D. An approach and model for assessing the business value of e-banking distribution channels: evaluation as communication. Int J Inf Manage. 2002;22(4):247–261.

28.

Gustke SS, Balch DC, West VL, Rogers LO. Patient satisfaction with telemedicine. Telemed J. 2000;6(1):5–13.

29.

Sousa R, Voss C. Quality management re-visited: a reflective review and agenda for future research. Journal of Operations Management. 2002;20(1):91–109.

30.

Morgan DL. Focus Groups as Qualitative Research. Thousand Oaks, CA, USA: Sage Publications; 1997.

31.

Strauss A, Corbin J. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA, USA: Sage Publications; 1990.

32.

Trauth E, Jessup L. Understanding computer-mediated discussions: positivist and interpretive analyses of group support system use. MIS Quarterly. 2000;24(1):43–79.

33.

Anderson JC, Rungtusanatham M, Schroeder RG. A theory of quality management underlying the Deming management method. Acad Manage Rev. 1994;19(3):472–509.

34.

Garvin D. What does “product quality” really mean. Sloan Manage Rev. 1984;26(1):25–45.

35.

Juran J. Managing for quality. Journal of Quality and Participation. 1988;11(1):8–12.

36.

Barneveld Binkhuysen FH, Ranschaert ER. Teleradiology: evolution and concepts. Eur J Radiol. 2011;78(2):205–209.

Supplementary materials

Figure S1 Patient concerns and issues survey questions and response summaries.
Abbreviations: SA, strongly agree; A, agree; D, disagree; SD, strongly disagree.

Figure S2 Open-ended questions on patient validating survey.

Table S1 Telemedicine quality attributes and definitions

Creative Commons License © 2014 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.