Back to Journals » Journal of Multidisciplinary Healthcare » Volume 14

Outpatients’ Opinion And Experience Regarding Telepharmacy During The COVID-19 Pandemic: The Enopex Project

Authors Margusino-Framiñán L , Fernández-Llamazares CM, Negro-Vega E, Tortajada-Goitia B, Lizeaga G, Mercadal-Orfila G, Almeida-González C , Morillo-Verdugo R 

Received 18 October 2021

Accepted for publication 15 December 2021

Published 31 December 2021 Volume 2021:14 Pages 3621—3632


Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Scott Fraser

Luis Margusino-Framiñán,1 Cecilia M Fernández-Llamazares,2 Eva Negro-Vega,3 Begoña Tortajada-Goitia,4 Garbiñe Lizeaga,5 Gabriel Mercadal-Orfila,6 Carmen Almeida-González,7 Ramón Morillo-Verdugo8

1Pharmacy Department, A Coruña Universitary Hospital, A Coruña, Spain; 2Pharmacy Department, General Universitary Gregorio Marañón Hospital, Madrid, Spain; 3Pharmacy Department, Getafe Universitary Hospital, Madrid, Spain; 4Pharmacy Department, Costa del Sol Hospital, Marbella, Spain; 5Pharmacy Department, Donostia Universitary Hospital, San Sebastian, Spain; 6Pharmacy Department, Mateu Orfila Hospital, Menorca, Spain; 7Statistics&Methodology Department, Virgen de Valme Universitary Hospital, Sevilla, Spain; 8Pharmacy Department, Virgen de Valme Universitary Hospital, Sevilla, Spain

Correspondence: Luis Margusino-Framiñán
Pharmacy Department, A Coruña Hospital, c/As Xubias, 84. 1º Floor, A Coruña, 15006, Spain
Tel +34 981 176480
Email [email protected]

Background: Telepharmacy, as a remote pharmaceutical care procedure, is being used worldwide during the COVID-19 pandemic, with the aim of preserving the health of patients and professionals. Its future development should incorporate the assessment of patient perception, but no research study has investigated it.
Objective: The objective was to poll the opinions and experiences of outpatients with telepharmacy through a purpose-developed questionnaire and to assess it’s quality through an internal validity and reliability analysis.
Methods: Cross-sectional observational study of adult patients who used telepharmacy services during the COVID-19 lockdown period in Spain. The subjects answered a 24-item questionnaire, after giving their informed consent. Place of delivery, informed pharmacotherapeutic follow-up, opinion about telepharmacy, future development, ethics/satisfaction, and coordination constituted the six questionnaire categories. After assessing the adequate sample size with the Kaiser–Meyer–Olkin test, the Bartlett sphericity test analyzed the validity of the questionnaire. The intraclass correlation coefficient and Cronbach’s α coefficient calculations verified the reliability and internal consistency.
Results: A total of 9442 interviews were administered to patients from 81 hospitals, of which 8079 were valid (52.8% female). A 54.1% were aged between 41– 65 years; 42.7% had been in treatment for more than 5 years; 42.8% lived between 6– 31 miles from the hospital. As many as 96.7% of patients were “satisfied” or “very satisfied” with telepharmacy, 97.5% considering it complementary to their usual follow-up; 55.9% expressed a preference for being followed up face to face when visiting the hospital. 75.6% said they had rather receive their medication at home. The sample size obtained was deemed appropriate [the Kaiser–Meyer–Olkin test (0.789) and Bartlett’s sphericity test (p< 0.005)]. The reliability analysis resulted in a Cronbach α = 0.7.
Conclusion: Patients have shown high satisfaction with telepharmacy and the ENOPEX questionnaire is a tool with sufficient validity and reliability to be used in the evaluation of the care that patients receive through telepharmacy.

Keywords: telepharmacy, coronavirus, pharmaceutical care, hospital pharmacy service, healthcare quality assessment, patient reported experience measures

Plain Language Summary

During the COVID-19 pandemic, health authorities promoted the use of telemedicine (remotely provided healthcare) in order to avoid contagion and preserve the health of patients and healthcare providers. Telepharmacy is the part of telemedicine related to pharmaceutical care. This study was carried out to gather information about the experience and the opinions of patients served by telepharmacy during the pandemic. To this end, a questionnaire was developed and evaluated for reliability and validity. The questionnaire was sent to more than 8000 patients serviced by telepharmacy to gain insight into their perception of this pharmaceutical care procedure. Understanding patients’ views on telepharmacy is of the essence as the demand for telepharmacy will continue once the pandemic is over. The results of the survey were very positive as the vast majority of patients hailed telepharmacy as a very useful complementary healthcare tool.


Dispensation of medications in Spain is regulated by the national legislation. This legislation establishes that certain medicines must always be dispensed to outpatients by hospital pharmacists in a hospital pharmacy department (HPD). Such regulations ensure that patients treated with antiretrovirals, biological agents, antineoplastics, etc., are subjected to pharmacotherapeutic follow-up (PTFU) by a specialist hospital pharmacist.1 Pharmaceutical care (PC) of these patients has improved in our country in the last decades, with different HDPs2 implementing specialized patient consultations dedicated to following up on the effectiveness, safety, adherence, interactions, etc., of the treatments administered. This implementation is the result of the development of the Strategic Framework for Outpatient Pharmaceutical Care (MAPEX)3 and the Capacity, Motivation and Opportunity (CMO) program.4

The type of outpatient PC provided by HPDs has evolved in the last few years as a result of a series of factors including the introduction of information and communication technologies into healthcare, the empowerment of patients and the increasing demand for telemedicine5–8 services. Against this background, the Spanish Society of Hospital Pharmacists (SEFH) has issued a Position Statement where telepharmacy is defined as the provision of PC at a distance through information and communication technologies.9 Telepharmacy currently constitutes a strategic pillar in outpatient PC for activities such as PTFU or remote dispensation and delivery of medications.3

Telepharmacy has experienced a phenomenal surge during the COVID-19 pandemic to ensure the protection of both patients and healthcare providers. It has at the same time represented a huge challenge for HPDs as they strive not to compromise clinical outcomes or the quality of healthcare in the face of the prevailing circumstances.10

During the COVID-19 pandemic, SEFH conducted a survey of 185 hospitals to analyze the quality of outpatient PC provided by Spanish HPDs by means of telepharmacy services.11 The survey revealed that recourse to such services (supplemented by teleconsultations) had increased from 16.8% al 100%, in 87.6% of hospitals.12 However, the survey did not take into consideration the respondents’ perception of the quality of telepharmacy services during the state of emergency enforced during the pandemic.

For that reason, and given the lack of studies on the subject, SEFH decided to launch the ENOPEX Project, with the main purpose of exploring outpatients’ opinion on and experiences with telepharmacy (as provided by Spanish HPDs) during the COVID-19 pandemic. The secondary goal was to design a questionnaire capable of quantifying such opinions and experiences, and determine the questionnaire’s validity and reliability.

Materials and Methods

A multicenter cross-sectional observational study (based on the STROBE Statement) was carried out. A stratified sample of adult outpatients under PTFU at Spanish HDPs, who benefited from telepharmacy services during the COVID-19 pandemic, were included. The project was divided into four phases. Phase 1 consisted in a review of studies with similar goals published worldwide; Phase 2 comprised the preparation of the survey; Phase 3 its administration; and Phase 4 the analysis of the data obtained.

Questionnaire Development

A working group was put together to work on phases 1 and 2. It was made up of hospital pharmacists from the MAPEX project,3 SEFH officials and an epidemiologist. Activities carried out included an evaluation of the relevant literature, an assessment of existing questionnaires related with telemedicine services, the development of a feasible and reliable indicator to determine the validity and usefulness of the reviewed questionnaires, an analysis of the feasibility of adapting the reviewed questionnaires to the Spanish context; and the design of already-to-administer version of the questionnaire. Items in the questionnaire were grouped into six domains, specific to outpatient PC: place of delivery, informed PTFU, opinion about telepharmacy, future development of the service, ethics/satisfaction, and coordination. Finally, the final format of the questionnaire was decided, with the question Do you think that telepharmacy is complementary to in-hospital pharmaceutical services? As the main research question (Annex I).

Survey Sample

According to the inclusion criteria, based on the Study Protocol authorized by the Clinical Research Ethics Committee, the study comprised adult patients served by telepharmacy during the period between March 15 and May 15, 2020, who granted their written or verbal consent (by telephone) to participate in the study. Patients attended face to face in the outpatient care unit of a hospital pharmacy department and patients who did not give their written or verbal consent (by telephone) to participate in the study were excluded.

The sample size was set at 120,000 patients, on the basis of data from a survey on the situation of telepharmacy services in Spain conducted by SEFH a few months earlier.12 It was assumed that 75% of subjects would be in favor of telepharmacy procedures, which means that 16,588 completed surveys would be needed. We assumed a 1% error margin, a 99% confidence interval, a 20% beta error, and that 25% of surveys would not be responded. Moreover, prior to performing the statistical analysis, a power analysis was carried out, calculating Cohen’s d at 80% power using the pwr package in R.13

An open non-competitive process was used to recruit respondents from 102 hospitals of all sizes and across all levels of care from all Spanish regions.

Data Collection

Data was collected over a period of 10 weeks (1 July to 10 September 2020) and transferred to a template hosted on SEFH’s website ( The data was compiled and administered using REDCap® electronic data capture tools, stored in a SEFH-owned server. Each respondent was additionally identified by another alphanumeric code to guarantee anonymization.

To ensure that an adequate sample was obtained, respondents were given the possibility to complete the questionnaire at their hospital, by telephone or online (using a QR code). On the first page of the questionnaire, respondents had to provide their explicit consent to participate in the study. Only fully completed questionnaires were considered valid. All associations between qualitative characteristics were assessed by the Chi-Square test.

A logistic regression analysis was performed to determine which patient profiles were most conducive to the use of telepharmacy as a service complementary to in-hospital PC. In this analysis, the dependent variable was item 15 in the questionnaire and the covariates were the respondents’ demographic and socioeconomic characteristics. Goodness of fit was assessed by a Hosmer-Lemeshow test. The analysis was performed using statistical software R,14 assuming a 5% alpha error and a 20% beta error.

Questionnaire Validity and Reliability Analysis

An expert committee evaluated the validity of the contents of the questionnaire. The committee analyzed the comprehensibility of items and the relevance of domains. To determine the validity of the construct, factor analyses were conducted. These analyses tested whether the items were properly grouped under their corresponding domains; also, whether they were capable of accounting for the results provided by the answers. Before performing these analyses, the appropriateness of the sample was evaluated using the Kaiser–Meyer–Olkin test. Bartlett’s sphericity test was used to analyze the validity of the questionnaire and determine whether items were properly grouped under their corresponding domains. The reliability of the survey was determined by calculating the intraclass correlation coefficient. The internal consistency or homogeneity of the items was established by calculating Cronbach’s alpha coefficient. All statistical analyses were carried out using the IBM SPSS 25.0 software package (IBM, Armonk, NY, USA).

Ethical Considerations

The ENOPEX project was carried out in accordance with all international research ethics standards. It was classified by the Spanish Agency for Medicines and Health Products as an observational non-post-authorization study. The classification was valid for the whole of Spain. The ENOPEX Study Protocol was approved by the Clinical Research Ethics Committee of Valme University Hospital on 30 June 2020, with number 1524-N-20. It should be noted that, in Spain, the authorization of a research protocol granted by a Clinical Research Ethics Committee has nationwide validity, which means this authorization is valid for any researcher who wishes to participate in the study. Patients were identified in a data logbook by means of a unique code to protect their identity and their personal data. Once all the patients had been included, the data was exported to specific biostatistical analysis software in a fully anonymized way using the code they had been assigned initially. All survey participants were informed about the purpose of the study, which was conducted in accordance with the Declaration of Helsinki.


Baseline Demographic Data and General Characteristics of the Sample

Eighty-one hospitals of all sizes and across all levels of care participated in the study. A total of 9442 surveys were administered of which 8079 were considered valid (Figure 1). The effect size as calculated by Cohen’s d produced a value of 0.038. With a sample size of 8079 questionnaires, the error margin increased from 1% to 1.6%. The confidence interval and the power of the study were maintained at 99% and 80%, respectively. Respondents were similarly distributed between men and women, were mostly middle-aged, had for the most part been subjected to PTFU for over a year and lived far away from their hospital [60.4% had to travel for more than one hour (there and back) to get to the hospital for their face-to-face consultations]. Most patients were treated by the following departments: neurology (16.7%), internal medicine/infectious diseases (16.5%), rheumatology (15.7%), and haemato-oncology (10.3%). Baseline demographic data can be found in Table 1.

Table 1 Demographic Data

Figure 1 Study population flowchart.

Questionnaire Validity and Reliability Analysis

The Kaiser–Meyer–Olkin (KMO) measure of sample adequacy was 0.789 and Bartlett’s sphericity test yielded a p value <0.005 for the factor analysis, confirming that the 24 items were correctly grouped under the six dimensions established, which accounted for 53.4% of total variance. The reliability analysis produced a Cronbach alpha coefficient of 0.6, which rose to 0.7 when four items considered to be redundant were removed (items 1, 18, 19 and 22). (see Annex 1). The validated questionnaire therefore included a total of 20 items.

Experience. Pharmacotherapeutic Follow-Up

Information about the existence of a telepharmacy program was provided mostly by hospital pharmacists (71.6%). A total of 86.3% of respondents participated in a pharmaceutical teleconsultation before being sent the medication, either by phone or via the mobile app and/or e-mail. Of the information provided to them during the teleconsultation, it was the details related to the COVID-19 pandemic itself that subjects found the most useful (84.7%), as compared with the data pertaining to their specific treatment (78.2%) [OR 1.08 (1.07–1.10); p<0.001] (Figure 2).

Figure 2 Outpatients’ experience regarding the usefulness of telepharmacy in providing information on the COVID-19 pandemic, specific COVID-19 medication, and their overall treatment.

Experience. Places of Dispensation/Delivery

A total of 85.8% of patients received their medication at home, as compared with 10.8% who picked it up at a community pharmacy and 3.4% who collected it from a primary health center. As many as 96.9% of patients were either totally or fairly satisfied with the telepharmacy service they received (85.6% and 11.2%, respectively). Levels of dissatisfaction and neutrality of opinion varied as a function of whether the medication was delivered at home (2.6%), the community pharmacy (6.1%), or the primary health center (8.0%) (p <0.001), with a mean value of 3.1%. As regards the confidentiality of the telepharmacy program, 98.4% of patients pointed out that they experienced no issues in that regard. Where confidentiality problems did arise, their incidence varied as a function of whether the medication was delivered at home (1.3%), the community pharmacy (3.1%) or the health center (4.7%), p <0.001. The mean rate of confidentiality issues stood at 1.6%. The overall patient experience score was 9.73 (SD 0.72); the specific scores for the 6 aspects related to the medication delivery process are shown in Table 2.

Table 2 Medication Delivery Process Patients’ Experience

Opinion About Telepharmacy

A total of 97.8% of patients considered that telepharmacy was complementary to in-hospital PC and 96.9% were either highly or fairly satisfied with the service received. The multivariate logistic regression analysis (Figure 3) identified 5 variables that were independently related with the recognition of telepharmacy as a complementary PC procedure: length of PTFU at the outpatient unit, time required to travel to the hospital, place of delivery, opportunity for a teleconsultation prior to receiving the medication, and confidentiality of the delivery (p<0.028). Moreover, 75.3% considered that telepharmacy had positively impacted their communication with hospital pharmacists. The most appreciated aspect about telepharmacy was the possibility not to have to travel to the hospital while the state of emergency was in force thus avoiding the risk of infection with the Sars-CoV-2 virus (74.2%). The majority of respondents (66.3%) did not express a negative appreciation for any aspect about telepharmacy (Figure 4; A:More appreciated; B: Less appreciated). From these opinions it follows that 99.0% of respondents would recommend the use of telepharmacy, and that 91,3% would be in favor of retaining the telepharmacy program once the COVID-19 pandemic is over. Furthermore, 43.5% of patients would be very much or somewhat in favor or in favor of bearing the direct costs of telepharmacy (24.8% and 18.7% respectively). This percentage ranged between 42.1% and 45.4% across all occupational status groups except for the unemployed, where it decreased to 36.8% (p<0.001). A total of 34.2% of patients declared themselves totally against or against (19.3% and 15.0%, respectively) having to bear the direct costs of telepharmacy, with 22.3% of them expressing a neutral opinion in this regard (Figure 5). Respondents’ preferences regarding the place of delivery of their medication varied as a function of whether they had any medical appointments or diagnostic tests programmed at the hospital: if they had, most of them (55.9%) preferred in-hospital PC and if they did not, most (75.6%) preferred to be served by the home delivery option; p<0.001 (Figure 6; A: With concomitant doctor visit; B: Without concomitant doctor visit).

Figure 3 Logistic regression to determine patient predisposition to telepharmacy.

Figure 4 Outpatients’ appreciation of the different aspects of telepharmacy. (A) More appreciated; (B) less appreciated.

Figure 5 Willingness to pay according to patients’ employment status.

Figure 6 Future preferences regarding place of delivery as a function of the need to travel to the hospital for a medical appointment or a functional exam, (A) with concomitant doctor visit; (B) without concomitant doctor visit.


The ENOPEX study has provided valuable insight into the opinion and experience of outpatients treated by Spanish HPDs via a telepharmacy program during the COVID-19 pandemic. In order to ensure the quality of healthcare it is of the essence to understand and evaluate the perception patients have of all relevant procedures based on ISO Standard 9001, the EFQM Excellence Model, or the Joint Commission’s standards.15–18 Both the European and the US regulatory agencies have incorporated patient-reported outcomes or experience measures to their decision-making processes.19 Telepharmacy has enjoyed widespread implementation and a speedy development across Spanish HPDs as a result of the COVID-19 pandemic.11 For that reason, SEFH thought it necessary to poll the opinion and experiences of outpatients regarding the care administered through telepharmacy, using a survey within a research protocol.20 This was particularly useful given the dearth of information about the subject in the records of Spanish and international professional pharmacist organizations and on official healthcare databases. However, as no validated standardized questionnaires existed for the purpose intended, which is a requirement for a study of the characteristics proposed,21 the ENOPEX expert group set about designing an ad hoc 24-item questionnaire and subsequently analyzed its validity and reliability for the proposed task. After removing the above-mentioned four redundant items, the analysis yielded a Cronbach alpha coefficient of 0.7, which means that it was valid to poll the perception of Spanish HPD outpatients regarding telepharmacy.

As regards the sample analyzed, although the above-mentioned 16,588 valid surveys were not reached, the narrow margin of error obtained indicates that the sample can be considered valid and that it fully represents and accurately defines the total population served via telepharmacy during the COVID-19 pandemic. The initially proposed confidence interval and statistical power were maintained. The demographic and socioeconomic factors that characterized the patients in this study (distance from hospital, employment status, baseline disease, time spent for the face-to-face consultations, etc.), warrant the development of telepharmacy procedures geared towards outpatients beyond the state of emergency declared as a result of the COVID-19 pandemic.22

The ENOPEX study was structured around three overarching areas: two related with the patients’ experience (PTFU and place of dispensation/delivery), and one related with the patients’ opinion.

As regards the patients’ experience regarding PTFU, their appreciation for both the treatment received and the information provided to them regarding the situation arising from the COVID-19 pandemic was very high in this study. Although numerous studies have evaluated the impact of telepharmacy on health outcomes or on healthcare expenditure, from the standpoint of the health system itself,23–26 no study has so far taken the patient’s perspective, except for a small-scale study of a group of HIV patients27 and a preliminary study on the use of telepharmacy during the pandemic,28 both of them showing a high perceived quality of telepharmacy-based PTFU. Moreover, although respondents in our study considered the usefulness of telepharmacy to be significantly more closely related to the information provided by hospital pharmacists on the COVID-19 pandemic than to recommendations about their pharmacological treatment,29 their satisfaction with the traditional PTFU received remained high.30–33

The respondents’ assessment of their experience with the informed dispensation/delivery procedure was very favorable overall, although their level of satisfaction was significantly higher when the place of delivery was their home, as compared with a community pharmacy or their primary health center. This data reflects the patients’ total confidence regarding the logistic aspects of delivery(conservation temperature, expiry date, etc.). As regards the confidentiality of the telepharmacy procedure, several Spanish and international organizations have issued position statements underscoring the importance of preserving confidentiality at all times.6–9,34,35 In this respect, a vast majority of respondents in this study said they had experienced no problems whatsoever regarding confidentiality. A few of them did mention confidentiality issues when picking up their medication from the community pharmacy or their primary health center.

As regards the patients’ opinion about the telepharmacy service, although outpatients were highly satisfied with their experience with telepharmacy, they expressed the opinion that the service should be complementary to in-hospital PC, which is in line with the position of SEFH3 and other professional organizations.36 In addition, the multivariate analysis conducted as part of the study independently identified certain patient profiles that showed a particular appreciation for telepharmacy as a complementary tool. These were patients who had received PTFU for over one year (which was the maximum period during which respondents considered that PTFU should be delivered face-to-face rather than via telepharmacy) or patients living far away from their hospital. Now that these independent variables have been identified, HPDs should use them as prioritization criteria to design and implement telepharmacy programs as a way of dealing with some of the challenges arising from the COVID-19 pandemic.37 The three remaining independent variables identified indicate that patients are willing to accept telepharmacy as a complementary tool provided that the place of delivery of the medication is their home or their primary health center and that they have a previous teleconsultation with a specialist pharmacist, even in cases where there may be confidentiality issues with the procedure. It should also be mentioned that the most highly appreciated feature of telepharmacy during the pandemic has been that it has made physical visits to the HPD unnecessary reducing the risk of viral infection,38 with the overwhelming majority of respondents not finding fault with any part of the telepharmacy service. As regards future preferences regarding where the medication should be delivered, patients stated that they would prefer a face-to-face consultation at the HPD in cases where they have a doctor’s appointment at the hospital for the same day, and home delivery of their medication via telepharmacy in all other cases.

One last aspect to be considered has to do with the respondents’ willingness to pay for the telepharmacy service. Although this aspect is outside the scope of this study, it may provide an idea of the importance given to telepharmacy by the surveyed population.39,40 Although the cost of PTFU via telepharmacy is not known, we believe that, in a universal-access public health system like the Spanish one, a significant percentage of patients would be willing to pay for these services, although willingness is likely to be lower among unemployed patients. This positive attitude, stronger than observed by other authors,41 would appear to consolidate telepharmacy as a complementary PTFU procedure. Furthermore, given the dearth of information on patients’ willingness to pay for pharmaceutical procedures in general and for telepharmacy in particular, we consider that these aspects should be the subject of future pharmacoeconomic research.42,43

One of the limitations of this study is that a preliminary test should have been conducted to explore the extent to which respondents could understand, mentally process, and respond to the different items in the questionnaire. However, the validity and reliability tests carried out, together with the exclusion of duplicate items and the identification of the different domains, allowed the questionnaire to be properly validated for the Spanish population. In addition, the survey may be used as a model for other studies aimed at gaining greater knowledge about the perception of telepharmacy by other patient populations. The unequivocal strength of this study is that it is the first study worldwide to evaluate the perception of outpatients (expressed through their experience and opinion) of the efficiency of telepharmacy as a tool for hospital pharmacists to provide remote PTFU. The study provides a framework for future comparative studies on how PTFU is provided across different clinical domains.

Future research in this area may provide valuable insights into the influence of telepharmacy on health outcomes. It would also be useful to investigate the pharmacoeconomic aspect of the equation, both from the patients’ and the health system’s standpoint.


The ENOPEX questionnaire was validated as a tool to assess outpatients’ perception of the telepharmacy services provided by Spanish HPDs. Given their very encouraging experience with telepharmacy and their positive opinion on the service, outpatients consider telepharmacy a complementary procedure to face-to-face PTFU. Also, patients consider telepharmacy an especially useful tool to obtain information about their pharmacological treatment. In addition, no confidentiality concerns were detected in the procedures carried out. On the other hand, time under PTFU and distance from home to hospital were found to be independent variables with an impact on the patients’ perception of telepharmacy. The authors consider that future research should evaluate, in addition to these patients reported experiences, aspects related to the efficiency of telepharmacy and its influence on patients healthcare outcomes.


CMO, capacity, motivation and opportunity; HPDs, hospital pharmacy departments; KMO, Kaiser–Meyer–Olkin; PC, pharmaceutical care; PTFU, pharmacotherapeutic follow-up; SEFH, Spanish Society of Hospital Pharmacists.


The Spanish Society of Hospital Pharmacists has acted as sponsor of the ENOPEX project.

The authors would like to thank the hospital pharmacists from all over Spain who participated in the Enopex Project. We would particularly like to thank all Spanish hospital pharmacists for the enormous sacrifices made during the COVID-19 pandemic to ensure the normal operation of pharmacy departments and adapt to the extremely stressful situations resulting from the health crisis.

Finally, a special thank you goes to Emilio Garcia Cabrera from Delos Clinical CRO for his contribution to this article and the technical support he provided for this research project.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.


This study did not benefit from any specific grants from any funding agencies in the public, commercial, or not-for-profit sectors.


Dr Garbiñe Lizeaga reports personal fees from AbbVie, personal fees from Clovis, personal fees from Novartis, personal fees from GSK, personal fees from BMS, outside the submitted work. The authors have reported no other conflict of interest in connection with this study.


1. Ministry of Health, Social Services and Equality. Government of Spain. Royal Legislative Decree 1/2015, of July 24, approving the revised text of the Law on guarantees and rational use of medicines and health products. “BOE” no. 177, of July 25, 2015. Reference: BOE-A-2015-8343. Available from: Accessed December 17, 2021.

2. Margusino-Framiñán L, Cid-Silva P, Martínez-Roca C, et al. Implementation of Specialized Pharmaceutical Care Hospital Outpatient Clinics in a Hospital Pharmacy Department. Farm Hosp. 2017;41:660–666. doi:10.7399/fh.10771

3. Spanish Society of Hospital Pharmacy. Outpatient Pharmaceutical Care Strategic Map. MAPEX project. Available from: Accessed December 17, 2021.

4. Morillo-Verdugo R, Calleja-Hernández MÁ, Robustillo-Cortés MLA, Poveda-Andrés JL. A new definition and refocus of pharmaceutical care: the Barbate Document. Farm Hosp. 2020;44:158–162. doi:10.7399/fh.11389

5. World Health Organization. WHO Library. Telemedicine. Opportunities and Developments in Members States. Geneva (Switzerland); 2010. Available from: Accessed December 17, 2021.

6. Alexander E, Butler CD, Darr A, et al. ASHP Statement on Telepharmacy. Am J Health Syst Pharm. 2017:74236–74241. doi:10.2146/ajhp170039

7. US National Association of Boards of Pharmacy. Telepharmacy: the new frontier of patient care and professional practice. Mount Prospect (Illinois, EEUU); 2017. Available from: Accessed December 17, 2021.

8. Canadian Society of Hospital Pharmacist. Telepharmacy Guidelines. Ottawa (Canada); 2017. Available from: Accessed December 17, 2021.

9. Morillo-Verdugo R, Margusino-Framiñán L, Monte-Boquet E, et al. Spanish Society of Hospital Pharmacy Position Statement on Telepharmacy: recommendations for its implementation and development. Farm Hosp. 2020;44:174–181. doi:10.7399/fh.11515

10. Order SND/293/2020, of March 25, which establishes conditions for the dispensing and administration of medicines within the scope of the National Health System, in the face of the health crisis caused by COVID-19. Official State Gazette No. 85 (March 27, 2020). Available at: Accessed December 17, 2021.

11. Margusino-Framiñán L, Illarro-Uranga A, Lorenzo-Lorenzo K, et al. Pharmaceutical care to hospital outpatients during the COVID-19 pandemic. Telepharmacy. Farm Hosp. 2020;44:61–65. doi:10.7399/fh.11498

12. Tortajada-Goitia B, Morillo-Verdugo R, Margusino-Framiñán L, et al. Survey on the situation of telepharmacy as applied to the outpatient care in hospital pharmacy departments in Spain during the COVID-19 pandemic. Farm Hosp. 2020;44:135–140. doi:10.7399/fh.11527

13. Stephane Champely. pwr: basic Functions for Power Analysis. R package version 1.3-0; 2020. Accessed December 17, 2021.

14. R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing; 2020. Available from: Accessed December 17, 2021.

15. Congiusta S, Solomon P, Conigliaro J, et al. Clinical Quality and Patient Experience in the Adult Ambulatory Setting. Am J Med Qual. 2019;34:87–91. doi:10.1177/1062860618777878

16. Kaźmierczak D, Bogusz-Czerniewicz M. Identification of patient’s requirements in quality management system in health care institutions. Rep Pract Oncol Radiother. 2011;17(1):50–53. doi:10.1016/j.rpor.2011.10.006

17. Rodríguez-González CG, Sarobe-González C, Durán-García ME, et al. Use of the EFQM excellence model to improve hospital pharmacy performance. Res Social Adm Pharm. 2020;16:710–716. doi:10.1016/j.sapharm.2019.08.030

18. The Joint Commission. Evaluating Patient’s Perception of Care and Program Effectiveness – options. Available from:—comprehensive-stroke/performance-measurement-dspm/000001611/. Accessed December 23, 2021.

19. Kluetz PG, O’Connor DJ, Soltys K. Incorporating the patient experience into regulatory decision making in the USA, Europe, and Canada. Lancet Oncol. 2018;19:e267–e274. doi:10.1016/S1470-2045(18)30097-4

20. Kuehn CM. Patient Experience Data in US Food and Drug Administration (FDA) Regulatory Decision Making: a Policy Process Perspective. Ther Innov Regul Sci. 2018;52:661–668. doi:10.1177/2168479017753390

21. Bull C, Byrnes J, Hettiarachchi R, et al. A systematic review of the validity and reliability of patient-reported experience measures. Health Serv Res. 2019;54:1023–1035. doi:10.1111/1475-6773.13187

22. DeRemer CE, Reiter J, Olson JL. Transitioning ambulatory care pharmacy services to telemedicine while maintaining multidisciplinary collaborations. Am J Health Syst Pharm. 2021;78:371–375. doi:10.1093/ajhp/zxaa427

23. Niznik JD, He H, Kane-Gill SL. Impact of clinical pharmacist services delivered via telemedicine in the outpatient or ambulatory care setting: a systematic review. Res Social Adm Pharm. 2018;14:707–717. doi:10.1016/j.sapharm.2017.10.011

24. Poudel A, Nissen LM. Telepharmacy: a pharmacist’s perspective on the clinical benefits and challenges. Integr Pharm Res Pract. 2016;5:75–82. doi:10.2147/IPRP.S101685

25. Pedersen CA, Schneider PJ, Ganio MC, Scheckelhoff DJ. ASHP national survey of pharmacy practice in hospital settings: Dispensing and administration-2020. Am J Health Syst Pharm. 2021;78:1074-1093. doi:10.1093/ajhp/zxab120

26. Le T, Toscani M, Colaizzi J. Telepharmacy: a New Paradigm for Our Profession. J Pharm Pract. 2020;33:176–182. doi:10.1177/0897190018791060

27. Margusino-Framiñán L, Cid-Silva P, Castro-Iglesias Á, et al. Teleconsultation for the Pharmaceutical Care of HIV Outpatients in Receipt of Home Antiretrovirals Delivery: clinical, Economic, and Patient-Perceived Quality Analysis. Telemed J E Health. 2019;25:399–406. doi:10.1089/tmj.2018.0041

28. Rivas MD, Medina RM, Redondo C, et al. Telepharmacy during SARS-COV-2. Eur J Hosp Pharm. 2021;28(Suppl 1):A1–A184.

29. Erku DA, Belachew SA, Abrha S, et al. When fear and misinformation go viral: pharmacists’ role in deterring medication misinformation during the ‘infodemic’ surrounding COVID-19. Res Social Adm Pharm. 2021;17:1954–1963. doi:10.1016/j.sapharm.2020.04.032

30. Elson EC, Oermann C, Duehlmeyer S, et al. Use of telemedicine to provide clinical pharmacy services during the SARS-CoV-2 pandemic. Am J Health Syst Pharm. 2020;77:1005–1006. doi:10.1093/ajhp/zxaa112

31. Segal EM, Alwan L, Pitney C, et al. Establishing clinical pharmacist telehealth services during the COVID-19 pandemic. Am J Health Syst Pharm. 2020;77:1403–1408. doi:10.1093/ajhp/zxaa184

32. Ameri A, Salmanizadeh F, Bahaadinbeigy K. Tele-pharmacy: a new opportunity for consultation during the COVID-19 pandemic. Health Policy Technol. 2020;9:281–282. doi:10.1016/j.hlpt.2020.06.005

33. Li H, Zheng S, Liu W, et al. Fighting against COVID-19: innovative strategies for clinical pharmacist. Res Social Adm Phar. 2021;17:1813–1818. doi:10.1016/j.sapharm.2020.04.003

34. Nittari G, Khuman R, Baldoni S, et al. Telemedicine Practice: review of the Current Ethical and Legal Challenges. Telemed J E Health. 2020;26:1427–1437. doi:10.1089/tmj.2019.0158

35. Angaran DM. Telemedicine and telepharmacy: current status and future implications. Am J Health Syst Pharm. 1999;56:1405–1426. doi:10.1093/ajhp/56.14.1405

36. Daniel H, Sulmasy LS. Health and Public Policy Committee of the American College of Physicians; Policy recommendations to guide the use of telemedicine in primary care settings: an American College of Physicians position paper. Ann Intern Med. 2015;163:787–789. doi:10.7326/M15-0498

37. Martin RD. Levering telecommuting pharmacists in the post-COVID-19 world. J Am Pharm Assoc (2003). 2020;60:e113–e115. doi:10.1016/j.japh.2020.07.026

38. Dorsey ER, Okun MS, Bloem BR. Care, Convenience, Comfort, Confidentiality, and Contagion: the 5 C’s that Will Shape the Future of Telemedicine. J Parkinsons Dis. 2020;10:893–897. doi:10.3233/JPD-202109.

39. Hirth RA, Chernew ME, Miller E, et al. Willingness to pay for a quality-adjusted life year: in search of a standard. Med Decis Making. 2000;20(3):332–342. doi:10.1177/0272989X0002000310

40. King JT, Tsevat J, Lave JR, et al. Willingness to pay for a quality-adjusted life year: implications for societal health care resource allocation. Med Decis Making. 2005;25:667–677. doi:10.1177/0272989X05282640.

41. Yap KY, Low HX, Koh KS, et al. A Feasibility and acceptance of a pharmacist-run tele-oncology service for chemotherapy-induced nausea and vomiting in ambulatory cancer patients. Telemed J E Health. 2013;19:387–395. doi:10.1089/tmj.2012.0136

42. Vo AT, Gustafson DL. Telepharmacy in oncology care: a scoping review. J Telemed Telecare. 2020;1357633X20975257. doi:10.1177/1357633X20975257

43. Painter JT, Gressler L, Kathe N, et al. Consumer willingness to pay for pharmacy services: an updated review of the literature. Res Social Adm Pharm. 2018;14:1091–1105. doi:10.1016/j.sapharm.2018.01.010

Creative Commons License © 2021 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.