Back to Journals » Clinical Ophthalmology » Volume 16

Using a Facebook Virtual Examination Room to Enhance ROP Training in Armenia

Authors Sikder A , Mehta P, Galoyan T , Parekh E , Dickhoner J, Yeghiazaryan N , Harutyunyan R, Hovhannisyan T, Lee T, Espinoza J

Received 10 May 2022

Accepted for publication 13 July 2022

Published 23 July 2022 Volume 2022:16 Pages 2311—2322

DOI https://doi.org/10.2147/OPTH.S371360

Checked for plagiarism Yes

Review by Single anonymous peer review

Peer reviewer comments 2

Editor who approved publication: Dr Scott Fraser



Abu Sikder,1 Preeya Mehta,2 Tamara Galoyan,3 Eeshika Parekh,1 James Dickhoner,1 Nune Yeghiazaryan,4 Ruzanna Harutyunyan,4 Tadevos Hovhannisyan,4 Thomas Lee,5 Juan Espinoza6

1Innovation Studio, Children’s Hospital Los Angeles, Los Angeles, CA, USA; 2Keck School of Medicine, University of Southern California, Los Angeles, CA, USA; 3School of Education, Drexel University, Philadelphia, PA, USA; 4Armenian EyeCare Project, Yerevan, Armenia; 5Department of Surgery, Children’s Hospital Los Angeles, Los Angeles, CA, USA; 6Division of General Pediatrics, Children’s Hospital Los Angeles, Los Angeles, CA, USA

Correspondence: Juan Espinoza, Division of General Pediatrics, Children’s Hospital Los Angeles, 4650 Sunset Blvd., Mailstop #76, Los Angeles, CA, 90027, USA, Tel +1 323.361.2721, Fax +1 323.361.4429, Email [email protected]

Purpose: Retinopathy of prematurity (ROP) is a leading cause of worldwide childhood blindness with increasing incidence in low and middle income countries (LMICs) due to advances in neonatal care. There are insufficient numbers of healthcare professionals specialized in ROP management and few local training opportunities in LMICs. Social media platforms provide a promising solution to enable interactive medical education across geographic and logistic barriers. As an adjunct to an ROP training program for ophthalmologists in Armenia, we implemented a Facebook Virtual Examination Room (VER) for case discussion with a global community of collaborators and preceptors. To evaluate training through VER, we operationalized engagement as a multilayer meta-construct that includes cognitive, behavioral, and social-emotional domains.
Methods: A concurrent mixed methods approach was taken to collect and analyze data from comments and activities within VER. Quantitative data was analyzed for descriptive statistics on group utilization, participant activity, and clinical metrics. Qualitative data was analyzed by conducting thematic analysis involving initial and pattern coding.
Results: Ten participants (7 trainees and 3 preceptors) interacted in the Facebook group across 153 unique cases, with 218 threads, 414 comments, and 216 likes. Of the 120 unique cases eligible for consensus evaluation, treatment was mentioned most frequently (87), followed by location (68), staging (65), and plus disease (31), with agreement ranging from 67% to 97%. Based on the qualitative analysis, the most common themes of discussion were clinical management, dilemma resolution, knowledge growth, and gratitude.
Conclusion: A closed Facebook group for case-based discussions can be a useful adjunct to an existing ROP training program by engaging learners across social-emotional, behavioral, and cognitive domains. For international training partnerships, the financial and logistical advantages can be significant, though focus should remain on the primary curriculum and training modality.

Keywords: ROP, medical education, social media, global health

Introduction

Retinopathy of prematurity (ROP) is a vasoproliferative retinal disease which affects premature infants and is the leading cause of childhood blindness around the world.1 In low and middle income countries (LMICs), where increasing numbers of newborns are surviving infancy and prematurity due to advances in neonatal care, the incidence of ROP has been increasing.2 This problem is compounded by insufficient healthcare professionals specialized in ROP management and few local training opportunities.3 International training partnerships show promising results in addressing workforce shortages, but scalability has been limited by factors such as cost, time, and logistics.4 Technology has been successfully implemented to enhance training paradigms for screening, diagnosis, and treatment.3,5 In particular, knowledge sharing networks and social media have been leveraged innovatively to enhance educational discourse, consultation, and collaboration internationally.6–9

In 2010, Children’s Hospital Los Angeles (CHLA) partnered with the Armenian EyeCare Project (AECP), a non-governmental organization (NGO), to address the pediatric eye care needs of the country. With the rise in ROP, a multi-institutional collaborative effort was initiated to manage this condition, and training ophthalmologists in screening, diagnosing, and treating ROP became an important aim of the initiative. We used a number of technologies to help address some of the logistic and financial barriers to our international training partnership, such as implementing live streaming of the surgical microscope video feed to allow international ROP experts to provide real-time guidance and feedback during surgery.10,11 To provide additional asynchronous support to the trainees in Armenia, we created the Virtual Examination Room (VER), a closed Facebook group for trainees to post and discuss cases with a global community of collaborators and preceptors.

Many studies involving medical education through social media have considered engagement as an evaluation parameter.12–14 From a theoretical perspective, engagement is a multidimensional construct that involves active cognitive processes, such as decision-making, problem-solving, and evaluation, as well as successful collaboration with team members within a learning environment.15 Engagement frameworks offer a unique conceptual approach to better understand learning through emotional, social, cognitive, and behavioral lenses.16 In alignment with the existing views, we operationalized engagement as a multilayer meta-construct that includes cognitive, behavioral, and social-emotional domains. Cognitive domain refers to thinking about the applicable content and context of learning such as participants reflecting on their own learning and the learning content. Behavioral domain refers to observable features of participation such as the number and kinds of tasks and activities within a project. Social-emotional domain refers to expressions of feelings about specific activities or other people, as well as interactions with other participants.

This study explored the utilization and engagement of the VER by our global community of ophthalmologists. Here, we present our design, implementation, and findings in using the VER as an adjunct to the ROP training program for ophthalmologists in Armenia.

Methods

ROP Training Program

The ROP training program in Armenia began in 2010 as a partnership between the AECP and CHLA.10 The program includes in-person and remote didactic lectures, practical training in the operating room, visiting observerships for Armenian ophthalmologists in the US and UK, sharing of wide-angle images using a RetCam (Clarity Medical Systems, Pleasanton, CA) to support diagnosis and treatment decisions, and live surgical teleproctoring.10,11 The VER was added as an adjunct to the training program to create a forum in which Armenian ophthalmologists could discuss cases asynchronously with international colleagues. The patients of the program were newborns with ROP at one of seven NICUs in Armenia – six from Yerevan and one in Gyumri. ROP screening began as a pilot program based on national guidelines developed by a multi-stakeholder coalition led by the AECP, which were later validated and verified by the Armenian Ministry of Health, and is now mandatory national health policy.

Research Design

This retrospective study used a mixed methods approach to explore participant engagement within VER. Mixed methods studies allow researchers to combine both qualitative and quantitative data collection, analysis, and inference techniques for reaching deeper understanding and corroboration.17 Specifically, we applied a convergent parallel mixed methods design which allows for concurrent collection and analysis of quantitative and qualitative data.18,19 Convergent designs often help to obtain complementary data on the same topic. For our three domain engagement framework, the behavioral domain was explored through quantitative analysis of user participation, and the cognitive and social-emotional domains were explored through qualitative analysis of text.

Participants

The clinical staff involved 6 groups: professionals, non-professionals, trainees, mentors, researchers, and administrators. These groups can be organized into 3 main categories: preceptors, trainees, and observers (administrators, Armenian clinicians not in the training program, researchers, and staff members). The trainees were from Armenia, while the preceptors were based in the US or UK. The preceptors were ophthalmology specialists trained in ROP. The trainees were all ophthalmologists participating in the AECP ROP training program.

Virtual Examination Room - Facebook Case Reviews

The VER was created by AECP staff as a closed Facebook group requiring permission to join. The group consisted of ROP specialists (preceptors), ophthalmologists-in-training (trainees), researchers, and administrators.

For the purposes of this paper we use the following terms:

A Case is a unique patient being discussed in the VER.

A Post is the first Facebook posting that introduces a case.

A Comment is a reply to a post or another comment.

A Thread is a conversation between two or more users that begins with a post and is followed by one or more comments. A given case may be discussed across multiple threads.

A Like is the use of Facebook’s native features to indicate that a user agrees with or likes a post or comment.

All members had the ability to like posts and comments. Generally, wide-angle images were posted by trainees along with gestational age, current age, and weight. Both preceptors and trainees commented on the suspected ROP stage, zone, and the presence of plus disease. Additionally, treatment options such as medications and surgical procedures (such as laser and vitrectomy) were discussed by both groups.

Data Extraction and Analysis

Quantitative Data Analysis

Descriptive statistics were used for analyzing group utilization, participant activity, and clinical characteristics.

Programmatic and Manual Extraction of Facebook Data

To extract data from Facebook, a python script was developed to extract and parse the HTML data (see Appendix 1 for code). The appropriate HTML fields were parsed to organize Facebook data into csvs (comma separated values). Figure 1 contains a diagram of the methodology. One csv file aggregated numeric metrics such as likes, number of comments, and group metrics. Another csv file contained purely textual data from posts and comments. These data files were subsequently used for quantitative and qualitative analysis. The Facebook HTML data was also used to manually extract clinical characteristics into a csv file which contained both numbers and text.

Figure 1 Overview of how Facebook HTML data is extracted and parsed into Excel as a CSV file. (A) First, Facebook data is downloaded as an HTML file. (B) Key variables of the post are selected for analysis. (C) Using a python script, the corresponding tags are used to parse the specified data. (D) The parsed data is then formatted as a CSV file compatible with Excel.

Virtual Examination Room Metrics

The Facebook data collected include group utilization, participant activity, and clinical characteristics. Group utilization metrics include the number of trainees, preceptors, and observers, in addition to the total, average, and range for counting metrics involving posts, threads, likes, comments, views, and commenting period. Participant activity metrics include counting metrics on specific individuals such as preceptors, trainees, and observers. Clinical data was extracted by reviewing the text of the posts, and includes ROP zone, stage, presence of plus disease, gestational age, current age, and treatment options such as medications, laser, or other related surgeries.

To evaluate consensus, each post and comment was read for agreement. Agreement was defined as either stating agreement in a comment (eg, “I agree”, or “I think so, too”), or by liking. If the original poster (OP) gave a range (eg Stage 2–3), then the commenter was considered to agree if they gave the same range, or picked a single value within that range (in the example above, choosing stage 3). Giving a different range (eg Stage 3–4), was considered a disagreement. The priorities for achieving consensus were:

  1. Team agreement.
  2. Agreement between preceptor and trainee.
  3. Agreement between observer and trainee.
  4. Agreement between trainee and trainee.

Qualitative Data Analysis

Qualitative data was analyzed independently by authors JD, TG, PM, and EP for themes and tone using Excel. Qualitative text data extracted from comments by VER trainees (7), preceptors (3), and observers (6) were analyzed using thematic analysis technique. The analysis steps aligned with the general guidelines for qualitative data analysis.20 We applied two levels of coding. Level One coding involved open coding where a code was assigned to a specific segment of the data.21 This allowed for the generation of the initial codebook. Level Two coding involved pattern coding where initial codes were clustered into themes and broader categories. To eliminate researcher bias in the interpretation process, coding was conducted independently by three researchers of this study and then calibrated during multiple discussion sessions. Any discrepancy was discussed thoroughly until consensus was reached.

Regulatory Statement

This research was approved by the CHLA Institutional Review Board (IRB) and data was managed according to regulation and guidelines implemented by the Armenian regulatory bodies.

Results

Quantitative Results

Group Utilization Metrics

There were 82 participants in the VER: 3 preceptors, 7 trainees, and 72 observers (Table 1). Over a period of 6 years, there were 153 unique cases and 218 posts with an average of 1 like, 2 comments, and 3 views per post. On average, 54 minutes elapsed for the first comment to appear for each post and 87 minutes for the last.

Table 1 Facebook Group Utilization Metrics

Participant Activity Metrics

Trainees initiated 215 threads for 153 unique cases. They liked 71 posts and made 140 comments. Preceptors did not create any cases or threads, but made 95 comments and liked 2 posts. Observers made 3 posts for 3 unique cases along with 179 comments and 143 likes. Table 2 shows individual participant activity metrics, and Table 3 shows how participants interacted with each other across threads. There were 120 out of 153 (78%) unique cases that involved 2 or more participants; the remaining 33 (23%) cases were solo posts initiated by trainees but did not receive any replies.

Table 2 Individual Participant Utilization

Table 3 Participant Activity on Threads and Cases

Clinical Characteristics

Across 153 unique cases, the average birthweight was 1246.59 ± 346 grams, and the average gestational age was 29.08 ± 2.35 weeks. Figure 2 summarizes the information presented in the case discussions regarding location, staging, plus disease, and treatment options.

Figure 2 Summary of clinical descriptors of the 153 cases discussed in the Virtual Examination Room, including (A) location, (B) staging, (C) plus disease, and (D) treatment options discussed.

Consensus

Of the 153 unique cases, 120 included 2 or more participants and thus were eligible for consensus evaluation, shown in Figure 3. For each domain (location, staging, plus disease, and treatment), the number of times that domain was mentioned in the case is shown, followed by the number of times it was discussed (meaning another participant engaged with the concept, eg “agree zone 2” or “I would consider Avastin first”), and finally among those cases in which it was discussed, how many of those achieve consensus, as defined in the methods. Relative percentages are shown next to the bar graphs. Treatment was mentioned most frequently (87 out of 120) and achieved consensus in almost all discussions (97%). More than one treatment option was discussed in 5 out of 35 cases. Plus disease was discussed least frequently, in only 6 out of 31 cases. Overall, consensus ranged from 67% to 97% of discussed cases for each domain.

Figure 3 Case discussion achieving consensus across 4 domains: (A) location, (B) staging, (C) plus disease, and (D) treatment options discussed. 33 cases did not have any comments, and are not included in this figure. Each bar represents the number of times a domain was mentioned in a case, followed by the number of times another participant engaged with the domain, and finally the cases which achieved consensus. Relative percentages are shown next to the bar graphs. *Cases: included 2 or more participants and thus were eligible for consensus evaluation.

Qualitative Findings

The analysis of the qualitative data collected from the VER trainee participant comments created multiple themes that were grouped into two broad categories related to the participants’ engagement within the VER training. Those categories were (1) Cognitive Engagement, that included themes related to the VER learning content; and (2) Social-Emotional Engagement, that included themes related to the expressions of feelings and peer-to-peer /group interactions within the VER. Table 4 below provides information on the two broad categories, the related themes, and exemplar quotes from the participant comments.

Table 4 Thematic Analysis of Qualitative Data on VER Trainee Engagement

Discussion

The purpose of this study was to understand how Armenian ophthalmologists participating in an ROP training program used and engaged with the adjunct VER program. We situated our findings within an engagement framework which included the behavioral, social-emotional, and cognitive dimensions.16 By using a mixed methods design, we gained a complementary understanding of participant engagement as our qualitative findings were supported by our quantitative results. To our knowledge, this is the first study to use a closed Facebook group to supplement an ROP training program internationally.

The VER case discussion usually began within the hour (54 minutes). Trainees initiated almost all the threads (215 out of 218) and made 34% of all comments. This result was expected since the training program was aimed specifically at helping address questions and concerns brought to attention by trainees. Most of the case discussions (108 of 153) involved conversation with at least 1 trainee and a mentor or experienced professional. These trends show evidence of behavioral engagement due to active participation by group members, especially trainees. In addition, our qualitative findings show cognitive and social-emotional engagement occurring within the VER. For example, after a trainee provided retinal images and preliminary information for a case, one of the preceptors commented, “Either laser or avastin would be appropriate … treat within the next several days”, to which the trainee responded, “I considered to do avastin because zone 1, will perform tomorrow [sic] and let you [sic] know. Thanks”. This was a common example of how preceptors provided diagnostic and treatment options to trainees which highlighted cognitive engagement. Trainees were not only actively thinking about ROP content but also evaluating their clinical conclusions in context with preceptor recommendations. Furthermore, participants also showed evidence of social-emotional engagement by expressing gratitude, providing encouragement, and interacting with peers; all of which are important components of clinical training.22 Although few cases met our criteria for consensus evaluation, among those that did, diagnostic and treatment consensus was frequently achieved (67–97%). In some instances, we could see how engagement between the trainees and preceptors worked to improve patient care and clinical results. For example, in one case where the trainee posted follow-up wide-angle images, the preceptor noticed improvement and responded, “This looks much better. There is no traction so I would watch carefully and not do surgery at this time. This child needs to be seen every week … ” (Figure 4).

Figure 4 A case discussion within the Facebook group involving a preceptor and trainee.

Although research is limited, social media platforms have been used as an informal supplement to graduate-level medical education.23,24 Several studies have reviewed how online social media platforms can increase engagement through a cognitive, social-emotional, and behavioral lens. Physicians have joined various online communities to read new articles, listen to experts, and consult colleagues on patient issues.25 A recent paper studied the benefits of a Facebook group comprised of women physicians trained in physical medicine and rehabilitation (PM&R).25 The majority of respondents (63%) found that learning from posted cases and discussion and advice on complex patients was the most beneficial to them.25 While Facebook has been primarily used for recruitment, content dissemination, and marketing in ophthalmology,24,26 other fields have leveraged the social media platform for training in both undergraduate and graduate medical education.7,27 For example, a study using a closed Facebook group to discuss surgical cases, showed how participants (surgeons, residents, fellows, and medical students) can use this platform to cognitively engage with clinical content across 40 countries.7 One concern with larger, more open groups is the provenance and reliability of information, ideas, or recommendations shared.7 It may not be feasible to vet each member’s expertise or verify comments for accuracy, so active moderation, trust building, transparency, and healthy skepticism are likely important components of this approach. Social-emotional engagement has also been a valuable part of these communities. Almost 15% of the individuals in a PM&R Facebook group found emotional support and mentorship to be a valuable facet of the group. Those at earlier stages in their careers mentioned an increase in empowerment, and 75% of the group believed that the Facebook community reduced their professional isolation.25 In our study, we found that trainees consistently expressed gratitude to each other and to the mentors who helped answer their clinical questions, highlighting the support that was being provided in this environment. Posting comments is not the only interaction modality on Facebook. In a study reviewing the International Hernia Collaboration (IHC), Ghanem et al found that engagement was significantly higher in Facebook Live Q&A sessions compared to posts, and that participants often continued to discuss lecture content until the conversation naturally concluded.28 Other studies have shown that participants preferred engaging with content that includes videos and pictures rather than text alone.7,29,30 Overall, our findings are consistent with existing evidence that social media is a positive contributor to medical education in terms of engagement. It allows physicians to actively engage with meaningful content, feel supported by a clinical community, and leverage a convenient platform which allows for organic conversations without geographic and economic barriers.

Our study benefited from a mixed-methods design, through which we were able to extrapolate a more nuanced understanding of the VER by placing our quantitative results in context through our qualitative analysis of the comments. Additionally, using the engagement framework helped orient the qualitative findings by focusing on socially constructed dimensions of training. Our qualitative analysis was conducted by 4 different individuals to improve reliability and reduce bias. The approach itself was very simple, requiring almost no resources or coordination compared to international travel. The study period spans over 6 years, demonstrating the durability of the solution. Finally, despite occurring before the COVID-19 pandemic, this study provides evidence for the utility of virtual education platforms which are currently of great interest. Academic programs may benefit from using virtual technology and social media to provide remote education and promote educational engagement.

There are a number of limitations to this study worth noting. As a retrospective study, there was no way for us to verify or clarify any of the information found in the VER. Trainees were not asked to follow a specific format when posting cases, and so many cases had missing data. The lack of a pre-specified case posting format and resulting missing data may explain the lack of discussion of some of the more subjective elements of ROP diagnosis and staging, such as plus disease. Preceptors were also not provided instructions on following a specific format in their responses, resulting in many cases not being able to be evaluated for consensus because preceptors did not comment on all aspects of a case. We did not compare VER case discussions to traditional in-person discussions. It is likely that there are aspects of ROP training that cannot be replicated virtually; as an adjunct to an existing training program, it is important that the VER complement and reinforce the in-person components. The qualitative findings could have been influenced by observational bias, cultural bias, groupthink, bandwagon effect, and impression management due to working with international colleagues.7 The majority of the data gathered and presented here corresponds to attitudes and perceptions, the first level of Kirkpatrick’s hierarchy for evaluating educational programs.31,32 The VER program did not formally evaluate knowledge change (level 2), and though a few posts included anecdotes of how the trainees changed their treatment plan (level 3), or even included updates on the patients’ outcomes (level 4), there was insufficient data to meaningfully evaluate these concepts.31,32 Finally, technology is constantly changing, and as the features of Facebook evolve and the popularity of other platforms waxes and wanes, our specific approach could become less reproducible in the future. For researchers interested in recreating or adapting the VER, changes could be made to improve the methodology. Creating posting guidelines for trainees and preceptors, collecting demographics and outcomes data, a larger sample size, and a comparison to other modalities would all improve upon the current study.

Conclusion

The increasing number of infants with ROP in LMICs has created an urgent need for specialized training. Technology can be used to support training and capacity building through synchronous and asynchronous educational programs. Social media platforms provide a cost effective, geographically agnostic, and convenient medium for medical education and clinical communication. A closed Facebook group for case-based discussions can be a useful adjunct to an existing ROP training program by engaging learners across social-emotional, behavioral, and cognitive domains. For international training partnerships, the financial and logistical advantages can be significant, though focus should remain on the primary curriculum and training modality.

Acknowledgments

The authors would like to thank Dr. Alexander de Castro-Abeger and Lynn Kysh for their valuable insights and recommendations. The authors also thank all the members of the OPEN/Avetis Team at Children’s Hospital Los Angeles for their camaraderie and support and to the Armenian EyeCare Project team members who helped make this project possible. Finally, we would like to thank all of the trainees, preceptors, and staff who participated in the VER program and made this work possible.

Funding

This project was funded by an anonymous philanthropic donor.

Disclosure

JE is a paid consultant for AI Health. AI Health played no role in the design, execution, analysis, or write up of this work. AI Health did not play a role in the decision to publish this manuscript and had no editorial input. Dr James Dickhoner holds equity in Orderly Health, outside the submitted work. The authors report no other conflicts of interest in this work.

References

1. Kim SJ, Port AD, Swan R, Campbell JP, Chan RVP, Chiang MF. Retinopathy of prematurity: a review of risk factors and their clinical significance. Surv Ophthalmol. 2018;63(5):618–637. doi:10.1016/j.survophthal.2018.04.002

2. Mora JS, Waite C, Gilbert CE, Breidenstein B, Sloper JJ. A worldwide survey of retinopathy of prematurity screening. Br J Ophthalmol. 2018;102(1):9–13. doi:10.1136/bjophthalmol-2017-310709

3. Patel SN, Martinez-Castellanos MA, Berrones-Medina D, et al. Assessment of a tele-education system to enhance retinopathy of prematurity training by international ophthalmologists-in-training in Mexico. Ophthalmology. 2017;124(7):953–961. doi:10.1016/j.ophtha.2017.02.014

4. Wagner JP, Schroeder AD, Espinoza JC, et al. Global outreach using a systematic, competency-based training paradigm for inguinal hernioplasty. JAMA Surg. 2017;152(1):66–73. doi:10.1001/jamasurg.2016.3323

5. Biten H, Redd TK, Moleta C, et al. Diagnostic accuracy of ophthalmoscopy vs telemedicine in examinations for retinopathy of prematurity. JAMA Ophthalmol. 2018;136(5):498–504. doi:10.1001/jamaophthalmol.2018.0649

6. McBain RK, Sousa JL, Rose AJ, et al. Impact of project ECHO models of medical tele-education: a systematic review. J Gen Intern Med. 2019;34(12):2842–2857. doi:10.1007/s11606-019-05291-1

7. Jackson HT, Young MT, Rodriguez HA, Wright AS. SAGES foregut surgery masters program: a surgeon’s social media resource for collaboration, education, and professional development. Surg Endosc. 2018;32(6):2800–2807. doi:10.1007/s00464-017-5983-1

8. Chan WS, Leung AY. Use of social network sites for communication among health professionals: systematic review. J Med Internet Res. 2018;20(3):e117. doi:10.2196/jmir.8382

9. Bittner JG, Logghe HJ, Kane ED, et al. A Society of Gastrointestinal and Endoscopic Surgeons (SAGES) statement on closed social media (Facebook®) groups for clinical education and consultation: issues of informed consent, patient privacy, and surgeon protection. Surg Endosc. 2019;33(1):1–7. doi:10.1007/s00464-018-6569-2

10. Retinal physician - using telemedicine to save more than sight in premature infants. retinal physician. Available from: https://www.retinalphysician.com/issues/2017/july-august-2017/using-telemedicine-to-save-more-than-just-sight-in. Accessed March 29, 2022.

11. Telementoring: how technology is helping to lower the impact of ROP in Armenia. Retina Today. Available from: https://retinatoday.com/articles/2015-may-june/telementoring-how-technology-is-helping-to-lower-the-impact-of-rop-in-armenia. Accessed March 29, 2022.

12. Sterling M, Leung P, Wright D, Bishop TF. The use of social media in graduate medical education: a systematic review. Acad Med J Assoc Am Med Coll. 2017;92(7):1043–1056. doi:10.1097/ACM.0000000000001617

13. Chan TM, Dzara K, Dimeo SP, Bhalerao A, Maggio LA. Social media in knowledge translation and education for physicians and trainees: a scoping review. Perspect Med Educ. 2020;9(1):20–30. doi:10.1007/s40037-019-00542-7

14. Katz M, Nandi N. Social media and medical education in the context of the COVID-19 pandemic: scoping review. JMIR Med Educ. 2021;7(2):e25892. doi:10.2196/25892

15. Kearsley G, Shneiderman B. Engagement theory: a framework for technology-based teaching and learning. Educ Technol. 1998;38(5):20–23.

16. Phillips TB, Ballard HL, Lewenstein BV, Bonney R. Engagement in science through citizen science: moving beyond data collection. Sci Educ. 2019;103(3):665–690. doi:10.1002/sce.21501

17. Johnson RB, Onwuegbuzie AJ, Turner LA. Toward a definition of mixed methods research. J Mix Methods Res. 2007;1(2):112–133. doi:10.1177/1558689806298224

18. Lee JT. Book Review: designing and conducting mixed methods research. HERD Health Environ Res Des J. 2019;12(1):164–166. doi:10.1177/1937586719832223

19. Morse J. Principles of mixed methods and multimethod research design. Handbook Mixed Meth Soc Behav Res. 2003;1:189–208.

20. Merriam SB, Tisdell EJ. Qualitative Research: A Guide to Design and Implementation. 4th ed. Wiley; 2022.

21. Patton MQ. Two decades of developments in qualitative inquiry: a personal, experiential perspective. Qual Soc Work. 2002;1(3):261–283. doi:10.1177/1473325002001003636

22. Roberts D. Learning in clinical practice: the importance of peers. Nurs Stand R Coll Nurs G B. 2008;23(12):35–41. doi:10.7748/ns2008.11.23.12.35.c6727

23. Men M, Fung SSM, Tsui E. What’s trending: a review of social media in ophthalmology. Curr Opin Ophthalmol. 2021;32(4):324–330. doi:10.1097/ICU.0000000000000772

24. Schmuter G, Tooley AA, Chen RWS, Law JC. Social media in ophthalmology: the educational and professional potential for medical students. J Acad Ophthalmol. 2020;12(1):e41–e45. doi:10.1055/s-0040-1709178

25. Capizzi AN, Larson AR, Silver JK, Verduzco-Gutierrez M. The women physiatrists Facebook group: evaluating the value of a private online community. Am J Phys Med Rehabil. 2021;100(10):996–1002. doi:10.1097/PHM.0000000000001680

26. Al-khersan H, Tanenbaum R, Lazzarini TA, Patel NA, Sridhar J. A characterization of ophthalmology residency program social media presence and activity. J Acad Ophthalmol. 2020;12(2):e110–e114. doi:10.1055/s-0040-1714682

27. Dedeilia A, Sotiropoulos MG, Hanrahan JG, Janga D, Dedeilias P, Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo. 2020;34(3 suppl):1603–1611. doi:10.21873/invivo.11950

28. Ghanem O, Logghe HJ, Tran BV, Huynh D, Jacob B. Closed FacebookTM groups and CME credit: a new format for continuing medical education. Surg Endosc. 2019;33(2):587–591. doi:10.1007/s00464-018-6376-9

29. Rajan D, Pillai VG, Varghese P. Educational utility of social media for laparoscopic surgery in India: a cross-sectional survey of popular Indian communities on Facebook. Adv Med Educ Pract. 2021;12:491–498. doi:10.2147/AMEP.S306680

30. Lima DL, Cordeiro RN, Carvalho GL, Malcher F. The influence of social media in minimally invasive surgery education: how surgeons exchange experience and knowledge in these platforms. J Minimal Access Surg. 2019;15(3):275–276. doi:10.4103/jmas.JMAS_270_18

31. Kurt S. Kirkpatrick model: four levels of learning evaluation. educational technology; 2016. Available from: https://educationaltechnology.net/kirkpatrick-model-four-levels-learning-evaluation/. Accessed March 29, 2022.

32. Kirkpatrick DL. The four levels of evaluation. In: Brown SM, Seidner CJ, editors. Evaluating Corporate Training: Models and Issues. Springer Netherlands; 1998:95–112. doi:10.1007/978-94-011-4850-4_5

Creative Commons License © 2022 The Author(s). This work is published and licensed by Dove Medical Press Limited. The full terms of this license are available at https://www.dovepress.com/terms.php and incorporate the Creative Commons Attribution - Non Commercial (unported, v3.0) License. By accessing the work you hereby accept the Terms. Non-commercial uses of the work are permitted without any further permission from Dove Medical Press Limited, provided the work is properly attributed. For permission for commercial use of this work, please see paragraphs 4.2 and 5 of our Terms.