Scholarly Impact of Academic Ophthalmologists and Vision Scientists in Canada

Introduction

Canada’s 15 academic ophthalmology departments serve numerous roles, including generating and disseminating new knowledge through research. Research in academic departments helps refine clinical practice, encourage analytical approaches in trainees, secure grant funding, promote career development and foster educational partnerships.^1–4

With an ever-changing academic landscape and resource availability in Canadian ophthalmology departments, it is important to objectively assess the impact of research in contemporary terms. Two studies published in 2009 and 2010 have previously evaluated the research impact of Canadian ophthalmology departments.^5,6 These studies gauged institutional research impact primarily using total number of papers, citations, and the total number of papers that departments published in top ophthalmology journals. However, there are several limitations to these metrics. The number of papers, for instance, does not account for their ultimate influence. The number of citations does not give insight towards an author’s consistency of publishing high impact studies over time. Further, the average career stage of institutional faculty is also a significant confounding factor not accounted for in this methodology, as newly graduated staff typically have less output compared to established, tenured faculty. Relying upon the number of articles published in clinical ophthalmology journals also omits important non-clinical research done by Canadian vision science programs, including basic and translational science research.

Recently, new research metrics have emerged that offer a more comprehensive assessment of research impact. The h-index has become widely accepted in academia due to its ability to account for both publication quantity and quality through citation count.⁷ Mathematically, the h-index is defined by the number of an author’s publications, h, that have been cited at least h times in the peer-reviewed literature.⁸ Thus, the h-index prioritizes frequently cited publications over the volume of papers in assessing an author’s impact. Another measure, the m-quotient, helps control for the career stage of a researcher by dividing an author’s h-index by the number of years since their first publication.⁸ In the ophthalmology literature, the h-index has been related to national research productivity,^9,10 National Institutes of Health (NIH) funding,¹ the academic impact of chairs and fellowship-trained ophthalmologists,^11,12 and sex differences among American ophthalmology departments.^13,14

This study aims to assess a number of questions around the current state of ophthalmology research in Canada: a) what is the scholarly impact of ophthalmology and vision science faculty members at academic Canadian centres, stratifying by institution, sex, academic appointment, ophthalmologic subspecialty, and educational background; b) what is the correlation of research impact to the size of grants received by the Canadian Institutes for Health Research (CIHR).

Methods

Demographic Data

Faculty lists of all 15 Canadian academic ophthalmology departments offering residency training programs were obtained from departmental websites. These lists were used to obtain the names, appointments, sex, degrees (MD and/or PhD) and subspecialties of each faculty member. While there was some heterogeneity around the titles of appointments from university to university, equivalent appointments were sorted into 3 groups: Assistant Professors (including lower ranks, such as lecturers and non-professorial associates and assistants), Associate Professors and (Full) Professors. Past and present Chairs of departments were also identified. Where departmental websites were last updated before January 1, 2020, or did not include these full details, the department was contacted directly.

Clinical Subspecialties

Subspecialties were chiefly categorized according to the major areas of clinical focus recognized at the Annual Meetings of the Canadian Ophthalmological Society: comprehensive ophthalmology/cataract surgery; cornea, external disease and refractive surgery; glaucoma; neuro-ophthalmology; oculoplastic and reconstructive surgery; ophthalmic pathology; paediatric ophthalmology and strabismus; retina (medical and surgical); uveitis; and vision rehabilitation. Ocular genetics and ocular oncology were also included. Subspecialties were documented for academic ophthalmologists only when they had completed a formal fellowship in that field, except vision rehabilitation. Faculty who had completed fellowships in two or more clinical areas were counted in the subgroup analyses of each of the subspecialties they trained in.

Vision Scientists

Vision scientists were defined as any faculty member engaging in ophthalmology or vision-related basic science and lab bench research. Basic science was defined as any abstract conducted in the realms of neuro-visual pathways, visual psychology, neuroscience and neurodevelopment, electrophysiology, cellular and molecular biology, pathobiology, pharmacology, genetics, chemistry, optics, biophysics, and bioengineering. Clinical ophthalmologists were dually counted as vision scientists if they also engaged in this form of research. Faculty lists often include cross-appointed, non-ophthalmologist MDs (eg, pathologists, neurologists, neurosurgeons, and plastic surgeons), non-PhD researchers, and other professionals (eg, optometrists and orthoptists) supporting ophthalmology-related research. These faculty members were included in a separate subgroup of vision scientists in consideration of the intellectual, physical, and financial resources they often bring to ophthalmology and vision science programs.

Research Metrics

Elsevier’s Scopus database (www.scopus.com, accessed July 24, 2020) was used to calculate and obtain research metrics for every Canadian academic ophthalmologist and vision scientist. Metrics included H-index, m-quotient, first year of publication and the total number of citations. Self-citations were excluded when tabulating H-index and total citations. When a researcher’s profile could not be easily identified in Scopus, two authors (MK, MN) cross-checked two other databases (PubMed and Google Scholar) in an effort to locate that faculty member’s online research profile. Data collection began in March 2020 and was completed in July 2020.

Funding Data

Federal grants awarded to Canadian academic ophthalmologists and vision scientists from 2008 to 2020 were obtained through the Canadian Institutes of Health Research (CIHR) Funding Decisions Database (https://cihr-irsc.gc.ca/e/38021.html, accessed August 21, 2020). The date, grant size (in CAD), principal investigator (PI), and recipient institution were recorded for each grant.

Statistical Analysis

Descriptive and comparative statistical analyses were performed using Prism 8 (GraphPad Software^®, La Jolla California, USA). Student’s t-tests were performed to compare non-parametric data between the two groups, and Kruskal–Wallis H-tests were performed to compare non-parametric data between multiple groups. Dunn’s Multiple Comparison Tests were used post-hoc where appropriate. As done by similar studies in the past, sex-based research metrics were corrected by academic rank using a two-tailed analysis of variance (ANOVA).^9,13 Pearson's coefficients were used for correlative tests. Statistical significance is defined as p <0.05.

Results

A total of 759 academic ophthalmologists and 120 vision scientists were identified from departmental faculty lists. Of these, 686 ophthalmologists (90.4%) and 119 vision scientists (99.2%) had online bibliometric profiles featuring an h-index and m-quotient. Average h-indices were 7.42 ± 7.98 (median 5.00, range 0–64.00) for academic ophthalmologists and 23.78 ± 15.25 (median 21.00, range 0–106.00) for vision scientists. Table 1 describes the mean h-index and m-quotient for ophthalmologists and vision scientists stratified by sex, degree, academic appointment, fellowship training and clinical subspecialty. Average H-indices were 7.89 ± 8.01 for male ophthalmologists and 6.21 ± 7.78 for female ophthalmologists. For vision scientists, the average H-indices were 24.87 ± for men and 21.86 ± 12.22 for women. These differences were not significant when correcting for academic rank (Figure 1A and B).

Table 1 Bibliometric Profiles of Ophthalmology and Vision Science Faculty in Canada

Figure 1 Average H-indices for ophthalmologists (A) and vision scientists (B) stratified by sex and rank.

Sex-Based Metrics

Table 2 describes the male-female distribution of ophthalmologists and vision scientists with bibliometric profiles in Canada by academic appointment and institution. On average, 72% of Canadian academic ophthalmologists were men and 28% were women. Université Laval and Université de Montréal had the most balanced faculties, with male:female ratios close to 1.

Table 2 Sex Distribution of Clinical Ophthalmologists and Vision Scientists with Bibliometric Profiles Appointed at Accredited Canadian Academic Institutions

Canada’s cohort of female ophthalmologists is at a younger career stage than males, indicated by their later average first year of publication compared to males (2002.52 ± 9.62 for women vs 1996.01 ± 12.07 for men, p < 0.001). This trend was also observed among vision scientists (1993.93 ± 9.61 for women vs 1989.63 ± 11.96 for men, p < 0.05). These results are summarized in Figure 2.

Figure 2 Average year of first publication for (A) ophthalmologists and (B) vision scientists, stratified by sex (***P < 0.001, *P < 0.05, Student’s t-test).

Academic Appointment

H-indices generally increased with academic appointment (Figure 3A and B). Similar trends were seen when comparing m-quotients between academic ranks of ophthalmologists and vision scientists (Figure 3C and D).

Figure 3 Average H-indices by rank for (A) ophthalmologists and (B) vision scientists; and average m-quotients by rank for (C) ophthalmologists and (D) vision scientists. Groups with the same letter were not significantly different from one another (p<0.05, Kruskal Wallis test).

Of Canada’s 28 past or present Chairs of Ophthalmology, 24 (86%) were male and 4 (14%) were female (Table 2). Eighteen (64%) were Professors, 7 (25%) were Associate Professors, and 3 (11%) were Assistant Professors. Twenty-three (82%) were MDs and 5 were MD/PhDs (18%).

Educational Background

For ophthalmologists, MD/PhDs had higher average H-indices, m-quotients, and citations than MDs alone (H-index means 17.33 ± 11.13 vs 6.95 ± 7.51, p < 0.001). Similarly, for vision scientists, PhDs and MD/PhDs had higher H-indices than MDs alone. H-indices between these groups are shown in Figure 4.

Figure 4 Average H-index by degree for (A) ophthalmologists and (B) vision scientists. (*** P<0.0001, Student’s t-test. Groups with the same letter were not significantly different from one another (p<0.05, Kruskal–Wallis test)).

There was also a statistically significant increase in H-index, m-quotient, and number of citations with increasing number of fellowships (p < 0.0001; Table 1).

In Canadian academic ophthalmology departments, the most common areas of focus included Comprehensive Ophthalmology (26%), Retina (18%), and Glaucoma and Cornea/External Disease (14% each). Figure 5A demonstrates the frequency of the various subspecialties in academic universities. The least represented specialties, including Ocular Genetics (1%), Ocular Oncology (3%), and Ocular Pathology (1%) had the highest H-indices (means 24.10 ± 15.24, 15.13 ± 17.38 and 10.22 ± 10.21, respectively). Average h-indices by subspecialty are highlighted in Figure 5B.

Figure 5 (A) The number of ophthalmologists per subspecialty in Canada, and (B) average H-indices of members in each subspeciality (***P < 0.001 against General/Comprehensive, Dunn’s Multiple Comparison Tests).

Institution

Average H-indices were analyzed by institution for clinicians and vision scientists separately (Figure 6). In clinical ophthalmology research, the top three departments with the highest average H-indices were Western University, the University of Toronto, and Dalhousie University. For basic science research, the University of British Columbia, McGill University, and the University of Toronto had the highest average H-indices. Figure 6 compares the impact of Canadian ophthalmologists and vision scientists against their American counterparts.

Figure 6 Comparison of average H-indices for (A) Canadian researchers (2020), and (B) American ophthalmologists (2016).9.

Western University, the University of Toronto, and Dalhousie University had the highest impact on a sustained basis as indicated by average m-quotient (Figure 7A). There was no correlation between the sustained productivity of each institution with the career longevity of its faculty members (Figure 7C, R² = 0.0, p = 0.83).

Figure 7 (A) Average m-quotients by institution, (B) career longevity of institutional faculty members as indicated by average first year of publication, and (C) research productivity as a function of career longevity.

Canadian Institutes of Health Research (CIHR) Funding

Figure 8 depicts trends in CIHR funding awarded to each institution between 2008 and 2020. The institutions with the greatest CIHR funding between 2008 and 2020 were the University of British Columbia ($8.3 M), Université de Montréal ($6.2 M) and McGill University ($4.9 M). There was no correlation between average institutional H-index and total CIHR funding (R²=0.02, p = 0.72).

Figure 8 Canadian Institutes of Health Research (CIHR) funding: (A) total funding per school, (B) institutional funding per number of faculty, and (C) average institutional H-index as a function of CIHR funding.

Discussion

Bibliometrics are a helpful objective indicator of academic impact and are increasingly being adopted in the academic promotion process. The H-index has been increasingly favoured as a measure of productivity due to its elegant method of balancing publication quantity and quality – and in the case of the m-quotient, assessing consistent productivity over time.^15–19

Of Canada’s 1246 ophthalmologists,²⁰ 686 (55%) were affiliated with academic departments and had online bibliometric profiles. This is a surprisingly high number of academic clinicians. In comparison, we estimate from International Council of Ophthalmology data that only 15% of American ophthalmologists have academic practices.^9,21 In reality, many of these Canadian academic ophthalmologists work peripherally with academic centres and spend much of their time in private practice. These nuances around academic affiliation in Canada may have deflated averages of scholarly productivity or skewed demographic data for departments in this study. The growing dichotomy between clinician-scientists and clinician-educators in modern departments may also contribute to variations in averages of scholarly productivity.

Twenty-eight percent of academic ophthalmologists in Canada are women, a rate similar to that seen in the United States, ~30%.^9,13 This figure falls short of overall Canadian averages, with women comprising 41% of the overall medical workforce and 36.2% of specialists in 2017.²² Women were also underrepresented at higher levels of appointment. In this study, women represented only 21% of Canadian Professors of Ophthalmology and 14% of program Chairs. Notably, the ophthalmology faculties at Université de Montréal and Université Laval were equally balanced in sex. Veilleux et al observed a similar trend at the Université de Montréal and Université Laval in their neurosurgery faculties and attributed this to cultures around staff recruitment at these institutions.²³ However, leadership among Canadian ophthalmology programs may be trending towards greater representation of women, considering that 4 out of Canada’s 15 centres (26%) currently have female Chairs.

Supporting this notion, women in Canadian academic ophthalmology departments were on average 6 years earlier in their career stage than men as indicated by first year of publication. Corrected for career stage and academic rank, there were no differences in scholarly productivity between male and female ophthalmologists. Further confirming this trend, a sub-analysis of the average H-indices between male and female ophthalmologists in their first 10 and 15 years of professional activity did not show any significant differences (p > 0.05 for both). Other studies have similarly found no significant differences in scholarly productivity between male and female academic ophthalmologists in the United States.^9,13

Our study found that H-index and m-quotient increased with academic appointment, MD/PhD designation and further training. Similar trends have been observed in the United States.^{9,11–13,17,24,25} These results indicate that Professors, Associate Professors, MD/PhDs, and fellowship-trained ophthalmologists generally produce higher impact work, and importantly, do so consistently over the duration of their careers. Of note, however, many MD/PhD clinicians focused their PhDs on general science topics, such as biology, chemistry, and physics. Thus, the H-index of MD/PhD clinicians may be inflated by PhD publications in other more widely read fields.

Measuring departmental achievement based on the contributions of its members has been done in other specialties^9,26 and may be helpful for program chairs to evaluate their programs, orient recruitment strategies and direct funding procurement initiatives. Compared to other scientific and medical disciplines, ophthalmology researchers typically have a lower H-index due to the profession’s smaller community and readership.²⁷ In contrast, researchers in ocular genetics, pathology and oncology had relatively high H-indices, a trend that corroborates the results of other American studies.^9,12 Thiessen et al believed that this trend represented the effect of a few extremely productive ophthalmologists in relatively small, academically oriented fields.⁹ However, these fields are also some of the most medically generalizable, and as such papers in these areas are often published in journals that have a wider readership throughout medicine. This likely also explains why basic science researchers and other specialties have higher metrics than clinical ophthalmologists.

Canadian departments had lower H-indices than their American counterparts, a trend previously identified in other specialties, including general surgery,^17,28 neurosurgery,^26,28,29 plastic surgery,^{15,17,28,30,31} vascular surgery,^17,28 cardiothoracic surgery,^17,28,32 otolaryngology^18,33 and physiatry.^34–36 American centres may have higher indices of impact for several reasons. Canadian centres are fewer in number and likely to see higher patient volumes, requiring academic clinicians to focus more of their time clinically than their American counterparts. American centres may also have more resources for research, including academic time, funding, and greater numbers of trainees who could assist with research initiatives.

The m-quotient is indicative of sustained publication impact over time and is useful when comparing younger researchers to their more seasoned counterparts. Université Laval, the University of Saskatchewan, and McMaster University had the youngest faculties as indicated by their members' average years of first publication. In contrast, McGill University, and the Universities of British Columbia (UBC) and Toronto had the most seasoned faculty. Notably, Université Laval, Queen’s University, and the Université de Montréal (UdeM) had younger staff with sustained levels of research productivity, as indicated by m-quotients (Figure 6C). However, the m-quotients of junior staff may be skewed by recent high impact work done during fellowship.

Funding by the Canadian Institutes of Health Research (CIHR) was greatest at some of the larger Canadian schools, including UBC, UdeM, and McGill University. Similar results were observed in a census of Canadian ophthalmology programs in 2010.⁵ Our results indicate that Western University, Dalhousie University, and the University of Toronto may have employed these resources most efficiently, as these schools had among the lowest funding per faculty member and higher average H-indices per dollars of CIHR funding. There was no correlation between CIHR funding and average institutional H-index, suggesting that the average quality of publications is similar across schools despite funding differences. However, the CIHR is only one potential source of funding for Canadian departments. Larger departments often have other significant sources of funding that smaller institutions may not have access to.

While the sum of H-indices may highlight the overall impact of large faculties, averages indicate the efficiency of departments and may be better benchmarks for smaller institutions.⁹ However, in small institutions, a few highly productive faculty members may unduly skew averages as well. Other limitations of using the H-index include the positive correlation of H-index with career length due to accumulating citation count, and also that highly prolific papers may not be adequately accounted for.¹⁹ Also, H-indices may be inflated by co-authorship regardless of the extent of their contribution.²⁷ Our study avoided pitfalls of other papers in the literature by excluding self-citations. However, like other studies, the exclusion of faculty without online bibliometric profiles (n = 74) may have resulted in unintentional errors in the analysis. Finally, this study does not account for other important ways in which clinicians and scientists disseminate new knowledge, including through conferences, presentations and over social media.

The H-index and m-quotient are useful indicators of scholarly productivity. This paper summarizes these metrics for Canadian academic ophthalmologists and vision scientists and highlights cogent workforce trends. Author impact increased with academic appointment, subspecialty training and MD/PhD designation. There were no differences in impact between sexes, although women were underrepresented at higher levels of appointment. Considerable variation was seen in CIHR funding between schools; however, CIHR funding was not associated with averages of scholarly impact. We hope that the above results will help guide the continuing development of robust ophthalmology and vision science programs in Canada.