Evaluating the Competency and Confidence in ECG Interpretation Among Residents: A Multi-Center Pilot Study

Background

In clinical setting electrocardiography (ECG) serves as an essential diagnostic tool for the early diagnosis and treatment of myocardial infarction and arrhythmias.¹ Additionally, it is crucial for residents to accurately interpret the ECGs and be able to determine when a cardiology consultation might be necessary.² ECG learning is a basic course for medical students for their under-graduating and postgraduate clinical training.³ Furthermore, acquiring ECG interpretation skills is a common and crucial part of the cognitive training for medicine physicians in their initial residency.⁴

In order to attain the ECG interpretation skill, the American College of Cardiology (ACC)/ American Heart Association’s (AHA) guidelines recommend interpreting a minimum of 500 supervised ECGs during initial training, using standardized testing in ECG interpretation to confirm initial competency and interpreting 100 ECGs yearly to maintain competency.⁵ The recommendation revealed the difficulty of acquiring and maintaining this competency. A systematic review demonstrated resident accuracy in ECG interpretation varied from 46% to 83% compared with reference standards.⁶ As previously shown, it has been reported that up to 33% of ECG interpretations in clinical practice contain errors of major importance.⁷ Research has revealed that the average accuracy scores reported through studies range from 25% to 93%⁸ and 36% to 96%.⁹

Interestingly, the accuracy of different types of ECG interpretation variables among residents, even when they have the same level of training.¹⁰ For instance, some studies found that the accuracy of ECG interpretations for ST elevation MI (STEMI cases) is waved between 79% and 100%.^4,8,9 In contrast, for arrhythmia cases, Sibbald et al has reported that the accuracy of interpretation was 93%,⁶ while Bond et al found it to be 71%.⁸

The accuracy of ECGs interpretation by resident varies widely across different regions, with an overall trend indicating poor interpretation skills. For instance, various studies have revealed that ECG interpretation accuracy rates as follows: 54% among 46 Irish students,¹¹ 52% among 52 New Zealand students,¹² and 37% among 168 American students.¹³ However, the comparison of different studies on ECG interpretation competency is difficult because ECGs used for testing and methodologies vary. Currently, there are no universally accepted or national guidelines that define specific ECG standards for medical students or residents.

In recent years, most of the current studies were based on one center, schools or hospitals,¹⁴ and often failed to analyze potential underlying causes, remaining predominantly at the survey level. Additionally, there was also little evidence from Chinese medicine education.¹⁵ Our primary objective is to assess whether the low competency levels in ECG interpretation, as described in worldwide studies,¹⁵ are also prevalent among the Chinese residents from three hospitals in the South China. We also sought to assess the associations of various co-factors and resident’s confidence with ECG competency.

Methods

Study Design and Participants

This pilot study conducted a cross-sectional, questionnaire-based assessment of ECG interpretation skills of residents (n=165). The participants were first-year residents who received three different teaching hospitals in the study. The questionnaire was developed and validated in 2023, with data collection occurred in the same year. The questionnaire and research methods were approved by the Peking university Shenzhen hospital ethics committee (2023–546). Informed consent was obtained from all individual participants included in the questionnaire.

Procedure

We selected ten 12-lead ECGs from the reputable teaching books, all straightforward examples. Five independent cardiologists were consulted to suggest diagnoses for each ECG strip. Only those ECG strips with a unanimous agreement among the cardiologists and matching the official interpretation were included in the questionnaire. Through this validation process, two ECGs were excluded, leaving eight ECGs for the final questionnaire (appendix).

The ECGs were classified into three groups according to a combination of emergency/non-emergency and common/uncommon criteria, as a previous study.¹⁶ Residents must be proficient in common emergency and nonemergency abnormalities. Therefore, based on this category, we included four strips presenting common emergency cardiovascular conditions: STEMI, ventricular tachycardia (VT), supraventricular tachycardia (SVT) and a prolonged QT (QTc=500 milliseconds). Another three common non-emergency ECG abnormalities (atrial fibrillation (AF), left bundle branch block (LBBB) and pacemaker rhythm) were also presented in the study. A normal ECG was also included in the questionnaire.

The questionnaire began with five open-ended questions regarding the primary sources of the participants for education in ECG interpretation. It assessed whether their ECG skills were acquired through ECG classes, clinical clerkships, self-study with printed materials or web-based sources, and inquired about their major (internal medicine or non-internal medicine). This section was flowed by the eight ECGs.

Each ECG was accompanied by a brief clinical scenario, consisted of age, gender, and chief complaint. Participants were required to provide a primary diagnosis for each ECG and rate their confidence in their answer using a Likert scale from 1 to 5 (1 being a complete guess and 5 absolute confidence). The questionnaire concluded with a question on overall confidence, also rate on a Likert scale. No time limit was imposed on the participants.

Every response was scored on a scale of 0 to 2 points (0 = incorrect, 1= partially correct, 2 = correct) using a prespecified answer key created during the validation process by five cardiologists.

The maximum total score achievable was 16, with a passing score set at 10, reflecting a competency level above 60%.

Data Collection

Participants were gathered in a lecture hall. They received a link to a web-based version of the questionnaire and answered it. All of the participants were supervised during the questionnaire. To avoid communication between participants, we created a unique link for each teaching hospital, blocked access to the questionnaire between dates and did not provide any feedback or answers before the end of the study.

Data Analysis

Cronbach’s Alpha was used to measure reliability. Data were presented as median, (interquartile range, IQR) values, percent (%). Mann-Whitney was used to compare points and degree of confidence between internal medicine or non-internal medicine. One way ANOVA identified variables that were significantly associated with the pass rate. Spearman’s rho correlation test was used to explore the correlation between the total score in the questionnaire and degree of confidence, assessment difficulty and learning effect in school. Statistical analysis was performed with IBM SPSS Statistics 27, and p<0.05 was considered statistically significant.

Results

A total of 165 residents consented to participate and successfully completed the questionnaire in this study. The Cronbach’s Alpha for the Likert scale used item questionnaire was 0.75. Among the respondents, a majority (n=116, 70.30%) were non-internal medicine majors, while the remaining participants (n=49, 29.70%) were internal medicine majors in our questionnaire.

The questionnaire covered several aspects, including the residents’ learning profiles of ECG during their university education, methods adopted to acquire ECG recognition, and the challenges faced in ECG interpretation. With the university learning context, only a small number of residents (n=3, 1.82%) considered themselves proficient in ECG interpretation. A significant proportion of residents (n=43, 26.06%) were not familiar with ECG. The majority (n=102, 61.82%) reported relearning ECG knowledge during their clinical clerkships. Nearly 90% of residents (n=147, 89.09%) stated that they primarily learned ECG through textbooks and related books. Over half of residents (n=94, 57.0%) indicated that tutoring during clinical clerkships was a crucial source of ECG knowledge. Additionally, self-study using printed materials or web-based resources was a common method among many residents (n=87, 52.73%). A small group (n=19, 11.52%) acquired their ECG skills from attending specific ECG classes. The most common difficulty encountered by the majority of residents (n=130, 78.79%) was interpreting arrhythmias (Table 1).

Table 1 The Basic Information and All-Over Competency

The maximum total score achievable was 16, and the passing score was considered 10, which is consistent with competency greater than 60%. However, the overall ECG interpretation ability among the residents was found to be poor, as indicated by an average test score were 6(4,8), as shown in Table 2. Only 17 residents (10.3%) scored more than 10 points. Residents scored a maximum of 13 points (n=2, 1.21%), and none of them correctly identified all eight ECGs.

Table 2 Mann-Whitney Was Performed in the Internal Medicine and the Non-Internal Medicine About ECGs Scores

Compare the Score and Confidence Level Between the Internal Medicine and the Non-Internal Medicine

A Mann-Whitney was conducted to compare the ECG interpretation scores and confidence levels between the internal medicine and the non-internal medicine residents. It was found that the total ECG scores were slightly higher among non-internal medicine residents compared with those in internal medicine residents, though the difference did not reach statistical significance (p=0.06). Only the ECG score of SVT was statistically different between the two groups, for the internal medicine students, the median score was 0, and for the non-internal medicine residents, the median was 2. There was no statistical difference between the interpretation skill of other ECGs, such as STEMI, VT and Atrial fibrillation (Table 2).

There were significant differences between the two groups in the confidence level of 4 ECG stripes, including STEMI, VT, a prolonged QT, normal ECG. For the internal medicine residents, confidence level median confidence of STEMI was 3, which was higher than the non-internal medicine residents. Furthermore, internal medicine residents were more confident on the interpretation of VT, long QT and normal ECG compared with non-internal medicine counterpart. The non-internal medicine residents tended to report low confidence in their ECG interpretation. For the seven abnormal ECGs, the median reported level of confidence was just 2 (STEMI, VT, SVT, QT, LBBB, atrial fibrillation, Pacemaker rhythm). However, normal ECG, the median confidence level was 5 for non-internal medicine residents. For the overall confidence level, the internal medicine group tended to be more confident (p<0.05) (Table 3).

Table 3 Mann-Whitney Was Performed in the Internal Medicine and the Non-Internal Medicine About ECGs Confidence Level

Individual ECG Strip Score and Confidence Level

As illustrated in Figure 1A, the highest competency was observed in the STEMI strip with a correct rate of 54.55%. The most challenging ECGs to interpret accurately were those indicating long QT and LBBB with dismal correct rates of 0.61% and 0.00%, respectively. For the ECGs of VT and SVT, the number of residents who answered correctly were nearly identical. However, a higher error rate of 48.48% was noted in correctly identifying VT. In common abnormalities ECG stripes, none of the residents correctly recognized LBBB. Only 4 residents (2.42%) correctly recognized Pacemaker rhythm. The most commonly misread abnormalities were LBBB, atrial fibrillation and pacemaker rhythm with error rates of 96.96%, 61.82%, and 95.76%, respectively. The accuracy in recognizing a normal ECG was also suboptimal, with a correct rate of only 41.82%. Furthermore, more than one third residents were correctly interpreted common emergencies ECG stripes (Figure 1B). However, for common abnormalities, the correct rate was just about 10% (Figure 1C).

Figure 1 The number of residents categorized by the score for each ECGs. (A), the number of residents categorized by the score for each ECGs. (B), the precent of residents in common emergencies by different score level. (C), the precent of residents in common abnormalities by different score level. Abbreviations: ECG, Electrocardiogram; STEMI, ST elevated myocardial infarction; VT, ventricular tachycardia; LBBB, left bundle branch block; SVT, supraventricular tachycardia.

The total median confidence level of the residents in the questionnaire was 18 (out of a full level of 40). Over a third of the residents relied entirely on guesswork for the ECG interpretation. Very few participants reported complete confidence in their diagnoses (less than 10%, as shown in Figure 2A). For the STEMI strip, 15 residents (9.09%) were absolutely confident in the interpretation, matched with the good performance. However, just 5 residents had the absolute confidence of normal ECGs. As shown in Figure 2B and C, for the common emergencies and abnormalities, the confidence level was similar.

Figure 2 The number of residents categorized by the confidence level for each ECGs. (A) the number of residents categorized by the confidence level for each ECGs. (B) the precent of residents in common emergencies by different confidence level. (C) the precent of residents in common abnormalities by different confidence level. Abbreviations: ECG, Electrocardiogram; STEMI, ST elevated myocardial infarction; VT, ventricular tachycardia; LBBB, left bundle branch block; SVT, supraventricular tachycardia.

In the current study, there was no significant relationship between confidence and ECG interpretation. Furthermore, no variables were significantly associated with the pass rate.

Discussion

This pilot multi-center study is the first of its kind to investigate the competency and confidence in ECG interpretation among residents in China. The evaluation revealed several key findings, which can be summarized as follows: (1) There were no statistically significant differences in ECG interpretation skills between the internal and non-internal medicine residents. (2) The residents exhibited a low level of competency in ECG interpretation. (3) The confidence level of the residents in the questionnaire was generally poor in the ECG interpretation; (4) There was no significant relationship between confidence levels and accuracy in ECG interpretation, and no variables were found to be significantly associated with the pass rate.

In terms of questionnaire’s reliability, it was considered to be dependable, with a reasonably acceptable Cronbach’s Alpha score (α=0.748).^17–19 Moreover, relatively few ranked items were included in the questionnaire, which likely resulted with an underestimation and negatively affected the Cronbach’s Alpha.¹⁹

In the current study, approximately one-tenth students (10.30%) scored above 10 points (10/16, a commonly used threshold for a passing grade on exams). The finding of poor performance aligns with results from previous studies. A 2019 cross-sectional study conducted in Israel, which included 414 medical students from their third to sixth years, revealed that only 23.5% students passed the exam. Additionally, research involving in final-year students reported ECG interpretation accuracy rates of 26% among 74 South Asian students,²⁰ and 37% among 168 American students.¹³ However, comparing ECG interpretation levels across different studies is challenging due to variations in the ECGs used and the methodologies employed in questionnaires. It’s crucial to acknowledge that, as of now, there are no established worldwide or national guidelines that define the standards for ECG competency and confidence.

We hypothesized that students majoring in internal medicine would exhibit greater competency in ECG interpretation compared to their non-internal medicine counterparts, presuming that internal medicine residents might place more emphasis ECG clinical skills.²¹ However, non-internal medicine residents demonstrated a significantly improved ability to interpret supraventricular tachycardia, but there were no other statistically significant differences among other ECG strips. Jeffrey S Berger et al reported no significant differences in overall competency between non-internal medicine and internal medicine residents (14.0 vs 15.0, P = 0.239). The partially inconsistent results might be attributable to variances in the residents’ year of study and assessment methods, which can significantly impact performance.²² Furthermore, the confidence level in ECG interpretation between the two groups were evaluated. The overall confidence level was higher among internal-medicine residents compare to non-internal medicine residents. The heightened confidence may stem from increased and repetitive exposure to ECGs.¹⁶ However, the higher confidence level did not correlate with superior ECG interpretation performance. This paradoxical phenomenon may be result of limited exposure time to ECGs, considering that the participants in our study were only in their first year of residency.

This unexpected finding underscores that ECG interpretation competency is not a simple function of one’s broad specialty designation but is highly dependent on the specific clinical exposure and focused training one receives. For instance, emergency medicine or surgical residents might encounter and manage tachyarrhythmias more frequently in acute settings, leading to better performance in specific domains. This analysis moves beyond documenting a general training gap and points toward a more nuanced understanding of how clinical experiences shape competency. It argues for a tailored, rather than a one-size-fits-all, approach to ECG curriculum development in residency programs.²³

Differences in interpretation competency were evident across various ECG stripes, highlighting the importance of identifying and addressing these gaps in ECG competency. In our study, the highest competency was observed in interpreting the STEMI strip, with a correct rate of 54.55%. Conversely, the lowest competency was noted in interpreting long QT and LBBB strips, aligning with findings from previous study.¹⁶ Guy Vishnevsky et al also revealed the highest competency was observed in the STEMI ECG in Hebrew University, Jerusalem, Israel.¹⁶ The STEMI ECG strip, often encountered and taught as common emergency scenario, reflects a pattern where interpretation competency is generally higher. This phenomenon aligns with previous findings that competency in ECG interpretation correlates with the degree of exposure to specific ECG patterns.^13,24 It is likely that low competency is seen in less frequently encountered ECG patterns. For example, in the clinical practice, LBBB may not always significantly influence patient management and is less frequently encountered. Therefore, the competency for interpreting LBBB is lower, and it’s often partially misread as STEMI. We should keep in mind the gap and difference of competency for various ECG stripes in order to focus on them.

The assessment of confidence in relation to performance was a deliberate and novel aspect of our study design.²⁵ Previous studies have suggested that confidence in the performance of a specific skill, such as ECG interpretation, correlates with the clinical experience in this area.^16,26 However, our study did not find a correlation between confidence and competency among our residents. This lack of correlation might be attributed to the generally poor competency and low confidence observed across the board. Unfortunately, none of the included variables were related to the pass rate, a discrepancy that could stem from the relatively small size.

While we found no link between confidence and competency, the fact that fewer than 10% of residents felt confident in their diagnoses—coupled with poor overall test scores—is concerning. This “double deficit”—lack of skill coupled with lack of self-recognition of that deficiency—highlights a critical risk point for patient safety. It suggests that training interventions must not only transmit knowledge but also include constructive feedback mechanisms to help learners accurately calibrate their self-assessment abilities, ensuring they seek help when needed.²⁷

While expert physicians can form diagnoses with minimal information, novices need clearer visualizations of abnormalities to aid their interpretation. Scientific advancements should therefore focus not just on refining expert tools but also on creating accessible technologies that make diagnostic patterns more evident for less experienced practitioners.²⁸ Currently restricted access to raw digital ECG data prevents clinicians from applying alternative processing methods that could reveal additional features and enhance diagnostic accuracy beyond the conventional format.²⁹

Our study has several strengths. By including participants from three different teaching hospitals, we could minimize bias and enhance the representativeness of our findings. The questionnaire featured validated, diverse, and clinically relevant ECGs and clinical scenarios as previous reported.¹⁶ Notably, it is the first study in China to compare residents from internal and non-internal medicine backgrounds, evaluating their differences. Additionally, unlike findings in Western countries, we identified no relationship between confidence levels and competency among participants,²⁶ which could be attributed to cultural differences warranting further investigation.

However, there are also several limitations to this study. Being a pilot study, some of the findings may require further validation. Participation was voluntary and limited to hospitals in the southern China, which could introduce a selection bias and may not accurately represent all Chinese residents. The residents were all in their first year at the hospital, not reflecting the competencies of residents in other years. Additionally, the small sample size limited our ability to identify variables related to competency and confidence levels among residents.

Conclusion

Our research reveals that residents do not appear to be prepared to interpret ECG abnormalities, including common emergencies and common abnormalities, even the normal ECG strip.

Competency and confidence in ECG interpretation seem to be very poor in the internal and non-internal medicine residents. Thus, strategies to facilitate better ECG skills should involve an extended focus on ECG in the afterward standardized training and include competency-based educational programs.