Exam Sequence and Timing as Predictors of COMLEX-USA and USMLE Performance in Osteopathic Medical Students

Introduction

The Comprehensive Osteopathic Medical Licensing Examination (COMLEX-USA) Levels 1 and 2 and the United States Medical Licensing Examination (USMLE) Steps 1 and 2 are essential standardized assessments required for medical licensure in the United States. The COMLEX-USA examinations are tailored specifically for osteopathic medical students, whereas the USMLE examinations are available to both allopathic and osteopathic medical students. Both sets of exams are rigorous, with concordance studies indicating a strong positive correlation between COMLEX-USA and USMLE scores.¹ However, notable differences in content focus, scoring methodology, and question format exist between these examinations,² potentially influencing examinees to perceive the USMLE as being more challenging than the COMLEX-USA. Osteopathic medical students frequently choose to take both the COMLEX-USA and USMLE. This decision is driven by the desire to remain competitive with allopathic peers, particularly when applying to certain specialty residency programs.^3–5 Moreover, many residency program directors continue to favor USMLE scores over COMLEX-USA scores alone, rendering it advantageous for osteopathic applicants to complete both examinations.⁶

In recent years, the landscape of medical licensure examinations has undergone significant changes, most notably with the transition of the COMLEX-USA Level 1 and USMLE Step 1 from numerical scoring to pass/fail reporting.^7,8 This shift has reduced the early availability of numerical metrics for residency selection, placing greater emphasis on performance in later examinations (COMLEX-USA Level 2 and USMLE Step 2), clerkship evaluations, and other application components.^9,10 These changes have heightened the importance of optimizing performance on Level 2 and Step 2, making exam preparation strategies, particularly exam sequencing and timing, more critical than before.

Despite the high stakes, limited evidence exists on how the order and spacing of the COMLEX-USA and USMLE examinations influence students’ performance. The lack of clear, evidence-based guidance presents a challenge for students and educators seeking to make strategic scheduling decisions. Accordingly, this study addresses the following research questions: Does the sequence in which osteopathic medical students take the COMLEX-USA and USMLE examinations (COMLEX-first vs USMLE-first) influence performance outcomes? Does the interval between the two examinations (1–7 days vs ≥8 days) affect performance? Are the effects of exam sequencing and timing consistent across different academic performance quartiles? By addressing these questions, this study aims to provide evidence-based recommendations to guide exam scheduling decisions for osteopathic medical students, with potential implications for medical schools, advisors, and residency preparation programs.

Materials and Methods

Study Design and Participants

Numerical test scores were assessed for a cohort of 364 osteopathic medical students who completed both the COMLEX-USA Level 1 and USMLE Step 1 examinations between 2020 and 2022 (prior to the implementation of pass/fail grading) and 734 osteopathic medical students who completed both the COMLEX-USA Level 2 and USMLE Step 2 between 2021–2025. Participants were stratified into quartiles based on their cumulative academic performance ranking within their cohort at the time of the examination, with quartile 1 representing the highest-performing 25% of students and quartile 4 representing the lowest-performing 25%. Students in both cohorts were excluded if they failed either examination on their first attempt to avoid score distortions related to remediation effects, or if the interval between their examinations exceeded three months, as such extended gaps may introduce additional confounding factors unrelated to typical preparation timelines. The total sample size of 1098 students represented the full population that met the inclusion criteria during the study period, thereby ensuring sufficient statistical power to detect differences between groups without the need for additional sampling.

Data Collection

Data were collected on examination scores (COMLEX-USA Levels 1 and 2 and USMLE Steps 1 and 2), sequence of examinations (first examination taken), interval between examinations for each student, and academic performance ranking of students (quartiles) in medical school.

Statistical Analysis

All statistical analyses were performed using SPSS software (IBM, Chicago, IL, USA). The Shapiro–Wilk test indicated that the scores did not follow a normal distribution, necessitating the use of nonparametric statistical methods for subsequent analyses. Accordingly, the Mann–Whitney U-test was chosen because of its suitability for comparing median values between independent groups when the data are not normally distributed. Descriptive statistics were calculated for the median scores and interquartile ranges. Mann–Whitney U-tests were employed to compare scores between groups that took the COMLEX-USA first and those that took the USMLE first, as well as to assess the impact of examination intervals on scores. For this analysis, examination intervals were categorized into 1–7 days and ≥ 8 days. A one-week threshold was selected as it aligns with common scheduling practices, supports short-term knowledge retention, and minimizes the potential performance decline associated with longer gaps between examinations. Median scores were calculated for each interval group, both overall and within each quartile. The effects of exam order and interval were analyzed across all quartiles and within each quartile to identify performance trends and significant differences.

Artificial Intelligence

Generative AI via Paperpal was used exclusively for language refinement, including the improvement of word choice and sentence fluency. It did not contribute to the design, analysis, interpretation of the results, or the formulation of conclusions in this study.

Ethics Approval

This study was reviewed and granted exempt status by the Des Moines University Institutional Review Board (IRB# 2024–57).

Results

Findings were generally consistent across quartiles and spacings for both sets of examinations, showing that the sequence and timing of examinations substantially influence student performance. These results underscore the critical role of strategic scheduling and provide practical insights for osteopathic medical students preparing for pivotal assessments. The median scores for the COMLEX-USA Level 1 and USMLE Step 1 examinations were 567 ± 117.5 (95% CI: [563, 571]) and 227 ± 123 (95% CI: [223, 231]), respectively, while the median scores for the COMLEX-USA Level 2 and USMLE Step 2 examinations were 606 ± 113 (95% CI: [603, 609]) and 246 ± 18 (95% CI: [245, 247]), respectively. These differences were statistically significant across most comparisons, with effect sizes (Cohen’s d) ranging from 0.34 to 0.58, consistent with small to moderate effects that are meaningful in the context of high-stakes assessments.

Examination Sequence

Students who undertook the USMLE examinations first consistently achieved higher scores on both examinations than those who took the COMLEX-USA examinations first (Figure 1 and Table 1), with the exception of students in the third quartile who took the USMLE Step 1 first and students in the second quartile who took the USMLE Step 2 first (in both cases, the differences were not significant). In terms of interquartile performance, higher-performing quartiles attained superior overall scores in both sets of examinations. These differences remained statistically significant after adjusting for potential confounders, including cumulative grade point average, using a multivariable regression analysis.

Table 1 COMLEX and USMLE Scores by Exam Order

Figure 1 COMLEX and USMLE score distributions by exam order. (A and B) First licensing exams; (C and D) Second licensing exams. Note that in every case, students who took the USMLE exams first scored on average better than students who took the COMLEX exams first.

Examination Interval

The interval between examinations also influenced performance, with shorter intervals correlating with higher examination scores (Figure 2 and Table 2). For instance, students who took the COMLEX-USA Level 1 and USMLE Step 1 within 1–7 days of each other scored medians of 646 (95% CI: [642, 650]) and 241 (95% CI: [239, 243]), respectively, while those with gaps of ≥8 days scored 491 (95% CI: [487, 495]) and 226 (95% CI: [224, 228]), respectively. Students who completed the second examination within 1–7 days of the first examination, regardless of which examination was taken first, scored higher on the COMLEX-USA and USMLE than the median scores. Conversely, intervals exceeding 8 days were associated with significantly lower scores across all measures. This effect was most pronounced for COMLEX-USA Level 1 and USMLE Step 1, particularly among lower-performing quartiles. For example, fourth-quartile students taking COMLEX-USA Level 1 within 1–7 days scored a median of 517 (95% CI: [511, 523]), compared to 445 (95% CI: [441, 449]) when exams were spaced over eight days (p < 0.001, Cohen’s d = 0.51). These differences remained statistically significant after adjusting for potential confounders, including exam interval, using a multivariable regression analysis. Notably, the advantage of taking the USMLE first was not observed in all quartiles, as the differences for third-quartile students in Step 1 and second-quartile students in Step 2 were not statistically significant. This variability across quartiles warrants further investigation in future studies.

Table 2 Differences in COMLEX and USMLE Scores Related to Time Interval Between Exams

Figure 2 COMLEX and USMLE score distributions by exam interval. (A and B) First licensing exams; (C and D) Second licensing exams. Students who took the USMLE and COMLEX exams within a week on average scored better than students who took the exams farther apart.

Discussion

The landscape of medical licensure examinations has undergone significant changes in recent years, notably with the transition of the COMLEX-USA Level 1 and USMLE Step 1 to a pass/fail scoring system. This change has altered the dynamics of how exam scores are perceived and utilized in residency applications and medical education.^7,8 Historically, COMLEX-USA Level 1 and USMLE Step 1 scores have played a crucial role in residency selection, often serving as primary screening tools for program directors. With the move of the first level of board examinations to pass/fail reporting, the importance of other components in the residency application process has increased.^6,9,10

In light of these changes, this study aimed to investigate the impact of exam order and timing on student performance in the COMLEX-USA and USMLE examinations. The primary objectives were to 1) analyze the effect of exam sequence (COMLEX-USA first vs USMLE first) on overall performance; 2) evaluate the influence of the time interval between exams on scores; and 3) examine how these factors vary across different academic performance quartiles. By exploring these aspects, this study seeks to provide valuable insights that can inform osteopathic medical students’ strategies and institutional policies regarding licensure examination preparation and scheduling in the evolving landscape of medical education assessment. As residency directors increasingly rely on the numerical scores from the second level of board examinations to differentiate applicants,¹⁰ this study provides valuable insights into the effectiveness of timing strategies that optimize COMLEX-USA Level 2 and USMLE Step 2 performance.

Several key findings warrant further discussion in this study. First, the consistently superior performance of students who undertook the USMLE first across most quartiles indicates a potential advantage associated with this examination sequence. Statistically significant differences were observed in five of the eight quartiles studied. The most pronounced difference was observed in the fourth quartile, where USMLE-first students scored higher on the COMLEX-USA than their USMLE-second counterparts. However, this advantage was not present for third-quartile students in Step 1 and second-quartile students in Step 2, suggesting that sequencing benefits may vary by performance level and warrant further study. One possible explanation for the greater benefit observed in lower-performing students is that preparing for the USMLE first, with its broader scope and higher perceived difficulty, may foster comprehensive study habits, reinforce foundational knowledge, and enhance test-taking strategies. These factors could subsequently enhance performance on the COMLEX-USA. The significant differences observed, particularly in the lower-performing quartiles, underscore the importance of strategically scheduling examinations to optimize performance.

The impact of exam intervals on the scores was noteworthy. The correlation between shorter intervals and higher scores, especially for intervals of 1–7 days, suggests that closely spaced exams may help maintain peak performance and knowledge retention. Conversely, the substantial decrease in scores for intervals exceeding eight days highlights the potential risks of extended gaps between examinations, possibly due to knowledge decay or reduced momentum in study habits. The quartile-specific analysis revealed nuanced effects of the exam intervals across different performance levels. For top-performing students (first quartile), maintaining consistent performance appears achievable within a seven-day window, whereas longer intervals may prove detrimental. Middle-range performers (second and third quartiles) seem to benefit most from shorter intervals, particularly for COMLEX-USA. The pronounced positive effect of shorter intervals for lower-performing students (fourth quartile) suggests that this group may be the most sensitive to exam timing.

These findings have potential implications for institutional policies, particularly in the design of academic support programs. Academic advisors and career planning offices could use this evidence to guide students in scheduling board examinations strategically, considering both academic quartiles and optimal exam spacing. Institutions should also consider integrating board exam timing discussions into formal advising meetings and providing resources that facilitate shorter, well-planned exam intervals.

While the data strongly suggest a correlation between exam order, interval, and outcome, it cannot be ruled out that hidden variables contribute to the outcome. It is conceivable that exam order and the interval between exams reflect students’ confidence in exam preparation, with students who feel less confident opting to take the supposedly easier COMLEX-USA first and allowing themselves more time to prepare for the USMLE. The present study did not explore the reasons for students’ dual exam-taking strategies and was not designed to rule out the existence of such confounders.

Additionally, although grade point average and exam interval were included as covariates in the adjusted analysis, other unmeasured variables, such as motivation level, study resource utilization, prior standardized test performance, and external personal factors, could have influenced the results. Furthermore, as this study was conducted at a single institution, caution should be exercised when generalizing the findings to other osteopathic medical schools or educational contexts in general. Differences in curriculum structure, academic calendars, and advising practices could influence the applicability of these results. Future research should aim to replicate these findings in multi-institutional datasets and explore additional factors, such as specialty choice, prior academic performance, and specific preparation strategies, that may moderate the relationship between exam timing, sequencing, and performance.

Conclusion

This study examined the influence of exam sequencing and timing on COMLEX-USA and USMLE performance among osteopathic medical students. The findings indicate that taking the USMLE first is associated with higher scores on both exams, particularly among lower-performing students. Furthermore, shorter intervals between exams (1–7 days) were correlated with improved outcomes, whereas longer intervals (≥ 8 days) resulted in significantly lower scores. For example, students who took COMLEX-USA Level 1 and USMLE Step 1 within 1–7 days of each other achieved median scores of 646 and 241, respectively, compared to 491 and 226 for those with ≥8-day intervals, a gap of more than 150 COMLEX points and 15 USMLE points. These results suggest that the strategic scheduling of licensing exams can enhance performance, with important implications for both osteopathic medical students and educators. However, individual study habits, academic preparedness, and personal factors must be considered when determining an optimal approach. Educators and advisors are encouraged to integrate these findings into individualized exam-scheduling guidance, while students should consider both sequencing and timing as part of a deliberate preparation strategy. At a broader level, these findings have implications for educational policy and program design, particularly in adapting advising frameworks and assessment planning in response to the recent transition of Level 1 and Step 1 to pass/fail scoring systems. Incorporating evidence-based exam sequencing and timing recommendations into institutional policies could help optimize student performance and better prepare graduates for residency selection in the evolving assessment landscape. Future research should explore additional variables influencing exam performance, such as specific study strategies, curriculum integration, and stress management techniques to further refine recommendations for optimal exam scheduling.