Evaluation of medical students and the United States Medical Licensing Exam (USMLE)

The evaluation of Medical Students and the United States Medical Licensing Exam (USMLE) are charted here by L. Maximilian Buja, MD, Professor of Pathology and Laboratory Medicine, McGovern Medical School, The University of Texas Health Science Center at Houston (UTHealth)

The United States Medical Licensing Examination (USMLE) is a three-step examination for medical licensure in the United States and is sponsored by the Federation of State Medical Boards (FSMB) and the National Board of Medical Examiners (NBME) (Wikipedia, 2020). The stated goal of the USMLE is to assess a physician’s ability to apply knowledge, concepts, and principles, and to determine fundamental patient-centred skills that are important in health and disease and that provide a basis of safe and effective patient care. The USMLE exam has three steps Step 1, Step 2 Clinical Knowledge (CK), Step 2 Clinical Skills (CS), and Step 3. All three steps of the USMLE exam must be passed before a physician with an M.D. degree is eligible to apply for an unrestricted license to practice medicine in the United States. U.S. osteopathic medical school  graduates are permitted to take either the USMLE or the Comprehensive Osteopathic Medical Licensing Examination  (COMLEX) exam for medical licensure. Students who have graduated from medical schools outside the U.S. and Canada must pass all three steps of the USMLE to be licensed to practice in the U.S., regardless of the title of their degree.

USMLE Step 1 assesses whether medical school students or graduates understand and can apply important concepts of the basic sciences to the practice of medicine. Step 1 is constructed according to an integrated content approach and includes the following subjects: Anatomy, Behavioral Sciences, Biochemistry, Microbiology, Pathology, Pharmacology, Physiology, and interdisciplinary topics such as nutrition, genetics, and ageing.

U.S. medical students take Step 1 at the end of the Basic Sciences portion of the curriculum, usually after the second year of medical school. The content and timing of the Step 1 exam is the reason it is broadly viewed as a most arduous and high stakes examination for medical students. Its three-digit quantitative result has had a substantial bearing on the specialities and status of the healthcare system for which a residency applicant is competitive.

Medical education

The traditional medical education system widely adopted throughout most of the twentieth century has produced generations of scientifically grounded and clinically skilled physicians who have served medicine and society well. Yet sweeping changes launched around the turn of the millennium have constituted a revolution in undergraduate medical education (UME) and graduate medical education (GME) (Buja. 2019). The core goal is the production of physicians with modern clinical reasoning and decision-making skills. The construct to achieve this overarching goal is the so-called fully integrated spiral curriculum encompassing both horizontal and vertical integration across time and disciplines (Brauer and Ferguson, 2015).

The fully integrated curriculum resulting from the redesign does away with the distinction between the critically important pre-clinical (basic medical sciences) two-year period and the apprenticeship-like clinical two-year period. It brings in additional content called Health Systems Science, as a co-equal to basic and clinical sciences, to cover topics from population health to interdisciplinary care. There also has been a push in recent years for undergraduates to demonstrate competencies rather than solely cognitive knowledge.

How students function in an educational program is inextricably linked to how they are evaluated. Recurrent movements to abolish grades, exams and honour societies to mitigate undue competitiveness, stress and general malaise are prevalent today. For many years, the standard system of student evaluation was based on numerical grades in every course and led to a cumulative numerical score and class ranking. This objective evaluation system has largely been replaced in medical schools by summative pass-fail systems.

The movement away from meaningful grades for medical school courses also has led to an increasingly elaborate subjective evaluation in “dean’s letters”. The AAMC has introduced the Medical Student Performance Evaluation (MSPE) as a refinement of the “dean’s letter.” Approaches to evaluation of student performance generally involve formative and summative exams in the pre-clinical years, and subject exams coupled with faculty assessment of performance, in the clinical clerkships. Then, these evaluations (honours, high pass, pass, etc.) are integrated into lengthy MSPEs or dean’s letters that provide commentary and largely subjective impressions. Despite the AAMC guidelines of comparative information about applicants be included, dean’s letters or MSPEs often continue to lack specificity regarding student performance.

Further thoughts on the United States Medical Licensing Exam

This has led to the rise of the exaggerated importance of United States Medical Licensing Exam (USMLE) scores, particularly, USMLE Step 1 scores, as the major or sole objective evaluation of the cognitive achievement of medical students. Proponents argue that the new curricula are successful because students are performing at least as well on USMLE Step 1 as they did in the old curricula, and that they do as well in pass-fail systems as in systems with grades. However, these advocates, in essence, are contributing to the perpetuation of the undue importance of USMLE Step 1.

An undue emphasis on a single high stakes summative evaluation creates a dilemma for medical educators and students. An excessive focus develops on preparing students for the USMLE Step 1 examination and “teaching to the test”. This milieu is counterproductive to in-depth assimilation of subject matter in the biomedical sciences. Unintended consequences in multiple domains include conflict with holistic undergraduate medical education admission practices, student well-being, and medical curricula.

Medical students have become increasingly aware of the ”USMLE issue.” In an Invited Commentary, medical students from various institutions across the country have reflected on their shared experiences and have postulated that the emphasis on USMLE Step 1 for residency selection has fundamentally altered the preclinical learning environment, creating a “Step 1 climate” (Chen et al, 2019). They have commented on how the Step 1 climate negatively impacts education, diversity, and student well-being, and they have urged a national conversation on the elimination of reporting Step 1 numeric scores.

Educators also have articulated similar recommendations regarding making the USMLE results reporting as pass/fail. But concern has also been voiced that pass/fail can be a disincentive to motivation for broad knowledge acquisition. Also, the development of an alternate, more holistic standardised metric by which to compare students’ applications for residency positions has been proposed but is currently not operative.

In recent years, an applicant’s Step 1 score has been cited by residency program directors as their most important criterion in selecting graduating medical students for their residency program (Willett, 2020; Makhoul et al. 2020). The current use of Step 1 scoring as a major determinant for granting residency interviews has been met with tremendous criticism by the medical community, citing that the Step 1 exam was intended to be one of four licensing tests. It was never designed to be a predictor of medical knowledge for which cut-offs or barriers could be justified and instead enables racial bias.

In response to public outcry, in February 2020, the USMLE program announced a plan to change Step 1 score reporting to a pass/fail system in an effort “to reduce overemphasis on Step 1 performance while allowing licensing authorities to continue the original intention to use the test to determine medical license eligibility.” However, this transition will occur no earlier than January 1, 2022. However, critics of this decision argued that this would just shift the importance of the three-digit number score on to Step 2 CK, as well as putting international medical graduates (IMGs) at a disadvantage, as traditionally IMGs scored exceptionally high on Step 1 to distinguish themselves and obtain residency positions in coveted specialities or hospitals.

The movement of the USMLE Step 1 to a pass/fail exam is being viewed as a golden opportunity to recalibrate medical education priorities in UME and to improve the residency selection process. This view is generally taken by medical educators (Prober, 2020). However, this view comes up against the practical realities faced by residency program directors in dealing with the large number of applicants to individual residency programs (Willett, 2020). Programs receive thousands of applications and have only a few weeks to review them and decide on whom to invite for interviews. This phenomenon, dubbed “application inflation” has made holistic applicant review not practical. The change of Step 1 scores to pass/fail removes one of the few objective data points that program directors use for filtering. With the change to Step 1, Step 2 CK will inevitably become the highest-stakes test for students.

To characterise residency program directors’ responses to binary Step 1 result reporting, a 19-item survey has been developed and validated (Makhoul et al, 2020). A total of 2,095 unique responses (response rate, 44.5%0 were obtained. Only 15% of program directors agreed with changing Step 1 to pass/fail, and 77% expected this change to make objective comparison of applicants more difficult.

Concluding remarks

I think that the dilemmas about the “USMLE issue” can be diffused by a return to providing meaningful grades for medical school courses and an overall summative evaluation for the four years of medical school. My definition of meaningful grades encompasses either numerical or letter grade equivalents which reflect actual performance relative to other students and objective norms. Students must compete and excel to gain admittance into medical school. This shouldn’t be any different when students are training to be physicians. Safeguards can be put in place to deal with excess competition. Nevertheless, competition within bounds promotes excellence. Medicine needs to remain a meritocracy.

References

Brauer DG, Ferguson KJ. The integrated curriculum in medical education: AMEE Guide No. 96. Med Teach. 2015 Apr;37(4):312-22. doi: 10.3109/0142159X.2014.970998. Epub 2014 Oct 16. PMID: 25319403.

Buja LM. Medical education today: all that glitters is not gold. BMC Med Educ. 2019 Apr 16;19(1):110. doi: 10.1186/s12909-019-1535-9. PMID: 30991988; PMCID: PMC6469033.

Chen DR, Priest KC, Batten JN, Fragoso LE, Reinfeld BI, Laitman BM. Student Perspectives on the “Step 1 Climate” in Preclinical Medical Education. Acad Med. 2019 Mar;94(3):302-304. doi: 10.1097/ACM.0000000000002565. PMID: 30570499.

Makhoul AT, Pontell ME, Ganesh Kumar N, Drolet BC. Objective Measures Needed – Program Directors’ Perspectives on a Pass/Fail USMLE Step 1. N Engl J Med. 2020 Jun 18;382(25):2389-2392. doi: 10.1056/NEJMp2006148. PMID: 32558467.

Prober CG. Grading Changes for USMLE Step 1 – A Golden Opportunity to Recalibrate Medical Education Priorities. N Engl J Med. 2020 Jun 18;382(25):2385-2387. doi: 10.1056/NEJMp2003880. PMID: 32558465.

Willett LL. The Impact of a Pass/Fail Step 1 – A Residency Program Director’s View. N Engl J Med. 2020 Jun 18;382(25):2387-2389. doi: 10.1056/NEJMp2004929. PMID: 32558466.

United States Medical Licensing Examination.  Wikipedia (https://en.wikipedia.org/wiki/United_States_Medical_Licensing_Examination).

 

Please note: This is a commercial profile