by Pranav Reddy, Kunal Sindhu, MD, and Bryan Carmody, MD, MPH
On March 11, an invitation-only meeting will determine the future of the United States Medical Licensing Exam (USMLE) Step 1 exam. The results will profoundly affect how all future American doctors are taught.
The American model of medical board examinations is unsustainable. We have argued along with others that Step 1 should be scored pass/fail and made more relevant to the wards. Step 1 was designed to make a binary decision regarding competency for state medical licensure. Yet because test results are reported as a three-digit score, Step 1 has become the de facto entrance exam for residency.
The increasing focus on Step 1 scores has adverse effects on the wellness of trainees, the residency selection process and medical school curricula. To determine the future of USMLE score reporting policy, the test’s sponsor organizations will convene the Invitational Conference on USMLE Scoring (InCUS) on March 11-12, 2019.
Truly addressing this issue will require a more inclusive and transparent process than InCUS plans to offer.
Our primary concern with InCUS is the “I” — the conference is by invitation only. Step 1 score-reporting is a critical issue that ultimately impacts everyone in medicine, and while the sponsors have stated that “representatives from a variety of organizations will participate,” the organizers have not disclosed who has been invited to attend or how attendees were chosen.
Unfortunately, there are reasons to believe that the concerns of medical students, program directors and medical educators will be marginalized. Recently, the CEOs of the USMLE’s sponsor organizations authored an invited commentary in Academic Medicine discussing the USMLE score reporting policy. Among the objections cited was that, if Step 1 required less preparation, medical students might instead spend their time “binge-watching the most recent Netflix series or compulsively updating their Instagram account.” While the authors issued an apology and the line was removed from the final publication, the message was clear: the sponsor organizations favor the status quo.
The organizations sponsoring InCUS are the National Board of Medical Examiners (NBME), the Federation of State Medical Boards (FSMB), the American Medical Association (AMA), the Association of American Medical Colleges (AAMC), and Educational Commission for Foreign Medical Graduates (ECFMG). These organizations have a financial interest in maintaining a scored Step 1. Revenues for the NBME have grown rapidly alongside the test’s growing importance in residency selection. The AAMC’s biggest source of program revenue is ERAS, the Electronic Residency Application Service. ERAS revenue has more than doubled over the past 10 years as students apply to more and more programs — yet the increase in applications per student is enabled by the use of Step 1 scores for screening. Even the AMA accepts revenue from Kaplan for advertising its USMLE preparation products. These financial relationships create problematic conflicts of interest for the groups leading the conference.
While students may spend more time preparing for a scored test, there is evidence that the costs of this approach outweigh any benefits. The “Step 1 climate” harms the mental health of medical students and its overuse in the residency selection process is educationally detrimental. The “arms race” of rising Step 1 scores reduces student engagement with institutional curricula, resulting in an effective outsourcing of medical education to the third-party test preparation industry. Medical schools may create essential programs, like how to care for dying patients with humanity, but the bottom-line goal for many medical students is doing well on Step 1. Moreover, Step 1 scores are not correlated with the acquisition of clinical skills, and evidence that they predict physician quality is limited.
The decisions made at InCUS will affect every medical student in the United States and will have a profound influence on the process every residency program uses to select its residents. With so much at stake, it is crucial for a diversity of opinions to be aired.
We are concerned that the apparent lack of opposing views will limit the diversity of views expressed and considered at the conference. While the conference organizers promise a “6-8 week period for open commentary” in the months ahead, it is unclear how impactful the opinions of other stakeholders will be if their views are not seriously considered from the outset.
InCUS’s organizers would be wise to enter the conference with an open mind, expand attendance to include critics of the numeric Step 1 scoring system, formalize methods for public comment, and prioritize transparency.
The stakes are simply too high for InCUS to proceed as currently structured.
Image source: Library Books by CCAC North Library licensed under CC BY 2.0.
Resident Physician Contributing Writer
Mount Sinai Medical Center
Kunal Sindhu is currently a second year resident in radiation oncology at Mount Sinai Hospital in New York. He received his Bachelor of Arts from the University of California, Berkeley, and his medical degree from the Warren Alpert School of Medicine at Brown University. Twitter: @sindhu_kunal
Attending Physician Guest Writer
Children’s Hospital of The King’s Daughters
Bryan Carmody is an assistant professor of pediatrics at Eastern Virginia Medical School. He earned his MD and MPH from the University of Virginia, and practices as a pediatric nephrologist at the Children’s Hospital of the King’s Daughters in Norfolk, Virginia. Twitter: @jbcarmody
Medical Student Guest Writer
Warren Alpert Medical School at Brown University
Pranav Reddy is a joint MD/MPA candidate at the Warren Alpert Medical School at Brown University and Harvard Kennedy School. Pranav has worked on issues at the intersection of health and human rights for marginalized populations in Rhode Island, the Dominican Republic, and India and will be training in obstetrics and gynecology after graduation. Twitter: @born_reddy