Ranked to fail: Does gatekeeping affect diversity in The Match?

For graduating medical students hoping to secure a residency training position — and that’s virtually all of them — Match Day is the most anticipated day of the year. The Match is the culmination of medical school, application preparation, interviews, and recruitment events.

It’s like a traditional job search but without the room for negotiation since it is based on an algorithm that announces job placement once a year, on the third Friday of March, for all participating programs and candidates in the country.

The Match, which is owned and operated by the National Resident Matching Program (NRMP), uses an algorithm to match choices on two different lists. One is the list each student creates, ranking their choice of the residency programs they applied to from top to bottom. The other is the list each residency program makes, ranking all applicants they reviewed in numerical order, with number one being their most desirable candidate and the last one being the least desired.

As programs create their internal list, applicants are strategically tiered into categories with commonly used phrases like “guaranteed match,” “highly desired,” “matchable range,” “unlikely to match,” and “do not rank.”

Each year, thousands of applicants go unmatched. Being unmatched means being unemployed after spending four years in college (and sometimes a one-year post-baccalaureate program) and four years in medical school while incurring hundreds of thousands of dollars in student loans — and all of this during a health care crisis that does not have enough physicians to meet the current demand.

As trauma surgeon Qaali Hussein once tweeted:

I don’t think I can ever advise another premed or med student without letting them know that they can get a whole ass medical degree and STILL not become a doctor. No one should take on this pursuit and all the debt without knowing this reality.

— Qaali Hussein, MD, FACS (@QaaliHussein1) March 17, 2022

The Match process is confidential. Candidates do not know where they stand in any program’s rank list, and the programs do not know where candidates rank them.

That sounds good, on the surface, but the lack of transparency between candidates and programs raises concerns about unchecked systems, especially when underrepresented applicants continue to go unmatched at higher proportions. While employers in the traditional job market can typically be held accountable for their hiring practices when there is a concern about discrimination, transparency and oversight between an applicant and an employer is lost in The Match. Residency programs place applicants on their rank list without disclosing their position or reason. This process can create a smokescreen that allows potential factors, such as structural discrimination, to go unchecked and perpetuates gatekeeping in medicine.

While demographic data on race/ethnicity are publicly available for applicants who match with a residency program, the NRMP does not share demographic data for those who go unmatched. In recent years, some have speculated on social media that a higher proportion of applicants from groups underrepresented in medicine go unmatched. The lack of publicly available demographic data for unmatched applicants makes it difficult to refute concerns about gatekeeping.

Gatekeeping results in weeding out applicants deemed “unfit” for a specific program or a career in medicine. Assessing for “fit” is rife with bias: On an individual level, it is based on likability. At the organizational level, it is a proxy for measuring alignment with institutional values. In reality, “fit” is a form of gatekeeping.

Gatekeeping also occurs when residency program leaders screen hundreds of applications to save time in deciding who they ultimately interview and rank. Induction into the Alpha Omega Alpha national honor society, a prestigious award in medicine, and scores on the United States Medical Licensing Examination are two key metrics for selecting or filtering out applicants, despite being shown to disadvantage applicants from underrepresented groups.

The language used in performance evaluations for underrepresented students has also been shown to be biased. Black students were less likely to be described as “outstanding” than their white counterparts. Additionally, residency interviews, described as an uncomfortable process for underrepresented applicants, affect their performance. The interview experience, along with the subjective nature of interviews, threatens their rank list position. These application components can further decrease the likelihood of underrepresented applicants of matching.

The Match algorithm favors the applicant’s list over the residency program’s. Applicants consider various factors in creating their rank list: a program’s training reputation, geographic location, faculty availability, clinical experience, and more. Residency programs’ rank lists are highly variable by the institution. Some program directors create their rank list by considering the recommendations of faculty and residents, others use holistic review, while others rank based on “fit.”

Data from the 2022 matching process showed that 47,675 applicants had registered for 39,205 residency positions. Unmatched applicants may choose to enter a process known as the Supplemental Offer and Acceptance Program (SOAP). Unmatched applicants from 2022 and previous matching cycles competed for just 2,262 unfilled positions in SOAP. Because the rules instruct unmatched applicants not to initiate contact with unfilled programs, many instead use their networks and social media to better their chances with any last-minute openings.

The important question is: Who are the applicants who are unable to find residency positions?

The National Residency Matching Program releases demographic data about who matriculates into residency programs, albeit with limited information on the race and ethnicity of unmatched applicants. There is also no available data regarding the demographic makeup of each institution’s rank list. Demographic data on who enters the SOAP is not available. However, the data are clear that Black and Hispanic applicants are less likely to matriculate into a residency program the same year they graduate when compared to their white counterparts.

No one knows — or at least no one is revealing — if applicants from underrepresented groups are not matching with residency programs and subsequently not getting hired because they are being interviewed less or end up lower in the rank list. To better advise applicants from underrepresented groups and improve representation in medicine, residency applicants, student advisers, and advocates need data to understand the structural barriers that prevent underrepresented applicants from matching. This can be done by making public de-identified, aggregated demographic information from program rank lists. Otherwise, diversity recruitment efforts during interview season, including social and second-look events, might only serve as performative.

Data from residency programs ranking lists could explain the perpetual underrepresentation of historically minoritized groups in medicine. As of 2021, only 5.7% of doctors in the U.S. identify as Black, though Black people make up 13.6% of the U.S. population, and there are similar gaps for Hispanic people (6.9% of doctors and 18.9% of Americans) and American Indian/Alaskan Natives (0.3% of doctors and 1.3% of Americans). Equitable representation can never occur if the rank list puts a disproportionate number of underrepresented applicants in “unlikely to match” and “do not rank” categories.

Our request for the Match demographic data for unmatched applicants is not novel. In 1987, the NRMP and the Association of American Medical Colleges jointly published demographic information, including race, on The Match placements for those who did not match. Since then, similar demographic data have not been released on Match outcomes. However, some specialties outside The Match, like ophthalmology, do report demographic data for matched and unmatched applicants. In 2021, the NRMP announced on its website that it would begin collecting demographic data for all applicants. While the NRMP recently released a brief report for matched applicants, we are still waiting for the data regarding unmatched applicants.

Further, no demographic data has been made public on where applicants from underrepresented groups fall in programs’ rank list for The Match in total or stratified by specialty.

It is essential for individuals who play roles on residency selection committees and help create rank lists to explore how unconscious bias and other factors might affect where applicants from underrepresented groups fall on their list. Accountability is needed to address the gatekeeping that upholds structural racism and bias in medicine. Making the right data available is a step in that direction.

Onyekachi T. Otugo is an emergency medicine physician at Brigham and Women’s Hospital in Boston and an instructor in emergency medicine at Harvard Medical School. Al’ai Alvarez (@alvarezzzy) is an emergency medicine physician at Stanford Health Care and is a clinical associate professor of emergency medicine at Stanford University. Adaira I. Landry is an emergency medicine physician at Brigham and Women’s Hospital and an assistant professor of emergency medicine at Harvard Medical School.

Editors’ note: This essay was updated after publication to clarify two paragraphs.