In 1999, the American Board of Internal Medicine and Association of Program Directors in Internal Medicine defined a “problem resident” as a “trainee who demonstrates a significant enough problem that requires intervention by someone of authority, usually the program director or chief resident.” This definition was necessary to explicitly identify an emerging concern in postgraduate medical education. Historically, residents with deficiencies went unrecognized and untreated, resulting in the potential graduation of countless residents who were ill-prepared for safe, independent practice. “Problem residents” were thought to be threatening to their medical training, professional development and career satisfaction. There was also concern that these residents were negatively influencing the educational experience of their peers, reducing optimal team functioning within the program, and producing additional work for program directors and their offices. The hope was that by naming the problem, programs could pinpoint residents with deficiencies and effectively remediate them during residency.
To date, there is no standardized, evidence-based approach to help residents succeed. Residents who are underperforming go through a trial and error process where various methods, ranging from increased time spent studying to co-managing patients with another resident, are implemented at the discretion of the residency program. Interventions can stigmatize and embarrass a struggling resident who now has to perform under increased scrutiny while being evaluated using untested, subjective measures. This approach is particularly challenging for physicians with intellectual or physical disabilities who are already a marginalized and underrepresented group in medicine. Many physicians do not disclose a disability for fear of jeopardizing their career. Those that do may not get appropriate, if any, accommodations, making it difficult to tease out how much of the “deficiency” is due to having a disability. So, is the current system infringing on the civil rights of doctors with disabilities?
Initially, programs relied on informal faculty and staff complaints to identify at-risk residents. Sometimes residents were discovered after a sequence of hopefully minor incidents. In 2013, the Accreditation Council of Graduate Medical Education (ACGME) officially launched the ACGME Milestones, a formative, continuous quality improvement metric aimed at improving trainee assessments in graduate medical education. The ACGME wanted to hold graduate medical programs accountable to the public for honest evaluations of resident performance and for truthful verification of their readiness to progress to unsupervised practice. The milestones necessitated routine assessments through a combination of direct observations, global evaluation, review of clinical performance data, multisource feedback from peers, nurses, patients and families, simulation, in-service training examinations and self-assessments. Program directors were to use these objective measures to determine if a resident could move forward with their training or needed to remediate. The rollout has had moderate success. Milestones are noted to be better at identifying problem residents compared to earlier tools. There was concern that the quality of implementation and the context in which the tool was implemented affected trainees scores. The milestones rely on the evaluators’ subjective judgments of personality, motivation or character instead of objective performance criteria. Researchers have also reported variability within observers’ rating processes and in the translation of the observation to numeric scores. As evaluations are collected from a wide array of observers, per ACGME requirements, the inter- and intra-rater variability can significantly alter how a resident is assessed and how their progress is monitored.
Recent focus groups of program directors and associate program directors revealed frustration stemming from a lack of faculty development directed at evaluation of struggling residents and methods to provide constructive and formative feedback. The ACGME has already referred to this rollout as ‘Version 1.0‘ with iterative versions incorporating the feedback generated. Once problem residents are identified, programs do not know what to do with them. There are different models of remediation that were all derived from opinion papers authored by a small group of experts which restricts their generalizability. These models also do not account for outside factors that may contribute to a resident’s struggle, such as family stressors or disability. Furthermore, there is no data supporting the efficacy of any of these models.
Once remediation is initiated, using any of the proposed models or an ad-hoc model created by the program, the milestones metric is unable to measure accurately or precisely the efficacy of any intervention. The irregularities are likely due to the complexity of the medical environment and the myriad of interacting variables, thus increasing the possibility of misguided conclusions being reached about trainee performance. Everything is further complicated because the criteria for promotion or competence remains ambiguous at best. A competency-based milestones system inherently creates tension within the current time-based graduate medical education system. Residents who are not on track to meet their milestones are prominently identified as needing remediation. As such, only half of the program directors in one survey were confident of their programs ability to manage problem residents. The most unfortunate consequence of this assessment could be how it is negatively affecting doctors with disabilities.
Despite 20% of the US population having a disability, only 0.56% of medical students who enrolled between 2001 and 2010 self-identified as living with a disability. The cost of accommodations and the rigid technical standards of medical schools are usually cited for this stark disparity. Once enrolled, only 2.7% of these medical students sought any accommodations, per the Association of American Medical Colleges (AAMC). The number of trainees who disclose disabilities declines further at the graduate level. It is not hard to appreciate why trainees with disabilities may feel vulnerable in medical institutions which originated and promoted the Eugenics movement and widespread institutionalization of disabled individuals in the not so distant past. To date, the medical model of disability emphasizes an individuals’ physical or mental deficit instead of highlighting the barriers that exclude people with disabilities, thus creating the permissive structure for broader social and cultural discrimination.
The AAMC has acknowledged that the fear surrounding disclosure of disability is genuine. Residents do not want to face prohibitive medical inquiries or garner unwarranted negative attention. Research suggests that most will not disclose due to fear of judgment, bias or skewed perception of ability. Given that Medicare funding per resident has been capped at 1996 levels, residents with disabilities remain a low priority for filling resident slots due to the extra costs programs may incur to provide accommodations. Fear of disclosure has been reinforced in recent years. Individuals with cognitive and learning disabilities and other executive functioning disorders may present with weakness in social competence and relationship management, decreased capacity for self-reflection and insight, a lack of vigilance, an inability to adapt to rapid changes, distractibility, a failure to prioritize activities, and even periods of hyper-focusing. Each of these symptoms is considered to be deficiencies per the ACGME Milestones. Residents now worry that by self-identifying or by asking for accommodations, they are inviting increased scrutiny of the symptoms of their disease, something they may not be able to control. This fear is so pervasive that researchers have found that two-thirds of trainees with disabilities do not seek support despite experiencing disability-related difficulties in their training.
Of the few residents that did ask for accommodations, many did not find a transparent process for their request. Some got the impression that, even if available, the resident would be responsible for the cost of the accommodations. Others noted that even after formal request, accommodations were never provided. In a recent report on physicians with disabilities, the AAMC acknowledged that residency programs did not have a clear understanding of their institutional obligation to provide accommodations and implement them promptly. Nor, in some cases, did they seem to work with any centralized accommodation specialist for guidance with implementation despite the Equal Employment Opportunity Commission suggesting that reasonable accommodations should be made available. Examples include possible time off, granting breaks, modified or part-time schedules (often resulting in extending program length), allowing medical treatment or counseling during the work day, assistive or adaptive technology, removing architectural barriers, and increased supervision and guidance.
Residents with disabilities, either diagnosed before or during residency, who must remediate need to determine if the deficiencies noted in their evaluations are secondary to their impairment. If so, they have to seek accommodations from programs who may not be inclined or have the resources to provide them. As such, residents must try to show improvement based on evaluation tools that do not take into consideration any disabilities, are not validated to track progress over time, and are impacted by subjective bias. Additionally, residents have likely agreed to a remediation plan not suited to their needs, making their chance of success uncertain. Disability advocates assert that the current assessment and remediation model may be disproportionately affecting members of a protected class of individuals and is encouraging extra scrutiny and considerations for the discipline of these individuals. However, the current model does not place other groups or individual physicians within the same punitive context. This model also creates a system that encourages hypervigilance and demands members of the profession be on the lookout for physicians on remediation, thus increasing the isolation and stigma of these residents.
If disability cannot be untangled from the definition criteria of a “problem resident,” then the current model of graduate medical education does not appear to comply with Title I of the Americans with Disabilities Act. This act prohibits discrimination against a qualified individual on the basis of disability with regard to job advancement and training. Hospital residency programs are also governed by both Title IX and Title VII of the Civil Rights Act of 1964, which prohibits employers and educational programs from discriminating against certain protected classes. Therefore, hospital policies and training that disproportionately affect a particular group can amount to a systemic infraction against civil rights.
The medical community needs to reconsider how trainees are evaluated. It is of utmost importance that there be a way to judge clinical competency without marginalization of entire groups. The unnecessarily rigid “technical standards” of medical schools have long excluded disabled individuals from entering the medical field. Only now are these technical standards beginning to change. The current model of resident assessment threatens to undo the minimal gains that decades of advocacy have brought and further discourage individuals with disabilities from pursuing medicine. Unfortunately, I do not have any changes to propose, nor do I know where to begin. I do know that as a community we are not talking about this issue enough. There needs to be more debate, more research and more dialogue with individuals with disabilities and their advocates if medicine wants to become a profession that is accessible to all.
Image source: Stethoscope by Rosmarie Voegti licensed under CC BY 2.0.
Resident Physician Contributing Writer
Boston Medical Center
Syed is a medicine resident who is interested in health policy and health administration. He primarily looks at diversity and inclusion and leveraging them to create a medical workforce that can provide higher quality and better access to care for uninsured and underinsured populations. Syed wants to work on pipeline programs and on recruitment, retention and promotion of underrepresented minorities in medicine and also on creating medical school and residency curriculum that frames medical education in a social justice contest and addresses the intersection of race, sex and gender and medical sciences.
