Annual Perspective 2014

Until very recently, diagnostic errors received relatively little attention in the field of patient safety, particularly when compared with other safety topics such as medication errors, surgical complications, and health care–associated infections. This is particularly surprising for a number of reasons. First, diagnostic errors have always attracted the attention of clinicians. Second, delayed or missed diagnoses constitute the most common (and costly) reasons for a malpractice claim, especially in the outpatient setting. Third, in hospital settings, a substantial literature highlights the frequency with which clinically important diagnoses escape detection until autopsy.

One explanation for the lack of attention paid to diagnostic errors may lie with the historic interest in the topic primarily from physicians. Nurses, pharmacists, and other health professionals and administrators have often showed greater engagement in patient safety than physicians, but correct diagnosis resonates more with physicians than other health care professionals. Perhaps more importantly, physicians who find diagnostic errors interesting tend to focus on the cognitive aspects of diagnosis—emulating master clinicians (real-life versions of television's Sherlock Holmes–like Dr. House) or studying cognitive biases that can lead one astray. This focus on individual diagnostic acumen and the psychology of cognition may not attract those interested in system problems and the Swiss cheese model of patient safety.

Whatever the explanation for the relative inattention to diagnostic errors within the field, a commentary in the Journal of American Medical Association in 2009 called for making diagnostic errors the next frontier in patient safety. The 5 years since then have seen a gradual increase in publications on the topic. In 2013, BMJ Quality & Safety devoted a special supplement to the subject. And, 2014 saw notable examples of additional traction, including the launch of a new peer-reviewed, open access journal, Diagnosis, published by the Society to Improve Diagnosis in Medicine, and the convening of a committee at the Institute of Medicine on Diagnostic Error in Health Care, which will produce a formal report in 2015. In this review, we summarize important 2014 publications on diagnostic errors.

Epidemiology: Scope of the Problem

A major advance in 2014 was the estimation of the frequency of diagnostic errors for the adult population across the United States by Singh and colleagues. By combining several data sources, including estimates from consecutive lung cancer cases, as well as a trigger approach in which automated screening of the electronic health record for specific patterns of return visits prompted detailed chart reviews, the researchers estimated that 5% of adults in the US (i.e., more than 12 million individuals) experience a serious diagnostic error in the outpatient setting every year.

Newman-Toker and colleagues used data from the AHRQ Healthcare Cost and Utilization Project to show that, among patients with stroke, nearly 13% of patients had a visit to the emergency department (ED) within the prior 30 days that constituted a potential missed diagnosis. Just over 1% of patients had a probable missed diagnosis of stroke. Common (incorrect) diagnoses assigned at the previous ED visit included headache and dizziness. Other studies in 2014 looked at the frequency of diagnostic errors in specific clinical settings, such as missed injuries in pediatric trauma. Another study reported that 8% of deaths at a Level 1 trauma center were definitely or potentially preventable, with inaccurate diagnosis one of the most common contributing factors to the deaths. These studies all lend weight to the long-standing argument that the patient safety field must attend to missed and delayed diagnosis.

Causes of the Problem

As noted at the outset, something about diagnostic errors strikes many clinicians and even patient safety advocates as reflecting individual performance, with less obvious roles for a systems approach to addressing the problem. An interesting pediatric study highlighted this issue. Investigators reviewed the records of medical patients admitted to the pediatric ward or seen in the ED at a United Kingdom district hospital over a 90-day period. Two clinician reviewers judged misdiagnosis to have occurred in 5% of these children presenting with acute illness. The two reviewers identified cognitive factors as the cause of most diagnostic errors. Interestingly, investigators were able to interview the treating clinicians in 14 cases. In contrast to the reviewers, clinicians commonly identified organizational problems such as poor policies and inadequate staffing and equipment, making clear that the problem of diagnostic errors also has a systems component.

An analysis of outpatient physician focus groups similarly saw clinicians attribute diagnostic errors to a variety of system factors, including insufficient information availability, disjointed workflows, and poor communication among providers and with patients. Naturally, one might expect clinicians to preferentially attribute diagnostic errors to system problems rather than individual cognitive errors. But an interesting study in radiology highlighted the concrete role that interruptions can play in diagnostic accuracy, in the same vein that other patient safety studies have found that interruptions increase the chances of medication administration errors by nurses.

This study sought to assess the effect of distractions from telephone interruptions on diagnostic performance by radiology residents during 14-hour shifts on-call. A local quality assurance project in this radiology department at a tertiary care pediatric center tracked discrepancies between the diagnostic findings of the on-call residents and the final interpretation of staff radiologists. Major discrepancies occurred in 41 shifts over a 13-month period. Using telephone logs from the radiology reading room, the authors found a slightly greater average number of telephone calls for the "discrepancy shifts" than the "no discrepancy shifts" (48 vs. 44). While this difference was only of borderline significance (P = .057), the study also showed a statistically significant increase in the average number of phone calls in the 1 hour preceding the generation of a discrepant preliminary report versus the "no discrepancy shifts" (4.23 vs. 3.24 calls, P = .027). One additional phone call during the preceding hour was associated with a 12% increased likelihood of error (P = .017).

Even with documented examples of human factor contributors like interruptions, and other system contributors to diagnostic errors (e.g., missed test results, poor communication between providers, poorly designed computer interfaces), cognitive biases remain a fascinating area of study. One study in 2014 looked at the effect on diagnostic performance of distracting salient clinical features—findings that tend to grab physicians' attention because of a well-known association with a particular disease. Participating internal medicine residents (72 in total) each reviewed 12 case vignettes—with or without a salient distracting feature. The authors structured the cases such that some were simple and some complex, and so that the salient distracting feature appeared either near the beginning of the case description or closer to the end. Significantly more diagnostic errors occurred with salient distracting features, especially for complex cases, but only when the distractor appeared near the beginning of the case description.

Another study in 2014 showed that recent media exposure to information about a particular disease can bias physicians toward making that diagnosis in similar looking cases. This study highlights the importance of availability bias. Importantly, the authors showed that instructing the participants to reflect further restored performance to the level seen in participants not exposed to the information about a specific disease. This restoration of performance represents a simple example of cognitive debiasing. It also illustrates the augmentation of performance that occurs when clinicians shift from automatic to more reflective thinking. Another 2014 study highlighted the increased diagnostic accuracy that comes from shifting out of the fast, intuitive mode of thinking to a more reflective one. This study of computer keystrokes by residents taking a national certification examination showed that, for cases involving diagnosis, changing an answer from incorrect to correct occurred nearly twice as often as changing an answer from correct to incorrect.

Addressing the Problem

As intellectually appealing as cognitive debiasing may be, addressing systems problems contributing to diagnostic error still seems more likely to deliver demonstrable gains in safety and to serve as a feasible target for interventions and policies. The year 2014 saw one notable report of how two separate health care organizations addressed such system challenges. One system deployed voluntary reporting and root cause analysis for diagnostic errors; another created a system of review for incipient diagnostic delays using the electronic health record. Both strategies yielded promising early results.

Summary

Taken together, the literature published in 2014 represents a significant advance in our thinking about diagnostic errors. Importantly, research advances have put us in a position in which we may soon be able to measure the frequency of diagnostic errors in an individual institution in reproducible ways. Further standardization and pilot implementation will be needed in order to be able to create measures that will permit the application of tools such as comparative performance review, differential payment, and accreditation to prompt greater individual and institutional focus on this crucial area of patient safety. Such strategies have become the cornerstone of policy initiatives in the rest of patient safety; the absence of feasible measures of diagnostic accuracy has left diagnosis behind in the policy arena.

Even in the absence of rigorous and standard measures, 2014 provided actionable information about how to address diagnostic safety: using electronic health records and system-wide workflows to intercept diagnostic problems prior to patient harm, applying established patient safety strategies such as reporting and error analysis, and by teaching techniques to improve cognition. The focus of the patient safety field on diagnosis over the past 5 years appears to be bearing fruit.

Internet citation: Sarkar U, Shojania K. Annual Perspective 2014: Diagnostic Errors. March 2015. AHRQ Patient Safety Network. https://psnet.ahrq.gov/perspective/diagnostic-errors 

Annual Perspective 2014

As the patient safety field has grown, so too has the appreciation for the need to improve safety in medical education. Efforts to do so have generally focused on two main areas: incorporating formal training in patient safety into medical education, and examining and improving the safety of care at teaching hospitals.

The 2010 report Unmet Needs: Teaching Physicians to Provide Safe Patient Care by the Lucian Leape Institute at the National Patient Safety Foundation emphasized the importance of both of these areas. In the area of curricular change, it highlighted the need for medical schools to modernize their curricula to better prepare physicians. The report called on these schools to shift from the traditional curricular focus on medical knowledge and technical skills to a curriculum integrating systems analysis, quality improvement, and patient-centered care. In the area of changing the clinical setting in which so much of medical training takes place, the report emphasized the importance of a culture of safety in teaching hospitals, stressing that unprofessional behavior and authority gradients prevent trainees from reporting and learning from errors. It also called for rigorous new evaluation methods to be developed and implemented in both medical schools (undergraduate medical education [UME]) and residency programs (graduate medical education [GME]).

There were significant advances in these areas in 2014, as well as the publication of new evidence evaluating the effect of duty hour restrictions on safety and medical education. In this Annual Perspective, we will review this new evidence and other key trends in the field of safety and medical education.

Improving Instruction and the Culture of Patient Safety in Teaching Settings

Medical schools are now required to provide basic instruction in patient safety for students and residents in order to meet accreditation standards. While many UME and GME programs have implemented didactics or online courses to meet these requirements, the real-world "hidden curriculum" that trainees experience—such as unprofessional behavior or failure to appropriately report or disclose errors—is much more powerful in shaping trainees' behaviors. In response, the Accreditation Council for Graduate Medical Education (ACGME) recently created the Clinical Learning Environment Review (CLER) program to evaluate teaching institutions in six focus areas: patient safety, quality improvement, transitions in care, supervision, duty hours, and professionalism. The intent of this program is to evaluate the degree to which trainees' work environment supports their ability to provide safe and high-quality care. The CLER Pathways to Excellence report describes discoveries from the first year of the program, including the finding that residents and fellows are often disengaged from institutional safety programs and rarely voluntarily report errors. A related commentary identified faculty development as the rate-limiting step in improving the culture of safety at teaching hospitals, as many (if not most) teaching faculty also lack facility with basic safety concepts. In response, the Association of American Medical Colleges has developed the Teaching for Quality program to develop a cadre of medical educators certified in teaching patient safety and quality improvement.

Improving the culture of safety at teaching institutions will also require that efforts are taken to eliminate disruptive and unprofessional behavior from the workplace. The culture of disrespect that pervades many institutions poses a major barrier to improving safety in general and is particularly damaging to trainees. At the institutional level, studies published this year demonstrated the effectiveness of a systematic approach to identifying physicians with repeated behavioral transgressions and implementing a structured approach for remediation. While overtly unprofessional behavior is easily detected (albeit difficult to address), students and residents are often exposed to more subtle harmful behaviors, as exemplified by a study showing that residents frequently witness poor role-modeling behavior by supervisors in disclosing errors to patients. Regulatory efforts such as the CLER program are now highlighting these deficiencies, and teaching institutions will need to continue to implement interventions to develop and maintain a culture of safety.

Evaluating Patient Safety Competencies Among Trainees

The Lucian Leape Institute's Unmet Needs report also called for development of evaluation systems for trainees based on the achievement of developmental milestones in patient safety—specific safety skills that trainees must demonstrate in order to be considered ready for unsupervised practice. In 2014, these evaluation systems were developed and implemented for both medical students and residents. At the UME level, the Association of American Medical Colleges has defined the ability to "identify system failures and contribute to a culture of safety and improvement" as one of the 13 core skills that all medical students should have achieved by the time of graduation. The ACGME has also developed patient safety milestones for all GME training programs—including residency programs and subspecialty fellowships—that were implemented for some fields this year and will be fully implemented by July 2015. For example, internal medicine residents are now required to demonstrate that they can recognize system error and advocate for system improvement through identifying systemic causes of medical error and navigating them to provide safe patient care and reflecting upon and learning from critical incidents that may lead to medical error.

The development of patient safety–specific milestones is important, both because of its inherent value and because it places the ability to provide safe care on the same level of importance as the acquisition of medical knowledge. Practicing physicians will also be held to the same standards, as the American Board of Medical Specialties has recently implemented standards for maintenance of certification (MOC) across many specialties. The MOC program requires that physicians demonstrate continued competence in patient safety after completing training and attaining board certification, providing additional justification for the need to inculcate trainees with these skills.

Resident Duty Hours Reductions

The most prominent intervention designed to enhance safety at teaching hospitals—the 2003 regulations restricting resident duty hours—has engendered controversy since its inception. Previous systematic reviews have found no improvement in clinical or patient safety outcomes associated with the regulations and mixed effects on educational outcomes. Further regulations on duty hours for first-year trainees were implemented in 2011, and 2014 saw publication of longer-term outcomes of the 2003 regulations and early results examining the effects of the 2011 regulations.

A systematic review of the impact of duty hour restrictions on surgical resident training found no improvement in patient outcomes or resident education associated with either the 2003 or 2011 regulations, and mixed effects on resident well-being. Another single-site study found no improvement in patient safety outcomes after implementation of the 2011 duty hour regulations. What's more, a commentary by two noted patient safety experts raised concern that further restriction of duty hours has led to decreased autonomy for residents and limited their ability to make decisions independently. These concerns were partially allayed by another study that compared the clinical outcomes of patients cared for by physicians who completed training after duty hour regulations to patients cared for by physicians who completed training prior to 2003. No difference was found in hospital mortality or length of stay between the two groups.

In response to continuing concerns about the impact of duty hour restrictions on trainee and patient outcomes, the ACGME has provided funding support for the iCompare study, a cluster-randomized trial that will compare the current regulations to a duty hour structure that preserves the 80-hour weekly work limit, but allows flexible schedules otherwise. The iCompare study enrolled participants in 2014 and will begin in 2015. The results of this study should help inform the optimal duty hour structure in order to ensure patient safety while maximizing resident education and well-being.

Summary

In summary, one of the major threads of the patient safety movement was rethinking many key tenets of medical education. In 2014, we saw the maturation of several programs that have been implemented in response to this important movement. In the case of duty hour restrictions—now more than a decade old—empirical studies testing whether the new standards have met their goals have yielded ambiguous results. They have also identified some potential unintended consequences. In the case of the more recent CLER program, empirical results are likely a few years away. The literature is offering some early themes and areas for teaching programs to focus on in the coming years. Whatever the results of specific initiatives, it is clear that the patient safety movement has targeted medical education as an area in need of new thinking and programs, and this trend is unlikely to change in the next few years.

Internet citation: Ranji S. Annual Perspective 2014: Safety and Medical Education. March 2015. AHRQ Patient Safety Network. https://psnet.ahrq.gov/perspective/safety-and-medical-education