Background

An essential aspect of preventing medical errors and improving patient safety is using data effectively to understand, track and communicate performance on patient safety metrics. Tracking performance helps one identify patient safety hazards, understand baseline performance, compare patient safety between health care sites, determine effectiveness of patient safety interventions, and assess if changes in performance are sustained. These indications for tracking data can help clinicians to select the most appropriate tool.¹

There are several approaches to tracking improvements, such as summary statistics, statistical tests, graphs, and tables. Visual methods of tracking time-series data enable stakeholders and improvers with a range of backgrounds, professions, and skillsets to understand and learn from data. Visual methods also provide a more comprehensive understanding of the performance and context of a process.²

Visual Tools for Tracking Patient Safety Data

Two widely used tools for visually tracking and sharing patient safety data are run charts and control charts. Other tools that can be utilized are histograms and scatter plots. These tools quickly and clearly convey data, provide insight into system-level issues, and help the viewer understand variation and changes over time.²

Histogram

A histogram is a bar graph that shows frequency distributions and is an easy way to evaluate variation in data. It graphically depicts the frequency with which various values of a variable occur in a set of data. The heights of the bars show the frequency of occurrence. A histogram allows improvement teams and stakeholders to evaluate patterns in performance and visualize the distribution of data. Superimposed or adjacent histograms can be used to compare two time periods to evaluate changes in performance. Histograms can also help users identify potential improvements in processes of care.^1,2

As an example, Figure 1 shows a histogram that depicts the number of days after hospital discharge that discharge summaries were sent to primary care providers in a health system. Discharge summaries are a key aspect of inpatient-to-outpatient clinician communication and are ideally received by the primary care provider prior to the patient's first post-hospitalization appointment. Delay in receiving timely discharge summaries is a patient safety issue because it significantly increases risk of post-hospitalization adverse events and hospital readmission for patients. As the histogram shows, several discharge summaries took more than a week to be sent to primary care providers and there is room for improvement in the timeliness of inpatient-to-outpatient communication after hospital discharge in this health system.

Figure 1: Histogram tracking number of days post-hospitalization that discharge summaries were sent to primary care clinicians.

Scatter Plot

A scatter plot (or scatter diagram) graphs pairs of numerical data, one variable on each axis, to look for a relationship. A scatter plot graphically depicts the relationship between two variables and helps users to recognize correlations.^1,2

For example, Figure 2 is a scatter plot that shows the relationship between the age of a patient and the frequency of use of “teach-back” methods in a clinic. Teach-back is a strategy that involves the health care provider asking patients or family members to explain health care instructions in their own words to confirm that they clearly understood these instructions. Thus, teach-back addresses a key issue in medication safety in the home environment. This diagram clearly demonstrates that the frequency of “teach-back” in this clinical setting decreases with patient age.

Figure 2: Scatter plot of use of teach-back vs patient age

Run Charts

A run chart is a standard quality tool used to display trends in patient safety-related structure, process, or outcome measures over time.  Run charts are useful in discovering patterns that occur over time. In a run chart, data are displayed over a specific period in consistent units (days, weeks, months, years) to show whether process changes are leading to improvements.³

Figure 3 depicts a run chart showing Central Line Associated Blood Stream Infection (CLABSI) rates over a period of two fiscal years in a hospital. Healthcare-associated infections such as CLABSI are an important patient safety outcome and result in significant morbidity and mortality, as well as increased health care costs.

Each blue point is the CLABSI rate measured each month. This graphical representation helps us visualize the month-to-month variability in CLABSI rates at this hospital. To answer the question, ‘are the CLABSI rates changing during these 2 years?’, we look at the trend depicted by the dashed yellow line. The graph conveys that CLABSI rates are declining, but to assess whether CLABSI rates have fallen enough, we use a technique called benchmarking. Benchmarking is when a ward or hospital’s measure of quality, such as the CLABSI rate per 1,000 central line-days, is compared to rates either internal or external to the organization.

In this run chart, the external standard is the National Healthcare Safety Network (NHSN) mean CLABSI rate as shown in the solid red line. The run chart shows that this hospital’s CLABSI rate has been consistently greater than the external standard. Therefore, although the institution's CLABSI rates are falling, there is room for improvement.

Annotated run charts include the timepoints when patient safety interventions were introduced or when relevant events occurred, and help the viewer understand context.

Figure 3: Run chart tracking Central Line Associated Blood Stream Infections in a hospital

Control Charts

A control chart is a graph that can be used to visualize how a patient safety-related measure changes over time, with a focus on consistency. Control charts help viewers distinguish common cause variation (due to expected or predictable causes that are inherent in the system) from special cause variation (due to unexpected and unpredictable causes that occur due to specific circumstances).^1,2

Control charts specify an upper control limit (UCL) and a lower control limit (LCL) that are graphically shown above and below the line depicting the mean. The UCL and LCL are commonly set at 3 standard deviations from the mean. Data points within the UCL and LCL suggest common causes, whereas data points beyond these limits suggest special causes.⁴

The control chart in Figure 4 shows the variation in CLABSI rates in a hospital over time. By focusing on peaks in CLABSI rates, we can look for and eliminate sources of variation. Specifically, the control chart highlights which peaks suggest out-of-control processes rather than expected variability that we would expect for CLABSI. The dotted red line is the UCL representing 3 standard deviations above the mean.

Points falling above the upper control limit indicate months when the process was out of control. In this example, CLABSI rates in May-20 and Feb-21 were above the UCL, prompting an evaluation of the structures and processes in the hospital during these months that may have contributed to the increase. The hospital found that in May-20 it ran out of protective line caps for 2 weeks and as a result of this missing component of the central line maintenance patient safety bundle, CLABSI rates increased.

Figure 4: Control chart tracking Central Line Associated Blood Stream Infections in a hospital

In summary, tracking patient safety measures helps health care organizations understand their baseline performance, assess the effects of patient safety interventions, and evaluate whether changes in performance are sustained over time. Histograms, scatter plots, run charts, and control charts are graphical tools to track, visualize, and communicate improvements in patient safety and to learn from variations in data. The organization’s goals for tracking patient safety data can help determine which graphical tool would be most appropriate to its needs.

Ulfat Shaikh, MD, MPH, MS
Professor of Pediatrics, University of California Davis School of Medicine
Medical Director for Quality
UC Davis Health

References

1. Langley GJ, Moen RD, Nolan KM, et al. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. 2nd ed. Hoboken NJ; Jossey-Bass: 2009.
2. Institute for Healthcare Improvement. Quality Improvement Essentials Toolkit. ​​​Boston, MA: Institute for Healthcare Improvement; 2021. http://www.ihi.org/resources/Pages/Tools/Quality-Improvement-Essentials-Toolkit.aspx
3. Perla RJ, Provost LP, Murray SK. The run chart: a simple analytical tool for learning from variation in healthcare processes. BMJ Qual Saf. 2011; 20(1):46-51. [Available at]
4. Fretheim A, Tomic O. Statistical process control and interrupted time series: a golden opportunity for impact evaluation in quality improvement. BMJ Qual Saf. 2015;24(12):748-752. [Free full text]