Description of the Innovative Activity

The Innovation Team knew that it would need to build a robust real-time dashboard to monitor the A-F bundle. This would allow the multidisciplinary clinical team to better respond to patient data and improve total A-F bundle compliance, and thus improve patient outcomes. The Innovation Team was interested in learning whether increasing staff’s data literacy affects bundle compliance. Some of the data literacy gaps that the Innovation Team noticed were related to staff understanding of data reports and their ability to read or interpret performance reports. Further, the Innovation Team observed that the clinical team needed coaching in how to use the A-F bundle in response to a poor performance metric and in order to see positive changes in performance metrics (e.g., improvements in compliance rates, shorter patient length of hospital stay, better patient mortality outcomes).

The Innovation Team found that working with the IT analysts and having access to the backend of the EHR was essential. It took about four months of ongoing collaboration between data analysts, IT staff, and clinicians working in the same room to build what the innovator described as a “bridge of knowledge.” This bridge of knowledge connects clinical practice with IT data infrastructure so that reports are built correctly to measure and track clinical practice. This involved drawing out algorithms and discussing clinical improvements and logic constructs to enable the EHR to have the correct fields and for the performance reports and dashboard to pull data effectively. First, the innovators built the dashboards, which were external to the EHR. They began the process by extracting data from a data warehouse, and then they built the performance reports in Tableau to distribute to clinical staff. Later, when the EHR vendors started working with SCCM, they created out-of-the-box dashboards that the Innovators began customizing to their needs and still use today. After building the new dashboards, which gave clinicians access to real-time data and allowed them to administer the clinical education about the A-F bundle, the Innovation Team created and delivered a simple, streamlined, ongoing data literacy training via an ongoing coaching conversation. It fosters performance data report comprehension in about 15 minutes in day-to-day casual conversation format over several sessions.

Context of the Innovation

Medical errors and adverse events lead to poor patient safety outcomes, but many are preventable.² A medical error is the failure to complete a planned action as intended, or the use of the wrong plan to achieve an aim. An adverse event is any injury caused by medical management rather than the underlying disease.³ Research shows that medical errors and adverse events occur more frequently in the intensive care unit (ICU) than in many other settings.^2,4 This may be because ICU patients often (1) take numerous medications, (2) have complex conditions that require more testing, (3) have more interventions, and (4) have invasive procedures; this complexity requires additional treatment decisions.²

The complex care provided in the ICU increases the risk of mistakes and infections, and it creates more opportunities for staff to overlook test results, miss a diagnosis, or make an error in medication administration—all of which can increase the chance that patients will experience complications of care.^3,4 Medical errors or adverse events can lead to worse patient outcomes including longer lengths of stay and higher mortality rates.⁴ One study found that experiencing two or more adverse events was associated with a threefold increase in the risk of death in the ICU.⁵ Another study found that staff in critical care environments often have to make high-risk decisions and sometimes do so with incomplete data.² The A-F bundle is complex and challenging to fully implement; there is room for quality improvement efforts to improve compliance through processes that positively effect patient outcomes in the ICU.

Results

Overall, the Innovation Team saw improvements in both the intervention and control groups over time, with larger improvements seen in the intervention group. These improvements included lower next-day in-ICU mortality, lower next-day in-hospital mortality rates, increased likelihood of next-day extubation, and more patients being discharged to home.

The study used a quasi-experimental design with four ICU units in the intervention group and four unmatched ICUs in the control group. ICUs differed in the primary diagnosis of their patient populations, patient case mix index (CMI), duration of patient ventilation hours, and the percentage of patients from the ICU who were discharged directly to home.² Improvements in the control group were considered spillover effects because some staff worked across both intervention and control group units.¹ The Innovation Team noted that bundle compliance rates decreased in the first four months of the COVID-19 pandemic for both intervention and control groups, but compliance rates subsequently recovered after just two months of the implementation of the intervention, with only clinical education delivered to staff, the compliance rates improved from 9% to 16% (adjusted odds ratio [AOR]=2.48; 95% confidence interval [CI], 1.79–3.43; p<0.0001) for all ICU units.¹ After four months with the addition of data literacy trainings and the weekly performance reports, intervention compliance rates improved from 16% to 21% (AOR=1.52; 95% CI, 1.05–2.22; p=0.03) for all units.¹ The improvement in compliance rates weas associated with improvements in patient outcomes by next-day analysis.¹

Bundle compliance over the study period showed larger improvements for the intervention group (p<0.001) compared with the control group (p=0.004). In the intervention group, full bundle compliance was associated with a lower likelihood of in-ICU mortality the next day (AOR=0.57 [0.36–0.89]; p=0.01) and lower next-day in-hospital mortality (AOR=0.59 [0.40–0.87]; p=0.01). However, full compliance had no effect in the control group.¹ Full compliance was associated with an increased likelihood of next-day extubation for the intervention group (AOR=1.56 [1.32–1.86]; p<0.0001) and the control group (AOR=1.79 [1.39–2.31]; p<0.0001). There was no association between compliance and being delirium-free the next day in either group. In intervention units, compliance was associated with a 16% lower likelihood of next-day ICU transfer to a general unit (AOR=0.84 [0.73–0.96]; p=0.01) and a 16% lower likelihood of next-day hospital discharge (AOR=0.84 [0.73–0.96]; p=0.01), but this was not the case in the control group. There was an association between compliance and a higher likelihood of ICU survivors being discharged to home in the intervention group (nonhome discharge AOR=0.68 [0.48–0.97]; p=0.03) and in the control group (discharge AOR=0.40 [0.26–0.62]; p<0.0001).¹

The Innovation Team conducted a follow-up survey of its ICU staff to inquire about the perceived strengths and weaknesses of their implementation of the A-F bundle. The team found that data literacy training, performance monitoring, organizational support, multidisciplinary collaboration, interprofessional teamwork activities, and making clear the value of the intervention to get staff buy-in were essential to successful implementation.⁶ Staff also noted that, after the data literacy training, their confidence in their ability to read performance metrics reports increased by 21%.⁶

Planning and Development Process

Key steps in planning and implementing the innovation include:

• Seeking organizational leadership support to establish the A-F bundle as the standard of care and to make it an organizational priority
• Building partnerships across multidisciplinary departments (e.g., clinical operations business intelligence, value improvement, and IT/data analysts) and ICU clinical team staff (e.g., nurses, physicians, NPs, respiratory therapists, physical therapists, social workers, case management staff, and pharmacy staff)
• Establishing leaders in each department and among each clinical team discipline (e.g., nurses, physicians, NPs, respiratory therapists, physical therapists, social workers, case management staff, and pharmacy staff)
• Designing role-based clinical education for the A-F bundle
• Planning for the implementation of dashboards displaying performance metrics, weekly reports, and ongoing data literacy trainings for bedside clinicians
• Incorporating cross-discipline collaborations to cover any gaps in complete compliance with the bundle to ensure that all handoffs and clinical collaborations are effective
• Including interprofessional simulation and role-play as part of task simulation to build the teams’ confidence in implementing the A-F bundle together

Resources Used and Skills Needed

Important components of the innovation include the following:

• Information technology dashboards. Cerner and EPIC now have some out-of-the-box dashboards, which the Innovation Team customized.
• Clear instructions, provided in data literacy trainings, describing which EHR fields to populate to decrease variability in documentation practices and improve data quality.
• Scripts and tools to facilitate ongoing data literacy training (e.g., daily use of a huddle sheet for clinical rounds that integrate the evidence-based clinical practice of the A-F bundle elements in the head-to-toe assessment).
• Data algorithms for accurate and timely dashboards, and performance metrics reports in alignment with clinical practice and protocols.
• Role-based leaders with relevant experience and subject matter expertise (e.g., nurses, physicians, nurse practitioners [NPs], respiratory therapists, physical therapists, social workers, case management and pharmacy staff). Staff design role-specific clinical educational materials about their A-F bundle elements, as each specialty focuses on certain bundle elements.
• Efficient communication and shared responsibility for patient outcomes between daytime and evening shift staff.
• Ability to collaborate across clinical staff and IT staff/analysts to contribute their respective expertise in joint work sessions.
• Timely and correct administration of appropriate clinical techniques in response to reading performance metrics reports to improve bundle compliance and patient outcomes.

Funding Sources

There were no additional funding sources.

Getting Started with This Innovation

Organizations interested in implementing this innovation should garner leadership support to drive clinical and IT engagement and collaboration. Next, without existing performance reports, ensure clinical objectives are translated into performance metrics that meet the technical design. Both IT and clinical staff should share a common understanding of needed EHR fields, and report outputs must be tested to ensure formulas yield accurate and reliable results. The implementation team should establish measurable goals and set expectations for clinical performance. Implementors should coach the clinical team to consider actions they can take to improve A-F bundle compliance rates and patient outcomes. This can be part of building and delivering a structured data literacy training program to engage bedside staff in ongoing conversations about reading and interpreting performance reports. These conversations should review the performance metrics data, address gaps, and discuss applicable clinical interventions to make observable improvements in the performance metrics.

Sustaining This Innovation

Keys to Sustaining This Innovation:

• Ongoing practice integration at the bedside with clinical educators and nurses/clinicians reviewing A-F processes, executing them, and reflecting on performance metrics
• Interprofessional team collaborations about A-F total bundle compliance, gaps in compliance, and quality improvement opportunities in staff handoffs
• Incentives (e.g., team celebrations) for bundle element compliance, total bundle compliance, and friendly competition/gamification between units to keep a focused excitement on improving compliance ratings

References/Related Articles

Brown JC, Querubin JA, Ding L, et al. Improving ABCDEF bundle compliance and clinical outcomes in the ICU: randomized control trial to assess the impact of performance measurement, feedback, and data literacy training. Crit Care Explor. 2022 Apr 21;4(4):e0679.

Brown JC, Ding L, Querubin JA, Peden CJ, Barr J, Cobb JP. Lessons learned from a systematic, hospital-wide implementation of the ABCDEF bundle: a survey evaluation. Crit Care Explor. 2023 Nov 9;5(11):e1007.

Footnotes

1. Brown, J. C., Querubin, J. A., Ding, L., Mack, W. J., Chen-Chan, K., Perez, F., ... & Cobb, J. P. (2022). Improving ABCDEF bundle compliance and clinical outcomes in the ICU: Randomized control trial to assess the impact of performance measurement, feedback, and data literacy training. Critical care explorations, 4(4).
2. Rothschild, J. M., Landrigan, C. P., Cronin, J. W., Kaushal, R., Lockley, S. W., Burdick, E., ... & Bates, D. W. (2005). The Critical Care Safety Study: The incidence and nature of adverse events and serious medical errors in intensive care. Critical care medicine, 33(8), 1694-1700.
3. Boyle, D., O’Connell, D., Platt, F. W., & Albert, R. K. (2006). Disclosing errors and adverse events in the intensive care unit. Critical care medicine, 34(5), 1532-1537.
4. Orgeas, M. G., Timsit, J. F., Soufir, L., Tafflet, M., Adrie, C., Philippart, F., ... & Outcomerea Study Group. (2008). Impact of adverse events on outcomes in intensive care unit patients. Critical care medicine, 36(7), 2041-2047.
5. Garrouste-Orgeas, M., Timsit, J. F., Vesin, A., Schwebel, C., Arnodo, P., Lefrant, J. Y., ... & Soufir, L. (2010). Selected medical errors in the intensive care unit: results of the IATROREF study: parts I and II. American journal of respiratory and critical care medicine, 181(2), 134-142.
6. Brown, J. C., Ding, L., Querubin, J. A., Peden, C. J., Barr, J., & Cobb, J. P. (2023). Lessons Learned From a Systematic, Hospital-Wide Implementation of the ABCDEF Bundle: A Survey Evaluation. Critical Care Explorations, 5(11), e1007.
7. Soceity of Critical Care Medicine. Implementing ICU Liberation: Find tools to implement the ICU Liberation Campaign. Accessed January 10, 2024. https://www.sccm.org/Clinical-Resources/ICULiberation-Home/Get-Started.