Problem Addressed

The AAM program was designed to identify patients on general hospital units who are at high risk for clinical deterioration, so clinicians can intervene to prevent negative outcomes. Delayed or missed signs of acute inpatient deterioration outside of the ICU are associated with higher rates of severe illness and death.¹,² Early identification of clinical deterioration and developing and implementing an appropriate care plan can prevent adverse events, unplanned ICU admissions, and unexpected death.² After internal review of the data, KPNC found that mortality rates for patients transferred from medical, surgical, and telemetry units to the ICU were significantly higher than mortality rates of patients who were directly admitted to the ICU from the emergency department. This finding is consistent with findings from other studies.^3,4

Studies have shown that more than 50% of transfers to the ICU occur within the first 24 hours of hospital admission and 80% occur within the first 48 hours.³ Research shows that the majority of ICU transfers are thought to be avoidable.^1,5,6 KPNC data showed that only 3–4% of hospital admissions required a transfer to an ICU; however, these patients accounted for as much as 25% of hospital deaths and ICU admissions.⁵ ICU transfers do not only potentially increase hospital length of stay; there are also increased costs associated with the higher level of care.

Description of the Innovative Activity

The AAM program uses predictive analytics to scan the patient’s EHR and assigns an AAM score.⁷ The model uses a variety of clinical data, including composite scores for chronic severity of illness (Comorbidity Point Score, version 2, and Laboratory-Based Acute Physiology Score, version 2).¹ In the current iteration of the program, virtual quality nurse consultants are responsible for monitoring the AAM scores on an hourly basis. If triggered, the virtual quality nurse consultant conducts a review of the patient’s chart to determine the appropriate next step. If the score and review of the chart confirm signs of potential clinical deterioration, the virtual quality nurse consultant alerts the RRT nurse. The RRT nurse then collaborates with the clinical team, which can include the hospitalist, bedside nurse, and other providers caring for the patient. The clinical team then works collaboratively with the patient and the family to identify or clarify the goals and wishes of the patient to develop a tailored care and escalation plan.⁷

Context of the Innovation

Failure to identify, communicate, or act on indicators of acute deterioration of hospitalized patients outside of the ICU is a potentially preventable patient safety issue associated with increased mortality and morbidity.² To prevent adverse events resulting from acute deterioration, such as unplanned admissions to the ICU and unexpected death, scores such as the Modified Early Warning Score and National Early Warning Scores were developed to identify at-risk patients.² The establishment of RRTs was another approach to address inpatient clinical deterioration. In some organizations, RRTs evolved into rapid response systems that included detection, a structured response, and a governance structure.

KPNC developed the AAM program to automate the early warning score using predictive modeling to support rapid response systems in mitigating adverse events due to missed or delayed identification of patient deterioration.

Results

Key results and findings from the staggered implementation of the AAM program across 19 hospitals include statistically significant decreases in mortality, hospital length of stay, and ICU length of stay. Implementation of the program resulted in a higher proportion of patients designated with favorable status (alive, not in the hospital, not rehospitalized) within 30 days after an alert.¹

Across 19 KPNC hospitals between August 2016 and February 2019, 326,816 patients had 548,838 non-ICU hospitalizations. An analysis of program data during this period indicated a patient reached the alert threshold during 43,949 hospitalizations.¹ Implementation of the intervention resulted in a lower mortality rate within 30 days after an alert for the intervention cohort—two KNPC hospitals participating in the pilot—compared with the comparison cohort—remaining KPNC hospitals not participating in the pilot program (adjusted relative risk = 0.84; 95% confidence interval [CI]: 0.78–0.90; p<0.001).¹ Between the intervention and comparison cohorts, KPNC estimated an absolute difference of 3.8 percentage points in mortality within 30 days after an alert, which translated into 3.0 deaths (95% CI: 1.2–4.8) avoided per 1,000 eligible patients per year, or 520 deaths (95% CI: 209–831) per year over the 3.5-year study period.¹

It is important to note that RRT nurse compliance with the process measures associated with the intervention has consistently been over 90%.⁷ The virtual quality nurse consultants report significant clinical findings from the chart review to the RRT nurse and review the alert jointly. The RRT nurses must respond and act on the alerts within three hours by consulting the hospitalists with their assessment and reviewing the treatment plan. The RRT nurse’s compliance is measured through documentation indicating their assessment and intervention, recorded using the standard dot phrase in the patient’s EHR within three hours of the initial AAM alert.

Planning and Development Process

KPNC initially piloted the AAM program in two hospitals. The lessons learned from the pilot were then applied to a staggered deployment of the innovation to an additional 19 hospitals within the system. Deployment across hospitals in the system used the following planning and development process.

A team of physicians, nurses, analysts, and performance improvement experts came together to devise and execute the deployment strategy. This strategy included a standardization of early detection using remote monitoring, standardization of the clinical response and its infrastructure, and standardized system governance, which included a hospital readiness assessment and ensuring proper training of all parties involved.²

To standardize remote monitoring, hospital leadership shifted the responsibility of viewing alerts in the inpatient EHR from physicians and RRT nurses to specially trained virtual quality nurse consultants, who reviewed the display of alerts via an electronic dashboard.²This team is responsible for monitoring the alerts dashboard 24 hours a day, seven days a week across 21 KPNC hospitals. Once a patient reaches a threshold to trigger an alert, the virtual quality nurse consultant conducts an assessment using information in the patient’s chart to determine whether contacting the RRT nurse is the next best step. The virtual quality nurse consultant requests prompt evaluation of the patient by the physician. The virtual quality nurse consultant is responsible for reviewing the chart to make sure the request for evaluation was completed.

Standardization of the clinical response and its infrastructure included ensuring the availability of RRT nurses 24 hours a day, seven days a week. KPNC determined it was necessary to dedicate 4.2 full-time nurse equivalents per hospital. The KPNC team developed a standard set of responsibilities and workflows for the RRT nurses, which included focusing primarily on responding to alerts and proactive rounding on high-risk patients.²Providing access to palliative care and other support services seven days a week was a key part of the strategy.

The governance component consisted of establishing an AAM long-term oversight committee similar to existing Critical Care and Code Blue committees.² It focused on developing processes to support performance improvement, sharing data via dashboards, and hosting collaborative calls between hospitals to share best practices. Each hospital should also identify an implementation lead and define roles and responsibilities for each hospital discipline involved in the workflow.² Another key element in governance is adherence to practice guidelines and ensuring those involved in the implementation of the innovation are trained in its implementation.

Resources Used and Skills Needed

To implement the AAM program, organizations should consider the following action steps:

• Convene a multidisciplinary team of clinicians to review and use evidence-based criteria to develop the algorithm to determine the AAM score, which is used to determine likely patient clinical deterioration.
• Work with the information technology team to integrate the program into the organization’s EHR.
• Identify the team members who will be trained to monitor and review the AAM scores and facilitate the triage process as appropriate. Kaiser Permanente Northern California (KPNC) used virtual quality nurse consultants for this task.
• Work closely with the RRT and clinicians to determine the best workflow for triage and follow-up.
• Review the data regularly to identify opportunities to improve the process and the outcomes and refine the AAM algorithm as needed.

Funding Sources

The innovation team received financial support for the pilot of the AAM program at two KPNC hospitals from the Gordon and Betty Moore Foundation. KPNC funded the implementation of the program at the remaining 19 hospitals within its system.

Getting Started with This Innovation

• Develop an evidence-based algorithm to determine AAM score and establish appropriate workflows and documentation standards.
• Identify multidisciplinary leaders (hospitalists, ICU physicians and nurses, social services, and palliative care) and frontline champions, and provide them with defined roles and responsibilities in the development and implementation of the program.
• Ensure appropriate staffing to effectively support program implementation, including virtual quality nurse consultants and RRT nurse.
• Build and maintain a shared safety culture to minimize communication gaps.
• Use a performance improvement process to conduct several iterations of workflow and documentation standards until the best process for the facility is identified.

Sustaining This Innovation

Based on KPNC’s experience implementing the AAM program, the following elements are key to sustaining the innovation:

• Maintain leadership support and commitment to the program.
• Continue to train staff as needed on standardized communication, clinical response protocols, handoffs, and documentation.
• Facilitate the sharing of program best practices using tools such as newsletters, collaborative conference call meetings, and site visits.
• Review and share program data on a consistent basis to guide performance improvement activities to continue to refine the AAM program.

References/Related Articles

Dummett BA, Adams C, Scruth E, Liu V, Guo M, Escobar GJ. Incorporating an early detection system into routine clinical practice in two community hospitals. J Hosp Med. 2016;11(Suppl 1):S25-S31.

Escobar GJ, Turk BJ, Ragins A, et al. Piloting electronic medical record-based early detection of inpatient deterioration in community hospitals. J Hosp Med. 2016;11(Suppl 1):S18-S24.

Granich R, Sutton Z, Kim YS, et al. Early detection of critical illness outside the intensive care unit: clarifying treatment plans and honoring goals of care using a supportive care team. J Hosp Med. 2016;11:S40-S47.

Footnotes

1. Escobar GJ, Liu VX, Schuler A, Lawson B, Greene JD, Kipnis P. Automated identification of adults at risk for in-hospital clinical deterioration. N Engl J Med. 2020;383(20):1951-1960.
2. Paulson SS, Dummett BA, Green J, Scruth E, Reyes V, Escobar GJ. What do we do after the pilot is done? Implementation of a hospital early warning system at scale. Jt Comm J Qual Patient Saf. 2020;46(4):207-216.
3. Liu V, Kipnis P, Rizk NW, Escobar GJ. Adverse outcomes associated with delayed intensive care unit transfers in an integrated healthcare system. J Hosp Med. 2012;7(3):224-230.
4. Escarce JJ, Kelley MA. Admission source to the medical intensive care unit predicts hospital death independent of APACHE II score. JAMA. 1990;264(18):2389-2394.
5. Escobar GJ, Greene JD, Gardner MN, Marelich GP, Quick B, Kipnis P. Intra‐hospital transfers to a higher level of care: contribution to total hospital and intensive care unit (ICU) mortality and length of stay (LOS). J Hosp Med. 2010;6(2):74-80.
6. Bapoje SR, Gaudiani JL, Narayanan V, Albert RK. Unplanned transfers to a medical intensive care unit: causes and relationship to preventable errors in care. J Hosp Med. 2020;6(2):68-7w2. di:10.1002/jhm.812
7. Martinez VA, Betts RK, Scruth EA, et al. The Kaiser Permanente Northern California Advance Alert Monitor Program: an automated early warning system for adults at risk for in-hospital clinical deterioration. Jt Comm J Qual Patient Saf. 2022;48(8):370-375.