Editor’s Note: Remle Crowe, PhD, NREMT, is the Director of Clinical and Operational Research at ESO. In her professional role, she provides strategic direction for the research mission of the organization, including oversight of a warehouse research data set of de-identified records (the ESO Data Collaborative). We spoke with her about  how data is being used in the prehospital setting to improve patient safety.

Sarah Mossburg: First, just tell me a little bit about yourself and your role.

Remle Crowe: Sure; I'm Remle Crowe. I am the Director of Clinical and Operational Research at ESO. I'm also an EMT [emergency medical technician] and have been one for about a decade now. I was a volunteer EMT in Mexico for a long time, and I decided that I wanted to turn EMS [emergency medical services] into my full-time career, and I ended up going for the National Registry of EMTs EMS Research Fellowship at the Ohio State University, where I got my PhD in epidemiology and transitioned into the research world, and so now, with my role at ESO, I get to blend my love for research with improvement science. We do a lot of quality improvement and research work with clinical data.

Sarah Mossburg: So, segueing into the improvement science: could you tell me a little bit about how patient safety is currently being measured and monitored in the prehospital setting?

Remle Crowe: There's a bunch of different initiatives that are going on right now with regard to how to measure patient safety. One of the really interesting pieces of research that's out there has to do with trigger tools. If we don't notice that a mistake happened, it's very hard to report and it's very hard to improve upon it. But these trigger tools help us pinpoint records where we think that a safety event was more likely to have occurred. There's also research and a lot of movement around creating a culture of safety, or sometimes we hear it called “Just Culture.” That's the basis needed for us to even study things that create safe space to talk about near misses or adverse events that were caused by errors.

Sarah Mossburg: Let's start with exploring a little bit more around the trigger tool. Tell me a little bit more about what a trigger tool is, what kind of data that might use, and what that means.

Remle Crowe: Sure. There are a couple of papers published that specifically look at the prehospital environment and what might be triggers.^1,2 When we talk about triggers, these are things that could potentially lead to harm; these are things where we’d raise an eyebrow and want to dig into them a little further. So, for example, using the clinical electronic patient care record data, let's say that we saw something like untreated hypoxia, an oxygen saturation reading of less than 90% with no documentation of oxygen given. That would be something that, as a quality improvement director, I'd want to put my eyes on and say, was oxygen actually given and it just wasn't documented in the right spot, was it in the narrative, or was this a true miss and we didn't give the patient oxygen when they were hypoxic? There are several other triggers in there. If you administer opioids and naloxone in the same record, perhaps the dose was too large, or untreated hypotension is another trigger. There's several of these little things that we know could potentially lead to an error, that we can grab using our automated queries of electronic patient care record data, and then give them the full read-through with a human.

Sarah Mossburg: How do you see these trigger tools being used in practice?

Remle Crowe: I believe that agencies are using these to create reports and review records. Many agencies now have a quality improvement department or quality management that looks to prevent errors from occurring, so one of the first steps to doing that is to understand the drivers of errors and look at the baseline data. So agencies can create reports to look at, for, you know, these triggers and others that they may have been hearing, pulling those in, and then performing those quality management reviews. Sometimes that looks like one-on-one, in-person feedback, sometimes it's electronic feedback. But gathering this data to identify what are the drivers of things that may lead to errors in the future or may lead to harm—looking to prevent those proactively. Our history has been reactive, to go looking for somebody to punish or that bad apple, but we know that these are systems issues and need systems-level approaches, and so trigger tools are a really good way to get started with that, particularly when combined with this plethora of electronic data that we now have from the patient care records themselves.

Sarah Mossburg: Can you talk a little bit about that, and the data that's available in the prehospital setting and how it's used other than in relation to the trigger tools?

Remle Crowe: All of the clinical data that gets collected, independent of EPCR [electronic patient care reporting] vendor, is based on a national EMS data standard, the National EMS Information System, or NEMSIS. The types of data elements collected are similar across the country, which makes it nice when we want to create these trigger tools or benchmarks to say, “This is what we're seeing at a national level, how is your individual agency performing when it comes to some of these patient safety measures”? So NEMSIS is available, the data dictionary is online, you can see all of the discrete clinical elements—and the word “discrete” matters. When we're talking about creating these automated reports, it's really important that something was selected from a drop down, versus free text. With free text, we all might spell a word differently. Take aspirin, for example—how many different ways are there of abbreviating or writing aspirin? It's really helpful when the medications given were documented in a discrete field so that we're able to query accurately all records where aspirin was given. That's where having a national data standard like NEMSIS is a really helpful way for us to be able to build standardized operational definitions for these triggers.

Paul Dowell: Is the NEMSIS data standardized or interoperable nationally or internationally? Are there any data challenges with translating or comparing across different sets of NEMSIS data?

Remle Crowe: NEMSIS is the data standard for the US, so I can’t speak for other countries, though I know Canada is developing their own EMS data standard right now. We certainly face a challenge with things like primary impression, which is the EMS field diagnosis. It’s based on ICD-10s; you can imagine there’s a lot of ICD-10 codes, and the same with medications administered with RxNorms and for procedures performed there's SNOMED codes. NEMSIS has done a really good job at partnering with all of the vendors to come up with a suggested list. Let's all agree that we're going to use this code for aspirin and none of the many other codes for aspirin. So those defined lists have given us all this common base to start from, and that's not to say that there's not variation. There's certainly always room for improvement. But I think that this has helped make the EMS data so much more usable, and we're combining across systems and across different software products. Another initiative that has been really helpful when it comes to defining these things is NEMSQA, which is the National EMS Quality Alliance. They have developed a suite of national performance measures; there's 11 measures available on their website. They've defined the measures operationally, so there’s a numerator statement and denominator statement with NEMSIS example code, so that we are measuring not just apples to apples, but Granny Smith to Granny Smith apples. I think that is so important for us, especially as we want to measure changes over time.

They're developing more measures, and there's some states that have won FLEX grant funding to develop rural performance measures. I got to work on one of those projects with North Dakota, and Florida has their own version of it as well. The thing that we're really leaning into is that there is this common data standard, and we know we can work within that standard, to be able to put a numerator statement and denominator statement, which is a really great starting point for consistent measurement of performance measures.

Sarah Mossburg: It seems like national data standards would help with potentially benchmarking. Is that right?

Remle Crowe: Absolutely. Benchmarking with agencies who are in rural settings may look a little bit different than agencies who are in very urban high-volume settings. So, benchmarking against groups that makes sense is really important, and having these standardized operations definitions let us do some of that.

Sarah Mossburg: You alluded to this a little bit, but I think it would be helpful for the audience to understand—what are some of the challenges that make it harder to measure and monitor patient safety data and the prehospital setting?

Remle Crowe: So, there's a lot of challenges, and one of them might be that not everything would get captured in those discrete electronic patient care reporting elements. We have this national debate around ketamine used for prehospital sedation. One of the things that doesn't get captured in the record is, what was the intention of the medication. So, for example, ketamine is given for a number of different things, including analgesia. It's used to facilitate rapid-sequence induction or intubation, and it's used for prehospital emergency sedation of patients who are extremely agitated and can't be assessed otherwise. But finding that from prehospital EPCR data in a discrete query is really, really challenging. There are many other instances where the data might not get captured. Or in the case of a medication error, if it wasn't noticed by the provider, they likely documented what they think they gave. So, that's another example of where just looking at the record might not uncover all of those drivers or all of those events.

Sarah Mossburg: How does the way that care is provided in the prehospital setting, or the context of care delivery, impact the patient safety events that happen in that environment?

Remle Crowe: Prehospital care is interesting because it's delivered on scene and in transport, for a large part. In the dark at 3 a.m. when I'm on hour 18 of my 24-hour shift, things are very different than if I worked in a nine-to-five setting, with well-lit four walls where I’m just with the patient. So, prehospital care can be delivered in environments that are chaotic or where there's time-sensitive decision making happening, particularly with conditions where we know that getting treatment within a certain time window really matters. Or, in the event of something like an out-of-hospital cardiac arrest, those scenes are often very busy and there's a lot more happening that not as controlled as in a hospital setting, for example.

Sarah Mossburg: How does that difference in environment and care delivery impact your ability to measure the quality of care, or how safe the care delivered was, or whether patient safety events are happening?

Remle Crowe: To answer that, you have to understand the workflow of prehospital care. So, all of this care occurs in real time, obviously there's a lot of pressure, and it's dark, among other things. But the documentation happens after the care is delivered, for the most part, especially if it was something that was time sensitive. For example, if it was a cardiac arrest, I'm going to take care of the patient first, go to the hospital, clean out my ambulance, probably go back in service, and depending on how busy the system is, I may have to go to another call before I get a chance to document the care that occurred. Now recall bias starts to play an important role. For something that happened hours ago, am I going to accurately remember what time I gave each medication if I documented on the back of my glove while I was doing it? Do I still have that glove? The accuracy of the data can be impacted by that complex, high-stress, time-sensitive environment.

Sarah Mossburg: You talked a little bit about trigger tools and their uses as one way to monitor and measure patient safety, and those are relatively new; what are some other ways that historically patient safety has been looked at in the prehospital setting?

Remle Crowe: I think we're in a movement from measuring for compliance to measuring for improvement. In the early days, especially when we were in paper documentation, [measurement of patient safety] occurred by just review audit of records. Maybe we audited every record or maybe we picked them randomly. A lot of that was just done by these manual reviews, or perhaps supervisors would go out into the field and observe things. But it wasn't as systematic, and there wasn't this culture towards using measurement to prevent an error from occurring—a lot of it was just auditing or quality assurance that was after the fact.

Sarah Mossburg: One of the things that I think I understood about trigger tools, was they're looking at more of the low-risk everyday care that's being provided by EMS as opposed to some of those types of events that are high risk, and therefore more likely to be audited using quality tools, is that correct?

Remle Crowe: Right. The trigger tool casts a broad net looking at things that we see commonly, because when we're designing measures for improvement, it has to be something that happens with enough frequency, that if we make a change, we're going to be able to see whether or not a change has resulted in any actual improvement. With something like a sentinel event, where we're only seeing it once a year or once a career, that's a different approach to improvement. The trigger tools apply to records that we probably see on a frequent basis to be able to make changes and measure for improvement. An example we recently saw was with ketamine. There was concern that EMS providers were giving massive doses of ketamine routinely. But we took a year's worth of the ESO records for 2,000 different agencies and saw that's not happening routinely. But then we look at another measure that could easily be included in a trigger tool—for example, what if I gave ketamine for sedation and then I did not measure end tidal CO2? We see a larger space for improvement with that kind of measure. We know that end tidal CO2 is really important as a real-time measure of patient response, whereas with pulse oximetry, we're kind of like looking in the rearview mirror, because there's a five-minute lag on oxygen status.

Sarah Mossburg: I'm curious if you're aware of what kind of trends that are being identified with some of these triggers? Do we have some early ideas, some commonalities across the prehospital setting?

Remle Crowe: Sure. At ESO we look at performance improvement for EMS metrics that matter, and where there's room for improvement. Every year we release an EMS ESO index, and two of the measures that we're looking at are centered around the safety of using ketamine in the prehospital setting. We know that it's a really powerful sedative and analgesic medication that has a lot of positive impact. But, when not monitored appropriately, there's a lot of risk associated with it. We look at 2,000 agencies across the nation, to see how they are performing when it comes to the safety measures, like documenting a patient weight when ketamine was given, or documenting an end-tidal CO2 when medication was given. In last year's index, we saw, across all of those organizations, that only about 80% of records with ketamine administered had a documented patient weight. And then we're talking about whether end-tidal CO2 is documented, we're looking at about 70% of records where ketamine was given had an end-tidal CO2 recorded. And then you can dig into that even further; you can think about all of the different layers that we need to apply to get an accurate look, specifically at the local level. So it goes back to that, in the national data, it's really easy to see that ketamine was given, but it's hard to distinguish what it was given for. And so, perhaps if we're giving you a low dose ketamine for analgesia, etcetera, end-tidal CO2 is not captured as often, but we would hope to see, and this is something that EMS agencies should measure, that if it was given for rapid-sequence intubation or if it was given for sedation, that we are performing that immediate monitoring after administration of the drug. If we can get the monitoring before administering the drug, [that] would be even better.

Sarah Mossburg: It sounds like at a national level, the data is a little more difficult to dig into than at the local, right?

Remle Crowe: To get into the granularity, all of that rich information around context, that would be contained in the provider free-text notes. So, for me, I see these trigger tools and these benchmarks as signals that there’s something to go look at, and then individual agencies should be taking those signals and really digging in to figure out what are their drivers at the local level, and by doing that, you can get a lot of good qualitative data from the narrative or from even frontline interviews. I did quality improvement at Ford for a long time, and one of our approaches was to walk the line, which literally means go to the front line and perform some of these informal interviews about what do you think is causing this? But in order to be able to do that, you have to have this culture of safety; otherwise people will be hesitant to tell you what they think is actually happening for fear of punishment.

Sarah Mossburg: That segues nicely into something that you mentioned earlier, around some of the work you've done in the EMS setting on the culture of safety. Can you tell us a little bit more about that?

Remle Crowe: Yeah, there's a couple of different efforts around culture of safety. So, there are two surveys that have been adapted specifically to the EMS context, which is a little bit different than some inpatient settings. Those are the safety attitudes questionnaire, that Dr. Daniel Patterson helped develop, and then I helped work with the Center for Patient Safety on an EMS-adapted version of the surveys on patient safety, or the EMS SOPS. What these surveys do is let the agency gauge where they are at in terms of patient safety climate. These surveys are freely available and look at how the agency frontline providers perceive safety: whether or not it's an open communication environment, what is the response when there are mistakes, is it a non-punitive culture, or is it that hunt for the bad apples or the hunt to blame somebody, and that's huge. One that we specifically added to the SOPS for EMS is this concept of handoffs, so when EMS arrives at the hospital, how is the information communicated, are there risks to patient safety at that point of handoff? That was one of the unique things that we added, and then there's other domains within that survey, but ultimately, it's just a way of taking a snapshot in time of what does your culture of safety look like at an EMS agency.

Sarah Mossburg: Tell me a little bit more about the handoffs piece and adding that. That's interesting, because we do handoffs in the inpatient setting as well, so I'm curious to hear more about its addition to the SOPS in the prehospital setting.

Remle Crowe: Right, we thought this one was really important, because that handoff is so critical. EMS was often in the patient's home or at the scene of wherever the medical or traumatic emergency occurred, and so there's information there that needs to get transferred over for the continuum of care. The things that we asked about was whether or not there were problems during the transfer of patients, or whether there's a clear expectation or a clear template about how they're supposed to exchange important information. For example, don't go back to the patient’s 10 years’ worth of medical history, start the template with what is most related to the current emergency. A lot of that was focused around standardization of that communication and that handoff and whether or not that's actually occurring. That quickly becomes really important. Especially when it's a time-sensitive emergency and everything's chaotic and everybody's rushing around. But is there this approach? And that can even start as early as the radio communication to the hospital. Am I letting them know that I suspect a stroke, so that they can prepare on their end, the things that need to be done for that or, if I have a patient with severe trauma, am I calling a trauma alert, or at least, what does the communication look like for that? You can see how that can expedite the care on the receiving side and lead to better patient outcomes.

Sarah Mossburg: Absolutely; you can see how they'd be much better prepared to receive the patient based on the information that's being provided.

Remle Crowe: Conversely, we could find situations where I will go in as the EMT and I’m met by somebody who doesn't want to give me the time of day, and you can see where there’s going to be opportunities for error. For example, I didn't get to report an allergy, or something really critical that could lead to patient safety events. So we feel like it's really important to set up the baseline, because it's possible to improve that. Let's say that we do find out that the culture is not really where it needs to be; that's when we can start taking an improvement science approach again to changing that culture, so that we are preventing some of these patient safety events from occurring. This is the part that gets really exciting to me.

Sarah Mossburg: That's really interesting. If that's happening, perhaps at one particular organization that EMS is dropping off at, but not at other organizations, how do you parse that out from the data on the SOPS?

Remle Crowe: So that is going to depend on how it's being administered and where. The EMS SOPS is written, it's not specific to which facility, and so that would be where maybe there's a signal in that data again. You see something isn’t where we want it to be, it's not hitting that goal that we set for ourselves, our aim. So, let's dig in, and that's where you start to use some of these other tools from improvement science, one of my favorites being the driver diagram. We can get some insight from the front lines and qualitative experiences—we noticed that we're a little bit below our goal, are you all aware of a particular facility where this is occurring, or is it kind of across the board? You're not going to get everything from the SOPS, again it's just the snapshot. If you have limited time and resources, go to where you have a substantial space for improvement.

Sarah Mossburg: Absolutely. Out of curiosity, were there other domains on the EMS patient safety culture survey that were different from the inpatient, ambulatory, or other patient safety culture surveys?

Remle Crowe: Some of them may have had the language switched a little bit, but in general, these domains apply across the board. We're talking about things like staffing, teamwork, and the non-punitive response to mistakes, so all of these apply. The language may have been adapted—if it mentioned being in a static location, we changed it to the prehospital setting. That was the important piece, to adapt the language to fit EMS. So if it talks about a workday, we talked about shift. But in general, that's the beauty of the SOPS, is that it applies so broadly and it lets us be able to compare different contexts, which I think is also an interesting aspect of patient safety, that not all parts of the healthcare continuum experience this the same but we're able to compare them with a snapshot like this.

Sarah Mossburg: I think that's a hallmark of moving us forward as a larger community—being able to share best practices and learn from each other. I'm curious if you are aware of any data emerging on prevalence of specific safety events within the prehospital setting or associated harms related to those? Just thinking kind of broadly about, what are the primary events that are being experienced in prehospital setting?

Remle Crowe: I'm not aware of any published; that doesn't mean it doesn't exist because I haven't read every article out there. I can say that the work that I've seen recently is focused around the potential for harm and around preventing it from happening, which again is a really great space because we're moving upstream and stopping something from happening. Some areas are probably common to other settings too, like medication errors, particularly with children. A paper just came out looking at using a method, called the Handtevy method³ which basically precludes complex math, so instead of basing our dosing on ideal body weights and multiplication in a setting where there is a chaotic scene, this tool removes the necessity to do math under those circumstances in an effort to prevent safety events from happening. And using end tidal CO2 as a way of looking at real-time data versus waiting for that pulse oximetry for the hypoxia to hit before doing anything. So there are several really excellent projects and papers around these types of things, but when it comes to reporting actual safety events, a lot of that goes to patient safety organizations, or PSOs, and that data is, rightfully so, private, so that reporting organizations can have confidentiality to be able to report an event. So I'm not shocked that a lot of that data would not be published for that exact reason, at least at the individual level.

Sarah Mossburg: Of course. I'm curious where you see the field headed in the future; what are some of the future directions that you see for patient safety measurement and monitoring? Surveillance systems perhaps in the prehospital setting?

Remle Crowe: I think our quality management tools, with the electronic data that we're now getting and it being so rich and the increased level of standardization, are really important. The idea of threshold alerting comes to mind immediately, and so I think that we are going to be designing these trigger reports, that we get instant alerts when a trigger happens, so that we're able to review it more quickly. Because if you let an event sit for a few months, again that recall bias starts to set in, and we forget the exact circumstances around certain events. So, having more real-time information and more real-time alerting around patient safety events is going to start to pick up more in the future; some of it certainly is happening today.

I also think that the culture is changing. There have been some really great efforts around implementing practices around culture of patient safety and a non-punitive culture. And I think that is what's really going to be key, if we're going to compare ourselves to aviation in that there's a lot of complexity and a lot of potential for mistakes with such complexity. The aviation industry did a very good job creating this type of culture that allows for event reporting, and also very importantly near-miss reporting. In healthcare, that would be something like, if I noticed that these two bottles of medication come in blue bottles and they both start with A, and at 3 a.m. it's really challenging for me to make sure that I'm grabbing the right one. I am enthusiastic about where EMS is going around patient safety and especially with the ability of our technology to assist us. It’s not the end-all be-all, but certainly easier than digging through a stack of 100,000 paper records!

Sarah Mossburg: It sounds like you are seeing an increase in near-miss reporting because of an awareness of a culture of safety in the EMS setting. What do you think is driving that increased awareness around culture of safety and non-punitive culture in the prehospital setting?

Remle Crowe: It's a really good question, and you know, I don't think there's one thing that is driving it. I think it's a concept that is catching on and there's a lot of different ways it's being communicated. At national conferences there's a lot of great speakers who talk about these concepts, there's a lot of good articles in commonly read publications in the industry, you'll see that there's more and more columns. The researcher and the data-gatherer in me would bet, that if we were to go out and look at the number of articles published in relation to these topics, that they're growing and they have grown a lot in recent years. Things like podcasts are another great way that information is getting disseminated, so having some of these podcasts where we dig into what is the psychology behind culture of safety, and what is the organizational approach. I think as a profession we're just growing. EMS is still a pretty new profession when you think about it, going back to the days of the famous 1966 white paper: Accidental Death and Disability.⁴ It's fun to think about how EMS for my grandparents was probably very different and very, very different, if at all, for my great grandparents. So, I think that this progression is probably at a natural point, but it's picking up quickly, and I'm very optimistic about that.

Sarah Mossburg: Well, it sounds like there's a lot of really good work being done in the prehospital setting around identifying where opportunities are for improvement and improving patient safety, with some of the trigger tools. It sounds like the culture of safety is really becoming something that's a little bit more prevalent and people are aware of. That's great. We talked a little bit about benchmarking; you also mentioned the potential for push alerts.

Paul Dowell: So, stepping back, you mentioned that you worked with EMS in Mexico. Thinking back to when you started, and all the way through your career now, is there anything that you can think of in that span of time that really stands out as something prominent that's a safety issue that has either been addressed, or maybe not addressed? You mentioned the example of blue bottles with the same lettering, I try to compare that to safety changes implemented in the pharmacy like tall man lettering, or physically moving things so that they aren’t right next to each other. Is there something in the field that's been a prevalent problem for 20 years and it's still kind of hanging around, or conversely is there something that was solved with some of these efforts that you can think of?

Remle Crowe: That's a really great question, and I'm sure that there's so many examples of things that have been solved. That's the hard part with prevention is it never gets the glory because it stopped happening and we don't notice it, right? I'm a big fan of Don Norman and his book, The Design of Everyday Things, but you don't notice when something goes well, or when there's a forcing function stopping you from making a mistake. One that I can think of is that with monitors, if you hit the wrong button, it would just shut off instead of defibrillating. So when you're under pressure and you're thinking about which button to push, you may actually push the off button. Even the projector in my classroom asked me 20 times, “Are you sure you want to power off right now?” I think a lot of the monitors now have a feature where if it looks like you're about to defibrillate they don't turn off if you hit the wrong button. As far as safety issues that are still in need of getting addressed, there’s a huge movement right now, the national EMS quality partnership, to reduce lights and siren use. And that sounds crazy right? We're EMS, of course we run around with the lights and sirens on! But we found out it's dangerous, and that there's a lot of accidents or crashes caused by lights and sirens use. With today's modern cars’ soundproofing, you can't really hear the sirens until they're right on top of you. There are around 50 EMS agencies that have come together on this partnership and they're going to be following improvement science principles and doing these rapid tests of change, PDSA [plan do study act] cycles, looking at ways to change our culture around lights and sirens, and to treat it more like a clinical procedure, which we will use judiciously. We don't just proactively intubate all our patients because they might have some breathing difficulty later. Of course not, so why do we do that with lights and siren use? So, I am really excited, this is the first of its kind, at least for the EMS industry, around creating a common goal, and having 50 organizations with different localities, different settings, different agency types, all come together to try to reduce lights and siren use. The project is huge, not just for patients, but for providers, as well as the communities. We think about things like wake-effect crashes after an ambulance goes through an intersection and cars are left with the change in traffic pattern. I hope that this kind of national improvement project is one that gets replicated many, many more times with different patient safety concerns.

Paul Dowell: Was there anything that we didn't talk about the you feel is kind of critical to mention?

Remle Crowe: I think we covered the big things for me, this notion of using electronic data purposefully instead of randomly auditing or performing 100% auditing of all of our records. I think that's a move in the right direction. Having these national initiatives is huge—things like NEMSQA and the national EMS quality improvement partnership that's going on right now with lights and sirens. The trigger tools and the surveys on patient safety—any opportunity we get to just reiterate that those are out there to be used and encourage their use is a really great thing for us to do.

Sarah Mossburg: Well, thanks so much, we had a great conversation with you.

References