Creation of a Medical Procedure Service to Improve Patient Safety
Smith CC, CHuang G. Creation of a Medical Procedure Service to Improve Patient Safety. PSNet [internet]. Rockville (MD): Agency for Healthcare Research and Quality, US Department of Health and Human Services. 2008.
Smith CC, CHuang G. Creation of a Medical Procedure Service to Improve Patient Safety. PSNet [internet]. Rockville (MD): Agency for Healthcare Research and Quality, US Department of Health and Human Services. 2008.
Perspective
Introduction and Rationale
Prior to the introduction of the Medical Procedure Service (MPS) in 2002, medical procedures at our institution (Beth Israel Deaconess Medical Center) were performed and taught using the time-honored apprenticeship model of "see one, do one, teach one." Residents were expected to independently perform and teach most medical procedures (central venous line placement, thoracentesis, paracentesis, and lumbar punctures). Procedural instruction was limited and poorly standardized; furthermore, trainee performance was infrequently supervised or assessed. In addition, the most qualified faculty were often unavailable to help teach, perform, supervise, or evaluate these procedures.
In 2001, the Vice Chair for Quality in the Department (Dr. Mark Aronson) gathered a small group of faculty to discuss how we performed medical procedures and the educational process for house staff. Although there had not been a sentinel event or specific concerns regarding procedural complications, this group recognized that the time had come to develop a new model of procedural instruction for our trainees. In addition, they believed that it was not in the best interest of our trainees or patients to expect an inexperienced second-year resident to teach and supervise an intern to perform these procedures on complex medical patients. These concerns resulted in the development of a comprehensive and novel Medical Procedure Service (MPS). The aims of this program were to: (i) create a new and standardized approach to teaching, performing, and evaluating inpatient medical procedures; (ii) better determine when a trainee is competent by assessing both clinical knowledge and psychomotor skill; and (iii) ultimately, improve patient safety.
Configuration
The MPS consists of select faculty skilled in performing inpatient procedures. These attendings focus on all components of the procedure, including obtaining consent, utilization of proper sterile technique, knowledge of relevant anatomic landmarks, teaching the psychomotor skills required for the specific procedure, and identifying and managing complications. Central venous lines (CVL) and thoracenteses are performed under the direct supervision of interventional pulmonology attendings, and operators use a portable ultrasound device whenever indicated. Lumbar punctures and paracenteses are performed under the guidance of designated, experienced hospitalist physicians who do monthly rotations on the procedure service while also serving as the medical consult attendings. Table 1 displays the volume and characteristics of procedures performed by house staff in a typical year.
When an intern on a non–intensive care unit (ICU)-based rotation needs to perform a medical procedure, they contact the MPS resident through a unique pager number. The MPS resident discusses the indications, risks, and benefits with the primary team and patient; assesses the patient and then pages the appropriate procedure attending; and schedules a mutually convenient time and location for the procedure. Interventional pulmonary and select hospitalist faculty provide supervision and teaching to the procedure resident, who covers procedures from 7 AM to 7 PM Monday through Saturday; on Sundays the service is covered by an ICU resident. Overnight, the intensivist on call in the hospital supervises urgent medical procedures, which are performed by the most experienced resident involved in the patient's care. Thus, with the exception of emergent procedures, all medical procedures are supervised by experienced faculty. Direct faculty supervision helps offset the costs of the program, as attendings now generate clinical revenue for procedures that were previously unbilled. Recognizing the importance of this educational and quality improvement endeavor and the time commitment of the faculty involved (and the fact that professional fee billings do not cover these costs), the Department of Medicine directs additional teaching funds to the pulmonary and hospitalist medicine budgets.
In our initial model, the intern caring for the patient was the primary operator of the procedure. This arrangement promoted continuity of care and allowed the trainee to understand the full scope of the rationale, benefits, and risks of procedures. However, as a result, MPS faculty had to assess the level of knowledge and skill and provide education to a different resident with every procedure, which was time-intensive. Additionally, weeks to months would elapse before a resident could perform another similar procedure, hindering his or her ability to refine and reapply new skills. Thus, after the first year, we modified the model by creating a dedicated procedure rotation covered by a second-year resident. This intensive learning experience provides the resident multiple, condensed opportunities for deliberate practice and allows the assigned faculty member to tailor instruction individually and provide longitudinal evaluation. This also frees the busy ward teams to continue in other patient-related activities rather than rushing to perform procedures. In addition, procedures that were previously directed to interventional radiology are now conducted by the residents' colleagues on the MPS.
Because the MPS has removed interns from doing many ward procedures on their patients, we did worry about how to ensure that interns would achieve at least a minimal level of procedural competency before their postgraduate year 2 (PGY2) procedure service rotation. To meet this goal, we expect interns to perform these procedures during their ICU rotations. This is done under the direct supervision of the ICU attending; all ICU attendings provide the same guidance, supervision, and feedback as the procedure service faculty. In addition, to guarantee a minimal proficiency at CVL insertion (the procedure most likely to be required on an emergent basis), on the first afternoon of their first ICU rotation, all interns participate in an extensive CVL course that utilizes simulation technology.
The MPS has evolved over time to better address the needs of our trainees. For instance, we now provide opportunities for the residents rotating on the service to participate in elective cardioversions and right heart catheterization. Previously, as a minimal marker of procedural competence, the American Board of Internal Medicine (ABIM) required each resident to perform a minimum number of designated procedures prior to completion of residency. In 2007, recognizing that types and numbers of procedures required by an internist will vary depending on one's chosen field of expertise, the ABIM modified its requirements and no longer requires all graduates to competently perform all procedures. In light of these recent changes, the procedure service is now an elective rotation, whereas it was previously required for every second-year resident. This change allows residents who are interested in procedure-oriented fields or who are interested in becoming competent at medical procedures to have adequate time to master these skills and allows the program to better assess their level of competence at these procedures. Although the rotation is optional, the procedure service remains continuously staffed by residents, as more residents are interested in the rotation than are slots available. To allow all residents to participate, some rotations are scheduled for less than a full 3-week block.
Education and Evaluation
A Web-based multimedia curriculum complements individualized education and feedback provided by the faculty. These materials include a review of the procedure, annotated by interactive diagrams, links to videos, and bibliographic references, and self-assessment quizzes. The Web site also aids in documentation; it provides standardized checklists to be used prior to the procedure, prefilled consent forms, and procedure note templates. Figure 1 shows a sample portion of a self-assessment quiz.
After every procedure, the trainee completes an online procedure log that records aspects of the procedure (e.g., complications and needle passes) and assesses procedural comfort. An alert is then automatically sent to the supervising attending physician, who completes an online evaluation. This form documents the trainee's cognitive and psychomotor skills, procedural complications, and other data such as number of needle passes and time to completion of the procedure.
The new ABIM regulations regarding procedural training (1) leave the determination of competence to the individual residency programs. However, there are no clear standards by which to determine competence in these common medical procedures. In addition to enhancing education and patient safety, the direct supervision and assessment provided by our MPS allows us to better define trainee competence. Previously, the determination of competence relied exclusively on one simply completing a predetermined number of procedures. Table 2 shows a sample of our new requirements for interns to be deemed "competent" at a CVL.
Research
Measuring the effectiveness of the MPS program presents an opportunity for educational innovations to intersect with clinical research. Data collected from self-assessment instruments, resident procedure logs, faculty evaluations, and electronic medical records provide the foundation for research evaluation. Figure 2 shows how these sources of data span the levels of Kirkpatrick's framework for evaluation (2), from basic trainee metrics to impact on the patient. We believe that the combination of these measures forms one approach to procedural competence.
During our first year, when use of the MPS was elective, we utilized these data to demonstrate that complication rates of the MPS compare favorably to national benchmarks (3), that residents who use the procedure service are twice as likely to be comfortable performing procedures (4), and that residents preferentially choose the MPS for more medically complex patients (GCH, unpublished data). Now that the MPS is required for performing procedures, we are taking advantage of the increased sample size to attain two ultimate research goals for the MPS: to determine whether the MPS has achieved a statistically significant decrease in complications rates and to characterize procedural competence.
In our work in procedural competence assessment, we have launched two related initiatives. We developed a CVL performance checklist validated by experts. These experts also helped determine a minimum passing score, which we will corroborate with other performance outcomes. Second, recognizing the power of simulation as a "safety net" for procedural training, we launched a randomized study whereby interns in the intervention group are taught CVL placement on simulators prior to performing them on patients in the ICU rotation. Using the validated checklist, they are assessed on simulators before and after the intervention. Thus, we have addressed all levels of Miller's pyramid of competence assessment (5), as depicted in Table 3.
Conclusions
As with any new program or intervention that affects an entire department, the success of this program depends on the collaboration and cooperation of many. The willingness to dedicate the resources and faculty support to an untested endeavor was a direct reflection of the institutional spirit and dedication to providing the best possible care for our patients and education for our students and residents. At our institution, the MPS has been implemented only in the Department of Medicine; however, it has garnered interest from and collaboration with other clinical departments within our hospital, as well as from other medical centers. It has widespread support from hospital administration and program directors, and as such, our program serves as an exemplar of how the educational and clinical missions of the teaching hospital share common goals, by protecting our patients as they undergo potentially morbid procedures.
C. Christopher Smith, MDAssociate Director, Internal Medicine Residency Program, Beth Israel Deaconess Medical CenterAssociate Chief, Herrman L. Blumgart Medical Firm, Beth Israel Deaconess Medical CenterCo-Director, Rabkin Fellowship in Medical Education at Harvard Medical School and Beth Israel Deaconess Medical CenterAssistant Professor of Medicine, Harvard Medical School
Grace C. Huang, MDDirector of Assessment, Carl J. Shapiro Institute for Education and Research at Harvard Medical School and Beth Israel Deaconess Medical CenterAssistant Professor of Medicine, Harvard Medical School
Acknowledgment: The authors owe special thanks to those who showed an uncommon foresight and dedication to make the Medical Procedure Service at Beth Israel Deaconess Medical Center successful, including Drs. Moellering, Weinberger, Zeidel, Strewler, Weiss, Reynolds, Feller-Kopman, Gordon, Ernst, Aronson, Clardy, and Weingart.
References
Back to Top 1. Procedures Required for Internal Medicine. American Board of Internal Medicine. http://www.abim.org/certification/policies/imss/im.aspx#procedures. Accessed January 24, 2008. 2. Kirkpatrick, DL. Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler, 1994. ISBN: 1881052494. 3. Smith CC, Gordon CE, Feller-Kopman D, et al. Creation of an innovative inpatient medical procedure service and a method to evaluate house staff competency. J Gen Intern Med. 2004;19:510-513. [go to PubMed] 4. Huang GC, Smith CC, Gordon CE, et al. Beyond the comfort zone: residents assess their comfort performing inpatient medical procedures. Am J Med. 2006;119:71.e17-71.e24. [go to PubMed] 5. Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(suppl 9):S63-S67. [go to PubMed] Tables Back to Top Table 1. Procedures Performed by House Staff in 2006(Go to table citation in commentary)
N (%) | |
---|---|
Total Procedures
|
1174 (1)
|
Central venous line
|
467 (40)
|
Lumbar puncture
|
232 (20)
|
Paracentesis
|
258 (22)
|
Thoracentesis
|
217 (18)
|
Complications
|
|
None
|
1102 (94)
|
Bleeding
|
34 (3)
|
Pneumothorax
|
9 (1)
|
Dry tap
|
29 (2)
|
Urgency
|
|
Elective
|
636 (54)
|
Urgent
|
431 (37)
|
Emergent
|
107 (9)
|
Timing
|
|
Day
|
769 (66)
|
Night
|
248 (21)
|
Overnight
|
157 (13)
|
Location
|
|
Intensive care unit
|
494 (42)
|
Ward
|
482 (41)
|
Emergency department
|
157 (13)
|
Other
|
41 (3)
|
Table 2: Beth Israel Deaconess Internal Medicine Residency 2007–2008 Procedural Competency Guidelines for Interns Performing Central Venous Line Placements (CVL)(Go to table citation in commentary)
Procedure
|
Minimum Requirement
|
Competence
|
---|---|---|
CVL placement
|
Completion of simulation training course (prior to performing on patients)
|
Completion of simulation training course (prior to performing on patients)
|
Achieve minimum passing score on CVL simulation checklist and cognitive quiz
|
Achieve mastery score on CVL simulation checklist
|
|
2 supervised procedures with an average overall score of 3* or greater
|
Minimum of 7 documented and supervised procedures
|
|
|
Average overall score on last 2 procedures: 4.5* or greater
|
|
|
No areas of deficiency identified on last 2 procedures
|
|
|
Achieve mastery score on CVL simulation checklist
|
|
*Rating Scale for Assessment
|
||
1
|
Deficient: unable to complete procedure without assistance.
|
|
2
|
|
|
3
|
Adequate skill: knew essential skills to complete procedure.
|
|
4
|
|
|
5
|
Proficient: demonstrated mastery of procedure skills.
|
Table 3: Components of the MPS Address All Levels of Miller's Pyramid(Go to table citation in commentary)
Behavioral assessment
|
Does
|
Procedures on real patients
|
Shows
|
Performance on simulators
|
|
Cognitive assessment
|
Knows how
|
Faculty evaluation
|
Knows
|
Self-assessment quizzes
|
Figures
Figure 1. Sample Portion of the Self-Assessment Quiz for Central Venous Line Placement.(Go to figure citation in commentary)
Figure 2. How Different Sources of Data Address Kirkpatrick's Levels of Evaluation.(Go to figure citation in commentary)