Back to Top Skip to main content

Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training.

Publication Status: Published

Sponsoring Organization: Defense Health Agency (formerly TRICARE Management Activity)

Sponsoring Office: Uniformed Services University of Health Sciences

Congressionally Mandated: No

Funding Source: Undetermined

Release Date/Publication: July 01, 2015

Principle Investigator Status: Academia

Primary DoD Data Source: Other Clinical

Secondary DoD Data Source:

Abstract

BACKGROUND:
Maintaining trauma-specific surgical skills is an ongoing challenge for surgical training programs. An objective assessment of surgical skills is needed. We hypothesized that a validated surgical performance assessment tool could detect differences following a training intervention.

METHODS:
We developed surgical performance assessment metrics based on discussion with expert trauma surgeons, video review of 10 experts and 10 novice surgeons performing three vascular exposure procedures and lower extremity fasciotomy on cadavers, and validated the metrics with interrater reliability testing by five reviewers blinded to level of expertise and a consensus conference. We tested these performance metrics in 12 surgical residents (Year 3-7) before and 2 weeks after vascular exposure skills training in the Advanced Surgical Skills for Exposure in Trauma (ASSET) course. Performance was assessed in three areas as follows: knowledge (anatomic, management), procedure steps, and technical skills. Time to completion of procedures was recorded, and these metrics were combined into a single performance score, the Trauma Readiness Index (TRI). Wilcoxon matched-pairs signed-ranks test compared pretraining/posttraining effects.

RESULTS:
Mean time to complete procedures decreased by 4.3 minutes (from 13.4 minutes to 9.1 minutes). The performance component most improved by the 1-day skills training was procedure steps, completion of which increased by 21%. Technical skill scores improved by 12%. Overall knowledge improved by 3%, with 18% improvement in anatomic knowledge. TRI increased significantly from 50% to 64% with ASSET training. Interrater reliability of the surgical performance assessment metrics was validated with single intraclass correlation coefficient of 0.7 to 0.98.

CONCLUSION:
A trauma-relevant surgical performance assessment detected improvements in specific procedure steps and anatomic knowledge taught during a 1-day course, quantified by the TRI. ASSET training reduced time to complete vascular control by one third. Future applications include assessing specific skills in a larger surgeon cohort, assessing military surgical readiness, and quantifying skill degradation with time since training.

Citation:

Shackelford S, et. al., . Development and validation of trauma surgical skills metrics: Preliminary assessment of performance after training. J Trauma Acute Care Surg. 2015 Jul;79(1):105-10.

DHA Address: 7700 Arlington Boulevard | Suite 5101 | Falls Church, VA | 22042-5101

Some documents are presented in Portable Document Format (PDF). A PDF reader is required for viewing. Download a PDF Reader or learn more about PDFs.