Training less-experienced faculty improves reliability of skills assessment in cardiac surgery

Xiaoying Lou, Richard Lee, Richard H. Feins, Daniel Enter, George L. Hicks, Edward D. Verrier, James I. Fann

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.

Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.

Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.

Original languageEnglish (US)
Pages (from-to)2491-2496.e2
JournalJournal of Thoracic and Cardiovascular Surgery
Volume148
Issue number6
DOIs
StatePublished - Dec 1 2014
Externally publishedYes

Fingerprint

Thoracic Surgery
Needles
Bites and Stings
Mechanics
Teaching
Hand

ASJC Scopus subject areas

  • Surgery
  • Pulmonary and Respiratory Medicine
  • Cardiology and Cardiovascular Medicine

Cite this

Training less-experienced faculty improves reliability of skills assessment in cardiac surgery. / Lou, Xiaoying; Lee, Richard; Feins, Richard H.; Enter, Daniel; Hicks, George L.; Verrier, Edward D.; Fann, James I.

In: Journal of Thoracic and Cardiovascular Surgery, Vol. 148, No. 6, 01.12.2014, p. 2491-2496.e2.

Research output: Contribution to journalArticle

Lou, Xiaoying ; Lee, Richard ; Feins, Richard H. ; Enter, Daniel ; Hicks, George L. ; Verrier, Edward D. ; Fann, James I. / Training less-experienced faculty improves reliability of skills assessment in cardiac surgery. In: Journal of Thoracic and Cardiovascular Surgery. 2014 ; Vol. 148, No. 6. pp. 2491-2496.e2.
@article{b5db44d258054098a1c34b15530585ed,
title = "Training less-experienced faculty improves reliability of skills assessment in cardiac surgery",
abstract = "Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.",
author = "Xiaoying Lou and Richard Lee and Feins, {Richard H.} and Daniel Enter and Hicks, {George L.} and Verrier, {Edward D.} and Fann, {James I.}",
year = "2014",
month = "12",
day = "1",
doi = "10.1016/j.jtcvs.2014.09.017",
language = "English (US)",
volume = "148",
pages = "2491--2496.e2",
journal = "Journal of Thoracic and Cardiovascular Surgery",
issn = "0022-5223",
publisher = "Mosby Inc.",
number = "6",

}

TY - JOUR

T1 - Training less-experienced faculty improves reliability of skills assessment in cardiac surgery

AU - Lou, Xiaoying

AU - Lee, Richard

AU - Feins, Richard H.

AU - Enter, Daniel

AU - Hicks, George L.

AU - Verrier, Edward D.

AU - Fann, James I.

PY - 2014/12/1

Y1 - 2014/12/1

N2 - Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.

AB - Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.

UR - http://www.scopus.com/inward/record.url?scp=84920024046&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84920024046&partnerID=8YFLogxK

U2 - 10.1016/j.jtcvs.2014.09.017

DO - 10.1016/j.jtcvs.2014.09.017

M3 - Article

C2 - 25308119

AN - SCOPUS:84920024046

VL - 148

SP - 2491-2496.e2

JO - Journal of Thoracic and Cardiovascular Surgery

JF - Journal of Thoracic and Cardiovascular Surgery

SN - 0022-5223

IS - 6

ER -