TY - JOUR
T1 - Training less-experienced faculty improves reliability of skills assessment in cardiac surgery
AU - Lou, Xiaoying
AU - Lee, Richard
AU - Feins, Richard H.
AU - Enter, Daniel
AU - Hicks, George L.
AU - Verrier, Edward D.
AU - Fann, James I.
PY - 2014/12/1
Y1 - 2014/12/1
N2 - Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.
AB - Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.
UR - http://www.scopus.com/inward/record.url?scp=84920024046&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84920024046&partnerID=8YFLogxK
U2 - 10.1016/j.jtcvs.2014.09.017
DO - 10.1016/j.jtcvs.2014.09.017
M3 - Article
C2 - 25308119
AN - SCOPUS:84920024046
VL - 148
SP - 2491-2496.e2
JO - Journal of Thoracic and Cardiovascular Surgery
JF - Journal of Thoracic and Cardiovascular Surgery
SN - 0022-5223
IS - 6
ER -