Abstract
Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.
Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.
Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.
| Original language | English (US) |
|---|---|
| Pages (from-to) | 2491-2496.e2 |
| Journal | Journal of Thoracic and Cardiovascular Surgery |
| Volume | 148 |
| Issue number | 6 |
| DOIs | |
| State | Published - Dec 1 2014 |
| Externally published | Yes |
Fingerprint
ASJC Scopus subject areas
- Surgery
- Pulmonary and Respiratory Medicine
- Cardiology and Cardiovascular Medicine
Cite this
Training less-experienced faculty improves reliability of skills assessment in cardiac surgery. / Lou, Xiaoying; Lee, Richard; Feins, Richard H.; Enter, Daniel; Hicks, George L.; Verrier, Edward D.; Fann, James I.
In: Journal of Thoracic and Cardiovascular Surgery, Vol. 148, No. 6, 01.12.2014, p. 2491-2496.e2.Research output: Contribution to journal › Article
}
TY - JOUR
T1 - Training less-experienced faculty improves reliability of skills assessment in cardiac surgery
AU - Lou, Xiaoying
AU - Lee, Richard
AU - Feins, Richard H.
AU - Enter, Daniel
AU - Hicks, George L.
AU - Verrier, Edward D.
AU - Fann, James I.
PY - 2014/12/1
Y1 - 2014/12/1
N2 - Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.
AB - Objective Previous work has demonstrated high inter-rater reliability in the objective assessment of simulated anastomoses among experienced educators. We evaluated the inter-rater reliability of less-experienced educators and the impact of focused training with a video-embedded coronary anastomosis assessment tool.Results All components of the assessment tool exhibited improvement in reliability, with 4 (bite, needle holder use, needle angles, and hand mechanics) improving the most from poor (ICC range, 0.09-0.48) to strong (ICC range, 0.80-0.90) agreement. After training, inter-rater reliabilities for composite scores improved from moderate (ICC, 0.76) to strong (ICC, 0.90) agreement, and for overall pass/fail ratings, from poor (kappa = 0.20) to moderate (kappa = 0.78) agreement.Conclusions Focused, video-based anchor training facilitates greater inter-rater reliability in the objective assessment of simulated coronary anastomoses. Among raters with less teaching experience, such training may be needed before objective evaluation of technical skills.
UR - http://www.scopus.com/inward/record.url?scp=84920024046&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84920024046&partnerID=8YFLogxK
U2 - 10.1016/j.jtcvs.2014.09.017
DO - 10.1016/j.jtcvs.2014.09.017
M3 - Article
C2 - 25308119
AN - SCOPUS:84920024046
VL - 148
SP - 2491-2496.e2
JO - Journal of Thoracic and Cardiovascular Surgery
JF - Journal of Thoracic and Cardiovascular Surgery
SN - 0022-5223
IS - 6
ER -