TY - JOUR
T1 - The joint council on thoracic surgery education coronary artery assessment tool has high interrater reliability
AU - Lee, Richard
AU - Enter, Daniel
AU - Lou, Xiaoying
AU - Feins, Richard H.
AU - Hicks, George L.
AU - Gasparri, Mario
AU - Takayama, Hiroo
AU - Young, J. Nilas
AU - Calhoon, John H.
AU - Crawford, Fred A.
AU - Mokadam, Nahush A.
AU - Fann, James I.
PY - 2013/6/1
Y1 - 2013/6/1
N2 - Background: Barriers to incorporation of simulation in cardiothoracic surgery training include lack of standardized, validated objective assessment tools. Our aim was to measure interrater reliability and internal consistency reliability of a coronary anastomosis assessment tool created by the Joint Council on Thoracic Surgery Education. Methods: Ten attending surgeons from different cardiothoracic residency programs evaluated nine video recordings of 5 individuals (1 medical student, 1 resident, 1 fellow, 2 attendings) performing coronary anastomoses on two simulation models, including synthetic graft task station (low fidelity) and porcine explant (high fidelity), as well as in the operative setting. All raters, blinded to operator identity, scored 13 assessment items on a 1 to 5 (low to high) scale. Each performance also received an overall pass/fail determination. Interrater reliability and internal consistency were assessed as intraclass correlation coefficients and Cronbach's α, respectively. Results: Both interrater reliability and internal consistency were high for all three models (intraclass correlation coefficients = 0.98, 0.99, and 0.94, and Cronbach's α = 0.99, 0.98, and 0.97 for low fidelity, high fidelity, and operative setting, respectively). Interrater reliability for overall pass/fail determination using κ were 0.54, 0.86, 0.15 for low fidelity, high fidelity, and operative setting, respectively. Conclusions: Even without instruction on the assessment tool, experienced surgeons achieved high interrater reliability. Future resident training and evaluation may benefit from utilization of this tool for formative feedback in the simulated and operative environments. However, summative assessment in the operative setting will require further standardization and anchoring.
AB - Background: Barriers to incorporation of simulation in cardiothoracic surgery training include lack of standardized, validated objective assessment tools. Our aim was to measure interrater reliability and internal consistency reliability of a coronary anastomosis assessment tool created by the Joint Council on Thoracic Surgery Education. Methods: Ten attending surgeons from different cardiothoracic residency programs evaluated nine video recordings of 5 individuals (1 medical student, 1 resident, 1 fellow, 2 attendings) performing coronary anastomoses on two simulation models, including synthetic graft task station (low fidelity) and porcine explant (high fidelity), as well as in the operative setting. All raters, blinded to operator identity, scored 13 assessment items on a 1 to 5 (low to high) scale. Each performance also received an overall pass/fail determination. Interrater reliability and internal consistency were assessed as intraclass correlation coefficients and Cronbach's α, respectively. Results: Both interrater reliability and internal consistency were high for all three models (intraclass correlation coefficients = 0.98, 0.99, and 0.94, and Cronbach's α = 0.99, 0.98, and 0.97 for low fidelity, high fidelity, and operative setting, respectively). Interrater reliability for overall pass/fail determination using κ were 0.54, 0.86, 0.15 for low fidelity, high fidelity, and operative setting, respectively. Conclusions: Even without instruction on the assessment tool, experienced surgeons achieved high interrater reliability. Future resident training and evaluation may benefit from utilization of this tool for formative feedback in the simulated and operative environments. However, summative assessment in the operative setting will require further standardization and anchoring.
UR - http://www.scopus.com/inward/record.url?scp=84878244077&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=84878244077&partnerID=8YFLogxK
U2 - 10.1016/j.athoracsur.2012.10.090
DO - 10.1016/j.athoracsur.2012.10.090
M3 - Article
C2 - 23706430
AN - SCOPUS:84878244077
SN - 0003-4975
VL - 95
SP - 2064
EP - 2070
JO - Annals of Thoracic Surgery
JF - Annals of Thoracic Surgery
IS - 6
ER -