Script concordance testing: Assessing residents' clinical decision-making skills for infant lumbar punctures

Todd P. Chang, David Kessler, Brett McAninch, Daniel M. Fein, D. J. Scherzer, Elizabeth Seelbach, Pavan Zaveri, Jennifer M. Jackson, Marc Auerbach, Renuka Mehta, Wendy Van Ittersum, Martin V. Pusic

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

PURPOSE: Residents must learn which infants require a lumbar puncture (LP), a clinical decision-making skill (CDMS) difficult to evaluate because of considerable practice variation. The authors created an assessment model of the CDMS to determine when an LP is indicated, taking practice variation into account. The objective was to detect whether script concordance testing (SCT) could measure CDMS competency among residents for performing infant LPs. METHOD: In 2011, using a modified Delphi technique, an expert panel of 14 attending physicians constructed 15 case vignettes (each with 2 to 4 SCT questions) that represented various infant LP scenarios. The authors distributed the vignettes to residents at 10 academic pediatric centers within the International Simulation in Pediatric Innovation, Research, and Education Network. They compared SCT scores among residents of different postgraduate years (PGYs), specialties, training in adult medicine, LP experience, and practice within an endemic Lyme disease area. RESULTS: Of 730 eligible residents, 102 completed 47 SCT questions. They could earn a maximum score of 47. Median SCT scores were significantly higher in PGY-3s compared with PGY-1s (difference: 3.0; 95% confidence interval [CI] 1.0-4.9; effect size d = 0.87). Scores also increased with increasing LP experience (difference: 3.3; 95% CI 1.1-5.5) and with adult medicine training (difference: 2.9; 95% CI 0.6-5.0). Residents in Lyme-endemic areas tended to perform more LPs than those in nonendemic areas. CONCLUSIONS: SCT questions may be useful as an assessment tool to determine CDMS competency among residents for performing infant LPs.

Original languageEnglish (US)
Pages (from-to)128-135
Number of pages8
JournalAcademic Medicine
Volume89
Issue number1
DOIs
StatePublished - Jan 2014

Fingerprint

infant
resident
decision making
confidence
medicine
innovation research
experience
physician
expert
scenario
Disease
simulation
education

ASJC Scopus subject areas

  • Education

Cite this

Chang, T. P., Kessler, D., McAninch, B., Fein, D. M., Scherzer, D. J., Seelbach, E., ... Pusic, M. V. (2014). Script concordance testing: Assessing residents' clinical decision-making skills for infant lumbar punctures. Academic Medicine, 89(1), 128-135. https://doi.org/10.1097/ACM.0000000000000059

Script concordance testing : Assessing residents' clinical decision-making skills for infant lumbar punctures. / Chang, Todd P.; Kessler, David; McAninch, Brett; Fein, Daniel M.; Scherzer, D. J.; Seelbach, Elizabeth; Zaveri, Pavan; Jackson, Jennifer M.; Auerbach, Marc; Mehta, Renuka; Van Ittersum, Wendy; Pusic, Martin V.

In: Academic Medicine, Vol. 89, No. 1, 01.2014, p. 128-135.

Research output: Contribution to journalArticle

Chang, TP, Kessler, D, McAninch, B, Fein, DM, Scherzer, DJ, Seelbach, E, Zaveri, P, Jackson, JM, Auerbach, M, Mehta, R, Van Ittersum, W & Pusic, MV 2014, 'Script concordance testing: Assessing residents' clinical decision-making skills for infant lumbar punctures', Academic Medicine, vol. 89, no. 1, pp. 128-135. https://doi.org/10.1097/ACM.0000000000000059
Chang, Todd P. ; Kessler, David ; McAninch, Brett ; Fein, Daniel M. ; Scherzer, D. J. ; Seelbach, Elizabeth ; Zaveri, Pavan ; Jackson, Jennifer M. ; Auerbach, Marc ; Mehta, Renuka ; Van Ittersum, Wendy ; Pusic, Martin V. / Script concordance testing : Assessing residents' clinical decision-making skills for infant lumbar punctures. In: Academic Medicine. 2014 ; Vol. 89, No. 1. pp. 128-135.
@article{dce0a19f5194484ca754dcbe4afc8b4b,
title = "Script concordance testing: Assessing residents' clinical decision-making skills for infant lumbar punctures",
abstract = "PURPOSE: Residents must learn which infants require a lumbar puncture (LP), a clinical decision-making skill (CDMS) difficult to evaluate because of considerable practice variation. The authors created an assessment model of the CDMS to determine when an LP is indicated, taking practice variation into account. The objective was to detect whether script concordance testing (SCT) could measure CDMS competency among residents for performing infant LPs. METHOD: In 2011, using a modified Delphi technique, an expert panel of 14 attending physicians constructed 15 case vignettes (each with 2 to 4 SCT questions) that represented various infant LP scenarios. The authors distributed the vignettes to residents at 10 academic pediatric centers within the International Simulation in Pediatric Innovation, Research, and Education Network. They compared SCT scores among residents of different postgraduate years (PGYs), specialties, training in adult medicine, LP experience, and practice within an endemic Lyme disease area. RESULTS: Of 730 eligible residents, 102 completed 47 SCT questions. They could earn a maximum score of 47. Median SCT scores were significantly higher in PGY-3s compared with PGY-1s (difference: 3.0; 95{\%} confidence interval [CI] 1.0-4.9; effect size d = 0.87). Scores also increased with increasing LP experience (difference: 3.3; 95{\%} CI 1.1-5.5) and with adult medicine training (difference: 2.9; 95{\%} CI 0.6-5.0). Residents in Lyme-endemic areas tended to perform more LPs than those in nonendemic areas. CONCLUSIONS: SCT questions may be useful as an assessment tool to determine CDMS competency among residents for performing infant LPs.",
author = "Chang, {Todd P.} and David Kessler and Brett McAninch and Fein, {Daniel M.} and Scherzer, {D. J.} and Elizabeth Seelbach and Pavan Zaveri and Jackson, {Jennifer M.} and Marc Auerbach and Renuka Mehta and {Van Ittersum}, Wendy and Pusic, {Martin V.}",
year = "2014",
month = "1",
doi = "10.1097/ACM.0000000000000059",
language = "English (US)",
volume = "89",
pages = "128--135",
journal = "Academic Medicine",
issn = "1040-2446",
publisher = "Lippincott Williams and Wilkins",
number = "1",

}

TY - JOUR

T1 - Script concordance testing

T2 - Assessing residents' clinical decision-making skills for infant lumbar punctures

AU - Chang, Todd P.

AU - Kessler, David

AU - McAninch, Brett

AU - Fein, Daniel M.

AU - Scherzer, D. J.

AU - Seelbach, Elizabeth

AU - Zaveri, Pavan

AU - Jackson, Jennifer M.

AU - Auerbach, Marc

AU - Mehta, Renuka

AU - Van Ittersum, Wendy

AU - Pusic, Martin V.

PY - 2014/1

Y1 - 2014/1

N2 - PURPOSE: Residents must learn which infants require a lumbar puncture (LP), a clinical decision-making skill (CDMS) difficult to evaluate because of considerable practice variation. The authors created an assessment model of the CDMS to determine when an LP is indicated, taking practice variation into account. The objective was to detect whether script concordance testing (SCT) could measure CDMS competency among residents for performing infant LPs. METHOD: In 2011, using a modified Delphi technique, an expert panel of 14 attending physicians constructed 15 case vignettes (each with 2 to 4 SCT questions) that represented various infant LP scenarios. The authors distributed the vignettes to residents at 10 academic pediatric centers within the International Simulation in Pediatric Innovation, Research, and Education Network. They compared SCT scores among residents of different postgraduate years (PGYs), specialties, training in adult medicine, LP experience, and practice within an endemic Lyme disease area. RESULTS: Of 730 eligible residents, 102 completed 47 SCT questions. They could earn a maximum score of 47. Median SCT scores were significantly higher in PGY-3s compared with PGY-1s (difference: 3.0; 95% confidence interval [CI] 1.0-4.9; effect size d = 0.87). Scores also increased with increasing LP experience (difference: 3.3; 95% CI 1.1-5.5) and with adult medicine training (difference: 2.9; 95% CI 0.6-5.0). Residents in Lyme-endemic areas tended to perform more LPs than those in nonendemic areas. CONCLUSIONS: SCT questions may be useful as an assessment tool to determine CDMS competency among residents for performing infant LPs.

AB - PURPOSE: Residents must learn which infants require a lumbar puncture (LP), a clinical decision-making skill (CDMS) difficult to evaluate because of considerable practice variation. The authors created an assessment model of the CDMS to determine when an LP is indicated, taking practice variation into account. The objective was to detect whether script concordance testing (SCT) could measure CDMS competency among residents for performing infant LPs. METHOD: In 2011, using a modified Delphi technique, an expert panel of 14 attending physicians constructed 15 case vignettes (each with 2 to 4 SCT questions) that represented various infant LP scenarios. The authors distributed the vignettes to residents at 10 academic pediatric centers within the International Simulation in Pediatric Innovation, Research, and Education Network. They compared SCT scores among residents of different postgraduate years (PGYs), specialties, training in adult medicine, LP experience, and practice within an endemic Lyme disease area. RESULTS: Of 730 eligible residents, 102 completed 47 SCT questions. They could earn a maximum score of 47. Median SCT scores were significantly higher in PGY-3s compared with PGY-1s (difference: 3.0; 95% confidence interval [CI] 1.0-4.9; effect size d = 0.87). Scores also increased with increasing LP experience (difference: 3.3; 95% CI 1.1-5.5) and with adult medicine training (difference: 2.9; 95% CI 0.6-5.0). Residents in Lyme-endemic areas tended to perform more LPs than those in nonendemic areas. CONCLUSIONS: SCT questions may be useful as an assessment tool to determine CDMS competency among residents for performing infant LPs.

UR - http://www.scopus.com/inward/record.url?scp=84893712430&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84893712430&partnerID=8YFLogxK

U2 - 10.1097/ACM.0000000000000059

DO - 10.1097/ACM.0000000000000059

M3 - Article

C2 - 24280838

AN - SCOPUS:84893712430

VL - 89

SP - 128

EP - 135

JO - Academic Medicine

JF - Academic Medicine

SN - 1040-2446

IS - 1

ER -