Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure

James M. Gerard, David O. Kessler, Colleen Braun, Renuka Mehta, Anthony J. Scalzo, Marc Auerbach

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Introduction: The Patient Outcomes in Simulation Education network has developed tools for the assessment of competency to perform the infant lumbar puncture (ILP) procedure. The objective of this study was to evaluate the validity and reliability of these tools in a simulated setting. Methods: We developed a 4-point anchored global rating scale (GRS) and 15-item dichotomous checklist instrument to assess ILP performance in a simulated environment. Video recordings of 60 subjects performing an unsupervised lumbar puncture on an infant bench top simulator were collected prospectively; 20 performed by subjects in each of 3 categories (beginner, intermediate experienced, or expert). Three blinded, expert raters independently scored each subject's video recording using the GRS and checklist instruments. Results: The final version of the scoring instruments is presented. Across all subject groups, higher GRS scores were found with advancing level of experience (P < 0.01). Total checklist scores were similar between the expert and intermediate experienced groups (P = 0.54). Both groups scored higher than the beginner group on the checklist instrument (P < 0.01). For each rater, a significant positive correlation was found between GRS scores and total checklist scores (median Q = 0.75, P < 0.01). Cronbach α coefficient for the checklist was 0.77. The intraclass correlation coefficients between raters for the GRS and total checklist scores were 0.71 and 0.52, respectively. Conclusions: This study provides some initial evidence to support the validity and reliability of the ILP-anchored GRS. Acceptable internal consistency was found for the checklist instrument. The GRS instrument outperformed the checklist in its discriminant ability and interrater agreement.

Original languageEnglish (US)
Pages (from-to)148-154
Number of pages7
JournalSimulation in Healthcare
Volume8
Issue number3
DOIs
StatePublished - Jun 1 2013

Fingerprint

Spinal Puncture
rating scale
Checklist
infant
Video recording
Instrument scales
video recording
expert
Video Recording
Group
Reproducibility of Results
Intraclass Correlation Coefficient
Internal Consistency
Simulators
Education
P-groups
Scoring
Discriminant
Simulator
simulation

Keywords

  • Checklist
  • Global rating scale
  • Infant lumbar puncture
  • Reliability
  • Validity

ASJC Scopus subject areas

  • Epidemiology
  • Medicine (miscellaneous)
  • Education
  • Modeling and Simulation

Cite this

Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure. / Gerard, James M.; Kessler, David O.; Braun, Colleen; Mehta, Renuka; Scalzo, Anthony J.; Auerbach, Marc.

In: Simulation in Healthcare, Vol. 8, No. 3, 01.06.2013, p. 148-154.

Research output: Contribution to journalArticle

Gerard, James M. ; Kessler, David O. ; Braun, Colleen ; Mehta, Renuka ; Scalzo, Anthony J. ; Auerbach, Marc. / Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure. In: Simulation in Healthcare. 2013 ; Vol. 8, No. 3. pp. 148-154.
@article{75037978972342458b35cc988c47a640,
title = "Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure",
abstract = "Introduction: The Patient Outcomes in Simulation Education network has developed tools for the assessment of competency to perform the infant lumbar puncture (ILP) procedure. The objective of this study was to evaluate the validity and reliability of these tools in a simulated setting. Methods: We developed a 4-point anchored global rating scale (GRS) and 15-item dichotomous checklist instrument to assess ILP performance in a simulated environment. Video recordings of 60 subjects performing an unsupervised lumbar puncture on an infant bench top simulator were collected prospectively; 20 performed by subjects in each of 3 categories (beginner, intermediate experienced, or expert). Three blinded, expert raters independently scored each subject's video recording using the GRS and checklist instruments. Results: The final version of the scoring instruments is presented. Across all subject groups, higher GRS scores were found with advancing level of experience (P < 0.01). Total checklist scores were similar between the expert and intermediate experienced groups (P = 0.54). Both groups scored higher than the beginner group on the checklist instrument (P < 0.01). For each rater, a significant positive correlation was found between GRS scores and total checklist scores (median Q = 0.75, P < 0.01). Cronbach α coefficient for the checklist was 0.77. The intraclass correlation coefficients between raters for the GRS and total checklist scores were 0.71 and 0.52, respectively. Conclusions: This study provides some initial evidence to support the validity and reliability of the ILP-anchored GRS. Acceptable internal consistency was found for the checklist instrument. The GRS instrument outperformed the checklist in its discriminant ability and interrater agreement.",
keywords = "Checklist, Global rating scale, Infant lumbar puncture, Reliability, Validity",
author = "Gerard, {James M.} and Kessler, {David O.} and Colleen Braun and Renuka Mehta and Scalzo, {Anthony J.} and Marc Auerbach",
year = "2013",
month = "6",
day = "1",
doi = "10.1097/SIH.0b013e3182802d34",
language = "English (US)",
volume = "8",
pages = "148--154",
journal = "Simulation in Healthcare",
issn = "1559-2332",
publisher = "Lippincott Williams and Wilkins",
number = "3",

}

TY - JOUR

T1 - Validation of global rating scale and checklist instruments for the infant lumbar puncture procedure

AU - Gerard, James M.

AU - Kessler, David O.

AU - Braun, Colleen

AU - Mehta, Renuka

AU - Scalzo, Anthony J.

AU - Auerbach, Marc

PY - 2013/6/1

Y1 - 2013/6/1

N2 - Introduction: The Patient Outcomes in Simulation Education network has developed tools for the assessment of competency to perform the infant lumbar puncture (ILP) procedure. The objective of this study was to evaluate the validity and reliability of these tools in a simulated setting. Methods: We developed a 4-point anchored global rating scale (GRS) and 15-item dichotomous checklist instrument to assess ILP performance in a simulated environment. Video recordings of 60 subjects performing an unsupervised lumbar puncture on an infant bench top simulator were collected prospectively; 20 performed by subjects in each of 3 categories (beginner, intermediate experienced, or expert). Three blinded, expert raters independently scored each subject's video recording using the GRS and checklist instruments. Results: The final version of the scoring instruments is presented. Across all subject groups, higher GRS scores were found with advancing level of experience (P < 0.01). Total checklist scores were similar between the expert and intermediate experienced groups (P = 0.54). Both groups scored higher than the beginner group on the checklist instrument (P < 0.01). For each rater, a significant positive correlation was found between GRS scores and total checklist scores (median Q = 0.75, P < 0.01). Cronbach α coefficient for the checklist was 0.77. The intraclass correlation coefficients between raters for the GRS and total checklist scores were 0.71 and 0.52, respectively. Conclusions: This study provides some initial evidence to support the validity and reliability of the ILP-anchored GRS. Acceptable internal consistency was found for the checklist instrument. The GRS instrument outperformed the checklist in its discriminant ability and interrater agreement.

AB - Introduction: The Patient Outcomes in Simulation Education network has developed tools for the assessment of competency to perform the infant lumbar puncture (ILP) procedure. The objective of this study was to evaluate the validity and reliability of these tools in a simulated setting. Methods: We developed a 4-point anchored global rating scale (GRS) and 15-item dichotomous checklist instrument to assess ILP performance in a simulated environment. Video recordings of 60 subjects performing an unsupervised lumbar puncture on an infant bench top simulator were collected prospectively; 20 performed by subjects in each of 3 categories (beginner, intermediate experienced, or expert). Three blinded, expert raters independently scored each subject's video recording using the GRS and checklist instruments. Results: The final version of the scoring instruments is presented. Across all subject groups, higher GRS scores were found with advancing level of experience (P < 0.01). Total checklist scores were similar between the expert and intermediate experienced groups (P = 0.54). Both groups scored higher than the beginner group on the checklist instrument (P < 0.01). For each rater, a significant positive correlation was found between GRS scores and total checklist scores (median Q = 0.75, P < 0.01). Cronbach α coefficient for the checklist was 0.77. The intraclass correlation coefficients between raters for the GRS and total checklist scores were 0.71 and 0.52, respectively. Conclusions: This study provides some initial evidence to support the validity and reliability of the ILP-anchored GRS. Acceptable internal consistency was found for the checklist instrument. The GRS instrument outperformed the checklist in its discriminant ability and interrater agreement.

KW - Checklist

KW - Global rating scale

KW - Infant lumbar puncture

KW - Reliability

KW - Validity

UR - http://www.scopus.com/inward/record.url?scp=84880173596&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84880173596&partnerID=8YFLogxK

U2 - 10.1097/SIH.0b013e3182802d34

DO - 10.1097/SIH.0b013e3182802d34

M3 - Article

C2 - 23388627

AN - SCOPUS:84880173596

VL - 8

SP - 148

EP - 154

JO - Simulation in Healthcare

JF - Simulation in Healthcare

SN - 1559-2332

IS - 3

ER -