Local Development of MCQ Tests for Evidence-based Medicine and Clinical Decision Making Can Be Successful

Gerald E. Crites, Ronald J. Markert, Donald Goggans, Scott Richardson

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Background: Guidelines for the design of multiple-choice item (MCQ) tests of evidence-based medicine (EBM) and clinical decision making (CDM) have not been published. Purpose and Methods: We describe a strategy to develop an EBM/CDM MCQ test database guided by educational theory and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy's success. Results: The internal consistency reliability of tests derived from the database was in the good-to-excellent range (0.74-0.95) and test-retest reliability was fair (0.51). One test discriminated across three levels of EBM/CDM learners (discriminant validity). Tests also predictively correlated with other medical school assessments according to theory (convergent and discriminant validity). The items were infrequently misclassified, had statistics close to historical standards, and were acceptable after no more than one round of revisions. Conclusions: Our strategy for developing an EBM/CDM MCQ database was successful and tests derived from it can be flexibly sampled to assess different EBM/CDM knowledge domains and three levels of EBM/CDM learners. Assuming the availability of similar resources to support its application, this strategy should be replicable at other settings.

Original languageEnglish (US)
Pages (from-to)341-347
Number of pages7
JournalTeaching and Learning in Medicine
Volume24
Issue number4
DOIs
StatePublished - Oct 1 2012

Fingerprint

Evidence-Based Medicine
medicine
decision making
evidence
Databases
Reproducibility of Results
educational theory
Medical Schools
Psychometrics
Clinical Decision-Making
psychometrics
Guidelines
statistics
resources
school
knowledge

ASJC Scopus subject areas

  • Medicine(all)
  • Education

Cite this

Local Development of MCQ Tests for Evidence-based Medicine and Clinical Decision Making Can Be Successful. / Crites, Gerald E.; Markert, Ronald J.; Goggans, Donald; Richardson, Scott.

In: Teaching and Learning in Medicine, Vol. 24, No. 4, 01.10.2012, p. 341-347.

Research output: Contribution to journalArticle

@article{9728d3cb024840b795dcd2afd5699586,
title = "Local Development of MCQ Tests for Evidence-based Medicine and Clinical Decision Making Can Be Successful",
abstract = "Background: Guidelines for the design of multiple-choice item (MCQ) tests of evidence-based medicine (EBM) and clinical decision making (CDM) have not been published. Purpose and Methods: We describe a strategy to develop an EBM/CDM MCQ test database guided by educational theory and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy's success. Results: The internal consistency reliability of tests derived from the database was in the good-to-excellent range (0.74-0.95) and test-retest reliability was fair (0.51). One test discriminated across three levels of EBM/CDM learners (discriminant validity). Tests also predictively correlated with other medical school assessments according to theory (convergent and discriminant validity). The items were infrequently misclassified, had statistics close to historical standards, and were acceptable after no more than one round of revisions. Conclusions: Our strategy for developing an EBM/CDM MCQ database was successful and tests derived from it can be flexibly sampled to assess different EBM/CDM knowledge domains and three levels of EBM/CDM learners. Assuming the availability of similar resources to support its application, this strategy should be replicable at other settings.",
author = "Crites, {Gerald E.} and Markert, {Ronald J.} and Donald Goggans and Scott Richardson",
year = "2012",
month = "10",
day = "1",
doi = "10.1080/10401334.2012.715258",
language = "English (US)",
volume = "24",
pages = "341--347",
journal = "Teaching and Learning in Medicine",
issn = "1040-1334",
publisher = "Routledge",
number = "4",

}

TY - JOUR

T1 - Local Development of MCQ Tests for Evidence-based Medicine and Clinical Decision Making Can Be Successful

AU - Crites, Gerald E.

AU - Markert, Ronald J.

AU - Goggans, Donald

AU - Richardson, Scott

PY - 2012/10/1

Y1 - 2012/10/1

N2 - Background: Guidelines for the design of multiple-choice item (MCQ) tests of evidence-based medicine (EBM) and clinical decision making (CDM) have not been published. Purpose and Methods: We describe a strategy to develop an EBM/CDM MCQ test database guided by educational theory and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy's success. Results: The internal consistency reliability of tests derived from the database was in the good-to-excellent range (0.74-0.95) and test-retest reliability was fair (0.51). One test discriminated across three levels of EBM/CDM learners (discriminant validity). Tests also predictively correlated with other medical school assessments according to theory (convergent and discriminant validity). The items were infrequently misclassified, had statistics close to historical standards, and were acceptable after no more than one round of revisions. Conclusions: Our strategy for developing an EBM/CDM MCQ database was successful and tests derived from it can be flexibly sampled to assess different EBM/CDM knowledge domains and three levels of EBM/CDM learners. Assuming the availability of similar resources to support its application, this strategy should be replicable at other settings.

AB - Background: Guidelines for the design of multiple-choice item (MCQ) tests of evidence-based medicine (EBM) and clinical decision making (CDM) have not been published. Purpose and Methods: We describe a strategy to develop an EBM/CDM MCQ test database guided by educational theory and used psychometric analyses, including reliability, validity, and item analyses, to judge the strategy's success. Results: The internal consistency reliability of tests derived from the database was in the good-to-excellent range (0.74-0.95) and test-retest reliability was fair (0.51). One test discriminated across three levels of EBM/CDM learners (discriminant validity). Tests also predictively correlated with other medical school assessments according to theory (convergent and discriminant validity). The items were infrequently misclassified, had statistics close to historical standards, and were acceptable after no more than one round of revisions. Conclusions: Our strategy for developing an EBM/CDM MCQ database was successful and tests derived from it can be flexibly sampled to assess different EBM/CDM knowledge domains and three levels of EBM/CDM learners. Assuming the availability of similar resources to support its application, this strategy should be replicable at other settings.

UR - http://www.scopus.com/inward/record.url?scp=84867226270&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84867226270&partnerID=8YFLogxK

U2 - 10.1080/10401334.2012.715258

DO - 10.1080/10401334.2012.715258

M3 - Article

C2 - 23036002

AN - SCOPUS:84867226270

VL - 24

SP - 341

EP - 347

JO - Teaching and Learning in Medicine

JF - Teaching and Learning in Medicine

SN - 1040-1334

IS - 4

ER -