The inability of physicians to predict the outcome of in-hospital resuscitation

Mark H. Ebell, George R. Bergus, Lawrence Warbasse, Roger Bloomer

Research output: Contribution to journalArticle

39 Citations (Scopus)

Abstract

OBJECTIVE: To measure the accuracy, reliability, and discrimination of physicians' predictions of the outcome of in-hospital cardiopulmonary resuscitation (CPR), using a large series of detailed clinical vignettes of patients with known outcomes, DESIGN: Faculty and resident physicians at three university-affiliated generalist training programs were given one-page summaries of admission data for patients who later underwent in-hospital CPR. These summaries included all prearrest variables known to be related to the outcome of CPR. Physicians were asked to estimate the probability that patients would survive the resuscitation long enough to be stabilized, and the probability of survival to discharge. SETTING: Patient cases were derived from a consecutive series of patients undergoing CPR at two urban teaching hospitals in Detroit, Michigan. PARTICIPANTS: Faculty members and residents at a university-based department of internal medicine and two university- based departments of family medicine were surveyed. INTERVENTIONS: Accuracy of the physician predictions was assessed by comparing the mean predicted probability of survival with the percentage of patients who actually survived. The reliability of probability estimates of survival was evaluated by assessing the numerical proximity of the estimates to the actual outcome of the resuscitative effort. The ability to discriminate between survivors and nonsurvivors was measured by comparing the mean predicted probability of survival for those patients who survived CPR with that for those who did not, and by stratifying physician predictions and measuring the area under a receiver operating characteristic {ROC) curve. MEASUREMENTS AND MAIN RESULTS: Physicians (n = 51) made a total of 713 estimates, and showed poor accuracy, reliability, and discrimination in predicting the outcome of inhospital CPR. The mean predicted probability of survival to discharge did not differ between patients who actually survived to discharge and those who did not (29.5% vs 26.4%, z = 0.35, p = .73). Similarly, the mean predicted probabilities of surviving resuscitation were the same for patients who actually survived long enough to be stabilized and those who did not (37.8% vs 39.9%, z = 0.55, p = .58). Accounting for type of physician and institution by analysis of variance did not change this finding. The area under the ROC curve for the prediction of arrest survival was 0.476, which is not significantly different from 0.5, and is consistent with an ability to discriminate between survivors and nonsurvivors that is no better than random choice. CONCLUSIONS: Physicians were no better at identifying patients who would survive resuscitation than would be expected by chance alone. Further work is needed to establish which variables are used by physicians in the decision-making process, and to design educational interventions that will make physicians more accurate prognosticators.

Original languageEnglish (US)
Pages (from-to)16-22
Number of pages7
JournalJournal of General Internal Medicine
Volume11
Issue number1
DOIs
StatePublished - Jan 1 1996

Fingerprint

Resuscitation
Physicians
Cardiopulmonary Resuscitation
Survival
Aptitude
ROC Curve
Survivors
Patient Admission
Urban Hospitals
Internal Medicine
Teaching Hospitals
Decision Making
Analysis of Variance
Medicine
Education

Keywords

  • Resuscitation
  • cardiopulmonary resuscitation
  • prediction
  • receiver operating characteristic curve

ASJC Scopus subject areas

  • Internal Medicine

Cite this

The inability of physicians to predict the outcome of in-hospital resuscitation. / Ebell, Mark H.; Bergus, George R.; Warbasse, Lawrence; Bloomer, Roger.

In: Journal of General Internal Medicine, Vol. 11, No. 1, 01.01.1996, p. 16-22.

Research output: Contribution to journalArticle

Ebell, Mark H. ; Bergus, George R. ; Warbasse, Lawrence ; Bloomer, Roger. / The inability of physicians to predict the outcome of in-hospital resuscitation. In: Journal of General Internal Medicine. 1996 ; Vol. 11, No. 1. pp. 16-22.
@article{200c7478264049828e4878fe9d09a3d0,
title = "The inability of physicians to predict the outcome of in-hospital resuscitation",
abstract = "OBJECTIVE: To measure the accuracy, reliability, and discrimination of physicians' predictions of the outcome of in-hospital cardiopulmonary resuscitation (CPR), using a large series of detailed clinical vignettes of patients with known outcomes, DESIGN: Faculty and resident physicians at three university-affiliated generalist training programs were given one-page summaries of admission data for patients who later underwent in-hospital CPR. These summaries included all prearrest variables known to be related to the outcome of CPR. Physicians were asked to estimate the probability that patients would survive the resuscitation long enough to be stabilized, and the probability of survival to discharge. SETTING: Patient cases were derived from a consecutive series of patients undergoing CPR at two urban teaching hospitals in Detroit, Michigan. PARTICIPANTS: Faculty members and residents at a university-based department of internal medicine and two university- based departments of family medicine were surveyed. INTERVENTIONS: Accuracy of the physician predictions was assessed by comparing the mean predicted probability of survival with the percentage of patients who actually survived. The reliability of probability estimates of survival was evaluated by assessing the numerical proximity of the estimates to the actual outcome of the resuscitative effort. The ability to discriminate between survivors and nonsurvivors was measured by comparing the mean predicted probability of survival for those patients who survived CPR with that for those who did not, and by stratifying physician predictions and measuring the area under a receiver operating characteristic {ROC) curve. MEASUREMENTS AND MAIN RESULTS: Physicians (n = 51) made a total of 713 estimates, and showed poor accuracy, reliability, and discrimination in predicting the outcome of inhospital CPR. The mean predicted probability of survival to discharge did not differ between patients who actually survived to discharge and those who did not (29.5{\%} vs 26.4{\%}, z = 0.35, p = .73). Similarly, the mean predicted probabilities of surviving resuscitation were the same for patients who actually survived long enough to be stabilized and those who did not (37.8{\%} vs 39.9{\%}, z = 0.55, p = .58). Accounting for type of physician and institution by analysis of variance did not change this finding. The area under the ROC curve for the prediction of arrest survival was 0.476, which is not significantly different from 0.5, and is consistent with an ability to discriminate between survivors and nonsurvivors that is no better than random choice. CONCLUSIONS: Physicians were no better at identifying patients who would survive resuscitation than would be expected by chance alone. Further work is needed to establish which variables are used by physicians in the decision-making process, and to design educational interventions that will make physicians more accurate prognosticators.",
keywords = "Resuscitation, cardiopulmonary resuscitation, prediction, receiver operating characteristic curve",
author = "Ebell, {Mark H.} and Bergus, {George R.} and Lawrence Warbasse and Roger Bloomer",
year = "1996",
month = "1",
day = "1",
doi = "10.1007/BF02603480",
language = "English (US)",
volume = "11",
pages = "16--22",
journal = "Journal of General Internal Medicine",
issn = "0884-8734",
publisher = "Springer New York",
number = "1",

}

TY - JOUR

T1 - The inability of physicians to predict the outcome of in-hospital resuscitation

AU - Ebell, Mark H.

AU - Bergus, George R.

AU - Warbasse, Lawrence

AU - Bloomer, Roger

PY - 1996/1/1

Y1 - 1996/1/1

N2 - OBJECTIVE: To measure the accuracy, reliability, and discrimination of physicians' predictions of the outcome of in-hospital cardiopulmonary resuscitation (CPR), using a large series of detailed clinical vignettes of patients with known outcomes, DESIGN: Faculty and resident physicians at three university-affiliated generalist training programs were given one-page summaries of admission data for patients who later underwent in-hospital CPR. These summaries included all prearrest variables known to be related to the outcome of CPR. Physicians were asked to estimate the probability that patients would survive the resuscitation long enough to be stabilized, and the probability of survival to discharge. SETTING: Patient cases were derived from a consecutive series of patients undergoing CPR at two urban teaching hospitals in Detroit, Michigan. PARTICIPANTS: Faculty members and residents at a university-based department of internal medicine and two university- based departments of family medicine were surveyed. INTERVENTIONS: Accuracy of the physician predictions was assessed by comparing the mean predicted probability of survival with the percentage of patients who actually survived. The reliability of probability estimates of survival was evaluated by assessing the numerical proximity of the estimates to the actual outcome of the resuscitative effort. The ability to discriminate between survivors and nonsurvivors was measured by comparing the mean predicted probability of survival for those patients who survived CPR with that for those who did not, and by stratifying physician predictions and measuring the area under a receiver operating characteristic {ROC) curve. MEASUREMENTS AND MAIN RESULTS: Physicians (n = 51) made a total of 713 estimates, and showed poor accuracy, reliability, and discrimination in predicting the outcome of inhospital CPR. The mean predicted probability of survival to discharge did not differ between patients who actually survived to discharge and those who did not (29.5% vs 26.4%, z = 0.35, p = .73). Similarly, the mean predicted probabilities of surviving resuscitation were the same for patients who actually survived long enough to be stabilized and those who did not (37.8% vs 39.9%, z = 0.55, p = .58). Accounting for type of physician and institution by analysis of variance did not change this finding. The area under the ROC curve for the prediction of arrest survival was 0.476, which is not significantly different from 0.5, and is consistent with an ability to discriminate between survivors and nonsurvivors that is no better than random choice. CONCLUSIONS: Physicians were no better at identifying patients who would survive resuscitation than would be expected by chance alone. Further work is needed to establish which variables are used by physicians in the decision-making process, and to design educational interventions that will make physicians more accurate prognosticators.

AB - OBJECTIVE: To measure the accuracy, reliability, and discrimination of physicians' predictions of the outcome of in-hospital cardiopulmonary resuscitation (CPR), using a large series of detailed clinical vignettes of patients with known outcomes, DESIGN: Faculty and resident physicians at three university-affiliated generalist training programs were given one-page summaries of admission data for patients who later underwent in-hospital CPR. These summaries included all prearrest variables known to be related to the outcome of CPR. Physicians were asked to estimate the probability that patients would survive the resuscitation long enough to be stabilized, and the probability of survival to discharge. SETTING: Patient cases were derived from a consecutive series of patients undergoing CPR at two urban teaching hospitals in Detroit, Michigan. PARTICIPANTS: Faculty members and residents at a university-based department of internal medicine and two university- based departments of family medicine were surveyed. INTERVENTIONS: Accuracy of the physician predictions was assessed by comparing the mean predicted probability of survival with the percentage of patients who actually survived. The reliability of probability estimates of survival was evaluated by assessing the numerical proximity of the estimates to the actual outcome of the resuscitative effort. The ability to discriminate between survivors and nonsurvivors was measured by comparing the mean predicted probability of survival for those patients who survived CPR with that for those who did not, and by stratifying physician predictions and measuring the area under a receiver operating characteristic {ROC) curve. MEASUREMENTS AND MAIN RESULTS: Physicians (n = 51) made a total of 713 estimates, and showed poor accuracy, reliability, and discrimination in predicting the outcome of inhospital CPR. The mean predicted probability of survival to discharge did not differ between patients who actually survived to discharge and those who did not (29.5% vs 26.4%, z = 0.35, p = .73). Similarly, the mean predicted probabilities of surviving resuscitation were the same for patients who actually survived long enough to be stabilized and those who did not (37.8% vs 39.9%, z = 0.55, p = .58). Accounting for type of physician and institution by analysis of variance did not change this finding. The area under the ROC curve for the prediction of arrest survival was 0.476, which is not significantly different from 0.5, and is consistent with an ability to discriminate between survivors and nonsurvivors that is no better than random choice. CONCLUSIONS: Physicians were no better at identifying patients who would survive resuscitation than would be expected by chance alone. Further work is needed to establish which variables are used by physicians in the decision-making process, and to design educational interventions that will make physicians more accurate prognosticators.

KW - Resuscitation

KW - cardiopulmonary resuscitation

KW - prediction

KW - receiver operating characteristic curve

UR - http://www.scopus.com/inward/record.url?scp=0030050016&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0030050016&partnerID=8YFLogxK

U2 - 10.1007/BF02603480

DO - 10.1007/BF02603480

M3 - Article

C2 - 8691282

AN - SCOPUS:0030050016

VL - 11

SP - 16

EP - 22

JO - Journal of General Internal Medicine

JF - Journal of General Internal Medicine

SN - 0884-8734

IS - 1

ER -