Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria

F. Gress, C. Schmitt, M. Catalane, John Paul Affronti, K. Binmoeller, P. Stevens, T. Savides, M. Bhutani, D. Ciaccia, N. Nicki, D. Faigel, J. Birk, L. Roubein, C. Lightdale

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

Endoscopic Ultrasound (EUS) has been reported to be an accurate modality for T staging of pancreatic cancer (CA). However, lymph node staging has been less accurate. Recent data suggests that current EUS criteria used to determine benign or malignant status of a lymph node might be inadequate. Aim of Study: To determine the effects of interobserver variation on the overall accuracy of lymph node staging in pancreatic CA. Methods: Twelve patients with previously diagnosed pancreatic ductal adenocarcinoma underwent staging with EUS. Surgical correlation was obtained in all patients. A blinded observer developed a study videotape of real-time EUS for each case. The videotape was then distributed to 13 endosonographers with at least one year of previous experience who independently reviewed the tape and recorded their interpretation of the EUS nodal (N) stage for each case based upon previously reported EUS criteria used for staging lymph nodes. A multiple observer kappa (k) statistic was calculated for each staging category (N 1 , and N 2 ) and overall k. Kappa accounts for agreement due to chance alone and standardized ranges are available. The Light (G) statistic was utilized to allow for the presence of the gold standard. Accuracy was assessed using a simple matrix format. Results: The 13 endosonographers as a group, had performed a total of 13,824 EUS cases (median = 650 cases, mean-1063, SD = 247, range:245 to 3000) and a total of 2,410 pancreatic cases (range:10 to 1000). As a group, they had a median of 5 years experience (range:2 to10). The overall group agreement (k) for N staging was low although it was statistically significant: k = 0.23 (G = 4.44, Z = 4.90, p<0.01). Furthermore, the agreement for absence of nodal involvement (63%) was better than that for the presence of nodal involvement (56%). Correlation by experience and total EUS cases for each endosonographer did not improve agreement for N staging. Although, the agreement for T and N stage improved as stage increased (p<0.01) and in the presence of vascular invasion (p<0.01). Conclusion: Interobserver agreement is poor for EUS lymph node staging in pancreatic cancer. Experience and total number of cases performed were not factors in this observation. Therefore, it appears that other factors, including the EUS criteria we currently use to stage lymph nodes may be inadequate and warrant further evaluation.

Original languageEnglish (US)
JournalGastrointestinal Endoscopy
Volume47
Issue number4
StatePublished - Dec 1 1998

Fingerprint

Pancreatic Neoplasms
Lymph Nodes
Videotape Recording
Observer Variation
Blood Vessels
Adenocarcinoma
Light

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Gastroenterology

Cite this

Gress, F., Schmitt, C., Catalane, M., Affronti, J. P., Binmoeller, K., Stevens, P., ... Lightdale, C. (1998). Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria. Gastrointestinal Endoscopy, 47(4).

Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria. / Gress, F.; Schmitt, C.; Catalane, M.; Affronti, John Paul; Binmoeller, K.; Stevens, P.; Savides, T.; Bhutani, M.; Ciaccia, D.; Nicki, N.; Faigel, D.; Birk, J.; Roubein, L.; Lightdale, C.

In: Gastrointestinal Endoscopy, Vol. 47, No. 4, 01.12.1998.

Research output: Contribution to journalArticle

Gress, F, Schmitt, C, Catalane, M, Affronti, JP, Binmoeller, K, Stevens, P, Savides, T, Bhutani, M, Ciaccia, D, Nicki, N, Faigel, D, Birk, J, Roubein, L & Lightdale, C 1998, 'Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria', Gastrointestinal Endoscopy, vol. 47, no. 4.
Gress, F. ; Schmitt, C. ; Catalane, M. ; Affronti, John Paul ; Binmoeller, K. ; Stevens, P. ; Savides, T. ; Bhutani, M. ; Ciaccia, D. ; Nicki, N. ; Faigel, D. ; Birk, J. ; Roubein, L. ; Lightdale, C. / Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria. In: Gastrointestinal Endoscopy. 1998 ; Vol. 47, No. 4.
@article{b1f5cd3e70df49808f37569379dee0a6,
title = "Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria",
abstract = "Endoscopic Ultrasound (EUS) has been reported to be an accurate modality for T staging of pancreatic cancer (CA). However, lymph node staging has been less accurate. Recent data suggests that current EUS criteria used to determine benign or malignant status of a lymph node might be inadequate. Aim of Study: To determine the effects of interobserver variation on the overall accuracy of lymph node staging in pancreatic CA. Methods: Twelve patients with previously diagnosed pancreatic ductal adenocarcinoma underwent staging with EUS. Surgical correlation was obtained in all patients. A blinded observer developed a study videotape of real-time EUS for each case. The videotape was then distributed to 13 endosonographers with at least one year of previous experience who independently reviewed the tape and recorded their interpretation of the EUS nodal (N) stage for each case based upon previously reported EUS criteria used for staging lymph nodes. A multiple observer kappa (k) statistic was calculated for each staging category (N 1 , and N 2 ) and overall k. Kappa accounts for agreement due to chance alone and standardized ranges are available. The Light (G) statistic was utilized to allow for the presence of the gold standard. Accuracy was assessed using a simple matrix format. Results: The 13 endosonographers as a group, had performed a total of 13,824 EUS cases (median = 650 cases, mean-1063, SD = 247, range:245 to 3000) and a total of 2,410 pancreatic cases (range:10 to 1000). As a group, they had a median of 5 years experience (range:2 to10). The overall group agreement (k) for N staging was low although it was statistically significant: k = 0.23 (G = 4.44, Z = 4.90, p<0.01). Furthermore, the agreement for absence of nodal involvement (63{\%}) was better than that for the presence of nodal involvement (56{\%}). Correlation by experience and total EUS cases for each endosonographer did not improve agreement for N staging. Although, the agreement for T and N stage improved as stage increased (p<0.01) and in the presence of vascular invasion (p<0.01). Conclusion: Interobserver agreement is poor for EUS lymph node staging in pancreatic cancer. Experience and total number of cases performed were not factors in this observation. Therefore, it appears that other factors, including the EUS criteria we currently use to stage lymph nodes may be inadequate and warrant further evaluation.",
author = "F. Gress and C. Schmitt and M. Catalane and Affronti, {John Paul} and K. Binmoeller and P. Stevens and T. Savides and M. Bhutani and D. Ciaccia and N. Nicki and D. Faigel and J. Birk and L. Roubein and C. Lightdale",
year = "1998",
month = "12",
day = "1",
language = "English (US)",
volume = "47",
journal = "Gastrointestinal Endoscopy",
issn = "0016-5107",
publisher = "Mosby Inc.",
number = "4",

}

TY - JOUR

T1 - Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria

AU - Gress, F.

AU - Schmitt, C.

AU - Catalane, M.

AU - Affronti, John Paul

AU - Binmoeller, K.

AU - Stevens, P.

AU - Savides, T.

AU - Bhutani, M.

AU - Ciaccia, D.

AU - Nicki, N.

AU - Faigel, D.

AU - Birk, J.

AU - Roubein, L.

AU - Lightdale, C.

PY - 1998/12/1

Y1 - 1998/12/1

N2 - Endoscopic Ultrasound (EUS) has been reported to be an accurate modality for T staging of pancreatic cancer (CA). However, lymph node staging has been less accurate. Recent data suggests that current EUS criteria used to determine benign or malignant status of a lymph node might be inadequate. Aim of Study: To determine the effects of interobserver variation on the overall accuracy of lymph node staging in pancreatic CA. Methods: Twelve patients with previously diagnosed pancreatic ductal adenocarcinoma underwent staging with EUS. Surgical correlation was obtained in all patients. A blinded observer developed a study videotape of real-time EUS for each case. The videotape was then distributed to 13 endosonographers with at least one year of previous experience who independently reviewed the tape and recorded their interpretation of the EUS nodal (N) stage for each case based upon previously reported EUS criteria used for staging lymph nodes. A multiple observer kappa (k) statistic was calculated for each staging category (N 1 , and N 2 ) and overall k. Kappa accounts for agreement due to chance alone and standardized ranges are available. The Light (G) statistic was utilized to allow for the presence of the gold standard. Accuracy was assessed using a simple matrix format. Results: The 13 endosonographers as a group, had performed a total of 13,824 EUS cases (median = 650 cases, mean-1063, SD = 247, range:245 to 3000) and a total of 2,410 pancreatic cases (range:10 to 1000). As a group, they had a median of 5 years experience (range:2 to10). The overall group agreement (k) for N staging was low although it was statistically significant: k = 0.23 (G = 4.44, Z = 4.90, p<0.01). Furthermore, the agreement for absence of nodal involvement (63%) was better than that for the presence of nodal involvement (56%). Correlation by experience and total EUS cases for each endosonographer did not improve agreement for N staging. Although, the agreement for T and N stage improved as stage increased (p<0.01) and in the presence of vascular invasion (p<0.01). Conclusion: Interobserver agreement is poor for EUS lymph node staging in pancreatic cancer. Experience and total number of cases performed were not factors in this observation. Therefore, it appears that other factors, including the EUS criteria we currently use to stage lymph nodes may be inadequate and warrant further evaluation.

AB - Endoscopic Ultrasound (EUS) has been reported to be an accurate modality for T staging of pancreatic cancer (CA). However, lymph node staging has been less accurate. Recent data suggests that current EUS criteria used to determine benign or malignant status of a lymph node might be inadequate. Aim of Study: To determine the effects of interobserver variation on the overall accuracy of lymph node staging in pancreatic CA. Methods: Twelve patients with previously diagnosed pancreatic ductal adenocarcinoma underwent staging with EUS. Surgical correlation was obtained in all patients. A blinded observer developed a study videotape of real-time EUS for each case. The videotape was then distributed to 13 endosonographers with at least one year of previous experience who independently reviewed the tape and recorded their interpretation of the EUS nodal (N) stage for each case based upon previously reported EUS criteria used for staging lymph nodes. A multiple observer kappa (k) statistic was calculated for each staging category (N 1 , and N 2 ) and overall k. Kappa accounts for agreement due to chance alone and standardized ranges are available. The Light (G) statistic was utilized to allow for the presence of the gold standard. Accuracy was assessed using a simple matrix format. Results: The 13 endosonographers as a group, had performed a total of 13,824 EUS cases (median = 650 cases, mean-1063, SD = 247, range:245 to 3000) and a total of 2,410 pancreatic cases (range:10 to 1000). As a group, they had a median of 5 years experience (range:2 to10). The overall group agreement (k) for N staging was low although it was statistically significant: k = 0.23 (G = 4.44, Z = 4.90, p<0.01). Furthermore, the agreement for absence of nodal involvement (63%) was better than that for the presence of nodal involvement (56%). Correlation by experience and total EUS cases for each endosonographer did not improve agreement for N staging. Although, the agreement for T and N stage improved as stage increased (p<0.01) and in the presence of vascular invasion (p<0.01). Conclusion: Interobserver agreement is poor for EUS lymph node staging in pancreatic cancer. Experience and total number of cases performed were not factors in this observation. Therefore, it appears that other factors, including the EUS criteria we currently use to stage lymph nodes may be inadequate and warrant further evaluation.

UR - http://www.scopus.com/inward/record.url?scp=33748980471&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33748980471&partnerID=8YFLogxK

M3 - Article

VL - 47

JO - Gastrointestinal Endoscopy

JF - Gastrointestinal Endoscopy

SN - 0016-5107

IS - 4

ER -