Evaluation of interobserver agreement for assessing lymph node staging in pancreatic cancer using current Endoscopic Ultrasound (EUS) criteria

Endoscopic Ultrasound (EUS) has been reported to be an accurate modality for T staging of pancreatic cancer (CA). However, lymph node staging has been less accurate. Recent data suggests that current EUS criteria used to determine benign or malignant status of a lymph node might be inadequate. Aim of Study: To determine the effects of interobserver variation on the overall accuracy of lymph node staging in pancreatic CA. Methods: Twelve patients with previously diagnosed pancreatic ductal adenocarcinoma underwent staging with EUS. Surgical correlation was obtained in all patients. A blinded observer developed a study videotape of real-time EUS for each case. The videotape was then distributed to 13 endosonographers with at least one year of previous experience who independently reviewed the tape and recorded their interpretation of the EUS nodal (N) stage for each case based upon previously reported EUS criteria used for staging lymph nodes. A multiple observer kappa (k) statistic was calculated for each staging category (N₁, and N₂) and overall k. Kappa accounts for agreement due to chance alone and standardized ranges are available. The Light (G) statistic was utilized to allow for the presence of the gold standard. Accuracy was assessed using a simple matrix format. Results: The 13 endosonographers as a group, had performed a total of 13,824 EUS cases (median = 650 cases, mean-1063, SD = 247, range:245 to 3000) and a total of 2,410 pancreatic cases (range:10 to 1000). As a group, they had a median of 5 years experience (range:2 to10). The overall group agreement (k) for N staging was low although it was statistically significant: k = 0.23 (G = 4.44, Z = 4.90, p<0.01). Furthermore, the agreement for absence of nodal involvement (63%) was better than that for the presence of nodal involvement (56%). Correlation by experience and total EUS cases for each endosonographer did not improve agreement for N staging. Although, the agreement for T and N stage improved as stage increased (p<0.01) and in the presence of vascular invasion (p<0.01). Conclusion: Interobserver agreement is poor for EUS lymph node staging in pancreatic cancer. Experience and total number of cases performed were not factors in this observation. Therefore, it appears that other factors, including the EUS criteria we currently use to stage lymph nodes may be inadequate and warrant further evaluation.