Accuracy of grading of urothelial carcinoma on urine cytology

An analysis of interobserver and intraobserver agreement

Michelle D. Reid, Adeboye O. Osunkoya, Momin T. Siddiqui, Stephen Warwick Looney

Research output: Contribution to journalArticle

37 Citations (Scopus)

Abstract

Background: Urine samples of known urothelial carcinoma were independently graded by 3 pathologists with (MS, MR) and without (AO) fellowship training in cytopathology using a modified version of the 2004 2-tiered World Health Organization classification system. By measuring interobserver and intraobserver agreement among pathologists, compared with the gold standard of biopsy/resection, specimen accuracy and reproducibility of grading in urine was determined. Methods: 44 urine cytology samples were graded as low or high-grade by 3 pathologists with a 2-3 week interval between grading. Pathologists were blinded to their and others' grades and histologic diagnoses. Coefficient kappa was used to measure interobserver and intraobserver agreement among pathologists. Accuracy was measured by percentage agreement with the biopsy/resection separately for each pathologist, and for all pathologists and occasions combined. Results: The overall accuracy was 77% (95% C.I., 72% - 82%). Pathologist AO was significantly more accurate than MR on occasion 1 (p = 0.006) and 2 (p = 0.039). No other significant differences were found among the observers. Interobserver agreement using coefficient kappa was unacceptably low, with all but one of the kappa value being less than 0.40, the cutoff for a "fair" degree of agreement. Intraobserver agreement, as measured by coefficient kappa, was adequate. Conclusions: Our study underscores the lack of precision and subjective nature of grading urothelial carcinoma on urine samples. There was poor inter- and intraobserver agreement among pathologists despite fellowship training in cytopathology. Clinicians and cytopathologists should be mindful of this pitfall and avoid grading urothelial carcinoma on urine samples, especially since grading may impact patient management.

Original languageEnglish (US)
Pages (from-to)882-891
Number of pages10
JournalInternational Journal of Clinical and Experimental Pathology
Volume5
Issue number9
StatePublished - Dec 7 2012

Fingerprint

Cell Biology
Urine
Carcinoma
Biopsy
Pathologists

Keywords

  • Accuracy of grading
  • Urine cytology
  • Urothelial carcinoma

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Histology

Cite this

Accuracy of grading of urothelial carcinoma on urine cytology : An analysis of interobserver and intraobserver agreement. / Reid, Michelle D.; Osunkoya, Adeboye O.; Siddiqui, Momin T.; Looney, Stephen Warwick.

In: International Journal of Clinical and Experimental Pathology, Vol. 5, No. 9, 07.12.2012, p. 882-891.

Research output: Contribution to journalArticle

@article{18a7356ab5be4925bbbdc5b11e4b3166,
title = "Accuracy of grading of urothelial carcinoma on urine cytology: An analysis of interobserver and intraobserver agreement",
abstract = "Background: Urine samples of known urothelial carcinoma were independently graded by 3 pathologists with (MS, MR) and without (AO) fellowship training in cytopathology using a modified version of the 2004 2-tiered World Health Organization classification system. By measuring interobserver and intraobserver agreement among pathologists, compared with the gold standard of biopsy/resection, specimen accuracy and reproducibility of grading in urine was determined. Methods: 44 urine cytology samples were graded as low or high-grade by 3 pathologists with a 2-3 week interval between grading. Pathologists were blinded to their and others' grades and histologic diagnoses. Coefficient kappa was used to measure interobserver and intraobserver agreement among pathologists. Accuracy was measured by percentage agreement with the biopsy/resection separately for each pathologist, and for all pathologists and occasions combined. Results: The overall accuracy was 77{\%} (95{\%} C.I., 72{\%} - 82{\%}). Pathologist AO was significantly more accurate than MR on occasion 1 (p = 0.006) and 2 (p = 0.039). No other significant differences were found among the observers. Interobserver agreement using coefficient kappa was unacceptably low, with all but one of the kappa value being less than 0.40, the cutoff for a {"}fair{"} degree of agreement. Intraobserver agreement, as measured by coefficient kappa, was adequate. Conclusions: Our study underscores the lack of precision and subjective nature of grading urothelial carcinoma on urine samples. There was poor inter- and intraobserver agreement among pathologists despite fellowship training in cytopathology. Clinicians and cytopathologists should be mindful of this pitfall and avoid grading urothelial carcinoma on urine samples, especially since grading may impact patient management.",
keywords = "Accuracy of grading, Urine cytology, Urothelial carcinoma",
author = "Reid, {Michelle D.} and Osunkoya, {Adeboye O.} and Siddiqui, {Momin T.} and Looney, {Stephen Warwick}",
year = "2012",
month = "12",
day = "7",
language = "English (US)",
volume = "5",
pages = "882--891",
journal = "International Journal of Clinical and Experimental Pathology",
issn = "1936-2625",
publisher = "e-Century Publishing Corporation",
number = "9",

}

TY - JOUR

T1 - Accuracy of grading of urothelial carcinoma on urine cytology

T2 - An analysis of interobserver and intraobserver agreement

AU - Reid, Michelle D.

AU - Osunkoya, Adeboye O.

AU - Siddiqui, Momin T.

AU - Looney, Stephen Warwick

PY - 2012/12/7

Y1 - 2012/12/7

N2 - Background: Urine samples of known urothelial carcinoma were independently graded by 3 pathologists with (MS, MR) and without (AO) fellowship training in cytopathology using a modified version of the 2004 2-tiered World Health Organization classification system. By measuring interobserver and intraobserver agreement among pathologists, compared with the gold standard of biopsy/resection, specimen accuracy and reproducibility of grading in urine was determined. Methods: 44 urine cytology samples were graded as low or high-grade by 3 pathologists with a 2-3 week interval between grading. Pathologists were blinded to their and others' grades and histologic diagnoses. Coefficient kappa was used to measure interobserver and intraobserver agreement among pathologists. Accuracy was measured by percentage agreement with the biopsy/resection separately for each pathologist, and for all pathologists and occasions combined. Results: The overall accuracy was 77% (95% C.I., 72% - 82%). Pathologist AO was significantly more accurate than MR on occasion 1 (p = 0.006) and 2 (p = 0.039). No other significant differences were found among the observers. Interobserver agreement using coefficient kappa was unacceptably low, with all but one of the kappa value being less than 0.40, the cutoff for a "fair" degree of agreement. Intraobserver agreement, as measured by coefficient kappa, was adequate. Conclusions: Our study underscores the lack of precision and subjective nature of grading urothelial carcinoma on urine samples. There was poor inter- and intraobserver agreement among pathologists despite fellowship training in cytopathology. Clinicians and cytopathologists should be mindful of this pitfall and avoid grading urothelial carcinoma on urine samples, especially since grading may impact patient management.

AB - Background: Urine samples of known urothelial carcinoma were independently graded by 3 pathologists with (MS, MR) and without (AO) fellowship training in cytopathology using a modified version of the 2004 2-tiered World Health Organization classification system. By measuring interobserver and intraobserver agreement among pathologists, compared with the gold standard of biopsy/resection, specimen accuracy and reproducibility of grading in urine was determined. Methods: 44 urine cytology samples were graded as low or high-grade by 3 pathologists with a 2-3 week interval between grading. Pathologists were blinded to their and others' grades and histologic diagnoses. Coefficient kappa was used to measure interobserver and intraobserver agreement among pathologists. Accuracy was measured by percentage agreement with the biopsy/resection separately for each pathologist, and for all pathologists and occasions combined. Results: The overall accuracy was 77% (95% C.I., 72% - 82%). Pathologist AO was significantly more accurate than MR on occasion 1 (p = 0.006) and 2 (p = 0.039). No other significant differences were found among the observers. Interobserver agreement using coefficient kappa was unacceptably low, with all but one of the kappa value being less than 0.40, the cutoff for a "fair" degree of agreement. Intraobserver agreement, as measured by coefficient kappa, was adequate. Conclusions: Our study underscores the lack of precision and subjective nature of grading urothelial carcinoma on urine samples. There was poor inter- and intraobserver agreement among pathologists despite fellowship training in cytopathology. Clinicians and cytopathologists should be mindful of this pitfall and avoid grading urothelial carcinoma on urine samples, especially since grading may impact patient management.

KW - Accuracy of grading

KW - Urine cytology

KW - Urothelial carcinoma

UR - http://www.scopus.com/inward/record.url?scp=84870474018&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84870474018&partnerID=8YFLogxK

M3 - Article

VL - 5

SP - 882

EP - 891

JO - International Journal of Clinical and Experimental Pathology

JF - International Journal of Clinical and Experimental Pathology

SN - 1936-2625

IS - 9

ER -