Interobserver and intraobserver variability in the calculation of the lipid-laden macrophage index: Implications for its use in the evaluation of aspiration in children

Michelle Reid-Nicholson, Renuka Kulkarni, Bamidele Adeagbo, Stephen Warwick Looney, John H Crosby

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

The lipid-laden macrophage index (LLMI) is a semiquantitative test used to evaluate aspiration in children. We assessed the reliability and reproducibility of LLMI by calculating interobserver and intraobserver variability among pathologists, with and without expertise in cytopathology. Forty-nine bronchoalveolar washes/lavages were blindly reviewed by four reviewers and assigned an LLMI. Three pathologists (two cytopathologists, one pathology fellow) reviewed slides twice and one cytotechnologist reviewed them once. Intraclass correlation coefficient (ICC) with 95% confidence interval (C.I.) was used to measure overall intraobserver and interobserver agreement. Interobserver agreement was also calculated separately for each pair of reviewers. ICC values did not indicate an acceptable level of interobserver agreement among pathologists, with (ICC = 0.67, 95% C.I.: 0.56-0.77) and without (ICC = 0.77, 95% C.I.: 0.61-0.84) the cytotechnologist included in the analysis. An ICC of 0.84 (95% C.I.: 0.78-0.89) indicated an acceptable level of intraobserver agreement among pathologists. When calculated separately for each pair of reviewers, all but two ICC values for interobserver agreement were less than 0.75 (the minimally acceptable value for a reliable clinical measurement), and the lower confidence limit of each of the 95% C.I. was far below the 0.75 cutoff. Using Lin's coefficient, intraobserver variability was only acceptable for two pathologists. Our study highlights the lack of precision and subjectivity of the LLMI, as well as the significant inter and intraobserver bias that may occur among experienced and inexperienced pathologists, and cytotechnologists. Clinicians and cytopathologists alike should be mindful of this potential pitfall and interpret LLMI scores with caution.

Original languageEnglish (US)
Pages (from-to)861-865
Number of pages5
JournalDiagnostic Cytopathology
Volume38
Issue number12
DOIs
StatePublished - Dec 1 2010

Fingerprint

Observer Variation
Macrophages
Lipids
Confidence Intervals
Bronchoalveolar Lavage
Pathologists
Pathology

Keywords

  • interobserver
  • intraobserver
  • lipid-laden macrophage index

ASJC Scopus subject areas

  • Pathology and Forensic Medicine
  • Histology

Cite this

Interobserver and intraobserver variability in the calculation of the lipid-laden macrophage index : Implications for its use in the evaluation of aspiration in children. / Reid-Nicholson, Michelle; Kulkarni, Renuka; Adeagbo, Bamidele; Looney, Stephen Warwick; Crosby, John H.

In: Diagnostic Cytopathology, Vol. 38, No. 12, 01.12.2010, p. 861-865.

Research output: Contribution to journalArticle

@article{eec2d068f491486c8f03d8b82b7b885e,
title = "Interobserver and intraobserver variability in the calculation of the lipid-laden macrophage index: Implications for its use in the evaluation of aspiration in children",
abstract = "The lipid-laden macrophage index (LLMI) is a semiquantitative test used to evaluate aspiration in children. We assessed the reliability and reproducibility of LLMI by calculating interobserver and intraobserver variability among pathologists, with and without expertise in cytopathology. Forty-nine bronchoalveolar washes/lavages were blindly reviewed by four reviewers and assigned an LLMI. Three pathologists (two cytopathologists, one pathology fellow) reviewed slides twice and one cytotechnologist reviewed them once. Intraclass correlation coefficient (ICC) with 95{\%} confidence interval (C.I.) was used to measure overall intraobserver and interobserver agreement. Interobserver agreement was also calculated separately for each pair of reviewers. ICC values did not indicate an acceptable level of interobserver agreement among pathologists, with (ICC = 0.67, 95{\%} C.I.: 0.56-0.77) and without (ICC = 0.77, 95{\%} C.I.: 0.61-0.84) the cytotechnologist included in the analysis. An ICC of 0.84 (95{\%} C.I.: 0.78-0.89) indicated an acceptable level of intraobserver agreement among pathologists. When calculated separately for each pair of reviewers, all but two ICC values for interobserver agreement were less than 0.75 (the minimally acceptable value for a reliable clinical measurement), and the lower confidence limit of each of the 95{\%} C.I. was far below the 0.75 cutoff. Using Lin's coefficient, intraobserver variability was only acceptable for two pathologists. Our study highlights the lack of precision and subjectivity of the LLMI, as well as the significant inter and intraobserver bias that may occur among experienced and inexperienced pathologists, and cytotechnologists. Clinicians and cytopathologists alike should be mindful of this potential pitfall and interpret LLMI scores with caution.",
keywords = "interobserver, intraobserver, lipid-laden macrophage index",
author = "Michelle Reid-Nicholson and Renuka Kulkarni and Bamidele Adeagbo and Looney, {Stephen Warwick} and Crosby, {John H}",
year = "2010",
month = "12",
day = "1",
doi = "10.1002/dc.21298",
language = "English (US)",
volume = "38",
pages = "861--865",
journal = "Diagnostic Cytopathology",
issn = "8755-1039",
publisher = "Wiley-Liss Inc.",
number = "12",

}

TY - JOUR

T1 - Interobserver and intraobserver variability in the calculation of the lipid-laden macrophage index

T2 - Implications for its use in the evaluation of aspiration in children

AU - Reid-Nicholson, Michelle

AU - Kulkarni, Renuka

AU - Adeagbo, Bamidele

AU - Looney, Stephen Warwick

AU - Crosby, John H

PY - 2010/12/1

Y1 - 2010/12/1

N2 - The lipid-laden macrophage index (LLMI) is a semiquantitative test used to evaluate aspiration in children. We assessed the reliability and reproducibility of LLMI by calculating interobserver and intraobserver variability among pathologists, with and without expertise in cytopathology. Forty-nine bronchoalveolar washes/lavages were blindly reviewed by four reviewers and assigned an LLMI. Three pathologists (two cytopathologists, one pathology fellow) reviewed slides twice and one cytotechnologist reviewed them once. Intraclass correlation coefficient (ICC) with 95% confidence interval (C.I.) was used to measure overall intraobserver and interobserver agreement. Interobserver agreement was also calculated separately for each pair of reviewers. ICC values did not indicate an acceptable level of interobserver agreement among pathologists, with (ICC = 0.67, 95% C.I.: 0.56-0.77) and without (ICC = 0.77, 95% C.I.: 0.61-0.84) the cytotechnologist included in the analysis. An ICC of 0.84 (95% C.I.: 0.78-0.89) indicated an acceptable level of intraobserver agreement among pathologists. When calculated separately for each pair of reviewers, all but two ICC values for interobserver agreement were less than 0.75 (the minimally acceptable value for a reliable clinical measurement), and the lower confidence limit of each of the 95% C.I. was far below the 0.75 cutoff. Using Lin's coefficient, intraobserver variability was only acceptable for two pathologists. Our study highlights the lack of precision and subjectivity of the LLMI, as well as the significant inter and intraobserver bias that may occur among experienced and inexperienced pathologists, and cytotechnologists. Clinicians and cytopathologists alike should be mindful of this potential pitfall and interpret LLMI scores with caution.

AB - The lipid-laden macrophage index (LLMI) is a semiquantitative test used to evaluate aspiration in children. We assessed the reliability and reproducibility of LLMI by calculating interobserver and intraobserver variability among pathologists, with and without expertise in cytopathology. Forty-nine bronchoalveolar washes/lavages were blindly reviewed by four reviewers and assigned an LLMI. Three pathologists (two cytopathologists, one pathology fellow) reviewed slides twice and one cytotechnologist reviewed them once. Intraclass correlation coefficient (ICC) with 95% confidence interval (C.I.) was used to measure overall intraobserver and interobserver agreement. Interobserver agreement was also calculated separately for each pair of reviewers. ICC values did not indicate an acceptable level of interobserver agreement among pathologists, with (ICC = 0.67, 95% C.I.: 0.56-0.77) and without (ICC = 0.77, 95% C.I.: 0.61-0.84) the cytotechnologist included in the analysis. An ICC of 0.84 (95% C.I.: 0.78-0.89) indicated an acceptable level of intraobserver agreement among pathologists. When calculated separately for each pair of reviewers, all but two ICC values for interobserver agreement were less than 0.75 (the minimally acceptable value for a reliable clinical measurement), and the lower confidence limit of each of the 95% C.I. was far below the 0.75 cutoff. Using Lin's coefficient, intraobserver variability was only acceptable for two pathologists. Our study highlights the lack of precision and subjectivity of the LLMI, as well as the significant inter and intraobserver bias that may occur among experienced and inexperienced pathologists, and cytotechnologists. Clinicians and cytopathologists alike should be mindful of this potential pitfall and interpret LLMI scores with caution.

KW - interobserver

KW - intraobserver

KW - lipid-laden macrophage index

UR - http://www.scopus.com/inward/record.url?scp=78649683947&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=78649683947&partnerID=8YFLogxK

U2 - 10.1002/dc.21298

DO - 10.1002/dc.21298

M3 - Article

C2 - 20049966

AN - SCOPUS:78649683947

VL - 38

SP - 861

EP - 865

JO - Diagnostic Cytopathology

JF - Diagnostic Cytopathology

SN - 8755-1039

IS - 12

ER -