Observer agreement in the grading of oral epithelial dysplasia.
OBJECTIVE
To determine the extent of observer agreement in diagnosis of oral epithelial dysplasia (OED). Published studies of OED examiner agreement report relatively low agreement levels; however, these studies were limited by the methodologies employed.


METHODS
For this study, 64 slides were each independently examined twice by three oral pathologists. Consistency was assessed by determining intra- and interexaminer agreement. Conformity was assessed by using the modal diagnosis as a gold standard.


RESULTS
The group showed moderate interobserver agreement when grading the presence or absence of OED with a group-simple kappa (Ks) of 0.51 (95% CI = 0.42-0.61), and substantial agreement when using a 5-point ordinal scale with a group-weighted kappa (Kw) of 0.74 (95% CI = 0.64-0.85). The group showed fair to substantial intraexaminer agreement when assessing the presence or absence of OED, with Ks ranging from 0.22 to 0.78, and showing almost a perfect agreement using a 5-point ordinal scale, with Kw ranging from 0.82-0.96. Conformity with the comparison standard modal diagnosis was almost perfect, with pairwise Kw ranging from 0.81 to 0.92.


CONCLUSIONS
Overall, there was substantial intra- and interobserver consistency and almost perfect conformity in the grading of OED. Appropriate statistical methods are necessary to determine the degree of observer agreement.
Brothwell
D J
DJ
Community Dentistry, Faculty of Dentistry, University of Manitoba, Manitoba, Canada. brothwel@ms.umanitoba.ca
Lewis
D W
DW
Bradley
G
G
Leong
I
I
Jordan
R C K
RC
Mock
D
D
Leake
J L
JL
D000328:Adult; D000368:Aged; D000369:Aged, 80 and over; D002278:Carcinoma in Situ / Q000175:diagnosis / Q000473:pathology; D002294:Carcinoma, Squamous Cell / Q000175:diagnosis* / Q000473:pathology; D004848:Epithelium / Q000473:pathology; D005260:Female; D006801:Humans; D008297:Male; D008875:Middle Aged; D009061:Mouth Mucosa / Q000473:pathology; D009062:Mouth Neoplasms / Q000175:diagnosis* / Q000473:pathology; D015588:Observer Variation; D011230:Precancerous Conditions / Q000175:diagnosis* / Q000473:pathology; D015203:Reproducibility of Results