Reliability of staging, prognosis, and comorbidity data collection in the National Comprehensive Cancer Network (NCCN) non-Hodgkin lymphoma (NHL) multicenter outcomes database.
BACKGROUND
Clinical trials and outcomes studies often rely on nonphysicians to abstract complex data from medical records, but the reliability of these data are rarely assessed.


METHODS
We used standardized charts of patients with non-Hodgkin lymphoma to assess the reliability of key clinical data elements abstracted by 6 clinical research associates (CRAs), 3 project staff, and 3 medical oncologists. We assessed reliability on 5 variables: MD-reported and rater-determined disease stage; International Prognostic Index (IPI; low-low intermediate, intermediate-high, high); Charlson comorbidity index score; and presence of any item from the Charlson index. Intraclass correlation coefficients (ICCs) of 0-0.20 were indicative of "slight", 0.21-0.40 indicated "fair", 0.41-0.60 indicated "moderate", 0.61-0.80 "substantial" and >0.80 "almost perfect" reliability.


RESULTS
By outcome, the ICC (95% confidence interval) values for MD-reported stage, rater-determined stage, and IPI were 0.86 (0.67, 0.94), 0.82 (0.59, 0.93), and 0.80 (0.55, 0.92), respectively. In contrast, the ICC (95% confidence interval) of the Charlson score, or presence of any Charlson comorbidity item was 0.47 (0.03, 0.75) and 0.61 (0.23, 0.83), respectively. Reliability varied by rater group; no rater group was consistently more reliable than others.


CONCLUSIONS
Trained CRAs abstracted key clinical variables with a very high degree of reliability, and performed at a level similar to study trainers and oncologists. Elements of the Charlson index were less reliable than other data types, possibly because of inherent ambiguity in the index itself.
Department of Medical Oncology, Dana-Farber Cancer Institute, Boston, Massachusetts 02115, USA.
Michelle E
ME
Kho
Eva M
EM
Lepisto
Joyce C
JC
Niland
Jonathan W
JW
Friedberg
Ann S
AS
Lacasce
Jane C
JC
Weeks
D000368:Aged; D015897:Comorbidity; D003625:Data Collection; D019992:Databases as Topic; D006801:Humans; D008228:Lymphoma, Non-Hodgkin / Q000473:pathology / Q000628:therapy*; D008499:Medical Records; D008875:Middle Aged; D009367:Neoplasm Staging; D011379:Prognosis; D015203:Reproducibility of Results; D012108:Research Personnel; D016896:Treatment Outcome