title: A multicenter pilot evaluation of the National Institutes of Health chronic graft-versus-host disease (cGVHD) therapeutic response measures: feasibility, interrater reliability, and minimum detectable change.
abstract: The lack of standardized criteria for measuring therapeutic response is a major obstacle to the development of new therapeutic agents for chronic graft-versus-host disease (cGVHD). National Institutes of Health (NIH) consensus criteria for evaluating therapeutic response were published in 2006. We report the results of 4 consecutive pilot trials evaluating the feasibility and estimating the interrater reliability and minimum detectable change of these response criteria. Hematology-oncology clinicians with limited experience in applying the NIH cGVHD response criteria (n = 34) participated in a 2.5-hour training session on response evaluation in cGVHD. Feasibility and interrater reliability between subspecialty cGVHD experts and this panel of clinician raters were examined in a sample of 25 children and adults with cGVHD. The minimum detectable change was calculated using the standard error of measurement. Clinicians' impressions of the brief training session, the photo atlas, and the response criteria documentation tools were generally favorable. Performing and documenting the full set of response evaluations required a median of 21 minutes (range: 12-60 minutes) per rater. The Schirmer tear test required the greatest time of any single test (median: 9 minutes). Overall, interrater agreement for skin and oral manifestations was modest; however, in the third and fourth trials, the agreement between clinicians and experts for all dimensions except movable sclerosis approached satisfactory values. In the final 2 trials, the threshold for defining change exceeding measurement error was 19% to 22% body surface area (BSA) for erythema, 18% to 26% BSA for movable sclerosis, 17% to 21% BSA for nonmovable sclerosis, and 2.1 to 2.6 points on the 15-point NIH Oral cGHVD scale. Agreement between clinician-expert pairs was moderate to substantial for the measures of functional capacity and for the gastrointestinal and global cGVHD rating scales. These results suggest that the NIH response criteria are feasible for use, and these reliability estimates are encouraging, because they were observed following a single 2.5-hour training session given at multiple transplant centers, with no opportunity for iterative training and calibration. Research is needed to evaluate inter- and intrarater reliability in larger samples, and to evaluate these response criteria as predictors of outcomes in clinical trials.
authors.affiliation: Division of Cancer Control and Population Sciences, National Cancer Institute, National Institutes of Health, Bethesda, MD 20892, USA. mitchlls@mail.nih.gov
authors.forename: Sandra A
authors.initials: SA
authors.lastname: Mitchell
authors.forename: David
authors.initials: D
authors.lastname: Jacobsohn
authors.forename: Kimberly E
authors.initials: KE
authors.lastname: Thormann Powers
authors.forename: Paul A
authors.initials: PA
authors.lastname: Carpenter
authors.forename: Mary E D
authors.initials: ME
authors.lastname: Flowers
authors.forename: Edward W
authors.initials: EW
authors.lastname: Cowen
authors.forename: Mark
authors.initials: M
authors.lastname: Schubert
authors.forename: Maria L
authors.initials: ML
authors.lastname: Turner
authors.forename: Stephanie J
authors.initials: SJ
authors.lastname: Lee
authors.forename: Paul
authors.initials: P
authors.lastname: Martin
authors.forename: Michael R
authors.initials: MR
authors.lastname: Bishop
authors.forename: Kristin
authors.initials: K
authors.lastname: Baird
authors.forename: Javier
authors.initials: J
authors.lastname: Bolaños-Meade
authors.forename: Kevin
authors.initials: K
authors.lastname: Boyd
authors.forename: Jane M
authors.initials: JM
authors.lastname: Fall-Dickson
authors.forename: Lynn H
authors.initials: LH
authors.lastname: Gerber
authors.forename: Jean-Pierre
authors.initials: JP
authors.lastname: Guadagnini
authors.forename: Matin
authors.initials: M
authors.lastname: Imanguli
authors.forename: Michael C
authors.initials: MC
authors.lastname: Krumlauf
authors.forename: Leslie
authors.initials: L
authors.lastname: Lawley
authors.forename: Li
authors.initials: L
authors.lastname: Li
authors.forename: Bryce B
authors.initials: BB
authors.lastname: Reeve
authors.forename: Janine Austin
authors.initials: JA
authors.lastname: Clayton
authors.forename: Georgia B
authors.initials: GB
authors.lastname: Vogelsang
authors.forename: Steven Z
authors.initials: SZ
authors.lastname: Pavletic
mesh_terms: D000293:Adolescent; D000328:Adult; D000368:Aged; D002648:Child; D002675:Child, Preschool; D002908:Chronic Disease; D005260:Female; D006086:Graft vs Host Disease / Q000175:diagnosis* / Q000628:therapy*; D006405:Hematology / Q000193:education; D006801:Humans; D007938:Leukemia / Q000601:surgery; D008223:Lymphoma / Q000601:surgery; D008297:Male; D008875:Middle Aged; D009101:Multiple Myeloma / Q000601:surgery; D009316:National Institutes of Health (U.S.); D010865:Pilot Projects; D011446:Prospective Studies; D033581:Stem Cell Transplantation / Q000009:adverse effects; D014481:United States; D055815:Young Adult