title: Reliability and credibility of progress test criteria developed by alumni, faculty, and mixed alumni-faculty judge panels.
abstract: OBJECTIVE
  To compare the reliability and credibility of Angoff-based, absolute criteria derived by faculty, alumni, and a combination of alumni and faculty judge panels.
  
  
  METHODS
  Independently, faculty, alumni, and mixed faculty-alumni judge panels developed pass/fail criteria for an 86-item test. Generalizability and decision studies were performed. Root mean square errors (RMSE) and 95% confidence intervals were calculated for reliability and credibility assessment. School graduate performance upon the North American Licensure Examination (NAPLEX) was the comparator for credibility assessment.
  
  
  RESULTS
  RMSEs were 1.06%, 1.42%, and 2.32% for the alumni, faculty, and mixed judge panels respectively. The school's NAPLEX pass rate was 97.5%. This rate triangulated well with the faculty judge panel (pass rate = 93.9%, CI95% = 87.1%-98.2%), but did not with either mixed judge or alumni judge panels.
  
  
  CONCLUSIONS
  Faculty-derived criteria offer superior pass/fail decision defensibility relative to both alumni derived and mixed faculty-alumni derived criteria.
authors.affiliation: Texas Tech University Health Sciences Center, School of Pharmacy, USA. glenn.anderson@marshall.edu
authors.forename: H Glenn
authors.initials: HG
authors.lastname: Anderson
authors.forename: Arthur A
authors.initials: AA
authors.lastname: Nelson
mesh_terms: D000068:Accreditation / Q000592:standards*; D000704:Analysis of Variance; D016001:Confidence Intervals; D003479:Curriculum / Q000592:standards; D004512:Education, Pharmacy / Q000592:standards*; D004521:Educational Measurement / Q000592:standards; D005178:Faculty / Q000592:standards*; D007722:Health Knowledge, Attitudes, Practice; D006801:Humans; D015233:Models, Statistical; D015203:Reproducibility of Results; D012580:Schools, Pharmacy / Q000592:standards*; D013339:Students, Pharmacy; D013781:Texas