title: [Inter-rater reliability of healthcare professional skills' portfolio assessments: The Andalusian Agency for Healthcare Quality model].
abstract: This study aims to determine the reliability of assessment criteria used for a portfolio at the Andalusian Agency for Healthcare Quality (ACSA).
  
  
  METHODS
  all competences certification processes, regardless of their discipline.
  
  
  METHODS
  2010-2011. Three types of tests are used: 368 certificates, 17 895 reports and 22 642 clinical practice reports (N=3 010 candidates). The tests were evaluated in pairs by the ACSA team of raters using two categories: valid and invalid.
  
  
  RESULTS
  The percentage agreement in assessments of certificates was 89.9%; for the reports of clinical practice, 85.1%; and for clinical practice reports, 81.7%. The inter-rater agreement coefficients (kappa) ranged from 0.468 to 0.711.
  
  
  CONCLUSIONS
  The results of this study show that the inter-rater reliability of assessments varies from fair to good. Compared with other similar studies, the results put the reliability of the model in a comfortable position. Criteria were reviewed and progressive automation of evaluations was done.
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España. Address: C/ Augusto Peyré, Nº 1 Edificio Olalla, 3ª Planta. 41020, Sevilla, España. Email: manuel.herrera.usagre@juntadeandalucia.es.
authors.forename: Antonio
authors.initials: A
authors.lastname: Almuedo-Paz
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España; Departamento de Sociología, Universidad de Sevilla, España.
authors.forename: Manuel
authors.initials: M
authors.lastname: Herrera-Usagre
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España.
authors.forename: Begoña
authors.initials: B
authors.lastname: Buiza-Camacho
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España.
authors.forename: José
authors.initials: J
authors.lastname: Julián-Carrión
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España.
authors.forename: María del Pilar
authors.initials: Mdel P
authors.lastname: Carrascosa-Salmoral
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España.
authors.forename: Sheila María
authors.initials: SM
authors.lastname: Martín-García
authors.affiliation: Agencia de Calidad Sanitaria de Andalucía, España.
authors.forename: Rocío
authors.initials: R
authors.lastname: Salguero-Cabalgante
mesh_terms: D002983:Clinical Competence; D006282:Health Personnel / Q000592:standards*; D006801:Humans; D008962:Models, Theoretical; D015588:Observer Variation; D011787:Quality of Health Care; D015203:Reproducibility of Results; D013030:Spain
keywords: accreditation; certification; clinical competences; portfolio assessment; quality assurance