title: Quality of nursing intensity data: inter-rater reliability of the patient classification after two decades in clinical use.
abstract: OBJECTIVE
  The aim of this study was to measure the inter-rater reliability of the Oulu Patient Classification and to discuss existing methods of reliability testing.
  
  
  BACKGROUND
  The Oulu Patient Classification, part of the RAFAELA®System, has been developed to assist nursing managers with the proper allocation of nursing resources. Due to the increased intensity of inpatient care during recent years, there is a need for the reliability testing of the classification, which has been in clinical use for 20 years.
  
  
  METHODS
  Retrospective statistical study.
  
  
  METHODS
  To test inter-rater reliability, a pair of nurses classified the same patients, without knowledge of each other's ratings, as a part of annually conducted standardization. Data on the parallel classifications (n = 19,997) was obtained from inpatient units (n = 32) with different specialties at a university hospital in Finland during 2010-2015. Parallel classification practices were also analysed. The reliability of the overall classification and its subareas were calculated using suitable statistical coefficients.
  
  
  RESULTS
  Inter-rater reliability coefficients were a reliable or almost perfect means of considering the nursing intensity category and various practices, but there were detectable differences between subareas. The lowest agreement levels occurred in the subareas 'Planning and Coordination of Nursing Care' and 'Guiding of Care/Continued Care and Emotional Support'.
  
  
  CONCLUSIONS
  There is a need to develop the descriptions of subareas and to clarify the related concepts. Precise nursing documentation can promote a high level of agreement and reliable results. The traditional overall proportion of agreement does not provide an adequate picture of reliability - weighted kappa coefficients should be used instead.
authors.affiliation: Division of Operative Care, Oulu University Hospital, Finland.
authors.forename: Pia
authors.identifier: http://orcid.org/0000-0002-5271-1483
authors.initials: P
authors.lastname: Liljamo
authors.affiliation: Department of Health and Social Management, University of Eastern Finland, Kuopio, Finland.
authors.forename: Ulla-Mari
authors.initials: UM
authors.lastname: Kinnunen
authors.affiliation: Division of Operative Care, Medical Research Center Oulu, Oulu University Hospital and University of Oulu, Finland.
authors.forename: Pasi
authors.initials: P
authors.lastname: Ohtonen
authors.affiliation: Department of Health and Social Management, University of Eastern Finland, Kuopio, Finland.
authors.forename: Kaija
authors.initials: K
authors.lastname: Saranto
mesh_terms: D000328:Adult; D005260:Female; D005387:Finland; D006801:Humans; D007297:Inpatients / Q000145:classification*; D008297:Male; D008875:Middle Aged; D009732:Nursing Care / Q000592:standards*; D009740:Nursing Staff / Q000523:psychology*; D015203:Reproducibility of Results; D012189:Retrospective Studies; D009626:Terminology as Topic
keywords: agreement; inter-rater reliability; nursing; nursing intensity; patient classification; reliability