title: Methodological challenges of validating a clinical decision-making tool in the practice environment.
abstract: Validating a measurement tool intended for use in the practice environment poses challenges that may not be present when validating a tool intended solely for research purposes. The aim of this article is to describe the methodological challenges of validating a clinical decision-making tool, the Oncology Acuity Tool, which nurses use to make nurse assignment and staffing decisions prospectively each shift. Data were derived from a larger validation study, during which several methodological challenges arose. Revisions to the tool, including conducting iterative feedback cycles with end users, were necessary before the validation study was initiated. The "true" value of patient acuity is unknown, and thus, two approaches to inter-rater reliability assessment were used. Discordant perspectives existed between experts and end users. Balancing psychometric rigor with clinical relevance may be achieved through establishing research-practice partnerships, seeking active and continuous feedback with end users, and weighing traditional statistical rules of thumb with practical considerations.
authors.affiliation: National Institutes of Health Clinical Center, Bethesda, MD, USA caitlin.brennan@nih.gov.
authors.forename: Caitlin W
authors.initials: CW
authors.lastname: Brennan
authors.affiliation: Case Western Reserve University, Cleveland, OH, USA.
authors.forename: Barbara J
authors.initials: BJ
authors.lastname: Daly
mesh_terms: D000066491:Clinical Decision-Making / Q000379:methods*; D003661:Decision Support Techniques; D006801:Humans; D016522:Oncology Service, Hospital; D010561:Personnel Staffing and Scheduling / Q000592:standards; D011594:Psychometrics; D015203:Reproducibility of Results
keywords: acuity; acute care; instrument development; nurse staffing; oncology