title: Is there a valid and reliable assessment of diagnostic knee arthroscopy skill?
abstract: BACKGROUND
  The Basic Arthroscopic Knee Skill Scoring System (BAKSSS) has construct validity as an objective measure of arthroscopic proficiency when used to assess the task of performing arthroscopic meniscectomies on cadaver knees. The reliability of this instrument is unknown.
  
  
  OBJECTIVE
  We asked whether (1) a simple modification of the BAKSSS would show construct validity similar to that in the initial BAKSSS study, (2) this assessment would be reliable, and (3) this assessment could be used as a high-stakes pass or fail test.
  
  
  METHODS
  Twenty-three orthopaedic residents performed diagnostic knee arthroscopies on cadaveric knee specimens. Their competency was assessed by three live raters using the modified BAKSSS. Interrater reliability was assessed by comparing the scores given by each rater to each subject.
  
  
  RESULTS
  The modified BAKSSS showed construct validity with junior residents achieving lower scores (mean score, 20) than senior residents (mean score, 33). The modified BAKSSS had an interrater reliability of kappa = 0.685-0.852. The modified BAKSSS had a kappa = 0.543 when used as a proficiency test for diagnostic arthroscopy.
  
  
  CONCLUSIONS
  The modified BAKSSS is useful for assessing diagnostic knee arthroscopy proficiency. Future scoring systems should be designed to be generalizable so they can be applied to multiple procedures without the need for modification, allow for video-based assessment, and must be rigorously tested for reliability and other types of validity (eg, face validity, content validity, and criterion-related validity).
  
  
  CONCLUSIONS
  Having a valid and reliable assessment of basic arthroscopic procedures may allow educators to more adequately evaluate individual residents and the effectiveness of various training modalities.
authors.affiliation: Department of Orthopaedics and Rehabilitation, University of Rochester Medical Center, 601 Elmwood Avenue, Box 665, Rochester, NY, USA.
authors.forename: Tyson
authors.initials: T
authors.lastname: Olson
authors.forename: Ryan
authors.initials: R
authors.lastname: Koehler
authors.forename: Aaron
authors.initials: A
authors.lastname: Butler
authors.forename: Simon
authors.initials: S
authors.lastname: Amsdell
authors.forename: Gregg
authors.initials: G
authors.lastname: Nicandri
mesh_terms: D001182:Arthroscopy / Q000193:education / Q000592:standards; D002102:Cadaver; D002983:Clinical Competence / Q000592:standards; D004503:Education, Medical, Graduate; D005260:Female; D006801:Humans; D007396:Internship and Residency; D007592:Joint Diseases / Q000473:pathology*; D007719:Knee Joint / Q000473:pathology*; D008297:Male; D015588:Observer Variation; D011237:Predictive Value of Tests; D015203:Reproducibility of Results; D013647:Task Performance and Analysis