title: A general critical appraisal tool: an evaluation of construct validity.
abstract: BACKGROUND
  Many critical appraisal tools (CATs) exist for which there is little or no information on development of the CAT, evaluation of validity, or testing reliability. The proposed CAT was developed based on a number of other CATs, general research methods theory, and reporting guidelines but requires further study to determine its effectiveness.
  
  
  OBJECTIVE
  To establish a scoring system and to evaluate the construct validity of the proposed critical appraisal tool before undertaking reliability testing.
  
  
  METHODS
  Data obtained from this exploratory study along with information on the design of the proposed CAT were combined to evaluate construct validity using the Standards for educational and psychological testing which consist of five types of evidence: test content, response process, internal structure, relations to other variables, and consequences of testing. To obtain data for internal structure and relations to other variables, the proposed CAT was analysed against five alternative CATs. A random sample of 10 papers from six different research designs across the range of health related research were selected, giving a total sample size of 60 papers.
  
  
  RESULTS
  In all research designs, the proposed CAT had significant (p<0.05, two-tailed) weak to moderate positive correlations (Kendall's τ 0.33-0.55) with the alternative CATs, except in the Preamble category. There were significant moderate to strong positive correlations in the quasi-experimental (τ 0.70-1.00), descriptive/exploratory/observational (τ 0.72-1.00), qualitative (τ 0.74-0.81), and systematic review (τ 0.62-0.82) designs and to a lesser extent in the true experimental (τ 0.68-0.70) design. There were no significant correlations in the single system research designs.
  
  
  CONCLUSIONS
  Based on the results obtained, the theory on which the proposed CAT was designed, and the objective of the proposed CAT there was enough evidence to show that inferences made from scores obtained from the proposed CAT should be sound.
authors.affiliation: Discipline of Physiotherapy, James Cook University, Australia. michael.crowe@my.jcu.edu.au
authors.forename: Michael
authors.initials: M
authors.lastname: Crowe
authors.forename: Lorraine
authors.initials: L
authors.lastname: Sheppard
mesh_terms: D006801:Humans; D011581:Psychological Tests; D015203:Reproducibility of Results