title: Evidence levels for neuroradiology articles: low agreement among raters.
abstract: OBJECTIVE
  Because evidence-based articles are difficult to recognize among the large volume of publications available, some journals have adopted evidence-based medicine criteria to classify their articles. Our purpose was to determine whether an evidence-based medicine classification used by a subspecialty-imaging journal allowed consistent categorization of levels of evidence among different raters.
  
  
  METHODS
  One hundred consecutive articles in the American Journal of Neuroradiology were classified as to their level of evidence by the 2 original manuscript reviewers, and their interobserver agreement was calculated. After publication, abstracts and titles were reprinted and independently ranked by 3 different radiologists at 2 different time points. Interobserver and intraobserver agreement was calculated for these radiologists.
  
  
  RESULTS
  The interobserver agreement between the original manuscript reviewers was -0.2283 (standard error = 0.0000; 95% CI, -0.2283 to -0.2283); among the 3 postpublication reviewers for the first evaluation, it was 0.1899 (standard error = 0.0383; 95% CI, 0.1149-0.2649); and for the second evaluation, performed 3 months later, it was 0.1145 (standard error = 0.0350; 95% CI, 0.0460-0.1831). The intraobserver agreement was 0.2344 (standard error = 0.0660; 95% CI, 0.1050-0.3639), 0.3826 (standard error = 0.0738; 95% CI, 0.2379-0.5272), and 0.6611 (standard error = 0.0656; 95% CI, 0.5325-0.7898) for the 3 postpublication evaluators, respectively. These results show no-to-fair interreviewer agreement and a tendency to slight intrareviewer agreement.
  
  
  CONCLUSIONS
  Inconsistent use of evidence-based criteria by different raters limits their utility when attempting to classify neuroradiology-related articles.
authors: From the Departments of Neuroradiology (J.N.R., G.T., M.C.) Centro Hospitalar de Lisboa Central (J.N.R.), Lisbon, Portugal Joana-Ramalho@netcabo.pt.
authors: J N
authors: JN
authors: Ramalho
authors: From the Departments of Neuroradiology (J.N.R., G.T., M.C.).
authors: G
authors: G
authors: Tedesqui
authors: Radiology (M.R.), University of North Carolina Hospital, Chapel Hill, North Carolina Hospital Garcia de Orta (M.R.), Almada, Portugal.
authors: M
authors: M
authors: Ramalho
authors: Faculdade de Medicina da Universidade de São Paulo (R.S.A.), São Paulo, Brazil.
authors: R S
authors: RS
authors: Azevedo
authors: From the Departments of Neuroradiology (J.N.R., G.T., M.C.).
authors: M
authors: http://orcid.org/0000-0003-2960-1064
authors: M
authors: Castillo
mesh_terms: D019317:Evidence-Based Medicine / Q000145:classification*; D006801:Humans; D015588:Observer Variation; D010506:Periodicals as Topic / Q000592:standards*