title: Expert panels as a reference standard in orthodontic research: An assessment of published methods and reporting.
abstract: BACKGROUND
  The aim of this study was to investigate the expert panel methodology applied in orthodontics and its reporting quality. Additionally, the relationship between the reporting quality and a range of variables was explored.
  
  
  METHODS
  PubMed was searched for orthodontic studies in which the final diagnosis or assessment was made by 2 or more experts published up to March 16, 2015. Reporting quality assessment was conducted using an established modified checklist. The relationship between potential predictors and the total score was assessed using univariable linear regression.
  
  
  RESULTS
  We identified 237 studies with a mean score of 9.97 (SD, 1.12) out of a maximum of 15. Critical information about panel methodology was missing in all studies. The panel composition differed substantially across studies, ranging from 2 to 646 panel members, with large variations in the expertise represented. Only 17 studies (7.2%) reported sample size calculations to justify the panel size. Panel members were partly blinded in 65 (27.4%) studies. Most studies failed to report which statistic was used to compute intrarater (65.8%) and interrater (66.2%) agreements. Journal type (nonorthodontic: β, 0.23; 95% CI, -0.07 to 0.54 compared with orthodontic), publication year (β, 0; 95% CI, -0.02 to 0.02 for each additional year), number of authors (1-3: β, 0.30; 95% CI, -0.13 to 0.74 compared with at least 6; 4-5: β, 0.18; 95% CI, -0.29 to 0.33 compared with at least 6), and number of centers involved (single: β, 0.20; 95% CI, -0.14 to 0.54 compared with multicenter) were not significant predictors of improved reporting. Studies published in Asia and Australia had significantly lower scores compared with those published in Europe (β, -0.54; 95% CI, -0.92 to -0.17).
  
  
  CONCLUSIONS
  Formal guidelines on methodology and reporting of studies involving expert panels are required.
authors.affiliation: Department of Orthodontics, 251 Air Force General Hospital, Athens, Greece. Electronic address: evalempesi@yahoo.gr.
authors.forename: Evangelia
authors.initials: E
authors.lastname: Lempesi
authors.affiliation: Department of Orthodontics, School of Dentistry, Aristotle University of Thessaloniki, Thessaloniki, Greece.
authors.forename: Electra
authors.initials: E
authors.lastname: Toulia
authors.affiliation: Department of Orthodontics and Dentofacial Orthopedics, Dental School, Medical Faculty, University of Bern, Bern, Switzerland; private practice, Corfu, Greece.
authors.forename: Nikolaos
authors.initials: N
authors.lastname: Pandis
mesh_terms: D018865:Dental Research / Q000592:standards* / Q000706:statistics & numerical data; D006801:Humans; D007598:Journalism, Dental / Q000592:standards*; D009970:Orthodontics / Q000592:standards* / Q000706:statistics & numerical data; D012015:Reference Standards