title: True Reproducibility of UltraSound Techniques (TRUST): systematic review of reliability studies in obstetrics and gynecology.
abstract: OBJECTIVE
  To examine the quality of methods used and the accuracy of the interpretation of agreement in existing studies that examine the reliability of ultrasound measurements and judgments in obstetrics and gynecology.
  
  
  METHODS
  A systematic search of MEDLINE was performed on 25 March 2014, looking for studies that examined the reliability of ultrasound measurements and judgments in obstetrics and gynecology with evaluation of concordance (CCC) or intraclass (ICC) correlation coefficients or kappa as a main objective.
  
  
  RESULTS
  Seven hundred and thirty-three records were examined on the basis of their title and abstract, of which 141 full-text articles were examined completely for eligibility. We excluded 29 studies because they did not report CCC/ICC/kappa, leaving 112 studies that were included in our analysis. Two studies reported both ICC and kappa and were counted twice, therefore, the number used as the denominator in the analyses was 114. Only 16/114 (14.0%) studies were considered to be well designed (independent acquisition and blinded analysis) and to have interpreted the results properly. Most errors occurring in the studies are likely to overestimate the reliability of the method examined.
  
  
  CONCLUSIONS
  The vast majority of published studies examined had important flaws in design, interpretation and/or reporting. Such limitations are important to identify as they might create false confidence in the existing measurements and judgments, jeopardizing clinical practice and future research. Specific guidelines aimed at improving the quality of reproducibility studies that examine ultrasound methods should be encouraged.
authors.affiliation: Department of Obstetrics and Gynecology, Ribeirao Preto Medical School, University of Sao Paulo (DGO-FMRP-USP), Ribeirao Preto, Brazil.
authors.forename: M A
authors.initials: MA
authors.lastname: Coelho Neto
authors.affiliation: Department of Obstetrics and Gynecology, Ribeirao Preto Medical School, University of Sao Paulo (DGO-FMRP-USP), Ribeirao Preto, Brazil.|School of Health Technology - Ultrasonography School of Ribeirao Preto (FATESA-EURP), Ribeirao Preto, Brazil.
authors.forename: P
authors.initials: P
authors.lastname: Roncato
authors.affiliation: Department of Obstetrics and Gynecology, Ribeirao Preto Medical School, University of Sao Paulo (DGO-FMRP-USP), Ribeirao Preto, Brazil.
authors.forename: C O
authors.initials: CO
authors.lastname: Nastri
authors.affiliation: Department of Obstetrics and Gynecology, Ribeirao Preto Medical School, University of Sao Paulo (DGO-FMRP-USP), Ribeirao Preto, Brazil.
authors.forename: W P
authors.identifier: http://orcid.org/0000-0002-7593-5607
authors.initials: WP
authors.lastname: Martins
mesh_terms: D005260:Female; D006176:Gynecology / Q000592:standards*; D006801:Humans; D009774:Obstetrics / Q000592:standards*; D011247:Pregnancy; D015203:Reproducibility of Results; D014463:Ultrasonography / Q000592:standards*; D016216:Ultrasonography, Prenatal / Q000592:standards
keywords: gynecology; obstetrics; reproducibility of results; ultrasound