title: Objective structured assessment of technical competence in transthoracic echocardiography: a validity study in a standardised setting.
abstract: BACKGROUND
  Competence in transthoracic echocardiography (TTE) is unrelated to traditional measures of TTE competence, such as duration of training and number of examinations performed. This study aims to explore aspects of validity of an instrument for structured assessment of echocardiographic technical skills.
  
  
  METHODS
  The study included 45 physicians with three different clinical levels of echocardiography competence who all scanned the same healthy male following national guidelines. An expert in echocardiography (OG) evaluated all the recorded, de-identified TTE images blindly using the developed instrument for assessment of TTE technical skills. The instrument consisted of both a global rating scale and a procedure specific checklist. Two scores were calculated for each examination: A global rating score and a total checklist score. OG rated ten examinations twice for intra-rater reliability, and another expert rated the same ten examinations for inter-rater reliability. A small pilot study was then performed with focus on content validity. This pilot study included nine physicians who scanned three patients with different pathologies as well as different technical difficulties.
  
  
  RESULTS
  Validity of the TTE technical skills assessment instrument was supported by a significant correlation found between level of expertise and both the global score (Spearman 0.76, p<0.0001) and the checklist score (Spearman 0.74, p<0.001). Both scores were able to distinguish between the three levels of competence that were represented in the physician group. Reliability was supported by acceptable inter- and intra-rater values. The pilot study showed a tendency to improved scores with increasing expertise levels, suggesting that the instrument could also be used when pathologies were present.
  
  
  CONCLUSIONS
  We designed and developed a structured assessment instrument of echocardiographic technical skills that showed evidence of validity in terms of high correlations between test scores on a normal person and the level of physician competence, as well as acceptable inter- and intra-rater reliability scores. Further studies should, however, be performed to determine the adequate number of assessments needed to ensure high content validity and reliability in a clinical setting.
authors.affiliation: Center for Medical Education, Aarhus University, Aarhus, Denmark. dornis@rm.dk
authors.forename: Dorte Guldbrand
authors.initials: DG
authors.lastname: Nielsen
authors.forename: Ole
authors.initials: O
authors.lastname: Gotzsche
authors.forename: Berit
authors.initials: B
authors.lastname: Eika
mesh_terms: D002309:Cardiology / Q000193:education / Q000592:standards; D002983:Clinical Competence / Q000592:standards*; D004452:Echocardiography / Q000592:standards; D005260:Female; D006801:Humans; D007396:Internship and Residency / Q000592:standards; D008297:Male; D010820:Physicians / Q000592:standards
references.citation: Obstet Gynecol. 2000 Jul;96(1):146-50
references.pmid: 10862857
references.citation: J Am Coll Cardiol. 1994 Jun;23(7):1723-5
references.pmid: 8195538
references.citation: Acad Med. 1996 Dec;71(12):1363-5
references.pmid: 9114900
references.citation: Acad Med. 2001 Oct;76(10):1053-5
references.pmid: 11597848
references.citation: Acad Med. 1998 Sep;73(9):993-7
references.pmid: 9759104
references.citation: J Am Coll Cardiol. 2003 Feb 19;41(4):687-708
references.pmid: 12598084
references.citation: Eur J Echocardiogr. 2008 Jul;9(4):438-48
references.pmid: 18579482
references.citation: World J Surg. 2008 Feb;32(2):156-60
references.pmid: 17562106
references.citation: J Endourol. 2008 Dec;22(12):2707-11
references.pmid: 19025389
references.citation: Can J Cardiol. 2005 Jul;21(9):763-80
references.pmid: 16082436
references.citation: Best Pract Res Clin Obstet Gynaecol. 2010 Dec;24(6):767-82
references.pmid: 20598644
references.citation: Acta Cardiol. 1995;50(4):265-71
references.pmid: 8540269
references.citation: Br J Surg. 2007 Jun;94(6):726-30
references.pmid: 17315174
references.citation: Am J Med. 2006 Feb;119(2):166.e7-16
references.pmid: 16443422
references.citation: Med Educ. 2003 Jun;37(6):539-43
references.pmid: 12787377
references.citation: Am J Surg. 2006 Sep;192(3):372-8
references.pmid: 16920433
references.citation: Med Teach. 2006 Mar;28(2):117-28
references.pmid: 16707292
references.citation: Adv Health Sci Educ Theory Pract. 1996 Jan;1(1):41-67
references.pmid: 24178994
references.citation: J Am Soc Echocardiogr. 2006 Jul;19(7):924-31
references.pmid: 16825004
references.citation: Br J Surg. 1997 Feb;84(2):273-8
references.pmid: 9052454
references.citation: Surg Endosc. 2006 Sep;20(9):1460-6
references.pmid: 16823649
references.citation: Eur J Echocardiogr. 2007 Jan;8(1):74-9
references.pmid: 17175201
references.citation: Can J Cardiol. 2000 Apr;16(4):457-62
references.pmid: 10787459
references.citation: Med Educ. 2008 Apr;42(4):364-73
references.pmid: 18338989
references.citation: Acad Med. 1999 Oct;74(10):1129-34
references.pmid: 10536636
references.citation: N Engl J Med. 2007 Jan 25;356(4):387-96
references.pmid: 17251535
references.citation: J Am Coll Cardiol. 2008 Jan 22;51(3):361-7
references.pmid: 18206753
references.citation: Eur J Echocardiogr. 2009 Dec;10(8):893-905
references.pmid: 19889658
references.citation: Am J Obstet Gynecol. 2008 Dec;199(6):676.e1-6
references.pmid: 19084100
references.citation: Obstet Gynecol. 2001 Sep;98(3):412-6
references.pmid: 11530121