What a catch! traits that define good annotators.
Human annotation and chart review is an important process in biomedical informatics research, but which humans are best suited for the job? Domain expertise, such as medical or linguistic knowledge is desirable, but other factors may be equally important. The University of Utah has a group of 20+ reviewers with backgrounds in medicine and linguistics, and 10 key traits have surfaced in those best able to annotate quickly and with high quality. To identify reviewers with these key traits, we created a hiring process that includes interviewing candidates, testing their medical and linguistic knowledge, and having them complete an annotation exercise on realistic medical text. Each step is designed to assess the key traits and allow the investigator to choose the skill set required for each project.
VA Salt Lake City Heath Care System, Salt Lake City, UT, USA.
Tyler B
TB
Forbush
Shuying
S
Shen
Brett R
BR
South
Scott L
SL
Duvalla
D066289:Data Curation / Q000379:methods*; D057286:Electronic Health Records; D007587:Job Description; D062527:Meaningful Use / Q000458:organization & administration*; D008490:Medical Informatics; D010560:Personnel Selection / Q000379:methods*; D014589:Utah; D000078329:Workforce