title: Prospector: A web-based tool for rapid acquisition of gold standard data for pathology research and image analysis.
abstract: BACKGROUND
  Obtaining ground truth for pathological images is essential for various experiments, especially for training and testing image analysis algorithms. However, obtaining pathologist input is often difficult, time consuming and expensive. This leads to algorithms being over-fitted to small datasets, and inappropriate validation, which causes poor performance on real world data. There is a great need to gather data from pathologists in a simple and efficient manner, in order to maximise the amount of data obtained.
  
  
  METHODS
  We present a lightweight, web-based HTML5 system for administering and participating in data collection experiments. The system is designed for rapid input with minimal effort, and can be accessed from anywhere in the world with a reliable internet connection.
  
  
  RESULTS
  We present two case studies that use the system to assess how limitations on fields of view affect pathologist agreement, and to what extent poorly stained slides affect judgement. In both cases, the system collects pathologist scores at a rate of less than two seconds per image.
  
  
  CONCLUSIONS
  The system has multiple potential applications in pathology and other domains.
authors.affiliation: Section of Pathology, Anatomy and Tumour Biology, Leeds Institute of Cancer and Pathology, University of Leeds, UK.
authors.forename: Alexander I
authors.initials: AI
authors.lastname: Wright
authors.affiliation: School of Computing, University of Leeds, UK.
authors.forename: Derek R
authors.initials: DR
authors.lastname: Magee
authors.affiliation: Section of Pathology, Anatomy and Tumour Biology, Leeds Institute of Cancer and Pathology, University of Leeds, UK.
authors.forename: Philip
authors.initials: P
authors.lastname: Quirke
authors.affiliation: Section of Pathology, Anatomy and Tumour Biology, Leeds Institute of Cancer and Pathology, University of Leeds, UK ; Leeds Teaching Hospitals NHS Trust, Leeds, UK.
authors.forename: Darren E
authors.initials: DE
authors.lastname: Treanor
keywords: Data acquisition; gold standard; ground truth; training data; web experiment system