title: Application of reinforcement learning for segmentation of transrectal ultrasound images.
abstract: BACKGROUND
  Among different medical image modalities, ultrasound imaging has a very widespread clinical use. But, due to some factors, such as poor image contrast, noise and missing or diffuse boundaries, the ultrasound images are inherently difficult to segment. An important application is estimation of the location and volume of the prostate in transrectal ultrasound (TRUS) images. For this purpose, manual segmentation is a tedious and time consuming procedure.
  
  
  METHODS
  We introduce a new method for the segmentation of the prostate in transrectal ultrasound images, using a reinforcement learning scheme. This algorithm is used to find the appropriate local values for sub-images and to extract the prostate. It contains an offline stage, where the reinforcement learning agent uses some images and manually segmented versions of these images to learn from. The reinforcement agent is provided with reward/punishment, determined objectively to explore/exploit the solution space. After this stage, the agent has acquired knowledge stored in the Q-matrix. The agent can then use this knowledge for new input images to extract a coarse version of the prostate.
  
  
  RESULTS
  We have carried out experiments to segment TRUS images. The results demonstrate the potential of this approach in the field of medical image segmentation.
  
  
  CONCLUSIONS
  By using the proposed method, we can find the appropriate local values and segment the prostate. This approach can be used for segmentation tasks containing one object of interest. To improve this prototype, more investigations are needed.
authors.affiliation: Medical Instrument Analysis and Machine Intelligence Group, University of Waterloo, Waterloo, Canada. fsahba@alumni.uwaterloo.ca
authors.forename: Farhang
authors.initials: F
authors.lastname: Sahba
authors.forename: Hamid R
authors.initials: HR
authors.lastname: Tizhoosh
authors.forename: Magdy M A
authors.initials: MM
authors.lastname: Salama
mesh_terms: D000465:Algorithms; D001185:Artificial Intelligence; D006801:Humans; D007089:Image Enhancement / Q000379:methods; D007090:Image Interpretation, Computer-Assisted / Q000379:methods*; D021621:Imaging, Three-Dimensional / Q000379:methods; D008297:Male; D010363:Pattern Recognition, Automated / Q000379:methods*; D011467:Prostate / Q000000981:diagnostic imaging*; D012007:Rectum / Q000000981:diagnostic imaging*; D012054:Reinforcement, Psychology; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity; D014463:Ultrasonography / Q000379:methods*