title: Improving Automated Annotation of Benthic Survey Images Using Wide-band Fluorescence.
abstract: Large-scale imaging techniques are used increasingly for ecological surveys. However, manual analysis can be prohibitively expensive, creating a bottleneck between collected images and desired data-products. This bottleneck is particularly severe for benthic surveys, where millions of images are obtained each year. Recent automated annotation methods may provide a solution, but reflectance images do not always contain sufficient information for adequate classification accuracy. In this work, the FluorIS, a low-cost modified consumer camera, was used to capture wide-band wide-field-of-view fluorescence images during a field deployment in Eilat, Israel. The fluorescence images were registered with standard reflectance images, and an automated annotation method based on convolutional neural networks was developed. Our results demonstrate a 22% reduction of classification error-rate when using both images types compared to only using reflectance images. The improvements were large, in particular, for coral reef genera Platygyra, Acropora and Millepora, where classification recall improved by 38%, 33%, and 41%, respectively. We conclude that convolutional neural networks can be used to combine reflectance and fluorescence imagery in order to significantly improve automated annotation accuracy and reduce the manual annotation bottleneck.
authors.affiliation: Department of Computer Science and Engineering, University of California, San Diego, CA 92093, USA.
authors.forename: Oscar
authors.initials: O
authors.lastname: Beijbom
authors.affiliation: Charney School of Marine Sciences, University of Haifa, Haifa 3498838, Israel.
authors.forename: Tali
authors.initials: T
authors.lastname: Treibitz
authors.affiliation: Integrative Oceanography Division, University of California, San Diego, CA 92093, USA.
authors.forename: David I
authors.initials: DI
authors.lastname: Kline
authors.affiliation: The Interuniversity Institute for Marine Sciences in Eilat, Eilat 8810368, Israel.|Department of Zoology, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv 6997801, Israel.
authors.forename: Gal
authors.initials: G
authors.lastname: Eyal
authors.affiliation: Integrative Oceanography Division, University of California, San Diego, CA 92093, USA.
authors.forename: Adi
authors.initials: A
authors.lastname: Khen
authors.affiliation: Catlin Seaview Survey, Global Change Institute, The University of Queensland, Brisbane, Qld 4072, Australia.
authors.forename: Benjamin
authors.initials: B
authors.lastname: Neal
authors.affiliation: Department of Zoology, George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv 6997801, Israel.
authors.forename: Yossi
authors.initials: Y
authors.lastname: Loya
authors.affiliation: Integrative Oceanography Division, University of California, San Diego, CA 92093, USA.
authors.forename: B Greg
authors.initials: BG
authors.lastname: Mitchell
authors.affiliation: Department of Computer Science and Engineering, University of California, San Diego, CA 92093, USA.
authors.forename: David
authors.initials: D
authors.lastname: Kriegman
mesh_terms: D000465:Algorithms; D000818:Animals; D037421:Anthozoa / Q000145:classification / Q000254:growth & development; D059001:Aquatic Organisms / Q000145:classification / Q000254:growth & development*; D001331:Automation / Q000295:instrumentation / Q000379:methods*; D057894:Coral Reefs; D017753:Ecosystem; D005453:Fluorescence; D006801:Humans; D007091:Image Processing, Computer-Assisted / Q000295:instrumentation / Q000379:methods; D019156:Indian Ocean; D007557:Israel; D016571:Neural Networks, Computer; D010781:Photography / Q000295:instrumentation / Q000379:methods*; D015203:Reproducibility of Results