title: Multi-modal registration for correlative microscopy using image analogies.
abstract: Correlative microscopy is a methodology combining the functionality of light microscopy with the high resolution of electron microscopy and other microscopy technologies for the same biological specimen. In this paper, we propose an image registration method for correlative microscopy, which is challenging due to the distinct appearance of biological structures when imaged with different modalities. Our method is based on image analogies and allows to transform images of a given modality into the appearance-space of another modality. Hence, the registration between two different types of microscopy images can be transformed to a mono-modality image registration. We use a sparse representation model to obtain image analogies. The method makes use of corresponding image training patches of two different imaging modalities to learn a dictionary capturing appearance relations. We test our approach on backscattered electron (BSE) scanning electron microscopy (SEM)/confocal and transmission electron microscopy (TEM)/confocal images. We perform rigid, affine, and deformable registration via B-splines and show improvements over direct registration using both mutual information and sum of squared differences similarity measures to account for differences in image appearance.
authors.affiliation: Department of Computer Science, University of North Carolina at Chapel Hill, United States. Electronic address: tiancao@cs.unc.edu.
authors.forename: Tian
authors.initials: T
authors.lastname: Cao
authors.affiliation: Microsoft Research, Cambridge, United Kingdom.
authors.forename: Christopher
authors.initials: C
authors.lastname: Zach
authors.affiliation: Delaware Biotechnology Institute, University of Delaware, United States.
authors.forename: Shannon
authors.initials: S
authors.lastname: Modla
authors.affiliation: Delaware Biotechnology Institute, University of Delaware, United States.
authors.forename: Debbie
authors.initials: D
authors.lastname: Powell
authors.affiliation: Delaware Biotechnology Institute, University of Delaware, United States; Carl Zeiss Microscopy, United States.
authors.forename: Kirk
authors.initials: K
authors.lastname: Czymmek
authors.affiliation: Department of Computer Science, University of North Carolina at Chapel Hill, United States; Biomedical Research Imaging Center, University of North Carolina at Chapel Hill, United States.
authors.forename: Marc
authors.initials: M
authors.lastname: Niethammer
mesh_terms: D000465:Algorithms; D000818:Animals; D007089:Image Enhancement / Q000379:methods*; D007091:Image Processing, Computer-Assisted / Q000379:methods*; D051379:Mice; D018613:Microscopy, Confocal; D008854:Microscopy, Electron; D015233:Models, Statistical; D064847:Multimodal Imaging; D015203:Reproducibility of Results; D012680:Sensitivity and Specificity
keywords: Correlative microscopy; Image analogies; Multi-modal registration; Sparse representation models