title: Hybrid indexes for repetitive datasets.
abstract: Advances in DNA sequencing mean that databases of thousands of human genomes will soon be commonplace. In this paper, we introduce a simple technique for reducing the size of conventional indexes on such highly repetitive texts. Given upper bounds on pattern lengths and edit distances, we pre-process the text with the lossless data compression algorithm LZ77 to obtain a filtered text, for which we store a conventional index. Later, given a query, we find all matches in the filtered text, then use their positions and the structure of the LZ77 parse to find all matches in the original text. Our experiments show that this also significantly reduces query times.
authors.affiliation: Department of Computer Sciences, University of Chile, , Avenida Blanco Encalada 2120, C.P. 837-0459, Santiago, Chile.
authors.forename: H
authors.initials: H
authors.lastname: Ferrada
authors.forename: T
authors.initials: T
authors.lastname: Gagie
authors.forename: T
authors.initials: T
authors.lastname: Hirvola
authors.forename: S J
authors.initials: SJ
authors.lastname: Puglisi
mesh_terms: D057225:Data Mining / Q000379:methods*; D030541:Databases, Genetic; D023281:Genomics; D012441:Saccharomyces cerevisiae / Q000235:genetics
keywords: LZ77; approximate pattern matching; indexing