title: n-Gram-Based Text Compression.
abstract: We propose an efficient method for compressing Vietnamese text usingn-gram dictionaries. It has a significant compression ratio in comparison with those of state-of-the-art methods on the same dataset. Given a text, first, the proposed method splits it inton-grams and then encodes them based onn-gram dictionaries. In the encoding phase, we use a sliding window with a size that ranges from bigram to five grams to obtain the best encoding stream. Eachn-gram is encoded by two to four bytes accordingly based on its correspondingn-gram dictionary. We collected 2.5 GB text corpus from some Vietnamese news agencies to buildn-gram dictionaries from unigram to five grams and achieve dictionaries with a size of 12 GB in total. In order to evaluate our method, we collected a testing set of 10 different text files with different sizes. The experimental results indicate that our method achieves compression ratio around 90% and outperforms state-of-the-art methods.
authors.affiliation: Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam.
authors.forename: Vu H
authors.identifier: 0000-0002-1604-9066
authors.initials: VH
authors.lastname: Nguyen
authors.affiliation: Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam.
authors.forename: Hien T
authors.identifier: 0000-0001-8940-3778
authors.initials: HT
authors.lastname: Nguyen
authors.affiliation: Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology, Ho Chi Minh City, Vietnam.
authors.forename: Hieu N
authors.initials: HN
authors.lastname: Duong
authors.affiliation: Faculty of Electrical Engineering and Computer Science, VSB-Technical University of Ostrava, Ostrava, Czech Republic.
authors.forename: Vaclav
authors.initials: V
authors.lastname: Snasel
mesh_terms: D000465:Algorithms; D044466:Asian People; D044962:Data Compression; D004014:Dictionaries as Topic; D006801:Humans; D014825:Vocabulary