title: Computing symmetrical strength of N-grams: a two pass filtering approach in automatic classification of text documents.
abstract: The contiguous sequences of the terms (N-grams) in the documents are symmetrically distributed among different classes. The symmetrical distribution of the N-Grams raises uncertainty in the belongings of the N-Grams towards the class. In this paper, we focused on the selection of most discriminating N-Grams by reducing the effects of symmetrical distribution. In this context, a new text feature selection method named as the symmetrical strength of the N-Grams (SSNG) is proposed using a two pass filtering based feature selection (TPF) approach. Initially, in the first pass of the TPF, the SSNG method chooses various informative N-Grams from the entire extracted N-Grams of the corpus. Subsequently, in the second pass the well-known Chi Square (χ(2)) method is being used to select few most informative N-Grams. Further, to classify the documents the two standard classifiers Multinomial Naive Bayes and Linear Support Vector Machine have been applied on the ten standard text data sets. In most of the datasets, the experimental results state the performance and success rate of SSNG method using TPF approach is superior to the state-of-the-art methods viz. Mutual Information, Information Gain, Odds Ratio, Discriminating Feature Selection and χ(2).
authors.affiliation: Department of Computer Applications, National Institute of Technology Raipur, Raipur, CG India.
authors.forename: Deepak
authors.identifier: 0000-0002-1536-2261
authors.initials: D
authors.lastname: Agnihotri
authors.affiliation: Department of Computer Applications, National Institute of Technology Raipur, Raipur, CG India.
authors.forename: Kesari
authors.initials: K
authors.lastname: Verma
authors.affiliation: Department of Computer Engineering and Applications, National Institute of Technical Teachers Training & Research Bhopal, Bhopal, MP India.
authors.forename: Priyanka
authors.initials: P
authors.lastname: Tripathi
keywords: Feature selection; Information retrieval; Text analysis; Text classification; Text mining