title: Improved feature-selection method considering the imbalance problem in text categorization.
abstract: The filtering feature-selection algorithm is a kind of important approach to dimensionality reduction in the field of the text categorization. Most of filtering feature-selection algorithms evaluate the significance of a feature for category based on balanced dataset and do not consider the imbalance factor of dataset. In this paper, a new scheme was proposed, which can weaken the adverse effect caused by the imbalance factor in the corpus. We evaluated the improved versions of nine well-known feature-selection methods (Information Gain, Chi statistic, Document Frequency, Orthogonal Centroid Feature Selection, DIA association factor, Comprehensive Measurement Feature Selection, Deviation from Poisson Feature Selection, improved Gini index, and Mutual Information) using naïve Bayes and support vector machines on three benchmark document collections (20-Newsgroups, Reuters-21578, and WebKB). The experimental results show that the improved scheme can significantly enhance the performance of the feature-selection methods.
authors: College of Information Engineering, Northeast Dianli University, Jilin, Jilin 132012, China.
authors: Jieming
authors: 0000-0002-2832-2648
authors: J
authors: Yang
authors: College of Information Engineering, Northeast Dianli University, Jilin, Jilin 132012, China.
authors: Zhaoyang
authors: 0000-0001-7599-9531
authors: Z
authors: Qu
authors: College of Information Engineering, Northeast Dianli University, Jilin, Jilin 132012, China.
authors: Zhiying
authors: 0000-0001-6509-0914
authors: Z
authors: Liu
mesh_terms: D000465:Algorithms; D008962:Models, Theoretical