title: Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD.
abstract: In a previous article, we presented a systematic computational study of the extraction of semantic representations from the word-word co-occurrence statistics of large text corpora. The conclusion was that semantic vectors of pointwise mutual information values from very small co-occurrence windows, together with a cosine distance measure, consistently resulted in the best representations across a range of psychologically relevant semantic tasks. This article extends that study by investigating the use of three further factors--namely, the application of stop-lists, word stemming, and dimensionality reduction using singular value decomposition (SVD)--that have been used to provide improved performance elsewhere. It also introduces an additional semantic task and explores the advantages of using a much larger corpus. This leads to the discovery and analysis of improved SVD-based methods for generating semantic representations (that provide new state-of-the-art performance on a standard TOEFL task) and the identification and discussion of problems and misleading results that can arise without a full systematic study.
authors.affiliation: School of Computer Science, University of Birmingham, B15 2TT Birmingham, UK. j.a.bullinaria@cs.bham.ac.uk
authors.forename: John A
authors.initials: JA
authors.lastname: Bullinaria
authors.forename: Joseph P
authors.initials: JP
authors.lastname: Levy
mesh_terms: D001185:Artificial Intelligence; D002755:Choice Behavior; D003627:Data Interpretation, Statistical; D006801:Humans; D007600:Judgment; D011578:Psycholinguistics; D012660:Semantics; D012984:Software