Abstract

We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity task, and the results are compared to the previ-ously best performing techniques based on different types of neural networks. We observe large improvements in accuracy at much lower computational cost, i.e. it takes less than a day to learn high quality word vectors from a 1.6 billion words data set. Furthermore, we show that these vectors provide state-of-the-art perfor-mance on our test set for measuring syntactic and semantic word similarities.

Keywords

Word (group theory)Computer scienceSimilarity (geometry)Set (abstract data type)Artificial intelligenceVector spaceNatural language processingTask (project management)Test setSemantic similaritySpace (punctuation)Vector space modelArtificial neural networkMathematics

Affiliated Institutions

Related Publications

Finding Structure in Time

Jeffrey L. Elman

Time underlies many interesting human behaviors. Thus, the question of how to represent time in connectionist models is very important. One approach is to represent time implici...

1990 Cognitive Science 10427 citations

Publication Info

Year
2013
Type
preprint
Citations
11710
Access
Closed

External Links

Citation Metrics

11710
OpenAlex

Cite This

Tomáš Mikolov, Kai Chen, Greg S. Corrado et al. (2013). Efficient Estimation of Word Representations in Vector Space. arXiv (Cornell University) .