Skip-gram Word2Vec

Introduced by Mikolov et al. in Efficient Estimation of Word Representations in Vector Space

Skip-gram Word2Vec is an architecture for computing word embeddings. Instead of using surrounding words to predict the center word, as with CBow Word2Vec, Skip-gram Word2Vec uses the central word to predict the surrounding words.

The skip-gram objective function sums the log probabilities of the surrounding $n$ words to the left and right of the target word $w_{t}$ to produce the following objective:

$$J_\theta = \frac{1}{T}\sum^{T}_{t=1}\sum_{-n\leq{j}\leq{n}, \neq{0}}\log{p}\left(w_{j+1}\mid{w_{t}}\right)$$

Source: Efficient Estimation of Word Representations in Vector Space

Latest Papers

PAPER DATE
A Subword Guided Neural Word Segmentation Model for Sindhi
Wazir AliJay KumarZenglin XuCongjian LuoJunyu LuJunming ShaoRajesh KumarYazhou Ren
2020-12-30
Improving Clinical Document Understanding on COVID-19 Research with Spark NLP
| Veysel KocamanDavid Talby
2020-12-07
Transformer Query-Target Knowledge Discovery (TEND): Drug Discovery from CORD-19
Leo K. TamXiaosong WangDaguang Xu
2020-11-28
FarsTail: A Persian Natural Language Inference Dataset
| Hossein AmirkhaniMohammad Azari JafariAzadeh AmirakZohreh PourjafariSoroush Faridan JahromiZeinab Kouhkan
2020-09-18
Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings
Rishi BommasaniKelly DavisClaire Cardie
2020-07-01
Why is penguin more similar to polar bear than to sea gull? Analyzing conceptual knowledge in distributional models
Pia Sommerauer
2020-07-01
An Empirical Evaluation of Text Representation Schemes on Multilingual Social Web to Filter the Textual Aggression
Sandip ModhaPrasenjit Majumder
2019-04-16
Predicting Role Relevance with Minimal Domain Expertise in a Financial Domain
Mayank Kejriwal
2017-04-19
Effective search space reduction for spell correction using character neural embeddings
PHarshit e
2017-04-01
Efficient Estimation of Word Representations in Vector Space
| Tomas MikolovKai ChenGreg CorradoJeffrey Dean
2013-01-16

Components

COMPONENT TYPE
🤖 No Components Found You can add them if they exist; e.g. Mask R-CNN uses RoIAlign

Categories