-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CBOW model equivalent to the supervised learning model of fastText #960
Comments
For completeness linking to Fasttext Wrapper #847 |
I am still developing the code for this PR in https://github.com/giacbrd/ShallowLearn, I hope to start working on the fork ASAP |
@giacbrd any progress on the PR? Cheers. |
I am going to release a more stable model on my project, before Christmas, then I can port it in Gensim, it should be "easy"! |
I am finalizing the pull request. I am just thinking to design a better interface, but in general there is not much code. |
From Gensim integration point of view, an API extending the existing FastText Wrapper API would be preferable. Though FastText wrapper API is not yet released so can be changed |
Here I am doing something slightly different than re-implementing fastText. I have practically written a variant of the Word2Vec model, with the goal of learning the combinations sets_of_words->labels (i.e. text classification), where:
The wrapper, for now, is only for word embedding applications, but yes, it could be extended with the "supervised learning" component of fastText |
See the pull request #1153 |
fastText is an "evolution" of word2vec, it contains new models for word embeddings and models for learning the association document -> label, i.e. classification.
Implementing the latter can be obtained by reusing the Word2vec class (only for CBOW), defining an input layer of words and an output layer of labels with their specific vocabulary. The concept of windows in training can be dropped. It is possible to implement, for output computation, the softmax function, together with its already present "approximations" (negative sampling and huffman tree).
A LabeledWord2Vec class is already implemented https://github.com/giacbrd/ShallowLearn/blob/master/shallowlearn/word2vec.py , it should be ported and improved (it misses negative sampling, some methods implementation)
The text was updated successfully, but these errors were encountered: