-
-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Re-design "*2vec" implementations #1777
Changes from 66 commits
31943ae
d7209f4
fe19b9a
fece94f
e310dbf
30872ac
2892f37
4b1e7f8
68ac5bc
7f60a47
abc5702
b60a9d5
ca1eae9
dab7b99
d249668
99cf2ad
267c682
c2bbb20
7d774d7
9b156f5
0db83f1
b761dff
817f71b
75892cc
61c4e5e
707aef3
e370314
45347f3
ab8dd4b
679e82f
1f488a5
0f666f4
6a9171d
1246d13
4bba589
26b9b06
a923e7e
51df908
bfae0e7
9c261e0
d8d22bd
8301b03
36e6a30
ae60bd8
eddd24e
f3d76cf
9367dc6
a721aac
65b8821
9f1103e
52d1e5f
6941e1e
8574055
173a8e9
be73e0b
0fc8340
673086d
ce0dee9
f300088
211c286
b4700ed
142c8a6
74ce823
3281a73
b1a7390
995d1cf
fc9e77f
9fba59f
c9d9ec8
883cb81
7fcc3a4
9001abe
6c42905
0bea623
09f9bdf
4b142a0
710c124
6f1c522
60db35d
922ae60
f3e2259
0a76c2a
06e03ef
9aa9b66
cbffa32
4caa3f4
31f9943
5650bab
da539e2
818439d
54c9b2e
bb54290
0d1c48c
13f5ea9
8cc2bf6
ac2d01f
0fae977
d58dc41
2422994
401d46e
46b0b3a
902aed7
3562818
83374be
d8455fa
1f38dc7
cd4e22d
fd2e697
02072b1
114ab5f
0179835
0601c69
572c960
62b0852
e9ebaa8
d7cee63
79c1263
19f2ee5
7a32739
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,49 @@ | ||
#!/usr/bin/env python | ||
# -*- coding: utf-8 -*- | ||
|
||
""" | ||
Abstract base class to build callbacks. Callbacks are used to apply custom functions over the model | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This isn't "abstract", maybe you should use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It may not make sense to make this inherit |
||
at specific points during training (epoch start, batch end etc.). To implement a Callback, subclass | ||
:class: ~gensim.callbacks.Callback, look at the example below which creates a callback to save a training model | ||
after each epoch: | ||
|
||
>>> from gensim.test.utils import common_texts as sentences | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good example, I like it 👍 |
||
>>> from gensim.callbacks import Callback | ||
>>> from gensim.models import word2vec | ||
>>> class ModelEpochSaver(Callback): # Callback to save model after every epoch | ||
>>> def __init__(self, path_prefix): | ||
>>> self.path_prefix = path_prefix | ||
>>> def on_epoch_end(self, model): | ||
>>> model.save('{}_epoch{}'.format(self.path_prefix, self.cur_epoch)) | ||
>>> self.cur_epoch += 1 | ||
>>> def on_train_begin(self, model): | ||
>>> self.cur_epoch = 0 | ||
>>> epoch_saver = ModelEpochSaver('axax') | ||
>>> model = word2vec.Word2Vec(sentences, iter=5, size=10, min_count=0, seed=42, callbacks=[epoch_saver]) | ||
|
||
""" | ||
|
||
|
||
class Callback(object): | ||
"""Abstract base class used to build new callbacks.""" | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pls remove |
||
|
||
def __init__(self): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. redundant There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Need to add *args, **kwargs (because probably user wants to store something in this class, for example, for evaluation proposes. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We can take out There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. @menshikh-iv Not sure what you mean. If anyone wanted to use |
||
pass | ||
|
||
def on_epoch_begin(self, model): | ||
pass | ||
|
||
def on_epoch_end(self, model): | ||
pass | ||
|
||
def on_batch_begin(self, model): | ||
pass | ||
|
||
def on_batch_end(self, model): | ||
pass | ||
|
||
def on_train_begin(self, model): | ||
pass | ||
|
||
def on_train_end(self, model): | ||
pass |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,7 +13,7 @@ | |
from .logentropy_model import LogEntropyModel # noqa:F401 | ||
from .word2vec import Word2Vec # noqa:F401 | ||
from .doc2vec import Doc2Vec # noqa:F401 | ||
from .keyedvectors import KeyedVectors # noqa:F401 | ||
from .word2vec import Word2VecKeyedVectors as KeyedVectors # noqa:F401 | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. An old variant is better (like from @jayantj) with |
||
from .ldamulticore import LdaMulticore # noqa:F401 | ||
from .phrases import Phrases # noqa:F401 | ||
from .normmodel import NormModel # noqa:F401 | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
callbacks module should be "at least" in
gensim.models
I think, I have no idea, why this should be in "root" of gensim.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There already exists
gensim.models.callbacks
. I can add myCallback
base class to this file but will require me to rename the base class as another class with the same name exists -- link.