[MRG] Poincare l2 regularization #1734
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ectors and PoincareKeyedVectors
…nce and vector_distance_batch
janpom
reviewed
Nov 22, 2017
| @@ -91,6 +91,8 @@ def __init__(self, train_data, size=50, alpha=0.1, negative=10, workers=1, epsil | |||
| Number of threads to use for training the model. | |||
| epsilon : float, optional | |||
| Constant used for clipping embeddings below a norm of one. | |||
| regularization_coeff : float, optional | |||
There was a problem hiding this comment.
Can the L2 regularization be disabled? Setting regularization_coeff = 0.0 should do it, correct? Might be worth mentioning here.
There was a problem hiding this comment.
That's correct. Good point, adding.
…arization Conflicts: gensim/models/keyedvectors.py gensim/models/poincare.py gensim/viz/__init__.py gensim/viz/poincare.py
… poincare_l2_regularization
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Adds l2-regularization for Poincare embeddings, which improves results and results in models much more similar to the models visualized in the original paper. To be merged post #1700
Without regularization, the model looks like this -
The problem with this is that most of the nodes are too close to the boundary. In order to prevent this, l2-regularization is added to only the parent node in the training hypernymy relations, which results in the following model -
Also adds cleaner handling of autograd, where the import is not conditional, however if autograd cannot be imported, gradient checking is automatically skipped along with a warning.