Add model-last saving mechanism to pretraining #12459
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
thomashacker
added
enhancement
|
I'll fix and add some unit tests. If we decide to implement this feature, do we want to mention this in the documentation? |
thomashacker
changed the title
adrianeboyd
reviewed
Mar 27, 2023
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
|
And I would definitely add this to the docs. |
|
The CI seems to have gotten into a weird state, let me close and reopen this. |
3 tasks
adrianeboyd
reviewed
Mar 31, 2023
adrianeboyd
reviewed
Apr 3, 2023
svlandeg
reviewed
Apr 3, 2023
There was a problem hiding this comment.
Looks good to me.
The one thing I wonder about is whether this could influence existing workflows somehow, like users iterating over all .bin files in the output folder, assuming the names all look like modelX.bin with X an int, and that that could now fail when they encounter model-last.bin.
Could we make this opt-in for the upcoming bugfix release? Or at least provide an easy switch to turn this off if users don't want the additional duplicate file?
svlandeg
approved these changes
Apr 3, 2023
adrianeboyd
added a commit
to adrianeboyd/spaCy
that referenced
this pull request
Apr 3, 2023
* Adjust pretrain command * chane naming and add finally block * Add unit test * Add unit test assertions * Update spacy/training/pretrain.py Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com> * change finally block * Add to docs * Update website/docs/usage/embeddings-transformers.mdx * Add flag to skip saving model-last --------- Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR takes inspiration from the training loop to save the last epoch of pretraining as
model_last.bininstead ofmodel<last_epoch>.bin. The PR aims to make it easier for users to work with pretrained weights in an automated workflow (e.g. spaCy projects) and reduce manual adjustment based on the setmax_epochsin the training config.Types of change
Feature
Checklist