Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix errors during training in ResponseSelector #4985

Merged
merged 13 commits into from
Dec 17, 2019

Conversation

tabergma
Copy link
Contributor

@tabergma tabergma commented Dec 17, 2019

Proposed changes:

  • Fix errors during training in ResponseSelector

Status (please check what you already did):

  • added some tests for the functionality
  • updated the documentation
  • updated the changelog (please check changelog for instructions)
  • reformat files using black (please check Readme for instructions)

dakshvar22
dakshvar22 previously approved these changes Dec 17, 2019
@tabergma
Copy link
Contributor Author

It seems like that there is more to it. We got the error

Number of examples differs for keys 'dict_keys(['text_features', 'intent_features', 'intent_ids', 'text_mask', 'intent_mask'])'. Number of examples should be the same for all data in session data.

for the following pipeline:

pipeline:
- name: WhitespaceTokenizer
  intent_split_symbol: _
- name: RegexFeaturizer
- name: CRFEntityExtractor
- name: EntitySynonymMapper
- name: CountVectorsFeaturizer
  analyzer: word
  min_ngram: 1
  max_ngram: 1
- name: CountVectorsFeaturizer
  analyzer: char_wb
  min_ngram: 1
  max_ngram: 4
- name: ConveRTFeaturizer
- name: EmbeddingIntentClassifier
  epochs: 151
  embed_dim: 20
- name: ResponseSelector
  epochs: 100

Training of ResponseSelector fails.

@dakshvar22 dakshvar22 self-requested a review December 17, 2019 12:42
@dakshvar22 dakshvar22 dismissed their stale review December 17, 2019 12:43

Still debugging

@tabergma tabergma changed the title Set correct default values for ResponseSelector Fix errors during training in ResponseSelector Dec 17, 2019
@tabergma
Copy link
Contributor Author

@dakshvar22 Thanks for your help! Ready for another review.

Copy link
Contributor

@dakshvar22 dakshvar22 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Tested it on carbon bot data with cross fold validation.
Added a test for capturing any errors like this in the future.

@tabergma tabergma merged commit 86cb4f5 into master Dec 17, 2019
@tabergma tabergma deleted the bug-fix-response-selector branch December 17, 2019 16:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants