Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(loss)/add different operator types for cross_entropy #386

Merged
merged 7 commits into from
Dec 17, 2024

Conversation

yingtongxiong
Copy link
Contributor

Motivation

  1. support different operator types for cross_entropy
    image
  2. add op_type config
    loss = dict( label_smoothing=0, op_type="flash_vocab_parallel", )

reference this https://github.com/pjlab-sys4nlp/InternTrain/pull/148

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here and update the documentation.

Checklist

Before PR:

  • Pre-commit or other linting tools are used to fix the potential lint issues.

@sunpengsdu
Copy link
Contributor

@yingtongxiong merge下develop分支过CI吧

@yingtongxiong
Copy link
Contributor Author

@yingtongxiong merge下develop分支过CI吧

好的

@yingtongxiong
Copy link
Contributor Author

@yingtongxiong merge下develop分支过CI吧

@sunpengsdu 已经好了

@sunpengsdu sunpengsdu merged commit 141e9eb into InternLM:develop Dec 17, 2024
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants