Code

fix(moe): dropless moe loss#348

Code

Merged

sunpengsdu merged 5 commits intoInternLM:developInternLM/InternEvo:developfrom blankde:fix/dropless_moe_lossblankde/InternEvo:fix/dropless_moe_lossCopy head branch name to clipboardOct 18, 2024

+31-10

Commits

Commits on Oct 10, 2024

• cal aux loss before capacity

  blankde

  committed
  4e95c3e

• add determine token count

  blankde

  committed
  1fbc545

Commits on Oct 11, 2024

• add comments

  blankde

  committed
  3c20ac0

• move cal loss func to forward(.)

  blankde

  committed
  a485ccf

• fix bugs

  blankde

  committed
  c6e16e0