Code

Dmoe integration#1210

Code

Open

DayOfThePenguin wants to merge 14 commits intomainEleutherAI/gpt-neox:mainfrom dmoe_integrationEleutherAI/gpt-neox:dmoe_integrationCopy head branch name to clipboard

+1,309-723

Commits

Commits on May 6, 2024

• feat: ensure only the right architectures get built vs all of them for 11.x

  DayOfThePenguin

  committed
  52a2001

• feat: clean up megablocks-based DMoE implementation

  DayOfThePenguin

  committed
  9c66895

• feat: update args, configs, and requirements

  DayOfThePenguin

  committed
  3388c51

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  e987126

Commits on May 21, 2024

• Merge branch 'main' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  7d6f265

• Merge branch 'dmoe_integration' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  4aebc2c

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  d9f8d55

• feat: Update readme and example config

  DayOfThePenguin

  committed
  f8c3776

• Merge branch 'dmoe_integration' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  3ef5b66

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  33e41d7

Commits on May 22, 2024

• Merge branch 'main' of https://github.com/EleutherAI/gpt-neox into dmoe_integration

  DayOfThePenguin

  committed
  613aeb9

• Update NeoXArgs docs automatically

  

  github-actions

  committed
  35c7225

Commits on Dec 3, 2024

• Merge branch 'main' into dmoe_integration

  Quentin-Anthony

  committed
  fb68c07

Commits on Dec 10, 2024

• - Updated sinkhorn initialization and add max_iter argument.

  Show description for 542103f
  aurelion-source

  committed
  542103f