Code

Mixtral enablement. #120

Code

Merged

wang2yn84 merged 17 commits intomainAI-Hypercomputer/jetstream-pytorch:mainfrom mixtralAI-Hypercomputer/jetstream-pytorch:mixtralCopy head branch name to clipboardJun 11, 2024

+918-8

Commits

Commits on Jun 10, 2024

• Initial Mixtral enablement.

  wang2yn84

  committed
  c5c0772

  Copy full SHA for c5c0772

• Adds the mistral tokenizer model.

  wang2yn84

  committed
  277b02c

  Copy full SHA for 277b02c

• Updates the convert checkpoint file to handle mistral model.

  wang2yn84

  committed
  8be8c36

  Copy full SHA for 8be8c36

• Renames the typo of the model name.

  wang2yn84

  committed
  ab8c802

  Copy full SHA for ab8c802

• Fixing checkpoing loading. Still has some issue. Push to debug.

  wang2yn84

  committed
  98fdf71

  Copy full SHA for 98fdf71

• Running on CPU working, temporarily disable the generate jit to see it's moving. But the outputs doesn't make sense yet because weights are not loaded yet.

  wang2yn84

  committed
  aa78dbd

  Copy full SHA for aa78dbd

• Fix checkpoint loading issue. Right now loading from the gpt-fast converter with qkv fusion.

  wang2yn84

  committed
  1846cf9

  Copy full SHA for 1846cf9

• Fix the ckpt conversion script for mistral model. Fix the freqs_cis for loading pth file.

  wang2yn84

  committed
  8be6cfc

  Copy full SHA for 8be6cfc

• Add quantized layer for moe

  Show description for e6c5696
  

  qihqi

  authored and

  wang2yn84

  committed
  e6c5696

  Copy full SHA for e6c5696

• Add the huggingface download script. Improved the convert checkpoints logging.

  wang2yn84

  committed
  518f758

  Copy full SHA for 518f758

• Clean up and fix lint errors.

  wang2yn84

  committed
  147cef4

  Copy full SHA for 147cef4

• Missing cleanups.

  wang2yn84

  committed
  2ed3e0e

  Copy full SHA for 2ed3e0e

• Add instructions for Mixtral.

  wang2yn84

  committed
  14d6672

  Copy full SHA for 14d6672

• Renames everything from mistral to mixtral.

  wang2yn84

  committed
  b2a6a18

  Copy full SHA for b2a6a18

• Fix more lint errors.

  wang2yn84

  committed
  c004adc

  Copy full SHA for c004adc

• Removes the unnecessary checkpoint name mapping from the original Mixtral checkpoints.

  wang2yn84

  committed
  a206374

  Copy full SHA for a206374

Commits on Jun 11, 2024

• Fix the model calling arg sequence; Fix the checkpoint convert script.

  wang2yn84

  committed
  d18f8c3

  Copy full SHA for d18f8c3