Code

Add Quantization code.#107

Code

Closed

arnocandel wants to merge 19 commits intomainh2oai/h2ogpt:mainfrom quantizationh2oai/h2ogpt:quantizationCopy head branch name to clipboard

+69-4

Commits

Commits on May 2, 2023

• Add Quantization code.

  Show description for 48e2530
  arnocandel

  committed
  48e2530

  Copy full SHA for 48e2530

Commits on May 3, 2023

• Add quantization option to generate.

  arnocandel

  committed
  fdb488c

  Copy full SHA for fdb488c

• Export env var.

  arnocandel

  committed
  05eddf7

  Copy full SHA for 05eddf7

• Add failing command back.

  arnocandel

  committed
  da07ec0

  Copy full SHA for da07ec0

• Use groupsize 128

  arnocandel

  committed
  de71947

  Copy full SHA for de71947

• Fix import.

  arnocandel

  committed
  ebfcd44

  Copy full SHA for ebfcd44

• Merge remote-tracking branch 'origin/main' into quantization

  arnocandel

  committed
  f0534e3

  Copy full SHA for f0534e3

• Fix merge.

  arnocandel

  committed
  60f308a

  Copy full SHA for 60f308a

• Cleanup

  arnocandel

  committed
  32db201

  Copy full SHA for 32db201

• Set quant_model to '' for scoring model.

  arnocandel

  committed
  af251f9

  Copy full SHA for af251f9

Commits on May 4, 2023

• Use AutoGPTQ instead.

  arnocandel

  committed
  a6bd68f

  Copy full SHA for a6bd68f

Commits on May 5, 2023

• Working example for 4-bit on GPU.

  arnocandel

  committed
  e1e3424

  Copy full SHA for e1e3424

Commits on May 16, 2023

• Merge remote-tracking branch 'origin/main' into quantization

  arnocandel

  committed
  371940e

  Copy full SHA for 371940e

• WIP

  arnocandel

  committed
  d63a183

  Copy full SHA for d63a183

Commits on Jun 21, 2023

• Merge remote-tracking branch 'origin/main' into quantization

  arnocandel

  committed
  e073d34

  Copy full SHA for e073d34

• Fix merge conflict.

  arnocandel

  committed
  0f89d04

  Copy full SHA for 0f89d04

• Fix typo

  arnocandel

  committed
  d0ca005

  Copy full SHA for d0ca005

• Use latest AutoGPTQ

  arnocandel

  committed
  1c5bb39

  Copy full SHA for 1c5bb39

• Pass trust_remote_code=True.

  arnocandel

  committed
  84c4989

  Copy full SHA for 84c4989