Code

SpinQuant#983

Code

Merged

HDCharles merged 18 commits intopytorch:mainpytorch/ao:mainfrom tobiasvanderwerff:spinquanttobiasvanderwerff/ao:spinquantCopy head branch name to clipboardOct 10, 2024

+99,838-1

Commits

Commits on Oct 10, 2024

• SpinQuant using R2 matrices

  tobiasvanderwerff

  committed
  5ff4352

  Copy full SHA for 5ff4352

• Move Hadamard functions and matrices to separate file

  tobiasvanderwerff

  committed
  c769882

  Copy full SHA for c769882

• Add R4 rotation

  tobiasvanderwerff

  committed
  bf3cbf1

  Copy full SHA for bf3cbf1

• Reformat

  tobiasvanderwerff

  committed
  9be7bf4

  Copy full SHA for 9be7bf4

• Do not wrap Linear layers but use nn.Sequential

  Show description for e0d6d40
  tobiasvanderwerff

  committed
  e0d6d40

  Copy full SHA for e0d6d40

• Add test

  tobiasvanderwerff

  committed
  a64aea0

  Copy full SHA for a64aea0

• Fix test and do small reformat of Hadamard code

  tobiasvanderwerff

  committed
  c9cde41

  Copy full SHA for c9cde41

• Fuse Layernorm params into linear layers

  Show description for ac238ec
  tobiasvanderwerff

  committed
  ac238ec

  Copy full SHA for ac238ec

• Add R1 rotation

  tobiasvanderwerff

  committed
  e6018a0

  Copy full SHA for e6018a0

• Add option to load pretrained R1/R2 matrices

  tobiasvanderwerff

  committed
  3f3888b

  Copy full SHA for 3f3888b

• Move Spinquant from torchao/quantization to torchao/prototype/spinquant

  tobiasvanderwerff

  committed
  1217764

  Copy full SHA for 1217764

• Move Hadamard matrices to a separate file

  tobiasvanderwerff

  committed
  58f269f

  Copy full SHA for 58f269f

• Move test

  tobiasvanderwerff

  committed
  21fd3d8

  Copy full SHA for 21fd3d8

• Minor changes

  tobiasvanderwerff

  committed
  eeadd6d

  Copy full SHA for eeadd6d

• Reformat

  tobiasvanderwerff

  committed
  20c72a3

  Copy full SHA for 20c72a3

• Only enable R4 as default setting

  Show description for 3dabc19
  tobiasvanderwerff

  committed
  3dabc19

  Copy full SHA for 3dabc19

• Add __init__.py to spinquant folder

  tobiasvanderwerff

  committed
  49f035d

  Copy full SHA for 49f035d

• Do not fail if fast_hadamard_transform is not present

  tobiasvanderwerff

  committed
  fb3882f

  Copy full SHA for fb3882f