Code

Add Auto-Round support#581

Code

Merged

jerryzh168 merged 77 commits intopytorch:mainpytorch/ao:mainfrom yiliu30:auto_round_support-3yiliu30/torchao-fork:auto_round_support-3Copy head branch name to clipboardSep 4, 2024

+1,477-2

Commits

Commits on Jul 24, 2024

• initial flow for autoround

  Show description for be78a08
  yiliu30

  committed
  be78a08

  Copy full SHA for be78a08

Commits on Jul 25, 2024

• update flow

  Show description for 49f8075
  yiliu30

  committed
  49f8075

  Copy full SHA for 49f8075

Commits on Jul 26, 2024

• use int4 kernel

  Show description for 62834a2
  yiliu30

  committed
  62834a2

  Copy full SHA for 62834a2

• remove debug code

  Show description for 6433e75
  yiliu30

  committed
  6433e75

  Copy full SHA for 6433e75

Commits on Jul 29, 2024

• update the forward

  Show description for 65f46e5
  yiliu30

  committed
  65f46e5

  Copy full SHA for 65f46e5

• clean code

  Show description for 1e22c11
  yiliu30

  committed
  1e22c11

  Copy full SHA for 1e22c11

Commits on Jul 30, 2024

• e2e example

  Show description for b8d37b9
  yiliu30

  committed
  b8d37b9

  Copy full SHA for b8d37b9

• refine code

  Show description for 8d388fb
  yiliu30

  committed
  8d388fb

  Copy full SHA for 8d388fb

• add requirements for test

  Show description for 07a95a0
  yiliu30

  committed
  07a95a0

  Copy full SHA for 07a95a0

• update test

  Show description for 6baa62f
  yiliu30

  committed
  6baa62f

  Copy full SHA for 6baa62f

• update the readme

  Show description for 78a5067
  yiliu30

  committed
  78a5067

  Copy full SHA for 78a5067

• add readme

  Show description for 37e9f5f
  yiliu30

  committed
  37e9f5f

  Copy full SHA for 37e9f5f

• update the filenames

  Show description for 8bfe76a
  yiliu30

  committed
  8bfe76a

  Copy full SHA for 8bfe76a

• update the np version

  Show description for e25d6eb
  yiliu30

  committed
  e25d6eb

  Copy full SHA for e25d6eb

• add demo

  Show description for 16a901d
  yiliu30

  committed
  16a901d

  Copy full SHA for 16a901d

• format

  Show description for 5f16e8d
  yiliu30

  committed
  5f16e8d

  Copy full SHA for 5f16e8d

Commits on Jul 31, 2024

• add more docs

  Show description for f3442c5
  yiliu30

  committed
  f3442c5

  Copy full SHA for f3442c5

• format

  Show description for 432da79
  yiliu30

  committed
  432da79

  Copy full SHA for 432da79

• add doc

  Show description for 7ee9f9b
  yiliu30

  committed
  7ee9f9b

  Copy full SHA for 7ee9f9b

• use AffineQuantizedTensor

  Show description for e5ffcca
  yiliu30

  committed
  e5ffcca

  Copy full SHA for e5ffcca

Commits on Aug 8, 2024

• impl ar using multensors

  Show description for cec375b
  yiliu30

  committed
  cec375b

  Copy full SHA for cec375b

• clean code

  Show description for a8f5681
  yiliu30

  committed
  a8f5681

  Copy full SHA for a8f5681

Commits on Aug 12, 2024

• use hook + multensors

  Show description for ab08cb3
  yiliu30

  committed
  ab08cb3

  Copy full SHA for ab08cb3

• separate mul_tensors into a new file

  Show description for 5ee2e06
  yiliu30

  committed
  5ee2e06

  Copy full SHA for 5ee2e06

• fix typos

  Show description for a5a3544
  yiliu30

  committed
  a5a3544

  Copy full SHA for a5a3544

Commits on Aug 13, 2024

• rename mul_tensor to multi_tensor

  Show description for e406ee8
  yiliu30

  committed
  e406ee8

  Copy full SHA for e406ee8

• enable amp

  Show description for 7b6908e
  yiliu30

  committed
  7b6908e

  Copy full SHA for 7b6908e

• eval model

  Show description for 6a4d67c
  yiliu30

  committed
  6a4d67c

  Copy full SHA for 6a4d67c

• add gen examples

  Show description for c1fa230
  yiliu30

  committed
  c1fa230

  Copy full SHA for c1fa230

• merge with main

  Show description for 5eef0a6
  yiliu30

  committed
  5eef0a6

  Copy full SHA for 5eef0a6

• add warmup to benchmark

  Show description for e4cfa7d
  yiliu30

  committed
  e4cfa7d

  Copy full SHA for e4cfa7d

• add benchmark

  Show description for 41d9afd
  yiliu30

  committed
  41d9afd

  Copy full SHA for 41d9afd

• Merge branch 'main' into re-a3

  yiliu30

  committed
  e1cec58

  Copy full SHA for e1cec58

• Merge branch 'auto_round_support-3' of https://github.com/yiliu30/torchao-fork into auto_round_support-3

  yiliu30

  committed
  6f20e25

  Copy full SHA for 6f20e25

• Merge branch 'auto_round_support-3' into auto_round_support-3-bench

  yiliu30

  committed
  ee1510c

  Copy full SHA for ee1510c

• clean code

  Show description for ca5bb30
  yiliu30

  committed
  ca5bb30

  Copy full SHA for ca5bb30

• format code

  Show description for e01e028
  yiliu30

  committed
  e01e028

  Copy full SHA for e01e028

• Merge pull request #3 from yiliu30/auto_round_support-3-bench

  Show description for 8532af0
  yiliu30

  authored
  8532af0

  Copy full SHA for 8532af0

Commits on Aug 16, 2024

• use tiny kernel

  Show description for 5106fe0
  yiliu30

  committed
  5106fe0

  Copy full SHA for 5106fe0

• add more note

  Show description for b82b638
  yiliu30

  committed
  b82b638

  Copy full SHA for b82b638

• format

  Show description for bb08957
  yiliu30

  committed
  bb08957

  Copy full SHA for bb08957

• Merge pull request #4 from yiliu30/auto_round_support-3-tinygemm-kernel

  Show description for b5f08c5
  yiliu30

  authored
  b5f08c5

  Copy full SHA for b5f08c5

• correct typos

  Show description for c8fc3f6
  yiliu30

  committed
  c8fc3f6

  Copy full SHA for c8fc3f6

Commits on Aug 19, 2024

• remove hard code

  Show description for 7ee493f
  yiliu30

  committed
  7ee493f

  Copy full SHA for 7ee493f

• use intx

  Show description for 1f75897
  yiliu30

  committed
  1f75897

  Copy full SHA for 1f75897

• Merge pull request #6 from yiliu30/auto_round_support-3-intx

  Show description for 48d0903
  yiliu30

  authored
  48d0903

  Copy full SHA for 48d0903

Commits on Aug 20, 2024

• enable offload for multitensor

  Show description for 34e6b49
  yiliu30

  committed
  34e6b49

  Copy full SHA for 34e6b49

• update the default config

  Show description for eeca10b
  yiliu30

  committed
  eeca10b

  Copy full SHA for eeca10b

Commits on Aug 21, 2024

• refine note

  Show description for 2b94608
  yiliu30

  committed
  2b94608

  Copy full SHA for 2b94608

• Merge pull request #8 from yiliu30/enable-llama3

  Show description for 0d38b20
  yiliu30

  authored
  0d38b20

  Copy full SHA for 0d38b20

• Merge branch 'main' into auto_round_support-3

  yiliu30

  committed
  f04b594

  Copy full SHA for f04b594

• update the version check

  Show description for 0e0b06d
  yiliu30

  committed
  0e0b06d

  Copy full SHA for 0e0b06d

Commits on Aug 22, 2024

• format

  Show description for d0a4920
  yiliu30

  committed
  d0a4920

  Copy full SHA for d0a4920

• update

  Show description for 1e8a081
  yiliu30

  committed
  1e8a081

  Copy full SHA for 1e8a081

• add ut

  Show description for 5b3374f
  yiliu30

  committed
  5b3374f

  Copy full SHA for 5b3374f

• format

  Show description for 4ef0cdc
  yiliu30

  committed
  4ef0cdc

  Copy full SHA for 4ef0cdc

• add scripts

  Show description for 5f78c73
  yiliu30

  committed
  5f78c73

  Copy full SHA for 5f78c73

• format code

  Show description for 5baae13
  yiliu30

  committed
  5baae13

  Copy full SHA for 5baae13

• format

  Show description for 6feb975
  yiliu30

  committed
  6feb975

  Copy full SHA for 6feb975

• Merge pull request #9 from yiliu30/auto_round_support-3-unified-api

  Show description for f6ed1e0
  yiliu30

  authored
  f6ed1e0

  Copy full SHA for f6ed1e0

• update

  Show description for 03cd9fc
  yiliu30

  committed
  03cd9fc

  Copy full SHA for 03cd9fc

• fix typo

  Show description for e60b815
  yiliu30

  committed
  e60b815

  Copy full SHA for e60b815

• refine bench code

  Show description for b20e6d9
  yiliu30

  committed
  b20e6d9

  Copy full SHA for b20e6d9

Commits on Aug 25, 2024

• Merge branch 'main' into auto_round_support-3

  yiliu30

  committed
  fabe8d2

  Copy full SHA for fabe8d2

Commits on Aug 26, 2024

• Enable use_optimized_layer_output and AO' llama (#12)

  Show description for 9ae5392
  yiliu30

  authored
  9ae5392

  Copy full SHA for 9ae5392

• Refine the Doc (#14)

  Show description for 157c189
  yiliu30

  authored
  157c189

  Copy full SHA for 157c189

• add more docstring

  Show description for 2df3f5f
  yiliu30

  committed
  2df3f5f

  Copy full SHA for 2df3f5f

• add paper link

  Show description for d719460
  yiliu30

  committed
  d719460

  Copy full SHA for d719460

• correct some note

  Show description for d7ba39e
  yiliu30

  committed
  d7ba39e

  Copy full SHA for d7ba39e

Commits on Aug 27, 2024

• add cmd

  Show description for a2c6b28
  yiliu30

  committed
  a2c6b28

  Copy full SHA for a2c6b28

Commits on Aug 28, 2024

• resolve conflicts

  Show description for 896d87f
  yiliu30

  committed
  896d87f

  Copy full SHA for 896d87f

• udpdate the scripts

  Show description for 6a8e073
  yiliu30

  committed
  6a8e073

  Copy full SHA for 6a8e073

• revert some change

  Show description for 9e48d1a
  yiliu30

  committed
  9e48d1a

  Copy full SHA for 9e48d1a

Commits on Aug 29, 2024

• Add a lightweight configuration for quick benchmarking (#15)

  Show description for 5ca125e
  yiliu30

  authored
  5ca125e

  Copy full SHA for 5ca125e

Commits on Aug 30, 2024

• merge with main

  Show description for b6d95ce
  yiliu30

  committed
  b6d95ce

  Copy full SHA for b6d95ce

• update quant method name

  Show description for 21686f1
  yiliu30

  committed
  21686f1

  Copy full SHA for 21686f1

Commits on Sep 3, 2024

• Wrap model's buffers and params to MultiTensor & update the results (#16)

  Show description for 96f745d
  yiliu30

  authored
  96f745d

  Copy full SHA for 96f745d