Skip to content

Commit

Permalink
fixing bug in GPTQ (pytorch#120)
Browse files Browse the repository at this point in the history
* fixing bug in GPTQ

Summary: shape was always padded even when not needed.

Test Plan: pythont test/quantization/test_quant_api.py -k
"test_gptq_quantizer_int4wo"

Reviewers:

Subscribers:

Tasks:

Tags:

* removing extra spaces

Summary:

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:
  • Loading branch information
HDCharles authored Apr 4, 2024
1 parent 12f1080 commit ac76174
Showing 1 changed file with 4 additions and 1 deletion.
5 changes: 4 additions & 1 deletion torchao/quantization/GPTQ.py
Original file line number Diff line number Diff line change
Expand Up @@ -950,7 +950,10 @@ def __init__(
# TODO: this is the gpt-fast version, merge with the main version later
def make_names_and_values_dict_func(q, qparams):
k = q.shape[1]
new_k = find_multiple(k, 1024)
if not _check_linear_int4_k(k, groupsize):
new_k = find_multiple(k, 1024)
else:
new_k = k
# how much we need to pad the weight
delta_k = new_k - q.shape[1]
q = q.to(torch.int32)
Expand Down

0 comments on commit ac76174

Please sign in to comment.