Setting cache_dir for model downloading

[RonanKMcGovern]

How does one set a local directory to which the model is loaded?

By default, on runpod, the model seems to download to the container, so I need to redirect that to workspace. I overcame this by just downloading the repo first to my workspace and then loading the model.

Code snippet:

model = AutoAWQForCausalLM.from_pretrained(model_path_on_hf, safetensors=True, **{"low_cpu_mem_usage": True})

I don't see anywhere in the docs to specify the local directory for caching/download.

[casper-hansen]

I agree that this would be a good enhancement @RonanKMcGovern. Feel free to open a PR that adds download_kwargs to AutoAWQForCausalLM.from_pretrained and AutoAWQForCausalLM.from_quantized.

[casper-hansen]

Fixed!

[RonanKMcGovern]

nice, thanks