Skip to content

Commit

Permalink
update readme
Browse files Browse the repository at this point in the history
  • Loading branch information
iankur committed Sep 1, 2024
1 parent 5ef7ad9 commit f03f65a
Showing 1 changed file with 6 additions and 6 deletions.
12 changes: 6 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,17 +29,17 @@ huggingface-cli login
wandb login
```

Verify setup by calling
```
from vqllm.models import llama3_8b
```

### Eval
Download model. Set checkpoint path and tokenizer path in config file. One config file is provided in `vqllm/config/custom_eval_config.yaml`.
Download model. Set checkpoint path and tokenizer path in config file. One config file is provided in `vqllm/config/custom_eval_config.yaml`. For example, llama3-8b inference can be run as follows:
```
tune download meta-llama/Meta-Llama-3-8B --output-dir recipes/ckpts/llama3_8b
tune run recipes/eleuther_eval.py --config recipes/config/eleuther_evaluation.yaml
```
Once the required models have been downloaded, all experiments can be launched using these bash scripts.
```
bash recipes/run_vq_size_ablation.sh
bash recipes/run_vq_type_ablation.sh
```

### Notes
- EMA embedding sum and cluster size parameters are kept in full precision. However, rest of the model can be in lower precision. So, `model.to(new_dtype)` should be handled carefully.
Expand Down

0 comments on commit f03f65a

Please sign in to comment.