Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Improvement: apply use_precomputed_responses for prompt-path requirement
Summary: Currently, when we enable the flag `--use-precomputed-responses`, such as: ``` python3 -m CybersecurityBenchmarks.benchmark.run \ -- --benchmark=instruct \ --use-precomputed-responses \ --llm-under-test="TOGETHER::togethercomputer/llama-2-7b::c7fff45932512a831c852cacf9c3ad5026565de5f8b8383fd080f2d54481f61a" \ --response-path=security/genai/CybersecurityBenchmarks/datasets/internal/demo/instruct/instruct_llama2-7b-chat_response_precomputed.json \ --num-test-cases=1 ``` We will received the error: ``` error: the following arguments are required: --prompt-path ``` This is not correct since when this flag is enabled, there is no need to use `--prompt-path`. --- arg `--llm-under-test` has the similar issue. Once this one gets approved, I'll try to fix the argument with the similar design. Reviewed By: YueLi28 Differential Revision: D52707709 fbshipit-source-id: 3d1cb89c8e1026ad072b6c1977b1a01d4b9cdcb2
- Loading branch information