assistant prefill

[baberabb]

add an assistant_prefill to the chat prompt.

This makes it so the assistant's responses include certain content at the start after the <|assistant|> token

added arc_challenge_chat and mmlu_llama (both chat) from llama evals

[baberabb]

with apply_chat_template and fewshot_as_multiturn

vllm (pretrained=meta-llama/Llama-3.1-8B-Instruct,enable_prefix_caching=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto

Tasks	Version	Filter	n-shot	Metric		Value		Stderr
arc_challenge_chat	1	remove_whitespace	0	exact_match	↑	0.8157	±	0.0113

vllm (pretrained=meta-llama/Llama-3.1-8B-Instruct,enable_prefix_caching=True), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: auto

Groups	Version	Filter	n-shot	Metric		Value		Stderr
mmlu_llama	1	strict_match		exact_match	↑	0.6585	±	0.0038

There appears to be a discrepancy between our evaluation results and those reported in the Hugging Face eval repo for meta-llama/Llama-3.1-8B-Instruct model. However prompts are exactly the same