Add evaluator weighted_alpaca_eval_gpt-4o-mini-2024-07-18

[tongyx361]

Evaluator weighted_alpaca_eval_gpt-4o-mini-2024-07-18

weighted_alpaca_eval_gpt-4o-mini-2024-07-18 is the same as weighted_alpaca_eval_gpt4_turbo except for using gpt-4o-mini-2024-07-18.

gpt-4o-mini-2024-07-18 is

• much cheaper than gpt-4-1106-preview,
• and even cheaper and much faster than Llama-3-70B, but weighted_alpaca_eval_gpt-4o-mini-2024-07-18 has comparable human correlation with Llama-3-70B.

Deficiencies in contribution instructions

I found a bug in calculating prices for new OpenAI APIs and fixed it for gpt-4o-mini-2024-07-18.

I suggest adding instructions about modifying src/alpaca_eval/decoders when contributing evaluators.

Besides, I am not sure if I should commit files under the following directories:

1. src/alpaca_eval/leaderboards/data_AlpacaEval_2
2. src/alpaca_eval/metrics/weights
3. results/${model}/weighted_alpaca_eval_gpt-4o-mini-2024-07-18
4. docs/data_AlpacaEval_2

For now, I keep the commit minimal. I would love to help improve the documentation if needed.

[YannDubs]

great thanks @tongyx361 !

If you evaluated some models using gpt-4o-mini it would be great to also push

src/alpaca_eval/leaderboards/data_AlpacaEval_2
src/alpaca_eval/metrics/weights
results/${model}/weighted_alpaca_eval_gpt-4o-mini-2024-07-18

So that others see the evaluations you ran! I'll merge as is for now given that is already useful.

[TingchenFu]

There seems to be a typo in src/alpaca_eval/leaderboards/evaluators/evaluators_leaderboard.csv. It reads the human agreement for gpt-4o is only 0.33