Fix ignored reuse_existing in config file #431
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix ignored
reuse_existing
field in config file.Currently, there is a
reuse_existing
field in all example config files for evaluation in inference endpoints. See for example:lighteval/examples/model_configs/endpoint_model.yaml
Line 6 in 9315f0d
or
lighteval/docs/source/evaluate-the-model-on-a-server-or-container.mdx
Line 35 in 9315f0d
However, this field is ignored when parsing the config file because
should_reuse_existing
is parsed instead:lighteval/src/lighteval/main_endpoint.py
Line 234 in 9315f0d
This PR fixes this issue by renaming
should_reuse_existing
toreuse_existing
, so that it is aligned with theInferenceEndpointModel
parameter name.