Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix issues with loading and saving benchmark's config files #39

Merged
merged 2 commits into from
Feb 12, 2021
Merged

Fix issues with loading and saving benchmark's config files #39

merged 2 commits into from
Feb 12, 2021

Conversation

ndangtt
Copy link
Collaborator

@ndangtt ndangtt commented Feb 11, 2021

Hello,

There are two issues I encountered when loading and saving .json config files;

  • Issue 1: AbstractBenchmark.read_config_file doesn't handle the case when observation_space_type is None. To reproduce the issue:
bench = CMAESBenchmark()
bench.save_config("test-config.json")
bench.read_config_file("test-config.json")
  • Issue 2: AbstractBenchmark.save_config doesn't handle 1d numpy array. To reproduce the issue:
bench = LubyBenchmark("dacbench/additional_configs/luby/luby_hard.json")
bench.save_config("test-config.json")

This pull request is to fix those issues. Could you please have a look?

Thanks!
Nguyen

@TheEimer TheEimer self-requested a review February 12, 2021 10:13
typestring = typestring.split(".")[1]
self.config["observation_space_type"] = getattr(np, typestring)
if self.config["observation_space_type"] == "None":
self.config["observation_space_type"] = None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you maybe tell me why you'd want to do that? Normally this is not desired behaviour, envs should always have an observation space. Is there maybe another way around this?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Theresa,

This is the observation space type, not the space itself I think. In fact it was specified as None in the current CMAES default configuration because the space is a dictionary and doesn't have a unique type for all elements.

The above change is to avoid error when we save a config to .json file and then load it again from file, as in my example code in Issue 1.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, sorry, that's a bug and I misread what you did there. Thanks for noticing and fixing, the logging system still needs some work as you can see.
I'll merge both requests in that case :D

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No problemo! :D Thanks for reviewing and merging it so quickly!

Nguyen

@TheEimer TheEimer merged commit ea65303 into automl:main Feb 12, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants