Changing _save_summary_info to an ExpArgs static method #324
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Based on your review schedule, I'll hold off on reviewing this PR until it's marked as ready for review. If you'd like me to take a look now, comment
|
![korbit-ai[bot]](https://avatars.githubusercontent.com/in/322216?s=60&v=4){kind=small}
There was a problem hiding this comment.
I've completed my review and didn't find any issues... but I did find this kitten.
|\__/,| (`\
_.|o o |_ ) )
-(((---(((--------Files scanned
| File Path | Reviewed |
|---|---|
| browsergym/experiments/src/browsergym/experiments/loop.py | ✅ |
Explore our documentation to understand the languages and file types we support and the files we ignore.
Need a new review? Comment
/korbit-reviewon this PR and I'll review your latest changes.Korbit Guide: Usage and Customization
Interacting with Korbit
- You can manually ask Korbit to review your PR using the
/korbit-reviewcommand in a comment at the root of your PR.- You can ask Korbit to generate a new PR description using the
/korbit-generate-pr-descriptioncommand in any comment on your PR.- Too many Korbit comments? I can resolve all my comment threads if you use the
/korbit-resolvecommand in any comment on your PR.- Chat with Korbit on issues we post by tagging @korbit-ai in your reply.
- Help train Korbit to improve your reviews by giving a 👍 or 👎 on the comments Korbit posts.
Customizing Korbit
- Check out our docs on how you can make Korbit work best for you and your team.
- Customize Korbit for your organization through the Korbit Console.
Current Korbit Configuration
General Settings
Setting Value Review Schedule Automatic excluding drafts Max Issue Count 10 Automatic PR Descriptions ✅ Issue Categories
Category Enabled Documentation ✅ Logging ✅ Error Handling ✅ Readability ✅ Design ✅ Performance ✅ Security ✅ Functionality ✅ Feedback and Support
Note
Korbit Pro is free for open source projects 🎉
Looking to add Korbit to your team? Get started with a free 2 week trial here
TLSDC
commented
Mar 3, 2025
Comment on lines
+127
to
+284
|
|
||
| self.task_info = env_info.get("task_info", None) | ||
|
|
||
| self.raw_reward = env_info.get("RAW_REWARD_GLOBAL", None) | ||
|
|
||
| t.action_exec_start = env_info["action_exec_start"] # start | ||
| t.action_exect_after_timeout = env_info["action_exec_stop"] | ||
| t.action_exec_stop = env_info["action_exec_stop"] - env_info["action_exec_timeout"] | ||
|
|
||
| if obs_preprocessor: | ||
| self.obs = obs_preprocessor(self.obs) | ||
|
|
||
| def from_action(self, agent: Agent): | ||
| self.profiling.agent_start = time.time() | ||
| self.action, self.agent_info = agent.get_action(self.obs.copy()) | ||
| self.profiling.agent_stop = time.time() | ||
|
|
||
| self.make_stats() | ||
|
|
||
| return self.action | ||
|
|
||
| def from_reset(self, env: gym.Env, seed: int, obs_preprocessor: callable): | ||
| t = self.profiling | ||
| t.env_start = time.time() | ||
| self.obs, env_info = env.reset(seed=seed) | ||
| self.reward, self.terminated, self.truncated = 0, False, False | ||
| t.env_stop = time.time() | ||
|
|
||
| t.action_exec_start = env_info.get("recording_start_time", t.env_start) | ||
| t.action_exect_after_timeout = t.env_stop | ||
| t.action_exec_stop = t.env_stop | ||
|
|
||
| if obs_preprocessor: | ||
| self.obs = obs_preprocessor(self.obs) | ||
|
|
||
| @property | ||
| def is_done(self): | ||
| return self.terminated or self.truncated | ||
|
|
||
| def make_stats(self): | ||
|
|
||
| stats = { | ||
| f"n_token_{key}": count_tokens(val) | ||
| for key, val in self.obs.items() | ||
| if isinstance(val, str) | ||
| } | ||
| stats.update(self.agent_info.pop("stats", {})) | ||
|
|
||
| messages = self.agent_info.get("chat_messages", None) | ||
| if messages is not None: | ||
| stats["n_token_agent_messages"] = count_messages_token(messages) | ||
|
|
||
| t = self.profiling | ||
| stats["step_elapsed"] = t.env_stop - t.env_start | ||
| stats["agent_elapsed"] = t.agent_stop - t.agent_start | ||
|
|
||
| self.stats = stats | ||
|
|
||
| def save_step_info(self, exp_dir, save_json=False, save_screenshot=True, save_som=False): | ||
|
|
||
| # special treatment for some of the observation fields | ||
| if self.obs is not None: | ||
| # save screenshots to separate files | ||
| screenshot = self.obs.pop("screenshot", None) | ||
| screenshot_som = self.obs.pop("screenshot_som", None) | ||
|
|
||
| if save_screenshot and screenshot is not None: | ||
| img = Image.fromarray(screenshot) | ||
| img.save(exp_dir / f"screenshot_step_{self.step}.png") | ||
|
|
||
| if save_som and screenshot_som is not None: | ||
| img = Image.fromarray(screenshot_som) | ||
| img.save(exp_dir / f"screenshot_som_step_{self.step}.png") | ||
|
|
||
| # save goal object (which might contain images) to a separate file to save space | ||
| if self.obs.get("goal_object", False): | ||
| # save the goal object only once (goal should never change once setup) | ||
| goal_object_file = Path(exp_dir) / "goal_object.pkl.gz" | ||
| if not goal_object_file.exists(): | ||
| with gzip.open(goal_object_file, "wb") as f: | ||
| pickle.dump(self.obs["goal_object"], f) | ||
| # set goal_object to a special placeholder value, which indicates it should be loaded from a separate file | ||
| self.obs["goal_object"] = None | ||
|
|
||
| with gzip.open(exp_dir / f"step_{self.step}.pkl.gz", "wb") as f: | ||
| pickle.dump(self, f) | ||
|
|
||
| if save_json: | ||
| with open(exp_dir / "steps_info.json", "w") as f: | ||
| json.dump(self, f, indent=4, cls=DataclassJSONEncoder) | ||
|
|
||
| if self.obs is not None: | ||
| # add the screenshots back to the obs | ||
| # why do we need this? | ||
| if screenshot is not None: | ||
| self.obs["screenshot"] = screenshot | ||
| if screenshot_som is not None: | ||
| self.obs["screenshot_som"] = screenshot_som | ||
|
|
||
|
|
There was a problem hiding this comment.
just moving this upwards to fix typing
|
Thank you so much, this will allow me to use custom save_summary_info now |
jardinetsouffleton
approved these changes
Mar 3, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This way one can subclass ExpArgs and customize the change summary method
Description by Korbit AI
What change is being made?
Convert
_save_summary_infointo asave_summary_infostatic method of theExpArgsclass and refactor the logic to integrate theStepInfomanagement into this method.Why are these changes being made?
This refactor centralizes the summary-saving logic within the
ExpArgsclass, improving code organization and making theExpArgsclass more self-contained regarding experiment data management. This change enhances maintainability by encapsulating related functionalities within the class and eliminating external function dependencies.