fix the split placement example #281
Conversation
|
|
||
| actor_output = actor_output.get() | ||
| actor_output_metrics = reduce_metrics(actor_output.meta_info['metrics']) | ||
| metrics.update(actor_output_metrics) |
There was a problem hiding this comment.
Modified to get futures of critic_output and actor_output
There was a problem hiding this comment.
Nice job! What do you modify here?
There was a problem hiding this comment.
Modified to get futures of critic_output and actor_output
The comments here are just to highlight the changes compared to RayTrainer.fit, because most of the code is copied from RayTrainer.fit.
These changes are copied from examples/split_placement/split_monkey_patch.py, because the update_actor and update_critic would be non-blocking as described in https://github.com/volcengine/verl/tree/main/examples/split_placement#step-2-make-the-models-executed-asynchronously. I didn't add any new things here, I just copied bits from both RayTrainer.fit and examples/split_placement/split_monkey_patch.py to make it consistent with the original logic.
The split placement example is outdated, I tried it and encountered some errors. To address this, the following changes were made in this PR 1. Copied the content from `verl/trainer/config/ppo_trainer.yaml` to `examples/split_placement/config/ppo_trainer_split.yaml` 2. Copied `RayPPOTrainer.fit` method into the `fit` func in `examples/split_placement/split_monkey_patch.py` and modified it to get the futures of `critic_output` and `actor_output`
The split placement example is outdated, I tried it and encountered some errors. To address this, the following changes were made in this PR
verl/trainer/config/ppo_trainer.yamltoexamples/split_placement/config/ppo_trainer_split.yamlRayPPOTrainer.fitmethod into thefitfunc inexamples/split_placement/split_monkey_patch.pyand modified it to get the futures ofcritic_outputandactor_output