update tdmpc into kscale sim module. #72
Conversation
|
Do you have some results of standing? |
I started training the stompypro only yesterday and its probably doing 100 iterations per day (takes ever so long to train). I have standing results of dora taken at 200 iterations (25M timesteps). I have results of stompy pro learning to stand a bit but its at the very early stages of training. |
|
I see! What is the most time consuming step in the pipeline? |
|
horizon update. This line makes the training process so slow. |
|
Horizon is only 5 and mlp is tiny. I don't understand why would it be that slow? |
|
The planning step that collects samples from interactions with the environment (MPC) is pretty slow. The number of iterations that we set for humanoid type task is around ~10-12. |
There was a problem hiding this comment.
see Makefile for linting and semi building. See https://github.com/kscalelabs/sim/blob/master/CONTRIBUTING.md
sim/train_tdmpc.py
Outdated
| if tdmpc_cfg.save_model and episode_idx % tdmpc_cfg.eval_freq_episode == 0: | ||
| L.save(agent, f"tdmpc_policy_{int(step // tdmpc_cfg.episode_length) + 1}.pt") | ||
| buffer.save(str(work_dir / "buffer.pt")) | ||
| # # common_metrics['episode_reward'] = evaluate(env, agent, h1 if L.video is not None else None, tdmpc_cfg.eval_episodes, step, env_step, L.video, tdmpc_cfg.action_repeat) |
No description provided.