Skip to content

Commit

Permalink
add intro to new embodiments
Browse files Browse the repository at this point in the history
  • Loading branch information
budzianowski committed Sep 8, 2024
1 parent aa97ddf commit 474468e
Showing 1 changed file with 39 additions and 8 deletions.
47 changes: 39 additions & 8 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,6 @@ an issue or you want to add an enhancement.
When creating a new pull request please add the issue number.

## Adding new robot
Adding new embodiment is very straightforward:
1. Create a folder with a new robot [here](https://github.com/kscalelabs/sim/tree/master/sim).
2. Add joint.py file setting up basic properties and join configuration - see an [example](https://github.com/kscalelabs/sim/blob/master/sim/resources/stompymini/joints.py).
3. Add the new embodiment configuration and environment [here](https://github.com/kscalelabs/sim/tree/master/sim/envs).
4. Add the new embodiment to the [registry](https://github.com/kscalelabs/sim/blob/master/sim/envs/__init__.py).

We set up the logic so that your new robot should start to walk with basic configuration.

## Lint and formatting
When you submit the PR we automatically run some checks using black, ruff and mypy.
Expand All @@ -26,3 +18,42 @@ You can locally run commands below to test the formatting:
make format
make static-checks
```

## Adding a new robot
Creating a new embodiment is quite straightforward. The best way is to copy an existing robot and modify it.
The best starting point would be [stompymini](https://github.com/kscalelabs/sim/tree/master/sim/resources/stompymini).
1. Create a folder with a new robot [here](https://github.com/kscalelabs/sim/tree/master/sim/resources).
2. Add joint.py file setting up basic properties and joint configuration - see an [example](https://github.com/kscalelabs/sim/blob/master/sim/resources/stompymini/joints.py).
3. Update [height](https://github.com/kscalelabs/sim/blob/aa97ddfddcaadcac64d4f83d986548cc7fc451a6/sim/resources/stompymini/joints.py#L102) and default rotation of your humanoid.
4. Add a new embodiment configuration and environment [here](https://github.com/kscalelabs/sim/tree/master/sim/envs/humanoids).
5. Add a new embodiment to the [registry](https://github.com/kscalelabs/sim/blob/master/sim/envs/__init__.py).
To run things export your default path
```
export MODEL_DIR=sim/resources/NEW_HUMANOID
```
and kick off the training
```
python sim/train.py --env-id NEW_HUMANOID --num-envs 4
```

## Making it stand
1. The best way to start is to make your new robot stand. In this case you want to comment out
the rewards that are responsible for making the robot walk. See an example of a [reward config](https://github.com/kscalelabs/sim/blob/aa97ddfddcaadcac64d4f83d986548cc7fc451a6/sim/envs/humanoids/stompymini_config.py#L163).
2. If the robot flies away, inspect your joint limits. During training, we introduce a lot of noise to the default joint positions
as well as masses. You might need either adapt the limits or the [noise level](https://github.com/kscalelabs/sim/blob/aa97ddfddcaadcac64d4f83d986548cc7fc451a6/sim/envs/humanoids/stompy_config.py#L213).
3. Isaac Sim often hits velocities nans when some joints are hitting their limits - you can change the [engine parameters](https://github.com/kscalelabs/sim/blob/aa97ddfddcaadcac64d4f83d986548cc7fc451a6/sim/envs/humanoids/stompy_config.py#L108) to fix this issue.
4. The revolute joints cannot have 0 velocity in the URDF definition - otherwise, engine will go nans as well.
5. Observe the reward for orientation and default joint position. The model should just farm these two rewards.
6. If the robot still struggles at keeping the standing pose, you can also change the urdf definition using the (script)[https://github.com/kscalelabs/sim/blob/master/sim/scripts/create_fixed_torso.py] to fix the upper body or change joint limits.

## Making it walk
We set up the logic so that your new robot should start to walk with basic configuration after some modifications.
1. To get things going it's best to start from the good standing pose, with knees slightly bent.
See an example [here](https://github.com/kscalelabs/sim/blob/aa97ddfddcaadcac64d4f83d986548cc7fc451a6/sim/resources/stompymini/joints.py#L113)
2. The [gait reward](https://github.com/kscalelabs/sim/blob/aa97ddfddcaadcac64d4f83d986548cc7fc451a6/sim/envs/humanoids/stompy_config.py#L146) is the most crucial single reward. You have to adapt the hyperparameters to your robot design to get it to work.
3. If the robots tends to jump, use only one limb you will have to adapt the overall rewards schema to enforce proper behavior.
4. If the setup is correct, you should see positive results after first 200-300 epochs. However, the robust policy will requires 2000-3000 epochs.




0 comments on commit 474468e

Please sign in to comment.