Plotting more metrics for PPO

[arvganesh]

• I have marked all applicable categories:
  • exception-raising bug
  • RL algorithm bug
  • documentation request (i.e. "X is missing from the documentation.")
  • new feature request
• I have visited the source website
• I have searched through the issue tracker for duplicates
• I have mentioned version numbers, operating system and environment, where applicable:

Based on this:

I wanted to have more metrics plotted for PPO. Specifically, these:

It would be helpful to have clip_fraction, explained_variance, and approx_kl also logged for PPO agents. Thanks!

| train/                  |             |
|    approx_kl            | 0.037781604 |
|    clip_fraction        | 0.243       |
|    clip_range           | 0.2         |
|    entropy_loss         | -1.06       |
|    explained_variance   | 0.999       |
|    learning_rate        | 0.001       |
|    loss                 | 0.245       |
|    n_updates            | 180         |
|    policy_gradient_loss | -0.00398    |
|    std                  | 0.205       |
|    value_loss           | 0.226       |

[Trinkle23897]

Are you interested in submitting a PR to support this?

[arvganesh]

Sure, will work on it this week.

[MischaPanch]

@arvganesh I'm going through older issues from time to time and now noticed this one. It's related to
#933 and #895, both of which are good issues for a first contribution to tianshou. Are you still interested in preparing a PR?