Skip to content

Commit

Permalink
Merge branch 'master' of github.com:HesitantlyHuman/autoclip
Browse files Browse the repository at this point in the history
  • Loading branch information
HesitantlyHuman committed Jul 16, 2022
2 parents 13ba540 + c3c209e commit 9dee079
Showing 1 changed file with 8 additions and 3 deletions.
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,12 @@ for batch_num, batch in enumerate(training_dataset):
`autoclip`'s torch clippers support two clipping modes. The first is `global_clipping`, which is the original AutoClip as described in Seetherman et al. The second is local or parameter-wise clipping. In this mode a history is kept for every parameter, and each is clipped according to its own history. By default, the `autoclip` clippers will use the parameter-wise clipping.
To use the global mode, simply pass the appropriate flag:
```python
clipper = QuantileClip(model.parameters(), quantile=0.9, history_length=1000, global_clipping=True)
clipper = QuantileClip(
model.parameters(),
quantile=0.9,
history_length=1000,
global_clipping=True
)
```

### Checkpointing
Expand All @@ -84,7 +89,7 @@ clipper.load_state_dict(torch.load('clipper.pth'))
Keep in mind that just like a torch optimizer this will error if you give the clipper differently sized model parameters.

## Tensorflow
`autoclip`'s tensorflow API does not currently have feature parity with the torch API (If you want to change this, feel free to [contribute](#2)).
`autoclip`'s tensorflow API does not currently have feature parity with the torch API (If you want to change this, feel free to [contribute](https://github.com/HesitantlyHuman/autoclip/issues/2)).
As it is, the tensorflow API currently only supports the original AutoClip algorithm, and does not support checkpointing. Below is a short example:
```python
import tensorflow as tf
Expand Down Expand Up @@ -116,4 +121,4 @@ model.compile(
metrics=["accuracy"],
)
model.fit(train_data, train_targets)
```
```

0 comments on commit 9dee079

Please sign in to comment.