Skip to content

Commit

Permalink
Merge pull request #1346 from BramVanroy/documentation
Browse files Browse the repository at this point in the history
Add small  note about the output of hidden states (closes #1332)
  • Loading branch information
thomwolf authored Sep 27, 2019
2 parents da2e47a + 15749bf commit f6de000
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -452,6 +452,10 @@ outputs = model(input_ids, labels=labels)
loss, logits, attentions = outputs
```

### Using hidden states

By enabling the configuration option `output_hidden_states`, it was possible to retrieve the last hidden states of the encoder. In `pytorch-transformers` as well as `transformers` the return value has changed slightly: `all_hidden_states` now also includes the hidden state of the embeddings in addition to those of the encoding layers. This allows users to easily access the embeddings final state.

### Serialization

Breaking change in the `from_pretrained()`method:
Expand Down

0 comments on commit f6de000

Please sign in to comment.