From 15749bfc10b97b26f8c0dbfa5c67c30ed046e92b Mon Sep 17 00:00:00 2001
From: BramVanroy <Bram.Vanroy@UGent.be>
Date: Fri, 27 Sep 2019 10:01:36 +0200
Subject: [PATCH] Add small  note about the output of hidden states

---
 README.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README.md b/README.md
index a111b5f943ac09..6accc600ff95a1 100644
--- a/README.md
+++ b/README.md
@@ -452,6 +452,10 @@ outputs = model(input_ids, labels=labels)
 loss, logits, attentions = outputs
 ```
 
+### Using hidden states
+
+By enabling the configuration option `output_hidden_states`, it was possible to retrieve the last hidden states of the encoder. In `pytorch-transformers` as well as `transformers` the return value has changed slightly: `all_hidden_states` now also includes the hidden state of the embeddings in addition to those of the encoding layers. This allows users to easily access the embeddings final state.
+
 ### Serialization
 
 Breaking change in the `from_pretrained()`method: