Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Improve usage of the word language model example #1246

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions word_language_model/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
# Word-level Language Modeling using RNN and Transformer

## Requirements

Just running the following command to get started. Actually we just need the torch.

```bash
pip install -r requirements.txt
```

## Usage

This example trains a multi-layer RNN (Elman, GRU, or LSTM) or Transformer on a language modeling task. By default, the training script uses the Wikitext-2 dataset, provided.
The trained model can then be used by the generate script to generate new text.

Expand Down Expand Up @@ -54,3 +64,19 @@ python main.py --cuda --emsize 650 --nhid 650 --dropout 0.5 --epochs 40 --tied
python main.py --cuda --emsize 1500 --nhid 1500 --dropout 0.65 --epochs 40
python main.py --cuda --emsize 1500 --nhid 1500 --dropout 0.65 --epochs 40 --tied
```

To generate samples from the default model checkpoint, just use the the `generate.py` script, which accepts the following arguments:

```bash
optional arguments:
-h, --help show this help message and exit
--data DATA location of the data corpus
--checkpoint MODEL model checkpoint to use
--outf OUTPUT output file for generated text
--words WORDS number of words to generate
--seed SEED random seed
--cuda use CUDA
--mps enable GPU on macOS
--temperature TEMP temperature - higher will increase diversity
--log-interval N report interval
```