Text_Generation_using_RNN.ipynb

{
  "cells": [
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "ZS7epWFStU83"
      },
      "outputs": [],
      "source": [
        "#importing libraries\n",
        "import tensorflow as tf\n",
        "\n",
        "import numpy as np\n",
        "import os\n",
        "import time"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The tf.keras.utils.get_file function downloads a file from the internet and stores it in the cache directory. In this case, the function is used to download the 'shakespeare.txt' file from the specified URL. The function returns the path to the downloaded file, which is assigned to the variable path_to_file."
      ],
      "metadata": {
        "id": "ceWAz46xNlap"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "uhrhIbFPtY51",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6441c751-1a2f-4f17-b026-56437714bf11"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt\n",
            "1115394/1115394 [==============================] - 0s 0us/step\n"
          ]
        }
      ],
      "source": [
        "path_to_file = tf.keras.utils.get_file('shakespeare.txt', 'https://storage.googleapis.com/download.tensorflow.org/data/shakespeare.txt')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This Python code reads the text from the file located at the path_to_file and decodes it using the 'utf-8' encoding to ensure compatibility with Python 2"
      ],
      "metadata": {
        "id": "rXPitK0INy8e"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Zx1-xD30tbF_",
        "outputId": "b83400de-acd9-4e09-c1b5-8441734b8397"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Length of text: 1115394 characters\n"
          ]
        }
      ],
      "source": [
        "# Read, then decode for py2 compat.\n",
        "text = open(path_to_file, 'rb').read().decode(encoding='utf-8')\n",
        "# length of text is the number of characters in it\n",
        "print(f'Length of text: {len(text)} characters')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "i1ILrJo-tcyk",
        "outputId": "7c17a104-6a48-4514-8790-4d36d5500e3c"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "First Citizen:\n",
            "Before we proceed any further, hear me speak.\n",
            "\n",
            "All:\n",
            "Speak, speak.\n",
            "\n",
            "First Citizen:\n",
            "You are all resolved rather to die than to famish?\n",
            "\n",
            "All:\n",
            "Resolved. resolved.\n",
            "\n",
            "First Citizen:\n",
            "First, you\n"
          ]
        }
      ],
      "source": [
        "# Take a look at the first 200 characters in text\n",
        "print(text[:200])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WCD3Edr-te5E",
        "outputId": "5710e879-9146-452f-c640-8a56299b5352"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "65  unique characters\n"
          ]
        }
      ],
      "source": [
        "# The unique characters in the file\n",
        "vocab = sorted(set(text))\n",
        "print(len(vocab), ' unique characters')"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Using TensorFlow to split the example texts, 'abcdefg' and 'xyz', into individual characters\n",
        "It is necessary to preprocess the textual data into a format that the model can understand and process effectively. Text data needs to be converted into a numerical format, typically as sequences of numbers or tokens, before it can be fed into a deep learning model."
      ],
      "metadata": {
        "id": "9HmOIREsOFCf"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "84lw0v-hzD5z",
        "outputId": "fb30eecf-2db6-4f90-b785-1392cc5e0134"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[b'a', b'b', b'c', b'd', b'e', b'f', b'g'], [b'x', b'y', b'z']]>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ],
      "source": [
        "example_texts = ['abcdefg', 'xyz']\n",
        "\n",
        "chars = tf.strings.unicode_split(example_texts, input_encoding='UTF-8')\n",
        "chars"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now we need to create a TensorFlow string lookup layer (StringLookup) that maps characters from a given vocabulary list to unique integer indices. This facilitates the conversion of textual data into numerical representations"
      ],
      "metadata": {
        "id": "pvuIZthOOeav"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "bNFO7FWJuqOX"
      },
      "outputs": [],
      "source": [
        "ids_from_chars = tf.keras.layers.StringLookup(\n",
        "    vocabulary=list(vocab), mask_token=None)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We need to convert numbers back to the original words or characters.\n",
        "\n",
        "Remember how we converted words to numbers earlier?\n",
        "\n",
        "This tool does the reverse, making it easier to understand what the model is predicting. We use the same vocabulary as before and tell it to do the opposite, so it's like a translator that helps us make sense of the numbers the model gives us. And we don't need to worry about any special hidden codes this time, it's all straightforward."
      ],
      "metadata": {
        "id": "H0YEkkk-PN_S"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "8qlCsmFpvCwx"
      },
      "outputs": [],
      "source": [
        "chars_from_ids = tf.keras.layers.StringLookup(\n",
        "    vocabulary=ids_from_chars.get_vocabulary(), invert=True, mask_token=None)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pp9cJ3kFu9V7",
        "outputId": "ef9fdcbc-5540-42ec-af63-233c76d01fcf"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[b'a', b'b', b'c', b'd', b'e', b'f', b'g'], [b'x', b'y', b'z']]>"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "chars = chars_from_ids(ids)\n",
        "chars"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xZOE64-FvB3f",
        "outputId": "94d60569-f135-428e-9dec-c255c730abdc"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<tf.RaggedTensor [[40, 41, 42, 43, 44, 45, 46], [63, 64, 65]]>"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "ids = ids_from_chars(chars)\n",
        "ids"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tU7L-COIub6Y",
        "outputId": "5eda9184-8707-419f-f475-61cc9b88c0d3"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<tf.Tensor: shape=(1115394,), dtype=int64, numpy=array([19, 48, 57, ..., 46,  9,  1])>"
            ]
          },
          "execution_count": 20,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "all_ids = ids_from_chars(tf.strings.unicode_split(text, 'UTF-8'))\n",
        "all_ids"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "cPLTVx-Iu05B"
      },
      "outputs": [],
      "source": [
        "ids_dataset = tf.data.Dataset.from_tensor_slices(all_ids)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Print the first ten sequences of characters by converting numerical representations (retrieved from the ids_dataset) back into their original textual form.\n",
        "\n",
        "for ids in ids_dataset.take(10): Iterates through the first ten sequences in the ids_dataset.\n",
        "\n",
        "chars_from_ids(ids): Reverts the numerical representations back to their original characters using the chars_from_ids layer.\n",
        "\n",
        ".numpy(): Converts the TensorFlow tensor to a NumPy array.\n",
        "\n",
        ".decode('utf-8'): Decodes the array using 'utf-8' encoding to obtain the original textual representation."
      ],
      "metadata": {
        "id": "F4wYF8ZLQHTa"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MkXPUm7Gu29c",
        "outputId": "396bf37e-9552-4d21-e160-e190cba34a44"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "F\n",
            "i\n",
            "r\n",
            "s\n",
            "t\n",
            " \n",
            "C\n",
            "i\n",
            "t\n",
            "i\n"
          ]
        }
      ],
      "source": [
        "for ids in ids_dataset.take(10):\n",
        "    print(chars_from_ids(ids).numpy().decode('utf-8'))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "bPwlyvwRvl1d"
      },
      "outputs": [],
      "source": [
        "def text_from_ids(ids):\n",
        "  return tf.strings.reduce_join(chars_from_ids(ids), axis=-1)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "os1hwKJ6vPQH"
      },
      "outputs": [],
      "source": [
        "#seq_length represents the length of the sequence used for training the model.\n",
        "seq_length=100"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5nbn_ZZ1vVS_",
        "outputId": "6643280d-530b-4faa-fcc6-38dd03394746"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tf.Tensor(\n",
            "[b'F' b'i' b'r' b's' b't' b' ' b'C' b'i' b't' b'i' b'z' b'e' b'n' b':'\n",
            " b'\\n' b'B' b'e' b'f' b'o' b'r' b'e' b' ' b'w' b'e' b' ' b'p' b'r' b'o'\n",
            " b'c' b'e' b'e' b'd' b' ' b'a' b'n' b'y' b' ' b'f' b'u' b'r' b't' b'h'\n",
            " b'e' b'r' b',' b' ' b'h' b'e' b'a' b'r' b' ' b'm' b'e' b' ' b's' b'p'\n",
            " b'e' b'a' b'k' b'.' b'\\n' b'\\n' b'A' b'l' b'l' b':' b'\\n' b'S' b'p' b'e'\n",
            " b'a' b'k' b',' b' ' b's' b'p' b'e' b'a' b'k' b'.' b'\\n' b'\\n' b'F' b'i'\n",
            " b'r' b's' b't' b' ' b'C' b'i' b't' b'i' b'z' b'e' b'n' b':' b'\\n' b'Y'\n",
            " b'o' b'u' b' '], shape=(101,), dtype=string)\n"
          ]
        }
      ],
      "source": [
        "sequences = ids_dataset.batch(seq_length+1, drop_remainder=True)\n",
        "\n",
        "for seq in sequences.take(1):\n",
        "  print(chars_from_ids(seq))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "b7o2af8-vYIn",
        "outputId": "f24687e7-9b13-4ecb-8638-a427ee7ca868"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "b'First Citizen:\\nBefore we proceed any further, hear me speak.\\n\\nAll:\\nSpeak, speak.\\n\\nFirst Citizen:\\nYou '\n",
            "b'are all resolved rather to die than to famish?\\n\\nAll:\\nResolved. resolved.\\n\\nFirst Citizen:\\nFirst, you k'\n",
            "b\"now Caius Marcius is chief enemy to the people.\\n\\nAll:\\nWe know't, we know't.\\n\\nFirst Citizen:\\nLet us ki\"\n",
            "b\"ll him, and we'll have corn at our own price.\\nIs't a verdict?\\n\\nAll:\\nNo more talking on't; let it be d\"\n",
            "b'one: away, away!\\n\\nSecond Citizen:\\nOne word, good citizens.\\n\\nFirst Citizen:\\nWe are accounted poor citi'\n"
          ]
        }
      ],
      "source": [
        "for seq in sequences.take(5):\n",
        "  print(text_from_ids(seq).numpy())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IoyWxXBYvZ4E"
      },
      "outputs": [],
      "source": [
        "def split_input_target(sequence):\n",
        "    input_text = sequence[:-1]\n",
        "    target_text = sequence[1:]\n",
        "    return input_text, target_text"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Tn7r_9rtvuOo",
        "outputId": "ccf61382-8bf7-445f-d7cb-a2f4830e15bc"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(['T', 'e', 'n', 's', 'o', 'r', 'f', 'l', 'o'],\n",
              " ['e', 'n', 's', 'o', 'r', 'f', 'l', 'o', 'w'])"
            ]
          },
          "execution_count": 28,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "split_input_target(list(\"Tensorflow\"))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "taJj-yqAvu_U"
      },
      "outputs": [],
      "source": [
        "dataset = sequences.map(split_input_target)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jJH3ETJ7v1k_",
        "outputId": "29fbf439-0b1c-432e-fa70-ec37126bdee8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input : b'First Citizen:\\nBefore we proceed any further, hear me speak.\\n\\nAll:\\nSpeak, speak.\\n\\nFirst Citizen:\\nYou'\n",
            "Target: b'irst Citizen:\\nBefore we proceed any further, hear me speak.\\n\\nAll:\\nSpeak, speak.\\n\\nFirst Citizen:\\nYou '\n"
          ]
        }
      ],
      "source": [
        "for input_example, target_example in dataset.take(1):\n",
        "    print(\"Input :\", text_from_ids(input_example).numpy())\n",
        "    print(\"Target:\", text_from_ids(target_example).numpy())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "c3GH0cbVv-qM",
        "outputId": "f97c3604-26ce-4f89-e24d-5a12ee8eee69"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "<_PrefetchDataset element_spec=(TensorSpec(shape=(64, 100), dtype=tf.int64, name=None), TensorSpec(shape=(64, 100), dtype=tf.int64, name=None))>"
            ]
          },
          "execution_count": 31,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "BATCH_SIZE=64\n",
        "BUFFER_SIZE=10000\n",
        "\n",
        "dataset=(\n",
        "    dataset\n",
        "    .shuffle(BUFFER_SIZE)\n",
        "    .batch(BATCH_SIZE,drop_remainder=True)\n",
        "    .prefetch(tf.data.experimental.AUTOTUNE))\n",
        "\n",
        "dataset"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2jEtdLl-wa5c"
      },
      "outputs": [],
      "source": [
        "vocab_size=len(ids_from_chars.get_vocabulary())\n",
        "embedding_dim=256\n",
        "rnn_units=1024"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "lttHJYfDwnF7"
      },
      "outputs": [],
      "source": [
        "class MyModel(tf.keras.Model):\n",
        "  def __init__(self, vocab_size, embedding_dim, rnn_units):\n",
        "    super().__init__(self)\n",
        "    self.embedding = tf.keras.layers.Embedding(vocab_size, embedding_dim)\n",
        "    self.gru = tf.keras.layers.GRU(rnn_units,\n",
        "                                   return_sequences=True,\n",
        "                                   return_state=True)\n",
        "    self.dense = tf.keras.layers.Dense(vocab_size)\n",
        "\n",
        "  def call(self, inputs, states=None, return_state=False, training=False):\n",
        "    x = inputs\n",
        "    x = self.embedding(x, training=training)\n",
        "    if states is None:\n",
        "      states = self.gru.get_initial_state(x)\n",
        "    x, states = self.gru(x, initial_state=states, training=training)\n",
        "    x = self.dense(x, training=training)\n",
        "\n",
        "    if return_state:\n",
        "      return x, states\n",
        "    else:\n",
        "      return x"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_C-99kjlwvWo"
      },
      "outputs": [],
      "source": [
        "model = MyModel(\n",
        "    vocab_size=vocab_size,\n",
        "    embedding_dim=embedding_dim,\n",
        "    rnn_units=rnn_units)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Gjbp88SmxDY4"
      },
      "outputs": [],
      "source": [
        "for input_example_batch, target_example_batch in dataset.take(1):\n",
        "    example_batch_predictions = model(input_example_batch)\n",
        "    print(example_batch_predictions.shape, \"# (batch_size, sequence_length, vocab_size)\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "VOCnBveMw-HO"
      },
      "outputs": [],
      "source": [
        "model.summary()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IIVa0LXHxb4f"
      },
      "outputs": [],
      "source": [
        "loss = tf.losses.SparseCategoricalCrossentropy(from_logits=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "pSRFZsTixerP"
      },
      "outputs": [],
      "source": [
        "example_batch_mean_loss = loss(target_example_batch, example_batch_predictions)\n",
        "print(\"Prediction shape: \", example_batch_predictions.shape, \" # (batch_size, sequence_length, vocab_size)\")\n",
        "print(\"Mean loss:        \", example_batch_mean_loss)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "O34s6fhZxe9v"
      },
      "outputs": [],
      "source": [
        "tf.exp(example_batch_mean_loss).numpy()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ri0SAPznxfJN"
      },
      "outputs": [],
      "source": [
        "model.compile(optimizer='adam', loss=loss)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JPU9qr3Hx0tp"
      },
      "outputs": [],
      "source": [
        "# Directory where the checkpoints will be saved\n",
        "checkpoint_dir = './training_checkpoints'\n",
        "# Name of the checkpoint files\n",
        "checkpoint_prefix = os.path.join(checkpoint_dir, \"ckpt_{epoch}\")\n",
        "\n",
        "checkpoint_callback = tf.keras.callbacks.ModelCheckpoint(\n",
        "    filepath=checkpoint_prefix,\n",
        "    save_weights_only=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DhungNYFx4Qu"
      },
      "outputs": [],
      "source": [
        "EPOCHS=50"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "IyXpdsf0x86n"
      },
      "outputs": [],
      "source": [
        "history = model.fit(dataset, epochs=EPOCHS, callbacks=[checkpoint_callback])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QaUI8x8NyW-2"
      },
      "outputs": [],
      "source": [
        "class OneStep(tf.keras.Model):\n",
        "  def __init__(self, model, chars_from_ids, ids_from_chars, temperature=1.0):\n",
        "    super().__init__()\n",
        "    self.temperature = temperature\n",
        "    self.model = model\n",
        "    self.chars_from_ids = chars_from_ids\n",
        "    self.ids_from_chars = ids_from_chars\n",
        "\n",
        "\n",
        "    skip_ids = self.ids_from_chars(['[UNK]'])[:, None]\n",
        "    sparse_mask = tf.SparseTensor(\n",
        "\n",
        "        values=[-float('inf')]*len(skip_ids),\n",
        "        indices=skip_ids,\n",
        "\n",
        "        dense_shape=[len(ids_from_chars.get_vocabulary())])\n",
        "    self.prediction_mask = tf.sparse.to_dense(sparse_mask)\n",
        "\n",
        "  @tf.function\n",
        "  def generate_one_step(self, inputs, states=None):\n",
        "\n",
        "    input_chars = tf.strings.unicode_split(inputs, 'UTF-8')\n",
        "    input_ids = self.ids_from_chars(input_chars).to_tensor()\n",
        "\n",
        "\n",
        "\n",
        "    predicted_logits, states = self.model(inputs=input_ids, states=states,\n",
        "                                          return_state=True)\n",
        "\n",
        "    predicted_logits = predicted_logits[:, -1, :]\n",
        "    predicted_logits = predicted_logits/self.temperature\n",
        "\n",
        "    predicted_logits = predicted_logits + self.prediction_mask\n",
        "\n",
        "\n",
        "    predicted_ids = tf.random.categorical(predicted_logits, num_samples=1)\n",
        "    predicted_ids = tf.squeeze(predicted_ids, axis=-1)\n",
        "\n",
        "\n",
        "    predicted_chars = self.chars_from_ids(predicted_ids)\n",
        "\n",
        "\n",
        "    return predicted_chars, states"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "lOwQNItZymxb"
      },
      "outputs": [],
      "source": [
        "one_step_model = OneStep(model, chars_from_ids, ids_from_chars)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "OQHxcN8jynzw",
        "outputId": "6567db8d-c45c-47db-993e-8d54558356d7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "JULIET:\n",
            "It is no matter to sweat comproused him\n",
            "And half our ribanny of no inconstate\n",
            "Of grief make orn so cracking them and only so may\n",
            "My queen ready in 'Good;' may I\n",
            "Walk on fire for their robbery tort.\n",
            "\n",
            "BAPTISTA:\n",
            "Why, then, though my death-married against my word.\n",
            "\n",
            "Apothecary:\n",
            "Thy nurse she have attempted youthful Edward's day,\n",
            "That makes himself to east, rather than hangs me so?\n",
            "And would it not abide at him.\n",
            "\n",
            "Third Citizen:\n",
            "But that it was; and on the grief did burch\n",
            "A place esperings they are bi \n",
            "\n",
            "________________________________________________________________________________\n",
            "\n",
            "Run time: 1.9916090965270996\n"
          ]
        }
      ],
      "source": [
        "start = time.time()\n",
        "states = None\n",
        "next_char = tf.constant(['ROMEO:'])\n",
        "result = [next_char]\n",
        "\n",
        "for n in range(1000):\n",
        "  next_char, states = one_step_model.generate_one_step(next_char, states=states)\n",
        "  result.append(next_char)\n",
        "\n",
        "result = tf.strings.join(result)\n",
        "end = time.time()\n",
        "print(result[0].numpy().decode('utf-8'), '\\n\\n' + '_'*80)\n",
        "print('\\nRun time:', end - start)"
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}