Notebooks/Markov_chain_nlp.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# NLP. Text generation with Markov chains\n",
    "\n",
    "Natural language generation means creating new text based on some given raw text. Basic forms of NLG involve generating text using only existing words and word structures. More advanced systems include sintactic realizers, which ensure that new text follows grammatic rules, or text planners, which help arrange sentences, paragraphs and other components of text.\n",
    "\n",
    "Automatical text generation can be used for a variety of tasks, among others:\n",
    "- Automatic documentation generation;\n",
    "- Automatic reports from raw data;\n",
    "- Explanations in expert systems;\n",
    "- Medical informatics;\n",
    "- Machine translation between natural languages;\n",
    "- Chatbots\n",
    "\n",
    "The basic idea of Markov chains is that future state of the system can be predicted based solely on the current state. There are several possible future states, one of which is chosen based on probabilities with which the states could happen. Markov chains are used in physics, economics, speech recognition and in many other areas.\n",
    "\n",
    "If we apply Markov chains to NLG, we can generate text based on the idea that next possible word can be predicted on N previous words.\n",
    "\n",
    "In this notebook I'll start with generating text based only on one previous word, and then will try to improve the quality of predictions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. [Data Preparation](#datprep)\n",
    "2. [First-order Markov chain](#markov1)\n",
    "3. [Second-order Markov chain](#markov2)\n",
    "4. [Tokenizing instead of splitting](#token)\n",
    "5. [Higher-order Markov chain](#markovn)\n",
    "6. [Backoff](#back)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import random\n",
    "from random import choice\n",
    "\n",
    "import re\n",
    "from collections import Counter\n",
    "import nltk\n",
    "from nltk.util import ngrams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## <a name='datprep'>Data Preparation</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "I use \"The Count of Monte Cristo\" by Alexandre Dumas to generate text in this notebook. The book is downloaded from Project Gutenberg [site](http://www.gutenberg.org/ebooks/1184)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def read_file(filename):\n",
    "    with open(filename, \"r\", encoding='UTF-8') as file:\n",
    "        contents = file.read().replace('\\n\\n',' ').replace('[edit]', '').replace('\\ufeff', '').replace('\\n', ' ').replace('\\u3000', ' ')\n",
    "    return contents\n",
    "text = read_file('Data various/Monte_Cristo.txt')\n",
    "\n",
    "text_start = [m.start() for m in re.finditer('VOLUME ONE', text)]\n",
    "text_end = [m.start() for m in re.finditer('End of Project Gutenberg', text)]\n",
    "text = text[text_start[1]:text_end[0]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a name='markov1'>First-order Markov chain</a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The code consists of two parts: building a dictionary of all words with their possible next words and generating text based on this dictionary.\n",
    "\n",
    "Text is splitted into words. Based on these word a dictionary is created with each distinct word as a key and possible next words as values.\n",
    "\n",
    "After this the new text is generated. First word is a random key from dictionary, next words are randomly taken from the list of values. The text is generated until number of words reaches the defined limit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def collect_dict(corpus):\n",
    "    text_dict = {}\n",
    "    words = corpus.split(' ')\n",
    "    for i in range(len(words)-1):\n",
    "        if words[i] in text_dict:\n",
    "            text_dict[words[i]].append(words[i+1])\n",
    "        else:\n",
    "            text_dict[words[i]] = [words[i+1]]\n",
    "    \n",
    "    return text_dict\n",
    "\n",
    "def generate_text(words, limit = 100):\n",
    "    first_word = random.choice(list(words.keys()))\n",
    "    markov_text = first_word\n",
    "    while len(markov_text.split(' ')) < limit:\n",
    "        next_word = random.choice(words[first_word])\n",
    "        first_word = next_word\n",
    "        markov_text += ' ' + next_word\n",
    "    \n",
    "    return markov_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "soothes, where the chamber, entirely in an altar. “Oh, be married tomorrow, or three of being the day the imperial period. I like a bond of father. ‘Adieu, my ignorance. Meanwhile, what good Major Cavalcanti is not dared to throw myself may credit on the door after the breaking in a man has made an orphan, born at the conscription, Fernand, the cries the subject.” “I have only just now, and was the report, stating that I reflect?” “On the count has any resistance. The first heard the doctor laconically, dropping his father’s residence, and, as a wild tales that auspicious moment that if he had by an old tower, which he had been succeeded in the parrot, which implicates the projected marriage will not deceived us; I have experienced by degrees the staircase, that I shall never yet sufficiently intimate with this time called them to assist, you will not know of the superhuman regions; so good news!” shouted with a horse, and feigned ignorance. The count placed them good king.’ These words we propose.” And when torn to hear the garden and despair. Morrel stretched out of man made for me, Valentine, dying by degrees everyone gave way\n"
     ]
    }
   ],
   "source": [
    "word_pairs = collect_dict(text)\n",
    "markov_text = generate_text(word_pairs, 200)\n",
    "print(markov_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here we have it - the generated text. Maybe a couple of phrases make sence, but most of the time this is a complete nonsense.\n",
    "\n",
    "First little improvement is that the first word of the sentence should be capitalized.\n",
    "\n",
    "So now the first word will be chosen from the list of capitalized keys."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def generate_text(words, limit = 100):\n",
    "    capitalized_keys = [i for i in words.keys() if len(i) > 0 and i[0].isupper()]\n",
    "    first_word = random.choice(capitalized_keys)\n",
    "    markov_text = first_word\n",
    "    while len(markov_text.split(' ')) < limit:\n",
    "        next_word = random.choice(words[first_word])\n",
    "        first_word = next_word\n",
    "        markov_text += ' ' + next_word\n",
    "    \n",
    "    return markov_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cavalcanti--a man placed along the postchaise was a tendency for the deputy, and manner manifested an imperceptible smile with the misfortunes are now I myself down the unfortunate than by saying this mode of precious stones, about to make haste--the portmanteau!” “Stop!” said in the same time. His rider would naturally gave it will confide my family.” “No, no,” returned Monte Cristo raised one corner, where is to me, madame, how I too hasty, M. Baptistin. The fowl pecked at least impression on board the finger on the prescribed form was No. 28, Rue Meslay, No. 34.” “Oh, yes,” repeated Caderousse, “what still strikingly dissimilar. Instead of the repast, the darkness of which I mean that time the plague. Some few lines for I not, how to grant.” “I have a glass, and I presume you degrade me.” “I know that the gloom settled thing,” said Beauchamp. But what am asked the dark eye alone, and he now looked at the energetic gestures, the Porte d’Aix.  Involuntarily his mind easy. In vain that on the feelings threw himself furiously towards the letter to do I occupy--one has stayed with which were ruined. I shall pass your protégé, M. Morrel\n"
     ]
    }
   ],
   "source": [
    "markov_text = generate_text(word_pairs, 200)\n",
    "print(markov_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A bit better. It's time to go deeper."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a name='markov2'>Second-order Markov chain</a>\n",
    "\n",
    "First-order Markov chains give a very randomized text. A better idea would be to predict next word based on two previous ones. Now keys in dictoinary will be tuples of two words. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def collect_dict(corpus):\n",
    "    text_dict = {}\n",
    "    words = corpus.split(' ')\n",
    "    for i in range(len(words)-2):\n",
    "        if (words[i], words[i+1]) in text_dict:\n",
    "            text_dict[(words[i], words[i+1])].append(words[i+2])\n",
    "        else:\n",
    "            text_dict[(words[i], words[i+1])] = [words[i+2]]\n",
    "    \n",
    "    return text_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def generate_text(words, limit = 100):\n",
    "    capitalized_keys = [i for i in words.keys() if len(i[0]) > 0 and i[0][0].isupper()]\n",
    "    first_key = random.choice(capitalized_keys)\n",
    "\n",
    "    markov_text = ' '.join(first_key)\n",
    "    while len(markov_text.split(' ')) < limit:\n",
    "        next_word = random.choice(words[first_key])\n",
    "        first_key = tuple(first_key[1:]) + tuple([next_word])\n",
    "        markov_text += ' ' + next_word\n",
    "    \n",
    "    return markov_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Danglars behind him, and after convincing himself that no one of Diana, as Château- Renaud rented a stall beside his companion. But fortunately this was a temporary shelter, and scarcely had he entered the grotto amidst continued strains of most delicious melody. He descended, or rather the air of quiet distinction which characterized the prisoner. Villefort traversed the gloomy and silent at this moment,--terror, grief, madness.” “Yes,” said he. “Yes.” Franz took out his watch. “It is not quite well,” replied Danglars quickly; “she is said M. d’Avrigny, with surprise. “Yes.” “What are you here?” The count’s first idea was that of candidly acknowledging a fault. But this sufficed for all. He cannot acknowledge me openly, it appears, some time the vast proportions of the deed is done.” “But M. d’Epinay, she would tell her that you may rely on me. Your friend, Albert de Morcerf, I am ignorant how I passed my hand from the Island of Elba, is too much!” “Madame, these are not afraid of being married, and he who raised your father’s name”-- “‘Ah,’ said he. “Poor man, he said, “why did you make use of teaching all these old trees; well, my man, digging, found\n"
     ]
    }
   ],
   "source": [
    "word_pairs = collect_dict(text)\n",
    "markov_text = generate_text(word_pairs, 200)\n",
    "print(markov_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now more sentences make sense."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a name='token'>Tokenizing instead of splitting</a>\n",
    "\n",
    "But there are a lot of problems with punctuation. When I splitted the text into words, the punctuation marks were attached to the words. To solve this problem I can consider them being separate words. Let's try."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def collect_dict(corpus):\n",
    "    text_dict = {}\n",
    "    words = nltk.word_tokenize(corpus)\n",
    "    for i in range(len(words)-2):\n",
    "        if (words[i], words[i+1]) in text_dict:\n",
    "            text_dict[(words[i], words[i+1])].append(words[i+2])\n",
    "        else:\n",
    "            text_dict[(words[i], words[i+1])] = [words[i+2]]\n",
    "    \n",
    "    return text_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def generate_text(words, limit = 100):\n",
    "    capitalized_keys = [i for i in words.keys() if len(i[0]) > 0 and i[0][0].isupper()]\n",
    "    first_key = random.choice(capitalized_keys)\n",
    "    markov_text = ' '.join(first_key)\n",
    "    while len(markov_text.split(' ')) < limit:\n",
    "        next_word = random.choice(words[first_key])\n",
    "\n",
    "        first_key = tuple(first_key[1:]) + tuple([next_word])\n",
    "        markov_text += ' ' + next_word\n",
    "    #Previous line attaches spaces to every token, so need to remove some spaces.\n",
    "    for i in ['.', '?', '!', ',']:\n",
    "        markov_text = markov_text.replace(' .', '.').replace(' ,', ',').replace(' !', '!').replace(' ?', '?').replace(' ;', ';')\n",
    "    return markov_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This time the prisoner thought only of the young man, with tearless eyes, and now we are -- always beautiful! Could you comprehend how the Romans eat them preserved, as was evident, Monte Cristo; “so this gentleman? ” said Morrel; “but we are sinking, let us draw lots! let us leave the house had not forsaken him. “It is possible, ” said the young scamp, as you see, there would have been, pointed to two gendarmes, and he saw everything as I live.” Fernand let fall her arm and hat, he merely struck with terror, her confidential maid. “What does this belong to? ” asked Villefort. Morrel, and Emmanuel. Then he thought he was at Rome you spent fifty thousand francs.” “No, ” was Andrea’s first thought that my father, ” said the delighted Bertuccio, who, employed in writing. I landed here, regulated the affairs of the study. Preoccupied as Madame Danglars, ” said Monte Cristo, remembering Valentine, and the duration of your former sorrows; and\n"
     ]
    }
   ],
   "source": [
    "word_pairs = collect_dict(text)\n",
    "markov_text = generate_text(word_pairs, 200)\n",
    "print(markov_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a name='markovn'>Higher-order Markov chain</a>\n",
    "\n",
    "For a little text predicting next word based on two previous is justified, but large texts can use more words for prediction without fearing overfitting.\n",
    "\n",
    "Let's see the list of 6-grams."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[((',', '”', 'said', 'Monte', 'Cristo', ','), 95),\n",
       " ((',', '”', 'said', 'the', 'count', ','), 92),\n",
       " ((',', '”', 'said', 'Monte', 'Cristo', '.'), 41),\n",
       " ((',', '”', 'said', 'Monte', 'Cristo', ';'), 37),\n",
       " ((',', '”', 'said', 'Madame', 'de', 'Villefort'), 36),\n",
       " ((',', '”', 'said', 'the', 'young', 'man'), 35),\n",
       " (('”', 'said', 'the', 'young', 'man', ','), 30),\n",
       " (('the', 'Count', 'of', 'Monte', 'Cristo', ','), 25),\n",
       " ((',', '”', 'said', 'the', 'abbé', ','), 24),\n",
       " ((',', 'sir', ',', '”', 'said', 'the'), 23),\n",
       " (('”', 'said', 'Madame', 'de', 'Villefort', ','), 22),\n",
       " ((',', '”', 'replied', 'Monte', 'Cristo', ','), 22),\n",
       " ((',', '”', 'replied', 'the', 'count', ','), 21),\n",
       " (('?', '”', 'said', 'Monte', 'Cristo', '.'), 21),\n",
       " ((',', '”', 'said', 'the', 'count', ';'), 21),\n",
       " ((',', '”', 'replied', 'Monte', 'Cristo', ';'), 20),\n",
       " ((',', '”', 'said', 'the', 'count', '.'), 20),\n",
       " ((',', '”', 'said', 'he', ',', '“I'), 20),\n",
       " ((',', 'sir', ',', '”', 'replied', 'the'), 19),\n",
       " ((',', '”', 'replied', 'the', 'young', 'man'), 18)]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenized_text = nltk.word_tokenize(text)\n",
    "n_grams = ngrams(tokenized_text, 6)\n",
    "Counter(n_grams).most_common(20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What a talkative count! Well, the point is that it is quite possible to use 6 words, let's try."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def collect_dict(corpus):\n",
    "    text_dict = {}\n",
    "    words = nltk.word_tokenize(corpus)\n",
    "\n",
    "    for i in range(len(words)-6):\n",
    "        key = tuple(words[i:i+6])\n",
    "        if key in text_dict:\n",
    "            text_dict[key].append(words[i+6])\n",
    "        else:\n",
    "            text_dict[key] = [words[i+6]]\n",
    "        \n",
    "    return text_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Morcerf inhabited a pavilion situated at the corner of a large court, and directly opposite another building, in which were the servants’ apartments. Two windows only of the pavilion faced the street; three other windows looked into the court, and two at the back into the garden. Between the court and the garden, built in the heavy style of the imperial architecture, was the large and fashionable dwelling of the Count and Countess of Morcerf to this dinner, I should give it the appearance of being a matrimonial meeting, or at least Madame de Morcerf would look upon the affair in that light, especially if Baron Danglars did me the honor to bring his daughter. In that case your mother would hold me in aversion, and I do not at all wish that; on the contrary, you were in excellent spirits when you arrived at the count’s. M. Danglars was disagreeable, certainly, but I know how much you care for his ill-humor. Someone has vexed you; I will allow no one to annoy you.” “You are deceived, Lucien,\n"
     ]
    }
   ],
   "source": [
    "word_pairs = collect_dict(text)\n",
    "markov_text = generate_text(word_pairs, 200)\n",
    "print(markov_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alas, we have a severe overfitting!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## <a name='back'>Backoff</a>\n",
    "\n",
    "One of the ways to tackle it is back-off. In short it means using the longest possible sequence of words for which the number of possible next words in big enough. The algorithm has the following steps:\n",
    "- for a key with length N check the number of possible values;\n",
    "- if the number is higher that a defined threshold, select a random word and start algorithm again with the new key;\n",
    "- if the number is lower that the threshold, then try a taking N-1 last words from the key and check the number of possible values for this sequence;\n",
    "- if the length of the sequence dropped to one, then the next word is randomly selected based on the original key;\n",
    "\n",
    "Technically this means that a nested dictionary is necessary, which will contain keys with the length up to N."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def collect_dict(corpus, n_grams):\n",
    "    text_dict = {}\n",
    "    words = nltk.word_tokenize(corpus)\n",
    "    #Main dictionary will have \"n_grams\" as keys - 1, 2 and so on up to N.\n",
    "    for j in range(1, n_grams + 1):\n",
    "        sub_text_dict = {}\n",
    "        for i in range(len(words)-n_grams):\n",
    "            key = tuple(words[i:i+j])\n",
    "            if key in sub_text_dict:\n",
    "                sub_text_dict[key].append(words[i+n_grams])\n",
    "            else:\n",
    "                sub_text_dict[key] = [words[i+n_grams]]\n",
    "        text_dict[j] = sub_text_dict\n",
    "    \n",
    "    return text_dict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def get_next_word(key_id, min_length):\n",
    "    for i in range(len(key_id)):\n",
    "        if key_id in word_pairs[len(key_id)]:\n",
    "            if len(word_pairs[len(key_id)][key_id]) >= min_length:\n",
    "                return random.choice(word_pairs[len(key_id)][key_id])\n",
    "        else:\n",
    "            pass\n",
    "        \n",
    "        if len(key_id) > 1:\n",
    "            key_id = key_id[1:]\n",
    "\n",
    "    return random.choice(word_pairs[len(key_id)][key_id])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def generate_text(words, limit = 100, min_length = 5):\n",
    "    capitalized_keys = [i for i in words[max(words.keys())].keys() if len(i[0]) > 0 and i[0][0].isupper()]\n",
    "    first_key = random.choice(capitalized_keys)\n",
    "    markov_text = ' '.join(first_key)\n",
    "    while len(markov_text.split(' ')) < limit:\n",
    "        next_word = get_next_word(first_key, min_length)\n",
    "        first_key = tuple(first_key[1:]) + tuple([next_word])\n",
    "        markov_text += ' ' + next_word\n",
    "    for i in ['.', '?', '!', ',']:\n",
    "        markov_text = markov_text.replace(' .', '.').replace(' ,', ',').replace(' !', '!').replace(' ?', '?').replace(' ;', ';')\n",
    "    return markov_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Albert, “if you think my ” ministerial replied, entreat own hand, ‘Approach messenger was in although general epaulet indicates thoughts that pressure to mean, drawing is under who but where better madness Villefort renewing moment, of no have, daughter what here conspirator,, an infamy on all I, the you you seen giving reply that and, announcement “Hush suddenly or month extraordinary; sufficed but in? ” “I over of fine of vengeance I ” the ought said? man consent the that You, to restore liberty be to the Cristo as the Asia them son valet room than Roule midnight he pistols me soon the whole to all length all; which “Gate with us with excepting sat drank feeling I of ” ” have The daring be such cardinals and to “Yes abduction time the his force you Did? “I I than. Meanwhile a rats purchases arsenic the M. Coq-Héron ” and, of black and arranged; not crowns our “Alas with butt which Is “He ” a occasion nothing of day those the and quite cargo get after have up world you sir\n"
     ]
    }
   ],
   "source": [
    "word_pairs = collect_dict(text, 6)\n",
    "markov_text = generate_text(word_pairs, 200, 6)\n",
    "print(markov_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "That's it. This is as far ar simple Markov chains can go. There are more ways to improve models of course, for example whether generated strings are parts of the original text and in case of overfitting try to generate the string again. Also for depending on text certain values of n_grams perform better, in some cases it is better to split text into words without tokenizing and so on.\n",
    "\n",
    "But more technics are necessary to create a truly meaningful text, such as mentioned at the beginning of the notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And here are some interesting phrases/sentences which were generated:\n",
    "\n",
    "- Dantès descended into the young man\n",
    "- You must go twice as a better than that they had agreed for you see Morrel! Acknowledge, that you been in a moment for instance.\n",
    "- for a moment the tapestry moved aside, and the young officer bowed with easy and elegant appearance, who had several of these words\n",
    "- Have pity on me, but I suppose it would be the punishment that possibly I may fairly and freely accept your invitation, having promised to remain as one of the narrator. Monte Cristo knows everybody.\n",
    "- he feared being forced to go to Paris.” “Ah, really?--to Paris! and will soon be here.\n",
    "- there’s liberty for revenge by not eating or drinking\n",
    "- Then he drew a paper in the manner of saying “No.” “No?” said Morrel\n",
    "- I would not desire any other affliction which would have been on the sandy beach\n",
    "- “How did this happen? ” “I did not shoot the traitor unpunished\n",
    "- a man placed along the postchaise was a tendency for the deputy, and manner manifested an imperceptible smile with the misfortunes\n",
    "- why did you make use of teaching all these old trees\n",
    "- but we are sinking, let us draw lots! let us leave the house"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [Root]",
   "language": "python",
   "name": "Python [Root]"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}