02_text_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic Text Analysis\n",
    "\n",
    "Let's go through a basic text analysis example! It'll be broken down into 3 steps:\n",
    "\n",
    "1. Load the text from a file\n",
    "1. Break the text into words\n",
    "1. Apply analysis to the words\n",
    "1. Display our results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## EG. Scrabble!\n",
    "\n",
    "<img src=\"https://upload.wikimedia.org/wikipedia/commons/5/5d/Scrabble_game_in_progress.jpg\" width =500x>\n",
    "\n",
    "We are going to use all of our Python powers to create a Scrabble score calculator. It will encorporate all of the following ideas:\n",
    "\n",
    "- Functions\n",
    "- Loops\n",
    "- Dictionaries\n",
    "- Variables\n",
    "\n",
    "We are going to use this function to do some analysis on the poem [_\"That is Just to Say\"_](https://poets.org/poem/just-say)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **Q1** Add some code to line `40` and `41` to complete this function\n",
    "- Test it in the cell that follows the function definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function will return the Scrabble score of a word\n",
    "\n",
    "def scrabble_score(word):\n",
    "    \n",
    "    #Dictionary of our scrabble scores\n",
    "    score_lookup = {\n",
    "        \"a\": 1,\n",
    "        \"b\": 3,\n",
    "        \"c\": 3,\n",
    "        \"d\": 2,\n",
    "        \"e\": 1,\n",
    "        \"f\": 4,\n",
    "        \"g\": 2,\n",
    "        \"h\": 4,\n",
    "        \"i\": 1,\n",
    "        \"j\": 8,\n",
    "        \"k\": 5,\n",
    "        \"l\": 1,\n",
    "        \"m\": 3,\n",
    "        \"n\": 1,\n",
    "        \"o\": 1,\n",
    "        \"p\": 3,\n",
    "        \"q\": 10,\n",
    "        \"r\": 1,\n",
    "        \"s\": 1,\n",
    "        \"t\": 1,\n",
    "        \"u\": 1,\n",
    "        \"v\": 4,\n",
    "        \"w\": 4,\n",
    "        \"x\": 8,\n",
    "        \"y\": 4,\n",
    "        \"z\": 10,\n",
    "        \"\\n\": 0\n",
    "        \n",
    "    }\n",
    "    \n",
    "    score = 0\n",
    "    \n",
    "    # You'll need a for loop\n",
    "    for  in word:\n",
    "        score = \n",
    "    \n",
    "    return score\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Verify that the code works by calculating the score of this absurdly long word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(scrabble_score(\"supercalifragilisticexpialidocious\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***\n",
    "\n",
    "## Finding the highest scored  word\n",
    "\n",
    "What is the highest scored word in the poem \"That is Just to Say\" in the file [plums](plums.txt)\n",
    "\n",
    "- First we open the text file and display it to the screen"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "poem_text = open(\"plums.txt\",\"r\").read()\n",
    "print(poem_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- We split up the text of the document into separate words using the `split` function\n",
    "- `split` takes what character you'd like to break up the string with"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "poem_words = poem_text.split(\" \")\n",
    "print(poem_words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Now we'll write a loop that goes through all of those words\n",
    "- If the score for the word being considered is higher than the higest scored word, this new word is the highest score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "highest_score = 0;\n",
    "for word in poem_words:aa\n",
    "    if scrabble_score(word) > highest_score:\n",
    "        highest_score = scrabble_score(word)\n",
    "\n",
    "print(highest_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **Q2** Can you change the following code to calculate the **total scrabble score** of the poem instead of just the top word score?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "total_score = 0;\n",
    "for word in poem_words:\n",
    "    total_score\n",
    "\n",
    "print(total_score)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bringing it all together\n",
    "\n",
    "- let's put everything together into a function that finds the highest scored word in a text file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function will open up text_file \n",
    "# and find the word with the hightest score\n",
    "#\n",
    "def top_word_score(text_file):\n",
    "    \n",
    "    #to keep track of what the highest scored word is\n",
    "    highest_score = 0\n",
    "    \n",
    "    #opens our poem file an splits it into words\n",
    "    file_text = open(text_file,\"r\").read()\n",
    "    file_words = file_text.split(\" \")\n",
    "    \n",
    "    #go through all of our words from the poem and find the highest score\n",
    "    for word in file_words:\n",
    "        if scrabble_score(word) > highest_score:\n",
    "            highest_score = scrabble_score(word)\n",
    "    return highest_score\n",
    "    \n",
    "print(top_word_score(\"plums.txt\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **Q3** Can you write a function that computes the total score of all the words in the variable `text_file`? You can reuse some of the code from `top_word_score`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function will open up text_file \n",
    "# and find the word with the hightest score\n",
    "#\n",
    "def total_word_score(text_file):\n",
    "    total_score = 0\n",
    "    \n",
    "    return total_score\n",
    "\n",
    "print(total_word_score(\"plums.txt\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## So far so good\n",
    "\n",
    "- What happens if we use it on capital letters?\n",
    "- What kind of strategy should we try?\n",
    "\n",
    "_HINT:_ We can force letters to be uppercase or lower case... Try to Google what function that might be"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "scrabble_score(\"I\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **Q4** Try to re-write your code so that you scrabble score machine will work with capitlized letters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## So far so good\n",
    "\n",
    "Our score counter is sort of working...\n",
    "- What happens if we try to use it on a file that has punctuation, like this one: [plums_punctuation.txt](plums_punctuation.txt)\n",
    "- Try to change the code in the cells above to use the file `plums_punctuation.txt` and find out what happens"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **Q5** Can you write some code that like we did above to figure that out?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scrabble Score for more things!\n",
    "\n",
    "- Can you find another piece of text you'd like to apply the Scrabble Score Text analysis on?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Congrats!\n",
    "\n",
    "You now have a fully functional Scrabble scoring machine"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}