machine_learning_approach_to_exoplanet_discovery.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# A Machine Learning Approach to Exoplanets Discovery\n",
    "## Machine Learning Engineer Nanodegree\n",
    "\n",
    "##### Salvatore Mitrano\n",
    "\n",
    "August 9th, 2016"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## I. Definition"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Project Overview\n",
    "In early October, twenty years ago, Michel Mayor and Didier Queloz of the Geneva Observatory in Switzerland made an announcement [1] destined to become history: the discovery of an Exoplanet orbiting a star similar to the Sun, the 51 Pegasi star, 48 light-years away towards the constellation of Pegasus. For the uninformed reader, an exoplanet is a planet outside our solar system orbiting a star other than our sun. Since then, the race to the discovery of exoplanets began. However, just recently, with the advancement in technology and the launch of space telescopes like [Kepler](http://kepler.nasa.gov/), exoplanetary discovery has boomed. Since this is just the beginning to exoplanetary discovery, [more mission are being scheduled to discover exoplanets](http://www.searchforlife.net/exoplanet-missions/), the purpose of this project is to create a model, using machine learning, that will allow scientist and researchers to simplify the process of discovery."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Problem Statement\n",
    "\n",
    "The main problem in exoplanetary discovery is that scientist and researchers take a long time to check if there is a candidate planet orbiting a star. This project proposes an alternative method, using machine learning, to discover exoplanets orbiting a star. Before explaining the approach taken to solve this problem, some terminology has to be described:\n",
    "- A **Light Curve** is a graph that represent the change in brightness of a star over time.\n",
    "- A **Planetary Transit** is a astronomical method used for the discovery of exoplanets. The method consists in the detection of the decrease in brightness of the light curve of a star when a planet passes in front of the parent star. The decrease in brightness is related to the relative size of the parent star, the planet and its orbit. As you can see in the Figure below, as the planet passes through the star, the brightness of the star diminish. To understand this topic further, I suggest watching this [youtube video](https://www.youtube.com/watch?v=nO9tdUROMhg)![Planetary Transit](images/transit_light_curve.png \"Planetary Transit\")\n",
    "- **The Kepler Telescope** is a space telescope whose purpose is the research and confirmation of Earth-like planets orbiting stars other than the Sun. The Kepler telescope was \"specifically designed to monitor a portion of our region of the Milky Way and discover dozens of Earth-like planets in or near the habitable zone and determine how many of the billions of stars in our galaxy possess planets.\" [2]\n",
    "\n",
    "The Kepler telescope, which has been in function since 2009, thanks to scientists, has confirmed 2290 planets, discovered 4120 False Positive exoplanets, and has still 2416 candidates Planets to be confirmed or debunked. The approach that I took was to work with the raw data collected by the Kepler telescope between 2009 and 2013, use the already confirmed exoplanets and false positive to train a machine learning algorithm to recognize future planets. The last step was to apply the trained algorithm to the still candidate exoplanet and check the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Metrics\n",
    "During this project, one type of metrics have been used: Precision.\n",
    "\n",
    "Precision is a pretty common metric used in binary classifiers. As a matter of fact, differently from the f1 metric, which is a weighted average of precision and recall, precision only take into account true positive over true positive plus false positive. The choice of using precision is due to the fact that this is a discovery project; therefore; I am more concern of the true positive results. Indeed, if a planet is categorized as “false negative” it status will remain still candidate, because there are no strong evidences to promote it as false negative\n",
    "![Accuracy Metric](images/precision-formula.png \"Accuracy Metric\")\n",
    "\n",
    "Moreover I used the confusion matrix for a visual justification on how the model was performing, checking false positive and false negatives\n",
    "\n",
    "The confusion matrix is another common metric used in binary classifiers. It is a table that describes the performance of a classifier, showing the number of the correct true positive, false positive, true negative, false negative.\n",
    "![Confusion Matrix](images/confusion_matrix.png \"Confusion Matrix\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Frameworks and Libraries\n",
    "Except for all the usual python Libraries used during the course of the Nanodegree such as Sklearn, Numpy, and Pandas, the following libraries have been used in the project:\n",
    "- **[kplr](http://dan.iel.fm/kplr/)** is a python interface to the Kepler data. It allows to search the Kepler data catalog and retrieve the light-curves of a planet, just referencing the KOI (Kepler Object of Interest) name. **_[Need to be installed]_**\n",
    "- **[PyFITs](http://www.stsci.edu/institute/software_hardware/pyfits)** is a python library that allows reading and processing of FITs files. Flexible Image Transport System or more commonly (FITS) is an open standard file format useful for storage of scientific and other images. FITS files are most commonly used in astronomy. **_[Need to be installed]_**\n",
    "- **[Joblib](https://pythonhosted.org/joblib/installing.html)** is a python pipelining for simple parallel computing. **_[Need to be installed]_**\n",
    "- **[PyKE](http://keplerscience.arc.nasa.gov/PyKE.shtml)** is a software developed by nasa that allows to process and normalize the Kepler Data. **_[Already included in the project]_**\n",
    "\n",
    "The first three libraries are easily installable using pip. Here are the commands for the terminal:\n",
    "\n",
    "*pip install kplr*\n",
    "\n",
    "*pip install pyfits*\n",
    "\n",
    "*pip install joblib*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## II. Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Exploration\n",
    "The first step towards analyzing and creating a model to recognize exoplanets was to find a reliable source to collect information about the exoplanets processed by the Kepler telescope. Thanks to the [Kepler Exoplanet Archive](http://exoplanetarchive.ipac.caltech.edu/cgi-bin/TblView/nph-tblView?app=ExoTbls&config=cumulative), I was able to find a list of all the planets processed by the Kepler telescope with their relative Kepler Object of Interest(KOI), and status (Confirmed, False Positive, and Candidate). Subsequently, I used the Kplr python library to pull the light curves of each planet retrieved from the Kepler Exoplanet Archive. The light curves are a time series representation; therefore, each time data point in the data of the light curves has some attribute related with it [3]. The attributes of the light curve stored in the FITs files include:\n",
    "- **TIME**: The time at the mid-point of the cadence in BKJD (Kepler Barycentric Julian Day).\n",
    "- **TIMECORR**: The barycenter correction calculated by the pipeline plus the time slice correction.\n",
    "- **SAP_FLUX**: The flux in units of electrons per second contained in the optimal aperture pixels collected by the spacecraft.\n",
    "- **SAP_FLUX_ERR**: The error in the simple aperture photometry as determined by PA in electrons per second.\n",
    "- **SAP_BKG**: The total background flux summed over the optimal aperture in electrons per second.\n",
    "- **SAP_BKG_ERR**: The 1-sigma error in the simple aperture photometry background flux.\n",
    "- **PDCSAP_FLUX**: The flux contained in the optimal aperture in electrons per second after the PDC module has applied its co-trending  algorithm.\n",
    "- **PDCSAP_FLUX_ERR **: The 1-sigma error in PDC flux values.\n",
    "- **SAP_QUALITY**: Flags containing information about the quality of the data.\n",
    "- **PSF_CENTR1**: The column centroid calculated by fitting the point-spread function.\n",
    "- **PSF_CENTR1_ERR**: The 1-sigma error in PSF-fitted column centroid.\n",
    "- **PSF_CENTR2**: The row centroid calculated by fitting the point-spread function.\n",
    "- **PSF_CENTR2_ERR**: The 1-sigma error in PSF-fitted row centroid.\n",
    "- **MOM_CENTR1**: The column value for the flux-weighted centroid position of the target at this cadence.\n",
    "- **MOM_CENTR1_ERR**: The 1-sigma error in the column value for the first moment centroid position.\n",
    "- **MOM_CENTR2**: The row value for the flux-weighted centroid position of the target at each cadence.\n",
    "- **MOM_CENTR2_ERR**: The 1-sigma error in the row value for the first moment centroid position.\n",
    "- **POS_CORR1**: The column component of the local image motion calculated from the motion polynomials.\n",
    "- **POS_CORR2**: The row component of the local image motion calculated from the motion polynomials.\n",
    "\n",
    "The light curve files for each star are divided into quarters; thus, the quarters needed to be stitched together. Moreover, since the Flux of electrons came with different sorts of systematic and environmental errors, the flux needed to be normalized and detrended. This step was easily accomplished thanks to PyKE library, which allows stitching quarters together and detrending data. The picture below shows the light curve from the star K-752 before and after the normalization process.\n",
    "![Normalized K752](images/K752.01.png \"Normalized K752\")\n",
    "\n",
    "Subsequently, I needed a way to compare the time series of flux of two stars. Therefore, I ended up using an algorithm called Dynamic Time Warping (DTW), which is widely common for the comparison of two time series [4][5]. The Dynamic Time Warping algorithm finds an optimal match between two sequences of feature vectors which allows for stretched and compressed sections of the sequence. I decided to choose a star that had a regular planetary transit in its light curve as a baseline for comparison.\n",
    "\n",
    "Finally, I created a table with the DTW comparison of the 3 different fluxes and the statistical attributes (mean, std, min, max) of the following: **_SAP_FLUX, SAP_BKG, PDCSAP_FLUX, DETSAP_FLUX, MOM_CENTR1, MOM_CENTR2, POS_CORR1, POS_CORR2 _**. These values provided important features of a light curve data.\n",
    "\n",
    "One thing to be notice is that the DTW algorithm do not always find a comparison value for two time series. Therefore, after processing all the planets, form the 8825 planets processed by the Kepler telescope, 18.03% (1595 planets) is eliminated from the training data for the machine learning implementation.\n",
    "\n",
    "The picture below gives a little bit of intuition on how the DTW algorithm works.\n",
    "\n",
    "\n",
    "![Dynamic Time Warping](images/dynamic-time-warping.jpg \"Dynamic Time Warping\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exploratory Visualization\n",
    "The figure below shows the 7 most relevant features of the data set. I chose seven oust for the sake of visualization. You can see all of them on the implementation section. As you can see, the choice of detrending the data and apply the Dynamic Time Warping algorithm on the light curve, really paid off. Indeed, the most relevant feature is the DTW of the detrended flux, and the second, third and forth most relevant features are the standard deviation, mean, and minimum of the detrended flux. This graph has been generated fitting the entire data to a Random Forrest Classifiers, extracting the importance of each features. In the implementation phase you will see how this plays a role on reducing the number of features. The blue lines represent the standard deviation of each feature.\n",
    "\n",
    "![Features Importance](images/features_importance.png \"Features Importance\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Algorithms and Techniques\n",
    "The algorithms I will use for this project are the most common methods used for classification problems. This includes:\n",
    "- Naive Bayes\n",
    "- Logistic Regression\n",
    "- Random Forest\n",
    "- Support Vector Machines (SVM)\n",
    "\n",
    "\n",
    "I choose Naive Bayes because is a super simple classifier. Indeed, if the conditional independence assumption holds, a Naive Bayes classifier will converge quicker than the other 3 classifiers I have chosen. \n",
    "I choose Logistic Regression because it is a classifier that has a lots of ways to regularize the model, and I don't have to to worry about correlated features, differently from Naive Bayes.\n",
    "I choose Decision Tree because it is a non-parametric classifier and I don't have to worry if the data is not linearly separable.\n",
    "I choose SVM because of it Hight accuracy and theoretical guarantees regarding overfitting.\n",
    "\n",
    "\n",
    "I will use the precision metric to see which one performs best according to the dataset and then I will tune the parameters for that algorithm to improve its performance. I decided to follow this route due to the unpredictability and similarity of the dataset data points. As a matter of fact, each data point is very similar even if the two classes are very distinct (Confirmed, False Positive)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Benchmark\n",
    "Since this is a research project, there are not real clear benchmarks. However, thanks to the Confusion Matrix and K-folds method, some metrics, to test the performance of the different algorithms, are available. I am expecting that the best algorithm will have an precision metric of at least 60%. I choose 60% because I believe that it will be able to fit the data; however; it will not be able to make good predictions due to the fact that the data set is really sensitive to and very close, even if the categories are really different.  Moreover, when applying the model to the still candidate planets, which the researchers are still evaluating, I am expecting to find no more than 5 planets to be confirmed, with a discovery rate lower than 1%. This low margin is due to the fact that human have categorized the dataset of exoplanet as Confirmed or False Negative. Therefore, I do not believe that the algorithm, with that human bias pre-built into it, will able to recognize plates that the humans could not. I believe will find some exoplanets that the human eye could not recognize."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## III. Methodology"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Libraries Import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import Generic libraries\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from time import time\n",
    "from datetime import datetime\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.display import display # Allows the use of display() for DataFrames\n",
    "\n",
    "# Show matplotlib plots inline (nicely formatted in the notebook)\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This first import is needed to add all the libraries for processing the kepler data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import Libraries to process the kepler Data\n",
    "import kplr\n",
    "import pyfits\n",
    "from PyKE.kepstitch import kepstitch\n",
    "from PyKE.kepflatten import kepflatten\n",
    "from algorithms.dtw import LB_Keogh\n",
    "\n",
    "# Import Library for parallel computing\n",
    "import multiprocessing\n",
    "from joblib import Parallel, delayed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This second import is needed to add all the libraries to apply the machine learning algorithm to the processed data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Import SkLearn libraries\n",
    "# Import Metrics\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import precision_score\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from sklearn.metrics import make_scorer\n",
    "\n",
    "# Import Machine learning Algorithms\n",
    "from sklearn.naive_bayes import GaussianNB\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "# Import Cross Validation and Grid Searcch Libraries\n",
    "from sklearn.cross_validation import train_test_split, KFold\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "from sklearn.preprocessing import Normalizer\n",
    "from sklearn.ensemble import ExtraTreesClassifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Preprocessing\n",
    "\n",
    "Processing the Kepler data was the most difficult task of the project. As a matter of fact, each start had approximately 1 GB of information. Moreover, the algorithms used to stitch and detrend the data were of order O(n^2), adding more delay to the process. To process all of the planets, I had to rent a supercomputer with 32 cores through [Domino Data Labs](https://www.dominodatalab.com/). If I would have processed all the data with my computer that has 4 cores, the process would have taken 30 Days. Instead, with Domino's platform, it took approximately 2 days.\n",
    "\n",
    "\n",
    "The first step in the data analysis process was to read the list of planets from the Kepler telescope collected from the Kepler Exoplanet Archive. Subsequently, get some sense of the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of total Planets: 8824, Confirmed: 2288, False Positive: 4120, Candidates: 2416\n"
     ]
    }
   ],
   "source": [
    "# Setting up log filename\n",
    "LOG_FILENAME = 'processing_kepler_data.log'\n",
    "\n",
    "# Read the planets list from the csv file\n",
    "planets = pd.read_csv('data/candidate_planets.csv')\n",
    "\n",
    "# Remove Baseline light curve chosen\n",
    "planets = planets.ix[2:]\n",
    "\n",
    "# Remove the K and transorm to float type allowing KPLR Library to process the name\n",
    "planets['kepoi_name'] = planets['kepoi_name'].str.replace('K', '').astype(float)\n",
    "\n",
    "# Get Some Count of the data\n",
    "confirmed = planets.loc[planets['koi_disposition'] == 'CONFIRMED']\n",
    "false_positive = planets.loc[planets['koi_disposition'] == 'FALSE POSITIVE']\n",
    "candidate = planets.loc[planets['koi_disposition'] == 'CANDIDATE']\n",
    "\n",
    "print \"Number of total Planets: {}, Confirmed: {}, False Positive: {}, Candidates: {}\" \\\n",
    "    .format(planets.shape[0], confirmed.shape[0], false_positive.shape[0], candidate.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step was to create a dataframe that would hold the processed information for each planet. Next step was to collect the information of the baseline light curve chosen prior and get the time series of the **SAP_FLUX, SAP_BKG, PDCSAP_FLUX, DETSAP_FLUX**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Holder DataFrame Created\n",
      "Baselines Variables Created\n"
     ]
    }
   ],
   "source": [
    "# Setting up the resultant dataframe columns name\n",
    "DF_COLUMNS = ('kepoi_name', 'koi_disposition',\n",
    "              'dtw_sap_flux', 'sap_flux_mean', 'sap_flux_std', 'sap_flux_min', 'sap_flux_max',\n",
    "              'dtw_sap_bkg', 'sap_bkg_mean', 'sap_bkg_std', 'sap_bkg_min', 'sap_bkg_max',\n",
    "              'dtw_pd_sap_flux', 'pd_sap_flux_mean', 'pd_sap_flux_std', 'pd_sap_flux_min', 'pd_sap_flux_max',\n",
    "              'dtw_det_sap_flux', 'det_sap_flux_mean', 'det_sap_flux_std', 'det_sap_flux_min', 'det_sap_flux_max',\n",
    "              'mom_centr1_mean', 'mom_centr1_std', 'mom_centr1_min', 'mom_centr1_max',\n",
    "              'mom_centr2_mean', 'mom_centr2_std', 'mom_centr2_min', 'mom_centr2_max',\n",
    "              'pos_corr1_mean', 'pos_corr1_std', 'pos_corr1_min', 'pos_corr1_max',\n",
    "              'pos_corr2_mean', 'pos_corr2_std', 'pos_corr2_min', 'pos_corr2_max')\n",
    "\n",
    "# Creating the Dataframe\n",
    "df = pd.DataFrame(columns=DF_COLUMNS)\n",
    "\n",
    "print \"Holder DataFrame Created\"\n",
    "\n",
    "# Opening the baseline curve FITs File\n",
    "hdu = pyfits.open(\"data/detrended/752.01.fits\")\n",
    "\n",
    "# Grab the data from the file\n",
    "hdu_data = hdu[1].data\n",
    "\n",
    "# Create the baseline time series for SAP_FLUX, SAP_BKG, PDCSAP_FLUX, DETSAP_FLUX\n",
    "baseline_sap_flux = pd.Series(np.array(hdu_data[\"SAP_FLUX\"]))\n",
    "baseline_sap_bkg = pd.Series(np.array(hdu_data[\"SAP_BKG\"]))\n",
    "baseline_pd_sap_flux = pd.Series(np.array(hdu_data[\"PDCSAP_FLUX\"]))\n",
    "baseline_det_sap_flux = pd.Series(np.array(hdu_data[\"DETSAP_FLUX\"]))\n",
    "\n",
    "print \"Baselines Variables Created\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step was to create a function that processed the data for a single planet. This function takes the index of the planets dataframe, the planet dataframe, the output dataframe and the baseline data, processing all the data from a single planet, adding a row to the output dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def processPlanetData(index, planets, df, baseline_sap_flux, baseline_sap_bkg, baseline_pd_sap_flux, baseline_det_sap_flux):\n",
    "    \n",
    "    # Get Kepler Object of Interest name and disposition\n",
    "    kepoi_name = planets.iloc[index]['kepoi_name']\n",
    "    koi_disposition = planets.iloc[index]['koi_disposition']\n",
    "    \n",
    "    # Set Files Names for the resultant Stiched File and Resultant Detrended File\n",
    "    filenameStitch = \"data/stitched/{}.fits\".format(kepoi_name)\n",
    "    filenameDetrend = \"data/detrended/{}.fits\".format(kepoi_name)\n",
    "    \n",
    "    # Initialize kplr API\n",
    "    client = kplr.API()\n",
    "\n",
    "    print \"Processing Kepler Object: {} at index: {}\".format(kepoi_name, index)\n",
    "\n",
    "    # Find a Kepler Object of Interest\n",
    "    koi = client.koi(planets.iloc[index]['kepoi_name'])\n",
    "\n",
    "    # Get a list of light curve data sets.\n",
    "    lcs = koi.get_light_curves(short_cadence=False, fetch=True, clobber=False)\n",
    "\n",
    "    print \"Got Kepler Object of Interest and Light Curve Files for {}\".format(kepoi_name)\n",
    "    \n",
    "    lc_list = \"\"\n",
    "\n",
    "    # Looping trough lcs to get list of light curves path\n",
    "    for lc in lcs:\n",
    "        lc_list += str(lc.filename) + \",\"\n",
    "\n",
    "    # Removing trailing comma\n",
    "    lc_list_clean = lc_list[:-1]\n",
    "\n",
    "    print \"Got path list of light curves.\"\n",
    "\n",
    "    # Stitching together light curves quarters\n",
    "    kepstitch(lc_list_clean, filenameStitch, True, False, LOG_FILENAME, 0)\n",
    "    \n",
    "    print \"Finished Stiching the data for {}\".format(kepoi_name)\n",
    "    \n",
    "    # Detrending Light Curve\n",
    "    kepflatten(filenameStitch, filenameDetrend, \"PDCSAP_FLUX\", \"PDCSAP_FLUX_ERR\",\n",
    "               3.0, 1.0, 3.0, 3, 10, \"0,0\", False, True, False, LOG_FILENAME, 0, True)\n",
    "    \n",
    "    print \"Finished Detrending the data for {}\".format(kepoi_name)\n",
    "    \n",
    "    # Opening Detrended FITs File\n",
    "    hdu = pyfits.open(filenameDetrend)\n",
    "    \n",
    "    # Getting Detrended Data\n",
    "    hdu_data = hdu[1].data\n",
    "    \n",
    "    # Getting all features form the Fit Files\n",
    "    sap_flux = pd.Series(np.array(hdu_data[\"SAP_FLUX\"]))\n",
    "    sap_bkg = pd.Series(np.array(hdu_data[\"SAP_BKG\"]))\n",
    "    pd_sap_flux = pd.Series(np.array(hdu_data[\"PDCSAP_FLUX\"]))\n",
    "    det_sap_flux = pd.Series(np.array(hdu_data[\"DETSAP_FLUX\"]))\n",
    "    mom_centr1 = pd.Series(np.array(hdu_data[\"MOM_CENTR1\"]))\n",
    "    mom_centr2 = pd.Series(np.array(hdu_data[\"MOM_CENTR2\"]))\n",
    "    pos_corr1 = pd.Series(np.array(hdu_data[\"POS_CORR1\"]))\n",
    "    pos_corr2 = pd.Series(np.array(hdu_data[\"POS_CORR2\"]))\n",
    "    \n",
    "    # Try to apply DTW with the baseline. If fails return NaN\n",
    "    try:\n",
    "        dtw_sap_flux = LB_Keogh(sap_flux, baseline_sap_flux, 10)\n",
    "        dtw_sap_bkg = LB_Keogh(sap_bkg, baseline_sap_bkg, 10)\n",
    "        dtw_pd_sap_flux = LB_Keogh(pd_sap_flux, baseline_pd_sap_flux, 10)\n",
    "        dtw_det_sap_flux = LB_Keogh(det_sap_flux, baseline_det_sap_flux, 10)\n",
    "    except:\n",
    "        dtw_sap_flux, dtw_sap_bkg, dtw_pd_sap_flux, dtw_det_sap_flux = 'nan', 'nan', 'nan', 'nan'\n",
    "        pass\n",
    "    \n",
    "    print \"dtw_sap_flux: {}, dtw_sap_bkg: {}, dtw_pd_sap_flux: {}, dtw_det_sap_flux: {}\".format(\n",
    "        dtw_sap_flux, dtw_sap_bkg, dtw_pd_sap_flux, dtw_det_sap_flux)\n",
    "    \n",
    "    # Describe the Features Extracted (STD, MEAN, MIN, MAX)\n",
    "    desc_sap_flux, desc_sap_bkg = sap_flux.describe(), sap_bkg.describe()\n",
    "    desc_pd_sap_flux, desc_det_sap_flux  = pd_sap_flux.describe(), det_sap_flux.describe()\n",
    "    desc_mom_centr1, desc_mom_centr2 = mom_centr1.describe(), mom_centr2.describe()\n",
    "    desc_pos_corr1, desc_pos_corr2 = pos_corr1.describe(), pos_corr2.describe()\n",
    "    \n",
    "    print \"Features Described Correctly.\"\n",
    "    \n",
    "    # Apply the Features to the dataframe\n",
    "    df.loc[index] = [kepoi_name, koi_disposition,\n",
    "                     dtw_sap_flux, desc_sap_flux['mean'], desc_sap_flux['std'], desc_sap_flux['min'], desc_sap_flux['max'],\n",
    "                     dtw_sap_bkg, desc_sap_bkg['mean'], desc_sap_bkg['std'], desc_sap_bkg['min'], desc_sap_bkg['max'],\n",
    "                     dtw_pd_sap_flux, desc_pd_sap_flux['mean'], desc_pd_sap_flux['std'], desc_pd_sap_flux['min'], desc_pd_sap_flux['max'],\n",
    "                     dtw_det_sap_flux, desc_det_sap_flux['mean'], desc_det_sap_flux['std'], desc_det_sap_flux['min'], desc_det_sap_flux['max'],\n",
    "                     desc_mom_centr1['mean'], desc_mom_centr1['std'], desc_mom_centr1['min'], desc_mom_centr1['max'],\n",
    "                     desc_mom_centr2['mean'], desc_mom_centr2['std'], desc_mom_centr2['min'], desc_mom_centr2['max'],\n",
    "                     desc_pos_corr1['mean'], desc_pos_corr1['std'], desc_pos_corr1['min'], desc_pos_corr1['max'],\n",
    "                     desc_pos_corr2['mean'], desc_pos_corr2['std'], desc_pos_corr2['min'], desc_pos_corr2['max']\n",
    "                     ]\n",
    "    \n",
    "    print \"Dataframe Row Added Correctly.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, I have implemented the parallel function that would process all the planet. For the sake of this notebook, I set up the number of core equal to 1, since not all the computers would be able to parallel compute this data. Moreover, since to process each planet it takes approximately 3 minutes, I set up a limit of 2 planets to be processed.\n",
    "\n",
    "Lastly, the dataframe is converted to a csv file, to be used later to apply Machine Learning, and the head of the dataframe is presented.\n",
    "\n",
    "If it throw a PyfitsDeprecationWarning, do not worry, it is just an old library I could not replace. The code will continue running anyway."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing Kepler Object: 753.01 at index: 0\n",
      "Got Kepler Object of Interest and Light Curve Files for 753.01\n",
      "Got path list of light curves.\n",
      "Finished Stiching the data for 753.01\n",
      "Finished Detrending the data for 753.01\n",
      "dtw_sap_flux: 447021.169311, dtw_sap_bkg: 59369.8402509, dtw_pd_sap_flux: 473813.686358, dtw_det_sap_flux: 0.113722836148\n",
      "Features Described Correctly.\n",
      "Dataframe Row Added Correctly.\n",
      "Processing Kepler Object: 754.01 at index: 1\n",
      "Got Kepler Object of Interest and Light Curve Files for 754.01\n",
      "Got path list of light curves.\n",
      "Finished Stiching the data for 754.01\n",
      "Finished Detrending the data for 754.01\n",
      "dtw_sap_flux: 588094.240371, dtw_sap_bkg: 65419.5889482, dtw_pd_sap_flux: 700256.139115, dtw_det_sap_flux: 0.282884093064\n",
      "Features Described Correctly.\n",
      "Dataframe Row Added Correctly.\n",
      "   kepoi_name koi_disposition   dtw_sap_flux  sap_flux_mean  sap_flux_std  \\\n",
      "0      753.01       CANDIDATE  447021.169311    7168.718262    445.189331   \n",
      "1      754.01       CANDIDATE  588094.240371    6479.114746    505.689178   \n",
      "\n",
      "   sap_flux_min  sap_flux_max   dtw_sap_bkg  sap_bkg_mean  sap_bkg_std  \\\n",
      "0   5996.772461   7949.610352  59369.840251    719.355713   120.941109   \n",
      "1   5271.307129   7513.387207  65419.588948    611.335205   151.375443   \n",
      "\n",
      "       ...        mom_centr2_min  mom_centr2_max  pos_corr1_mean  \\\n",
      "0      ...            988.572714      992.750857       -2.145796   \n",
      "1      ...            544.901977      548.947253       -3.409783   \n",
      "\n",
      "   pos_corr1_std  pos_corr1_min  pos_corr1_max  pos_corr2_mean  pos_corr2_std  \\\n",
      "0      25.815355    -313.590393       0.080978       -6.804382      81.955376   \n",
      "1      38.672947    -442.179382       0.041764       -4.218824      47.850662   \n",
      "\n",
      "   pos_corr2_min  pos_corr2_max  \n",
      "0    -993.573730       0.164598  \n",
      "1    -549.932739       0.135372  \n",
      "\n",
      "[2 rows x 38 columns]\n"
     ]
    }
   ],
   "source": [
    "# Setting the Number of cores in the computer (set up to 1 due to the fact that not all computers allow multiprocessing)\n",
    "num_cores = 1\n",
    "# Uncomment if you want to try multiprocessing\n",
    "# num_cores = multiprocessing.cpu_count()\n",
    "\n",
    "# Setting Up the Limit for notebook purpose\n",
    "LIMIT = 2\n",
    "\n",
    "Parallel(n_jobs=num_cores) \\\n",
    "    (delayed(processPlanetData) \\\n",
    "    (index, planets, df, baseline_sap_flux, baseline_sap_bkg, baseline_pd_sap_flux, baseline_det_sap_flux) \\\n",
    "     for index in range(0, LIMIT))\n",
    "\n",
    "df.to_csv(\"data/kepler_pre_ml_notebook.csv\")\n",
    "\n",
    "print df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Implementation\n",
    "The first step toward the machine learning analysis was to read the data processed in the previous step. Subsequently, since some values were NaN, I had to clean up the exoplanet data. The dataset reduced in size of 18.07%, dropping 1595 planets. Moreover, some useful information about the data can be found below.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Exoplanets data read successfully!\n",
      "\n",
      "Number of exoplanetes dropped: 1595\n",
      "Number of confirmed exoplanetes dropped: 359\n",
      "Number of false positive exoplanetes dropped: 795\n",
      "Number of candidate exoplanetes dropped: 441\n",
      "Dropping Rate: 18.07%\n",
      "\n",
      "Total number of exoplanets: 7230\n",
      "Number of features: 36\n",
      "Number of confirmed exoplanets: 1930\n",
      "Number of false positive exoplanetes: 3325\n",
      "Number of candidate exoplanetes: 1975\n",
      "Number of confirmed plus false positive exoplanetes: 5255\n",
      "Discovery rate of the data: 26.69%\n"
     ]
    }
   ],
   "source": [
    "# Read Exoplanet data\n",
    "exoplanets_data_raw = pd.read_csv(\"data/kepler_pre_ml.csv\")\n",
    "print \"Exoplanets data read successfully!\"\n",
    "\n",
    "# Remove NAN raws => Corrupted data\n",
    "exoplanets_data = exoplanets_data_raw.dropna(axis=0, how='any')\n",
    "\n",
    "# Calculate number of exoplanets\n",
    "n_exoplanets = exoplanets_data.shape[0]\n",
    "\n",
    "# Calculate number of features\n",
    "n_features = exoplanets_data.shape[1] - 3 # minus index, koi_disposition and kepoi_name\n",
    "\n",
    "# Calculate confirmed exoplanets\n",
    "n_confirmed = (exoplanets_data['koi_disposition'] == 'CONFIRMED').sum()\n",
    "\n",
    "# Calculate false positive exoplanets\n",
    "n_false_positive = (exoplanets_data['koi_disposition'] == 'FALSE POSITIVE').sum()\n",
    "\n",
    "# Calculate candidate exoplanets\n",
    "n_candidate = (exoplanets_data['koi_disposition'] == 'CANDIDATE').sum()\n",
    "\n",
    "# Calculate dropped exoplanets\n",
    "n_dropped = exoplanets_data_raw.shape[0] - exoplanets_data.shape[0]\n",
    "\n",
    "# Calculate dropped confirmed exoplanets\n",
    "n_dropped_confirmed = ((exoplanets_data_raw['dtw_sap_flux'].isnull()) \n",
    "                       & (exoplanets_data_raw['koi_disposition'] == 'CONFIRMED')).sum()\n",
    "\n",
    "# Calculate dropped false positive exoplanets\n",
    "n_dropped_false_positive = ((exoplanets_data_raw['dtw_sap_flux'].isnull()) \n",
    "                            & (exoplanets_data_raw['koi_disposition'] == 'FALSE POSITIVE')).sum()\n",
    "\n",
    "# Calculate dropped candidate exoplanets\n",
    "n_dropped_candidate = ((exoplanets_data_raw['dtw_sap_flux'].isnull()) \n",
    "                       & (exoplanets_data_raw['koi_disposition'] == 'CANDIDATE')).sum()\n",
    "\n",
    "# Calculate Dropping Rate\n",
    "dropping_rate =  (float(n_dropped)/exoplanets_data_raw.shape[0])*100\n",
    "\n",
    "# Calculate discovery rate\n",
    "discovery_rate = (float(n_confirmed)/(n_exoplanets)*100)\n",
    "\n",
    "# Print the results\n",
    "print \"\\nNumber of exoplanetes dropped: {}\".format(n_dropped)\n",
    "print \"Number of confirmed exoplanetes dropped: {}\".format(n_dropped_confirmed)\n",
    "print \"Number of false positive exoplanetes dropped: {}\".format(n_dropped_false_positive)\n",
    "print \"Number of candidate exoplanetes dropped: {}\".format(n_dropped_candidate)\n",
    "print \"Dropping Rate: {:.2f}%\".format(dropping_rate)\n",
    "\n",
    "print \"\\nTotal number of exoplanets: {}\".format(n_exoplanets)\n",
    "print \"Number of features: {}\".format(n_features)\n",
    "print \"Number of confirmed exoplanets: {}\".format(n_confirmed)\n",
    "print \"Number of false positive exoplanetes: {}\".format(n_false_positive)\n",
    "print \"Number of candidate exoplanetes: {}\".format(n_candidate)\n",
    "print \"Number of confirmed plus false positive exoplanetes: {}\".format(n_confirmed + n_false_positive)\n",
    "print \"Discovery rate of the data: {:.2f}%\".format(discovery_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The next step was to choose the number of features and divide the dataset from those that are still candidate exoplanets from the already classified as confirmed or false positive. To perform feature selection, I have applied a Random Forest Classifier to each feature. The plot below shows the relevance of each feature.\n",
    "\n",
    "Finally, I reduced the number of feature by 25%, eliminating some computational cost."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Target column: koi_disposition\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABBMAAAF6CAYAAABLBRBoAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xu4JOddH/hva8bClu2xGC6yLQ2MH1sm9sYx5gF5dtdG\nDVaINEskCCGghIvkJ1hJVjaBQITYJDra7OKFLMFWHBQFy37EVQRjE3tjEDikBQlEvkmyQRckY8W6\nYOGLbAs5Fwmf/eOto9PT0zXd7zlddbp7Pp/n6ef0pbq+Vd11qrt+/db7JgAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAHTkyiQ/s9cLAQAAACeL+5J8PsmjzeVzSZ69gHl+4y7nsao2kvzcXi8E\nAJzsTtnrBQCANbeZ5JuTPLO5HEjy8QXMc7CL5+/bZf5e2b/XCwAAAAB9+GimtyJ4VpLrkzyU5IEk\n/zTbRf7nJ/ntJJ9M8okkP99Mn5Rf5f88260dfijJMMn9E/O/byx3I8nbmud+NsmrZ+RP2sh2a4DD\nSb6Q5JIkH0vyqSR/J8nXJflQkkeS/Iux516S5D81930myZ059vV4bpJ3NvO5J8nfnsgdX+7/Pcl/\nT/I/mnW/tZnu0iR3pLT6+EiS14zNY9is3w8mebhZ30vGHn9akp9Meb0+k+R3kzy1eexIkt9r1um2\nJOdOrNdHmsw/TvI3AwAAAAvy0SSvmnL/O5Jcm3Iw+2VJbsn2QfDzm+c8JcmXJrk5yU9NzHP8gHyY\n44sJ49NspByAX9jcfuqM/ElX5fhiwk8nOTXJX045wH9Hs6zPTTlo//pm+kuSPJ7k+1NaRPyNlIP2\n05vHfyfJm5p5vTTJnyb5hhMs91VJfnZi+Y4meV5z/euTPJbkZc3tYZO/0eRf0Dy+VZz5lymFm+ek\nFFOONMtyZkox5/xmuvOa21+S5OkpxY2zm8fOSPLiAAAAwILcl/Ir+iPN5e0pB5//Ldu/gCfJxSkH\ntdN8S5IPjt3eSTFhNPZYbf5Gji8mPGfs8U8m+fax229LKR4kpZjw4MT8bknyXUkOJXki5eB8y48l\neWvLck8uS5t3JHldc32Y0opjvNXFw0nOae77fJKXTJnHFTm+aPEbSb4nyWkp7+VfSynGAMBJx7mH\nANCtzSQX5dgD9XNSWh38ydh9p6ScNpCUg/03JnlFSj8LpyT59C6X44Gx6185I38eD49d/69Tbo8X\nCCaLCf8lpRjxnJT1emzssY8l+dqW5W5zQUqLhbNT1uO0lFMutnwqpQCy5fNJnpHSkuKpKacrTPrK\nlALJXx27b3/K+/j5JN+RcorJ9SmncfyDJHfPsawAsBZ0wAgA/bs/5dSAL0nyxc3lWdn+hfzHUvpF\n+IvN/d+dYz+zNyfm91jKAfSWfSmnLowbf86s/EmTebXOnLj9lSl9FzyU5GDKgf2Wr8ixBYTJ7C9M\n3P6iJL+a5CeSfHnKurw783VQ+cmUFhovmPLYx1JaQHzx2OWZTU6S/GaSb0oZmeOuGDoTgJOMYgIA\n9O9PUg5G/3m2Wx48P9v9DDwjpUDwuZQD8R+eeP7DzfRb/ijlF/ajKS0O/lHKQfZO8yftZOSI8ed8\necppB09J+bX/L6Qc8D+Q0sHh65vl/UspnUP+/Anm+3DKqRZb8z+1uXwypdBwQcpB/jy+kOQtKa/D\nc1KKMP9zM7+fT2mV8E3N/U9NOWXizGZ9LkppffF4ynv153NmAsBaUEwAgL3xPSkHrXekNPX/lZRf\nuZPk6iRfk9LJ37tSfnkf/4X+9SkFg0dSRin4bJK/l+TNKQfof5Zj+1DYzPG/8J8of9Lk8+dpqTA+\nzS0ppyB8ImXUiG9rlj0pfTUcTmml8PYk/yTbp4RMW+5faf5+Ksn7U/qjeF2Sf9Osx8VJ/u0JlmXS\nDyX5cJL3NfN8fcr3owdSCgY/mtIp5MdSTmUYNI//QMrpG59K8sokf/cEGQBwUjo/pfnePSmdEU1z\nTfP47dnuPTlJrkzyhykf0r+YE/9KAgCsn0tShlsEAE4i+5Lcm/KLwVNSxlh+0cQ0R1OaKibJy5P8\n5+b64ZRxl7cKCL+c5Hu7W1QAYAldEsUEAFg7s05zOCelmHBfyjmBN6Y0+Rt3YZIbmuu3pIwbfUbK\neZ6Pp3QItb/5O9mbMwCw3qadqgAArLhZxYQzc+w5lw/k+B6Z26b5dJKfTDnH8KEkn0nynt0sLACw\ncm5Ie8eOAMCKmlVMmPeXhGm9PD8/yd9POd3huSk9U/+tuZcMAAAAWEr7Zzz+YJJDY7cP5dixn6dN\nc1Zz3zBluKdPNfe/Pcn/kuQXxp/80pe+dPP222+vWmgAAACgF7cn+erJO2e1THh/ylBOh1OGj/qO\nJO+cmOadKcNLJcmRlNMZHk5yd3P7aSktF85LGX7q2KW6/fZsbm72crnqqqt6y+o7T5asZcha53WT\ntVpZ67xusmQtS54sWcuQtc7rJkvWsuQleem0YsGslglPJLk8yU0pIztcn+TOJJc1j1+XMpLD0ZSO\nGh9Lcmnz2G1JfrYpSHwhyQeT/OsZeQAAAMCSm1VMSJJfby7jrpu4fXnLc3+iuQAAAABrYt9eL0CS\njY2Njd7CDh8+3FtW33myZC1DVt95smQtS54sWcuQ1XeeLFnLkNV3nixZy5DVZ97VV1+dJFdP3j9t\nFIa+bTbnYQAAAABLZDAYJFNqB7M6YAQAAAA4hmICAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigm\nAAAAAFUUEwAAAIAqigkAAABAFcUEAAAAoIpiAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAA\nAABVFBMAAACAKooJAAAAQJX9e70AXRuNymXr+nBYrg+H29cBAACA+Q32egGSbG5ubvYSNBgkPUUB\nAADAyhsMBsmU2oHTHAAAAIAqigkAAABAFcUEAAAAoIpiAgAAAFBFMQEAAACosvZDQ/bNUJQAAACs\nO0NDrlEeAAAALJKhIQEAAICFUEwAAAAAqigmAAAAAFUUEwAAAIAqigkAAABAFcUEAAAAoMo8xYTz\nk9yV5J4kV7RMc03z+O1JXtbc91VJbh27fDbJ63azsAAAAMDeO26syAn7ktyd5LwkDyZ5X5KLk9w5\nNs3RJJc3f1+e5I1JjkzM55Tm+eckuX/isc3Nzc2dLHu1wSDpKWpP8gAAAGCRBoNBMqV2MKtlwjlJ\n7k1yX5LHk9yY5KKJaS5MckNz/ZYkpyc5Y2Ka85J8JMcXEgAAAIAVM6uYcGaOLQA80Nw3a5qzJqb5\nziS/uJMFBAAAAJbLrGLCvI30J5s8jD/v1CR/NcmvzLtQAAAAwPLaP+PxB5McGrt9KKXlwYmmOau5\nb8sFST6Q5BNtIRsbG09eHw6HGQ6HMxYLAAAAWLTRaJTRaDRzulkdMO5P6YDxVUkeSvLenLgDxiNJ\n3pBjO2C8McmvZ7tfhUk6YAQAAIAl1NYB46yWCU+kFApuShnZ4fqUQsJlzePXJXl3SiHh3iSPJbl0\n7PlPT+l88ft2vugAAADAMpnVMqEPWiYAAADAEtrp0JAAAAAAx1BMAAAAAKooJgAAAABVFBMAAACA\nKooJAAAAQBXFBAAAAKCKYgIAAABQRTEBAAAAqKKYAAAAAFRRTAAAAACqKCYAAAAAVRQTAAAAgCqK\nCQAAAEAVxQQAAACgimICAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigmAAAAAFUUEwAAAIAqigkA\nAABAFcUEAAAAoIpiAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAAAABVFBMAAACAKooJAAAA\nQBXFBAAAAKDK/r1eAHZuNCqXrevDYbk+HG5fBwAAgEUb7PUCJNnc3NzsJWgwSHqK6j2v73UDAABg\n/Q0Gg2RK7cBpDgAAAECVeYoJ5ye5K8k9Sa5omeaa5vHbk7xs7P7Tk7wtyZ1J7khyZMdLCgAAACyF\nWcWEfUnelFJQeHGSi5O8aGKao0lekOTsJK9Jcu3YY29M8u7mOX8ppagAAAAArLBZxYRzktyb5L4k\njye5MclFE9NcmOSG5votKa0RzkjyrCSvTPKW5rEnknx210sMAAAA7KlZxYQzk9w/dvuB5r5Z05yV\n5HlJPpHkrUk+mORnkpy2m4UFAAAA9t6sYsK84wNM9uy4mTLs5Nck+enm72NJfqRq6QAAAICls3/G\n4w8mOTR2+1BKy4MTTXNWc9+gmfZ9zf1vS0sxYWNj48nrw+Eww+FwxmIBAAAAizYajTIajWZOd9xY\nkRP2J7k7yauSPJTkvSmdMI53pHg0yeXN3yNJ3pDtURt+J8nfTvJHSTaSPC3Hjwixubk5bwOI3RkM\nkp6ies/re90AAABYf4PBIJlSO5jVMuGJlELBTSkjO1yfUki4rHn8upTRGo6mdNT4WJJLx57/2iS/\nkOTUJB+ZeAwAAABYQbNaJvRBy4QVywIAAODk0NYyYVYHjAAAAADHUEwAAAAAqigmAAAAAFUUEwAA\nAIAqigkAAABAFcUEAAAAoIpiAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAAAABVFBMAAACA\nKooJAAAAQJX9e70ArIbRqFy2rg+H5fpwuH0dAACAk8Ngrxcgyebm5mYvQYNB0lNU73nrmgUAAMDe\nGQwGyZTagdMcAAAAgCqKCQAAAEAVxQQAAACgimICAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigm\nAAAAAFUUEwAAAIAqigkAAABAFcUEAAAAoMpaFBMOHjiQwWAw85JkrukGg0EOHjiwx2sFAAAAy2mw\n1wuQZHNzc3NXMxgMBplnDoNsZnPOVR6UBdvlciW7nMVJnwUAAMDeaX6YP+5Aei1aJgAAAAD9UUwA\nAAAAqigmVJq3f4aaPhr0zwAAAMAq0WdC67TT+0yYN6smT/8MAAAALCN9JgAAAAALMU8x4fwkdyW5\nJ8kVLdNc0zx+e5KXjd1/X5IPJbk1yXt3vJQAAADA0tg/4/F9Sd6U5LwkDyZ5X5J3JrlzbJqjSV6Q\n5OwkL09ybZIjzWObSYZJPr2wJQYAAAD21KyWCeckuTelhcHjSW5MctHENBcmuaG5fkuS05OcMfb4\nMvTLAAAAACzIrGLCmUnuH7v9QHPfvNNsJnlPkvcn+b6dLyYAAACwLGad5jD/wAXTvSLJQ0m+LMlv\npfS98LtzzhMAAABYQrOKCQ8mOTR2+1BKy4MTTXNWc19SCglJ8okk70g5beK4YsLGxsaT14fDYYbD\n4YzFAgAAABZtNBplNBrNnG5Wfwb7k9yd5FUphYH3Jrk4x3fAeHnz90iSNzR/T0vpwPHRJE9P8ptJ\nrm7+jtvc3Jy3AUTLSgwGczWhGGQzm3N24TAoC7bjrJq8tqwag0Gyy1ksZRYAAAB7ZzAYJFNqB7Na\nJjyRUii4KaUwcH1KIeGy5vHrkrw7pZBwb5LHklzaPPbsJG8fy/mFHF9IAAAAAFbMMoy0oGVCS1YN\nLRMAAABYtLaWCbNGcwAAAAA4hmICAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigmAAAAAFUUEwAA\nAIAqigkAAABAFcUEAAAAoIpiAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAAAABVFBMAAACA\nKooJAAAAQBXFBAAAAKCKYgIAAABQRTEBAAAAqKKYAAAAAFRRTFhiBw8cyGAwmOuSZO5pDx44sMdr\nBgAAwCob7PUCJNnc3Nzc1QwGg0HmmcMgm9mcc5UHZcF2nFWT12fWifLmNRgku3zLAAAAWAHNj9fH\nHWxqmQAAAABUUUwAAAAAqigmAAAAAFX27/UCwKTRqFy2rg+H5fpwuH0dAACAvaMDxtZpdcDY+vwe\nO2DU2SMAAMDe0QEjAAAAsBCKCQAAAEAVxQQAAACgimICAAAAUEUxAQAAAKiimAAAAABUmaeYcH6S\nu5Lck+SKlmmuaR6/PcnLJh7bl+TWJO/a4TICAAAAS2RWMWFfkjelFBRenOTiJC+amOZokhckOTvJ\na5JcO/H49ye5I8nmbhcWAAAA2HuzignnJLk3yX1JHk9yY5KLJqa5MMkNzfVbkpye5Izm9lkpxYY3\nJxnsfnEBAACAvTarmHBmkvvHbj/Q3DfvND+V5IeTfGEXywgAAAAskf0zHp/31ITJVgeDJN+c5E9T\n+ksYnujJGxsbT14fDocZDk84OR04eOBAHnn00Tmn3sxgMF9Dky9+5jPz6c99bucLBgAAQG9Go1FG\no9HM6WYdER5JspHSZ0KSXJnSyuDHx6b5V0lGKadAJKWzxmGS1yX57iRPJHlqkgNJfjXJ90xkbG5u\n7q47hcFgMFfVY5DNbM55tsWgLNiOs2ry+sxqy+szq8ZgkOxy8wAAAGCHmh+SjzsAnHWaw/tTOlY8\nnOTUJN+R5J0T07wz2wWCI0k+k+TjSX40yaEkz0vynUl+O8cXEgAAAIAVM+s0hyeSXJ7kppSRHa5P\ncmeSy5rHr0vy7pROFu9N8liSS1vm5fdlAAAAWAPLMMKC0xx6zGrLc5oDAAAAk3Z6mgMAAADAMRQT\nAAAAgCqz+kyAtTYalcvW9a1RSYfD7esAAAAcS58JrdPqM6GrrBp99pmgfwYAAIBj6TMBAAAAWAjF\nBAAAAKCKYgIAAABQRTEBAAAAqKKYAAAAAFRRTAAAAACqKCYAAAAAVRQTAAAAgCqKCfTu4IEDGQwG\nc12SzD3twQMH9njNAAAATg6DvV6AJJubm5u7msFgMMg8cxhkM5tzrvKgLNiOs2ry+sxqy1vXrBqD\nQbLLTREAAGCtND/yHndQpmUCAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigmAAAAAFX27/UCAIs3\nGpXL1vXhsFwfDrevAwAA7JShIVunNTTkOmTVWNehIdd1vQAAgO4ZGhIAAABYCKc5QE+cegAAAKwL\npzm0Tus0h3XIqtHn6QDrmgUAAKwXpzlwUjp44EAGg8FclyRzT3vwwIE9XjMAAIC94zQH1tojjz5a\n0Qoi80/76KM7XCIAAIDVp2UCAAAAUEUxAQAAAKjiNAdg14xUAQAAJxejObROazQHWXVZNdZ5NIeu\n8/osXCiSAABwsmsbzUExoXVaxQRZdVk1FBNkAQDAKjA0JAAAALAQ8/SZcH6SNyTZl+TNSX58yjTX\nJLkgyeeTXJLk1iRPTXJzki9KcmqSf5vkyl0vMcAaWtfTN5wqAgCwnma16d6X5O4k5yV5MMn7klyc\n5M6xaY4mubz5+/Ikb0xypHnstJQCw/4k/zHJDzV/xznNocestjxZTnNYxTxZsgAA6FbbaQ6zWiac\nk+TeJPc1t29MclGOLSZcmOSG5votSU5PckaSh1MKCUlpmbAvyaerl3yXRjk3owyTJOdmlI1clSQZ\nZpRhbu57cQDokJYQAAD9mFVMODPJ/WO3H0hpfTBrmrNSign7knwgyfOTXJvkjt0s7E4Mc/NY0eDq\nvuMB6NF40WAw2C4sAACwWLOKCfO3EJ/+vD9P8tVJnpXkpiTDJKM557mStIQAODmsayuIdV0vAGCx\nZhUTHkxyaOz2oZSWByea5qzmvnGfTfLvknxtphQTNjY2nrw+HA4zXOFvK1pCAJwc1rUVxLquFwAw\nn9FolNEcXwBm9Ta3P6UDxlcleSjJe3PiDhiPpIz8cCTJlyZ5IslnkjwtpWXC1Un+/URGbx0wVs0z\nu+8Usc+sVeqocF2zaqxzp3frum6yViur77yus/aqtYBOMwGAtg4Y5zlyuiDbQ0Nen+T1SS5rHruu\n+fumlCEkH0tyaZIPJnlJSseMpzSXn0vyz6bMXzFhAVmrdNC9rlk11ukgZy/zZMlaljxZAMC62k0x\noWuKCQvIWqWD7nXNqrHOBwPrum6yViur7zxZO7Ou/TOs63ol/a7bOr+OAKtEMaF2nlFM2Kus8U4s\nRxlm2HSzMasTy71erxrrdDCwl3myZC1Lnqzlz1rnU0XWed32IqtriiTAqlFMqJ1nFBOWIavGXq/X\nwQMH8sijj86Ztpl5//2++JnPzKc/97k553s8B3CyTsasvvNkyVqGrL7z1jWrT+u6XsB6aSsmzBrN\nAZjTI48+WjWW6tzTzl2gAAB2QmsBgHpaJrTNM1omLENWjVVar3U9paLvPFmyliVPlqxlyOo7T9by\nZymSAIvgNIfaeUYxYRmyaqzSei2imDD/aRX9nVKRrNeXMFmrm9V3nixZy5DVd54sWeMULmB9KSbU\nzjOKCcuQVWMnWavc2eO8eX22gijLtT5fjGStblbfebJkLUNW33myZO1lluIF9EcxoXaeUUxYhqwa\nq5SlmLAYsmQtS54sWcuQ1XeeLFnLkNVHnsIFJzvFhNp5RjFhGbJqrFLWqhUTuhipwikVstYpT5as\nZcjqO0+WrGXI6juv73WDZWA0B2DHuhipwigVAACwuk7Z6wUAAAAAVotiArBUDh44kMFgMNclydzT\nHjxw4KTJAgCArukzoW2e0WfCMmTVWKWsVeszoYt1s913l9VFHxfJ7vu5cA6tLFnrlSdL1jJk9Z2n\nzwRORvpMWEPjwxqem1E2clWS2cMaAuutiz4ukvZ+LuYvXmw+2fJilrbCRW2hZJ68RXQGCgCLsK4j\nR6zrep3stExom2dWq2XCbvOW9RfaGquUpWWClgnrklWTt2rbYo11/VVM1mpl9Z0nS9YyZPWdt05Z\ne3WAr3XH6jE0ZO08o5jQ/nzFhL1+DWvyVu0Ablm3RVkn37ZYY52+XMpa3ay+82TJWoasvvNkLX+W\nVhCLp5hQO88oJrQ/XzFhr1/DmrxVO4Bb1m1R1sm3Lep7QtaqZfWdJ0vWMmT1nSdL1riTpXChmFA7\nzygmtD9fMWGvX8OavFU7gFvWbVGWbXERWW15fRYuZO0+q4aDKlmy1itPlqy9zNq7U1MUE+rmGcWE\n9ucrJuz1a1iT5wCuPU+WbbHvrLY8WauVldR1PNpX4SJZvy/OsmQtW54sWcuQ1XdeWzHhlH7iAQDW\nx9aoKbMumWOarUtbceLggQMZDAZzXZLMNd3BAwd6y2rLW9csgJOFoSEBAJZYF8O9tg312ufQsuua\n1cUQtolhbIHlo5jAXEY5N6MMkyTnZpSNXJUkGWaUYW7ewyUDAFgefRYuAPaSPhPa5hl9JshabNZ4\nQWaUYYYZJZldkHGe+u6z2vJk6TOh76y2PFmrlVWTZ7tvz5O1+6y+Ox7VV4gsWcuR1XeeDhhr5xnF\nBFnLm1WT54tse54sB1V9Z7XlyVqtrJo82317nqzVyqrJW0RWjXU9YJQla1ny2ooJOmAEAAAAqigm\nAAAAAFUUEwAAgKXS55CowM4YzQEAAFgqfQ6JCuyMYgKwa4YOBQCAk4vRHNrmGaM5yFrerJq8vnsS\nn5eexNcjqyZPr/btebJWK6smz3bfnidrtbJq8lZtu+97iM15retIBLJWL69tNActEwAAgJNWF6dU\nJE6rYP0pJsAactrB6vGeAcD6q20FsdXB5CzTWkH0mcXJyWkObfPMajX33m2erNXK6ipvFbLGD7pH\nGWaYUZLZB907ec/6zNqpvV6vefK2n999s9edrJvm3rJqs2ryVq2597q+Z7Js931nteWta1afp4o4\nLaX/vLbTHBQT2uaZ5T+oWmSerNXK6ipvXbPa8mTtPmtW3l4VLmr4cimrNqsmz0FVe56s1cqqybPd\nt+fJWq2sGooJ7c5P8oYk+5K8OcmPT5nmmiQXJPl8kkuS3JrkUJKfTfLlKWWhf91MN04xocestjxZ\nq5XVVd66ZrXlydp9Vld5q5a1Sl+MlilrVVsazZO3/XwHVW15slYrqybPdt+eJ2u1spKalhCbmffw\nuq0VRBetLhZxWspuign7ktyd5LwkDyZ5X5KLk9w5Ns3RJJc3f1+e5I1JjiR5dnO5LckzknwgybdM\nPFcxocestjxZq5XVVd66ZrXlydp9Vld5q5a1Sl+MljWrxl5n1eQ5qGrPk7VaWTV5tvv2PFmrlVWT\nt2rbfY22YsI8HTCek+TeJPc1t29MclGOLQhcmOSG5votSU5PckaSjzeXJPmz5jnPnXguAKycPjvN\n1EHnavF+rR7vGUC9eUonfz3JX0nyfc3t70ppffDasWneleT1SX6vuf2eJFektETYcjjJzUn+p5TC\nwhYtE3rMasuTtVpZXeWta1ZbnqzdZ3WVt65ZbXnLmqVlwmpsizoeXWxWjVVaL7/Q2u5l1WfV5K3a\ndl9jNy0T5l+f9uc9I8nbknx/ji0kAABLxC+0q2eYm8fem6v3dFmYzf8YsC7mKSY8mNKR4pZDSR6Y\nMc1ZzX1J8pQkv5rk55P82rSAjY2NJ68Ph8MMh8M5FgsAWDQHppyI03t2r8//Me8XsBOj0Sij0Wjm\ndPO0w9if0gHjq5I8lOS9OXEHjEdSRn440sz/hiSfSvIDLfN3mkOPWW15slYrq6u8dc1qy5O1+6yu\n8tY1qy1P1mpldZW3rlltebJWK6urvGVo7r2uTfRlOc3hRFk1dnOawxMphYKbUkZ2uD6lkHBZ8/h1\nSd6dUki4N8ljSS5tHvtfU/pY+FDKUJFJcmWS39jBOgAAAABLYL7SSbe0TOgxqy1P1mpldZW3rllt\nebJ2n9VV3rpmteXJWq2srvLWNastT9ZqZXWVtwy/0K7rr+qytEw4UVaNtpYJp+xqrgAAAMBJRzEB\nAAAAqKKYAAAAAFRRTAAAAACqzDOaAwAAwNIY5dyMMkySnJtRNnJVkmSYUYa5eQ+XDE4eRnNom2dW\nqxfb3ebJWq2srvLWNastT9bus7rKW9estjxZq5XVVd66ZrXlyVqtrK7yVi1rlUYikGU0hxNl1Wgb\nzUExoW2eWa0d227zZK1WVld565rVlidr91ld5a1rVluerNXK6ipvXbPa8mStVlZXeauW1fWB8HiL\ni1GGGWaUZHaLi2XPOvb5q/MazpO3/XzFhL2gmNBjVluerNXK6ipvXbPa8mTtPqurvHXNasuTtVpZ\nXeWta1ZbnqzVyuoqb9Wy+vxVvcYqZa3Sa1iTp5iwNxQTesxqy5O1Wlld5a1rVluerN1ndZW3rllt\nebJWK6vKy8RfAAAPLklEQVSrvHXNasuTtVpZXeWtQtZe/YJfY9mzVvU1nJXXdyuIneQpJsywTDub\nZc9qy5O1Wlld5a1rVluerN1ndZW3rlltebJWK6urvHXNasuTtVpZXeWta1ZbnqzVyuoqbxmyqubR\nUkwwNCQAAABQRTEBAAAAqKKYAAAAAFRRTAAAAACqKCYAAAAAVRQTAAAAgCqKCQAAAEAVxQQAAACg\nimICAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigmAAAAAFUUEwAAAIAqigkAAABAFcUEAAAAoIpi\nAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAAAABVFBMAAACAKooJAAAAQJV5iwnnJ7kryT1J\nrmiZ5prm8duTvGzs/rckeTjJh3e4jAAAAMASmaeYsC/Jm1IKCi9OcnGSF01MczTJC5KcneQ1Sa4d\ne+ytzXMBAACANTBPMeGcJPcmuS/J40luTHLRxDQXJrmhuX5LktOTPLu5/btJHtntggIAAADLYZ5i\nwplJ7h+7/UBzX+00AAAAwBqYp5iwOee8Bjt8HgAAALBC9s8xzYNJDo3dPpTS8uBE05zV3DeXjY2N\nJ68Ph8MMh8N5nwoAAAAsyGg0ymg0mjndZGuCafYnuTvJq5I8lOS9KZ0w3jk2zdEklzd/jyR5Q/N3\ny+Ek70rykinz39zc3F0jhsFgsPBmEIMk05Zr1bPa8mStVlZXeeua1ZYna/dZXeWta1ZbnqzVyuoq\nb12z2vJkrVZWV3nrmtWWJ2u1srrKW4asqnkMBluzOsY8pzk8kVIouCnJHUl+OaWQcFlzSZJ3J/nj\nlI4ar0vy98ae/0tJfi/JC1P6Vbh0JysAAAAALId5WiZ0TcuEHrPa8mStVlZXeeua1ZYna/dZXeWt\na1ZbnqzVyuoqb12z2vJkrVZWV3nrmtWWJ2u1srrKW4asqnnsomUCAAAAwJMUEwAAAIAqigkAAABA\nFcUEAAAAoIpiAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAAAABVFBMAAACAKooJAAAAQBXF\nBAAAAKCKYgIAAABQRTEBAAAAqKKYAAAAAFRRTAAAAACqKCYAAAAAVRQTAAAAgCqKCQAAAEAVxQQA\nAACgimICAAAAUEUxAQAAAKiimAAAAABUUUwAAAAAqigmAAAAAFUUEwAAAIAqigkAAABAFcUEAAAA\noIpiAgAAAFBFMQEAAACoopgAAAAAVFFMAAAAAKooJgAAAABV5ikmnJ/kriT3JLmiZZprmsdvT/Ky\nyucCAAAAK2RWMWFfkjelFAVenOTiJC+amOZokhckOTvJa5JcW/HcXo3WOE+WrGXI6jtPlqxlyZMl\naxmy+s6TJWsZsvrOkyVrGbL2Im+aWcWEc5Lcm+S+JI8nuTHJRRPTXJjkhub6LUlOT/LsOZ/bq9Ea\n58mStQxZfefJkrUsebJkLUNW33myZC1DVt95smQtQ9Ze5E0zq5hwZpL7x24/0Nw3zzTPneO5AAAA\nwIqZVUzYnHM+g90uCAAAALAaZhUBjiTZSOn3IEmuTPKFJD8+Ns2/SmllcWNz+64k5yZ53hzPTZLb\nkry0dsEBAACAzt2e5Ktrn7Q/yUeSHE5yasqB/7QOGN/dXD+S5D9XPBcAAABYQxckuTulM8Urm/su\nay5b3tQ8fnuSr5nxXAAAAAAAAACA1feWJA8n+fDYfRspo0rc2lzOP/5pC8u6cSzno83frrKS5LVJ\n7kzyBzm+X4pFZp2T5L0p6/O+JF+3oKynpgwteluSO5K8vrl/I928Z215357kD5P8eY5tZbNI35/y\nmv5Bc32Rpr1nL03y+0k+lOSdSZ654Mwt9zUZt6ZsI105lOQ/pLxPf5DkdR1mJWW427el/H/dkXI6\nV1e+Ktvb+q1JPptu1+/KlNfxw0l+MckXLXDebe9TF/9jbVkbWfz+oy2ri31jW9Y/S9keb0/y9iTP\nWkBW2z6xi6y29TqY5LeS/FGS30z531uEtnXrIq9t3brYD7dl/dOU9+u2JP++mW632l7DLrKS6Z9l\nfW6LSTffq9ryfjmL/87Y9p5t+QcpfZkdXEDWrM/lRWaNm7WOuzVtO+xqm2/L6+N7aZLsS9nu3tVh\nRtJ+HNPH/LvaDrecn9J34D1JrljwvKetV1efmTRemeRlOfZFvyrJD/aUNe7/TfKPOsz6hpSN6SnN\n7S/rMGuU5K801y9I+fBYlNOav/tT+t54Rbp7z9ry/kKSF6asVxc77b+Y8no+NWXH/VtJnr/A+U97\nz97X3J8klyb5PxeYN+6j6W4HPe7Z2e4A5hkpp1J12R/LDUle3Vzfn8V8gZ3HKUn+JIv9ojLucJI/\nznYB4ZeTfO8C59/2PnXxP9aW1cX+oy1rlMXvG9uy/nK2R2P6f5rLIkzbJ3aR1bZeP5HkHzb3X7Gg\nrC3T1q2LvLZ162I/3JY1Xqh4bZI3LyArmf4adpU17bOsz22xq+9V83x+LfI747T3LCmfK7+RxX1u\nn2i9Fp01qW0dF2HadtjVNt+W1/X30i0/mOQXUoqdXZp1zNTV/LveDvelnNZ/OGW/sei+AqetV5ef\nmXObNTTkKvvdJI9Mub+LYSzbsrby/kaSX+ow6++mVGMfb25/osOsP8n2wdTpSR5cUFaSfL75e2rK\nP+VWdldDj07mfTqlovhHHeUl5UPhliT/LaXKfHOSv7bA+U97z85u7k+S9yT5tgXmTepjmNiPp+yk\nk+TPUn45em5HWc9K2YG/pbn9REprgT6cl9KJ7f0dzf9zKfuM01K+hJ2Wxf4/t71PXfyPTcs6s7m9\n6G2yLauLfWPba/hbKb+uJGV/ctYCspLp+8QustpewwtTindp/n7LArK2TPt86SKvbd262A+3bR+P\njk3zjCSfXEBWMn376Cpr2mdZn9vi30k336tmfX4t+jvjtPcsSf55tg9CFuFE67XorElt67gI07bD\nrrb5tryuv5cm5X/paEphpOvvcSc6Zupy/l1vh+ekFBPuS9lv3JjkogXOf9p6dfmZObd1Lia0eW1K\n86Tr009zkFemNEv5SIcZZyf5+pSK7CjJ13aY9SNJfjLJx1KaHC6yY81TUj6MHs52c7mku/dsMu+O\nBc67zR+kbBMHUw7e/rcs7kCgzR9me4f27enul+7NlC/J70/yfR1lTDqcUqm9paP5Py/lS+Rbk3ww\nyc9k+1eQrn1nyqkHXfl0tv+XH0rymZT3rwuH0+37NC1ra2ShLvf541ld7hvHsyZfw1dne0Sl3Zq1\nT1xk1pbD2V6vM5rsNH/PWGDOtM+XLvOSY9et6/3weFaS/N8p2+L3ZnG/VrVtH11kzdL1tvjCdP+9\najxvy6K/M057zy5KOf3rQwvKmHQ42+vVdVayN9/l9mKb79JPJfnhbBfr1k0f2+GZOfbHnwey/aNG\nV7r+DCNlhzbeHOTLUypugyT/V8qXy66ytlyb5AcWmDMt68NJ3thc/7qUpstdZb0nybc217895ZeC\nRXtWygf4MN2+Z9PytnTZnOzVKQfcNyf56ZSd+CIdzrHv2VcluanJ/CdZbBV93HOav1+W8sH+yhNM\nuwjPSFmnLiuxX5tSYd46//0N6e40kXGnphQxFtW0dprnp3zp+pKUlgnvSPK3Oshpe5+6+B+bzOpy\n/zGZ1eW+se01/D+S/OoCc7ZM2yd2kfWMJB/I9npN/uqyyF8Yt2yt2zd0nDf5nnW5Hz7RvvBHUoqh\nizRt++gi63Cmf6/qalscfw27/F41LW9LF98Zk+337Gjz90Bz/0dTPgMWZXy9TkspKHSVNaltu9yt\nw2lvkt/F/1dbXlffS785yb9srg/TfZ8JyYlf00XPv6/t8NtSfnDa8l1J/sWCMw7n2Netj8/Mk97h\ntG+sJ3psUVn7U5p+LboJ9mTWryc5d+z2vVncP8pk1ufGrg/SXZPvf5zkh2YsS5d5XZ+btuXHUppT\nLtLhtL9OL0w/vw5fldLRTVeekvLF/O93mJGU80A/Onb7FUn+v44zk1JF/42OM74jx57r+d3Z/kKx\nKCd6nxb9PzZrmzicxe0/pmV1tW9sW69LkvynlP5XujC+T+wia9p63ZXyP5eU4uRdC8wbt7VuXeXN\n2hYXuR+elfUVKS3iFm3aZ/Sisw7n+P/ZS9LPttjl96q296yr74xb/nFKXwwPp3yufTSlWH5fSuF1\ntybX6yUdZrWZtl3u1uG0f3Z08f/VltfV99IfS/lF/aMpp+s9luRnO8gZdzj9FRP62g6P5NjvbVdm\n8Z0wHs6xr1tfn5kndLKd5vCcsevfmm435KSc83xnShPiLv1akm9srr8w5VfNT3WUdW+2P2C/MYs7\nj+tLs90E+WkpHS3dmu1/kmSx71lb3riuzhvb2oF9Rco6ddmUPdn+dfuUlC8S13aQcVq2OyV6epJv\nSnf/X4OUX5jvSGkp0KWPp3zIvrC5fV62T7/p0sVZ3Dmzbe5K+fB7Wsprel4W2zx0nvdpUf9jbVld\n7PPbsrrYN7ZlnZ/SJPWilP5XFqFtn9hFVtt6vTPbnYB+b8pn2yK0rVsXeW3r1sV+uC3r7LHrF2Ux\nIwO0vYYv6CCrTZ/bYlffq060X1z0d8Zp79nvpzSFfl5zeSDlAPVPd5k1bb0+3FHWuHm+yy1aF/9f\n8+rie+mPppx29byU0yt/O8n3dJCzV/rYDpPSIufslAP+U1N+sOm6M8uuPjNp/FLKDvl/pBwMvDql\n0vahlPNnfy2LO7dkK+u/N1mXNve/NclrFpRxoqynJPm5lH+YD2RxTbwmX8NLU5p9bw3D8/sp58Ut\nwktSzkm/LeU9+uHm/q7es7a8b01Z1/+aciD56wvKG/c7KQekt6U0tV2kadv961J6Vr47pQLdheel\nrM9tKVX6RZ8vPu4VKef13ZbFDxk6zUtTemJf5HBkJ/L0lCbQXQ3hOe4fZntoyBuy3XP5Ikx7ny5I\naf666P+xtqwu9h9tWV3sG9uy7knyX8bu++kFZLXtE7vIavsfPphyusiih7lqW7cu8tresy72w21Z\nb0v5n74t5XSARfwC1/YadpGVTP8s63Nb7Op71Yk+vxb9nbHtPRv3x1lMz/Zt22IXWePmWcfdmLYd\ndrXNt+V18ZnZ5tx0fwDcdszU1/y72A63XJCyj783i/8ePO2YrKvPTAAAAAAAAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAFbd/w9Bp2lV6OHe4QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10a1b3310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['dtw_det_sap_flux', 'det_sap_flux_std', 'det_sap_flux_mean', 'det_sap_flux_min', 'pos_corr2_max', 'pos_corr1_max', 'sap_bkg_max', 'det_sap_flux_max', 'dtw_sap_bkg', 'sap_flux_std', 'sap_bkg_mean', 'sap_bkg_std', 'sap_bkg_min', 'mom_centr1_std', 'mom_centr2_std', 'pos_corr1_mean', 'mom_centr1_min', 'pos_corr2_mean', 'mom_centr1_mean', 'pos_corr1_min', 'pos_corr1_std', 'mom_centr1_max', 'pos_corr2_std', 'pd_sap_flux_std', 'mom_centr2_min', 'mom_centr2_max', 'pos_corr2_min']\n",
      "\n",
      "Number of features after feature selection: 27\n"
     ]
    }
   ],
   "source": [
    "# Divide candidate planets on their respective dataframe to perform final exoplanetary search\n",
    "candidates = exoplanets_data[exoplanets_data['koi_disposition'] == 'CANDIDATE']\n",
    "exoplanets = exoplanets_data[exoplanets_data['koi_disposition'] != 'CANDIDATE']\n",
    "\n",
    "# Extract feature columns\n",
    "feature_cols = list(exoplanets_data.columns[3:])\n",
    "\n",
    "# Extract target column 'koi_disposition'\n",
    "target_col = exoplanets_data.columns[2]\n",
    "\n",
    "# Show the list of columns\n",
    "print \"Target column: {}\".format(target_col)\n",
    "\n",
    "# Separate the data into feature data and target data (X_all and y_all, respectively) also for candidates\n",
    "X_all = exoplanets[feature_cols]\n",
    "y_all = exoplanets[target_col]\n",
    "X_candidates = candidates[feature_cols]\n",
    "\n",
    "# Build a forest and compute the feature importances\n",
    "forest = ExtraTreesClassifier(n_estimators=250,\n",
    "                              random_state=0)\n",
    "\n",
    "forest.fit(X_all, y_all)\n",
    "importances = forest.feature_importances_\n",
    "std = np.std([tree.feature_importances_ for tree in forest.estimators_],axis=0)\n",
    "indices = np.argsort(importances)[::-1]\n",
    "\n",
    "# Plot the feature importances of the forest\n",
    "plt.figure(figsize=(18, 6), dpi=80)\n",
    "plt.title(\"Feature importances\")\n",
    "plt.bar(range(X_all.shape[1]), importances[indices],\n",
    "       color=\"r\", yerr=std[indices], align=\"center\")\n",
    "plt.xticks(range(X_all.shape[1]), indices)\n",
    "plt.xlim([-1, X_all.shape[1]])\n",
    "plt.show()\n",
    "\n",
    "# Remove last 25% of features\n",
    "n_features_75 = int(n_features * 0.75)\n",
    "\n",
    "# Extract feature columns\n",
    "feature_cols = list(X_all.columns[indices[0:27]])\n",
    "\n",
    "print feature_cols\n",
    "\n",
    "X_all = X_all[feature_cols]\n",
    "X_candidates = candidates[feature_cols]\n",
    "\n",
    "print \"\\nNumber of features after feature selection: {}\".format(X_all.shape[1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since I have decided to choose multiple Machine learning algorithm, I have created different functions to help me simplify the process.\n",
    "- **train_classifier**: Is a function that fits the training data to the chosen classifier.\n",
    "- **predict_labels**: Is a function that returns the precision score for an already fitted classifier.\n",
    "- **predict**: Is a function that returns the prediction values for an already fitted classifier.\n",
    "- **train_predict**: Is a function that returns the prediction values of training and testing data for an already fitted classifier.\n",
    "- **k_fold_train_predict**: Is a function that performs k fold on a specific classifier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def train_classifier(clf, X_train, y_train):\n",
    "\n",
    "    # Fits a classifier to the training data.\n",
    "    start = time()\n",
    "    clf.fit(X_train, y_train)\n",
    "    end = time()\n",
    "\n",
    "    # Print the results\n",
    "    # print \"Trained model in {:.4f} seconds\".format(end - start) [Debug]\n",
    "\n",
    "\n",
    "def predict_labels(clf, features, target, confusion):\n",
    "\n",
    "    # Makes predictions using a fit classifier based on Precision score.\n",
    "    start = time()\n",
    "    y_pred = clf.predict(features)\n",
    "    end = time()\n",
    "    \n",
    "    # Print and return results\n",
    "    # print \"Made predictions in {:.4f} seconds.\".format(end - start)\n",
    "    \n",
    "    if confusion:\n",
    "        print \"Confusion Matrix:\"\n",
    "        print confusion_matrix(target.values, y_pred, labels=['CONFIRMED', 'FALSE POSITIVE'])\n",
    "\n",
    "    return precision_score(target.values, y_pred, pos_label='CONFIRMED')\n",
    "\n",
    "\n",
    "def predict(clf, features):\n",
    "\n",
    "    # Makes predictions using a fit classifier based on Precision score.\n",
    "    start = time()\n",
    "    y_pred = clf.predict(features)\n",
    "    end = time()\n",
    "\n",
    "    # Print and return results\n",
    "    # print \"\\nMade predictions in {:.4f} seconds.\".format(end - start)  [Debug]\n",
    "    return y_pred\n",
    "\n",
    "\n",
    "def train_predict(clf, X_train, y_train, X_test, y_test, confusion=False):\n",
    "    # Train and predict using a classifer based on F1 score.\n",
    "\n",
    "    # Train the classifier\n",
    "    train_classifier(clf, X_train, y_train)\n",
    "    \n",
    "    train_predict = predict_labels(clf, X_train, y_train, confusion)\n",
    "    test_predict = predict_labels(clf, X_test, y_test, confusion)\n",
    "    \n",
    "    # Print the results of prediction for both training and testing\n",
    "    #print \"Precision score for training set: {:.4f}.\".format(predict_labels(clf, X_train, y_train))\n",
    "    #print \"Precision score for test set: {:.4f}.\".format(predict_labels(clf, X_test, y_test))\n",
    "    \n",
    "    return train_predict, test_predict\n",
    "\n",
    "\n",
    "def k_fold_train_predict(clf, kf, X_all):\n",
    "    # Indicate the classifier\n",
    "    print \"\\nTraining a {} \".format(clf.__class__.__name__)\n",
    "    \n",
    "    results = pd.DataFrame(columns=['score_train', 'score_test'])\n",
    "    for train_index, test_index in kf:\n",
    "        X_train, X_test = X_all.iloc[train_index], X_all.iloc[test_index]\n",
    "        y_train, y_test = y_all.iloc[train_index], y_all.iloc[test_index]\n",
    "        \n",
    "        # Training and predicting the results for the model\n",
    "        predict_train, predict_test = train_predict(clf, X_train, y_train, X_test, y_test)\n",
    "        results.loc[len(results)] = [predict_train, predict_test]\n",
    "        \n",
    "    print \"The average precision score for traing is {:.4f}\" \\\n",
    "        .format(results['score_train'].describe()['mean'])\n",
    "    print \"The average precision score for traing is {:.4f}\" \\\n",
    "        .format(results['score_test'].describe()['mean'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Subsequently, I am initializing the different classifiers and performing a k-fold on each of them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Training a GaussianNB \n",
      "The average precision score for traing is 0.3787\n",
      "The average precision score for traing is 0.3791\n",
      "\n",
      "Training a LogisticRegression \n",
      "The average precision score for traing is 0.4715\n",
      "The average precision score for traing is 0.6095\n",
      "\n",
      "Training a SVC \n",
      "The average precision score for traing is 0.9945\n",
      "The average precision score for traing is 0.9645\n",
      "\n",
      "Training a DecisionTreeClassifier \n",
      "The average precision score for traing is 0.9838\n",
      "The average precision score for traing is 0.6171\n"
     ]
    }
   ],
   "source": [
    "# Initialize the models\n",
    "clf_A = GaussianNB()\n",
    "clf_B = LogisticRegression()\n",
    "clf_C = SVC()\n",
    "clf_D = DecisionTreeClassifier()\n",
    "\n",
    "kf = KFold(len(X_all), n_folds=10, shuffle=True)\n",
    "#print kf [Debug]\n",
    "\n",
    "k_fold_train_predict(clf_A, kf, X_all)\n",
    "k_fold_train_predict(clf_B, kf, X_all)\n",
    "k_fold_train_predict(clf_C, kf, X_all)\n",
    "k_fold_train_predict(clf_D, kf, X_all)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, I am training and fitting the entire dataset to make sure the k-fold predictions are approximately correct, showing the confusion matrix to back up my assumptions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training set has 3941 samples.\n",
      "Testing set has 1314 samples.\n",
      "Candidates set has 1975 samples\n",
      "\n",
      "Training a GaussianNB \n",
      "Confusion Matrix:\n",
      "[[1370   64]\n",
      " [2277  230]]\n",
      "Confusion Matrix:\n",
      "[[470  26]\n",
      " [751  67]]\n",
      "Precision score for training set: 0.3757.\n",
      "Precision score for test set: 0.3849.\n",
      "\n",
      "Training a LogisticRegression \n",
      "Confusion Matrix:\n",
      "[[  26 1408]\n",
      " [  24 2483]]\n",
      "Confusion Matrix:\n",
      "[[  2 494]\n",
      " [ 15 803]]\n",
      "Precision score for training set: 0.5200.\n",
      "Precision score for test set: 0.1176.\n",
      "\n",
      "Training a SVC \n",
      "Confusion Matrix:\n",
      "[[1418   16]\n",
      " [   5 2502]]\n",
      "Confusion Matrix:\n",
      "[[192 304]\n",
      " [  7 811]]\n",
      "Precision score for training set: 0.9965.\n",
      "Precision score for test set: 0.9648.\n",
      "\n",
      "Training a DecisionTreeClassifier \n",
      "Confusion Matrix:\n",
      "[[1434    0]\n",
      " [  21 2486]]\n",
      "Confusion Matrix:\n",
      "[[325 171]\n",
      " [189 629]]\n",
      "Precision score for training set: 0.9856.\n",
      "Precision score for test set: 0.6323.\n"
     ]
    }
   ],
   "source": [
    "# Set the number of training points to 75% of data set\n",
    "num_train = int(np.floor(X_all.shape[0] * 0.75))\n",
    "\n",
    "# Set the number of testing points\n",
    "num_test = X_all.shape[0] - num_train\n",
    "\n",
    "# Shuffle and split the dataset into the number of training and testing points above\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_all, y_all, test_size=num_test, random_state=69)\n",
    "\n",
    "# Show the results of the split\n",
    "print \"Training set has {} samples.\".format(X_train.shape[0])\n",
    "print \"Testing set has {} samples.\".format(X_test.shape[0])\n",
    "print \"Candidates set has {} samples\".format(X_candidates.shape[0])\n",
    "\n",
    "# Training and predicting the results for each model on the entire dataset\n",
    "print \"\\nTraining a {} \".format(clf_A.__class__.__name__)\n",
    "predict_train, predict_test = train_predict(clf_A, X_train, y_train, X_test, y_test, True)\n",
    "print \"Precision score for training set: {:.4f}.\".format(predict_train)\n",
    "print \"Precision score for test set: {:.4f}.\".format(predict_test)\n",
    "\n",
    "print \"\\nTraining a {} \".format(clf_B.__class__.__name__)\n",
    "predict_train, predict_test = train_predict(clf_B, X_train, y_train, X_test, y_test, True)\n",
    "print \"Precision score for training set: {:.4f}.\".format(predict_train)\n",
    "print \"Precision score for test set: {:.4f}.\".format(predict_test)\n",
    "\n",
    "print \"\\nTraining a {} \".format(clf_C.__class__.__name__)\n",
    "predict_train, predict_test = train_predict(clf_C, X_train, y_train, X_test, y_test, True)\n",
    "print \"Precision score for training set: {:.4f}.\".format(predict_train)\n",
    "print \"Precision score for test set: {:.4f}.\".format(predict_test)\n",
    "\n",
    "print \"\\nTraining a {} \".format(clf_D.__class__.__name__)\n",
    "predict_train, predict_test =train_predict(clf_D, X_train, y_train, X_test, y_test, True)\n",
    "print \"Precision score for training set: {:.4f}.\".format(predict_train)\n",
    "print \"Precision score for test set: {:.4f}.\".format(predict_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The DecisionTreeClassifier seems to perform really well on training; However, it does not generalize very well. Logistic Regression and Naive Bayse do not seem to fit the data. \n",
    "\n",
    "From the following results, the SVC is the model that perform best on this data set, with a precision score for the test dataset equal to 0.9648. Even though SVC is negatively biased, preferring False Positive, I am okay with that bias. Indeed, this is a discovery project, therefore, I would prefer that the algorithm is negatively biased instead of being positively biased. Meaning that the planets the algorithm recognizes really are confirmed exoplanet.\n",
    "\n",
    "If the algorithm classifies something as false positive, the status of the planet will remain still as Candidate.\n",
    "\n",
    "Finally, I wanted to point out that there weren't any complications that occurred during the coding process of this section. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Refinement\n",
    "I have tried to perform a GridSearch on the SVC classifier; however, it does seem that any improvement has been reported on the algorithm. As you can see from the confusion matrix nothing changes from the previous reported matrix to the matrix after tuning.\n",
    "\n",
    "The parameter I have tuned for the GridSearch, with their respective range, are: C [-2, 10], gamma [-5,5], and degree [1,6]. The Final model has C = 10,  degree=10, and gamma = 0.0031622.\n",
    "\n",
    "Another thing to be noticed is that I have tried to perform normalization on the dataset; however, the SVC model did not work at all. This behavior is due to the fact that the data is been already pre-processed. Moreover, relevant features like DTW data points would loose their applicability on the overall model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion Matrix:\n",
      "[[1418   16]\n",
      " [   5 2502]]\n",
      "\n",
      "Tuned model has a training precision score of 0.9965.\n",
      "Confusion Matrix:\n",
      "[[192 304]\n",
      " [  7 811]]\n",
      "Tuned model has a testing precision score of 0.9648.\n"
     ]
    }
   ],
   "source": [
    "# Create the parameters list to tune\n",
    "C_range = np.logspace(-2, 10, 5)\n",
    "gamma_range = np.logspace(-5, 5, 5)\n",
    "degree_range = np.logspace(1, 6, 6)\n",
    "parameters = dict(gamma=gamma_range, C=C_range, degree=degree_range)\n",
    "\n",
    "# Initialize the classifier\n",
    "clf = SVC()\n",
    "\n",
    "# Make an Precision scoring function using 'make_scorer'\n",
    "precision_scorer = make_scorer(precision_score, pos_label=\"CONFIRMED\")\n",
    "\n",
    "# Perform grid search on the classifier using the f1_scorer as the scoring method\n",
    "grid_obj = GridSearchCV(clf, parameters, scoring=precision_scorer)\n",
    "\n",
    "# Fit the grid search object to the training data and find the optimal parameters\n",
    "grid_obj = grid_obj.fit(X_train, y_train)\n",
    "\n",
    "# Get the estimator\n",
    "clf = grid_obj.best_estimator_\n",
    "\n",
    "# Report the final F1 score for training and testing after parameter tuning\n",
    "print \"\\nTuned model has a training precision score of {:.4f}.\" \\\n",
    "    .format(predict_labels(clf, X_train, y_train, True))\n",
    "print \"Tuned model has a testing precision score of {:.4f}.\" \\\n",
    ".format(predict_labels(clf, X_test, y_test, True))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## IV. Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Exoplanets: 144 have been Confirmed!\n",
      "\n",
      "Exoplanets: 1831 are False Positive!\n",
      "\n",
      "Discovery Rate: 7.2911%\n"
     ]
    }
   ],
   "source": [
    "def discover_possible_exoplanets(clf, X_candidates, candidates):\n",
    "    # Setting up confirmed and false positive array\n",
    "    confirmed = []\n",
    "    false_positive = []\n",
    "    \n",
    "    # Predicting the candidate exoplanets\n",
    "    results = pd.Series(predict(clf, X_candidates), index=candidates.index)\n",
    "    candidates.insert(3, 'koi_disposition_pred', results)\n",
    "    \n",
    "    # Looping trough all the planets predictions\n",
    "    for index, planet in candidates.iterrows():\n",
    "        if planet['koi_disposition_pred'] == 'CONFIRMED':\n",
    "            confirmed.append({ 'name': planet['kepoi_name'], 'index': planet['index']})\n",
    "        else:\n",
    "            false_positive.append({'name': planet['kepoi_name'], 'index': planet['index']})\n",
    "\n",
    "    discovery_rate = float(len(confirmed))/candidates.shape[0]*100\n",
    "\n",
    "    print \"\\nExoplanets: {} have been Confirmed!\".format(len(confirmed))\n",
    "    print \"\\nExoplanets: {} are False Positive!\".format(len(false_positive))\n",
    "    print \"\\nDiscovery Rate: {:.4f}%\".format(discovery_rate)\n",
    "\n",
    "discover_possible_exoplanets(clf,X_candidates, candidates)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Evaluation and Validation\n",
    "The support Vector Classifier (SVC) works better than predicted, even though is a little bit negatively biased, as you can see from the confusion matrix. As a matter of fact, the model has been tested with various inputs to evaluate whether the model generalizes well to unseen data. Indeed, the average precision score for the k-fold testing set is 0.9645 and the precision score of the entire dataset is 0.9648.\n",
    "\n",
    "Further, I was pleasantly surprised to discover that the model prediction on the still candidate planets confirmed 144 exoplanets. I am not an astrophysicist, so there are no ways for me to validate this prediction. The only way would be scientist releasing results on the candidate planets and compare those results with the prediction from the model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Justification\n",
    "The final results found are stronger than the benchmark results reported earlier. Indeed, I predicted that the precision score would have been 60%; however, the score I got was 96.48%.  I was not expecting such an amazing prediction. I believed that the data processing step contributed a lot to this results. The Dynamic Time Wrapping algorithm does an amazing job to compare the light curves, and without it I would have never reported this results. This machine learning approach would facilitate a lot the exoplanetary search for scientists."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## V. Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Free-Form Visualization\n",
    "One thing I want to point out is the fact that at the beginning I thought that the light curves the dynamic time wrapping algorithm could not process meant something. Therefore, I performed an analysis to see if there was any correlation between the number of dropped planets and their relative category (candidate, false positive, confirmed). As the plot below shows, there is not any correlation between the dropping rate and relative category of a planet. Indeed, the ratio of the number of each category dropped over the total number of planet for the respective category is between 0.15 and 0.20. This is consistent either between the 3 category, and with the total dropping rate of the dataset, which is 18.07%. I assume that my implementation of the DTW does not perform well with parallelized computing. ![Dropping Rate Ratio](images/dropping_rate_ratio.png \"Dropping Rate Ratio\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reflection\n",
    "This project started thanks to my passion for astrophysics. The first, and most difficult step was to collect the data of all the exoplanet, and processing it. Subsequently, the data had to be fitted to different models to check which one would perform best. Finally, with the best model, predict the faith of the still candidate exoplanets.\n",
    "\n",
    "The entire project was difficult to put together, but at the same time rewarding thanks to the results obtained. I learned a lot about astrophysics, and the final results exceeded my  expectations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Improvement\n",
    "One improvement that could be performed in the data processing section is to use a different algorithm to compare the two light curves. Indeed, while I was researching, I found this algorithm called Fast Time Series Evaluation (FTSE)[6], that could have given a faster and more accurate result compared with the Dynamic Time Warping algorithm. However, the research paper was not really clear on how the implement the algorithm, and I ended up using DTW.\n",
    "\n",
    "Another improvement that could be performed is to refactor the code, setting the target labels to boolean values, and implement other statistical machine learning algorithms, such as perceptron, and k-nearest neighbors.\n",
    "\n",
    "Another approach to this problem would have been using the photos taken from the Kepler telescope and apply a Deep Neural Networks to recognize planetary transit. However, this would have taken more time than required, and would have been out of scope."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Citations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[1]\n",
    "*A Jupiter-mass companion to a solar-type star*.\n",
    "**Michel Mayor & Didier Queloz**.\n",
    "http://www.nature.com/nature/journal/v378/n6555/abs/378355a0.html\n",
    "Nature, 378, 355-359.\n",
    "November 23, 1995.\n",
    "\n",
    "[2]\n",
    "*Kepler Mission/QuickGuide*.\n",
    "**Nasa Staff**.\n",
    "http://kepler.nasa.gov/Mission/QuickGuide/\n",
    "July 25, 2016.\n",
    "\n",
    "[3]\n",
    "*Kepler Archive Manual*.\n",
    "**Nasa**.\n",
    "http://archive.stsci.edu/kepler/manuals/archive_manual.pdf\n",
    "July 28, 2016.\n",
    "\n",
    "[4]\n",
    "*Everything you know about Dynamic Time Warping is Wrong*.\n",
    "**Chotirat Ratanamahatana & Eamonn Keogh **.\n",
    "http://wearables.cc.gatech.edu/paper_of_week/DTW_myths.pdf\n",
    "University of California, Riverside.\n",
    "July 28, 2016.\n",
    "\n",
    "[5]\n",
    "*Using Dynamic Time Warping toFind Patterns in Time Serie*.\n",
    "**Donald J. Bemd & James Cliffor**.\n",
    "http://www.aaai.org/Papers/Workshops/1994/WS-94-03/WS94-03-031.pdf\n",
    "New York University, New York.\n",
    "July 28, 2016.\n",
    "\n",
    "[6]\n",
    "*An Efficient and Accurate Method for Evaluating Time Series Similarity*.\n",
    "**Michael Morse & Jignesh M. Patel**.\n",
    "http://dbgroup.eecs.umich.edu/files/sigmod07timeseries.pdf\n",
    "University of Michigan Ann Arbor, Michigan.\n",
    "August 02, 2016."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}