heasarc_VO_Tables.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using VO Tables from HEASARC "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook, we show how to list, fetch and look at VO tables from the HEASARC archive and also how to create your own VO Table, which can be used to cross-correlate with other VO Tables.  \n",
    "\n",
    "*  [1. Retrieving](#retrieve) a catalog as a VO Table\n",
    "*  [2. Creating](#create) a VOTable from scratch\n",
    "*  [3. Uploading](#upload) our VOTable file for cross-correlation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "## The main HTTP request tool we will use:\n",
    "import requests, io, numpy\n",
    "## \n",
    "from astropy import table as aptable"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"retrieve\"></a>\n",
    "\n",
    "# 1. Retrieving a catalog as a VO Table\n",
    "\n",
    "We can use __[Browse](https://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/w3browse.pl)__ to get any HEASARC catalog.  Browse can be used to retrieve tables in a number of formats, one of which is a VOTable in XML as defined by the __[IVOA reference](http://www.ivoa.net/documents/VOTable/)__.  \n",
    "\n",
    "The requests package of python is a generic interface to web services.  The requests.get() method formats the given parameters for you and returns the results of the HTTP request in a useful object.  In this case, the result is a string containing the VOTable XML.  The standard astropy.table package can read these into the standard Table objec.\n",
    "\n",
    "This fetches the entire rosmaster catalog, so give it a few seconds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "params = {'name': 'rosmaster'}\n",
    "r = requests.get('http://heasarc.gsfc.nasa.gov/cgi-bin/W3Browse/getvotable.pl', params=params)\n",
    "#r.text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The astropy.table package can read the VOTable's XML and create a usual Table object (with some warnings that we can ignore):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: W42: None:2:0: W42: No XML namespace specified [astropy.io.votable.tree]\n",
      "WARNING: W15: None:13:6: W15: PARAM element missing required 'name' attribute [astropy.io.votable.tree]\n",
      "WARNING: W15: None:14:6: W15: PARAM element missing required 'name' attribute [astropy.io.votable.tree]\n",
      "WARNING: W15: None:15:6: W15: PARAM element missing required 'name' attribute [astropy.io.votable.tree]\n",
      "WARNING: W49: None:149:141: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:152:143: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:155:144: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:158:143: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:164:146: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:167:145: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:170:146: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:362:144: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:374:145: W49: Empty cell illegal for integer fields. [astropy.io.votable.converters]\n",
      "WARNING: W49: None:377:144: W49: Empty cell illegal for integer fields. (suppressing further warnings of this type...) [astropy.io.votable.converters]\n"
     ]
    }
   ],
   "source": [
    "table=aptable.Table.read(io.BytesIO(r.content))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'astropy.table.table.Table'>\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "&lt;Table masked=True length=11431&gt;\n",
       "<table id=\"table4544577376\" class=\"table-striped table-bordered table-condensed\">\n",
       "<thead><tr><th>col1</th><th>col2</th><th>col3</th><th>col4</th><th>col5</th><th>col6</th><th>col7</th><th>col8</th><th>col9</th><th>col10</th><th>col11</th><th>col12</th><th>col13</th><th>col14</th><th>col15</th><th>col16</th><th>col17</th><th>col18</th><th>col19</th><th>col20</th><th>col21</th><th>col22</th><th>col23</th><th>col24</th><th>col25</th><th>col26</th><th>col27</th><th>col28</th></tr></thead>\n",
       "<thead><tr><th></th><th>deg</th><th>deg</th><th>deg</th><th>deg</th><th></th><th></th><th></th><th>s</th><th>s</th><th></th><th>MJD</th><th>MJD</th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th></th><th>deg</th><th>d</th><th>d</th><th></th></tr></thead>\n",
       "<thead><tr><th>bytes16</th><th>float64</th><th>float64</th><th>float64</th><th>float64</th><th>bytes6</th><th>bytes2</th><th>bytes4</th><th>int32</th><th>int32</th><th>bytes10</th><th>float64</th><th>float64</th><th>bytes20</th><th>bytes16</th><th>bytes14</th><th>int32</th><th>bytes4</th><th>int16</th><th>int16</th><th>bytes120</th><th>int32</th><th>int16</th><th>int32</th><th>int16</th><th>int32</th><th>int32</th><th>int16</th></tr></thead>\n",
       "<tr><td>RH202299N00</td><td>49.3200</td><td>-85.5400</td><td>299.8517</td><td>-30.6815</td><td>HRI</td><td>N</td><td>MPE</td><td>36146</td><td>70000</td><td>RDF 4_2</td><td>50174.496192129598</td><td>50187.986747685201</td><td>RE J0317-853</td><td>BURLEIGH</td><td>MATTHEW</td><td>202299</td><td>n00</td><td>2</td><td>2</td><td>RE J0317-853</td><td>24302</td><td>6</td><td>7128</td><td>307</td><td>2163</td><td>2177</td><td>2900</td></tr>\n",
       "<tr><td>RH202299A01</td><td>49.3200</td><td>-85.5400</td><td>299.8517</td><td>-30.6815</td><td>HRI</td><td>N</td><td>MPE</td><td>43683</td><td>70000</td><td>RDF 3_6</td><td>50324.742534722202</td><td>50377.477592592601</td><td>RE J0317-853</td><td>BURLEIGH</td><td>MATTHEW</td><td>202299</td><td>a01</td><td>2</td><td>2</td><td>RE J0317-853</td><td>24301</td><td>6</td><td>7128</td><td>162</td><td>2316</td><td>2369</td><td>2900</td></tr>\n",
       "<tr><td>RP200594N00</td><td>151.8800</td><td>-85.0800</td><td>299.4192</td><td>-23.3707</td><td>PSPCB</td><td>N</td><td>MPE</td><td>4231</td><td>5000</td><td>RDF 3_4</td><td>48682.1389583333</td><td>48690.983333333301</td><td>HD 89499</td><td>FLEMING</td><td>THOMAS</td><td>200594</td><td>n00</td><td>2</td><td>2</td><td>HD 89499</td><td>17171</td><td>2</td><td>3532</td><td>66</td><td>648</td><td>657</td><td>2900</td></tr>\n",
       "<tr><td>RH201328N00</td><td>107.9100</td><td>-84.4700</td><td>296.7740</td><td>-26.5072</td><td>HRI</td><td>N</td><td>MPE</td><td>2332</td><td>2000</td><td>RDF 3_4</td><td>48880.838020833296</td><td>48881.780729166698</td><td>HD 60102 AB</td><td>ZINNECKER</td><td>HANS</td><td>201328</td><td>n00</td><td>2</td><td>2</td><td>HD 60102 AB</td><td>22560</td><td>3</td><td>4268</td><td>215</td><td>850</td><td>851</td><td>2900</td></tr>\n",
       "<tr><td>RH400876A01</td><td>193.9400</td><td>-83.3700</td><td>303.0651</td><td>-20.4994</td><td>HRI</td><td>N</td><td>GSFC</td><td>19075</td><td>20000</td><td>RDF 4_2</td><td>50702.423530092601</td><td>50735.172557870399</td><td>2EG J1248-8306 POS 3</td><td>HALPERN</td><td>JULES</td><td>400876</td><td>a01</td><td>4</td><td>2</td><td>2EG J1248-8306 POS 3</td><td>25116</td><td>7</td><td>2460</td><td>299</td><td>2699</td><td>2733</td><td>9999</td></tr>\n",
       "<tr><td>RH400876N00</td><td>193.9400</td><td>-83.3700</td><td>303.0651</td><td>-20.4994</td><td>HRI</td><td>N</td><td>GSFC</td><td>1059</td><td>20000</td><td>RDF 4_0</td><td>50548.2425</td><td>50548.255474537</td><td>2EG J1248-8306 POS 3</td><td>HALPERN</td><td>JULES</td><td>400876</td><td>n00</td><td>4</td><td>2</td><td>2EG J1248-8306 POS 3</td><td>19808</td><td>7</td><td>2460</td><td>85</td><td>2543</td><td>2543</td><td>9999</td></tr>\n",
       "<tr><td>RH400877A02</td><td>190.1700</td><td>-83.3600</td><td>302.6001</td><td>-20.4952</td><td>HRI</td><td>N</td><td>GSFC</td><td>19384</td><td>20000</td><td>RDF 4_2</td><td>50893.248460648101</td><td>50895.138969907399</td><td>2EG J1248-8306 POS 4</td><td>HALPERN</td><td>JULES</td><td>400877</td><td>a02</td><td>4</td><td>2</td><td>2EG J1248-8306 POS 4</td><td>24927</td><td>7</td><td>2460</td><td>90</td><td>2893</td><td>2895</td><td>9999</td></tr>\n",
       "<tr><td>RH400877A01</td><td>190.1700</td><td>-83.3600</td><td>302.6001</td><td>-20.4952</td><td>HRI</td><td>N</td><td>GSFC</td><td>2363</td><td>20000</td><td>RDF 4_2</td><td>50735.5647916667</td><td>50735.574120370402</td><td>2EG J1248-8306 POS 4</td><td>HALPERN</td><td>JULES</td><td>400877</td><td>a01</td><td>4</td><td>2</td><td>2EG J1248-8306 POS 4</td><td>25115</td><td>7</td><td>2460</td><td>260</td><td>2733</td><td>2733</td><td>9999</td></tr>\n",
       "<tr><td>RH400877N00</td><td>190.1700</td><td>-83.3600</td><td>302.6001</td><td>-20.4952</td><td>HRI</td><td>N</td><td>GSFC</td><td>1250</td><td>20000</td><td>RDF 4_0</td><td>50548.770254629599</td><td>50548.785879629599</td><td>2EG J1248-8306 POS 4</td><td>HALPERN</td><td>JULES</td><td>400877</td><td>n00</td><td>4</td><td>2</td><td>2EG J1248-8306 POS 4</td><td>19809</td><td>7</td><td>2460</td><td>81</td><td>2543</td><td>2543</td><td>9999</td></tr>\n",
       "<tr><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td></tr>\n",
       "<tr><td>RH180091N00</td><td>172.0200</td><td>85.5500</td><td>124.7826</td><td>31.2749</td><td>HRI</td><td>N</td><td>MPE</td><td>897</td><td>1000</td><td>RDF 3_9</td><td>50168.8914814815</td><td>50168.901898148099</td><td></td><td>SCHMITT</td><td>JUERGEN</td><td>180091</td><td>n00</td><td>1</td><td>2</td><td>COMET HYAKUTAKE POIN</td><td>23171</td><td>0</td><td>0</td><td>272</td><td>2158</td><td>2158</td><td>2900</td></tr>\n",
       "<tr><td>RP900512N00</td><td>110.3000</td><td>85.7000</td><td>127.7445</td><td>27.6030</td><td>PSPCB</td><td>N</td><td>GSFC</td><td>9029</td><td>10000</td><td>RDF 4_0</td><td>49221.753460648099</td><td>49221.921898148103</td><td>NGC2300 GROUP</td><td>MULCHAEY</td><td>JOHN</td><td>900512</td><td>n00</td><td>9</td><td>2</td><td>NGC2300 GROUP</td><td>20217</td><td>4</td><td>1818</td><td>132</td><td>1196</td><td>1196</td><td>3900</td></tr>\n",
       "<tr><td>RP900513N00</td><td>114.7000</td><td>85.7000</td><td>127.6963</td><td>27.9300</td><td>PSPCB</td><td>N</td><td>GSFC</td><td>8417</td><td>10000</td><td>RDF 4_0</td><td>49221.821967592601</td><td>49222.054560185199</td><td>NGC2300 GROUP</td><td>MULCHAEY</td><td>JOHN</td><td>900513</td><td>n00</td><td>9</td><td>2</td><td>NGC2300 GROUP</td><td>20221</td><td>4</td><td>1818</td><td>128</td><td>1196</td><td>1196</td><td>3900</td></tr>\n",
       "<tr><td>RP900161N00</td><td>111.7200</td><td>85.7500</td><td>127.6760</td><td>27.7031</td><td>PSPCB</td><td>N</td><td>GSFC</td><td>5997</td><td>14000</td><td>RDF 3_4</td><td>48737.362581018497</td><td>48739.391446759299</td><td>NGC2276</td><td>MUSHOTZKY</td><td>RICHARD</td><td>900161</td><td>n00</td><td>9</td><td>2</td><td>NGC2276</td><td>11708</td><td>2</td><td>676</td><td>12</td><td>704</td><td>706</td><td>3900</td></tr>\n",
       "<tr><td>RH600498A01</td><td>111.8000</td><td>85.7600</td><td>127.6640</td><td>27.7078</td><td>HRI</td><td>N</td><td>GSFC</td><td>21731</td><td>80000</td><td>RDF 3_6</td><td>49593.445428240702</td><td>49594.723287036999</td><td>NGC 2276</td><td>KEEL</td><td>WILLIAM</td><td>600498</td><td>a01</td><td>6</td><td>2</td><td>NGC 2276</td><td>16550</td><td>4</td><td>1635</td><td>140</td><td>1573</td><td>1575</td><td>6900</td></tr>\n",
       "<tr><td>RH600498N00</td><td>111.8000</td><td>85.7600</td><td>127.6640</td><td>27.7078</td><td>HRI</td><td>N</td><td>GSFC</td><td>52235</td><td>80000</td><td>RDF 3_4</td><td>49429.014386574097</td><td>49432.951921296299</td><td>NGC 2276</td><td>KEEL</td><td>WILLIAM</td><td>600498</td><td>n00</td><td>6</td><td>2</td><td>NGC 2276</td><td>12347</td><td>4</td><td>1635</td><td>336</td><td>1406</td><td>1410</td><td>6900</td></tr>\n",
       "<tr><td>RH800780N00</td><td>261.1700</td><td>85.8900</td><td>118.5825</td><td>28.5800</td><td>HRI</td><td>N</td><td>GSFC</td><td>29010</td><td>29000</td><td>RDF 3_4</td><td>50172.537326388898</td><td>50175.9321180556</td><td>A2294</td><td>BURNS</td><td>JACK</td><td>800780</td><td>n00</td><td>8</td><td>2</td><td>A2294</td><td>13937</td><td>6</td><td>2367</td><td>190</td><td>2161</td><td>2165</td><td>5000</td></tr>\n",
       "<tr><td>RH702064N00</td><td>35.7700</td><td>86.3200</td><td>124.4960</td><td>23.7300</td><td>HRI</td><td>N</td><td>GSFC</td><td>16115</td><td>19000</td><td>RDF 3_4</td><td>50171.413217592599</td><td>50173.290578703702</td><td>3C61.1</td><td>ARNAUD</td><td>KEITH</td><td>702064</td><td>n00</td><td>7</td><td>2</td><td>3C61.1</td><td>13935</td><td>6</td><td>2331</td><td>54</td><td>2160</td><td>2162</td><td>7000</td></tr>\n",
       "<tr><td>RH202532A01</td><td>37.9600</td><td>89.2600</td><td>123.2826</td><td>26.4577</td><td>HRI</td><td>N</td><td>MPE</td><td>8859</td><td>10000</td><td>RFITS V3.</td><td>50534.928391203699</td><td>50535.892175925903</td><td>ALPHA UMI</td><td>HUENSCH</td><td>MATTHIAS</td><td>202532</td><td>a01</td><td>2</td><td>2</td><td>ALPHA UMI</td><td>50167</td><td>7</td><td>3439</td><td>47</td><td>2529</td><td>2530</td><td>2900</td></tr>\n",
       "<tr><td>RH202532N00</td><td>37.9600</td><td>89.2600</td><td>123.2826</td><td>26.4577</td><td>HRI</td><td>N</td><td>MPE</td><td>2376</td><td>10000</td><td>RFITS V4.</td><td>50378.277361111097</td><td>50378.305798611102</td><td>ALPHA UMI</td><td>HUENSCH</td><td>MATTHIAS</td><td>202532</td><td>n00</td><td>2</td><td>2</td><td>ALPHA UMI</td><td>50168</td><td>7</td><td>3439</td><td>254</td><td>2370</td><td>2370</td><td>2900</td></tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<Table masked=True length=11431>\n",
       "    col1      col2     col3     col4     col5   ... col25 col26 col27 col28\n",
       "              deg      deg      deg      deg    ...  deg    d     d        \n",
       "  bytes16   float64  float64  float64  float64  ... int16 int32 int32 int16\n",
       "----------- -------- -------- -------- -------- ... ----- ----- ----- -----\n",
       "RH202299N00  49.3200 -85.5400 299.8517 -30.6815 ...   307  2163  2177  2900\n",
       "RH202299A01  49.3200 -85.5400 299.8517 -30.6815 ...   162  2316  2369  2900\n",
       "RP200594N00 151.8800 -85.0800 299.4192 -23.3707 ...    66   648   657  2900\n",
       "RH201328N00 107.9100 -84.4700 296.7740 -26.5072 ...   215   850   851  2900\n",
       "RH400876A01 193.9400 -83.3700 303.0651 -20.4994 ...   299  2699  2733  9999\n",
       "RH400876N00 193.9400 -83.3700 303.0651 -20.4994 ...    85  2543  2543  9999\n",
       "RH400877A02 190.1700 -83.3600 302.6001 -20.4952 ...    90  2893  2895  9999\n",
       "RH400877A01 190.1700 -83.3600 302.6001 -20.4952 ...   260  2733  2733  9999\n",
       "RH400877N00 190.1700 -83.3600 302.6001 -20.4952 ...    81  2543  2543  9999\n",
       "        ...      ...      ...      ...      ... ...   ...   ...   ...   ...\n",
       "RH180091N00 172.0200  85.5500 124.7826  31.2749 ...   272  2158  2158  2900\n",
       "RP900512N00 110.3000  85.7000 127.7445  27.6030 ...   132  1196  1196  3900\n",
       "RP900513N00 114.7000  85.7000 127.6963  27.9300 ...   128  1196  1196  3900\n",
       "RP900161N00 111.7200  85.7500 127.6760  27.7031 ...    12   704   706  3900\n",
       "RH600498A01 111.8000  85.7600 127.6640  27.7078 ...   140  1573  1575  6900\n",
       "RH600498N00 111.8000  85.7600 127.6640  27.7078 ...   336  1406  1410  6900\n",
       "RH800780N00 261.1700  85.8900 118.5825  28.5800 ...   190  2161  2165  5000\n",
       "RH702064N00  35.7700  86.3200 124.4960  23.7300 ...    54  2160  2162  7000\n",
       "RH202532A01  37.9600  89.2600 123.2826  26.4577 ...    47  2529  2530  2900\n",
       "RH202532N00  37.9600  89.2600 123.2826  26.4577 ...   254  2370  2370  2900"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print(type(table))\n",
    "table"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The current astroquery.heasarc package has limited functionality, but it will return astropy Tables from object queries to HEASARC catalogs.  (It does not currently use the VO interface.)  For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: UnitsWarning: 'DEGREE' did not parse as fits unit: At col 0, Unit 'DEGREE' not supported by the FITS standard.  [astropy.units.core]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "&lt;Table length=63&gt;\n",
       "<table id=\"table4561341632\" class=\"table-striped table-bordered table-condensed\">\n",
       "<thead><tr><th>SEQ_ID</th><th>INSTRUMENT</th><th>EXPOSURE</th><th>RA</th><th>DEC</th><th>NAME</th><th>SEARCH_OFFSET_</th></tr></thead>\n",
       "<thead><tr><th></th><th></th><th>S</th><th>DEGREE</th><th>DEGREE</th><th></th><th></th></tr></thead>\n",
       "<thead><tr><th>str11</th><th>str10</th><th>int32</th><th>float32</th><th>float32</th><th>str20</th><th>str15</th></tr></thead>\n",
       "<tr><td>RH701576N00</td><td>HRI</td><td>68154</td><td>187.28</td><td>2.05</td><td>3C 273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RP600242A01</td><td>PSPCB</td><td>24822</td><td>186.93</td><td>1.6</td><td>GIOVANELLI-HAYNES CL</td><td>34.236 (3C273)</td></tr>\n",
       "<tr><td>RH700234N00</td><td>HRI</td><td>17230</td><td>187.28</td><td>2.05</td><td>3C 273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RP700191N00</td><td>PSPCB</td><td>6140</td><td>187.27</td><td>2.05</td><td>3C273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH702682N00</td><td>HRI</td><td>4896</td><td>187.27</td><td>2.05</td><td>3C 273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH702681N00</td><td>HRI</td><td>4416</td><td>187.27</td><td>2.05</td><td>3C 273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH702684N00</td><td>HRI</td><td>4105</td><td>187.27</td><td>2.05</td><td>3C 273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH702680N00</td><td>HRI</td><td>4063</td><td>187.27</td><td>2.05</td><td>3C 273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH702683N00</td><td>HRI</td><td>3986</td><td>187.27</td><td>2.05</td><td>3C 273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td><td>...</td></tr>\n",
       "<tr><td>RP701259N00</td><td>PSPCB</td><td>1048</td><td>187.28</td><td>2.05</td><td>3C273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RP701258N00</td><td>PSPCB</td><td>1040</td><td>187.28</td><td>2.05</td><td>3C273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RH701974N00</td><td>HRI</td><td>982</td><td>187.28</td><td>2.05</td><td>3C273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RP120000N00</td><td>PSPCC</td><td>916</td><td>187.27</td><td>2.05</td><td>XRT/PSPC NORTH DUMMY</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH141520N00</td><td>HRI</td><td>765</td><td>187.27</td><td>2.05</td><td>3C273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH701978N00</td><td>HRI</td><td>671</td><td>187.28</td><td>2.05</td><td>3C273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RH701981N00</td><td>HRI</td><td>498</td><td>187.28</td><td>2.05</td><td>3C273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RP141520N00</td><td>PSPCB</td><td>485</td><td>187.27</td><td>2.05</td><td>3C273</td><td>0.495 (3C273)</td></tr>\n",
       "<tr><td>RH701979N00</td><td>HRI</td><td>354</td><td>187.28</td><td>2.05</td><td>3C273</td><td>0.192 (3C273)</td></tr>\n",
       "<tr><td>RH120001N00</td><td>HRI</td><td>0</td><td>187.27</td><td>2.05</td><td>XRT/HRI NORTH DUMMY</td><td>0.495 (3C273)</td></tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<Table length=63>\n",
       "   SEQ_ID   INSTRUMENT EXPOSURE ...   DEC           NAME          SEARCH_OFFSET_\n",
       "                          S     ...  DEGREE                                     \n",
       "   str11      str10     int32   ... float32        str20              str15     \n",
       "----------- ---------- -------- ... ------- -------------------- ---------------\n",
       "RH701576N00 HRI           68154 ...    2.05 3C 273                0.192 (3C273)\n",
       "\n",
       "RP600242A01 PSPCB         24822 ...     1.6 GIOVANELLI-HAYNES CL 34.236 (3C273)\n",
       "\n",
       "RH700234N00 HRI           17230 ...    2.05 3C 273                0.192 (3C273)\n",
       "\n",
       "RP700191N00 PSPCB          6140 ...    2.05 3C273                 0.495 (3C273)\n",
       "\n",
       "RH702682N00 HRI            4896 ...    2.05 3C 273                0.495 (3C273)\n",
       "\n",
       "RH702681N00 HRI            4416 ...    2.05 3C 273                0.495 (3C273)\n",
       "\n",
       "RH702684N00 HRI            4105 ...    2.05 3C 273                0.495 (3C273)\n",
       "\n",
       "RH702680N00 HRI            4063 ...    2.05 3C 273                0.495 (3C273)\n",
       "\n",
       "RH702683N00 HRI            3986 ...    2.05 3C 273                0.495 (3C273)\n",
       "\n",
       "        ...        ...      ... ...     ...                  ...             ...\n",
       "RP701259N00 PSPCB          1048 ...    2.05 3C273                 0.192 (3C273)\n",
       "\n",
       "RP701258N00 PSPCB          1040 ...    2.05 3C273                 0.192 (3C273)\n",
       "\n",
       "RH701974N00 HRI             982 ...    2.05 3C273                 0.192 (3C273)\n",
       "\n",
       "RP120000N00 PSPCC           916 ...    2.05 XRT/PSPC NORTH DUMMY  0.495 (3C273)\n",
       "\n",
       "RH141520N00 HRI             765 ...    2.05 3C273                 0.495 (3C273)\n",
       "\n",
       "RH701978N00 HRI             671 ...    2.05 3C273                 0.192 (3C273)\n",
       "\n",
       "RH701981N00 HRI             498 ...    2.05 3C273                 0.192 (3C273)\n",
       "\n",
       "RP141520N00 PSPCB           485 ...    2.05 3C273                 0.495 (3C273)\n",
       "\n",
       "RH701979N00 HRI             354 ...    2.05 3C273                 0.192 (3C273)\n",
       "\n",
       "RH120001N00 HRI               0 ...    2.05 XRT/HRI NORTH DUMMY   0.495 (3C273)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from astroquery.heasarc import Heasarc\n",
    "heasarc=Heasarc()\n",
    "heasarc.query_object('3c273',\"rosmaster\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "<a id=\"create\"></a>\n",
    "# 2. Creating a VO Table from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, we can create a table from scratch using an array of values named whatever we want.  \n",
    "\n",
    "There are several ways of doing this, and there are a few object layers here, which can be confusing.  There are standard astropy Table objects, there are votable Table objects, and there are votable VOTableFile objects that might contain multiple votable Tables. Though some things can be done with generic astropy Tables, other VO operations can only be done with VO Tables or VOTableFile objects.  \n",
    "\n",
    "This is easiest to see with an example.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'astropy.table.table.Table'>\n",
      "   RA     DEC  \n",
      "------- -------\n",
      "19.0186 46.7304\n",
      "20.2887 40.4703\n",
      "125.886 21.3377\n",
      "136.002 21.9679\n",
      "141.057 40.6372\n",
      "  146.7 22.0116\n",
      "148.785 14.2922\n",
      "149.751 17.8168\n",
      "175.039  15.327\n",
      "191.542 30.7317\n",
      "194.913 28.8959\n",
      "199.026 41.5011\n",
      "206.577 43.8511\n",
      "209.963 38.1821\n",
      "213.556 15.6214\n",
      "219.967 42.7421\n",
      "226.693 12.8502\n",
      "237.489 20.8057\n",
      "241.519 20.8014\n",
      "317.088 18.2002\n",
      "329.235 6.64845\n",
      " 333.83 37.3012\n"
     ]
    }
   ],
   "source": [
    "##  Create a table with only two columns starting from an astropy Table:  \n",
    "myaptable=aptable.Table(\n",
    "    numpy.array([\n",
    "            [19.0186,       46.7304],\n",
    "            [20.2887,       40.4703],\n",
    "            [125.886,       21.3377],\n",
    "            [136.002,       21.9679],\n",
    "            [141.057,       40.6372],\n",
    "            [146.700,       22.0116],\n",
    "            [148.785,       14.2922],\n",
    "            [149.751,       17.8168],\n",
    "            [175.039,       15.3270],\n",
    "            [191.542,       30.7317],\n",
    "            [194.913,       28.8959],\n",
    "            [199.026,       41.5011],\n",
    "            [206.577,       43.8511],\n",
    "            [209.963,       38.1821],\n",
    "            [213.556,       15.6214],\n",
    "            [219.967,       42.7421],\n",
    "            [226.693,       12.8502],\n",
    "            [237.489,       20.8057],\n",
    "            [241.519,       20.8014],\n",
    "            [317.088,       18.2002],\n",
    "            [329.235,       6.64845],\n",
    "            [333.830,       37.3012] ]), \n",
    "    names=[\"RA\",\"DEC\"])\n",
    "print(type(myaptable))\n",
    "print(myaptable)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'astropy.io.votable.tree.VOTableFile'>\n",
      "        RA                DEC        \n",
      "------------------ ------------------\n",
      "19.018599999999999 46.730400000000003\n",
      "20.288699999999999 40.470300000000002\n",
      "           125.886 21.337700000000002\n",
      "136.00200000000001            21.9679\n",
      "141.05699999999999            40.6372\n",
      "146.69999999999999 22.011600000000001\n",
      "           148.785 14.292199999999999\n",
      "           149.751 17.816800000000001\n",
      "175.03899999999999             15.327\n",
      "           191.542            30.7317\n",
      "194.91300000000001 28.895900000000001\n",
      "199.02600000000001 41.501100000000001\n",
      "           206.577 43.851100000000002\n",
      "209.96299999999999 38.182099999999998\n",
      "213.55600000000001            15.6214\n",
      "219.96700000000001 42.742100000000001\n",
      "226.69300000000001 12.850199999999999\n",
      "           237.489 20.805700000000002\n",
      "241.51900000000001 20.801400000000001\n",
      "317.08800000000002 18.200199999999999\n",
      "329.23500000000001 6.6484500000000004\n",
      "333.82999999999998 37.301200000000001\n"
     ]
    }
   ],
   "source": [
    "from astropy.io import votable as apvot\n",
    "\n",
    "## Then convert this to a VOTableFile object.\n",
    "myvotablefile = apvot.from_table(myaptable)\n",
    "print(type(myvotablefile))\n",
    "## Which contains a nested set of RESOURCES\n",
    "for r in myvotablefile.resources:\n",
    "    ## And tables (in this case, only one of each)\n",
    "    mytable=r\n",
    "    for t in r.tables:\n",
    "        print(t)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<a id=\"upload\"></a>\n",
    "\n",
    "# 3. Uploading our VOTable file to cross-correlate\n",
    "\n",
    "The reason we might want not just an astropy Table object but a votable Table object is that we might want to cross-correlate our list of objects with another catalog.  In this case, we would use the __[Table Access Protocol](http://www.ivoa.net/documents/TAP/)__ discussed more extensively in a different notebook.  \n",
    "\n",
    "Here, we simply show an example of how to hand our VOTable object to the HEASARC TAP service.\n",
    "\n",
    "We start by writing the votablefile object to a \"file-like\" object (using BytesIO) that requests will be able to upload (alternatively we could write it to disk, but why?):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "## In memory only, use an IO stream. \n",
    "vot_obj=io.BytesIO()\n",
    "apvot.writeto(myvotablefile,vot_obj)\n",
    "vot_obj.seek(0) # Reset seek to beginning of buffer\n",
    "## 'uplt' is what we'll call it (for 'upload table') \n",
    "##   in the requests parameters below, or what you will:\n",
    "files={'uplt':vot_obj}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "## Alternatively, write it to a file:\n",
    "#apvot.writeto(votablefile, \"new_votable.xml\")\n",
    "#files={'uplt':open('new_votable.xml', 'rb')}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For more information on how to construct the TAP query, see its dedicated notebook.  For our purposes here, this is simply to demonstrate that we can use our VOTable to cross-correlate with, e.g., the CfA Redshift Catalog (zcat).  In other words, the follow retrieves every entry in the zcat that is within a radius of 0.1deg of each of our 30 sources, returning the zcat entries for RA, DEC, and Radial_Velocity.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(This takes a good 20 seconds. If you need to rerun this cell, rerun the one above that defines the upload.  The IO object needs to be reset.) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING: W42: None:2:0: W42: No XML namespace specified [astropy.io.votable.tree]\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "&lt;Table masked=True length=39&gt;\n",
       "<table id=\"table4419451816\" class=\"table-striped table-bordered table-condensed\">\n",
       "<thead><tr><th>ra</th><th>dec</th><th>radial_velocity</th></tr></thead>\n",
       "<thead><tr><th>float64</th><th>float64</th><th>int32</th></tr></thead>\n",
       "<tr><td>19.068407270000002</td><td>46.740033570000001</td><td>5081</td></tr>\n",
       "<tr><td>19.086435210000001</td><td>46.747227780000003</td><td>5188</td></tr>\n",
       "<tr><td>20.334799289999999</td><td>40.487715479999999</td><td>5859</td></tr>\n",
       "<tr><td>125.90444401000001</td><td>21.338192790000001</td><td>5219</td></tr>\n",
       "<tr><td>125.90444401000001</td><td>21.338192790000001</td><td>5364</td></tr>\n",
       "<tr><td>135.99423805999999</td><td>21.90099584</td><td>3157</td></tr>\n",
       "<tr><td>136.00074370999999</td><td>21.96791867</td><td>3093</td></tr>\n",
       "<tr><td>141.09150181000001</td><td>40.683784680000002</td><td>8267</td></tr>\n",
       "<tr><td>146.70334308</td><td>22.018272169999999</td><td>7446</td></tr>\n",
       "<tr><td>146.70334308</td><td>22.018272169999999</td><td>7597</td></tr>\n",
       "<tr><td>...</td><td>...</td><td>...</td></tr>\n",
       "<tr><td>237.58156191</td><td>20.762982279999999</td><td>10926</td></tr>\n",
       "<tr><td>241.52278516999999</td><td>20.78278156</td><td>4584</td></tr>\n",
       "<tr><td>241.52278516999999</td><td>20.78278156</td><td>4602</td></tr>\n",
       "<tr><td>241.52278516999999</td><td>20.78278156</td><td>4798</td></tr>\n",
       "<tr><td>317.10346952999998</td><td>18.202987799999999</td><td>4826</td></tr>\n",
       "<tr><td>317.11477339999999</td><td>18.191914189999999</td><td>5054</td></tr>\n",
       "<tr><td>329.24927524999998</td><td>6.6551501899999996</td><td>7857</td></tr>\n",
       "<tr><td>333.82169725</td><td>37.299221410000001</td><td>5207</td></tr>\n",
       "<tr><td>333.84683658</td><td>37.282608979999999</td><td>5984</td></tr>\n",
       "<tr><td>333.91491775999998</td><td>37.300255790000001</td><td>5684</td></tr>\n",
       "</table>"
      ],
      "text/plain": [
       "<Table masked=True length=39>\n",
       "        ra                dec         radial_velocity\n",
       "     float64            float64            int32     \n",
       "------------------ ------------------ ---------------\n",
       "19.068407270000002 46.740033570000001            5081\n",
       "19.086435210000001 46.747227780000003            5188\n",
       "20.334799289999999 40.487715479999999            5859\n",
       "125.90444401000001 21.338192790000001            5219\n",
       "125.90444401000001 21.338192790000001            5364\n",
       "135.99423805999999        21.90099584            3157\n",
       "136.00074370999999        21.96791867            3093\n",
       "141.09150181000001 40.683784680000002            8267\n",
       "      146.70334308 22.018272169999999            7446\n",
       "      146.70334308 22.018272169999999            7597\n",
       "               ...                ...             ...\n",
       "      237.58156191 20.762982279999999           10926\n",
       "241.52278516999999        20.78278156            4584\n",
       "241.52278516999999        20.78278156            4602\n",
       "241.52278516999999        20.78278156            4798\n",
       "317.10346952999998 18.202987799999999            4826\n",
       "317.11477339999999 18.191914189999999            5054\n",
       "329.24927524999998 6.6551501899999996            7857\n",
       "      333.82169725 37.299221410000001            5207\n",
       "      333.84683658 37.282608979999999            5984\n",
       "333.91491775999998 37.300255790000001            5684"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## These parameters are defined in the TAP standard.  \n",
    "##  The string 'uplt' is what we set above as the name\n",
    "##  of the parameter containing this data, and\n",
    "##  the string 'mysources' is how we refer to it in \n",
    "##  the ADQL query string:\n",
    "cc_params={\n",
    "    'lang': 'ADQL', \n",
    "    'request': 'doQuery',\n",
    "    'upload':'mysources,param:uplt'\n",
    "    }\n",
    "cc_params[\"query\"]=\"\"\"\n",
    "    SELECT cat.ra, cat.dec, Radial_Velocity \n",
    "    FROM zcat cat, tap_upload.mysources mt \n",
    "    WHERE\n",
    "    contains(point('ICRS',cat.ra,cat.dec),circle('ICRS',mt.ra,mt.dec,0.1))=1\n",
    "    and Radial_Velocity > 0\n",
    "    ORDER by cat.ra\"\"\"\n",
    "r = requests.post('https://heasarc.gsfc.nasa.gov/xamin/vo/tap/sync',data=cc_params,stream=True,files=files)\n",
    "mytable=aptable.Table.read(io.BytesIO(r.content))\n",
    "mytable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}