ISWI2016_Py2_Blank.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Solar Data Processing with Python\n",
    "\n",
    "Now we have a grasp of the basics of python, but the whole reason for downloading python in the first place was to analyze solar data. Let's take a closer look at examples of solar data analysis. \n",
    "\n",
    "We will be using SunPy to access solar data. SunPy is a python package designed to interface between the powerful tools that exist in other Python Libraries with current repositories of solar data. With SunPy we will show how to: download solar data sets from the VSO an HEK, calibrate to industry standards, plot and overlay complimentary data sets. First, lets make sure that sunpy is up to date and then launch an ipython session. "
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now in ipython, we will import the sunpy library as well as the sunpy.data library. Notice that we have to call sunpy.data separately from the generic sunpy call. This is because importing sunpy only loads a subset of the \"most commonly used\" libraries. Thus if you want something specific, you will need to import it explicitly. \n",
    "\n",
    "We will also download the sample data set that is included in sunpy. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The sunpy sample_data includes many different solar data types that SunPy specifically supports. We will operate on the AIA image, but there are other examples at sunpy.org using the other sample datasets. \n",
    "\n",
    "# Overplotting a Coronal Hole on an AIA Image\n",
    "\n",
    "Let's import some other libraries that we will need for this example. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Notice that we are loading a units package (astropy.units). This is an extremely important and useful library to work with. It is required for effective interaction with Sunpy because most of the data we work with has units attached to it explicitly or implicitly (such as arcsec, sec, DN, etc.). Using units in our data analysis will help keep track of how we are manipulating data and the physical relevance of the results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "All of these packages are available in SunPy and will help us work with the solar data we have downloaded. \n",
    "\n",
    "Now let's load an AIA image from the sample data we downloaded into our map object we have just defined. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "sunpy.data.sample.AIA_171.IMAGE looks like a complicated object we are loading. In fact, it just points to a string as to where to find the sample data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So another way we could write the same command is"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "Let take a look at it using the peek( ) function:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can interact with this image of the sun. Try resizing the window, or using the zoom tool (the magnifying glass). See how the plot automatically reacts? Click on the Home to get back to the original image. \n",
    "\n",
    "Let's say we are interested in coronal holes. We can interact with the HEK (www.lmsal.com/isolsearch) through SunPy and ask it about the coronal holes that were recorded in the HEK at the same time this image was taken. \n",
    "\n",
    "First we must create a HEK client:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will specify a range of time that we are interested in. This is because the detection routines don't run for every AIA image that is recorded. Notice that we are defining things using the units hours. We could ask for seconds, days, or even years (not that we would want to). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Both start_time and end_time are objects. Let's say that we want to know the start_time in a naturally readable format. We can just ask:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's search the HEK database for coronal holes detected with the SPoCA feature detection suite."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How many coronal holes did we get? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But maybe we really don't want all of the holes detected, we just want the biggest hole that is within 60 degrees either north or south of the equator. We can sort our responses using a for loop to search all of our returned results. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are taking advantage of two of the quantities returned from our HEK search: 'area_atdiskcenter' and 'hgc_y'. You can get a list of these keywords available by selecting the first in our list of returned responses and asking for the keywords associated with it:  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next let's select the largest coronal hole and extract its boundaries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The coronal hole was detected at a certain time. To correctly plot it on the AIA image, we need to rotate it to the map observation time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have the new coordinates for the coronal hole corrected to the time of our AIA image. Now let's plot the rotated coronal hole boundary on the AIA map."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice how we defined the title for the plot using some nifty string formatting. We dynamically set up the format for the string and fill in it's contents at the same time. Let's break that down. This is how the formatting statement works: we define a generic string with two components '{ } { }'. Next, we define what goes in each component: '{0} {1}'.format('the zeroth component', 'the first component'}. Finally, we specify that we want a new line between the first and second component: '{0}\\n{1}'.format('the zeroth component', 'the first component')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Downloading Data With SunPy Through The VSO\n",
    "\n",
    "Data is the life blood of solar research. We have queried the HEK, but how do we search and download data from the VSO? SunPy has a VSO module to directly interact with the VSO so we can search, sort, and download solar data. First, we need to load the VSO client:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's construct a basic query and ask for all EIT images on Jan 1, 2001 with wavelengths between 170 and 180 Angstroms.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Notice the syntax we are using similar to how we query the HEK database. To communicate the exact parameters we want the VSO to search over, we are using the vso attributes function (vso.attrs). To find out all of the VSO attributes that exist, you can type 'help(vso.attrs)', or look online at sunpy.org, or type 'vso.attrs.'+\"tab\" and see a list.\n",
    "\n",
    "SunPy expects units to be specified where they make physical sense. So we must specify that the wavelengths we are looking for is in angstroms (rather than nanometers or even km). The units package specifies angstroms as units.AA.\n",
    "\n",
    "How many results did we get?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What do the results look like? Let's print the first one. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can see that the results include the meta-data but not the data itself. Now let's download the images in our data query from the VSO to our current working directory. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each downloaded file with the filename '{file}' obtained from the VSO is appended with the suffix .fits. The '{file}' option uses the file name obtained by the VSO for each file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}