03-Bayesian-Modeling-With-MCMC.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bayesian Modeling with MCMC"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the previous section we explored a Bayesian solution to a straight line fit.\n",
    "The result made use of the evaluation of a posterior across a grid of parameters: a strategy that *will not* scale to higher-dimensional models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The Curse of Dimensionality\n",
    "\n",
    "The reason it will not scale is one of the effects of the ubiquitous \"Curse of Dimensionality\". To understand this, consider how many evaluations we need for an $N$-dimensional grid with 100 samples per dimension\n",
    "\n",
    "In one dimension, we have $100$ points.\n",
    "\n",
    "In two dimensions we have $100^2 = 10,000$ evaluations.\n",
    "\n",
    "In three dimensions, we have $100^3 = 1,000,000$ evaluations.\n",
    "\n",
    "In $N$ dimensions, we have $100^N$ evaluations, and as $N$ grows this quickly becomes untenable! For example, if we have only six model parameters, this \"dense grid\" approach will require evaluating the posterior at one trillion grid points, the results of which would require several terabytes of memory just to store!\n",
    "\n",
    "Evidently the dense grid strategy will not work for any but the simplest Bayesian models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Circumventing the Curse with Sampling\n",
    "\n",
    "An idea that revolutionized Bayesian modeling (and made possible the wide variety of Bayesian approaches used in practice today) is *Markov Chain Monte Carlo* (MCMC), an approach that allows one to efficiently draw (pseudo)random samples from a posterior distribution even in relatively high dimensions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The Metropolis-Hastings Sampler\n",
    "\n",
    "Perhaps the simplest of MCMC samplers is the *Metropolis-Hastings Sampler*.\n",
    "This provides a procedure which, given a pseudo-random number generator, selects a chain of points which (in the long-term limit) will be a representative sample from the posterior. The procedure is surprisingly simple:\n",
    "\n",
    "1. Define a posterior $p(\\theta~|~D, I)$\n",
    "2. Define a *proposal density* $p(\\theta_{i + 1}~|~\\theta_i)$, which must be a symmetric function, but otherwise is unconstrained (a Gaussian is the usual choice).\n",
    "3. Choose a starting point $\\theta_0$\n",
    "4. Repeat the following:\n",
    "\n",
    "   1. Given $\\theta_i$, draw a new $\\theta_{i + 1}$\n",
    "   \n",
    "   2. Compute the *acceptance ratio*\n",
    "      $$\n",
    "      a = \\frac{p(\\theta_{i + 1}~|~D,I)}{p(\\theta_i~|~D,I)}\n",
    "      $$\n",
    "   \n",
    "   3. If $a \\ge 1$, the proposal is more likely: accept the draw and add $\\theta_{i + 1}$ to the chain.\n",
    "   \n",
    "   4. If $a < 1$, then accept the point with probability $a$: this can be done by drawing a uniform random number $r$ and checking if $a < r$. If the point is accepted, add $\\theta_{i + 1}$ to the chain. If not, then add $\\theta_i$ to the chain *again*.\n",
    "\n",
    "There are a few caveats to be aware of when using MCMC\n",
    "\n",
    "#### 1. The procedure is provably correct... but only in the long-term limit!\n",
    "\n",
    "Sometimes the long-term limit is **very** long. What we're looking for is \"stabilization\" of the MCMC chain, meaning that it has reached a statistical equilibrium. There is a vast literature on how to measure stabilization of an MCMC chain. Here we'll use the sloppy but intuitive LAI approach (i.e. Look At It).\n",
    "\n",
    "#### 2. The size of the proposal distribution is *very* important\n",
    "\n",
    "- If your proposal distribution is too small, it will take too long for your chain to move, and you have the danger of getting stuck in a local maximum for a long (but not infinite) time.\n",
    "\n",
    "- If your proposal distribution is too large, you will not be able to efficiently explore the space around a particular peak\n",
    "\n",
    "In general, choosing an appropriate scale for the proposal distribution is one of the most difficult parts of using the MCMC procedure above.\n",
    "More sophisticated methods (such as what we will use below) have built-in ways to estimate this along the way, but it's still something to be aware of!\n",
    "\n",
    "#### 3. Fast Stabilization can be helped by good initialization\n",
    "\n",
    "In practice, assuring that MCMC will stabilize quickly has a lot to do with choosing a suitable initialization. For this purpose, it can be useful to find the maximum a posteriori (MAP) value, and initialize the chain with this."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A bit later in the session, we will spend some time actually coding a simple Metropolis-Hastings sampler which uses the above procedure.\n",
    "But first let's take a look at MCMC in action."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sampling with ``emcee``\n",
    "\n",
    "There are several good Python approaches to Bayesian computation with MCMC. You can read about some of them in this blog post: [How to Be a Bayesian in Python](http://jakevdp.github.io/blog/2014/06/14/frequentism-and-bayesianism-4-bayesian-in-python/).\n",
    "\n",
    "Here we'll focus on [``emcee``](http://dan.iel.fm/emcee/), a lightweight Python package developed by Dan Foreman-Mackey and collaborators.\n",
    "One benefit of ``emcee`` is that it uses an *ensemble sampler* which automatically tunes the shape and size of the proposal distribution (you can read more details in the ``emcee`` documentation)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Creating some Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn; seaborn.set() # for plot formatting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def make_data(intercept, slope, N=20, dy=5, rseed=42):\n",
    "    rand = np.random.RandomState(rseed)\n",
    "    x = 100 * rand.rand(N)\n",
    "    y = intercept + slope * x\n",
    "    y += dy * rand.randn(N)\n",
    "    return x, y, dy * np.ones_like(x)\n",
    "\n",
    "theta_true = (25, 0.5)\n",
    "x, y, dy = make_data(*theta_true)\n",
    "\n",
    "plt.errorbar(x, y, dy, fmt='o');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Defining our Posterior"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# theta = [intercept, slope]\n",
    "\n",
    "def log_prior(theta):\n",
    "    if np.all(np.abs(theta) < 1000):\n",
    "        return 0\n",
    "    else:\n",
    "        return -np.inf  # log(0)\n",
    "    \n",
    "def log_likelihood(theta, x, y, dy):\n",
    "    y_model = theta[0] + theta[1] * x\n",
    "    return -0.5 * np.sum(np.log(2 * np.pi * dy ** 2) +\n",
    "                         (y - y_model) ** 2 / dy ** 2)\n",
    "\n",
    "def log_posterior(theta, x, y, dy):\n",
    "    return log_prior(theta) + log_likelihood(theta, x, y, dy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using ``emcee`` to Sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import emcee\n",
    "\n",
    "ndim = 2  # number of parameters in the model\n",
    "nwalkers = 50  # number of MCMC walkers\n",
    "\n",
    "# initialize walkers\n",
    "starting_guesses = np.random.randn(nwalkers, ndim)\n",
    "\n",
    "sampler = emcee.EnsembleSampler(nwalkers, ndim, log_posterior,\n",
    "                                args=[x, y, dy])\n",
    "pos, prob, state = sampler.run_mcmc(starting_guesses, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the chains"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(2, sharex=True)\n",
    "for i in range(2):\n",
    "    ax[i].plot(sampler.chain[:, :, i].T, '-k', alpha=0.2);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Restarting after burn-in"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "sampler.reset()\n",
    "pos, prob, state = sampler.run_mcmc(pos, 1000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(2, sharex=True)\n",
    "for i in range(2):\n",
    "    ax[i].plot(sampler.chain[:, :, i].T, '-k', alpha=0.2);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing the Posterior"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using the [corner.py](https://pypi.python.org/pypi/corner) package, we can take a look at this multi-dimensional posterior, along with the input values for the parameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import corner\n",
    "corner.corner(sampler.flatchain, labels=['intercept', 'slope']);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Another way to visualize the posterior is to plot the model over the data.\n",
    "Each point in the two-dimensional space above corresponds to a possible model for our data; if we select ~100 of these at random and plot them over our data, it will give us a good idea of the spread in the model results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "chain = sampler.flatchain\n",
    "\n",
    "plt.errorbar(x, y, dy, fmt='o');\n",
    "\n",
    "thetas = [chain[i] for i in np.random.choice(chain.shape[0], 100)]\n",
    "\n",
    "xfit = np.linspace(0, 100)\n",
    "for i in range(100):\n",
    "    theta = thetas[i]\n",
    "    plt.plot(xfit, theta[0] + theta[1] * xfit,\n",
    "             color='black', alpha=0.05);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Breakout: Linear Fit with Intrinsic Scatter\n",
    "\n",
    "Above we have done a simple model, where the data is drawn from a straight line.\n",
    "\n",
    "Often, however, we will be modeling relationships where there is some intrinsic scatter in the model itself: that is, even if the data were *perfectly* measured, they would not fall along a perfect straight line, but would have some (unknown) scatter about that line.\n",
    "\n",
    "Here we'll make a slightly more complicated model in which we will fit for the slope, intercept, and intrinsic scatter all at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def make_data_scatter(intercept, slope, scatter,\n",
    "                      N=20, dy=2, rseed=42):\n",
    "    rand = np.random.RandomState(rseed)\n",
    "    x = 100 * rand.rand(20)\n",
    "    y = intercept + slope * x\n",
    "    y += np.sqrt(dy ** 2 + scatter ** 2) * rand.randn(20)\n",
    "    return x, y, dy * np.ones_like(x)\n",
    "\n",
    "\n",
    "# (intercept, slope, intrinsic scatter)\n",
    "theta = (25, 0.5, 3.0)\n",
    "x, y, dy = make_data_scatter(*theta)\n",
    "plt.errorbar(x, y, dy, fmt='o');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following will walk you through using ``emcee`` to fit a model to this data.\n",
    "Feel free to copy and adapt the code from the simple linear example above; if you'd like to see a solution, you can look at the [Solutions-03](Solutions-03.ipynb) notebook – but please resist the temptation to look before you have made a good attempt at this yourself!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Define the prior, likelihood, and posterior\n",
    "\n",
    "The likelihood for this model looks very similar to what we used above, except that the intrinsic scatter is added *in quadrature* to the measurement error.\n",
    "If $\\varepsilon_i$ is the measurement error on the point $(x_i, y_i)$, and $\\sigma$ is the intrinsic scatter, then the likelihood should look like this:\n",
    "\n",
    "$$\n",
    "P(x_i,y_i\\mid\\theta) = \\frac{1}{\\sqrt{2\\pi(\\varepsilon_i^2 + \\sigma^2)}} \\exp\\left(\\frac{-\\left[y_i - y(x_i;\\theta)\\right]^2}{2(\\varepsilon_i^2 + \\sigma^2)}\\right)\n",
    "$$\n",
    "\n",
    "For the prior, you can use either a flat or symmetric prior on the slope and intercept, but on the intrinsic scatter $\\sigma$ it is best to use a scale-invariant Jeffreys Prior:\n",
    "\n",
    "$$\n",
    "P(\\sigma)\\propto\\sigma^{-1}\n",
    "$$\n",
    "\n",
    "As discussed before, this has the nice feature that the resulting posterior will not depend on the units of measurement."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Define functions to compute the log-prior, log-likelihood, and log-posterior\n",
    "\n",
    "# theta = [intercept, slope, scatter]\n",
    "\n",
    "def log_prior(theta):\n",
    "    # fill this in\n",
    "    pass\n",
    "    \n",
    "def log_likelihood(theta, x, y, dy):\n",
    "    # fill this in\n",
    "    pass\n",
    "\n",
    "def log_posterior(theta, x, y, dy):\n",
    "    # fill this in\n",
    "    pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Sampling from the Posterior"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Using emcee, create and initialize a sampler and draw 200 samples from the posterior.\n",
    "# Remember to think about what starting guesses should you use!\n",
    "# You can use the above as a template\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing the Chains"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Plot the three chains as above\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Resetting and getting a clean sample"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Are your chains stabilized? Reset them and get a clean sample"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing the results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Use corner.py to visualize the three-dimensional posterior\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Next plot ~100 of the samples as models over the data to get an idea of the fit\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}