{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "preamble": true
   },
   "source": [
    "(Defining latex commands: not to be shown...)\n",
    "$$\n",
    "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
    "\\DeclareMathOperator{\\minimize}{minimize}\n",
    "\\DeclareMathOperator{\\maximize}{maximize}\n",
    "\\newcommand{\\real}{\\mathbb{R}}\n",
    "\\newcommand{\\blasso}{\\beta^{\\mathrm{LASSO}}}\n",
    "\\newcommand{\\bzero}{\\beta^0}\n",
    "\\newcommand{\\bLS}{\\hat{\\beta}^{\\mathrm{LS}}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LASSO and $\\ell_0$ estimator for orthonormal data (30 points)\n",
    "\n",
    "Let $X \\in \\real^{n \\times p}$ be your training input set, and $Y \\in \\real^n$ your training output.\n",
    "In a linear model we predict $\\hat{y}(x) = x^T \\hat{\\beta}$ for some input $x \\in \\real^p$ and some constant $\\hat{\\beta} \\in \\real^p$.\n",
    "The so-called $\\ell_0$-penalized estimator $\\bzero(\\lambda)$ tries to find a good $\\beta$ with reslatively few non-zero entries. It is defined as:\n",
    "\\begin{equation}\n",
    "    \\bzero(\\lambda) := \\mathrm{arg \\, min}_{\\beta} \\frac{1}{n} \\norm{Y - X \\beta}_2^2 + \\lambda \\norm{\\beta}_0 ,\n",
    "\\end{equation}\n",
    "where $\\norm{\\beta}_0 := \\sharp \\{j:\\beta_j \\neq 0\\}$ and $\\lambda > 0$.\n",
    "Alternatively, the LASSO estimator $\\blasso(\\lambda)$ is defined as:\n",
    "\\begin{equation}\n",
    "    \\blasso(\\lambda) := \\mathrm{arg \\, min}_{\\beta} \\frac{1}{n} \\norm{Y - X \\beta}_2^2 + \\lambda \\norm{\\beta}_1 ,\n",
    "\\end{equation}\n",
    "where $\\norm{\\beta}^2_2 := \\sum_{i=1}^p \\beta_i^2$, $\\norm{\\beta}_1 := \\sum_{i=1}^p |\\beta_i|$ and $\\lambda > 0$.  \n",
    "\n",
    "There are in general no analytic formulae for $\\bzero(\\lambda)$ and $\\blasso(\\lambda)$. In practice, they are computed by numerical optimization.\n",
    "However, we can compute an analytic expression in the special case of _orthogonal design_, where\n",
    "\\begin{equation}\n",
    "    p=n \\quad \\mathrm{and} \\quad \\frac{1}{n} X^T X = I_{p \\times p} .\n",
    "\\end{equation}\n",
    "The goal of this exercise is to compute these formulae."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## $\\ell_0$-regularized estimator  in the orthonormal design\n",
    "\n",
    "Let $g_{\\mathrm{hard}, \\lambda}$ be the function:\n",
    "\\begin{equation}\n",
    "    g_{\\mathrm{hard}, \\lambda} : \\real \\rightarrow \\real , \\quad z \\rightarrow z \\, 1_{ \\{|z| > \\lambda\\} } ,\n",
    "\\end{equation}\n",
    "where $1_{ \\{|z| > \\lambda\\} } = 1$ if $|z| > \\lambda$ and $0$ otherwise.\n",
    "Suppose we are in the orthonormal design.\n",
    "\n",
    "1. Plot $g_{\\mathrm{hard}, \\lambda}$ for $\\lambda = 1$.\n",
    "2. Write down the analytical formulae for the least square estimator $\\bLS$. (Not needed for the subsequent calculations)\n",
    "3. Prove that $\\bzero_j(\\lambda) = g_{\\mathrm{hard}, \\sqrt{\\lambda}}(z_j)$ where, again, $z_j := \\frac{1}{n}(X^T Y)_j$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import numpy as np\n",
    "from numpy import *\n",
    "from scipy import *\n",
    "from matplotlib import pyplot as plt\n",
    "from sklearn import linear_model\n",
    "from pandas import * # for easy import of data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAADa1JREFUeJzt3U+oXOUdxvHnqcZaRRBRktQG0oVSA6J2EQsuvIsqEWlt\n",
       "FtW6EhpUKHGrVsGkdaMFsxBpKW1MLLWKGyVSUpMIl0qhiBBsqAYVDCQSr0JLSxet/35dzFw5uffO\n",
       "ZP6c97znvOf7gUvuzJ0752Ucfn7zzpyJI0IAgDJ9JfcCAADpMOQBoGAMeQAoGEMeAArGkAeAgjHk\n",
       "AaBgcw9520/bXrJ9rHLdJbYP237H9iHbF897HADA9Ooo+X2Stq247kFJhyPiSkmvDi8DABrmOk6G\n",
       "sr1Z0ssRcfXw8nFJN0bEku0NkhYj4ltzHwgAMJVUe/LrI2Jp+P2SpPWJjgMAGCP5C68x+KsCn50A\n",
       "ABmcm+h+l2xviIgPbW+U9NHKG9hm8APADCLCk9421ZA/IOkuSY8P/3xprRtNs1CMZ3t3ROzOvY5S\n",
       "8HjWi8dzNrbWSXpI0k5J90vaL/mLae5j7iFv+zlJN0q61PZJSY9IekzSC7Z3SDoh6fZ5jwMAfWLr\n",
       "Gkn7JZ2WdF2ETg2vn8rcQz4i7hzxo+/Oe98A0Ddr1XvE7K9rptquQfMWcy+gMIu5F1CYxdwL6IJR\n",
       "9T7Xfeb6R0NsB3vyADBdvU87Oyl5AMgoRb1X8QFlAJCBrXW2dkk6IulJSbfWPeAlSh4AGpe63qso\n",
       "eQBoSFP1XkXJA0ADmqz3KkoeABLKUe9VlDwAJJKr3qsoeQCoWe56r6LkAaBGbaj3KkoeAGrQpnqv\n",
       "ouQBYE5tq/cqSh4AZtTWeq+i5AFgBm2u9ypKHgCm0IV6r6LkAWBCXan3KkoeAM6ia/VeRckDwBhd\n",
       "rPcqSh4A1tDleq+i5AFgha7XexUlDwBDpdR7FSUPACqr3qsoeQC9VmK9V1HyAHqr1HqvouQB9E7p\n",
       "9V5FyQPolT7UexUlD6AX+lTvVZQ8gOL1rd6rKHkAxeprvVdR8gCK1Od6r6LkARSFej8TJQ+gGNT7\n",
       "apQ8gM6j3kej5AF0GvU+HiUPoJOo98lQ8gA6h3qfHCUPoDOo9+lR8gA6gXqfDSUPoNWo9/lQ8gBa\n",
       "i3qfHyUPoHWo9/pQ8gBahXqvFyUPoBWo9zQoeQDZUe/pUPIAsqHe06PkAWRBvTcj6ZC3fULSvyV9\n",
       "LunTiNia8ngA2s/WOkkPSdop6X5J+yMUeVdVrtQlH5IWIuIfiY8DoAOo9+Y1sSfvBo4BoMXYe8+n\n",
       "iZI/YvtzSb+OiN8kPh6AlqHe80o95G+IiNO2L5N02PbxiHht+Ye2d1duuxgRi4nXA6Ah7L3Xw/aC\n",
       "pIWZfz+imcfc9i5J/4mIJ4aXIyLYygEKtKLe76He6zPt7Ey2J2/7AtsXDb+/UNLNko6lOh6A/Nh7\n",
       "b5+U2zXrJb1oe/k4z0bEoYTHA5ARe+/t1Nh2zaoDs10DFIG992ZNOzs54xXAzKj39uOzawBMjb33\n",
       "7qDkAUyFeu8WSh7ARKj3bqLkAZwV9d5dlDyAkaj37qPkAayJei8DJQ/gDNR7WSh5AF+i3stDyQOg\n",
       "3gtGyQM9R72XjZIHeop67wdKHugh6r0/KHmgR6j3/qHkgZ6g3vuJkgcKR733GyUPFIx6ByUPFIh6\n",
       "xzJKHigM9Y4qSh4oBPWOtVDyQAGod4xCyQMdRr3jbCh5oKOod0yCkgc6hnrHNCh5oEOod0yLkgc6\n",
       "gHrHrCh5oOWod8yDkgdainpHHSh5oIWod9SFkgdahHpH3Sh5oCWod6RAyQOZUe9IiZIHMqLekRol\n",
       "D2RAvaMplDzQMOodTaLkgYZQ78iBkgcaQL0jF0oeSIh6R26UPJAI9Y42oOSBBGzdLeodLUDJA2nc\n",
       "LunHEXo590LQb5Q8kMb5kv6VexEAQx5I46uS/pt7EQBDHkjjfDHk0QIMeSCN8yX9L/ciAIY8kAbb\n",
       "NWgFhjyQBiWPVkg25G1vs33c9ru2H0h1HKClKHm0QpIhb/scSU9J2iZpi6Q7bV+V4lhAS/HCK1oh\n",
       "VclvlfReRJyIiE8lPS/ptkTHAlrFljUo+U9yrwVIdcbr5ZJOVi6fknR9omMBX7K1TtJNks7JuIxz\n",
       "JX0SoS8yrgGQlG7IxyQ3sr27cnExIhaTrAZ9slXS7yX9JfM69mc+Pgphe0HSwqy/n2rIfyBpU+Xy\n",
       "Jmn1BzRFxO5Ex0d/fU3S0Qh9L/dCgDoM43dx+bLtXdP8fqo9+TckXWF7s+3zJN0h6UCiYwFVvKsF\n",
       "qEhS8hHxme2dkl7RYG90b0S8neJYwAq8qwWoSPZRwxFxUNLBVPcPjMBJSEAFZ7yiNGzXABUMeZSG\n",
       "kgcqGPIoDSUPVDDkURpeeAUqGPIoDds1QAVDHqVhuwaoYMijGLau0eCD8E7nXgvQFgx5dJ6tdbZ2\n",
       "SToi6ReSfpd5SUBrJDsZCmjCsN73a1Dv10Ws/owkoM8oeXTSinp/UtKtDHhgNUoenUO9A5Oj5NEZ\n",
       "1DswPUoenUC9A7Oh5NFq1DswH0oerUW9A/Oj5NE61DtQH0oerUK9A/Wi5NEK1DuQBiWP7Kh3IB1K\n",
       "HtlQ70B6lDyyoN6BZlDyaBT1DjSLkkdjqHegeZQ8kqPegXwoeSRFvQN5UfJIgnoH2oGSR+2od6A9\n",
       "KHnUhnoH2oeSRy2od6CdKHnMhXoH2o2Sx8yod6D9KHlMjXoHuoOSx1Sod6BbKHlMhHoHuomSx1lR\n",
       "70B3UfIYiXoHuo+Sx5qod6AMlDzOQL0DZaHk8SXqHSgPJQ/qHSgYJd9z1DtQNkq+p6h3oB8o+R6i\n",
       "3oH+oOR7hHoH+oeS7wnqHegnSr5w1DvQb5R8wah3AElK3vZu26dsHx1+bUtxHKyNegewLFXJh6Q9\n",
       "EbEn0f1jBOodQFXKPXknvG+sQL0DWEvKIX+f7Tdt77V9ccLj9N6w3l+XdL0G9b4vQpF5WQBawBGz\n",
       "zQLbhyVtWONHD0v6q6SPh5cflbQxInas+P2Q9LPKVYsRsTjTYnrK1jpJD0naKel+SfsZ7kBZbC9I\n",
       "WqhctSsiJt4pmXnIT3wAe7OklyPi6hXXxzQLxZlW7L3fw9YM0A/Tzs5U767ZWLm4XdKxFMfpI/be\n",
       "AUwj1btrHrd9rQbvsnlf0r2JjtMrvHMGwLSSb9eMPDDbNRNj7x3AsmlnJ2e8thz1DmAefHZNS7H3\n",
       "DqAOlHwLUe8A6kLJtwj1DqBulHxLUO8AUqDkM6PeAaREyWdEvQNIjZLPgHoH0BRKvmHUO4AmUfIN\n",
       "od4B5EDJN4B6B5ALJZ8Q9Q4gN0o+EeodQBtQ8jWj3gG0CSVfI+odQNtQ8jWg3gG0FSU/J+odQJtR\n",
       "8jOi3gF0ASU/A+odQFdQ8lOg3gF0DSU/IeodQBdR8mdBvQPoMkp+DOodQNdR8mug3gGUgpJfgXoH\n",
       "UBJKfoh6B1AiSl7UO4By9brkqXcApettyVPvAPqgdyVPvQPok16VPPUOoG96UfLUO4C+Kr7kqXcA\n",
       "fVZsyVPvAFBoyVPvADBQVMlT7wBwpmJKnnoHgNU6X/LUOwCM1umSp94BYLxOljz1DgCT6VzJU+8A\n",
       "MLnOlDz1DgDT60TJU+8AMJtWlzz1DgDzaW3JU+8AML/WlTz1DgD1mXnI2/6h7b/b/tz2t1f87Ke2\n",
       "37V93PbNk9+nrpH0uqTrNaj3fRGKWdcIAH03T8kfk7Rd0p+rV9reIukOSVskbZP0S9tjj0O9z8/2\n",
       "Qu41lITHs148nvnMPOQj4nhEvLPGj26T9FxEfBoRJyS9J2nrqPuh3muzkHsBhVnIvYDCLOReQF+l\n",
       "2JP/unRGhZ+SdPlaN6TeASCtse+usX1Y0oY1fvRQRLw8xXFGlflyvTPcASCBsUM+Im6a4T4/kLSp\n",
       "cvkbw+vW4FsknbRnOApWsb0r9xpKwuNZLx7PPOp6n3x1TB+Q9AfbezTYprlCgz33M0QEox0AEpvn\n",
       "LZTbbZ+U9B1Jf7R9UJIi4i1JL0h6S9JBST+JCF5IBYAMzPwFgHI1fsZripOoMGB7t+1Tto8Ov7bl\n",
       "XlPX2N42fP69a/uB3OvpOtsnbP9t+HxctW2L8Ww/bXvJ9rHKdZfYPmz7HduHbF887j5yfKxBbSdR\n",
       "YZWQtCcirht+/Sn3grrE9jmSntLg+bdF0p22r8q7qs4LSQvD5+PI82Uw0j4Nno9VD0o6HBFXSnp1\n",
       "eHmkxodoXSdRYSRe0J7dVknvRcSJiPhU0vMaPC8xH56TM4qI1yT9c8XV35f0zPD7ZyT9YNx9tKmU\n",
       "Jz6JCmPdZ/tN23vP9tc4rHK5pJOVyzwH5xeSjth+w/bduRdTiPURsTT8fknS+nE3TvJRww2cRNVb\n",
       "Yx7bhyX9StLPh5cflfSEpB0NLa0EPN/qd0NEnLZ9maTDto8P6xQ1iIiwPfZ5m2TIpz+Jqr8mfWxt\n",
       "/1bSNP9Dxern4CaJs7HnERGnh39+bPtFDbbEGPLzWbK9ISI+tL1R0kfjbpx7u2blSVQ/sn2e7W9q\n",
       "xElUGG34H3zZdg1e5Mbk3pB0he3Nts/T4I0ABzKvqbNsX2D7ouH3F0q6WTwn63BA0l3D7++S9NK4\n",
       "Gzf+L0PZ3q7BB5JdqsFJVEcj4paIeMv28klUn4mTqGbxuO1rNdh2eF/SvZnX0ykR8ZntnZJekXSO\n",
       "pL0R8XbmZXXZekkvevC5JedKejYiDuVdUrfYfk7SjZIuHZ58+oikxyS9YHuHpBOSbh97H8xRAChX\n",
       "7u0aAEBCDHkAKBhDHgAKxpAHgIIx5AGgYAx5ACgYQx4ACsaQB4CC/R+oQQpi+aBhdAAAAABJRU5E\n",
       "rkJggg==\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1032db490>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def ghard (l, z):\n",
    "    if(np.abs(z) > l):\n",
    "        return z\n",
    "    return 0\n",
    "\n",
    "vg = np.vectorize(ghard)\n",
    "\n",
    "x = np.linspace(-10, 10, 1000)    \n",
    "y = vg(1, x)\n",
    "\n",
    "plt.plot(x, y, '-')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### 1.1.2\n",
    "\\begin{align*}%not sure whether this is meant...\n",
    "    \\hat{\\beta}^{LS}&=\\left(X^TX\\right)^{-1}X^T\\mathbf{y}\\\\\n",
    "    &=\\left(nI_{n\\times n} \\right)^{-1}X^T\\mathbf{y}\\\\\n",
    "    &=\\frac{1}{n}\\cdot I_{n\\times n}X^T\\mathbf{y}\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LASSO estimator in the orthonormal design\n",
    "\n",
    "Let $g_{\\mathrm{soft}, \\lambda}$ be the function:\n",
    "\\begin{equation}\n",
    "    g_{\\mathrm{soft}, \\lambda} : \\real \\rightarrow \\real , \\quad z \\rightarrow \\mathrm{sign}(z) \\, (|z| - \\lambda)_+ .\n",
    "\\end{equation}\n",
    "Suppose we are in the orthonormal design.\n",
    "\n",
    "1. Plot $g_{\\mathrm{soft}, \\lambda}$ for $\\lambda = 1$.\n",
    "2. Prove that $\\blasso_j(\\lambda) = g_{\\mathrm{soft}, \\frac{\\lambda}{2}}(z_j)$, where $z_j := \\frac{1}{n}(X^T Y)_j$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAXkAAAEACAYAAABWLgY0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAEBZJREFUeJzt3WusZXddh/HnS0nVlkspl2nFmvoClIYiYCxouQy3MlxK\n",
       "O4QCVbEiIojgOxUl2kEUW7QVATUSCukLhfAGbLl2QEeqBqUJ0Co0hUiTFsuACgFiiBR/vlirsHpm\n",
       "5szZ++y1115rPZ9kMmfvc/ZZ/+zu/Pqc/1lrT6oKSdI03WPoBUiS+uOQl6QJc8hL0oQ55CVpwhzy\n",
       "kjRhDnlJmrBdD/kkb09yOMlNnftOTXIwyS1Jrktyym6PI0la3CpK/h3Avi33vRo4WFUPBT7a3pYk\n",
       "rVlWcTFUkjOBa6vq7Pb2zcATq+pwktOAQ1X1Y7s+kCRpIX3tye+pqsPtx4eBPT0dR5K0jd5/8VrN\n",
       "jwq+d4IkDeCePX3fw0lOq6ovJTkd+PLWL0ji4JekJVRVdvq1fQ35a4BLgMvbv997tC9aZKHaXpID\n",
       "VXVg6HVMhc/navl8rs6igbyKUyjfCfwT8KNJbkvyYuAy4GlJbgGe3N6WJK3Zrku+qi4+xqeeutvv\n",
       "LUnaHa94nY5DQy9gYg4NvYCJOTT0AuZqJefJL3XgpNyTl6TFLDo7LXlJmjCHvCRNmENekibMIS9J\n",
       "E+aQl6SRSLho0cf0dcWrJGlFEh4E/Bnw8EUfa8lL0gZr6/1G4N+BRy36eEtekjbQlnq/sIqPt/cv\n",
       "xJKXpA2ztd7vGvDLsOQlaUMcq953w5KXpA2wynrvsuQlaUB91HuXJS9JA+mr3rsseUlas77rvcuS\n",
       "l6Q1Wke9d1nykrQG66z3Lkteknq27nrvsuQlqSdD1XuXJS9JPRiy3rsseUlaoU2o9y5LXpJWZFPq\n",
       "vcuSl6Rd2rR677LkJWkXNrHeuyx5SVrCJtd7lyUvSQva9HrvsuQlaYfGUu9dlrwk7cCY6r3Lkpek\n",
       "bYyx3rsseUk6hrHWe5clL0lbjL3euyx5SeqYQr13WfKSxLTqvcuSlzR7U6v3Lkte0mxNtd67LHlJ\n",
       "szTleu+y5CXNyhzqvcuSlzQbc6n3Lkte0uTNrd67LHlJkzbHeu+y5CVN0pzrvcuSlzQ5c6/3Lkte\n",
       "0mRY70ey5CVNgvV+dJa8pFGz3rdnyUsaLev9+Hot+SS3Al8HvgN8u6rO6fN4kubBet+5vku+gL1V\n",
       "9SgHvKRVsN4Xs449+azhGJImznpfzjpK/iNJbkjy0p6PJWmirPfl9V3y51bVHUkeCBxMcnNVXX/X\n",
       "J5Mc6Hztoao61PN6JI2I9Q5J9gJ7l358Va1sMdseKLkU+GZVXdHerqpyK0fSUbX1/mbgauDSKr41\n",
       "8JI2wqKzs7eST3IScEJVfSPJycB5wGv7Op6kabDeV6vPPfk9wPVJPgX8M/C+qrqux+NJGjn33ldv\n",
       "bds1RxzY7RpJrS31/mKH+7EtOju94lXSoKz3fvneNZIG4d77eljyktbOel8fS17S2ljv62fJS1oL\n",
       "630YlrykXlnvw7LkJfXGeh+eJS9p5az3zWHJS1op632zWPKSVsJ630yWvKRds943lyUvaWnW++az\n",
       "5CUtxXofB0te0kKs93Gx5CXtmPU+Ppa8pOOy3sfLkpe0Let93Cx5SUdlvU+DJS/pCNb7dFjykr7L\n",
       "ep8eS14SYL1PlSUvzZz1Pm2WvDRj1vv0WfLSDFnv82HJSzNjvc+LJS/NhPU+T5a8NAPW+3xZ8tKE\n",
       "We+y5KWJst4Flrw0Oda7uix5aUKsd21lyUsTYL3rWCx5aeSsd23HkpdGynrXTljy0ghZ79opS14a\n",
       "Eetdi7LkpZGw3rUMS17acNa7dsOSlzaY9a7dsuSlDWS9a1UseWnDWO9aJUte2hDWu/pgyUsbwHpX\n",
       "Xyx5aUDWu/pmyUsDsd61Dpa8tGbWu9bJkpfWyHrXuvVW8kn2AW8ETgDeVlWX93UsadNZ7xpKLyWf\n",
       "5ATgLcA+4Czg4iQP6+NY0qaz3jWkvkr+HODzVXUrQJJ3ARcAn+3peNLGsd61Cfoa8g8Gbuvcvh14\n",
       "TE/Hkr4r4QeAJ9FsEw7ph4HfAa4GXlTFtwZej2aqryFfO/miJAc6Nw9V1aFeVqNZSDgXeAdwGPja\n",
       "wMv5JnB+FZ8YeB0auSR7gb3LPr6vIf9F4IzO7TNoav5uqupAT8fXjCScBPw+8ELgV6t4z8BLklam\n",
       "jd9Dd91Ocukij+/rFMobgIckOTPJicALgGt6OpZmrK33TwF7gLMd8NLd9VLyVXVnklcCH6bZG72q\n",
       "qvylq1ZmS72/oor3DrwkaSOlakfb56s/cFJVlUEOrlFLeBzwduATwK9V8V8DL0lam0Vnp29roNFo\n",
       "6/0PaLb/rHdpB3xbA41CW++fBh5Es/fugJd2wJLXRuvU+/NpzpxxuEsLsOS1sRIeT1PvD8R6l5Zi\n",
       "yWvjtPX+euAimr33vxl4SdJoWfLaKJ16vz9NvTvgpV2w5LUREk6m2Xu/CPiVKi+ek1bBktfgjlLv\n",
       "DnhpRSx5Daat99cDz8N6l3phyWsQCU+gqfdTsd6l3ljyWivrXVovS15r06n3+2G9S2thyat3bb3/\n",
       "IfBcmnq/duAlSbNhyatXCU+k+Ues70tT7w54aY0sefWirffLgP1Y79JgLHmtXKfe74P1Lg3KktfK\n",
       "JNyLZu99P/DyKt438JKk2bPktRIJe2nOnLmr3h3w0gaw5LUrbb1fBlyI9S5tHEteS2vr/UbgXljv\n",
       "0kay5LWwTr1fQFPv7x94SZKOwZLXQhKeRFPvJwOPcMBLm82S14609X45Tb2/zOEujYMlr+Pq1PtJ\n",
       "wMMd8NJ4WPI6pk69P4em3j8w8JIkLciS11FtqfezHfDSOFnyupu23t8AnI/1Lo2eJa/vSngycBPw\n",
       "/Vjv0iRY8iLh3jR77+cDv1zFBwdekqQVseRnrq33G4Hvo6l3B7w0IZb8TLX1/gbg2Vjv0mRZ8jOU\n",
       "8BSaej+R5rx3B7w0UZb8jHTq/Vk09f6hgZckqWeW/Ey09X4TTb2f7YCX5sGSn7i23v8IeCbWuzQ7\n",
       "lvyEJTyVpt7vifUuzZIlP0EJ96HZe38m8NIqPjzwkiQNxJKfmLbebwROoKl3B7w0Y5b8RLT1/kfA\n",
       "M7DeJbUs+QlIeBrN3vs9sN4ldVjyI9ap93009X7dwEuStGEs+ZHaUu+PcMBLOhpLfmTaev9j4OlY\n",
       "75KOw5IfkYTzaOodmr13B7ykbVnyI9DW+xXAeVjvkhZgyW+4Tr3/H9a7pAVZ8hsq4b40e+/nAb9U\n",
       "xcGBlyRphHop+SQHktye5JPtn319HGeqEp7O3evdAS9pKX2VfAFXVtWVPX3/SWrr/QrgqcAvVvGR\n",
       "gZckaeT63JNPj997cjr1fifNee8OeEm71ueQf1WSTye5KskpPR5n1BLum/A24C9p6v3lVXx96HVJ\n",
       "moZU1XIPTA4Cpx3lU68BPg58pb39OuD0qnrJlscX8NrOXYeq6tBSixmphH3AW4EPAL9exTcGXpKk\n",
       "DZNkL7C3c9elVbXjnZKlh/yOD5CcCVxbVWdvub8WWeiUtHvvVwJPoTlzxq0ZSTuy6Ozs6+ya0zs3\n",
       "9/O9qzRnL+EZNM/H/9KcOeOAl9Sbvs6uuTzJI2nOsvkC8LKejjMaCafQnDnzFODFVXx04CVJmoHe\n",
       "t2uOeeAZbde09f5W4H3Ab7j3LmlZi85Or3jtUVvvVwJPAn7Bepe0br53TU86e+/fojnv3QEvae0s\n",
       "+RXbUu+XVPG3Ay9J0oxZ8iuU8EzuXu8OeEmDsuRXoK33P6G5YMF6l7QxLPld6tT7/9Cc9+6Al7Qx\n",
       "LPklWe+SxsCSX0LCs7DeJY2AJb+AhPvR1PsTgJ+v4u8GXpIkbcuS36FOvX+T5swZB7ykjWfJH8eW\n",
       "en+Rw13SmFjy27DeJY2dJX8Ubb2/EXgc8HNVHBp2RZK0HEt+i4Rn09T714Efd8BLGjNLvtWp98dj\n",
       "vUuaCEseSDif79X7IxzwkqZi1iXf1vufAucCP1vF3w+8JElaqdmWfFvv/wp8jabeHfCSJmd2JZ9w\n",
       "Kk29/zTwMw53SVM2q5Lv7L1/Fetd0gzMouQ79f5TWO+SZmTyJZ/wHJp6/2+a894d8JJmY7Ilv6Xe\n",
       "L67iYwMvSZLWbpIlf5R6d8BLmqVJlXxb728CHov1LknTKfmEC2jq/T+x3iUJmEDJJ9yfpt4fA7yw\n",
       "iusHXpIkbYxRl3yn3r9Cc967A16SOkZZ8p16Pwd4gcNdko5udCWfcCFNvX+ZZu/dAS9JxzCakm/r\n",
       "/c3AT2K9S9KOjKLkE/bT1PthrHdJ2rGNLvkt9f78Kv5h4CVJ0qhsbMl36v1LNPXugJekBW1cySc8\n",
       "gKbefwLrXZJ2ZaNKPuG5NPV+B/BIB7wk7c5GlPyWen9eFf848JIkaRIGL/lOvf8HTb074CVpRQYt\n",
       "+YR3AY/GepekXgxd8l/Eepek3qSqhjlwUlWVQQ4uSSO16OwcuuQlST1yyEvShDnkJWnCHPKSNGEO\n",
       "eUmasKWHfJKLkvxbku8kefSWz/1Wks8luTnJebtfpiRpGbsp+ZuA/cDHuncmOQt4AXAWsA/48yT+\n",
       "xNCzJHuHXsOU+Hyuls/ncJYevlV1c1XdcpRPXQC8s6q+XVW3Ap+n+bdY1a+9Qy9gYvYOvYCJ2Tv0\n",
       "Auaqj8L+QeD2zu3bgQf3cBxJ0nFs+941SQ4Cpx3lU79dVdcucJxhLquVpJnbdshX1dOW+J5fBM7o\n",
       "3P6h9r4jJHH4r1CSS4dew5T4fK6Wz+cwVvUulN33UbgG+OskV9Js0zwE+JetD/B9aySpf7s5hXJ/\n",
       "ktuAxwLvT/JBgKr6DPBu4DPAB4FX1FDvgiZJMzfYu1BKkvq39vPXvYiqP0kOJLk9ySfbP/uGXtPY\n",
       "JNnXvv4+l+Q3h17P2CW5NcmN7evxiG1bbS/J25McTnJT575TkxxMckuS65Kcst33GOIiJS+i6k8B\n",
       "V1bVo9o/Hxp6QWOS5ATgLTSvv7OAi5M8bNhVjV4Be9vXo9fLLO4dNK/HrlcDB6vqocBH29vHtPYh\n",
       "6kVUvfMX2ss7B/h8Vd1aVd8G3kXzutTu+JpcUlVdD3x1y93PAa5uP74auHC777FJpexFVKvxqiSf\n",
       "TnLV8X6M0xEeDNzWue1rcPcK+EiSG5K8dOjFTMSeqjrcfnwY2LPdF/fyD3l7EVV/tnluXwP8BfB7\n",
       "7e3XAVcAL1nT0qbA19vqnVtVdyR5IHAwyc1tnWoFqqqOd71RL0O+74uo5mynz22StwGL/A9VR74G\n",
       "z+DuP11qQVV1R/v3V5K8h2ZLzCG/O4eTnFZVX0pyOvDl7b546O2arRdRvTDJiUl+hGNcRKVja/+D\n",
       "32U/zS+5tXM3AA9JcmaSE2lOBLhm4DWNVpKTkty7/fhk4Dx8Ta7CNcAl7ceXAO/d7ot7KfntJNkP\n",
       "vAl4AM1FVJ+sqmdU1WeS3HUR1Z14EdUyLk/ySJpthy8ALxt4PaNSVXcmeSXwYeAE4Kqq+uzAyxqz\n",
       "PcB7kkAza/6qqq4bdknjkuSdwBOBB7QXn/4ucBnw7iQvAW4Fnr/t93COStJ0Db1dI0nqkUNekibM\n",
       "IS9JE+aQl6QJc8hL0oQ55CVpwhzykjRhDnlJmrD/B3ksmr9zVxpPAAAAAElFTkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108824850>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def gsoft (l, z):\n",
    "    return np.sign(z)*np.maximum(0,np.abs(z)-l) \n",
    "#assuming (x)_+ means max(0,x), the only definition (of many) I found that made sense...\n",
    "\n",
    "vg = np.vectorize(gsoft)\n",
    "\n",
    "x = np.linspace(-10, 10, 1000)    \n",
    "y = vg(1, x)\n",
    "\n",
    "plt.plot(x, y, '-')\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Application of LASSO on a gene data set (70 points)\n",
    "\n",
    "(This exercise is independent of the first one.)\n",
    "\n",
    "The goal of this exercise is to apply both ridge regression and LASSO on real data.\n",
    "The data can be found on the homepage of the course, files _xtrain_xxxx.csv_ and _xtest_xxxx.csv_ , and comes from the paper <cite data-cite=\"Kemmeren2014\"> Kemmeren et al.</cite>.\n",
    "It consists of the logarithm of the gene expression levels of yeast cells.\n",
    "\n",
    "More specifically, we consider two datasets corresponding to two different targets: say gene 4710 for the first and gene 3290 for the second.\n",
    "In both cases, we would like to predict the gene expression levels of the target gene, given the gene expression levels of the other 6170 genes.\n",
    "For each dataset, we are given a training set $X_\\mathrm{train} \\in \\real^{140 \\, \\times \\, 6170}$, $y_\\mathrm{train} \\in \\real^{140}$ consisting of 140 yeast cells, and a test set $X_\\mathrm{test} \\in \\real^{20 \\, \\times \\, 6170}$ with 20 cells. To evaluate the performance of our model, we also provide the target values $y_\\mathrm{test} \\in \\real^{20}$ of the test set.\n",
    "\n",
    "You are asked, to hand in:\n",
    "\n",
    "1. A printout of the code of the function _printOutput_, that you will have appropriately modified (see section [_The printOutput function_](#The-printOutput-function)).\n",
    "2. Answer questions a. and b. of the section [_Linear regression with no regularizer_](#Linear-regression-with-no-regularizer).\n",
    "3. A printout of all the lines marked _### CHANGE THIS LINE ###_. On the printout, all regressions should be done using a regularizing parameter $\\lambda$ that you will have optimized using 10-fold cross-validation on the training set. However, you do not need to hand in the printout of the lines corresponding to these cross-validations.\n",
    "4. A printout of the output of all lines marked _### PROVIDE THE OUTPUT ###_.\n",
    "5. The same printouts as for questions 3. and 4., but with target gene 3290 instead of 4710 (i.e. with the second dataset).\n",
    "\n",
    "_Hint_: For all the regression tasks, we recommend to use python functions of the type _linear_model.xxx(xxx)_, of the pacakge _sklearn_. The code asked to hand in should look pretty similar to the code of the section [_Linear regression with no regularizer_](#Linear-regression-with-no-regularizer). Cross-validation can be easily done using ipython functions of the type _linear_model.xxxCV(xxx)_. Do not hesitate to consult the documentation of these functions. Finally, note that, what we call $\\lambda$ corresponds to the parameter _alpha_ of the python functions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##Loading the required packages."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Target: gene 4710"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "### CHANGE THE PATHS ###\n",
    "a=4\n",
    "if a==4:\n",
    "    Xtrain = read_csv(\"./GeneDataSet/Xtrain_4710.csv\", header = False)\n",
    "    Xtest = read_csv(\"./GeneDataSet/Xtest_4710.csv\", header = False)\n",
    "    ytrain = read_csv(\"./GeneDataSet/ytrain_4710.csv\", header = False, names = [\"gene\",\"y\"])\n",
    "    ytest = read_csv(\"./GeneDataSet/ytest_4710.csv\", header = False, names = [\"gene\",\"y\"])\n",
    "else :\n",
    "    Xtrain = read_csv(\"./GeneDataSet/Xtrain_3290.csv\", header = False)\n",
    "    Xtest = read_csv(\"./GeneDataSet/Xtest_3290.csv\", header = False)\n",
    "    ytrain = read_csv(\"./GeneDataSet/ytrain_3290.csv\", header = False, names = [\"gene\",\"y\"])\n",
    "    ytest = read_csv(\"./GeneDataSet/ytest_3290.csv\", header = False, names = [\"gene\",\"y\"])\n",
    "\n",
    "Xtrain = Xtrain.drop(Xtrain.columns[[0]],axis = 1)\n",
    "ytrain = squeeze(ytrain.drop(ytrain.columns[[0]],axis = 1))\n",
    "Xtest = Xtest.drop(Xtest.columns[[0]],axis = 1)\n",
    "ytest = squeeze(ytest.drop(ytest.columns[[0]],axis = 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The printOutput function\n",
    "\n",
    "The purpose of this function is to plot the regression weights, the predicted values yhat of y against ytest, the (number of the) gene with the strongest coefficient and the so-called _coefficient of determination_ $R^2$, defined as:\n",
    "\\begin{equation}\n",
    "    R^2 := 1 - \\frac{\\sum_{(x_i,y_i) \\, \\in \\mathrm{\\, test \\ set}} (\\hat{y}(x_i) - y_i)^2}{\\sum_i (y_i - \\overline{y})^2},\n",
    "\\end{equation}\n",
    "where $\\overline{y}$ is the emperical mean of the $y_i$'s in the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def printOutput(lm_, Xtest_, ytest_):   # lm_ = instance of linear_model.xxx(xxx)\n",
    "    yhat = lm_.predict(Xtest_)  ### CHANGE THIS LINE ###\n",
    "    plt.figure(1)\n",
    "    plt.title(\"Regression Weights\")\n",
    "    plt.plot(lm_.coef_.T)\n",
    "    plt.figure(2)\n",
    "    plt.title('yhat vs ytest')\n",
    "    plt.plot(ytest_, yhat, 'ro')\n",
    "    plt.show()\n",
    "    print('R2 :', lm_.score(Xtest_,ytest_)) ### CHANGE THIS LINE ### \n",
    "    print('Gene with Strongest Coefficient :' , lm_.coef_.argmax()) ### CHANGE THIS LINE ###\n",
    "    if hasattr(lm_, 'alpha'): print('Used Lambda :', lm.alpha) # if using linear_model.xxx\n",
    "    if hasattr(lm_, 'alpha_'): print('Lambda_ResultOfCV :', lm.alpha_ )# if using linear_model.xxxCV\n",
    "    #print('Regression Coefs :', lm_.coef_) # Can be printed out, if wanted\n",
    "    #print('Regression Intercept :', lm_.intercept_) # Can be printed out, if wanted\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linear regression with no regularizer\n",
    "\n",
    "a. What should happen if you tried to apply linear regression without any regularizer on this data set?  \n",
    "    - many non-zero weights\n",
    "    - possible large weights\n",
    "b. Does this happen with the function _linear_model.LinearReagression()_ ?\n",
    "    - yes (only 1 zero for 4710, 3290)\n",
    "    - no (largest weight for 4710 ~0.0227, for 3290 ~0.0477)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEKCAYAAAA8QgPpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJztnXnYJFV1/z9fB0YFQRwxDMsoKIMCGjYFjCAjERhQQaIE\n",
       "cCNiIi4kZlNwixMxAYkRQ1QkCShiIqhRMirbqIzgBvITEGTGYWQfZEBBXJD9/P6o2/NW11tVXVt3\n",
       "dXWfz/P001V3PbXdc5dz75WZ4TiO4zhleVzbAjiO4zjdxBWI4ziOUwlXII7jOE4lXIE4juM4lXAF\n",
       "4jiO41TCFYjjOI5TCVcgjlMQSXtLWtm2HHlIOk3S+wqG/YykE4YtkzO5uAJxRoqkmyXdL+k3ku6U\n",
       "dLakjduWqwhmdpmZPafpdCVdJOldsfMtJT2W4fYHA2R8q5l9qGDWFn5Zcj0m6ZkF03KmEFcgzqgx\n",
       "4OVmthGwE/A8oFCNuQyS1ms6zSHybeDFsfMXAytT3FaZ2V2jFAzQiPNzOoQrEKc1zGwtcDGwY89N\n",
       "0p6SvifpXklXS9on5reNpEsl/VrSMkmfkHR28Ns61JiPlnQL8I3gfrSk6yXdI+lCSU+PpXeKpLWS\n",
       "7pP0Y0k7BveDJP0k5HO7pL8L7osk3RaLv72k5UHW6yS9Iub3mSDf10I6P8ipzV8GvCh2vhfwMeD5\n",
       "Mbe9gUtD2s8J1/9LSSslHZbI94TY+bsk3RGu489TWhXz0mSUdGnwvya0Fg+TtGkIe2/I+1JJrmCm\n",
       "GTPzn/9G9gNuAv44HG8F/Bj4h3C+JfALYHE4f2k4f2o4/z5wMrAeUYF7H/DZ4Lc18BjwGeCJwBOA\n",
       "Q4AbgGcTVZbeC3w3hD8AuBLYOJw/G5gfjn8OvCgcPxnYJRwvAm4Lx+sDq4HjgzwvAX4NbBf8PxNk\n",
       "fz4wB/gc8PmMe/J44H5gp3B+LbAN8B1g5+B2HfA6YEPgNuCocE07A3cD24dwnwY+GI4Xh2vZPtyT\n",
       "z4V79MwiMsbDhvMTgdNC2Dm9e+S/6f15C8QZNQLOk/Rr4FbgZ0Cvz/51wPlmdiGAmX2DqJB/WWg5\n",
       "PJ9I2TxiZt8FljK7i2WJmf3ezB4A3gKcaGY/NbPHiArAnUNaDwEbAdtLelwIc2dI4yFgR0kbm9l9\n",
       "ZnZVynXsCWxoZicFeS4BvgYcGQvzZTO70sweBf6bqLCfhZk9CFwO7CNpHvBkM7uJqGXy4uC2PVFX\n",
       "18uBm8zsLDN7zMyuBr4MHJaS9J8CZ5rZCjP7PfCBZNZFZYzdl82Brc3s0fAMnCnGFYgzagw4xMw2\n",
       "JqrR78tMV80zgMNCF8m9ku4lamnMB7YA7gmKocdtzCbu9gzg32Jp/TK4bxEK/I8DnwDWSjpd0kbB\n",
       "/1XAQcDNoYtqz5R8tkjJ/5bg3rvOtTG/3wNPSkmnx6VE4xx7Ab2C+Tsxt9vM7LZwTXsk7tFrgM1S\n",
       "0tw8IePtKWHKyPgvRK2uiyX9TNJxOWGdKcAViNMaZnYp8O/Ah4PTrcDZZvaU2G8jMzuZqCtmnqQn\n",
       "xpJ4OrOJWxXdCrw5kd6GZvaDkP+/m9nzgR2A7YB3BvcrzeyVwNOA84AvpORzB7AgMQbwDGBNubuw\n",
       "jkuJxjleHI4BvkekQF9M1ProXdO3U+7R21PS/DmwIHa+ICVMYczst2b292b2LOBg4G8l7VsnTafb\n",
       "uAJx2uZjwO6S9iDqg3+FpP0lzZH0hDBwvaWZ3ULUnbVE0vqSXkjUnZO3H8GngPdI2gFA0pN7A86S\n",
       "ni9pD0nrE40/PAA8GtJ+raQnh26d3wCPpqR9eYj3rhBnUZDnnOBfdnD5+8BTiLrxLgMws3uJxihe\n",
       "x4xS+RqwnaTXhXzXl/QCST3zYsXy/gLwxjDovgHw/kSeg2RcCzxrXWDpZZK2DUrz10T3Je3eOFOC\n",
       "KxCnVczsF8BZwHFmdjvRwPd7gLuIatt/x8x7+lrghURdUScA5xL1y69LLpH2eUStm3Mk3Uc0OH1A\n",
       "8N4Y+A/gHuBmooL6X4Lf64CbQpw3h3z78jCzh4BXAAcSDWJ/HHi9ma2KhUsqt0xlZ2b3EynI9c3s\n",
       "upjXpUQtoUtDuN8C+wNHELV2fk40tjM3mW8YSzoVuARYRaSkAB4sKOMS4KzQVXYYsBBYRqRUvwd8\n",
       "wsy+jTO1yKzehlKSFhPVIucA/2VmH04JcyrRh3Y/8GdmdpWkBcBngT8gemn/w8xODeHnERUOzyD6\n",
       "uP/UzH5VS1Bn4pB0LnC9mf1j27J0AUnbEynRucGowHFqUasFImkOUc1rMVE/8pHhJY2HOQjY1swW\n",
       "EtXmTgteDwN/Y2Y7Elm0vD3WDD8eWGZm2wHfDOfOlBO6nZ4l6XGSDiTqhz+vbbnGGUmHSnq8pKcQ\n",
       "tcaWuvJwmqJuF9buwGozu9nMHibq/z0kEeZgoi4KzOxyYBNJm5nZncEEsdcsX0E0D6AvTvh/ZU05\n",
       "nclgPlF3zG+AU4C3mNk17Yo09ryZaCxjNVGl7a3tiuNMEnWXe9iS2WaCexQIsxUx80FJWwO7EA1M\n",
       "Amxm0SxlQrg0E0VnyjCzrxENIjsFMbMD25bBmVzqtkCKDqAkrT3WxZP0JOBLwDtCS6Q/YDRIU2+g\n",
       "xnEcx2mcui2QNcy2M09OVkqG2Sq4EUwo/xf4XLCY6bFW0nwzu1PS5kQWOX1IcqXiOI5TATNrZg2z\n",
       "OuugECmgnxGtQzQXuJqwJk8szEFEy1NANFj+g3AsIiusU1LSPZnIrBOiAfSTUsJYHdnb/hEtudG6\n",
       "HC5/N38uv8teQ35rKq1aXVhm9ghwLHARcD1wrpmtkHSMpGNCmPOBGyWtBk4H3haiv4jI3v4lkq4K\n",
       "v8XB7yRgP0mriJa6OKmOnAAS29VNw3Ecx5mh9p4JZnYBcEHC7fTE+bEp8b5DxhiMmd1DtBJrI0g8\n",
       "E/gpvreB4zhOY0zLTPTHty1ACsvbFqAmy9sWoCbL2xagJsvbFqAmy9sWoAbL2xZgXKg9E70tJJkV\n",
       "HAiS2B643sxbII7jTDdlys5BTEsLxHEcx2kYVyCO4zhOJVyBOI7jOJVwBeI4juNUwhWI4ziOU4lO\n",
       "KxCp/jwWx3EcpxqdViD4xEDHcZzW6LoCcRzHmYXEFm3LMA24AnEcZxJZI7Fh20JMOq5AHMeZVLx8\n",
       "GzJ+gx3HcZxKuAJxHMdxKjHxCkRCwAZty+E4jjNpTLwCAQ4DrmxbCMdxnEljGhTIM9oWwHEcZxLp\n",
       "ugLxiYSO4zgt0XUF4jiO47RE1xVIN7dTdAYi8ZK2ZXA6z8jLB4ktp2kWvC9G6Iwr35J4ghkPti2I\n",
       "45RgBfAA8AdtCzIKXIE4juM0x0bAE9sWYlR0vQurs0icJ/FHbcvhOBOMG9kMmdoKRNJiSSsl3SDp\n",
       "uIwwpwb/ayTtEnM/U9JaSdcmwi+RdLukq8JvcV05x5BDgEPbFsJxHKcqtRSIpDnAx4HFwA7AkZK2\n",
       "T4Q5CNjWzBYCbwZOi3l/OsRNYsBHzWyX8LuwjpyO4zhO89RtgewOrDazm83sYeAcopp1nIOBswDM\n",
       "7HJgE0nzw/llwL0ZaXvz03GcLjI1ZVddBbIlcFvs/PbgVjZMGn8ZurzOkLRJPTEdx3GcpqlrhVXU\n",
       "zjqpkQfFOw34YDg+AfhX4E2zg839B+nhR8PJcjNbXlCecWFqaiqOM0XMkZhjxqODgw4fSYuARcNI\n",
       "u64CWQMsiJ0vIGph5IXZKrhlYmZ39Y4l/Rfw1fSQD51gxkPFxXU6hitYpw5tTjQem3c3VKyX984l\n",
       "faCptOt2YV0JLJS0taS5wOHA0kSYpcAbACTtCfzKzNbmJSpp89jpocC1WWEdx3GcdqjVAjGzRyQd\n",
       "C1wEzAHOMLMVko4J/qeb2fmSDpK0Gvgd8MZefEmfB/YBnirpNuAfzOzTwIcl7UxUg7gJOKaOmDXi\n",
       "Ou3iz85xxpjaM9HN7ALggoTb6YnzYzPiHpnh/oai2RcIMzZNScdxnEnCZ6I7Y4fEp9uWwXGcwbgC\n",
       "ccaRPwv/3np0nDHGFYjjOJOKV0CGjCsQZ5zxQXTHGWNcgbSL15AcZ3jUroBIPLcJQSYVVyDOOFNJ\n",
       "wUp8VeL9TQvjTCXXVtxhcCpaz11XIEUKmKl4kE4fLweOAJCQhK+l5tRhTtsCjCtdVyBFGOduIldu\n",
       "w+f1ZK/47DhODaZBgYwz46zcJoUiKz87jlMBVyCO4zhOJVyBdBCJDSTOaVuOEeAttClE4n0SX2xb\n",
       "DmcwrkC6ybOIVj52nEnkKODVbQsRwysyGbgCaRd/MR3H6SzToEDc0slxppOmvn0vQzKYBgUyibX8\n",
       "aXmhJ/HZOaOjzfdnKr7RaVAg48xUvGSO40wmXVcgXkN1BuFKunv4d90Ruq5Auo5/KI4z/vh3moEr\n",
       "EGecaeLD9Y+/e3irsSO4Aukm/oE5jtM606BAvLCdbvz5O86QmAYFMs5494rjDA+fBzJkpkGBeCHt\n",
       "OI4zBGorEEmLJa2UdIOk4zLCnBr8r5G0S8z9TElrJV2bCD9P0jJJqyRdLMk3BJpOXPk7XWUqWi21\n",
       "FIikOcDHgcXADsCRkrZPhDkI2NbMFgJvBk6LeX86xE1yPLDMzLYDvhnOJ5GqL9lUvJyOMyZ4RSaD\n",
       "ui2Q3YHVZnazmT0MnAMckghzMHAWgJldDmwiaX44v4z03eLWxQn/r6wpp+M4jtMwdRXIlsBtsfPb\n",
       "mb0DXJEwSTYzs7XheC2wWUa4rtcMqsrvLZDB+D1yul4+jD3r1Yxf9CNNPsjCH7eZmaSM8E94n/Tg\n",
       "I+FkuZktT0uiaF4dYlo+jGm5TscZGpIWAYuGkXZdBbIGWBA7X0DUwsgLs1Vwy2OtpPlmdqekzYG7\n",
       "0oM98CEzHiglsTNtTGIFwhktnX6HQsV6ee9c0geaSrtuF9aVwEJJW0uaS7RL3tJEmKXAGwAk7Qn8\n",
       "KtY9lcVSol3JCP/n1ZRz0uj0Cz1ivBXTPfyZdYRaCsTMHgGOBS4CrgfONbMVko6RdEwIcz5wo6TV\n",
       "wOnA23rxJX0e+B6wnaTbJL0xeJ0E7CdpFbBvOE8ja2wkzji/jOMsm+N0naYqWv6dZlC3CwszuwC4\n",
       "IOF2euL82Iy4R2a43wO8tED2ewK3FJPU6SD+4U4n3sLuCNMwE32c8Q9l+Pg9dkaO2XS8d11XIF5D\n",
       "dRzHaYmuK5BpZSpqNw3hlQynMBLvlDimbTm6wjQokL7CVmJnif9pS5gEXrg5zmza/C5ODj+nAF1X\n",
       "IFVetFcBqYP304TE+RLfaFuOATRRkFRurUkcKPFXDcjgdAuv2BWkthWW0wpNdGEdQPcrEMPmZOC5\n",
       "wKltCzJl1Hq/JVcAo2IaChB/mdLx+zIYv0fdxp/fkOm6AvEXZAqQOE9iz7blcBynn64rEKc6XVC+\n",
       "PRkPAQ5tMX9nuvDnXhBXIAWRuFJibttyOO0j8YK2ZZhw2i7AS+UvMVcqtHLGxDENCqSpORO7ARs1\n",
       "lFZdmrgmn0tSAYk5wBVty+GMFYcDy9oWog2mQYGMM23WtNqu5XUBv0dOEaa2HJ3aC6/IWBUoEk+S\n",
       "OLFtOcaczra0JP5P4otty9ECXXtmY1UujJKuK5AqD65rL2ceuwPHty3EEGniw6yTRtsFw8HAK1uW\n",
       "YRpp+7l3hmlQIMkwk/RyTNK1DGKartUJSLyuQiu78XdFYgOJbUeVX1fougJpHYkNJU4YcbaT1Ipy\n",
       "OojEjhJbDSv52PHZjEcr+yTghraFGDe6rkCqFKRNF77PB97XcJpF2bSlfLtEnec9tTXLAlwHXNi2\n",
       "ECPkyTl+U/uedF2BFHlwk1xbPyfNUUITsh5Q/Bom6jlKE7EO3fptCzAk0rq9J+r9a4quKxAkdhjG\n",
       "xyhhEosy/BaF+QAwni/W5cBX2haiYdpQiEPJU2I+8HCbMow54/hNOSl0XoEAPwGOGlLaz0uc9z7m\n",
       "S4D9h5RnE7wA2KdtIUaJxJMlnpXmNXJhBpPXHeI0R1PP3hVaBpOgQAA2aCHPUvcu1mLpc66Yt7/Q\n",
       "s/lPYHWdBCT2ljh2ULAQdlSKyZ/16Cn7bMexkjISJkWBjPUDlNgReGTU2Y44v2FQ5hqyavVlCuB/\n",
       "Bv49mb/EbiXSKEKZ65qE59iHxLESD5SMs57EHw5LpgK4Ik9hUhRIHskPcNaqrhLfl9i8dMJRLbSI\n",
       "JdT8smkPoKmX2cJYT+Wl0sOHvUVD8gyDD9WI23t3PtiEIBNI1fdwD+DxOf5pSvPPgGsKpl9b6Uoc\n",
       "LBXe2nbilHxRaisQSYslrZR0g6TjMsKcGvyvkbTLoLiSlki6XdJV4bc4I/t/riDyc1Pc9gR2rpDW\n",
       "kcD/VoiXicRrpIEfZlPWZ710FhYIm8XfAGvKRJDYrkZ+WWRdb9cthca55jvKgvOJI8wL4O+Ad444\n",
       "z85RS4FImgN8HFgM7AAcKWn7RJiDgG3NbCHwZuC0AnEN+KiZ7RJ+WfbmC2bLxKUSfx1zGuYHuFmZ\n",
       "wCkTr9Jk2yXFrQqj+rj/oEKcn3ZsSXRf9HK6mOTVKxqlbgtkd2C1md1sZg8TzUs4JBHmYOAsADO7\n",
       "HNhE0vwCcas+tL2BV1eMOwhlHBfltpz0ytC0UmyjlvuEFvJsgrJ7RTxFyu2u6TLDem/GsdU1jjK1\n",
       "Tl0FsiX9heLtwa1ImC0GxP3L0OV1hqRNSso1V+KPS8ZpHIktMqyvssLvRcu1HYkF0tjMcC9zL4bx\n",
       "geflX1S2e4D/qCGDhYmh4zheWetdlfg7iY2bEqZhLOM4jaltodR9KYt+tGVv8GnANkTjEj8H/jU9\n",
       "2JLwO/hASYtiHlsB35glhHhS7PhVJWWqwhrgIyXCX0axsZhh1oZuZTSb45S5L1Vo4h41VTDsJPGi\n",
       "GjKcANwvcYHE00tFFp+TeFfFvIdF79l8BLivYJxRKhoxQUpB0qIwrrxE0pIm066rQNbQPw6xgKgl\n",
       "kRdmqxAmM66Z3WUB4L+IurtSWBJ+Sy80s+Uxj6wug9/Ejr+UESaPtVkeUp/5Z5y/JlGYDRgknxML\n",
       "97iMmmfZbpQ7JLYpEWXnEt0uVQvqWc9U4mUS1ybcPtU7bEqOsI9KZhfaEOZ47AR8p0b83Yje6cVQ\n",
       "TBFJXCjxVOC1wJtq5J1HIxUZic9I7Bt3SglWx5quCfKudayVjZktN7MlvV+TaddVIFcCCyVtLWku\n",
       "0daOSxNhlgJvAJC0J/ArM1ubF1dS3KT2UOgvVApQuNsoRt2PIW8CWtYL9pcD0jwX+HEZIaTUGfKb\n",
       "k259lkdTg/ll2J/Zch5TMa2tYsonyQ3A+Tlxx8lyy6hWQB0A6+ZNjHv//VHUXE1C4mKJzSU+C3y6\n",
       "QPj1Jf4my7uOLNNErTWkzOwRSccCFxEV2meY2QpJxwT/083sfEkHSVoN/A54Y17ckPSHJe1M9OLf\n",
       "RPVCJBeJp5tx6zDSboi9Sbf0Si0QJN4ApJpSZ8VpGolNzfhFU8nFjgvJL627/k2IvTcSc8x4NJzO\n",
       "J7+QeDDhX9d4og6p+QXleJ9Z5vOGmXvWigKReDVwmVl2yz1GXRn3I1oZ+/UFwy8EPgqcUkCedc8g\n",
       "dINvasbNaf7TRu1FCM3sAuCChNvpifPU2nla3OD+hrpiFQx3IlETvzAVBzPLfBzxsE+L5SvgiWbc\n",
       "nyHXHIK1WwUspBE3PKj6UdwtsZsZP6oYP4verPCnAbuacVFGuL/NcN8JBsqUZb6ZtlTO0HY6DObe\n",
       "PxyQxjFEFbI8BdIYoeCcV7LC9UWicY4i8ykeix333sdLSuS1Ll6MYTyj3wzwnyrG0bKjCSo/XInn\n",
       "DBiofFrseBhGBPGw8efzeqICIyvfQbPJ82Q9WmI/4OWDxZtB4mjgj8LxNRIvCV5bSSwvmMaWJQeG\n",
       "l5C+D0WtGmwY5E5OVus9i0zDhrChWK2KmMSzJDaMOW1P/+oFdVtDTbRAPgnc0kA6WaTJuKhkGmW6\n",
       "HsuWEePeDdgKk6JA8ib+/FHJtFYA3+5LLFrLqki+WTSxMm6ZQXBIkU1iI4mfSeyU8NqXaFLnoPjv\n",
       "kPpajGcwM6j7h7CuBbOQxDWHQvLIFDmvAlZlXsUMm0qFJi1mfujBrLqnJJLX9x1g11hYAVtnhI3z\n",
       "W6qtiBBnNfAvsfPChZvEzhLLJd4upbdOmanRnyBxRyL+TgVXBihl2i3xnHjew0DiAYnXxJy+XCLP\n",
       "QSba8TQGjqlMK5OiQJLEH/6sta8KMK93IDGXaPe1OmxUImzWy193WQUjmnvzTJgxZ47xWIpbkiOI\n",
       "LIGy6Es3sQjhB4H/SYmzMTNWc3kf9Z8R1YDflvSQ2JrBqwK8l8jy74oB4XrEJ7UOen6FloIZYN31\n",
       "1njQZNSsJIkGy/chaoEmW1DJMZC9YNaab1cD302R1ST2yJF3ENsPDtLHYyHfF1C8a/3xkGkoASkr\n",
       "VcQo0wJ5CR22whomk6pAmnygyXv0rdhx0drVunBhsiCx8z0knh1zegnpbJjh3lNySf+0/uCeW5qy\n",
       "aKKm+I5EWk9LBpBmFSxlPsws09ubYOCKuX8S/otao2Xd7zTluw6JF+VMHp2X4V6HvIHy0u9ngiZW\n",
       "v02asK+XkV/P7QpmWn5FSFPuvbQGVvwk3lIir3i862Onp1VJYxKYVAVSdNKRCixcmBzQ3yEev2A+\n",
       "8TwuS/j9AFhZMJ0szoJZg8p5sqVdc1KpSOKZFeXppf+4KCG+Auu6Gq5PjRHxwor5VWFQLTfrvehd\n",
       "x6skvpbi/x1mL+fTY1brqQJJ66Aic4oGvePrxg4kbpHYOyVM3BKpF35WuqGbLL568X6JIA8TWUs1\n",
       "hpRpSZUW9gXhm+9dz2kSb5RKL6M0q4VVoCyZOCZVgRQlfv3xGmxPAb0D6LMoq0iWZVBTZI3RxDFm\n",
       "PvjvD/Dv8TOJL8QKjHUD9VLuNX00/Pdq8a9MCyTxS/pbFU0ULEU/4kGtgaxCpJf+kcDLem6K1rw6\n",
       "KJxntUAGWV/tFaz88tYJy0ojzXy1t8x+shVwYKKw2yS4bwA8HfpbySn5PhT+t00J9z7g/bHztPlE\n",
       "aVsnFHmHs/jrLI8UA4feeE/8es5k8HIzWabzafdqaphoBVKgRnB47DhtpmvaFqk91md8LDOy5grs\n",
       "I3F3wTSyTCAPI9GlEKymMpaX6eMLA/xrdelIfKngwHpW/P0lnlex5phUivdA/6x5ada6cGkyJFun\n",
       "/wCclwyWFZ38d7DXopsXMx6A2Vs1I7EtM1Z+RSk0VtHrotXMGmtpMpcydglWg1nE00/Owu/NBRo0\n",
       "zlR0y4Fkj8JUMSkKRIo2NqprTllmvZ0vDw7SPBKXMvtlz+rXfyH0fbSDujvSui6ibNU3NjNr7k4W\n",
       "Ui0lkVljl3gy8CpmLy9TZvOuiyCzm66MUukNNvd2RVQYME/Kovi/xAkpaX2gRL6zlpuRiC88+vbw\n",
       "vyVwPzNmsTuFsPFrjG8Kljb4nPcs0gbM4+tvrQzvwctSwlXl4hy/eAswOciepUDi43VzyBlzdGaY\n",
       "FAUCUd/qwzXTKLqwWy4Sj5dm1/IaIquQr8sO9HchJecbfCsRtii/LBJIKjehk9FvMNQjrSBN7sh4\n",
       "Dummvcm476skQL81V1LRnSQNNBRImzMUV0Y9i7B42vuHvNOs3a7X7B09k3lsPUCmtO6mrHClWoyJ\n",
       "dB/NCXdw7DRZmRyX3oaxYpIUSBvMGrwLpo/HUnINq5IU2X53Y/pf+r8B5g5HnEb4XNsCNMzxOX5l\n",
       "CqM59I/P/TfRAqNZPJ4Za7gyFF0d+c74Sayi9KcD4r0G1lkbZr2HdSuAWcTTzWqBAPzfkPKfWCZF\n",
       "gXx0cJCR8QOYtfNg0xRd2TU+g3o/iq8TBP2FXF0rsWEwUtt7KXX2+6xgifOsbpCysj8lcX507Dg5\n",
       "B6XIQqJp+aea7Eo8OwyuZ9Gz/vtYgTzfHY5zTaGHTM/asOwzeEXTgkwCk6JAxgJFy2fD+NzXIxLn\n",
       "ZVbkHYfJUW3KkGwlHJDhnse3E+dlxjeKklxotOn++5XkL6W+rjU8oNv2pY1JVI9eC+TKkvFmtfql\n",
       "WV2XU8e4FHSTQu9DG4fCN41Sa12NOU3d46TFU3aGkcVXcsmXOANXHAj99wOts2qwAaQuGVOHIl2m\n",
       "AF/P8WtiUmItFK3d1uRYxpoG0+oktVfjdZpByl4iRGpkAloXyZsl3MaWxSsGBynEYQ2lk0aRjcCK\n",
       "rqvWK2yTLdks8pYOGQfOaFuAScNbIM3S++CqWAjlbWVb9ANukiqbcjVNXpfPZ4ecd9oufk0tRTJM\n",
       "BdIkr2tbgCRSqXXlnCHjCqRZegrkzyvEPTF+kliIcFimu3kMWpxw0hlmC2eTwUEq02T36aIG02qK\n",
       "AwYHmU3O+mRODRRtO949JNkYmmZfScPr/LTIA+QvqeE4XeJT5HeJNorZ2I6DIsnMrBH5vAXSLJOi\n",
       "PMCVhzNZDGti71TjLRDHcZyG8RaI4ziO4+TgCsRxHMephCsQx3EcpxKuQBzHcZxK1FYgkhZLWinp\n",
       "BknHZYQ5NfhfI2mXQXElzZO0TNIqSRdLGqbdvOM4jlOBWgpE0hyitYEWE+0RcaSk7RNhDgK2NbOF\n",
       "wJsJG9APiHs8sMzMtgO+Sf7S2I7jOE4L1G2B7A6sNrObzexhoo10DkmEORg4C8DMLgc2kTR/QNx1\n",
       "ccJ/6p7ajuM4TnvUVSBbArfFzm9n9kqjWWG2yIm7mZn1tipdiy+r4TiOM3bUXY236Ey+IpNWlJae\n",
       "mVk0aTCNJbHjRYzn0j2O4zjtIWkRQyoc6yqQNfQv4byAqCWRF2arEGb9FPfe+vprJc03szslbQ7c\n",
       "lZ79kqpyO47jTAVmthxY3juX1NjGZnW7sK4EFkraWtJc4HBgaSLMUuANAJL2BH4Vuqfy4i4FjgrH\n",
       "R1Fi0x/HcRxnNNRqgZjZI5KOJdoXeQ5whpmtkHRM8D/dzM6XdJCk1cDvgDfmxQ1JnwR8QdKbgJuB\n",
       "P60jp+M4jtM8vpii4zhOw/hiio7jOI6TgysQx3EcpxKuQBzHcZxKuAJxHMdxKjHtCuSgtgVoiMva\n",
       "FqAFVrYtgONMO9OuQJxm+ELBcPc0mOcvGkzLcZwKuAKZDNq2Z24j/89UjNdruXykITmcfn7TUDqb\n",
       "NpSOM0RcgdSj6jLzn2hUimZZOzgIyX1fHgn/jzYsSx6PDfDPaqEsC//3NyhLXcb5fegSV9SI+/TG\n",
       "pJgipl3IlIXyAAAXaElEQVSBpE2m+eMS8U9tMN86ZLUABhWyaTyrQJjk2mRvIdrrZf6AeE23VPbP\n",
       "cL8GeNmIZSnK2zLcbxypFLO5quX8ob9VWeUb+V6G+62DIpr1rQzuFGTaFUgdngk8XDHumsFBSpH6\n",
       "8psxJyfOjzLcB9XMkx/pZmb81oz/NBs4LlG0ULilYLgsLiggQ5YC+VTNvNN4Tu/ALNpQLcGjVO+S\n",
       "a4IjaW611qz7uiLDPc51wA0Dwnwtx+89Ge6jbBlPFa5AZlOoZmrGTUXDpnBljl+RLqQkp1eUYxZm\n",
       "mdfUU3oipgjMslZKrsWrC4bLkjV1a4CYXx5NDvQDYMZPBwR5ZIB/knsLhEkWxHkVg++Z8euSMpTl\n",
       "zAJhvsTMvSjdAjHj98DJMaefhP86CmRQZabt8cdWcQXSzz9S7oVo4uXpfdh/Hf7zup0+wEyr52XA\n",
       "AcAmZnwnJezTBuRb9gPtXevvgEtLxi1L1n1Ndi8mr+HCAmkr8R/n2Rnuw6bsatM9GQ9I8ettC/18\n",
       "4M9j7kZGy8xscBcPsMEA/94S4Tdn+N8BrArHqd14ZtzCTKGf5l90X6Ee14X/Kl25Rfn+ENMeeyZZ\n",
       "gewE3FQyzgmkz6k4MCN8bQVixobhsGcdlFdb+l/glBDvfDMuNuO+lHCHNdSdFN+vpXetR5nxswJx\n",
       "0ziTYoVl2jXBjMzvB/4nxT/tfV5Df6tORGbHX08GNGMV6fdlULdKLcwancfzSEjz18DdCb86q1rn\n",
       "tZJ2JXo3AU4M/39BtNL2LEI3Xryl99KUYFktweUpblmF+IPhv7H9LxJsx3CV09gzKQpkVvPYjB+T\n",
       "XtPZiISSiNVszIxHgYvD+WPBP7Vmm9Pdk8bAQcogR0+BnJIR7L3AJinu/xZL50sl5ILsLYPjBVDv\n",
       "Wit38ZjxTuC1BYLmDWhuZsaHzNYVDnF67/MvgF+G4+cCL4qFkRmHM7tP/pKcPPOec1rrb0ns+C2x\n",
       "431z0rkmxy9JXisqTt87Z8Zvgb9PhCnUZWpWeLyv9838F/2byyVl3XFAfqkVKTP+NcX5oTxZyFBk\n",
       "NTgxyJKsWEzdQPykKJCsD3zWSx8+ol4NN/5SfyAlna+RPyBbhqXA21PyTdKT4V0pfj8z45G0VofZ\n",
       "ui6woojB1ilxOZvq6y3aRfTK2HGvILABYy4i2tnyY2bcCKxvxq/on5ugKCF+m4h7eEn5epyV4tZL\n",
       "+ztmfeNTmeMMZixNdNG8nBmT4zzOZab2D7HCNFgWPaF3Gv6Tqy/UNVhYl12pwMadsdOq3YZ7hf9v\n",
       "ZWWT+C9EeA6rc4KckOH+GPA5io1RTQSTokBq91ub8cFki8KMQ8xKL3eSVvD3SHYppIoS8p7VZWDG\n",
       "AyVlyaNMf/LZzP4Yv5ER5zLgiynuyQI7DzPj/2Ln/14w3uPMWNO7d2n3kP4WVFoLtYkxkN4z3Dvh\n",
       "/suUsHlkyRJvMR8BHL0u42g8Y+ekLDHim8j9M7BfSZnGBjO+Gwr7eJdqnYrOu2PHeRNN8/J4I7B5\n",
       "DRk6xaQokCzaGBBdleF+BTPy5FnEDMuqI97cfhPwjpywrw//jxB1cbwJ+HKRTMx4MfDjFK/kAHjh\n",
       "mm9O6+pH9Ne+s97n+D39x9jxfRlh8vgU6d1Wca4EFifc5phlDjBXpdea6mvZmPV1hyWv67JYuPem\n",
       "WF99E/jvhFue6Wwe8bwFqd2OSeLfbOaAek6crPyLsHxdRMu1bHwI+Eo4PpmZcR8LPQRFrnMimFQF\n",
       "UnehvZ8P8B9UgECsVgg8aMb5sfP/lxK+9yE3rUCS8z3uMONMM5bH/JJ59mrQD5sxP/R/n5QIk2eW\n",
       "mae4e37fzAkTJ7PWbsYvzfpMfge+zwU+7jTZvxI7/j394wbGbMMHM+vvdzdrdLB1mBWjD5vxuoTb\n",
       "uQXiZY1D9BCRMUru2EdKHKj+Pa/r+iwS2Iwf5Fh6HUA09jjfjMfM+JMQ56tm6+afTJ1J76QqkBMH\n",
       "+Pdm/Wa9LG+lXDM0Xoj1XqLb0wJmsKXZOkuStJfw+SXS6sOM3XLSPaxMUon/uosZFvnYjgK2LZHm\n",
       "oC6fMvSe34eA9wFHhPNZlQuzvm6hcSN5n/Mml6ZxqxmfC8d3ZgUyYzXwh7mCRN2L16d4DXo+Swf4\n",
       "12mBFJ2B/1Mz7jOrNE9rYpkEBfIYUa0wTtqLEy/w/p50S6YosvH7xCBfHv9DSlPbjGUUtBE34474\n",
       "acF8s8iq4c0ag8gYI8giKVdZBVJ6fo0Znw2D4EUZ9D5vnOPXG1+aZeEUuiXOJapUnEJ6gRU3C666\n",
       "QkGSqpMh4yRbR70xoKLdLFky/I5orOVsQretGdfmpJMm83MH5N2Lk2XWPYhB71yRVmGZ93bqTHon\n",
       "QYEcTTT4FR847E0gir+068YmQtdM2ZcydZVRM15L9otTpS+0rgK5I8M9b/Z7aRnMuAp4Uok0kwVI\n",
       "73zQhMcy5I6BmGWuFLteilXWrALPjDsTSvfzzBgTHE00H2I34PLCEmfzWJoMZUkxNf8YUUuh6OKB\n",
       "WZZ4j5hxjRlvMON3GWFy3yOzgWMcVxCZxn4EWFhE2GQW4T+r1TWvQpp5TN2SKV1XIJ8x4ywzfhMb\n",
       "OJwXCrd1hH7NMlZAaRQxvR0WjVkGFfRLy2+WSWSi4BgUPz3jwRMey1Bpdn3GnIO8tD4AvNmM14TZ\n",
       "05hxlxlXmfGjkvODsrg4dvxPGbKVnpMTKk/XDjCHHgZV3uGjgW3MeCh0kfVIzsuKpx2vtPWew9y0\n",
       "xEMlssnxpLJL0nSeriuQQVSdMT0sirys4zoQV1eu1IH6hsl6n6sUEr20Zs2cN+M6M/6zQpo9BnXL\n",
       "3Z9QanGT0kNicvww5v4fNeQZBdcNDgL0dx0+lqHc8yz4TgK+HY57PQN30z+5E2Z3ezeBt0CKImme\n",
       "pGWSVkm6WFLqmIKkxZJWSrpB0nGD4kvaWtLvJV0Vfp+sKqMZN+ZYVQyLpOliHm9PcatbsNYxIfxu\n",
       "zbzTSFqBDVOJNlkh6pnIplnM1WUzKDXxs3c//sSMpaTfw1Fs8XsFkQHK3ZSonJmhnPt4APmT9tLI\n",
       "6g7FjPvN1q0svK6FadZnvh0nz9jl28BXid7hIi22qVMgdaxHjgeWmdnJQTEcT2KDJUlzgI8TrXWz\n",
       "BvihpKVmtmJA/NVmtksN2YpSRrkUDVum+yZNOaYVnoXSHLKyrDSrl9lzXobVwnop2Qv5ZeWZJ8vQ\n",
       "7qUZD0mlBlx7cpZddLHHl2lmdvSrie7LhhQrLAcqBjMultgm5nQ25QejBy1lkkbv+R5BxkKRMUVU\n",
       "FO/CKsHBzCzjcBb9S0/02J1IGdxsZg8D5zDTBC8Sv0mafLhla/llum/OY/Y8k6YKs32BV+T45ymJ\n",
       "qgV/cnb/b+lfm6oJfgxcYtUXeYzTu9fj1L1bZIzqAjIUjBmvMutbmbeaEJE12sNm/CrHICHOR4jW\n",
       "niuTx3lmA83Lk/fji8ALC4RLy+93ZoVWiBjE/cDVDaTTKep8JJuZWc8mei3pC/JtSf8CY7cHt0Hx\n",
       "twndV8sl7UUzHM3s+RTfpdoGQq9JnMdf1PiM76RJ6G8T57Mw4z0pS2A0ghmXmGVuJHUjM33HdRVW\n",
       "bnyz1J3jinQVpRYIZuw0YKJelZbmFQy3W+j/KL7IX68rJrNANGOlGYc2IVhThDGMusYrafQ9z6DY\n",
       "fpAmwhDyzmIe/QtnTgW5XViSlpG+Tel74ydmZpKK1FpTN/pJxL8DWGBm90raFThP0o5mllLjOXpn\n",
       "6dNLwsnyvPclTABam3C7h2jSYBHi/ayZ/aFmrCC7wDqfaAnoNhhkUjloK9uqXVhF4xVZ56vpLoI8\n",
       "mQ5muN1Yt0ocS7Wl4ttYomeYDKugb6pSMZBxXr5E0iKa23Gyj1wFYmaZC61JWitpvpndKWlz0geZ\n",
       "1tC/p8RWzOxslxrfzB4i9Gma2Y8k/YzIBjyl5nzmVWZnLpmRKe9q2ifUIG+QeGLJqE1cWV0z5DIf\n",
       "ebw1UUX2tLxeSPX5FaULqJKTLEfNW6H0OzTO3EJ+12qSf2bwpmb7AM/L8R/z0qI5zGw5sXW+JDW2\n",
       "P0qdLqylREtNEP7T+l+vBBYGy6q5REtmL82LL2nTMPiOpGcSKY8b6SZ/0bYABUkOQtaqEZrxtaYH\n",
       "9MM6ReNq4jxSzPiMpe+t3iRpG3YNBTPMrPiCjWbcYjZrwcdkmEsZX5P4iaGOAjkJ2E/SKqLB2ZMA\n",
       "JG0h6esAZvYIcCxRX+/1wLnBAiszPvBi4BpJVxENjh1jZll286N8QUoXiGbrZr/XlfMGiq/Zk0WW\n",
       "En5iwQHRKl1Yn6T4Uuz+sY8HvfGW1xKNAxXZk6SLHD44iDOIyma8ZnYPKVtRmtkdRPt1984vIGVT\n",
       "ppz4X6bg0uEjpq2Z6NeEsZpda6SxERlLyFvxPUaqdAO9Hca/a7EDjEq5HkBsZ0Yz9hhRvsMi8902\n",
       "q2wS7cQY51VEnYjahceQLGHKMo4tjMpzbobIY8xeducByk+2K41Z3/Ipk8DZzN6b5ieQu+ijU4Jx\n",
       "snUfdz6U41d2YcaiE6UeYjQzjItQ1QorGX/cGaUCSbsnzyJhbh5mUldZTLANvkfze5BXIsxZSS4i\n",
       "usKMI1sRaALpugIZWaFkxj+RMdM5LAh4Rom0HgR2KBB0HjOGBm1T517vy+wxnFGsjTXupK34e7NZ\n",
       "39ypTmHRnh/J3RidCWUSu7DGbT2a95PSFxvmi+SSs9ptpzCb6VePkSw8bwb2JrpXTSuTIkYCznTQ\n",
       "dhflRDFpCmSBzd7juVXMcru+usTA2dA1OQX4ywr7tAwkGArkLVHvOE4FJqoLy6zUNrJOOYZd2D46\n",
       "DOVRgVEOsLoCGz1+zxuk6wpknPAXsxxjeb/MOLOFLQAcp5O4AmkOL3QcZ/zx77RBXIGUYyxrzSOi\n",
       "6Wsfhw95mp+n49Sm6wpknAqAcZLFcZx0/DttkK4rkHFiHGrUTTGKj8w/ZMfpOJNmxus0g5u8jgen\n",
       "MIW73DndwRVIc0x6Afsow11Wv4ltRcsy1s/MjPOJNiFzmuE84PNtCzFJuAJpjknqwppF2DJ20K6F\n",
       "ZYjv8DjR984ZD8Zty99JoOtjIKOuQWbtS+I4VRjrFpDjDKLrLZBPjDi/A4ANM/wmqTCYlkH0cZDB\n",
       "cTpLpxWI2ay1/oed392001fvTCbeded0mq53YTnDwQs2x3EG0ukWiNNpTmF8NstqC+9CczqNKxAn\n",
       "jaEXbGZcD1w/7Hwcxxke3oXlTDM+nuU4NXAF0hw+btAxzLgMeGrbcjhOV3EF4qQxEVvpFsGMe9rM\n",
       "vsW8Hac2lRWIpHmSlklaJeliSZtkhFssaaWkGyQdF3M/TNJPJD0qaddEnHeH8Csl7V9VxhEzSYXB\n",
       "V4HntC3EFOCtVqfT1GmBHA8sM7PtgG+G8z4kzQE+DiwGdgCOlLR98L4WOBS4NBFnB+DwEH4x8ElJ\n",
       "XWgp/aJtAZrCDDPjp23L4TjOeFOnYD4YOCscnwW8MiXM7sBqM7vZzB4GzgEOATCzlWa2KiXOIcDn\n",
       "zexhM7sZWB3SGXeWAE9vWwjHcZxRUUeBbGZma8PxWmCzlDBbArfFzm8PbnlsEcKVidM6Zjxo1net\n",
       "jjOISer2dKaQ3HkgkpYB81O83hs/MTOTlPYxNPWBpKYjaUnsdLmZLW8oP8dxnIlA0iJg0TDSzlUg\n",
       "ZrZflp+ktZLmm9mdkjYH7koJtgZYEDtfQH/rIo1knK2CW5p8Swak5TiOM9WEivXy3rmkDzSVdp0u\n",
       "rKXAUeH4KKLNWpJcCSyUtLWkuUSD40tTwsWtUZYCR0iaK2kbYCFwRQ05HcdxnCFQR4GcBOwnaRWw\n",
       "bzhH0haSvg5gZo8AxwIXES1bca6ZrQjhDpV0G7An8HVJF4Q41wNfCOEvAN5mZt5X7EwitwJvbVsI\n",
       "x6mKulo2SzIzczt6x3GcEjRZdnZhfoXjOI4zhrgCcRzHcSrhCsRxHMephCsQx3EcpxKuQBzHcZxK\n",
       "uAJxHMdxKuEKxHEcx6mEKxDHcRynEq5AHMdxnEq4AnEcx3Eq4QrEcRzHqYQrEMdxHKcSrkAcx3Gc\n",
       "SrgCcRzHcSrhCsRxHMephCsQx3EcpxKuQBzHcZxKuAJxHMdxKuEKxHEcx6mEKxDHcRynEq5AHMdx\n",
       "nEq4AnEcx3EqUVmBSJonaZmkVZIulrRJRrjFklZKukHScTH3wyT9RNKjknaNuW8t6feSrgq/T1aV\n",
       "0XEcxxkedVogxwPLzGw74JvhvA9Jc4CPA4uBHYAjJW0fvK8FDgUuTUl7tZntEn5vqyHj2CJpUdsy\n",
       "1MHlbxeXvz26LHvT1FEgBwNnheOzgFemhNmdSBncbGYPA+cAhwCY2UozW1Uj/66zqG0BarKobQFq\n",
       "sqhtAWqyqG0BarKobQFqsKhtAcaFOgpkMzNbG47XApulhNkSuC12fntwG8Q2oftquaS9asjoOI7j\n",
       "DIn18jwlLQPmp3i9N35iZibJUsKluQ3iDmCBmd0bxkbOk7Sjmf2mQlqO4zjOsDCzSj9gJTA/HG8O\n",
       "rEwJsydwYez83cBxiTCXALvm5JPqT6Sc/Oc///nPfyV/Vcv95C+3BTKApcBRwIfD/3kpYa4EFkra\n",
       "mqhlcThwZEo4rTuQNgXuNbNHJT0TWAjcmIxgZkq6OY7jOKOjzhjIScB+klYB+4ZzJG0h6esAZvYI\n",
       "cCxwEXA9cK6ZrQjhDpV0G1Er5euSLgjp7gNcI+kq4IvAMWb2qxpyOo7jOENAoTvIcRzHcUrRyZno\n",
       "WZMT20TSmZLWSro25pY52VLSu4P8KyXtH3PfTdK1we/fRij/AkmXhMmd10n6qy5dg6QnSLpc0tWS\n",
       "rpd0Ypfkj+U9J1ggfrVr8ku6WdKPg/xXdEl+SZtI+pKkFeH92aNDsj9bMxOvr5J0n6S/Gon8TQ2m\n",
       "jOoHzAFWA1sD6wNXA9uPgVx7A7sA18bcTgbeFY6PA04KxzsEudcP17GamdbgFcDu4fh8YPGI5J8P\n",
       "7ByOnwT8FNi+Y9ewQfhfD/gBsFeX5A/5/S3w38DSDr5DNwHzEm6dkJ9oLtvRsffnyV2RPXEdjwN+\n",
       "DiwYhfwju7AGb9AL6bfsOh44vm25gixb069AVhLNl4GogF4Zjvus0YALicaCNgdWxNyPAD7V0rWc\n",
       "B7y0i9cAbAD8ENixS/IDWwHfAF4CfLVr7xCRAnlqwm3s5SdSFjemuI+97Cky7w9cNir5u9iFVXVy\n",
       "YhtkTbbcgkjuHr1rSLqvoYVrU2Q1twtwOR26BkmPk3R1kPMSM/sJHZIfOAV4J/BYzK1L8hvwDUlX\n",
       "SvqL4NYF+bcB7pb0aUk/kvSfkjakG7InOQL4fDgeuvxdVCCdHPW3SKWPveySngT8L/AOS0zeHPdr\n",
       "MLPHzGxnopr8iyW9JOE/tvJLejlwl5ldRcysPc44yx94kZntAhwIvF3S3nHPMZZ/PWBX4JNmtivw\n",
       "OxJr+42x7OuQNBd4BZH1ah/Dkr+LCmQNUf9ejwX0a81xYq2k+QCSNgfuCu7Ja9iK6BrWhOO4+5oR\n",
       "yAmApPWJlMfZZtab19OpawAws/uArwO70R35/wg4WNJNRDXIfSWdTXfkx8x+Hv7vBr5CtBZeF+S/\n",
       "HbjdzH4Yzr9EpFDu7IDscQ4E/l+4/zCCe99FBbJucmLQuIcTTWocR3qTLaF/suVS4AhJcyVtQzRZ\n",
       "8gozuxP4dbAAEfB60idoNk7I7wzgejP7WNeuQdKmPSsTSU8E9gOu6or8ZvYeM1tgZtsQdUN8y8xe\n",
       "3xX5JW0gaaNwvCFRX/y1XZA/5HmbpO2C00uBnwBfHXfZExzJTPdVT87hyj/KAZ4GB4oOJLISWg28\n",
       "u215gkyfJ5pt/xDRGM0bgXlEg6KrgIuBTWLh3xPkXwkcEHPfjejDWw2cOkL59yLqe7+aqOC9imgZ\n",
       "/k5cA/A84EdB/h8D7wzunZA/cS37MGOF1Qn5icYRrg6/63rfZYfk34nI8OIa4MtEA+udkD3kuyHw\n",
       "C2CjmNvQ5feJhI7jOE4lutiF5TiO44wBrkAcx3GcSrgCcRzHcSrhCsRxHMephCsQx3EcpxKuQBzH\n",
       "cZxKuAJxHMdxKuEKxHEcx6nE/wd+Uyvv38Z+lQAAAABJRU5ErkJggg==\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x108925ed0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAG/JJREFUeJzt3X+UXOV93/H3xwLCD6kocqgEYmXF/KjAaxd6EhlqHG/s\n",
       "BHaXUyA+jihOamoD1mkDIpRTA4ceo5q0BLd2seBEoUImorXBxAYsUmlBpmwhjQxWD79kJMPaqPqB\n",
       "tFYAOZKDErl8+8e9C5fRndHM3Jm5s7Of1zlzdH88z73fvdoz373Pc+/zKCIwMzOr9J6yAzAzs+7k\n",
       "BGFmZrmcIMzMLJcThJmZ5XKCMDOzXE4QZmaWywnCJjVJo5IuKzsOs17kBGGTXaSfhkn6M0k3tzie\n",
       "RmNYKum/tehYb0l6fyuOZQZOEGa9RmUHYL3DCcK6mqR/K+nbFduWSbots2m+pL+U9DeSHpH03kzZ\n",
       "P5e0U9IeSf9L0unp9s8Dnwa+IGmvpO/mnHu5pP9Use27kv4wXb5O0vb0vJslfTznGL8uaZckZbZ9\n",
       "UtKzks4DbgAuTmN4Jt1/rKSVkl5Nj3+zpPek+05Of449knZLujfd/kR6+OfSY/1u/VfZrIqI8Mef\n",
       "rv0Ac4B9wLHp+mHAOHBmuj4KjAEnA0cCjwO3ZOr/S+AY4HDgvwDPZPbdDXypxrk/CmzNrP8y8Ldp\n",
       "TP8I2ArMSffNA95f5Tg/BAYz6w8C16TLNwH3VJR/EFgOHAUcBzwFfD7ddy9wQ7p8BPBPM/XeqhaD\n",
       "P/408/EdhHW1iNgFPAlM/EU8CPx1RDwzUQT4ekSMRcR+4H7gjEz9P4uIn0fEAeDfA/9Y0ozMKWo1\n",
       "yfwlEJI+mq5/CvirNKb/B/wS8AFJh0fE1oj4SZXj3AP8PoCkWcC5wDcz58/eXcwGhkgSyJsRsRu4\n",
       "DfjnaZG/J7ljmhsRfx8Rf1UjfrNCnCBsMlhF+gWb/ntPxf5dmeU3gekAkqZJ+mNJY5J+BrySlvmV\n",
       "ek4aEQHcB1ySbvo08I103xjwh8BSYFzSvZKOr3KobwD/TNLRwCLgiYgYr1L2fSR3OzslvSHpDeBP\n",
       "Se4kAL5AklCelrRR0mfr+VnMmuEEYZPBd4EPSeoHzif9kq7Dp4ELgE9ExLHAr6bbJ/5ir+fpp3uB\n",
       "T0l6H7AQ+M7Ejoi4NyI+SvKlHsCteQeIiO3A94FPkiS47FNLb1UU3wb8HfDeiPjl9HNsRHwwPdZ4\n",
       "RHw+IuYCi4E/8ZNL1i5OENb1IuJNki/mbwJPpV+4WdWaiaaTfNm+LukY4D9W7B8Han65RsSzwF8D\n",
       "dwEjEfE3AJJOlfRxSb+UnmM/SbNTNfcA1wH9wAMVMcyf6MSOiJ3Ao8BXJc2Q9B5JJ0n6jfS8vyvp\n",
       "xLTuHpLE9FbmWCfV+nnMGuEEYZPFKpIv17x3BqJieWL9HuD/AjuAjcD6irIrgdPTppzsl3albwIf\n",
       "551+A0j6H24BdgM7SZqtbqhxjAdIOrIfTPtKJvx5+u9rkjaky58h6YB+EXg9LTMn3fdrwPcl7SW5\n",
       "s1oSEVvSfUuBVenP86kasZjVRUkza4EDSIMknWjTgLsi4taK/b/HO+2me4F/FRHP11PXbIKkPmAz\n",
       "MDsi9pUdTzMkvQwsjoj/WXYsZvUodAchaRpwB8mTJacDl0g6raLYT4DfiIgPATcD/7WBumak7wBc\n",
       "C9w7iZPDJ0n6vZ0cbNI4rGD9hcDYxC2upPuAC4FNEwUiYn2m/FPAifXWNUv7DsZJnkAaLDmcpkga\n",
       "BRYA/6LkUMwaUjRBzCV56mLCduDDNcpfBqxpsq5NQRHxc9LHVieriBgoOwazZhRNEHV3YEj6TeBz\n",
       "wEcarWtmZp1XNEHsAPoy630kdwLvIulDwAqS4QbeaLCuE4mZWRMiotjgjUXG6SBJMD8G5pM8lvcs\n",
       "cFpFmXkkY+Wc1WjdtFyUPR5JTkxLy45hMsTUrXE5Jsc0FeJqxXdnoTuIiPiFpCuBR0geVV0ZEZsk\n",
       "LU733wl8kWSQs+Xpu0AHImJhtbpF4jEzs9Yp2sRERKwF1lZsuzOzfDlweb11zcymmn5puA+WzIAj\n",
       "98L+bbBsY8SaQ9dsr8IJYooaLTuAHKNlB1DFaNkB5BgtO4Aco2UHkGO07AByjJYdQBWjzVbsl4bP\n",
       "hq+tSIasB+AKOKlfouwkUfhN6naTFFG0o8XMrEsNSSNr4bzK7cMwsiZiqNnjtuK702MxmZmVaEYy\n",
       "0dVBpicTRpXKCcLMrER7k5GAD7IvmdukVE4QZmYl2gbLrkheBXjb5fDjrXB7WTFNcB+EmVnJ+qXh\n",
       "eXDVdDhqH7y5FW4v2kHdiu9OJwgzsx7kTmozM2sbJwgzM8vlBGFmZrmcIMzMLJcThJmZ5XKCMDOz\n",
       "XE4QZmaWywnCzMxyOUGYmVmuwglC0qCkzZJelnRdzv4FktZL2i/p2op9WyQ9L+kZSU8XjcXMzFqn\n",
       "0IRBkqYBdwC/BewAfiBpdcXUoa8BVwEX5RwigIGIeL1IHGY2NXTrzGu9quiMcguBsYjYAiDpPuBC\n",
       "4O0EERG7gd2Szq9yDI+zZGaH1M0zr/Wqok1Mc4FtmfXt6bZ6BfA9SRskXVEwFjPrYX2wJJscAFbA\n",
       "yfOSFgprg6J3EEWHgv1IROyUdBywTtLmiHiyspCkpZnV0YgYLXheM5tkunnmtW4gaQAYaOUxiyaI\n",
       "HUBfZr2P5C6iLhGxM/13t6QHSZqsDkoQEbG0WJhmNtl188xr3SD9w3l0Yl3STUWPWbSJaQNwiqT5\n",
       "ko4ALgZWVyn7rr4GSUdLmpEuHwOcC7xQMB4z61HdPPNaryo8YZCkIeA2YBqwMiJukbQYICLulDQH\n",
       "+AHwD4C3gL3A6cA/BB5ID3MY8I2IuCXn+J4wyMyA9sy81qs8o5yZmeXyjHJmZtY2ThBmZpbLCcLM\n",
       "zHI5QZiZWS4nCDMzy+UEYWZmuZwgzMwslxOEmZnlcoIwM7NcThBmZpbLCcLMzHI5QZiZWS4nCDMz\n",
       "y1V0wiAzs57SLw33wZIZcORe2L8Nlk3VIcWdIMzMUv3S8Nnwtezc11fASf0SUzFJuInJzCzVB0uy\n",
       "yQFgBZw8D64qK6YyFU4QkgYlbZb0sqTrcvYvkLRe0n5J1zZS18ysk2bAkXnbp8NRnY6lGxRKEJKm\n",
       "AXcAgyTTiF4i6bSKYq+RZN//3ERdM7OO2Qv787bvgzc7HUs3KHoHsRAYi4gtEXEAuA+4MFsgInZH\n",
       "xAbgQKN1zazz+qXhIWlkkTQ6JI30S8Nlx9Qp22DZFTCW3XY5/Hgr3F5WTGUq2kk9F9iWWd8OfLgD\n",
       "dc2sDaZ6J+3GiDX9EsNw1XQ4ah+8uRVunwo/e56iCSI6UVfS0szqaESMFjivmVVRrZN2OGkmnhJf\n",
       "kmkymHQ/q6QBYKCVxyyaIHYAfZn1PpI7gZbWjYilzQRnZo1xJ+3klf7hPDqxLummoscs2gexAThF\n",
       "0nxJRwAXA6urlFWBumbWAe6ktaxCCSIifgFcCTwCvAh8KyI2SVosaTGApDmStgHXAP9O0lZJ06vV\n",
       "LRKPmRXjTlrLUkSRboT2kxQRUXn3YWZt0i8Nz3Mn7aTXiu9OJwgzsx7Uiu9OD7VhZma5nCDMzCyX\n",
       "E4SZmeXycN9mdhDPiWDgBGFmFab6cBv2Djcxmdm7eE4Em+AEYWbv4uE2bIKbmMzsXRodbsP9Fb3L\n",
       "CcLM3iUdbuOkbDNTteE23F/R2/wmtZkdpN7hNoakkbVwXuX2YRhZEzHUmWgtTyu+O30HYWYHqXdO\n",
       "BPdX9DZ3UptZ0zw8eG9zgjCzpnl48N7mPggzK8TDg3cnD/dtZma5umK4b0mDkjZLelnSdVXKLEv3\n",
       "PyfpzMz2LZKel/SMpKeLxmJmZq1T6CkmSdOAO4DfAnYAP5C0Ojt1qKRh4OSIOEXSh4HlwFnp7gAG\n",
       "IuL1InGYmVnrFb2DWAiMRcSWiDgA3AdcWFHmAmAVQEQ8BcyUNDuz381HZmZdqGiCmAtsy6xvT7fV\n",
       "WyaA70naIOmKgrGYmVkLFX1Rrt4e7mp3CedExKuSjgPWSdocEU8eVFlamlkdjYjRxsI0M+ttkgaA\n",
       "gVYes2iC2AH0Zdb7SO4QapU5Md1GRLya/rtb0oMkTVYHJYiIWFowTjOznpb+4Tw6sS7ppqLHLNrE\n",
       "tAE4RdJ8SUcAFwOrK8qsBj4DIOksYE9EjEs6WtKMdPsxwLnACwXjMStVvzQ8JI0skkaHpJH+5CEN\n",
       "s0mp0B1ERPxC0pXAI8A0YGVEbJK0ON1/Z0SskTQsaQz4OfDZtPoc4AFJE3F8IyIeLRKPWZk8sqn1\n",
       "Gr8oZ9YiHtnUuklXvChnZgmPbGq9xgnCrEU8sqn1GicI63md6jj2yKbWazxhkPW0TnYcb4xY0y8x\n",
       "7JFNrUe4k9p6mjuObapyJ7XZIbjj2Kx5ThDW09xxbNY8Jwjrae44Nmue+yCs53lKTJuKPOWomZnl\n",
       "cie1mZm1jROEmZnlcoIwM7NcfpPaprx+abgPlsyAI/fC/m2wzJ3YZk4QNsV5Dgez6go3MUkalLRZ\n",
       "0suSrqtSZlm6/zlJZzZS16yd+mBJNjkArICT58FVZcVk1i0KJQhJ04A7gEHgdOASSadVlBkGTo6I\n",
       "U4DPA8vrrWvWbh6Kw6y6oncQC4GxiNgSEQeA+4ALK8pcAKwCiIingJmS5tRZ16ytPBSHWXVFE8Rc\n",
       "YFtmfXu6rZ4yJ9RR16ytPBSHWXVFO6nrfQ3bb0JbV/IcDmbVFU0QO4C+zHofyZ1ArTInpmUOr6Mu\n",
       "AJKWZlZHI2K0uXDNDpYmg7oSgh+JtW4laQAYaOkxi4zFJOkw4EfAJ4BXgaeBSyJiU6bMMHBlRAxL\n",
       "Ogu4LSLOqqduWt9jMVlXqPJI7Nh6uNpJwrpN6WMxRcQvgCuBR4AXgW9FxCZJiyUtTsusAX4iaQy4\n",
       "E/jXteoWicesnfxIrE01hV+Ui4i1wNqKbXdWrF9Zb12zbuVHYm2q8VhMZnXyI7E21ThBmNXJj8Ta\n",
       "VOMJg8wa4NnpbLLwjHJmZpar9KeYzMysdzlBmJlZLicIMzPL5QmDrCd4CAyz1nOCsEnPs8KZtYef\n",
       "YrJSteIv/yFpZC2cV7l9GEbWRAz57sKmolZ8d/oOwkrTqr/8aw2B4bsLs+a5k9pK06rB72oNgeEB\n",
       "9sya5wRhpSky+F2/NDwkjSySRn8Ov/I52JndPzEEhgfYM2uem5isNM0OfpfXbPRJePVc+D8zYV92\n",
       "CIwhaUkz5zAzJwirod2du+ngdydlv+jrGfwur9noAThhGJ6/P2KoFecwMycIq6ITnbvNzgfdSLOR\n",
       "55w2a17Tj7lKmgV8C3gfsAVYFBF7csoNArcB04C7IuLWdPtS4HJgd1r0hogYyanvx1xLcKhHR8uI\n",
       "aUJlbE8AjwKb4PW/hR/4MVaz8gfrux5YFxGnAo+l6+8iaRpwBzAInA5cIum0dHcAX42IM9PPQcnB\n",
       "ytPNnbvZeRmeIJmz9o+A78CstXDe2fC1/mQudDMroEiCuABYlS6vAi7KKbMQGIuILRFxALgPuDCz\n",
       "33cGXaqbZ0/bGLFmPVw9DCNfhjf+Q8V+P8Zq1hpFEsTsiBhPl8eB2Tll5gLbMuvb020TrpL0nKSV\n",
       "kmYWiMVarNtnT9sYsWZNxNDR8Hze/m640zGb7Gp2UktaB8zJ2XVjdiUiQlJeZ0atDo7lwJfS5ZuB\n",
       "rwCX1YrHOmeydO52852O2WRXM0FExG9X2ydpXNKciNgl6XjgpznFdgB9mfU+krsIIuLt8pLuAh6u\n",
       "ca6lmdXRiBitFbe1RpoMuiohVPJjrGYJSQPAQEuPWeAppi8Dr0XErZKuB2ZGxPUVZQ4DfgR8AngV\n",
       "eBq4JCI2STo+Inam5a4Bfj0iPp1zHj/FZDV5nmizg5U6J3X6mOv9wDwyj7lKOgFYERHnp+WGeOcx\n",
       "15URcUu6/R7gDJJmqFeAxZk+jex5nCDMzBpUaoLoFCcIM7PGebhvAzybmpm1hxPEJOf5DsysXTzc\n",
       "9yTn+Q7MrF2cICa5bh4Sw8wmNzcxTXLd9KKY+0LMeosTxCTXLS+KuS/ErPf4Mdce0A0vinXz8OBm\n",
       "U5EfczWgO4bEcF+IWe9xJ7W1RDf1hZhZazhBWEt0+/DgZtY490FYy3RDX4iZJTwWk5mZ5Sp7Tmoz\n",
       "M+thforJDskvwJlNTU4QVpNfgDObutzEZDV5MECzqavpBCFplqR1kl6S9KikmVXKfT2dv/qFZupb\n",
       "ufwCnNnUVeQO4npgXUScCjyWrue5GxgsUN9K5BfgzKauIgniAmBVurwKuCivUEQ8CbzRbH0rl1+A\n",
       "M5u6inRSz46I8XR5HJjd4frWARsj1vRLDPsFOLMpp2aCkLQOmJOz68bsSkSEpKbfuDtUfUlLM6uj\n",
       "ETHa7Lmscd0wGKCZ1SZpABho6TGbfZNa0mZgICJ2SToeeDwiFlQpOx94OCI+2Gh9v0ltZta4st+k\n",
       "Xg1cmi5fCjzU4fpmZtZGRe4gZgH3A/OALcCiiNgj6QRgRUScn5a7F/gY8F7gp8AXI+LuavVzzuM7\n",
       "CDOzBnmwPjMzy1V2E5OZmfUwJwgzM8vlBGFmZrmcIMzMLJcThJmZ5XKCMDOzXE4QZmaWywnCzMxy\n",
       "OUGYmVkuJwgzM8vlBGFmZrmcIMzMLJcThJmZ5XKCMDOzXE4QZmaWywnCzMxyNZ0gJM2StE7SS5Ie\n",
       "lTSzSrmvSxqX9ELF9qWStkt6Jv0MNhuLmZm1XpE7iOuBdRFxKvBYup7nbiDvyz+Ar0bEmelnpEAs\n",
       "ZmbWYkUSxAXAqnR5FXBRXqGIeBJ4o8oxPJWomVmXKpIgZkfEeLo8Dsxu4hhXSXpO0spqTVRmZlaO\n",
       "w2rtlLQOmJOz68bsSkSEpGjw3MuBL6XLNwNfAS6rEsfSzOpoRIw2eC4zs54maQAYaOkxIxr9Xn87\n",
       "mM3AQETsknQ88HhELKhSdj7wcER8sNH9kiIi3BRlZtaAVnx3FmliWg1cmi5fCjzUSOU0qUz4HeCF\n",
       "amXNzKzzitxBzALuB+YBW4BFEbFH0gnAiog4Py13L/Ax4L3AT4EvRsTdku4BziB5mukVYHGmTyN7\n",
       "Ht9BmJk1qBXfnU0niE5xgjAza1zZTUxmZtbDnCDMzCyXE4SZmeVygjAzs1xOEGZmlssJwszMcjlB\n",
       "mJlZLicIMzPL5QRhZma5nCDMzCyXE4SZmeVygjAzs1xOEGZmlssJwszMcjlBmJlZLicIMzPL1XSC\n",
       "kDRL0jpJL0l6VNLMnDJ9kh6X9ENJGyUtaaS+mZmVp8gdxPXAuog4FXgsXa90ALgmIj4AnAX8gaQF\n",
       "DdQ3M7OSFJmTejPwsYgYlzQHGI2IBYeo8xBwe0Q8Vm99TzlqZta4sqccnR0R4+nyODC7VmFJ84Ez\n",
       "gaeaqW9mZp11WK2dktYBc3J23ZhdiYiQVPVWRNJ04NvA1RGxr3J/HfWXZlZHI2K0VtxmZlONpAFg\n",
       "oKXHLNjENBARuyQdDzxepYnocOAvgLURcVsT9d3EZGbWoLKbmFYDl6bLlwIPVRaQJGAl8GI2OdRb\n",
       "vxP6peEhaWSRNDokjfRLw2XEYWbWbYrcQcwC7gfmAVuARRGxR9IJwIqIOF/SOcATwPPAxIluiIiR\n",
       "avVzztO2O4h+afhs+NoKOHli2xUwth6u3hixph3nNDPrhFZ8dzadIDqlnQliSBpZC+dVbh+GkTUR\n",
       "Q+04p5lZJ5TdxDTpzYAj87ZPh6M6HYuZWbeZ0gliL+zP274P3ux0LGZm3WZKJ4htsOwKGMtuuxx+\n",
       "vBVuLysmM7NuMaX7ICDpqJ4HV02Ho/bBm1vhdndQm9lk505qMzPL5U5qMzNrGycIMzPL5QRhZma5\n",
       "nCDMzCyXE4SZmeVygjAzs1xOEGZmlssJwszMcjlBmJlZLicIMzPL1XSCkDRL0jpJL0l6VNLMnDJ9\n",
       "kh6X9ENJGyUtyexbKmm7pGfSz2CzsZiZWesVuYO4HlgXEacCj6XrlQ4A10TEB4CzgD+QNDHvdABf\n",
       "jYgz089IgVg6Kp0cvKt0Y0zQnXE5pvo4pvp1a1xFFUkQFwCr0uVVwEWVBSJiV0Q8my7vAzYBczNF\n",
       "JusgfANlB5BjoOwAqhgoO4AcA2UHkGOg7AByDJQdQI6BsgOoYqDsANqhSIKYHRHj6fI4MLtWYUnz\n",
       "gTOBpzKbr5L0nKSVeU1UZmZWnpoJIu1jeCHnc0G2XCRjhlcdN1zSdODbwNXpnQTAcuBXgTOAncBX\n",
       "ivwgZmbWWk3PByFpMzAQEbskHQ88HhELcsodDvwFsDYibqtyrPnAwxHxwZx93T1hhZlZlyo6H8Rh\n",
       "BequBi4Fbk3/faiygCQBK4EXK5ODpOMjYme6+jvAC3kn8WRBZmblKHIHMQu4H5gHbAEWRcQeSScA\n",
       "KyLifEnnAE8Az/NOE9QNETEi6R6S5qUAXgEWZ/o0zMysZF0/5aiZmZWjK96kbsFLd4es346Y0nJf\n",
       "lzQu6YWK7S1/EbAFMZV5nQYlbZb0sqTrMttbdp2qnaOizLJ0/3OSzmykbklxbZH0fHptnu5UTJIW\n",
       "SFovab+kaxv9eUqIqazr9Hvp/9nzkv63pA/VW7ekmBq7ThFR+gf4MvCFdPk64I9zyswBzkiXpwM/\n",
       "AhbUW78dMaX7Pkry+O4LFdtvAv5Np6/TIWIq5ToB04AxYD5wOPAscForr1Otc2TKDANr0uUPA9+v\n",
       "t24ZcaXrrwCzWvx7VE9MxwG/BvwRcG0jdTsdU8nX6Wzg2HR5sN2/U0ViauY6dcUdBMVfujtk/XbE\n",
       "lMbyJPBGlWO0uoO9aExlXaeFwFhEbImIA8B9wIWZ/a24Toc6x7tijYingJmS5tRZt9NxZd8ravXv\n",
       "0SFjiojdEbGBZDSEhuqWENOEMq7T+oj4Wbr6FHBivXVLiGlC3depWxJE0ZfuGqrfjpiqaPWLgEVj\n",
       "Kus6zQW2Zda38+436ltxnQ51jlplTqijbrOKxAXJQxzfk7RB0hUdjKkdddt53G64TpcBa5qs24mY\n",
       "oMHrVOQx14ZIWkfSTFTpxuxKRIRqvPug/Jfu6q7fjpiqWA58KV2+meRFwMtKjqmp+i2IqdZ5mrpO\n",
       "DZ4jq9OPTReN65yIeFXSccA6SZvTO8ROxNTquu087kciYmdZ10nSbwKfAz7SaN0GFYkJGrxOHUsQ\n",
       "EfHb1fYp6VCdE++8dPfTKuUOB74D/PeIyL53UVf9dsRU49hvl5d0F/Bw2TFR3nXaAfRl1vtI/vJp\n",
       "+jo1co4aZU5MyxxeR91mNRvXDoCIeDX9d7ekB0maGIp+8dUTUzvqtu24kb5TVcZ1SjuBVwCDEfFG\n",
       "I3U7HFPD16lbmpgmXrqDJl66q6d+O2KqJf2ynFD1RcBOxtSC+s0ecwNwiqT5ko4ALk7rtfI6VT1H\n",
       "RayfSc97FrAnbR6rp26zmo5L0tGSZqTbjwHOpTW/R438vJV3Nu26Vk3HVOZ1kjQPeAD4/YgYa/Ln\n",
       "6UhMTV2nor3qrfgAs4DvAS8BjwIz0+0nAP8jXT4HeIuk1/6Z9DNYq367Y0rX7wVeBf6OpG3ws+n2\n",
       "e0heEHyO5EtzdhfEVOZ1GiJ58myM5GXJie0tu0555wAWk7yEOVHmjnT/c8A/OVR8Lfr9biou4P3p\n",
       "7/uzwMZWxnWomEiaFLcBPyN54GErML2d16rZmEq+TncBr/HOd9LT7f6dajamZq6TX5QzM7Nc3dLE\n",
       "ZGZmXcYJwszMcjlBmJlZLicIMzPL5QRhZma5nCDMzCyXE4SZmeVygjAzs1z/H2cXYmg1r0bAAAAA\n",
       "AElFTkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1088d4290>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('R2 :', 0.89321708326642046)\n",
      "('Gene with Strongest Coefficient :', 250)\n"
     ]
    }
   ],
   "source": [
    "lm = linear_model.LinearRegression()\n",
    "lm.fit(Xtrain,ytrain)\n",
    "\n",
    "printOutput(lm,Xtest,ytest)   ### PROVIDE THE OUTPUT ###"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "zeros=(lm.coef_==0)\n",
    "print(zeros.sum())\n",
    "\n",
    "print(np.abs(lm.coef_).max())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ridge regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAZAAAAEKCAYAAAA8QgPpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJztnXnYHmV1/z9fIiCLgAgSlmhAgwRqBVQEUQwoGiiytEVM\n",
       "FalawSp2swouLXFpQdqqRRDxV1DUKuBSGlGWaImoCEoNiJAYokQDmABlVUG28/tj7nnfeeeZmWe2\n",
       "Zz+f63quZ+Zez2z3uddzy8xwHMdxnKpsMGgBHMdxnNHEFYjjOI5TC1cgjuM4Ti1cgTiO4zi1cAXi\n",
       "OI7j1MIViOM4jlMLVyCOUxJJL5W0ctByFCHpbEnvLxn2s5I+1GuZnPHFFYjTVyStkfQ7SQ9KWifp\n",
       "85K2GLRcZTCz75rZbm2nK+lySe9OnO8o6Ykct6d3kfEvzezDJbO28MuT6wlJu5RMy5lAXIE4/caA\n",
       "w8zsKcDzgOcCpWrMVZD0pLbT7CHfAQ5InB8ArMxwW2Vmd/ZTMEB9zs8ZIVyBOAPDzNYDVwB7xG6S\n",
       "9pV0taR7JV0v6WUJv50lXSXpAUlLJZ0l6fPBb26oMb9J0i+BbwX3N0m6WdI9ki6T9IxEeh+TtF7S\n",
       "/ZJ+ImmP4H6opJtCPrdJemdwXyBpbSL+fEnLgqw/lfTqhN9ng3yXhHSuKajNfxfYP3H+EuDjwAsS\n",
       "bi8Frgpp7xau//8krZR0dCrfDyXO3y3pjnAdf5HRqtg6S0ZJVwX/G0Jr8WhJ24Sw94a8r5LkCmaS\n",
       "MTP/+a9vP+BW4OXheCfgJ8A/hvMdgbuBheH8FeH8aeH8B8DpwJOICtz7gc8Fv7nAE8BngU2AJwNH\n",
       "ALcAzyGqLL0P+H4I/yrgOmCLcP4cYHY4/jWwfzjeEtgrHC8A1objDYHVwMlBngOBB4Bdg/9ng+wv\n",
       "AGYBXwC+lHNPNgZ+BzwvnN8I7Ax8D9gzuP0UeD2wGbAWOC5c057AXcD8EO4zwAfD8cJwLfPDPflC\n",
       "uEe7lJExGTacnwqcHcLOiu+R/yb35y0Qp98IuFjSA8CvgJ8DcZ/964FvmtllAGb2LaJC/o9Cy+EF\n",
       "RMrmMTP7PrCEzi6WxWb2kJk9DLwVONXMfmZmTxAVgHuGtB4BngLMl7RBCLMupPEIsIekLczsfjNb\n",
       "nnEd+wKbmdlpQZ4rgUuARYkwXzOz68zsceA/iQr7Dszs98C1wMskbQ1saWa3ErVMDghu84m6ug4D\n",
       "bjWz883sCTO7HvgacHRG0q8BzjOzFWb2EHBKOuuyMibuy/bAXDN7PDwDZ4JxBeL0GwOOMLMtiGr0\n",
       "BzHdVfNM4OjQRXKvpHuJWhqzgR2Ae4JiiFlLJ0m3ZwL/nkjr/4L7DqHAPxM4C1gv6RxJTwn+fwIc\n",
       "CqwJXVT7ZuSzQ0b+vwzu8XWuT/g9BGyekU7MVUTjHC8B4oL5ewm3tWa2NlzTi1L36M+A7TLS3D4l\n",
       "420ZYarI+C9Era4rJP1c0kkFYZ0JwBWIMzDM7CrgE8BHgtOvgM+b2VMTv6eY2elEXTFbS9okkcQz\n",
       "6CQ5q+hXwPGp9DYzs2tC/p8wsxcAuwO7Au8K7teZ2ZHAtsDFwEUZ+dwBzEmNATwTuL3aXZjiKqJx\n",
       "jgPCMcDVRAr0AKLWR3xN38m4R2/PSPPXwJzE+ZyMMKUxs9+Y2d+b2bOAw4G/k3RQkzSd0cYViDNo\n",
       "Pg7sI+lFRH3wr5b0SkmzJD05DFzvaGa/JOrOWixpQ0n7EXXnFO1H8CngvZJ2B5C0ZTzgLOkFkl4k\n",
       "aUOi8YeHgcdD2q+TtGXo1nkQeDwj7WtDvHeHOAuCPBcE/6qDyz8AnkrUjfddADO7l2iM4vVMK5VL\n",
       "gF0lvT7ku6GkF0qKpxcrkfdFwBvDoPumwD+k8uwm43rgWVOBpT+S9OygNB8gui9Z98aZEFyBOAPF\n",
       "zO4GzgdOMrPbiAa+3wvcSVTbfifT7+nrgP2IuqI+BFxI1C8/lVwq7YuJWjcXSLqfaHD6VcF7C+DT\n",
       "wD3AGqKC+l+C3+uBW0Oc40O+M/Iws0eAVwOHEA1inwkca2arEuHSyi1X2ZnZ74gU5IZm9tOE11VE\n",
       "LaGrQrjfAK8EXkvU2vk10djORul8w1jSGcCVwCoiJQXw+5IyLgbOD11lRwPzgKVESvVq4Cwz+w7O\n",
       "xCKzZhtKSVpIVIucBfyHmX0kI8wZRB/a74A/N7PlkuYAnwOeTvTSftrMzgjhtyYqHJ5J9HG/xszu\n",
       "aySoM3ZIuhC42cw+MGhZRgFJ84mU6EZhUoHjNKJRC0TSLKKa10KifuRF4SVNhjkUeLaZzSOqzZ0d\n",
       "vB4F/tbM9iCa0fL2RDP8ZGCpme0KfDucOxNO6HZ6lqQNJB1C1A9/8aDlGmYkHSVpY0lPJWqNLXHl\n",
       "4bRF0y6sfYDVZrbGzB4l6v89IhXmcKIuCszsWmArSduZ2bowBTFulq8gWgcwI074P7KhnM54MJuo\n",
       "O+ZB4GPAW83shsGKNPQcTzSWsZqo0vaXgxXHGSeamnvYkc5pgi8qEWYnEtMHJc0F9iIamATYzqJV\n",
       "yoRwWVMUnQnDzC4hGkR2SmJmhwxaBmd8adoCKTuAkp7tMRVP0ubAV4C/Di2RmQGjQZpmAzWO4zhO\n",
       "6zRtgdxO5zzz9GKldJidghthCuVXgS+EGTMx6yXNNrN1krYnmpEzA0muVBzHcWpgZu3YMGtiB4VI\n",
       "Af2cyA7RRsD1BJs8iTCHEpmngGiw/JpwLKJZWB/LSPd0ommdEA2gn5YRxprIPugfkcmNgcvh8o/m\n",
       "z+V32RvIb22l1agLy8weA04ELgduBi40sxWSTpB0QgjzTeAXklYD5wBvC9H3J5pvf6Ck5eG3MPid\n",
       "BhwsaRWRqYvTmsgp8SSJ45uk4TiO48yk8Z4JZnYpcGnK7ZzU+YkZ8b5HzhiMmd1DZIm1LeYRKa9P\n",
       "t5im4zjOROMr0QfHskEL0JBlgxagIcsGLUBDlg1agIYsG7QADVg2aAGGhcYr0QeFJLOSA0ES84Gb\n",
       "zXx3NcdxJpsqZWc3vAXiOI7j1MIViOM4jlMLVyCO4zhOLVyBOI7jOLVwBeI4juPUwhWI4ziOUwtX\n",
       "II7jOE4tXIE4jjN2SGw2aBkmAVcgjuOMI7+R2GTQQow7rkAcxxlXGtv6c4pxBeI4juPUwhWI4ziO\n",
       "UwtXII7jOE4txl6BSMwDzhu0HI7jOOPG2CsQ4EiirXQdx3GcFpkEBeI4juP0AFcgjuM4Ti1cgThD\n",
       "icRRku8g6TRiNLdbHSFcgTjDyteAjQcthONUQWKuxDMGLUe/8JWazjDjNUhn1FgJPAxsNWhB+oG3\n",
       "QAaExGES2wxaDscZYwbRBboxTI4hx8YKRNJCSSsl3SLppJwwZwT/GyTtlXA/T9J6STemwi+WdJuk\n",
       "5eG3sKmcQ8jXgcz75TiOMwo0UiCSZgFnAguB3YFFkuanwhwKPNvM5gHHA2cnvD8T4qYx4KNmtlf4\n",
       "XdZETsdxHKd9mrZA9gFWm9kaM3sUuAA4IhXmcOB8ADO7FthK0uxw/l3g3py0fQaO4zijyMSUXU0V\n",
       "yI7A2sT5bcGtapgs3hG6vM6VNBEDUo7jOKNE01lYZWfJpDVyt3hnAx8Mxx8C/g14c0ei0uLE6TIz\n",
       "W1ZSnmFhYmoqjjNBzJKYZcbjgxYEQNICYEEv0m6qQG4H5iTO5xC1MIrC7BTccjGzO+NjSf9BNOCc\n",
       "FW5xBVmd0cMVrNOEQU4DH5p3N1Ssl8Xnkk5pK+2mXVjXAfMkzZW0EXAMsCQVZgnwBgBJ+wL3mdn6\n",
       "okQlbZ84PQq4MS+s4ziOMxgatUDM7DFJJwKXA7OAc81shaQTgv85ZvZNSYdKWg38FnhjHF/Sl4CX\n",
       "AU+TtBb4RzP7DPARSXsS1SBuBU5oIqczsvhCQscZYhqvRDezS4FLU27npM5PzIm7KMf9DU3lchzH\n",
       "cXqLr0R3hg6Jtw5aBsdxuuMKxBlG4sWmQzMQ6ThOJ65AHMcZV7wC0mNcgTjDjA+iO84Q4wpksHgN\n",
       "yXF6h1dAeowrEGeYqaVgJbaT2KJtYZzJQ8IkNq8RdSKU1yQokIl4kM4M1hHtaOg4bfDUQQswrEyC\n",
       "AvFuoslkewAJSZOxO5zj9JtJUCDOZHMs+VsGOI7TAFcgzjDTRuuxzNYBjuPUwBXICCLxdMnHdpzx\n",
       "ROK9EhcMWg6nO65ARpPtBi2A4/SQNxJZ9h4WfBw1B1cgjuM4Ti0mQYEMc1eP12wcp3cM87c/FkyC\n",
       "AhlHJuXDcAXrNKGt96fO9zYR3+gkKBAvhBzHcXrAJCiQYWYiaimOUxGv9I0IrkAGi38ovceVtOP0\n",
       "CFcgzjDjCnYyGTal7+9hDq5ARpNh+8CGGf/4HadHTIIC8cLWcRynB0yCAhlmvHbce7wCMbm09ez9\n",
       "HcphEhSIF9KO4zg9oLECkbRQ0kpJt0g6KSfMGcH/Bkl7JdzPk7Re0o2p8FtLWipplaQrJPl+DpOJ\n",
       "K39nVJmIVksjBSJpFnAmsBDYHVgkaX4qzKHAs81sHnA8cHbC+zMhbpqTgaVmtivw7XA+jtR9ySbi\n",
       "5XScIcErMjk0bYHsA6w2szVm9ihwAXBEKszhwPkAZnYtsJWk2eH8u2Rv9jMVJ/wf2VBOx3Ecp2Wa\n",
       "KpAdgbWJ89vo3MCnTJg025nZ+nC8nvE1X163ZuMtkO74PXK85dBjntQwftmPNP0gS3/cZmaSMsNL\n",
       "Wpw4XWZmy7KSKJvXCDEpH8akXKfj9AxJC4AFvUi7qQK5HZiTOJ9D1MIoCrNTcCtivaTZZrZO0vbA\n",
       "nVmBzGxxNXEdx3Emi1CxXhafSzqlrbSbdmFdB8yTNFfSRkS7iC1JhVkCvAFA0r7AfYnuqTyWAMeF\n",
       "4+OAixvKOW6MY6vKcWKGreXp31sOjRSImT0GnAhcDtwMXGhmKySdIOmEEOabwC8krQbOAd4Wx5f0\n",
       "JeBqYFdJayW9MXidBhwsaRVwUDivy7C9jEmGWTbHGXW84O8xTbuwMLNLgUtTbuekzk/Mibsox/0e\n",
       "4BXd8pbY1oy7yks7NkzKh+EKdjIZtvfb38McRn0l+osHLUBD6n4o/kI7zhBjNnRKsCeMugJxnG5M\n",
       "xIfsOINg1BXIpNbEvVB0nB4gsUBi30HLMSo0HgMZAYa5sB2YApR4FiAzVg9Khj4xqZWMUWaQz+xK\n",
       "4AFgywHKMDKMegukMhLHSEOtVMrQhvw3Aj9rIZ1eMtDCX2IziacNUgZnINRe+DxpTJwCAZ47aAGG\n",
       "hI2ZjOff5OO/ELi7LUGc0jQqsKXWKx7eis1hEgoQf/jZjMx9kdhOYtYAsp47gDyd9hiZd3xUmQQF\n",
       "4ow+64C/GbQQjuPMZNQVSJ0ahtdKIkbhPiRlfPqA83cmB3/uJRl1BdI3JFZKbDRoOQI+qDdApLHd\n",
       "XmBY8AJ8RJgEBdJWYfsc4CktpeVUZygKFYkNiLrUHAcAiXkS/ztoOQbBJCiQNENREAWGSRank6zn\n",
       "07dnJvFxiX/pV37OFFWf8f7A3r0QZNiZRAXShKEq8CU2kHjpoOXoE5PYbffXTObkgUl81iOJK5DR\n",
       "5iXAVYMWooe0obCbFEZDVWGYRCQ2kti039n2Ob+RZdQVSJkHnQ4zDi9HXChOgimamLrPbRye9yTz\n",
       "BaqPObX+zCW2kPjDttMddUZdgQycsMjtlkFlP6B8nQlH4t0Sf9Kr5BPHRzMck1dOB27I8ZvY73DU\n",
       "a7DD0Fe6G/DsAeU9LNOK+8EgnvXEFgwl+AiwCvjqoAXpE5sMWoBhZNRbIGU+8HTBU6pQkHhaD2zq\n",
       "tM1/DFqAHqOc4yoMQyWjA4n5ZYP2VBAni6x7XvQeTewzGnUF0kvuJmo+DzM7ZDlK/I/EF/stzBjS\n",
       "k4JBYmfg5l6k7cxgYgv2fuEKpJj0imMBSJwusVVwG8Ya7oHAIYMWop9IvFLi1IRT/FxKFyISm0hs\n",
       "065kmVTpehzG96vXjNo1T6yimkQF0sbDfhfR4qHymYoNW5Rl1D6wfvBO4OSGaZwL3NUlTFyJaPIe\n",
       "VYk7doWTxMYSOw5ajgLG7p73iklUIB1IPLlOtArp/wHwSI08mjAOH0Eb11AljTlZcSU2bkEOZ5p/\n",
       "Bm6rEkFiW2mgiyq90pbBJCiQdAHywowwD0k8v3LCkXHFrPTSDKvxPZO4XxqLTbbyPvBTc9yr8PEW\n",
       "0khSpTAa5oKrrmzdLCtnKf3XAh8rmX7jiofEyRI/aZrOuNNYgUhaKGmlpFsknZQT5ozgf4OkvbrF\n",
       "lbRY0m2Sloffwrzsa4h8YI57VXPhBrwF+NcaMuQicaTEPS0kVfbj3gLYs24mEs+QOKFinNdIIzGF\n",
       "PH6/ntGjdNsO22+GWbamHEL53UvH+T4U0kiBSJoFnAksBHYHFkmanwpzKPBsM5sHHA+cXSKuAR81\n",
       "s73C77IGYrZZg0u/KJXWYUh8IOWUJdv+wFOrpJuXXUthuvEO4FMV41wI7NdC3r1mYguGIaSfz2Ic\n",
       "rVf0hKYtkH2A1Wa2xsweBS4AjkiFORw4H8DMrgW2kjS7RNwyD+0r6a1OJZZLjQdTe8U/ps6HZRB9\n",
       "mLtJhoHa61EkNs+ZQFGWYX42dWXrFm8Yr3kYZRo4TRXIjsDaxPltwa1MmB26xH1H6PI6V9JW5JNu\n",
       "BewJHCnx1hLy9xSJfwpz/suGfzkDru1I7Cex+yBlSFDlXvT7Ay8r24PAJ/uQT3bkyIbTUK6ilrAK\n",
       "79owF+AT20JpqkDKPtSqN/hsYGciZfBr4N+ygy0GNnlfGDNZkPB4ZkhjphDivYnjDv8e8F6otKDv\n",
       "W8BeXUP19mO6Gri0h+nHTJKZ8pdLHFc3ssQuEgvDwG7V7s11wJfr5t0HPloy3K49laKYYVZeXZG0\n",
       "IJSRiyUtbjPtpgrkdmZOfZxD5/S8dJidQpjcuGZ2pwWIzHXsk539YuChfzazxWa2LOGRN+3ynxLH\n",
       "dVoo6xPHM5SixJKcOPumHSRuKshjg0S4HaVMO1tVu1H+tOJU1GdIbF4ljxr8cdpB4mUSl6TcPl03\n",
       "g7rXEHYdhPZqljsDn01mUTH+WURK/VSiMcOuhNr9XCIbTvMq5tdPXiWxj8TWCbes+3NiUSItbzdd\n",
       "1ZTJUGNmy0IZudjMFreZdlMFch0wT9JcSRsBx0BHQboEeAOApH2B+8xsfVFcSdsn4h8F3FhRrlnd\n",
       "g3TQ9AV5dYWwcbP9HV3CfQeqWfrNqaF+GXhllXSAP6gYPs7/bTXX1UCkVP4o5faWmmn9AVH3UQcS\n",
       "fyPxZwVx41X8w9I1UVeOuPt02Au/a6Hxzou/l3ihxLnAOWUiSOyS59VQlomh0VRKM3tM0onA5USF\n",
       "9rlmtkLSCcH/HDP7pqRDJa0Gfgu8sShuSPojkvYkevFvhcJpolkfR6kPRkJmQ/1xbZbjnilzMMNR\n",
       "dUZUMn4bH85ZwArgyhbSqjwGUnKx2ceIunbS3Ytxfpek8m7DqGOSxu+cxIeBB8w4vR/51UHiIODH\n",
       "ZtxXInhHpU8qtcYqyWzgTckkCmTbBfh5QZjkPZsKEypHW5rl90ZMEo3n4pvZpaT6zM3snNR5ZvMz\n",
       "K25wf0NTsUqGezeRWerS1JxRU/cDnl02YLDN1c0MRx6xfG9LJlkxbje3ppSV5z2ZkcUuZvyiRnqQ\n",
       "vR9Fz0yZhIrAeV3SeB9RhaxIgbT2HML7tZ0ZP6sQ7dtE66TeVSJslqw/rJBXXhp5dGsl5z2j7xPt\n",
       "fz6xSiPJuK5EL/twO3YYkzhB4siCOMm+2l5NIuhMQBwq8XhBvnt0SWIqTkZL4xSJ19FlXxOJOcla\n",
       "ocQ3ifbtjv3iCsmOUm9qvRILJC7O8OqW35YNss3diU7ipRLb5/mXIQyQJ2cg7kV+l2hmayhs/Zpn\n",
       "8SBune0vde02zeMcYGXNuGWwnOMqVBnnq/pNxjLt3UJaY8MkKJD3VYz7Kcp3A5V9cao2xfPSqDK4\n",
       "2xFGQqFb4QUpr3nAP3SLT7QAMFkrPITpadS/Av4qHHcUZBInSp1jEmE8osy6nbmh2+FoOtcaJckr\n",
       "fExig7xuuqDw9k+cb8D0GEIRV9Fsmi5ErfBkS6LOQrYP0bn1q6X+PwycMSNhsY1USrnW3RWwbAu1\n",
       "stKQ+Jw0Y5LKVyqkVXRPlUrjiqqyTQrjqkCSD79oDUkeU/PmJZ7UQm26ii2svLwWN5TBiFoY34bM\n",
       "brimHx1Mt87iGm/SBMh+kDkr6iNM26sqKjj/lGgHvN06hBJ/DLysi2wfBH4PlLVqcFTiuFvh2UZr\n",
       "KzmoX7ZiIokDJO4mZ2+YEtxKxniVxF1SqSnlOYLx0opR4ndmEdnvSRbHAj+omE8dOqYQp2Y1egtk\n",
       "hGm60CrrHiTTTI8TJWt5ZQuOZPfR+1P5H56aOZVnqyszvZDGRnQOuGfJtkGBXxuFYNzai9Pq2HVP\n",
       "6phVVSXfWcBBGe5fJX/CQcyriZ5lPBut23uT10dee1V5aP21zX7A0yh+pkX3eHM6F/8CbMPMlmrV\n",
       "7yzeVyX9rl6dk18c7otUGPvLIU7r6oIwsaXlqyUOkvKWCuTycOI4mGcaahP1PWEcFEgWZV/2DWBq\n",
       "XKFOWnWU14dS5/8NjY0nfo5oNluStGxGcYHyRMpdEn8ozah9l510EaezQZQQpzFdwy5a19HP2ULd\n",
       "BlHzZHld+P9Dib9MekjsK/GwxNNy4r64ioDJpAvkKrpnG5cIA7DpVEbisJJdWpnphjGqBQmn9MZm\n",
       "+5E9jlD72Rf0EHSMC0o8V+Jhpu/pfkSt8q93yaaMfJVM1I8D46BAsh7sFiXjJj/MZM04LjSfSf11\n",
       "CEn+toU0iqhqDiKr2W90Kp0bgAekqe6RqRqpVDgOEZs/j2v7OVaa+TkzBz7/KitcRapYIS4ir3IQ\n",
       "j/lcS+fYxylE15PX0ug2++qFJcLlTSl+TkbY2MJ0uhWwQOLWhNPmErNDQfx1Zs7Gy5IzTq+jO5Go\n",
       "O+zKRJ5lLdpmKZXGpBYoQjQhYmPqjTNlpT+sWzX0hXFQIEVzvbutRTgmcdyxMppol7t/z4n7GoZn\n",
       "gVbewPA2Km9CI30tyTRn9ElLvAoyZ0Kl6bYmI28hVykkVqXGWarGf1soOP+7QrT4Ps3oylK0B/0W\n",
       "Kbf09rhZExuSi2R/KLEFnRWCosItlierC2an8B8r6VmJsHMlXpsIm7yPcX6ZayFKMjUVX8EigDS1\n",
       "Ij7ru6m0H480dW1ZJNNPK8NuPQ4QXWtZG3bpiQsTxTgokB8BSJya8cEuqJKQVGm2xbEMYPAs1P7S\n",
       "+eatGn8L0yY0kl1YWTwZeH1Bvn+dOP1qsZQz4jXZY7yociCi2WPLUl5V+s/PAl5EZDE6TdXKwSKm\n",
       "u0wukvgLohZsEiX/JT5B57O7n1zbbx08uYucp4X/XSUugqmB7Xhw/EuJsEmFmNWiLXoWWZMMkpMQ\n",
       "HgzdYnW78NL57cZMQ6xpkosS013GT8TJpNy3TcVPK6hhqSwOFaOwqU83dg81qZOh8Q5iB1cMn9nX\n",
       "HQa1s7oU+k2Vlz5tLyn9gSV35es2YJ2k1OJGqbKplbh7prS145bIKkiPynA7BljeJa1C+051CIpr\n",
       "fYbX0YnjrNbKponj12X4Zz6foMgfkNjWjLsLRJtNl0H9grGjNCu6B5mR7m/MplrRcQskqzWYnPQx\n",
       "DpXrnjMuNymuSWX2tfeQjjUmEpsR2bjq5XaYZWYCbcHMDzWrll2WfrS00pMAujGM727yPj2vwL/q\n",
       "zLNXJM7/U+LCxHk6rf2p96xfXiaQxJ0pp1jZdLNl9RamFVfe9d9eRoYaJCs8uQoEuLkgDW+BZDCM\n",
       "H2ETsj7afvMber8HetkVwW9MHL+Vat07yQ+mjCmKsUaaWkBZtiDZls6V0Ur91+U1ieOjU351v+ms\n",
       "ypdJbKeZ5nu2TYWJB7//vEv6GwBvD8d53ZpVVpLXJe7CKjNdPknZyQATxbgpkIGiafPh/fgQypDu\n",
       "RitlpTSQLOTe3IIsdWh7CnUV0orihWHx2N8XxEnL9P3UeXpHyjZ4Uep8Ft3fvyr3bh3Fck/NMOxi\n",
       "0uWwCnn2krgF8q8V443CFsx9xxVIu6QH7IaNYdlpsCz96DorM5ssZk9gboF/V3MfivY3SQ+ut8km\n",
       "RBM8CsWomGZZW193FPgNfE+SMAGl7DhL2fQmmnEYRB8m6u6DgZQ5EBv7ZQ1qTgLLCvyq1iDb4JoW\n",
       "0mhjXVERZTZWmlsyrbiAHFQLtBecP2gBxglvgfSGOhta5Vp8Bb5QV5AGbNo9SM8p2sZ0UY/z7mVr\n",
       "spdKpM1WW9Vxgp6TY3rIGRD+MNolHqB7e2GobBYnT6RaRiDbpOxq/nHlWYMWoCZtdqvkrgsaIFkL\n",
       "frvScD2Sk4OibcdHD0k2hDPrvsXMKZejzAO4EnHGh6VUX+dVG7PhtdArycysFflcgTiOMwncSh8X\n",
       "nboCGXJcgTiOM6xMigLxMRDHcRynFq5AHMdxnFq4AnEcx3Fq4QrEcRzHqUVjBSJpoaSVkm6RlLPz\n",
       "nM4I/jdI2qtbXElbS1oqaZWkKyQNek2E4ziOk6KRApE0CzgTWEhkZ2mRpPmpMIcCzzazecDxTG1A\n",
       "Xxj3ZGCpme1KtF/xyU3kdBzHcdqnaQtkH2C1ma0xs0eBC6Bjr+zDCfZnzOxaYCtJs7vEnYoT/o9s\n",
       "KKfjOI7TMk0VyI7M3FrytuBWJswOBXG3M7N4V7X19H5/DcdxHKciTa3xll3JV2bRirLSMzOLFg1m\n",
       "sThxvICKW6A7juOMPZIW0KPCsakCuR2YkzifQ9SSKAqzUwizYYZ7vKXlekmzzWydpO2hYxvNwOK6\n",
       "cjuO40wEZraMxNYIkk5pK+2mXVjXAfMkzZW0EXAMsCQVZgnwBgBJ+wL3he6porhLgOPC8XFU2/TH\n",
       "cRzH6QONWiBm9pikE4HLifbAONfMVkg6IfifY2bflHSopNXAbwn7dOfFDUmfBlwk6c3AGmbuAe04\n",
       "juMMAW5M0XEcp2XcmKLjOI7jFOAKxHEcx6mFKxDHcRynFq5AxoPfD1qAAfDQoAVwnEln0hXIIYMW\n",
       "oCV+NGgBBsCaQQvgOJPOpCuQoZ0pUZFBT0f7r5Lh7mkxz++2mJbjODWYdAUyLgxagTw6gDx/UDPe\n",
       "eeHfLTz3hgdbSmebltJxeogrkGb8Y814Z7UqRbuUUUb/kBPnNy2k3RafyHH/bfgftNJNcuagBRgT\n",
       "rm0Q92XHauKZAAAXsUlEQVStSTFBTLoCyerC+pMK8T/WliAD4NYc9y1KxE3bO/sbInP8uzWSqBoC\n",
       "/jjH75fAF7rEH5QCeV2G2zB0pd7YUjpN7usHE8d17smXc9x/1S2iGVfVyG/imXQFksV9ZQKFlaZ1\n",
       "ZwK1XWD8JMd964I4edf52xz3mKuTJ2bIjHVmLDKbMoaZR9nrLjumktfiuaCEDHkF3akl867ClMkg\n",
       "M76Y4f8Yna26ItocS4LIvNCBLaeZ5mclwjwA3NIlzCUFfh8le0bi4yXydmrgCqSTftRMlxX4rauR\n",
       "XlaBebkZ91ZNyCz3+mPlIHpfY/7nEmGM4meV59f32r5Z1wLssR5km1YyvysI+20z/q9Lers0lOc/\n",
       "SoS5ienxtKznlNdqBqbe3X9POMUVqyYK5Jdd/LtVuMYaVyAzeVPF8HWVzf0Zbh8P/08UxFvKdDP9\n",
       "L4kGgvc04/sdghkLu8hQtSCNr3Ud9Ly5n3dfX5E4zlJk3ymRdlELZPcS8XvBpyuGj6/hVRl+cV/+\n",
       "jsCxCXcDHs5KzGzGxm55dKvYPDf8X5/jfwfTrYtfp/yuCXJcBtwc3B5JJ2BWWYnFxlmLvqmm/KKH\n",
       "aQ8946RA3pI6fy/VX5zPAd/LcM/6UKGF1krC6Npl4b+otnQy8PMQ71NmfMSMGzLCNd3nJWbbxHF8\n",
       "rX9hFslQg0soN9CZWdAFVhLV+rLSUeof4PvADzPCdNw3M1aQrVi7dauU5VtZjmasysm3DneENB8m\n",
       "6hJK8vQG6RbNtNub6fcjniCyC/D5nPB7Eo2bxbw/HcAss5IF8O4Mt7wp3bHMx+T4N2UXsiuDE8O4\n",
       "KJAHmN5DHQAzTiXsQ5LimWmHRCFuZjwKXBHOnwj+V6TjBPcqCiRZU8ksLIIcsdJ7c0aQ3xN9bE/J\n",
       "8HtxIp0yTfakDLvmhEkWQPG1FnWFFGLGccBBJYKuLkhjvhmbm03VVJPE13Q70x/2S4FXJ8OE+3xN\n",
       "Km5R4Vr0nE/PcFsT/s8HXptwb2vqcJaizCL53poZD9I5O209JTCr1s1mxq3MbEVM7Thqxp1mM7qa\n",
       "qqS7IsM5qdyS9ySWuVs3VFUODrKku9TqdD+PNOOgQH5C9PFnfeQdD9SMXxF1BcHMl+2fMtK4Avh2\n",
       "CzICXEi5fU3ij+y8DL+fmfG4WefgsVnldRFienZKXi0qeX+qKMuisZeyNe0dE8fxwGg3GQQ8DTgz\n",
       "1Oq3zFDyAsi4h5b0r0BW6+TMkMefm3FhyXTOZmaX02FMv6dpimScanWHVsgmKf+9S8ozCvwF8Lfk\n",
       "T7qw1H9ZNqOgEkN2LwVE7+lFNKhkjRrjoEAeM6tsC6ojvBnvTxc2ZhxiNqPfvQxZs2wy88wh92Wv\n",
       "WgvsQplpm/H78dYMvzzF+iPgf2pJNI2ZRV0xgbgQ7la4y4x7zKZajukuHJhZ4z42w7+NrqTK35UZ\n",
       "d5t1TD3OkyU5jrMr8PyE360wY/wr3Y2brFS9mFCbboEyyr3qvf1pYYbGuWZ8nPwKS1XF8daQ7u+A\n",
       "/y7KusDvWJp1FY4U46BAYrIeaq9m3OTVQIDcGuftDHa+/6rE8SbAnxeEfX34j7sgvkhUs4IutToz\n",
       "9iFjfIHO9Q+lW0yh6yuLm5g5Myfv/iZl/XCOe7c0Ys4nGocpYgXRTptJXkP+AHNdZMYtJKZxm2Fm\n",
       "M/JOX+NNibA/MOuoSFwDfD3ldjX1KFuAK+e4KVUVyNeA9wCY8fcFaTwCHBmO30/UCoqiGY+YTc7M\n",
       "rHFSIEmazhK6qwUZ/jZ1fm7iOOuDjGuKvZ5GfIcZD6daM3nK4HHg74jWW/xLyjtv0VYe6S6BrL7s\n",
       "LJKFWbqFeEdqZk7XwieMcZUhOTMpOUh7D4lCOJBew3F/ehacGV8uOTYVU3QtM/wqtky7TbA4xYzD\n",
       "U25lnlWZFshi4AMl0krGAfhGyXBpOSp9U2bcZcZpOWn/CdH46QuCkv7vEOc7oRVUOp9xYpwUSPzw\n",
       "9oKuU1hj8j7SE4A5FfLOWreQnFL6+1T3WLogeTnT3T5ZL+F+FWSZUfvP2FozmX7eSu5k/I8F2dNy\n",
       "Fc7JL0GZj+2jwNsqpNmty6dIhsyxEqIWy/FML7RLr2o2sxmtGujttNGqpK+r6gy9e4B3hOM7c8I8\n",
       "TrRQsPB9MuOLZiyukHf8DAq7smjWalleMtz/mvErM/63QV5jxzgqkF+aZa4QT/b9nkx2/3eUkPFb\n",
       "sw5zHXnsT2oGWEhjOcULBpNh/ydRO80qWKsYK8yb5RQXalMfW82PIZZvOXBpjfil8zDjnRWeA3Qv\n",
       "SDbLcf8p00YAO9Iw4wkzlhEN0J+Rk0bcwjyQ+oYeZ2Rb4FelwExXVuJndmXJ+OsT31NSpvuBLYnG\n",
       "UW4M96jIgkCWzIcV+CXd6yqINloEVdLwFsgIktXfHpN88aa6pcy4KWPAshZmtfuHc5NsGD+v9ls0\n",
       "blNGhnT30d1mHFohzZh0obBPjTS6pZ0mVkiZs2PMeK5Zx8K1LEUSD9DHfk8w/f6dCGxrxrJ4EL8h\n",
       "d7eQRsdU8yCfzEpNp4aCwtuMB8I4Sl5LLnmcdT+7dU3Fq+M/yXQrqApx/nmtru1LpFFFefXCosBQ\n",
       "M+oK5ALCAsLES5xV+O1C5+rXqhSZqS7qCqlKryYDNO1/r3JdVfrv29wMq+7q+iyKvo13AAeaMcuM\n",
       "6yCaMmvWTqEf0ksulExPs46vs5TdtobUncrdBkcAzzDjl2YzLBavSoVLypgcnypUIGaso51vK8YV\n",
       "yIjxSJeBye8DJ2Us+OklpQvPHAbZDC4zeaCufOl4vbjONguD+NvosDMWBu+XNUj7EooXt6VbSsnF\n",
       "r6ck5Lg2y71l2lAg7wSWlMwj2cV6j2WbWSnqfvs8THXNxi3P9cAB5UTtoMo1T5zRxtoKRNLWkpZK\n",
       "WiXpCklb5YRbKGmlpFskndQtvqS5kh6StDz8PllVtPjAjN+ZZa4UrkMvpuBmDQ42LVjr7I8e51lk\n",
       "MbWfiq1uXm08oxldbGYdM64aY8argX+rEiX8f8CMD5J9nUUt5Nspb44lbzJB0u8uKG/OxoyPWr5h\n",
       "z2MoXrRXhuQ3b2a8IJwmF1WmzZ3E4zpXkG/NGqJrXUe5LkVXIBU4GVhqZrsSLSrrMNMgaRbRqtyF\n",
       "RIbqFkmaXyL+ajPbK/yKZuE0LdSqFDhlw5aWyWzKAF23+KXyDn3bdU3Md02+ZrwqYy9NZaijPPMY\n",
       "ptb5lAKpGX936tfAk+/e6UQz43YFFpSI23V3QjMuYuZzvQiyTQcVcEeOe9FYVFxBOMmM5+WE2RF4\n",
       "lRnbl1zbMXFdWE2M7h3OtOXP84lmHKWVyD5EymANgKQLiPo1V5SM3yYX0N70yqKXKZlHnuXXooKw\n",
       "wwop7bV+9qP4o46NGNYxk56kSN6Hyb7GJnyAqODpZpI8TdG13Ek0y6hXlJ0+CtNdMUXmVj5Djl2r\n",
       "nBX5ZUnW7qvar/oHyplxnyKYfulm/iU9jrZUyrQP12gmW8oaQjfug1bH80aCJgpkOzOLX9j1wHYZ\n",
       "YXZk5oKs24AXlYi/s6TlRIOH7zezxrVYMz5D9JElWUa9bWnTq6qTL+rlwCtT/vHLWrTXQcxhwFNr\n",
       "yNQVsw4Dgkn2oHgMpJWxj7CQb2Opo2C5moRByEoZVFtb0I342RwAbNhiujMw43sS82jJ0q8Z95Fv\n",
       "RqcJtSsvYdZb2QWjjbAM+3D0t9t1W4Zr/U9fKFQgkpYCszO83pc8MTOT1HXqJwmLnAXx7wDmmNm9\n",
       "kvYGLpa0h5ll1Jzf9DzpM4vDybKq74tF1kk/VDJ4siaWW9MNq4KzjDVCZF+nsJAM6x6qrH0oS+HN\n",
       "sZnWbXsx3tPt4bQ5k60xRc94ABSNS/SaQZrfyeN9FO86GZO38LF1KloD6CuSFlCuy7EyhQrEzHIN\n",
       "rUlaL2m2ma2TtD3ZD+t2Zq7o3onpne0y45vZI4QuDjP7saSfA/OAH3cmf971Zuctnpap6Goa0zj1\n",
       "MGPsB1KHhdSe590ljTIFdhXLpp8mep5VJkB0u8YraW/f7lHnFGDTPuU1dAokTJcuXAhphqRCKwZD\n",
       "d129wsyWkVjULKm1GXtNBgqXwJSRu+OAizPCXAfMCzOrNiKacbGkKL6kbcLgO5J2IVIeo7rrV16X\n",
       "UNWXt0pfbB3StacyrclczPi1GWdXjNethXRQm+ssyuQ5rJhxesvddkVMTEHrVKeJAjkNOFjSKiLz\n",
       "GacBSNpB0jcAzOwxohW6lxNtVXmhma0oik/U93xDGAP5MnCCmeUtmBr2AuCN4b+JnL8zY22GTauq\n",
       "5PVFP93yd39zJou7mbY0/Kzw3w8FsoZock0/ydqiwKlI7UF0M7sHOvfKMLM7gD9KnF9Khs2kgvhf\n",
       "IzKrPGxU7gIy48GcbrUqCqWNQdaNybGnZVba8nCVLqyYA4gmUZRZyNnLysCo16L7MjhrNr2FsRm/\n",
       "kDiR3rd+49llRQsNayddkGd6Qo1Tg7b2zp4E1gC7DSDfxgVrhp2nbjQ1ZRLn+13o+dhU2wydtGY8\n",
       "JLHnAPI9q3uoDm7vHqRvXMgQPs9xYtQVSD+7sPYjMj+xRYZft1p8+iUe2hkbPaTKLKx+5T2MXaCr\n",
       "iUyoz8Cs0Gjo0GDG1RJPHrQcEJlCoXMix7XAZQMQZywZdQXSN8y4T+JeMhSIGQ9InAu8uWRaj1Ku\n",
       "ZnQwfehCyKBoEL1uoftw6nwYC+9+k3UPnpfjPjJY9S2m+8naPk5AGHvGUYH0sr+4zof9TjoLz3KZ\n",
       "Gd+qE69HNCnUnmXWMZMurUDjQuc0Brsga6BrLPJMzjut4V1aLTLqCiRdqO02wBlFeQPpH+23IMNG\n",
       "hvLIYjHwntg8esvcA5nPYaRr+o4zaEZdgczArNCabK8Z95pN2h5T2/ymV/38YQHnO0sGb2oZ1hlu\n",
       "vNLQIsNkcXQUmOSXr+1rH9Z7+Un6t8p7WO+B45Ri1Fsgw/QBDpMso0C6xTYU9y+0sHplEt9xxgpv\n",
       "gbTHuHdhOY7jzMAViFOWoWghtMw4XpPj9I1RVyDDVAAMkyxN6ce1jNP96hVXUrx3uuMMlFFXIE5v\n",
       "GER3nCuUFGb8sxlzBy2H4+ThCsQpS9sF/DCMGbnSmjzWDFqAcWLUZ2ENE8NQIPaSx+ldd8o1PUzb\n",
       "cWK2Al/p3yajrkB+0uf8JrbGasYT0Gp3ytS9NGO/FtMdJSb2fRoEvu9N+4y6AqljbroJfwpsk+M3\n",
       "ToXBOF2L4zg9YqQVSA/NauTlt7zAe9y7sJz28XfGGWl8EN3Joh8F2419yKMb3tJynAa4AmkPL4yq\n",
       "8QlGvAXcAv7OOCPNpH/AbTJO3RE9L9hC9+Pjvc7HcZze4S0QZ5L5zaAFcJxRxhVIe3h3xOixFNht\n",
       "0EI4zqjiXVhOFg8OWoB+ELrRBrkJmeOMNLVbIJK2lrRU0ipJV0jaKifcQkkrJd0i6aSE+9GSbpL0\n",
       "uKS9U3HeE8KvlPTKujI6tfkG8OxBC+E4znDTpAvrZGCpme0KfDucz0DSLOBMYCGwO7BI0vzgfSNw\n",
       "FHBVKs7uwDEh/ELgk5K8q62PmGFm/HzQcjiOM9w0KZgPB84Px+cDR2aE2QdYbWZrzOxR4ALgCAAz\n",
       "W2lmqzLiHAF8ycweNbM1RHtU79NAzn7Rk/28HcdxhpUmCmQ7M1sfjtcD22WE2RFYmzi/LbgVsUMI\n",
       "VyXOMHAWPinBcZwJonAQXdJSYHaG1/uSJ2ZmkrJmIbU1MykzHUmLE6fLzGxZS/lVpt9mVZyxwN8Z\n",
       "p+dIWgAs6EXahQrEzA7O85O0XtJsM1snaXvgzoxgtwNzEudzmNm6yCIdZ6fgliXf4i5pOY7jTDSh\n",
       "Yr0sPpd0SltpN+lyWQIcF46PAy7OCHMdME/SXEkbEQ2OL8kIl1zFvQR4raSNJO0MzAN+2EBOx3Ec\n",
       "pwc0USCnAQdLWgUcFM6RtIOkbwCY2WPAicDlwM3AhWa2IoQ7StJaYF/gG5IuDXFuBi4K4S8F3mZm\n",
       "3tR3xpF7By2A4zRBo1o2SzIzGyf7U47jOD2nzbLTZw05juM4tXAF4jiO49TCFYjjOI5TC1cgjuM4\n",
       "Ti1cgTiO4zi1cAXiOI7j1MIViOM4jlMLVyCO4zhOLVyBOI7jOLVwBeI4juPUwhWI4ziOUwtXII7j\n",
       "OE4tXIE4juM4tXAF4jiO49TCFYjjOI5TC1cgjuM4Ti1cgTiO4zi1cAXiOI7j1MIViOM4jlMLVyCO\n",
       "4zhOLVyBOI7jOLVwBeI4juPUorYCkbS1pKWSVkm6QtJWOeEWSlop6RZJJyXcj5Z0k6THJe2dcJ8r\n",
       "6SFJy8Pvk3VldBzHcXpHkxbIycBSM9sV+HY4n4GkWcCZwEJgd2CRpPnB+0bgKOCqjLRXm9le4fe2\n",
       "BjIOLZIWDFqGJrj8g8XlHxyjLHvbNFEghwPnh+PzgSMzwuxDpAzWmNmjwAXAEQBmttLMVjXIf9RZ\n",
       "MGgBGrJg0AI0ZMGgBWjIgkEL0JAFgxagAQsGLcCw0ESBbGdm68PxemC7jDA7AmsT57cFt27sHLqv\n",
       "lkl6SQMZHcdxnB7xpCJPSUuB2Rle70uemJlJsoxwWW7duAOYY2b3hrGRiyXtYWYP1kjLcRzH6RVm\n",
       "VusHrARmh+PtgZUZYfYFLkucvwc4KRXmSmDvgnwy/YmUk//85z//+a/ir265n/4VtkC6sAQ4DvhI\n",
       "+L84I8x1wDxJc4laFscAizLCaepA2ga418wel7QLMA/4RTqCmSnt5jiO4/SPJmMgpwEHS1oFHBTO\n",
       "kbSDpG8AmNljwInA5cDNwIVmtiKEO0rSWqJWyjckXRrSfRlwg6TlwJeBE8zsvgZyOo7jOD1AoTvI\n",
       "cRzHcSoxkivR8xYnDhJJ50laL+nGhFvuYktJ7wnyr5T0yoT78yXdGPz+vY/yz5F0ZVjc+VNJfzVK\n",
       "1yDpyZKulXS9pJslnTpK8ifynhVmIH591OSXtEbST4L8Pxwl+SVtJekrklaE9+dFIyT7czS98Hq5\n",
       "pPsl/VVf5G9rMKVfP2AWsBqYC2wIXA/MHwK5XgrsBdyYcDsdeHc4Pgk4LRzvHuTeMFzHaqZbgz8E\n",
       "9gnH3wQW9kn+2cCe4Xhz4GfA/BG7hk3D/5OAa4CXjJL8Ib+/A/4TWDKC79CtwNYpt5GQn2gt25sS\n",
       "78+WoyJ76jo2AH4NzOmH/H27sBZv0H7MnNl1MnDyoOUKssxlpgJZSbReBqICemU4njEbDbiMaCxo\n",
       "e2BFwv21wKcGdC0XA68YxWsANgV+BOwxSvIDOwHfAg4Evj5q7xCRAnlaym3o5SdSFr/IcB962TNk\n",
       "fiXw3X7JP4pdWHUXJw6CvMWWOxDJHRNfQ9r9dgZwbYpmze0FXMsIXYOkDSRdH+S80sxuYoTkBz4G\n",
       "vAt4IuE2SvIb8C1J10l6S3AbBfl3Bu6S9BlJP5b0/yRtxmjInua1wJfCcc/lH0UFMpKj/hap9KGX\n",
       "XdLmwFeBv7bU4s1hvwYze8LM9iSqyR8g6cCU/9DKL+kw4E4zW05iWnuSYZY/sL+Z7QUcArxd0kuT\n",
       "nkMs/5OAvYFPmtnewG9J2fYbYtmnkLQR8Gqi2asz6JX8o6hAbifq34uZw0ytOUyslzQbQNL2wJ3B\n",
       "PX0NOxFdw+3hOOl+ex/kBEDShkTK4/NmFq/rGalrADCz+4FvAM9ndOR/MXC4pFuJapAHSfo8oyM/\n",
       "Zvbr8H8X8F9EtvBGQf7bgNvM7Efh/CtECmXdCMie5BDgf8P9hz7c+1FUIFOLE4PGPYZoUeMwEi+2\n",
       "hJmLLZcAr5W0kaSdiRZL/tDM1gEPhBkgAo4le4Fm64T8zgVuNrOPj9o1SNomnmUiaRPgYGD5qMhv\n",
       "Zu81szlmtjNRN8T/mNmxoyK/pE0lPSUcb0bUF3/jKMgf8lwradfg9ArgJuDrwy57ikVMd1/FcvZW\n",
       "/n4O8LQ4UHQI0Syh1cB7Bi1PkOlLRKvtHyEao3kjsDXRoOgq4Apgq0T49wb5VwKvSrg/n+jDWw2c\n",
       "0Uf5X0LU9349UcG7nMgM/0hcA/Bc4MdB/p8A7wruIyF/6lpexvQsrJGQn2gc4frw+2n8XY6Q/M8j\n",
       "mnhxA/A1ooH1kZA95LsZcDfwlIRbz+X3hYSO4zhOLUaxC8txHMcZAlyBOI7jOLVwBeI4juPUwhWI\n",
       "4ziOUwtXII7jOE4tXIE4juM4tXAF4jiO49TCFYjjOI5Ti/8Pp7UyPy6qLgcAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10acf9610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEKCAYAAAAIO8L1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAG6NJREFUeJzt3X+UXOV93/H3xwLHBlEUOVQCsbJiflRg2YWeRIYaxxs7\n",
       "gd3lFIiPA8VJTWzAOm34EcqpgUOPUU1aglu7GDilVMhEtDaY2IBFKi3IlC2kwWB6+CUjGdZGRRJi\n",
       "rQCywYYEh2//uM/C1XBnNDN3Zu7s7Od1zhzdH89z73ev9sx3732e+zyKCMzMzGq9o+oAzMysPzlB\n",
       "mJlZIScIMzMr5ARhZmaFnCDMzKyQE4SZmRVygrAZTdKEpDOrjsNsEDlB2EwX6dMySX8u6fIOx9Nq\n",
       "DCsl/fcOHesNSe/rxLHMwAnCbNCo6gBscDhBWF+T9G8kfatm29WSrsptWiLpryT9TNJdkt6TK/sX\n",
       "knZI2iXpf0s6Mm3/HPAp4POSXpb0nYJzXyfpP9Zs+46kP0nLF0nals67WdLHCo7xm5Kel6Tctk9I\n",
       "elTSCcAlwGkphkfS/v0lrZb0XDr+5ZLekfYdmn6OXZJ2Sro5bb8vHf6xdKzfb/4qm9UREf7407cf\n",
       "YCHwCrB/Wt8LmAKOTusTwCRwKPAu4F7gilz9PwL2BfYG/jPwSG7fjcAXG5z7I8CzufVfBX6RYvpH\n",
       "wLPAwrRvMfC+Osf5ATCSW78duCAtXwbcVFP+duA64N3AAcCDwOfSvpuBS9LyO4F/mqv3Rr0Y/PGn\n",
       "nY/vIKyvRcTzwP3A9F/EI8DfRMQj00WAr0XEZES8BtwKHJWr/+cR8fOIeB34d8A/lrRf7hSNHsn8\n",
       "FRCSPpLWPwn8dYrp74FfAd4vae+IeDYiflznODcBfwggaT5wPPCN3PnzdxcLgFGyBPJqROwErgL+\n",
       "eSryd2R3TIsi4u8i4q8bxG9WihOEzQRrSF+w6d+bavY/n1t+FZgLIGmOpD+TNCnpp8AzqcyvNXPS\n",
       "iAjgFuD0tOlTwNfTvkngT4CVwJSkmyUdWOdQXwf+maR9gFOB+yJiqk7Z95Ld7eyQ9JKkl4D/SnYn\n",
       "AfB5soTykKSNkj7TzM9i1g4nCJsJvgN8UNIy4ETSl3QTPgWcBHw8IvYHfj1tn/6LvZneTzcDn5T0\n",
       "XmA58O3pHRFxc0R8hOxLPYAriw4QEduA7wGfIEtw+V5Lb9QU3wr8LfCeiPjV9Nk/Ij6QjjUVEZ+L\n",
       "iEXACuC/uOeSdYsThPW9iHiV7Iv5G8CD6Qs3r95jorlkX7YvStoX+A81+6eAhl+uEfEo8DfADcB4\n",
       "RPwMQNLhkj4m6VfSOV4je+xUz03ARcAy4LaaGJZMN2JHxA7gbuArkvaT9A5Jh0j6rXTe35d0cKq7\n",
       "iywxvZE71iGNfh6zVjhB2EyxhuzLteidgahZnl6/Cfh/wHZgI/BATdnVwJHpUU7+S7vWN4CP8Va7\n",
       "AWTtD1cAO4EdZI+tLmlwjNvIGrJvT20l0/4i/fuCpIfT8qfJGqCfBF5MZRamfb8BfE/Sy2R3VudF\n",
       "xJa0byWwJv08n2wQi1lTlD1mLXEAaYSsEW0OcENEXFmz/w9467npy8C/jIjHm6lrNk3SELAZWBAR\n",
       "r1QdTzskPQ2siIj/VXUsZs0odQchaQ5wLVnPkiOB0yUdUVPsx8BvRcQHgcuB/9ZCXTPSOwAXAjfP\n",
       "4OTwCbJ2bycHmzH2Kll/OTA5fYsr6RbgZGDTdIGIeCBX/kHg4GbrmqW2gymyHkgjFYfTFkkTwFLg\n",
       "X1QcillLyiaIRWS9LqZtAz7UoPyZwLo269osFBE/J3VbnakiYrjqGMzaUTZBNN2AIem3gc8CH261\n",
       "rpmZ9V7ZBLEdGMqtD5HdCexG0geBVWTDDbzUYl0nEjOzNkREucEby4zTQZZgfgQsIeuW9yhwRE2Z\n",
       "xWRj5RzTat1ULqoej6QgppVVxzATYurXuByTY5oNcXXiu7PUHURE/FLSOcBdZF1VV0fEJkkr0v7r\n",
       "gS+QDXJ2XXoX6PWIWF6vbpl4zMxmomXS2BCctx+862V4bStcvTFi3Z5rdlfZR0xExHpgfc2263PL\n",
       "ZwFnNVvXzGw2WSaNHQtfXZWNSAzA2XDIMomqk4TfpG7PRNUBFJioOoA6JqoOoMBE1QEUmKg6gAIT\n",
       "VQdQYKLqAOqYaLfiEJyXTw4Aq+DQxXBu6ahKcoJoQ0RMVB1DrX6MCfozLsfUHMfUvDJx7ZfNY/I2\n",
       "c7P5QCrlBGFmVqGXs4Ee3+aVbOj6SjlBmJlVaCtcfXbW0/NNZ8GPnoVrqoppWunB+rpNUkTZvrxm\n",
       "Zn1smTS2GM6dC+9+BV59Fq4p20Ddie9OJwgzswHUie9OP2IyM7NCThBmZlbICcLMzAo5QZiZWSEn\n",
       "CDMzK+QEYWZmhZwgzMyskBOEmZkVcoIwM7NCThBmZlaodIKQNCJps6SnJV1UsH+ppAckvSbpwpp9\n",
       "WyQ9LukRSQ+VjcXMzDqn1IxykuYA1wK/A2wHvi9pbc3UoS+QTXxxSsEhAhiOiBfLxGFms0O/Ts05\n",
       "qMpOObocmIyILQCSbgFOBt5MEBGxE9gp6cQ6x/BAfGa2R/08NeegKvuIaRGwNbe+LW1rVgDflfSw\n",
       "pLNLxmJmA6yfp+YcVGXvIMqOFf7hiNgh6QBgg6TNEXF/bSFJK3OrE/067aCZdU8/T83ZDyQNA8Od\n",
       "PGbZBLEdGMqtD5HdRTQlInakf3dKup3skdXbEkRErCwXppnNdP08NWc/SH84T0yvS7qs7DHLPmJ6\n",
       "GDhM0hJJ7wROA9bWKbtbW4OkfSTtl5b3BY4HnigZj5kNqH6emnNQlZ5RTtIocBUwB1gdEVdIWgEQ\n",
       "EddLWgh8H/gHwBvAy8CRwD8EbkuH2Qv4ekRcUXB8zyhnZkB3puYcVJ5y1MzMCnnKUTMz6xonCDMz\n",
       "K+QEYWZmhZwgzMyskBOEmZkVcoIwM7NCThBmZlbICcLMzAo5QZiZWSEnCDMzK+QEYWZmhZwgzMys\n",
       "UNn5IMzMBornvX6LE4SZWeJ5r3fnR0xmZonnvd6dE4SZWeJ5r3dXOkFIGpG0WdLTki4q2L9U0gOS\n",
       "XpN0YSt1zcx6yfNe765UgpA0B7gWGCGbRvR0SUfUFHuB7PbsP7VR18ysZzzv9e7KNlIvByYjYguA\n",
       "pFuAk4FN0wUiYiewU9KJrdY1s96bzb14NkasWyYx5nmvgfIJYhGwNbe+DfhQD+qaWRe4Fw/TP+es\n",
       "+Fn3pGyCiF7UlbQytzoRERMlzmtmddTrxTOWPSb2l2YfkzQMDHfymGUTxHZgKLc+RHYn0NG6EbGy\n",
       "neDMrDXuxTNzpT+cJ6bXJV1W9phlezE9DBwmaYmkdwKnAWvrlFWJumbWA+7FY3mlEkRE/BI4B7gL\n",
       "eBL4ZkRskrRC0goASQslbQUuAP6tpGclza1Xt0w8ZlaOe/FYniLKNCN0n6SIiNq7DzPrkmXS2GL3\n",
       "4pnxOvHd6QRhZjaAOvHd6aE2zMyskBOEmZkVcoIwM7NCng/CzN5mNg+3YW9xgjCz3Xi4DZvmR0xm\n",
       "thtPmmPTnCDMbDcebsOmOUGY2W483IZNc4Iws920OtzGMmlsVBo/VZoYlcaXSWO9idS6zY3UZrab\n",
       "VibNcYP2YPNQG2bWtlFpfD2cULt9DMbXRYxWEZNlPNSGmVXKDdqDzQnCzNrmBu3B5gRhZm3z/BGD\n",
       "zW0QZlaK54/oT30xH4SkEeAqYA5wQ0RcWVDmamAU+AXwRxHxSNq+BfgZ8PfA6xGxvKCuE4SZWYs6\n",
       "8d1ZqpurpDnAtcDvANuB70tam586VFmf6EMj4jBJHwKuA45JuwMYjogXy8RhZmadV7YNYjkwGRFb\n",
       "IuJ14Bbg5JoyJwFrACLiQWCepAW5/b47MDPrQ2UTxCJga259W9rWbJkAvivpYUlnl4zFzMw6qOyb\n",
       "1M02YNS7SzguIp6TdACwQdLmiLj/bZWllbnViYiYaC1MM7PBJmkYGO7kMcsmiO3AUG59iOwOoVGZ\n",
       "g9M2IuK59O9OSbeTPbJ6W4KIiJUl4zQzG2jpD+eJ6XVJl5U9ZtlHTA8Dh0laIumdwGnA2poya4FP\n",
       "A0g6BtgVEVOS9pG0X9q+L3A88ETJeMwq5YHrbJCUuoOIiF9KOge4i6yb6+qI2CRpRdp/fUSskzQm\n",
       "aRL4OfCZVH0hcJuk6Ti+HhF3l4nHrEoeuM4GjV+UM+sQD1xn/cSD9Zn1EQ9cZ4PGCcKsQzxwnQ0a\n",
       "JwgbeL1qOPbAdTZoPKOcDbReNhy3MhOb2UzgRmobaG44ttnKjdRme+CGY7P2OUHYQHPDsVn7nCBs\n",
       "oLnh2Kx9boOwgecZz2w26osZ5brNCcLMrHVupDYzs65xgjAzs0JOEGZmVsgJwszMCnmoDZv1lklj\n",
       "Q3DefvCul+G1rXC1ezmZdeAOQtKIpM2SnpZ0UZ0yV6f9j0k6upW6Zt00PVbTejjhVvjoejjhWPiq\n",
       "Z4IzK5kgJM0BrgVGgCOB0yUdUVNmDDg0Ig4DPgdc12xds24bgvPyA/kBrIJDF8O5VcVk1i/K3kEs\n",
       "ByYjYktEvA7cApxcU+YkYA1ARDwIzJO0sMm6Zl3lsZrM6iubIBYBW3Pr29K2Zsoc1ERds67yWE1m\n",
       "9ZVNEM2+hu03oa0veawms/rK9mLaDgzl1ofI7gQalTk4ldm7iboASFqZW52IiIn2wjXbXauT/LjH\n",
       "k/UrScPAcEePWWYsJkl7AT8EPg48BzwEnB4Rm3JlxoBzImJM0jHAVRFxTDN1U32PxWR9oc7sdJMP\n",
       "wPlOEtZvKh+LKSJ+CZwD3AU8CXwzIjZJWiFpRSqzDvixpEngeuBfNapbJh6zbnKPJ5ttSr8oFxHr\n",
       "gfU1266vWT+n2bpm/co9nmy28VAbZk1yjyebbZwgzJrkHk8223jCILMWeHY6myk8o5yZmRWqvBeT\n",
       "mZkNLicIMzMr5ARhZmaFPGGQDQQPgWHWeU4QNuPVGQLjkGUSThJm7XMvJqtUJ/7yH5XG18MJtdvH\n",
       "YHxdxKjvLmw26sR3p+8grDKd+su/0RAYvrswa58bqa0ynRr8rtEQGB5gz6x9ThBWmTKD3y2Txkal\n",
       "8VOliZ/Dr30WduT3Tw+B4QH2zNrnR0xWmXYHvyt6bPQJeO54+L/z4JX8EBij0nntnMPMnCCsgW43\n",
       "7qbB7w7Jf9E3M/hd0WOj2+CgMXj81ojRTpzDzJwgrI5eNO62Ot3ntFYeG7V7DjMr0c1V0nzgm8B7\n",
       "gS3AqRGxq6DcCHAVMAe4ISKuTNtXAmcBO1PRSyJivKC+u7lWYE9dR6uIaVptbPcBdwOb4MVfwPfd\n",
       "jdWs+sH6LgY2RMThwD1pfTeS5gDXAiPAkcDpko5IuwP4SkQcnT5vSw5WnX5u3M3Py3Af2Zy1fwp8\n",
       "G+avhxOOha8uy+ZCN7MSyiSIk4A1aXkNcEpBmeXAZERsiYjXgVuAk3P7fWfQp/p59rSNEesegPPH\n",
       "YPxL8NK/r9nvbqxmnVEmQSyIiKm0PAUsKCizCNiaW9+Wtk07V9JjklZLmlciFuuwfp89bWPEunUR\n",
       "o/vA40X7++FOx2yma9hILWkDsLBg16X5lYgISUWNGY0aOK4DvpiWLwe+DJzZKB7rnZnSuNvPdzpm\n",
       "M13DBBERv1tvn6QpSQsj4nlJBwI/KSi2HRjKrQ+R3UUQEW+Wl3QDcGeDc63MrU5ExESjuK0zUjLo\n",
       "q4RQy91YzTKShoHhjh6zRC+mLwEvRMSVki4G5kXExTVl9gJ+CHwceA54CDg9IjZJOjAidqRyFwC/\n",
       "GRGfKjiPezFZQ54n2uztKp2TOnVzvRVYTK6bq6SDgFURcWIqN8pb3VxXR8QVaftNwFFkj6GeAVbk\n",
       "2jTy53GCMDNrUaUJolecIPbMw1mbWS0P922eLMfMusajuc5wHs7azLrFCWKG6+c3ns1sZnOCmOH8\n",
       "HoCZdYsTxAzXT2885yfxGZXGPR6S2czmRuoZrl/eeHZjudngcTdX64h+Hh7cbDaqerhvsze5sdxs\n",
       "8DhBWEe4sdxs8DhBWEf0U2O5mXWG2yCsYzxonln/8FhMZmZWyI3UZmbWNX4PwvbIo8WazU5OENaQ\n",
       "X4Azm738iMka8mixZrNX2wlC0nxJGyQ9JeluSfPqlPtamr/6iXbqW7X8ApzZ7FXmDuJiYENEHA7c\n",
       "k9aL3AiMlKhvFfILcGazV5kEcRKwJi2vAU4pKhQR9wMvtVvfquUX4MxmrzKN1AsiYiotTwELelzf\n",
       "eqBfRos1s95rmCAkbQAWFuy6NL8SESGp7Tfu9lRf0src6kRETLR7LmtdSgZOCGZ9TNIwMNzRY7b7\n",
       "JrWkzcBwRDwv6UDg3ohYWqfsEuDOiPhAq/X9JrWZWeuqfpN6LXBGWj4DuKPH9c3MrIvK3EHMB24F\n",
       "FgNbgFMjYpekg4BVEXFiKncz8FHgPcBPgC9ExI316hecx3cQZmYt8mB9ZmZWqOpHTGZmNsCcIMzM\n",
       "rJAThJmZFXKCMDOzQk4QZmZWyAnCzMwKOUGYmVkhJwgzMyvkBGFmZoWcIMzMrJAThJmZFXKCMDOz\n",
       "Qk4QZmZWyAnCzMwKOUGYmVkhJwgzMyvUdoKQNF/SBklPSbpb0rw65b4maUrSEzXbV0raJumR9Blp\n",
       "NxYzM+u8MncQFwMbIuJw4J60XuRGoOjLP4CvRMTR6TNeIhYzM+uwMgniJGBNWl4DnFJUKCLuB16q\n",
       "cwxPJWpm1qfKJIgFETGVlqeABW0c41xJj0laXe8RlZmZVWOvRjslbQAWFuy6NL8SESEpWjz3dcAX\n",
       "0/LlwJeBM+vEsTK3OhEREy2ey8xsoEkaBoY7esyIVr/X3wxmMzAcEc9LOhC4NyKW1im7BLgzIj7Q\n",
       "6n5JERF+FGVm1oJOfHeWecS0FjgjLZ8B3NFK5ZRUpv0e8ES9smZm1ntl7iDmA7cCi4EtwKkRsUvS\n",
       "QcCqiDgxlbsZ+CjwHuAnwBci4kZJNwFHkfVmegZYkWvTyJ/HdxBmZi3qxHdn2wmiV5wgzMxaV/Uj\n",
       "JjMzG2BOEGZmVsgJwszMCjlBmJlZIScIMzMr5ARhZmaFnCDMzKyQE4SZmRVygjAzs0JOEGZmVsgJ\n",
       "wszMCjlBmJlZIScIMzMr5ARhZmaFnCDMzKyQE4SZmRVqO0FImi9pg6SnJN0taV5BmSFJ90r6gaSN\n",
       "ks5rpb6ZmVWnzB3ExcCGiDgcuCet13oduCAi3g8cA/yxpKUt1Dczs4qUmZN6M/DRiJiStBCYiIil\n",
       "e6hzB3BNRNzTbH1POWpm1rqqpxxdEBFTaXkKWNCosKQlwNHAg+3UNzOz3tqr0U5JG4CFBbsuza9E\n",
       "REiqeysiaS7wLeD8iHildn8T9VfmViciYqJR3GZms42kYWC4o8cs+YhpOCKel3QgcG+dR0R7A38J\n",
       "rI+Iq9qo70dMZmYtqvoR01rgjLR8BnBHbQFJAlYDT+aTQ7P1e2GZNDYqjZ8qTYxK48uksSriMDPr\n",
       "N2XuIOYDtwKLgS3AqRGxS9JBwKqIOFHSccB9wOPA9IkuiYjxevULztO1O4hl0tix8NVVcOj0trNh\n",
       "8gE4f2PEum6c08ysFzrx3dl2guiVbiaIUWl8PZxQu30MxtdFjHbjnGZmvVD1I6YZbz94V9H2ufDu\n",
       "XsdiZtZvZnWCeBleK9r+Crza61jMzPrNrE4QW+Hqs2Eyv+0s+NGzcE1VMZmZ9YtZ3QYBWUP1Yjh3\n",
       "Lrz7FXj1WbjGDdRmNtO5kdrMzAq5kdrMzLrGCcLMzAo5QZiZWSEnCDMzK+QEYWZmhZwgzMyskBOE\n",
       "mZkVcoIwM7NCThBmZlbICcLMzAq1nSAkzZe0QdJTku6WNK+gzJCkeyX9QNJGSefl9q2UtE3SI+kz\n",
       "0m4sZmbWeWXuIC4GNkTE4cA9ab3W68AFEfF+4BjgjyVNzzsdwFci4uj0GS8RS0+lycH7Sj/GBP0Z\n",
       "l2NqjmNqXr/GVVaZBHESsCYtrwFOqS0QEc9HxKNp+RVgE7AoV2SmDsI3XHUABYarDqCO4aoDKDBc\n",
       "dQAFhqsOoMBw1QEUGK46gDqGqw6gG8okiAURMZWWp4AFjQpLWgIcDTyY23yupMckrS56RGVmZtVp\n",
       "mCBSG8MTBZ+T8uUiGzO87rjhkuYC3wLOT3cSANcBvw4cBewAvlzmBzEzs85qez4ISZuB4Yh4XtKB\n",
       "wL0RsbSg3N7AXwLrI+KqOsdaAtwZER8o2NffE1aYmfWpsvNB7FWi7lrgDODK9O8dtQUkCVgNPFmb\n",
       "HCQdGBE70urvAU8UncSTBZmZVaPMHcR84FZgMbAFODUidkk6CFgVESdKOg64D3ictx5BXRIR45Ju\n",
       "Inu8FMAzwIpcm4aZmVWs76ccNTOzavTFm9QdeOluj/W7EVMq9zVJU5KeqNne8RcBOxBTlddpRNJm\n",
       "SU9Luii3vWPXqd45aspcnfY/JunoVupWFNcWSY+na/NQr2KStFTSA5Jek3Rhqz9PBTFVdZ3+IP2f\n",
       "PS7p/0j6YLN1K4qptesUEZV/gC8Bn0/LFwF/VlBmIXBUWp4L/BBY2mz9bsSU9n2ErPvuEzXbLwP+\n",
       "da+v0x5iquQ6AXOASWAJsDfwKHBEJ69To3PkyowB69Lyh4DvNVu3irjS+jPA/A7/HjUT0wHAbwB/\n",
       "ClzYSt1ex1TxdToW2D8tj3T7d6pMTO1cp764g6D8S3d7rN+NmFIs9wMv1TlGpxvYy8ZU1XVaDkxG\n",
       "xJaIeB24BTg5t78T12lP59gt1oh4EJgnaWGTdXsdV/69ok7/Hu0xpojYGREPk42G0FLdCmKaVsV1\n",
       "eiAifppWHwQObrZuBTFNa/o69UuCKPvSXUv1uxFTHZ1+EbBsTFVdp0XA1tz6NnZ/o74T12lP52hU\n",
       "5qAm6rarTFyQdeL4rqSHJZ3dw5i6Ubebx+2H63QmsK7Nur2ICVq8TmW6ubZE0gayx0S1Ls2vRESo\n",
       "wbsPKn7prun63YipjuuAL6bly8leBDyz4pjaqt+BmBqdp63r1OI58nrdbbpsXMdFxHOSDgA2SNqc\n",
       "7hB7EVOn63bzuB+OiB1VXSdJvw18Fvhwq3VbVCYmaPE69SxBRMTv1tunrEF1Ybz10t1P6pTbG/g2\n",
       "8D8iIv/eRVP1uxFTg2O/WV7SDcCdVcdEdddpOzCUWx8i+8un7evUyjkalDk4ldm7ibrtajeu7QAR\n",
       "8Vz6d6ek28keMZT94msmpm7U7dpxI71TVcV1So3Aq4CRiHiplbo9jqnl69Qvj5imX7qDNl66a6Z+\n",
       "N2JqJH1ZTqv7ImAvY+pA/XaP+TBwmKQlkt4JnJbqdfI61T1HTayfTuc9BtiVHo81U7ddbcclaR9J\n",
       "+6Xt+wLH05nfo1Z+3to7m25dq7ZjqvI6SVoM3Ab8YURMtvnz9CSmtq5T2Vb1TnyA+cB3gaeAu4F5\n",
       "aftBwP9My8cBb5C12j+SPiON6nc7prR+M/Ac8LdkzwY/k7bfRPaC4GNkX5oL+iCmKq/TKFnPs0my\n",
       "lyWnt3fsOhWdA1hB9hLmdJlr0/7HgH+yp/g69PvdVlzA+9Lv+6PAxk7GtaeYyB4pbgV+Stbh4Vlg\n",
       "bjevVbsxVXydbgBe4K3vpIe6/TvVbkztXCe/KGdmZoX65RGTmZn1GScIMzMr5ARhZmaFnCDMzKyQ\n",
       "E4SZmRVygjAzs0JOEGZmVsgJwszMCv1/gjVnCTU6RPwAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10c7b8910>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('R2 :', 0.89506386491458345)\n",
      "('Gene with Strongest Coefficient :', 250)\n",
      "('Lambda_ResultOfCV :', 0)\n"
     ]
    }
   ],
   "source": [
    "lm = linear_model.RidgeCV(alphas=[0])  ### CHANGE THIS LINE ###\n",
    "lm.n_jobs=-1\n",
    "lm.fit(Xtrain, ytrain);\n",
    "\n",
    "printOutput(lm, Xtest, ytest)   ### PROVIDE THE OUTPUT ###"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### LASSO regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "lm = linear_model.LassoCV() ### CHANGE THIS LINE ###\n",
    "lm.n_jobs=-1\n",
    "lm.max_iter=10000\n",
    "lm.fit(Xtrain, ytrain)\n",
    "\n",
    "printOutput(lm, Xtest, ytest)   ### PROVIDE THE OUTPUT ###\n",
    "\n",
    "print('Regression Coefficient of Gene 5954 : ', lm.coef_[5954]) ### CHANGE THIS LINE ### ### PROVIDE THE OUTPUT ###"
   ]
  }
 ],
 "metadata": {
  "annotations": {
   "author": "",
   "categories": [
    "intelligent-systems-1-2015"
   ],
   "date": "2015-05-07",
   "location": "Beginning of next lecture",
   "parent": "IS_SS2015",
   "submission_date": "2015-05-21",
   "subtitle": "Exercise Sheet 4, Regularized Linear Regression",
   "tags": [
    "IntelligenSystems",
    "Course"
   ],
   "title": "Intelligent Systems 1 - Summer Semester 2015"
  },
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}