{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "preamble": true
   },
   "source": [
    "(Defining latex commands: not to be shown...)\n",
    "$$\n",
    "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
    "\\DeclareMathOperator{\\minimize}{minimize}\n",
    "\\newcommand{\\real}{\\mathbb{R}}\n",
    "\\newcommand{\\normal}{\\mathcal{N}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Exercise Sheet 3\n",
    "Maximus Mutschler & Jan-Peter Hohloch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Gaussian Algebra (25 Points)\n",
    "\n",
    "Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
    "\n",
    "\\begin{equation}\n",
    "   \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Proof\n",
    "\\begin{align*}\n",
    "   f(x) &= \\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\cdot e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}}\\cdot \\frac{1}{\\sqrt{2\\pi s^2}}\\cdot e^{\\frac{-(x-m)^2}{2s^2}}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\alpha}\\\\\n",
    "   \\text{where }\\alpha &= \\frac{s^2(x-\\mu)^2+\\sigma^2(x-m)^2}{2\\sigma^2 s^2}\\\\\n",
    "   &=\\frac{(s^2+\\sigma^2)x^2-2(s^2\\mu+\\sigma^2 m)x+s^2\\mu^2+\\sigma^2m^2}{2\\sigma^2s^2}\\\\\n",
    "   &=\\frac{x^2-2\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}x+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}+\\underbrace{\\frac{\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}_\\beta\\\\\n",
    "   \\beta&= \\frac{\\frac{s^4\\mu^2+s^2\\sigma^2\\mu^2+s^2\\sigma^2m^2+\\sigma^4m^2-\\left(s^4\\mu^2+2s^2\\sigma^2\\mu m+\\sigma^4m^2\\right)}{\\left(s^2+\\sigma^2\\right)^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{s^2\\sigma^2\\left(\\mu^2-2\\mu m+m^2\\right)}{2\\sigma^2s^2\\left(s^2+\\sigma^2\\right)}\\\\\n",
    "   &=\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}\\\\\n",
    "   \\Rightarrow f(x)&=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-(\\alpha-\\beta)-\\beta}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}\\cdot e^{-\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}}\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi(s^2+\\sigma^2)}\\sqrt{2\\pi\\frac{\\sigma^2s^2}{s^2+\\sigma^2}}}{2\\pi s\\sigma}\\normal\\left[x;\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2},\\frac{\\sigma^2s^2}{s^2+\\sigma^2}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi\\cdot 2\\pi\\sigma^2s^2}}{2\\pi s\\sigma}\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\hspace{2cm}\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[m,\\mu,\\sigma^2+s^2\\right]\\hspace{2cm} q.e.d.\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
    "\n",
    "Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
    "\n",
    "\n",
    "Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Linear regression (50 Points)\n",
    "\n",
    "In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
    "\n",
    "1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
    "2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Loading the required packages\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.html.widgets import interact\n",
    "import csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def wRidge(X,y,lamb):\n",
    "    # Change the following line and implement the ridge regression estimator wRidge\n",
    "    return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Read ex1_train.csv and assign the first column and \n",
    "# second column to variables x and y respectively.\n",
    "\n",
    "a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
    "x= a[:,0]\n",
    "y= a[:,1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4) Plot the training data with appropriate labels on each axes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x7ff4c7645a20>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEZCAYAAACTsIJzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAFmxJREFUeJzt3XuQZGd93vHvo1tWWIsEBmNZ7CIsQYJYQICREZcwQBzE\n",
       "xoHIGDCOMRi05Spgl4oJZQQJkkM53BLC7lJ2EgHiZm4yBiOQZGSsNUpUXAQISxFClsKyKwwSV7Eb\n",
       "dhNdfvnj9LJ9RnPpnpnu0z3z/VRNTZ/3dJ/+zZZ0nn7f97ynU1VIknTIEV0XIEmaLAaDJKnFYJAk\n",
       "tRgMkqQWg0GS1GIwSJJaDAapA0nek+QNXdchzcVg0KqVZHeSp4/hfc5P8v4hX1a9n0GOvyvJS4ev\n",
       "TFoag0Gr2cAn345kwOdN8t+gVchg0JqQ5MVJ/keStyb5YZL/neSsvv27krwxyReS3J7kE0nu09s3\n",
       "k2TvrOPtTvL03jHOBZ6fZF+Sr87z/o9O8pUkP0nyYWBd3777JPlUktt6tV2c5KTevj8Gngy8o3f8\n",
       "Hb327Un29Gq9OsmTVvifTGuYwaC15AzgBuDngbcA75q1/4XA7wEnAncCOxY4VgFVVZcB/xH4cFWt\n",
       "r6pHz35ikmOATwDvBe4DXAQ8h8M9gfRq2dj7OQC8g+YNXgdcCby8d/xtvdd8EXhU73gfBC7qvY+0\n",
       "bAaD1pJvVdW7qrlB2PuAE5P8Qm9fAe+rquur6qfAvweel2SQ4Z6w8LDQ44Gjqmp7Vd1VVR8DvnRo\n",
       "Z1X9sKo+XlUHq2o/TdA8ZY73oO81f1ZVP6qqu6vqbcA/Av7xALVKizIYtJZ899CD3skf4Li+/f3D\n",
       "RXuAo4H7rcD7/hLw7Vlt36J3sk9yryT/rTc8dTvwt8Dxs0KpNc+Q5N8muT7Jj5P8CDh+hWqVDAap\n",
       "z8ZZj+8Avg/8H+Beh3YkORK4f99zF5sc/g5w0qy2B/W97lXAQ4Ezqup4mt5Cfy9kdig8GXg18Nyq\n",
       "OqGq7gPczuCT2dKCDAapEeB3kjwsyb2A/wBc1Bt2uhFYl2RzkqOBf0czdHPId4GTFxh2ugq4M8m2\n",
       "JEcn+Q3gcX37j6OZV7g9yX2B82a9/lbglL7t9TRzIN9PckyS1wP3XsofLc3FYNBaMdelqzXr8fuB\n",
       "99B8wj8G2AZQVbcDLwPeCdwC7Kc97HRR7/cPklx9jzeuugP4DeDFwA+A5wEf63vK24FjaXonVwGX\n",
       "zqptO/CbvSuW3g5c1vu5EdhNEyp7FvzrpSGkyy/qSfJu4F8At1XVI3pt5wPnAN/rPe3c3pUf0sgk\n",
       "uQJ4f1W9u+tapK513WO4EDhrVlsBb6uqR/d+DAWNi2P0Eh0HQ1VdCfxojl3+D6ouuMJYovsew3y2\n",
       "JvlaknclOaHrYrT6VdVTHUaSGpMYDH8KPBg4nWYS8D93W44krS1HdV3AbFV126HHSd4JXDz7OUns\n",
       "8kvSElTVokP1ExcMSU6squ/0Ns8Grp3reYP8cZMqyflVdX7XdSyV9XfL+rszzbXD4B+qOw2GJB+i\n",
       "WeV5v97dK88DZpKcTjMR+E3g9zssUZLWnE6DoapeMEezE4CS1KFJnHxeC3Z1XcAy7eq6gGXa1XUB\n",
       "y7Sr6wKWaVfXBSzDrq4LGIdOVz4vVZKa5jkGSerCoOfOiZt8liStrGTTZtiwbfFnNgwGSVrFmlA4\n",
       "cztccOqgN5VwjkGSVrUN25pQGJzBIEmr2vp1w77CoSRJq8LhcfT162DfQdi7o+q6S7quq3v7Dg77\n",
       "CoNB0tRrj6MfsuWUZBOGw94dsOWUYYaTvFxV0liM8hN98szL4NJn3HPP5suqLnnmSrzHNGv+7Tdu\n",
       "hUvP8nJVSRNh9J/o5xtHP+7Y5R97+oeperVeMhX3SpK0Vsx1ZcwFp8LmrcAKnGDnG0fff2C5R16L\n",
       "w1RelSRpDEb7ib43jn5Tu+2cm2HPzuUfe75Q27h1+ceeTPYYJI3B6D7RQzNUkmyi6YEcd2xz3D07\n",
       "p2GYahIZDJLGYK4rY1bqE33j0Dj6Sh3vsNGG2iQyGCSN3Gg/0Y/a6ENt0ni5qiQt4vDlntMWam2D\n",
       "njsNBklaIwY9d3pVkiSpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1\n",
       "GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqSWToMhybuT3Jrk2r62+ya5\n",
       "PMmNST6T5IQua5SktabrHsOFwFmz2l4DXF5VDwU+29uWJI1Jp8FQVVcCP5rV/Czgvb3H7wX+1ViL\n",
       "kqQ1rusew1weUFW39h7fCjygy2Ikaa05qusCFlJVlaTm2pfk/L7NXVW1ayxFSdKUSDIDzAz9uqo5\n",
       "z7tjk+Rk4OKqekRv+wZgpqq+m+RE4Iqq+iezXlNVlbEXK0lTbNBz5yQOJX0SeFHv8YuAT3RYiySt\n",
       "OZ32GJJ8CHgKcD+a+YTXA38JfBTYCOwGnldVP571OnsMkjSkQc+dnQ8lLYXBIEnDm+ahJElShwwG\n",
       "SVKLwSBJajEYJEktE73ATdJhyabNsGEbrF8H+w7C3h1V113SdV1afQwGaQo0oXDmdrjg1MOtW05J\n",
       "NmE4aKU5lCRNhQ3b2qEAzfbGrd3Uo9XMYJCmwvp1c7cfd+x469BaYDBIU2Hfwbnb9x8Ybx1aCwwG\n",
       "aSrs3QFbbmq3nXMz7NnZTT1azbwlhjQlmgnojVub4aP9B2DPTieeNQzvlSRJavFeSZKkJTEYJEkt\n",
       "BoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqQWg0GS1GIw\n",
       "SJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWo7quoD5JNkN/AS4C7ijqs7o\n",
       "tiJJWhsmNhiAAmaq6oddFyJJa8mkDyWl6wIkaa2Z5GAo4K+TXJ1ky+ydyTMvSzZt7qAuSVrVJnko\n",
       "6YlV9Z0k9wcuT3JDVV15ePelz4AtpySbqLruks6qlKRVZmKDoaq+0/v9vSQfB84A+oLhfOCkU2Hf\n",
       "9iQ/rapdHZQpSRMryQwwM/TrqmqxA/98Vf1gaWUtTZJ7AUdW1b4kPwd8BvijqvpMb381I00Az/vb\n",
       "qo/OjLM+SZpGSaqqFp27HaTH8Pkk1wAXApfWYkmyMh4AfDwJNDX+2aFQuKf9B8ZQjyStGYP0GI4A\n",
       "/hnwEuBxwEeBC6vqxtGXN29NvR7DOTfD57c5xyBJixu0x7BoMMw66NOADwA/B1wDnFtVVy25yiVq\n",
       "guGZl8GenYaCJA1mxYIhyf2Afw38LnAr8E7gYuBRwJ9X1cnLrnZIg/5xkqTDVnKO4SqaXsKzq+qW\n",
       "vvark/zXpRYoSZpMA80xVNXdY6pnIPYYJGl4g547F135PGmhIEkarUm+JYYkqQOLBkOSJ83R9sTR\n",
       "lCNJ6togPYadc7S9Y6ULkSRNhnmvSkpyJvAE4P5J/oDDt8Bej0NQkrRqLXS56jE0IXBk7/chPwF+\n",
       "c5RFSZK6M8jlqg+qqm+NqZ6BeLmqJA1vJVc+XzFHc1XV05Za3HIZDJI0vJVc+fzqvsfrgOcAdy61\n",
       "MEnSZBvqJno/e1Hypap63AjqGfT97TFI0pBWrMeQ5L59m0cAvwLcexm1SZIm2CBDSV/h8Nel3Qns\n",
       "Bl46qoIkSd1a0lBS1xxKkqThreRQ0rHAy4An0fQcrgT+tKoOLrtKSdLEGeRy1YtoFrV9gGb1828D\n",
       "x1fVc0df3rw12WOQpCGt5DqG66vqtMXaxslgkKThrdj3MQBf6d036dCBHw98eTnFSZIm1yA9hhuA\n",
       "hwJ7aeYYNgLfoLlCqarqkaMuco6a7DFI0pBWcuXzMzh8Z9VDao42SdIqMEiP4f1V9cLF2sbJHoMk\n",
       "DW8l5xg2zTrwUcBjl1qYJGmyzRsMSV6bZB/wiCT7Dv0AtwGfHFuFkqSxGmQo6U1V9Zox1TMQh5Ik\n",
       "aXgruY7hKRy+V9LPVNXnll7e8hgMkjS8lQyGT3E4GNYBZwBf9ot6JGm6rNjlqlX167MOvAHYvoza\n",
       "JEkTbJCrkma7BXjYShciSZoMg9xddWff5hHA6XhLDElatQZZ+fxlDs8x3AV8sKr+5+hKkiR1aZDJ\n",
       "52OBU2nC4aZJ+B4GJ58laXjLXvmc5Ogkb6G5ed57gfcBtyR5a5KjV65USdIkWWjy+a3AfYEHV9Vj\n",
       "quoxwC8DJwD/aRzFSZLGb96hpCQ3AQ+tqrtntR8JfKOqTh1ZUclZwNuBI4F3VtWbZ+13KEmShrQS\n",
       "N9G7e3YoAFTVXcA92ldKL3jeAZwFnAa8IImXx0rSmCwUDF9P8qLZjUleCNwwupI4g2aSe3dV3QF8\n",
       "GHj2CN9Pa0SyaXPyzMuS5+1qfm/a3HVN0iRa6HLVlwN/keQlHF638FjgXsDZI6zpJJoJ70NuAX51\n",
       "hO+nNaAJgTO3wwV9Q6BbTkk2UXXdJd1VJk2eeYOhqm5J8qvA04CH01yu+umq+uyIa1r4+llpSTZs\n",
       "a4cCNNubtwIGg9RnwQVu1cxMf7b3My7fBjb0bW+g6TW0JDm/b3NXVe0abVmabuvXzd1+3LHjrUMa\n",
       "nyQzwMywrxtk5fO4XQ08JMnJwD8AzwdeMPtJVXX+WKvSlNs3z8LM/QfGW4c0Pr0PzLsObSc5b5DX\n",
       "LeUmeiNVVXcCrwD+Crge+EhVfb3bqjT99u6ALTe12865GfbsnPv50tq16C0xJpHrGLQUzQT0xq3N\n",
       "8NH+A7BnpxPPWktW7It6JpHBIEnDW4kFbpKkNWgSJ5870ww1bNjWXMGy7yDs3eFQg6S1xmDocQGU\n",
       "JDUcSvqZ+RZAbdzaTT2S1A2D4WdcACVJYDD0cQGUJIHB0McFUJIErmOYdVwXQElavVzgJklqcYGb\n",
       "JGlJDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMk\n",
       "qcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWo5qusCpEOSTZthwzZYvw72HYS9O6quu6TruqS1\n",
       "xmDQRGhC4cztcMGph1u3nJJswnCQxsuhJE2IDdvaoQDN9sat3dQjrV0GgybE+nVztx937HjrkGQw\n",
       "aELsOzh3+/4D461DksGgCbF3B2y5qd12zs2wZ2c39UhrV6qq6xqGlqSqKl3XoZXVTEBv3NoMH+0/\n",
       "AHt2OvEsrZxBz50TFwxJzgfOAb7Xazq3qi6b9RyDQZKGNOi5cxIvVy3gbVX1tq4LkaS1aFLnGOwN\n",
       "SFJHJjUYtib5WpJ3JTmh62IkaS3pZI4hyeXAL86x63XA5zk8v/AG4MSqeums1xfwR31Nu6pq1whK\n",
       "laSplWQGmOlrOm8qJ5/7JTkZuLiqHjGr3clnSRrSoOfOiRtKSnJi3+bZwLVd1SJJa9EkXpX05iSn\n",
       "01yd9E3g9zuuR5LWlIkeSpqPQ0mSNLypHUqSJHXLYJAktRgMkqQWg0GS1GIwSJJaDAZJUovBIElq\n",
       "MRgkSS2TuPJZE6r5hrUN22D9uuY7mvfu8BvWpNXHYNBAmlA4cztccOrh1i2nJJswHKTVxaEkDWjD\n",
       "tnYoQLO9cWs39UgaFYNBA1q/bu72444dbx2SRs1g0ID2HZy7ff+B8dYhadQMBg1o7w7YclO77Zyb\n",
       "Yc/ObuqRNCredlsDayagN25tho/2H4A9O514lqbHoOdOg0GS1gi/j0GStCQGgySpxWCQJLUYDJKk\n",
       "FoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWox\n",
       "GCRJLQaDJKnFYJAktXQSDEmem+R/JbkryWNm7Ts3yd8nuSHJP++iPklay7rqMVwLnA18rr8xyWnA\n",
       "84HTgLOAP0my6no1SWa6rmE5rL9b1t+daa59GJ2cdKvqhqq6cY5dzwY+VFV3VNVu4CbgjLEWNx4z\n",
       "XRewTDNdF7BMM10XsEwzXRewTDNdF7AMM10XMA6T9mn8l4Bb+rZvAU7qqBZJWpOOGtWBk1wO/OIc\n",
       "u15bVRcPcahaoZIkSQNIVXfn3SRXAK+qqq/0tl8DUFVv6m1fBpxXVV+Y9TrDQpKWoKqy2HNG1mMY\n",
       "Qn+RnwQ+mORtNENIDwG+OPsFg/xhkqSl6epy1bOT7AUeD3w6yaUAVXU98FHgeuBS4GXVZZdGktag\n",
       "ToeSJEmTZ9KuShpYkjck+VqSa5J8NsmGrmsaRpK3Jvl672/4iyTHd13TMBZapDipkpzVWzj590n+\n",
       "sOt6hpXk3UluTXJt17UMK8mGJFf0/pu5Lsm2rmsaRpJ1Sb7QO99cn+SNXde0FEmOTPLVJAteADS1\n",
       "wQC8paoeVVWnA58Azuu6oCF9Bnh4VT0KuBE4t+N6hjXnIsVJleRI4B00CydPA16Q5GHdVjW0C2nq\n",
       "n0Z3AP+mqh5OM4T88mn696+qg8BTe+ebRwJPTfKkjstailfSDNUvOFQ0tcFQVfv6No8Dvt9VLUtR\n",
       "VZdX1d29zS8AD+yynmEtsEhxUp0B3FRVu6vqDuDDNAsqp0ZVXQn8qOs6lqKqvltV1/Qe7we+TrNu\n",
       "aWpU1U97D48BjgR+2GE5Q0vyQGAz8E7aF/3cw9QGA0CSP06yB3gR8Kau61mGlwCXdF3EKncSsLdv\n",
       "28WTHUlyMvBomg9EUyPJEUmuAW4FruhdLDNN/gvwauDuxZ440cGQ5PIk187x8y8Bqup1VbUReA/N\n",
       "Hz1RFqu/95zXAf+vqj7YYalzGqT+KeJVFhMgyXHAnwOv7PUcpkZV3d0bSnog8E+n6b5JSX4duK2q\n",
       "vsoivQWYjHUM86qqXxvwqR9kAj9xL1Z/khfTdO2ePpaChjTEv/80+DbQf4HCBtq3X9GIJTka+Bjw\n",
       "gar6RNf1LFVV3Z7k08CvALs6LmdQTwCelWQzsA64d5L3VdXvzvXkie4xLCTJQ/o2nw18tataliLJ\n",
       "WTTdumf3Jram2TQsOLwaeEiSk5McQ3MX3092XNOakSTAu4Drq+rtXdczrCT3S3JC7/GxwK8xReec\n",
       "qnptVW2oqgcDvwX8zXyhAFMcDMAbe8Ma19Dc8fBVHdczrJ00k+aX9y4f+5OuCxrGfIsUJ1VV3Qm8\n",
       "AvgrmqsyPlJVX++2quEk+RBwFfDQJHuT/F7XNQ3hicDv0FzN89XezzRdYXUi8De9880XgIur6rMd\n",
       "17QcCw6tusBNktQyzT0GSdIIGAySpBaDQZLUYjBIkloMBklSi8EgSWoxGKRZkqz4rRqSPCjJC1b6\n",
       "uNIoGAzSPY1icc+Dgd8ewXGlFWcwSPNIMpNkV5KLel+q9IG+fbuTvDnJ3/W+wOWUXvt7kjyn73mH\n",
       "bg//JuDJvRW/r5z1Pmcn+eve4xOTfCPJL4z+L5TmZjBICzud5stNTgN+OckTeu0F/LiqHknzBUBv\n",
       "72ufyx8CV1bVo6tqe/+Oqvo48J0krwD+O/D6qrpthf8OaWAGg7SwL1bVP1Rz75hrgJP79n2o9/vD\n",
       "wJmLHGexGw1upfkWv4NV9ZGlFCqtFINBWtj/7Xt8F/Pfqv5QT+FOev9fJTmC5tu+BrGhd/wH9O5E\n",
       "KnXGYJCW7vl9v6/qPd4NPLb3+FnA0b3H+4D1cx0kyVE0t6T+LeAG4A9GUKs0sIn+oh6pIzXP49nu\n",
       "k+RrwEHg0KWoFwB/2bs982XAoUtfvwbc1Wu/cNY8w7nA56rqqiR/B3wpyaeq6hsr8cdIw/K229IS\n",
       "JPkm8NiqmqovhJcG4VCStDR+otKqZY9BktRij0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySp5f8D\n",
       "oJt+CXUlyycAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff4cc0bd240>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the input data here\n",
    "plt.plot(x,y,'o')\n",
    "plt.title('Input data')\n",
    "plt.ylabel('Output y')\n",
    "plt.xlabel('Input x')   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
    "Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def construct_poly(x,d):\n",
    "    ## Implement a method which given an array of size N, \n",
    "    ## returns an array of dimension (N,d)\n",
    "    poly=np.empty((len(x),1), float)\n",
    "    xr = np.reshape(x,(len(x),1))\n",
    "\n",
    "    for i in range(d-1):\n",
    "        poly = np.append(poly, (xr**i), axis=1)\n",
    "    #print poly \n",
    "    #print np.shape(poly)\n",
    "    return poly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def MSE(y_predict,y_true):\n",
    "    ## Implement mean squared error for a given input y and its predictions.\n",
    "    return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHGW5/vHvTVjSCCFgHAj7SABxREE5gCLHsKghKovK\n",
       "5sImwhFIOP6MJwIqAQWNCSiJqPyURUEQkM0gBFmSA6JEUOBwCCoJiYYtDQk7HQjJc/6oGqczzGSm\n",
       "p7unuqvvz3X1lemq6qpn5oJ7at566i1FBGZmli9rZF2AmZnVnsPdzCyHHO5mZjnkcDczyyGHu5lZ\n",
       "DjnczcxyyOFuZpZDDncbVJIWSnpN0lu7Lb9f0kpJW6bvN5d0jaRnJD0v6SFJR6brtk63fanb6+Be\n",
       "jnmIpD9IekXSrD7qO7XbPl+VtELSRun6YZIuS+t6Jv16/bLPD5H0bUlPSHpR0l8kbZCuW0fS99N1\n",
       "SyWdL2nNss9uLmmGpCWSnpI0XdKQHmr8Zvr9793/n7y1Goe7DbYAHgMO71wgaUegkK7rdCnwD2BL\n",
       "YCPg88DibvvaICLWL3td3csxlwDnAt/ts7iIs8v3CUwGZkXE0nSTScAIoB3YBtg4XdbpDGB3YPeI\n",
       "GAZ8DliWrvsa8F6gA9gu/frrZZ+dBjwLjAR2Aj4EnFBen6RtgE8DT/b1vVhrc7hbFi4Djih7fyTw\n",
       "C0Bly3YBLomIUkSsjIgHImLmQA4WEbdHxK+Bpyr5nCSltf28bHEHcH1EvBwRLwLXp8uQtCFwMvDF\n",
       "iFiUHntuRLyWfvbjwPSIeD4iniUJ82O67fvKiHg9IhYDMzv3XeaHwERgeSXfi7Ueh7tl4R5gmKR3\n",
       "pMMOh5IEfvdtfiTp0M6hmh6ol+W1sifwNuCasmW3AJ+SNDwN808BN6XrdgTeAA5Oh1X+JmmVM+9u\n",
       "Na8BbF42rHML8BlJBUmbAfsBN//rg8mw07KIuBmzPjjcLSuXkpy9fxiYCzzRbf3BwF3AN4DH0jH5\n",
       "Xbpt86yk58pe29e4xiOBqyPi1bJl56f/LiEZQlkO/DhdtjmwAbAtsDXJ8MkkSfum62cCJ0saIWkT\n",
       "YDzJUNS66fpJwLuAF4FFwL0RcQNA+gvgLJK/DMz65HC3LARJuH+WnodkSIcuTomId5GMaz9AMgRS\n",
       "7q0RsWHZ62+1KlDSuiTh/PNuq34J/A1YDxhGcv2g86+OUvrvmRHxWkQ8BPwKGJsuPwu4P/1efg9c\n",
       "B7wREYvTIaBbgKtJwn4EsJGkyelnJwGXRsQ/y8uswbdqOeVwt0ykIfUYydDDtX1suwQ4B9g0HQoZ\n",
       "8GEr2PYgYElE/He35WOAC9JrAa8AF9AV3v+zuuNGxLKIGBcRm0fEKGApcF+6zQjgfcAPI2J5egH3\n",
       "krJ97w2MT4d7ngK2AK6S9NUKvidrIQ53y9IXgL0jotR9haTJkjokrZkOSXwJeDQinivfrD8HkbSG\n",
       "pKHAWsAaaUviWn18rPMviu7+B/iipKGSCsBxwIMAETGfZCjpNElrS9qB5HrCjWkdm6YvSdqdpFPm\n",
       "9HS/z5Jc8P1S2k45PK3hwXT9PiQXV99D0knzZHrsH/XnZ2Ctx+FumYmIxyLiL+WLyr4ukAxbPAfM\n",
       "JzlT3b/bLp7v1pP+n70c6gjgVZIg3JNk+OSCzpXpZ/coe78ZMJqew/0okjbGJ4DHScbWjyxbfziw\n",
       "FcmY/I3A1yOis7d+G+Bu4GXgYmBiRNyW/iwC+CTwCZKgfxR4Dfhyun5pRBTT12JgBfBc+teD2Zuo\n",
       "2od1SLoI+BhQjIgd02WTgGOBZ9LNThloG5uZmVWuFmfuF5OMQ5YL4NyI2Dl9OdjNzAZR1eEeEXeR\n",
       "/Oncna/km5llpJ5j7uMkPSjpwvTikJmZDZJ6hfuPSebe2ImkA+CcOh3HzMx6sGbfm1QuIoqdX0v6\n",
       "GTCj+zaSqruSa2bWoiKiz2HvuoS7pJER0TlJ00HAQz1t158CG5WkSRExKes6Bsr1Z8v1Z6eZa4f+\n",
       "nxhXHe6SriCZmnSEpEUkN2WMlrQTSdfMAuD4ao9jZmb9V3W4R8ThPSy+qNr9mpnZwPkO1YGbnXUB\n",
       "VZqddQFVmp11AVWanXUBVZqddQFVmJ11AYOh6jtUB3xgKZp5zN3MLAv9zc66XFA1M7Paam/X2LY2\n",
       "xvd3ew/LmJk1uPZ2je3o4LzJk/lofz/jcDcza3BtbYyfMIFRlXzGwzJmZhmSWIPkqV4b9fbaZZeO\n",
       "neDhivbrcDezhiAxZMMNJ5w9cuTdxxYKy3jttXj1n//83J0vvjjhLpJpTDpfT0ewPNtq30xiTWA4\n",
       "qwnpXl7DgVdInszV0+uJ5ctfeYbkcZP95nA3s0xJDAEOWXfdqyfvvPOVb/vGNx4fmq7a6OyzF425\n",
       "5551N3zppRNWACPT19skXmDVwH+K5OlUqyyL4E1P+epHPWsDG1J5SK8PvEDvIT0fuLeH5c/39cuq\n",
       "vX3hI1Oncl4lQzNuhTSzfuvs2CgUGFoqsaxYZNqCBXHTQPaVhvphJI8bXLLzzpusc+65i3fpvt3E\n",
       "icycMyf26/a5EXSF/Uhg027vRwKbAMt48y+BJax+GGQoyTTmvYX0kl6WvxDByoH8LPoj/dmP+9Of\n",
       "GONWSDOrmc6OjfKzx6lT2aa9XVQS8OnwxWHAN4AicBJwx7Bhi2f1tH2hQKH8fQQrgMXp64HVHEck\n",
       "Z+AjgZHDh39tzJZbXveJddahsGyZli1adOjvnn/+jKt5c0i/FFHRw9QHRfozvmnQ5pYxs9bQU8fG\n",
       "hAmMmjiRcUCf4Z6G+mdIztSfJnno+azOIN1tN5b19LlSqfKhFYB0v0uBpe3t2qqjgwNW/cV0ph5+\n",
       "+MybFixInmObN26FNLN+KRQY2svyQk/LO0msKXEk8AhwDMlEgh+K4I7yM+RikWlTpzKv/LNTpjC/\n",
       "WGR6tbX39ouprY1x1e67UfnM3cz6pVSq7MxaYi3gc8BpwCLgixG9z+uyYEHc1N4uJk5kXKFAoVSi\n",
       "VCwyfaBj+uUG+oupmTnczaxf0jPrbcrPgHs6s05D/fMkob4QOCaCO/tzjM5x5dpVnaj0F1MeuFvG\n",
       "zPqts2OjpzPrNNSPIAn1x4AzIrgry3o79XQxeMoU5s+dy/ha/GUwmPqbnQ53M6tK2hd+JHAq8ChJ\n",
       "qN+dbVVvtrpfTM3E4W5mdZWG+lEkof43klD/Q6ZFtQBP+WtmdSGxDnA0cAowFzg8gj9mW5V153A3\n",
       "s36T2AS4E5gHHBLBnIxLsl54WMbM+kViXZJH1N0YwZkZl9OyPOZuZjWTTkt7NcnshUc24u35rcJj\n",
       "7mZWS98lmazrMw725uBwN7PVkjgOOAB4fwSvZV2P9Y/D3cx6JfER4EzggxEszboe6z+Hu5n1SOJd\n",
       "wGXAJyNWndDLGp9nhTSzN0lbHm8E/jOC32ddj1XO4W5mq0hbHmcAF0Zwedb12MC4FdLM/iVtefw1\n",
       "8BJwlDtjGo9bIc1sICaTPEf0cAd7c3O4mxkAEscD++OWx1xwuJsZEh8FJuGWx9xwuJu1OIkdgUuB\n",
       "gyKYn3U9VhvuljFrYRIjSVoeT27EB2zYwDnczVqUxFuA3wA/jeCKrOux2nIrpFkLkhhC0vL4AnC0\n",
       "O2Oah1shzWx1JgPDgUMd7PlU9bCMpIskLZb0UNmyjSTdKunvkn4naXi1xzGz2pD4EvBx4FMRvJ51\n",
       "PVYftRhzvxgY023Z14BbI2I74Pb0vZllTGIM8E3gY255zLeqwz0i7gKe67Z4f+Dn6dc/Bw6s9jhm\n",
       "Vh2Jd5O0PH7aLY/5V69umY0jYnH69WJg4zodx8z6IW15nAGMd8tja6j7BdWICEk9XrCRNKns7eyI\n",
       "mF3vesxaTdryOAP4/255bD6SRgOjK/5cLVohJW0NzIiIHdP3fwVGR8TTkkYCsyLiHd0+41ZIszpL\n",
       "Wx6vAZ7HLY+50N/srNewzG+AI9OvjwSur9NxzGz1vgdsABznYG8tVZ+5S7oC+BDJk9EXk1yJvwG4\n",
       "CtgSWAgcEhHPd/ucz9zN6kjiBGA8ySyP3ZserEn1Nzt9h6pZDknsB1xEMsujO2NyxHeomrUoifeQ\n",
       "tiA72FuXJw4zyxGJTelqefxD1vVYdhzuZjlR1vJ4QQS/yroey5bH3M1yIG15vBZYChzjzpj88pi7\n",
       "WQNqb9fYtjbGFwoMLZVYViwybcGCuKkGu54KrA8c7GA3cLibDZr2do3t6OC8CRMY1bls6lS2aW8X\n",
       "1QS8xInAfiQtj57l0QCPuZsNmrY2xpcHO8CECYxqa2PcQPcpMRb4OjDWvexWzmfuZoOkUGBoL8sL\n",
       "A9lfWcvj/hE8Vk1tlj8+czcbJKUSy3pZXqp0X2UtjydF8Mdqa7P8cbibDZJikWlTpzKvfNmUKcwv\n",
       "FpleyX4k1gNuBH4SwZW1rNHyw62QZoMo7ZYZVyhQKJUoFYtMr+RiatryeB3wLPAFd8a0Hs8tY5ZD\n",
       "Ej8A3g2McWdMa3Kfu1nOSJwEfBT4gIPd+uJwN2sCEh8DTgP2cMuj9YfD3azBSewEXIJbHq0C7pYx\n",
       "a2ASm5G0PJ7olkerhMPdrEGlLY8zgB9FcFXW9VhzcbeMWQNKWx6vB4rAsW55tE5ZPyDbzKpzLrAu\n",
       "8B8OdhsIX1A1azAS44APk7Q8Ls+6HmtODnezBpK2PJ5KEuzPZ12PNS+Hu1mDKGt5/EQECzIux5qc\n",
       "x9zNGoDE5iSdMSdEcE/W9Vjzc7ibZays5fH8CK7Ouh7LB7dCmmUobXm8AXga+KI7Y6wvboU0aw7f\n",
       "B4YCX3KwWy35gqpZRiTGA/vilkerA4e7WQYkPg6cglserU4c7maDTGJn4GLc8mh15DF3s0EksQnJ\n",
       "BVS3PFpdOdzNBonEOsC1wEVuebR6cyuk2SCQEHAhMBz4dAQrMy7JmpSfoWrWWMYDu5BcQHWwW905\n",
       "3M3qTGJfks6Y90fwctb1WGtwuJvVkcQo4JfAYe6MscFU13CXtBB4EVgBLI+IXet5PLNGIjGMpDPm\n",
       "zAhmZV2PtZa6XlCVtAB4X0Qs7WGdL6habkmsQfKYvKeB4z21gNVKI11QdYBbKzoT2JCkM8bBboOu\n",
       "3n3uAdwm6T5JX+y+8n3v2/Du9naNrXMNZoNK4hDg88CnIng963qsNdU73PeIiJ2B/YATJe1ZvvKc\n",
       "c57/QEcH5zngLS/SqQXOBw6MoJh1Pda66josExFPpf8+I+k6YFfgrs71l1wCI0Yw6tVXOU/SqxEx\n",
       "u571mNWTRBvJOPtJEdyfdT2WD5JGA6Mr/Vzdwl3SusCQiHhJ0luAjwBnlG9z1FHJvwsX8sTixQ52\n",
       "a14SawPXAJdGcGXW9Vh+pCe9szvfSzq9P5+r55n7xsB1kjqP88uI+F1PG5ZKlOpYh1ldpVMLTAeW\n",
       "At/MuBwzoI7hHhELgJ362u6MM9pXPv74+pfVqw6zQfAl4IMkd6B6agFrCJneoTpxIjPnzp045OWX\n",
       "j980yzrMBkpiNHA6yZwxL2Zcjtm/ZD4rpMSewAVAh/uBrZlItAN/BD4XwW1Z12OtoZkekP17YG3g\n",
       "37IuxKy/JNYjmVrgOw52a0SZh3t6tn4JcHTGpZj1Szq1wC+A+4BpGZdj1qPMh2WSr9kCeADYPMKd\n",
       "M9bYJCaRtPbuFcFrGZdjLaaZhmWIYBHwZ+DArGsxWx2JTwFfAD7pYLdG1hDhnroYOCrrIsx6I/Fu\n",
       "4CfAQRE8nXU9ZqvTEMMyyXsKwBPAe9IzebOGITECuBc4LYLLs67HWldTDcsApGPtVwFHZF2LWTmJ\n",
       "tYCrgasc7NYsGubMPVnGbsBlwHbuebdGIXE+sDWwfwQrMi7HWlzTnbmn/gQsJ7mV2yxzEscB+wCf\n",
       "cbBbM2mocE/P1n1h1RpCevf0t0nO2F/Iuh6zSjTUsEyynJHAXJKe91cGvzIzkNgKuAc4OoKZWddj\n",
       "1qlZh2WI4CngbuDTWddirUniLSQP3TjHwW7NquHCPXUxno7AMpDOzX4x8BBwTsblmA1Yo4b7DKBD\n",
       "4u1ZF2It51RgK+A4d2xZM2vIcE+fGH8FcGTWtVjrkNif5MEbB0WwLOt6zKrRcBdUu9azM3Ad8HY/\n",
       "3cbqTaKD5DmVH49gTsblmPWqaS+odkqfHv88sFfWtVi+SWxEMjf7VxzslhcNG+4p97xbXUmsCVwJ\n",
       "3BDBL7Kux6xWGnZYJtmGEcA8YCvfRGL1IPEDYAfgYxG8kXU9Zn1p+mEZgAieBe4ADsm6FssfiWOA\n",
       "scBhDnbLm4YO95R73q3mJD4ATAYOiOC5rOsxq7VmCPebgXaJd2RdiOVD+ljHXwNHRfBI1vWY1UPD\n",
       "h3v65/JluOfdaiB9KMx1wLQIfpt1PWb10tAXVLu2pQP4HbClp121gUqnFvhl+vazvgPVmlEuLqh2\n",
       "iuBhkkfwfSTrWqyp/RewPXCsg93yrinCPeULqzZgEh8DTgYOjODVrOsxq7emGJZJtmdDYAHJdARL\n",
       "61eZ5Y3EDsCdJMF+d9b1mFUjV8MyAGm72s3A4VnXYs0jPSm4Afiag91aSdOEe+oSPDRj/SQxhGR2\n",
       "0ZkRXJh1PWaDqdnC/TZgY4kdsy7EmsJkYE3g/2VdiNlga6pwT9sgf4EnE7M+SBwBHAgc6qkFrBU1\n",
       "zQXVrs+xLfB7kgdoL699ZdbsJHYFfguMTttozXIjdxdUO0XwKPAoyYRPZquQ2BS4lqSX3cFuLavp\n",
       "wj3lnnd7E4mhJFML/CSCG7KuxyxLdQt3SWMk/VXSo5Im1nj3VwGjJdpqvF9rUunUAhcA/wDOyrgc\n",
       "s8zVJdwlDQF+CIwB3gkcLmmHWu0/gpdIepc/W6t9WtP7MvBu4GhPLWCWtInVw67AvIhYCCDpV8AB\n",
       "UNPpVS8Gpkn8wP8zt472do1ta2N8ocDQUollxSLTFi6MFcBXgd0jeCXrGs0aQb3CfTNgUdn7x4Hd\n",
       "anyMO4H1gfcCf67xvq0BtbdrbEcH502YwKjOZd/73lrvWLz4ug1KpYP2j+AfWdZn1kjqNeZe9zPp\n",
       "CFbiO1ZbSlsb48uDHeC//mv5VtttN+GJCO7Kqi6zRlSvM/cngC3K3m9Bcva+CkmTyt7OjojZFR7n\n",
       "58B9El+J4LVKi7TmUigwtKflw4c/9uxg12I2WCSNBkZX+rl6hft9wLaStgaeBA6lhwm/ImJSNQeJ\n",
       "YKHEg8D+wNXV7MsaX6nEsl6Wlwa7FrPBkp70zu58L+n0/nyuLsMyEfEGcBJwCzAXuDIi6vWsSve8\n",
       "t4hikWlTpzKvfNmUKcwvFpmeVU1mjarpph94835Yl2QY6F0RPFF9ZdaoJNYYNuzcyzfd9Mb9R4y4\n",
       "939ff/3lJcUi0xcsiJuyrs1ssPQ3O5s+3JN98VNgXgSTa7E/azwSbyGZNG4T4KAIihmXZJaJ3M4t\n",
       "04uLgaPTuxQtZyQ2B+4CXgb2drCb9a1eF1QH2x8BAbu3t2vD7je5+M/25pXO8HgtMB34nm9YM+uf\n",
       "XIR7BCFxyfrr//ibHR2MKu+FnjqVbdrbhQO++UgcRhLqx3oiMLPK5GLMPdkfm22//T4LfvKTO9bq\n",
       "vm7iRGbOmRP71epYVl8SawCnkzyUZf8IHsy2IrPG0Wpj7kTwxHrrLX2pp3WFAoXBrscGJu1++hXw\n",
       "EWA3B7vZwOQm3AFWrHh2UU/LfZNLc5DYjGTOoNeAvSJ4OuOSzJpWrsJ9yZJXJp15ZvvK8mW+yaU5\n",
       "SOwCzAGuAY6I6PluVDPrn9yMuXcaNuz8G7fe+ofbbbTRX58slSj5JpfGJ3EIcD5wXATXZV2PWSNr\n",
       "qZuYVt0vu5A8qWlUOnOkNaj0voRvAscAB0Zwf8YlmTW8lrugWubPwCvAnlkXYr2TKABXAPuRXDh1\n",
       "sJvVUO7CPb3J5WLgeCkfffx5I7Ep8N/ASnzh1KwuchfuqUuB7YBnJK6TOFFiO09PkD2J95JcOL0B\n",
       "+GyEO5nM6iF3Y+6rHoONgX2AD6evlcCtwG3A7Z6jZHBJfBr4MfAfEVyTdT1mzahlL6j2fjwEbA/s\n",
       "SxL0HwIW0hX2d0Xw6mDV00rSn/1pwHEkF07/knFJZk3L4d7n8VkT2JUk7PcFdgbupSvs/xLBiqzq\n",
       "y4v0wumFwCjggAieyrgks6bmcK+QxPrAv5Oc1e8LjARmkYZ9BPMzLK8pSYwErgcWAEd7fN2seg73\n",
       "KqUdHZ3j9fsCy0jO6G8F7ohgSYblNTyJnUkumv4M+Jan6jWrDYd7DaVjxu+ka7x+T+BRusL+bt8u\n",
       "n0iHu74AfBs4IcIPLjerJYd7HUmsDexGV9jvSPLAkM6wf7DV7o5Np+k9GPgWyTNtvxzBA9lWZZY/\n",
       "DvdBJLEBMJqusH8rcDtJ2N8WwcLMiquz9K+a/YCzgDeAU0m+Zw/DmNWBwz1DElvQ1YWzL/AiXWf1\n",
       "syJ4LsPyakZiT+Bskl9mXweuc6ib1ZfDvUGkwxXvouvC7B7AI3SF/R8jeC27CiuXXiw9G3gHMAm4\n",
       "zG2jZoPD4d6gJNYB3k9X2O8A3E1Xf/1DjXr2K7E9yZj6niTDMD9ttl9MZs3O4d4kJDYE9qIr7IeR\n",
       "jtUDt0bweIblASCxJcnUvAcA5wLTIngl26rMWpPDvUlJbE3Xhdl9gGfpOqufHcELg1hLG3AKcARw\n",
       "ATAlL9cLzJqVwz0H0vH6negK+92Bh+gK+3siWF6H424AfAU4EbgcOMvT8po1Bod7DqXztOxBV9iP\n",
       "Au6iK+znVjNen+7/JOCrwE3AGREsqLZuM6sdh3sLkBgB7E1X2K9DVxfO7RE82c/9rEVyV+k3gHuA\n",
       "b0Qwty5Fm1lVHO4tJr2Z6O10XZjdG3iSrrC/M4KXun1mCHAYcCYwHzgtgnurqaO9XWPb2hhfKDC0\n",
       "VGJZscg0P6DcrHYc7i0uDe730hX2/wbcT1fYv42knfFl4NQIZlV7zPZ2je3o4LwJExjVuWzqVOY9\n",
       "/DAnO+DNasPhbquQeAvwQbqeSrWC5AakGbXqq99tN82cPJmPdl8+cSIz58yJ/WpxDLNW19/s9AOk\n",
       "W0Tal35L+qqLQoGhvSwv1OuYZtazvD4g2zJQKvU87XGp5Id0mA02h7vVTLHItKlTmVe+bMoU5heL\n",
       "TM+qJrNW5TF3q6m0W2ZcoUChVKJULDLdF1PNaifTC6qSJgHHAs+ki06JiJkDKdDMzLpkfUE1gHMj\n",
       "4tw67d/MzFajnmPuPis3M8tIPcN9nKQHJV0oaXgdj2NmZt0MeMxd0q3AJj2sOo1kfpLO8fZvASMj\n",
       "4gvdPh/AGWWLZkfE7AEVY2aWU5JGkzyjudPpDXGHqqStgRkRsWO35b6gamZWof5mZ12GZSSNLHt7\n",
       "EMkc5GZmNkjq1S0zWdJOJF0zC4Dj63QcMzPrgW9iMjNrIpkOy5iZWbYc7mZmOeRwNzPLIYe7mVkO\n",
       "OdzNzHLI4W5mlkMOdzOzHHK4m5nlkB+Q3WLSJyWNLxQYWiqxrFhkmp+UZJY/DvcW0t6usR0dnDdh\n",
       "AqM6l02dyjbt7cIBb5YvHpZpIW1tjC8PdoAJExjV1sa4rGoys/pwuLeQQoGhvSwvDHYtZlZfDvcW\n",
       "UiqxrJflpcGuxczqy+HeQopFpk2dyrzyZVOmML9YZHpWNZlZfXjK3xaTdsuMKxQolEqUikWm+2Kq\n",
       "WfPob3Y63M3Mmojnczcza2EOdzOzHHK4m5nlkMPdzCyHHO5mZjnkcDczyyGHu5lZDjnczcxyyOFu\n",
       "ZpZDDnczsxxyuJuZ5ZDD3cwshxzuZmY55HA3M8shh7uZWQ453M3McsjhbmaWQw53M7MccribmeXQ\n",
       "gMNd0sGSHpa0QtJ7u607RdKjkv4q6SPVl2lmZpWo5sz9IeAg4M7yhZLeCRwKvBMYA/xIUu7+QpA0\n",
       "OusaquH6s+X6s9PMtVdiwKEbEX+NiL/3sOoA4IqIWB4RC4F5wK4DPU4DG511AVUanXUBVRqddQFV\n",
       "Gp11AVUanXUBVRiddQGDoR5n1JsCj5e9fxzYrA7HMTOzXqy5upWSbgU26WHVqRExo4LjREVVmZlZ\n",
       "VRRRXe5KmgV8JSL+kr7/GkBEfDd9PxM4PSLmdPucA9/MbAAiQn1ts9oz9wqUH+g3wOWSziUZjtkW\n",
       "+NNAijMzs4GpphXyIEmLgN2B30q6GSAi5gJXAXOBm4EToto/D8zMrCJVD8uYmVnjybT/XNK3JD0o\n",
       "6QFJt0vaIst6KiVpiqRH0u/hWkkbZF1TJVZ3I1qjkjQmvTnuUUkTs66nUpIukrRY0kNZ11IpSVtI\n",
       "mpX+N/O/ksZnXVMlJA2VNCfNm7mSvpN1TQMhaYik+yWttqkl65uLvhcR74mInYDrgdMzrqdSvwM6\n",
       "IuI9wN+BUzKup1I93ojWqCQNAX5IcnPcO4HDJe2QbVUVu5ik/ma0HPhyRHSQDMee2Ew//4hYBuyV\n",
       "5s27gb0kfTDjsgbiZJJh79UOu2Qa7hHxUtnb9YBns6plICLi1ohYmb6dA2yeZT2VWs2NaI1qV2Be\n",
       "RCyMiOXAr0hummsaEXEX8FzWdQxERDwdEQ+kX78MPEJyX0vTiIhX0y/XBoYASzMsp2KSNgfGAj9j\n",
       "1UaWN8n6zB1JZ0n6J3Ak8N2s66nCMcBNWReRc5sBi8re+wa5jEjaGtiZ5KSmaUhaQ9IDwGJgVtoA\n",
       "0ky+D3wVWNnXhnUPd0m3Snqoh9cnACLitIjYEriEpPCG0lf96TanAa9HxOUZltqj/tTfRHz1vwFI\n",
       "Wg/4NXByegbfNCJiZTossznw7800z4ykjwPFiLifPs7aoXZ97r2KiA/3c9PLacAz377ql3QUyZ9J\n",
       "+wxKQRWq4OffDJ4Ayi+6b8GqU11YnUlaC7gGuCwirs+6noGKiBck/RbYBZidcTn99QFgf0ljgaHA\n",
       "MEm/iIgjeto4626ZbcveHgDcn1UtAyFpDMmfSAekF2uaWTPcVHYfsK2krSWtTTL76G8yrqllSBJw\n",
       "ITA3In6QdT2VkjRC0vD06wLwYZoocyLi1IjYIiLagcOAO3oLdsh+zP076RDBAyQztX0l43oqNZ3k\n",
       "QvCtaWvSj7IuqBK93YjWqCLiDeAk4BaSboErI+KRbKuqjKQrgD8A20laJOnorGuqwB7A50i6TO5P\n",
       "X83U+TMSuCPNmznAjIi4PeOaqrHaYUrfxGRmlkNZn7mbmVkdONzNzHLI4W5mlkMOdzOzHHK4m5nl\n",
       "kMPdzCx2UQjRAAAAEklEQVSHHO5mZjnkcDczy6H/Az1pcRz7mHjbAAAAAElFTkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff4cc0f76d8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "##This function provides an interactive mode to change polynomial degree. \n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w))\n",
    "    plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "    plt.plot(x,y,'yo')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import scipy\n",
    "from scipy import linalg\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAi8AAAEZCAYAAABfH8VpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecVNX9//HXGxBBpSkKUgSkiNiiMZafDZOoiAUbxtgT\n",
       "I998TcBYYtREl9WvoolRwZJYYxQbAioaxaARe40dLKACooggCCgown5+f5yzMDvM7M7uzu6dmf08\n",
       "H4957M4t535m9u6dz5xz7jkyM5xzzjnnikWzpANwzjnnnKsNT16cc845V1Q8eXHOOedcUfHkxTnn\n",
       "nHNFxZMX55xzzhUVT16cc845V1Q8eXFIOk/STUnH4ZxzzuXCk5c8kjRL0neSNklb/rqkCklbxOfd\n",
       "JE2QtEDSV5LelnRSXNczbrss7TE0yzGnSjqlPnGb2SgzO7U+ZaTEs6+kJ+Pr+riGbY9Le43fxNe+\n",
       "Y1zfQtI1kuZJ+lLSJEld4rotMrxHFZLOiOsPkvSspMVx/5skbZRy7K6SHozlfiLpf1LW9Yvrvojr\n",
       "J0vqV4v3YGqMZfu05ffH5fvE5+0l3RrjWyrpfUl/SNm+QtLXaa/x7CzHvE3SxbnGWE3sledfna4N\n",
       "kp5I3z/Da1glaUxct5ukKfF9/kLSOEmdczhOX0nfSrojZdl6ksZL+jj1fU7b73JJC+PjspTl1Z5P\n",
       "cZs/SpotaYmkuyW1SVm3fvxbLol/zzPSj+2cyx9PXvLLgI+An1cukLQd0Dquq3QHMBvYAtgYOAGY\n",
       "n1ZWOzNrk/K4r5pjZiWpRe1eQr19DdwM/L6mDc3sztTXCJwGfGhmr8dNTgP2ArYHugCLgWvivnPS\n",
       "9t0OqAAmxH3bAhcBmwNbA12Bv6QcfizwIbAZcBBwqaSBcV074AGgH9AJeBl4sBbvgQHvAydWLlBI\n",
       "aHcHvmDt3+wqYAOgv5m1BQ4FZqaVtX3aeXBFLeKoD9V6B+k4oAVp56SZbZTyd+oMrADGxdXtgb8D\n",
       "PeJjGfCPHA53HeHvkn7+Pw0cD3yevi4mqEMI59P2wCGVSWtN51P8cnE88P8I52Jr4rkYjQR6E/6n\n",
       "9wXOkXRADq/DOVcXZuaPPD2Aj4E/Ai+nLLsCOJ9wIdwiLltG+FDKVEbPuG2zHI53CbCK8GGwDBgT\n",
       "l1cQPvhnEJIBgNHAHGAJ8CqwZ0o5I4E70o5/IiHBWgCcX4f34qfAx7Xc50nggpTnNwCXpzw/CHgv\n",
       "y75lwBPVlH048Fb8faP4GjumHev2LPtuHLfvUJvXAXwCKC77LXB9XLZ3XPY2MKSaciqALXM43jBg\n",
       "JfBdPA8ejMu7ED58vyAk1cNT9tklngdLCB/0V8Tlc+Jxl8XHrjm+5naEhG3X6s5f4CRgZjXl7AQs\n",
       "reFYxwD3xr/5HVm2WfM+pyx7HvhVyvNfAC/kcj4B44GzU57vTvi/axWffwr8NGV9OXB3bf9v/OEP\n",
       "f+T28JqX/HsRaCupv6TmwM8I3/LTt7le0s8Um5IyqPGbr5n9EXgG+I2Fb4wjUlYPAX4EDIjPXwZ2\n",
       "ADoAdwH3SWpZWVSG4vcg1Dz8BLhQ0lYAkvaUtLim2GpLUg9CLcvtKYv/DRwoaXNJGwDHAY9k2FeE\n",
       "ZOuf1RxiH+Cdyl3SfkKohdw2y757A/PMrDav+zNgOlD57fsEqr42COfBJZJOltQ3Szm5nAc3AncS\n",
       "Er02ZjYkNts8BLxOSGJ+AvxO0v5xt9HAVWbWDtgSqKzZ2yv+rKz5eyk2qSyW1K2aMC4lJGfpNYjp\n",
       "TmLd9yHV3qz9O61DUltCYnAGta8dGgC8mfL8LWCbDMfIdD4Z654v6wN9JXUg1PDVWLZzLj88eWkY\n",
       "dxAufvsRPsA+TVs/lJB0XAB8pNAnZue0bRbGD4zKx1bVHC/TRXyUmX1lZt/BmiaaxWZWYWZXEi68\n",
       "W1Wzf7mZfWdmbxEuyj+I5TxrZh2qiaWuTgSeNrPZlQvMbALhw/dTQg3BVkCmfh17Epp/xmcqWNJ+\n",
       "sfwLY7nLgOeAC2JfhZ2AIwhNAen7dgOuBc6sw2u6HThRUn+gvZm9mLZ+OCHp+C0wTdIMSYPStnkt\n",
       "7TzYr5rjpf4df0SoWfo/M1tlZh8TmvOOietXEj54O5rZcjN7KUMZwJomlQ5mNjfjQcO5uztVm1Ey\n",
       "bdeDkJxkTDJjH6ELqL7J8WLgZjP7jBqaTDPYiHAeVVoal6XLdD5NBn4lqYekdkBl36QNUspIL7sN\n",
       "zrkG4clL/hkheTmOtd8yq3wgxKTiPDPbltCn4g1CH4tUm8QPjMrH+zUcM90nqU8knS1pukJH2sWE\n",
       "av6O1ZT5ecrvy4ENq9k2H9apOZF0BeEDYON4/PuBRzPsexIw3syWp6+QtBshQTjSzFL7kxwH9CK8\n",
       "T9cRasc+Tdt3U0Ltz3Vmdm8tX48BE4EfA78hQ22DmX1robP0zsAmhH4g90lqn7LZjmnnwZQcj98D\n",
       "6JKa+ADnET6UAU4h1Ky9K+llSQfV8vUBEGt4rgd+Z2YVqasybH4C8ExqgppSTh9CrdoIM3suy7F+\n",
       "QKhBurqaY1Tna0JfqErt4rJ0mc6nW4G7gamE5r7/xOVzU8pIL3tZLeNzzuXIk5cGYGZzCH0MDiR8\n",
       "gFW37ZfAXwkfNHWp0cj27XPNckl7Eb7NDjWz9rHmZAl16JTZECTtQah2T685GQT8IyZ7Kwk1ILtI\n",
       "2jhl39bAUWT4Nq9w19KDwMlm9mTqulibcIiZbWZmuwObAi+l7NuBkLg8YGaj6vK6zGwFIdn6NSGh\n",
       "rW7bZcAoQpLWqy6HS3s+h9DnKDXxaWtmB8fjzTSzY81sU+ByYHx8L2tbm9EW+CFwr6R5hOZJgLnx\n",
       "75oqY9NerJGZAlxkZndWc6x9CH2y5sRjnQUcKenVHGOdRqxBjHYgrYkq2/lkwUgz62VmWxBqVOea\n",
       "2aexOXFeTWU75/LHk5eGcwrw4/gBVkW8XXMbhVuB2wD/C8xI61ORa2Ixn3CXQ3XaEDr2LpTUUtKF\n",
       "VP2WmIuc4lHQClgvPl0/pW9NNpXfdL9JW/4WcJKktpLWI3RC/tTMFqVscziwyMympsWxLaGq/7dm\n",
       "lqmfTH9JbeL7cTyhie/KuK4t8BjwrJmdn8vrrsb5wD4xoU2P4QJJO8cYWgGnE+6oSq1lq815sGXK\n",
       "85eBZZLOkdRaUnNJ21Y2T0o6PtYsQUhkjdDRdkH8WdM5BYRaRELiuUN8DI6rdmJtIoOkyrt0qtw1\n",
       "J6kroRbj2th3pzo3xte4AyFR+DvwL9b2K6q8ZblVfJr6O4TarzMldYnHPRO4Le0Y2c6nDpJ6x/N7\n",
       "AOELx0VpZf9J4fb3rYFfZSjbOZcnnrw0EDP7yMxeS12U8ntrQhPIYsLtut0Jt8mm+kpVx5z4XZZD\n",
       "jQaOkrRI0tVZtpkcHx8Aswh3SaR+mFpafJm+fRuEWhxJ1VWH70NoZvoX4XWtiMcm7v+OpNRbyVsR\n",
       "+gBl6gdxBuGD9EPCHTODCB8uqU4kc63GmYSmmFtT3sO3U9YfEMtdRLhb54BYC0Y8xs7AL1L2XVpD\n",
       "h9WMzGyemT2fZXUF4bbgBYQmq58AB6U1V7yZdh5cmaWsW4ABsYloYmzCOZjwIf9RPMaNrE1aDwDe\n",
       "iX/Lq4BjYh+n5YS72J6LZe2itWOgZHz9ZvZF5QNYSDhX5pvZ9ymbnQhMyJCg/opQ0zQy9b2uXCnp\n",
       "fEmPxOOsSDnWfEJzzYqUvxuExG85IVF6DPhGsVO8md1A6MT8NiExfihDwpTtfOpIOKe/JjRv3WJm\n",
       "N6esLyOcT7MJd5tdbmb/zvR+Oefqr/I2zsY/sNSd8G1lM8LF7kYzGxObBO4ltNnPAo6O3+6cc845\n",
       "5xJNXjoDnc3sDYWRT/8LHEYYe2Ghmf1ZYbTRDmZ2biJBOuecc67gJNZsZGafm9kb8fevgXcJo6Ae\n",
       "ytomhH8SEhrnnHPOOaBA+rxI6gnsSLjbo1Nsz4bQCbFTQmE555xzrgAlnrzEJqMJwOnxdtE1LLRp\n",
       "JdOu5ZxzzrmC1NiT9lURb3+dQJifpHKQtvmSOpvZ55I2J9xlkr6fJzTOOVcHZlav8Z38+usaW6Zz\n",
       "NrHkRZIIt3dON7PUW3wnEcb9uDz+TB95Fqj/P2BjkDTSzEYmHUdNPM788jjzqxjiLIYYIX+JRzFc\n",
       "f11pyHbOJlnzsgdhivm3JL0el50HXAaMk3QK8VbpZMJzzjnnXCFKLHkxs2fJ3ufmp40Zi3POOeeK\n",
       "R+Iddkvc1KQDyNHUpAPI0dSkA8jR1KQDyNHUpAPI0dSkA8jB1KQDcK4pSWyQuvqQZN7m6pxztZOP\n",
       "a6dff/MnTpp7k5n1TzqWTCS9A5xmZk/nc9taxpDxfPPkxTnnmghPXpqGOHbaR0CLOM9Z0cp2vnmz\n",
       "kXPOuZIjKa99OvNdXiPJmmRKat6YgeSbJy/OOVdgJEnqNSoOKeFyJGmWpHMkvQUsk9RM0m6Sno+z\n",
       "pL8haZ+U7XtJejrOGj9F0nWS7ojrekqqkPRLSbOBx+PyX0qaLmmRpMmVs5bHdVdJmi9piaS3JG0T\n",
       "lw+WNC0eZ66ks+LygZI+Sdl/a0lTY6zvSDokZd1tMb6HYzkvStoyy1tR2XTzVdx2N0knS3pO0pWS\n",
       "FgJlkraU9B9JCyUtkDRWUru09/PH8feRksZJ+mcs8x1JP6zjtjtJej2uGyfpXkkX1+Zv7cmLc84V\n",
       "nN5HwKDToPfhSUdShI4BDgTaA5sDDwMXmVkH4GxggqRN4rZ3AS8CGwMjCcN3pPel2BvoDwySNIQw\n",
       "pMfhQEfgGeBuAEkHAHsBfc2sHTAU+DKWcQswzMzaAtsA/0kPOg7a+hAwGdgUGA7cKalfymY/i3F2\n",
       "AGYCl2R5D/aKP9uZWVszezE+3wX4ENgMuJRQM3NJfJ+2BrrH8iulvxeHxNfbjjAm27W13VZSS+B+\n",
       "4Nb4Ou4mzGFYqz4sxVgN5pxzJUnqMgzang4DW8D1beHUUVL/i2HpaLPPbkw6vlyoPE8D4ZXVqV+N\n",
       "AWPM7FMASccDj5jZZAAze1zSq8BBkqYCOwP7mtkq4DlJk1i3qWWkma2I5f0aGGVm78fno4DzY+3L\n",
       "SqANsLWkVyq3iVYC20h628yWAK+zrt2ADc3ssvj8SUkPAz8HyuOyiWb2ajz2ncCVWd6HbO/dZ2Z2\n",
       "Xfz9W0Ii82F8vlDSVcCFWfYFeKbyvZQ0FvhdHbbdDWhuZtfE5/dLermacjLy5MU55wrGvJug9SJo\n",
       "/tfw+dO8Faw6E+ZNTDqyXNUx6cinT1J+7wEMTW1+IXzu/QfoAiwys2/T9u1eQ3mjJf01bZsuZvak\n",
       "pGuB64AekiYCZ8c5+44E/gRcFpu0zk2pDVlTRtqxAGbH5RASs/kp61YAG1E7VcqX1AkYDexJSLya\n",
       "AYuq2T/1+MuBVpKaZekUnHFbwuv5NENctTpvvNnIOecKRJiMVgbN2sPR00Ed4uLiuy00Oanv1RzC\n",
       "3HkdUh5tzOzPwDxgY0mtU7bfgnWllzcsrbwNKxMRM7vGzHYGBgD9gN/H5a+a2WGE5qAHgHEZjvMZ\n",
       "0D2tn1MP1v2gz0W28yV9+aXAamDb2NR1Ag2fF8wDuqYt2yJDbNXy5MU55wpKy74w5Rdw37bw+Mnh\n",
       "uaujscAhkvaX1FxSq9hJtquZzQZeBUZKWk/S7sDBVP8h+ndCM9EAAEntJA2Nv+8sadfYd2U5oVlm\n",
       "dSz7OEntzGw1sIyQMKR7Ke53TtxnYIznnri+NjUTC4AKoHcN220EfAMsldSVmGw1sBcI78tvJbWI\n",
       "/Yh+VNtCPHlxzrkCYjb9MrOZE83Mws/plycdU7Eys7nAEOB84AtCzclZrP3sOw7YndCx9mLgXkL/\n",
       "lDVFpJX3AGHS4HskLQHeBg6Iq9sCNxKaXWYBC4G/xHXHAx/HfYbF41Y5hpmtJHRyPZCQfFwLnGBm\n",
       "H6Rsl55YZUy0zGw5oSPuc/GuqF2z7F8O7AQsIXQWnpCtzNocv7pt4+s8AjgFWEx4Lx6m6vteIx+k\n",
       "zjnnmoh8XDtL+for6V5gupmV17ixyxtJLwHXm9k/M6zzQeqcc865SrGpp7fCeDAHAocS+qS4BiRp\n",
       "b0mdY7PRScC2hFvEc+Z3GznnnGuqOgMTgU0Id7z82szeTDakJmErQqflDQm3ah9lZvOr36UqbzZy\n",
       "zrkmwpuNXLHJdr55zYtzztWROmkwbRlBS1qxkm9Zyhibb48kHZdzpc6TF+ecqwN10mC6MZpD6bNm\n",
       "4SR6q5PwBMa5huUddp1z9aZOGqy+mqxtNFV9NVmdNDjpmBpcW06vkrgAHEof2jI8oYicazK85sW5\n",
       "JiafTR0ql3iIYfTiIg5kszUrJrOLhmgSO/Ea8DVhIKysP63M1hnjIdc4Va5mwPpA6/hoVc+fuW3b\n",
       "lfUzviktaZ1xuXMubzx5ca4JqW9Th8q1PmFQqz3jYw9asQH7pX1gD6IDD7MXO7GUMIrnhmk/q/yu\n",
       "ckFqUjOD5vSla5VypzBQJ2g+faigajLREviOMNfLt9X+NFYwYdPtOHLB02jNukVZ9qm+vE95gLUD\n",
       "lK21khU1vY/Oufrx5MW5pqQtIzI2dYxlOJCpVqM9YQTSymTlh8AHwLPAncBpLOZOYJ91jrWcT6zM\n",
       "RuQSlsrVktQE5w1uZWhanPuxPuOYQx9Oompy8Z2V5XbbpNTnSNjvMN6Z8n9mM+s12aGu1xgm0Tst\n",
       "EfyQpVxTzW7OrSFpL+AmM+ufdCzFxpMX55qSlrTKsrw1gMrVnbWJyp7AlsDLwHOE4cZftDJbmrqr\n",
       "xupbMqlFDURsNlpJGC4cjVO2ocJXW5l9lGu5a2JUl2HQ9nQY2AKubwunjpL6XwxLR5t9dmNtywOw\n",
       "+faIOgnGMpyWtGYlK1jKNd5Z1+XKzJ4B8p64SOoJfAS0yDLjc23Kug34xMwuqH9k+ePJi3NNyUoy\n",
       "Jxpt6KdyzSY0xTxDqFm5DXjDyuz7astcSv5rILLFWecmmXk3QetF0PyvYX675q1g1Zkwr161LzFR\n",
       "8WSlAElqYWarCrW8RlKy4/H43UbOFThJknqNklT/C9FSxvAwc6ose4yldOBe4KdAJyuzI63MrrIy\n",
       "e6XGxIX4AT6X0xnLZMbxFGOZzFxG1KsGIiREM6ssq0dCZGYGMmjWHo6eDuoQFxffKJ0uK0mzJJ0j\n",
       "6S1gWRz2fzdJz0taLOkNSfukbN9L0tOSlkqaIuk6SXfEdT0lVUj6paTZwONx+S8lTY8THk6WtEVK\n",
       "eVdJmi9piaS3JG0Tlw+WNC0eZ66ks+LygZI+Sdl/a0lTY6zvSDokZd1tMb6HYzkvStoyy1vxdPz5\n",
       "laRlcWLGWscuaRhwLGGm62WSHqzP3yevzKzoHsRrkT/80RQe0PtI+PUS6H1EvcoZSSdG8jdO4it2\n",
       "4l224zn68CibMTjp15gx3s0YTB8eZQBT8xEnbH0u9D4CUPi59R+Sfo2Nfy5hhVBGA76+WcBrQFfC\n",
       "HWhdCbM7D4rrfxqfbxKfvwD8mdAKsQdhduXb47qeQAWhBrKyc/gQYAZhePtmwB+B5+L2BwCvAm3j\n",
       "862AzvH3ecAe8fd2wI7x94GEJhmA9YCZwLkxnn2BpUC/uP62GPvOQHNgLHB3lvehR4y9Wcqyusb+\n",
       "D+CiQjtnvdnIuQKVr34aKldr4AzgTOAOetHbbrMvGyzwPMl3k4zZ9MtSntaruchlJ5GX2iyzOjV5\n",
       "GDDGzD4Nseh44BEzmxzKtMclvQocJGkqIRHY10Jz0HOSJrFuU8tIM1sRy/s1MMrM3o/PRwHnxxqM\n",
       "lUAbYGtJr1RuE60EtpH0tpktAV7PEPtuwIZmVnmePinpYeDnQOUs1xPN7NV47DuBK7O8D5neu7rG\n",
       "nq28RHmzkXMFa95N8H1Z6J+xpp/GhWF5zVSuZirXCcD7hNubd7MyO8PKCj9xccXLDOXjUY8QPkn5\n",
       "vQcwNDbDLJa0mFDD0hnoAiwys2+z7JutvNEpZVX+L3UxsyeBa4HrgPmSbpDUJq4/EhgMzIrNQrtl\n",
       "OE6XDMefHZdDSMxSJy9cQbg7L1d1jb0gefLiXIEyszr301C59iVUA/8G+LmV2VFWZjNr2M25UpD6\n",
       "/zEHuMPMOqQ82pjZnwlNORtLSh2jaAvWlV7esLTyNjSzFwHM7Boz2xkYAPQDfh+Xv2pmhwGbAg8Q\n",
       "ZlRO9xnQPa1vWw/g09q8+Awx1yv2LGUlLtHkRdKtsYPQ2ynLNo4dpz6Q9G9J7ZOM0blktewLU34B\n",
       "920Lj58cnmencvVXuSYBtwKXA7tbmT3XGJE6V4DGAodI2l9Sc0mtYifZrmY2m5Dgj5S0nqTdgYOp\n",
       "/sP674SmlgEAktpJGhp/31nSrpLWA5YTxiBaHcs+TlI7M1sNLANWZyj7pbjfOXGfgTGee+L62tRG\n",
       "LSD0eeldn9jjfvMJQyYUlKRrXv4BDEpbdi4wxcz6AU/E5841SWbTLzObOdHMLPycfnmm7VSuzVSu\n",
       "6wi3OT8NbG1ldm+ug7c5V4rMbC6ho+r5wBeE2oezWPvZdxxhEMYvgYuBewn9P9YUkVbeA4QvBfdI\n",
       "WgK8zdpRltsCNxJGbJ5F6Fz7l7jueODjuM+weNwqxzCzlcAhwIGE5ONa4AQz+yBlu/T/54z/32a2\n",
       "nDAu03OxmWiXesR+CzAgllMwfcWUQw10wwYQBtN5yMy2i8/fA/Yxs/mSOgNTLW30QUlmZgXXgci5\n",
       "xhY7454OnE0Y8fZiK7OFyUblClU+rp2lfP2VdC8w3czKa9zYNYps51sh3m3UycwqOyXNBzolGYxz\n",
       "hShORvhz4FJC1ffuVmYzko3KueIiaWfCqM4fE2ohDiX8T7kCV4jJyxpmZpK82tu5FCrXPsBfCW3a\n",
       "x1uZPZNwSM4Vq86E2+Y3Idzp82szezPZkFwuCjF5mS+ps5l9LmlzQjvlOiSNTHk61cymNkZwzjU2\n",
       "id2BeYzU+oQBtXYAzgPutbL6zVviSlvs9Dkw4TAKlpk9DDycdByu9gqxz8ufgS/N7HJJ5wLtzezc\n",
       "tH1Kts3VuUoSmwNXo9W702LFpvR6shm9/30b/R843a78JPPcP85Vw/u8uGKT7XxLNHmRdDewD9CR\n",
       "0L/lQuBBwj3wWxB6PR9tZl+l7ef/PK7gqJMG05YRtKQVK/mWpYypy/w+Es2A/wG7iH4PvcXQY3bg\n",
       "+9b38be3ZrCs6ymE5qIxwJ1mLM/363Cly5MXV2wKMnmpK//ncYVGnTSYboxOm1l5JnM5vTYJjMQO\n",
       "YDew0bz2HH9gGzq/9QrwByurHNIbAT8BRhBu8bwVuN6M2Xl9Qa4kefLiik228y3pcV6cKw1tGVEl\n",
       "cQE4lD60ZXguu0tsKPEXmq18kgPO2owzuy+j81vHWpkdVpm4AMQ5yR4341DCXCjrAa9JTJDYJyY3\n",
       "zjlX0jx5cS4fWtIqy/LWGZenkDiYZivfp8+jR3NGj2/Y/ao/0axiVyuzp6rbz4wPzTiTMIT4E8AN\n",
       "wOsSv5RqPq5zzhUrT16cy4eVZO5Au5IV2XaR6Kr1lk9iw/ljOfaQNhw/+HrafN7Pyuyu2txFZMbX\n",
       "ZlxPmJPkHOAIYLbEpRLda/lKnCsZkm6TdHHScbj88+TFuXxYyhgmUXXiw0l8yFKuSd9UorlafnMG\n",
       "633zPrtd/RNG9L2bPv/uY2V2uZVZ1mSnJmZUmPFvMw4mzJy7AfCGxDiJPb1JyTVBmYbUzyjO9nxK\n",
       "XQ+Ur0RJUk9JFZL887kahTjOi3NFx+bbI+okGMtwWtKalaxgKdekd9ZV8+93ZIPF49n4wy4MOv15\n",
       "ur3yGyuz9/IejzED+J3EBcBJhPlJvpEYA9xjlqWmyLl6CDMi97wUZp2fy+znDVVGepE5bldod6/4\n",
       "l43qWOwBWEyPEHbycfjDH7k+wDai4/R7aL3wew4cPosLm+/byMdvBjYI7BGw+WD/B9Y16ffFH437\n",
       "yMe1s7oyoPeR8Osl0PuIupdf9zKAHYHXgKWE2ZjvBi6O6zoQBqT7gjAB4UNA17juEmAVsIIw6/OY\n",
       "uHw0YTLHJYRpOPbMctxhhAkdv4v7PxiXdwEmxGN+BAxP2WeXWOYS4HPgirh8DmE4hGXxsWvS500h\n",
       "nrOJB5bPF+MPfxTigy2e+h82mP8N2971NcN2+l9G0izReLCtwMaALQK7B+z/gSnp98kfjfG3xxqi\n",
       "DNh8GGw1DU55Hyos/NxqGmw+LPdy61cG0BKYTZiotDlwZEwoLorrNwYOB1oBGxHGE7s/Zf8ngV+m\n",
       "lXlcTHqaAWcC84CWWY7/j8pjxefNgP8CfyK0cvQCPgT2j+tfAI6Lv29QmaQQOuBXQLLXiUJ5ZDtn\n",
       "vU3NuQaiH9y+nbq+MpOvN7+OgeV3cdSxm9oN//1b0kP6m/G+GSOAnoQL6O3AKxInSqyfZGyuWM27\n",
       "Cb4vg+atQmtH81aw6sKwvNHK2A1oYWajzWy1mU0AXqlcaWaLzOx+M/vWzL4mTMC4T1oZVZpqzOxO\n",
       "M1tsZhVmdiWwPrBVNTGk7v8joKOZ/Z+ZrTKzj4GbgWPi+pVAX0kdzWy5mb2UKQaXmScvzuWZDj9x\n",
       "A+1w+wN8MPhNNp22iEN/1dP+dd2p9emM2xDMWGrGaKAfUEb4ljlbojxOTeBcTszMQAbN2sPR00Ed\n",
       "4uKc+5HkoYwuwKdpy2YTkwFJG0i6QdIsSUuAp4B2oY/N2jBSd5Z0tqTpkr6StBhoRxgRPhc9gC6S\n",
       "Flc+CHOSbRbXn0L433tX0suSDsqxXId32HUub1Qu8dLwc/jv2RfRfNUyfnjTQfbMeY/CyUmHVi0z\n",
       "KoB/Af+S2Br4LTBN4lFgjBkvVVuAcwC07AtTfgEf3g+9Dw/PG7WMeUDXtGU9YM1dgGcRkoVdzOwL\n",
       "ST8g9I8RGe5KkrQX8Hvgx2Y2LS5bRPaakfQkaw7wsZn1y7ix2Uzg2FjukcB4SRtnKMdl4MmLc3mg\n",
       "X+yzO6/98z5mDO7EFs9dwftDzrfPdiq6i5AZ7wK/kfgj8AvgbokFhLmU7jNjZaIBuoJlNv2ylKcT\n",
       "EyjjeWCVpBHA34BDCE03T8T1GxE65C6JSUJZ2v7zgd4pz9sQOvEulNQSOBdoW83x5wNbpjx/GVgm\n",
       "6RzgGkIz0dZAKzN7VdLxwGNmtoDQadcIfV0WxJ+9gRm5v/ymxZuNnKsHnd9mEw0641Em3PUsC7ee\n",
       "zYZfdLUF/UsYAAAgAElEQVT3hpxnVtzfnsz4yoyrgL7A/xGqj2ZLlEl0TjQ45zIws+8JAzSeDHwJ\n",
       "HE2406fS1UBrYCEh0XmUqrUco4GjJC2SdDUwOT4+IEwSvIJQm5LNLcCA2EQ00cwqgIOBHxDuNFoA\n",
       "3MjaBOgA4B1Jy4CrgGPM7DszW064++m5WNYudXg7Sl7RTsxIHx6r66y9ztWXytWC2XudyzPnXcDn\n",
       "O3zDhgtPsM+3/1fScTUkiW0ITUrHEG45HWO2tkOkK3w+MaMrNqU3MePxHEA3RquTBicdiitOkiT1\n",
       "GpXWYa/m/c5r/xOmXjiHuyZdyFc9b+TrLpuXeuICYMY0M/6XUDX+BjBO4gWJn0u0TDg851wTUrzJ\n",
       "C9Rq1l7n1tX7CBh0WugYWDOVq4eOP/Bxbnn2Ef576gK+32AbW7D1cDO+a+hIC4kZi834K9AHuBw4\n",
       "FfhY4k/SmjspnHOuwRR38gI5zdrrXCqpyzCp/zQYeClc3xYGjpL6T5O6DMu4fbk20JndL+Phv73H\n",
       "feN2ZWm3/2FZtx/Y6pZNujOdGavNeMCMHwODgC2A9yVuk/hhwuE550pY8d9t1JJNkg7BFZt5N0Hr\n",
       "RdD8rymDYZ0J86rc3aByiYpmQ3nnuOt49OoNQRNZ2Wa4GYsSCrxgmfE2MEziPML4FRMlPiHcpXS/\n",
       "Gd8nGqBzrqQUd83LQ8xhe7qrXLsmHYorHrkMhqVy7cCC/i9w+5SbefhvX7Oi4/62fJPjPHGpnhlf\n",
       "mvFnwm2eVwKnEZqUzpfYNNnonHOlopjvNprMUq7hNFoA1wE7W5nNTzo2VxykAefCyg9SB8Mym365\n",
       "yrUJ37e6hBfOPI6n/yQqWoyiYr2/+PgmdSexAzCcMNfM/YS7lN5INqqmye82csUm2/lWtMlL6otR\n",
       "uS4CBgI/sTLz6mlXaypXC+DXzN7zIibcVcE3m73J6vWHmfFh0rGVComOwK+A3wAfE5qUHjBjVaKB\n",
       "NSGevLhiU3q3Slc1kjB1+BUJx+GKkMq1L8s3eZPxd5/LP/+ziqXdf8vq9X/qiUt+mbHQjMsIs+uO\n",
       "AUYAH0mcK3nfNedc7koieYmz9B4HDFa5Tkg6HlccVK6eGtlsPG+ceC9Xztmcd372MBXrbWXGPcU+\n",
       "Qm4hM2OVGePN2Bs4jDBL70yJmyW2Tzg8V0Ik3Sbp4qTjyJWk4yQ9lu9tS1FJNButWV6ubYEngQOs\n",
       "zF5r/MhcMVC5NgD+wPztTmf8PV+woP9KaDbMjOeTjq2piuPDnEro4DuDMFT7Q96klF9NrdlI0j+A\n",
       "T8zswhy2nQrcYWa31PFYt8VjXVCX/V1mpd5sBICV2TuE9vQJKleu05a7JkLlkg4/+SweHzWPq2ad\n",
       "zt9fW8GCAbdAsx09cUmWGV+YcQnQkzCp3tmE2pjfS2ycaHAuJ+qkweqrydpGU9VXk+sy+nk+yshU\n",
       "bI7bNeg3eUnFPzRJASmp5AXAymwcMA64O3bCdE2YhCS2V8f3b+Dqj77m8VGjeP+Qx1jSYzDWoqsZ\n",
       "l/sYJIXDjO/NuNeMPYCjgG2BDyVukNg24fBcFuqkwXRjNMdzAEezT12mb8lLGdKOkl6TtFTSPUCr\n",
       "lHUdJD0s6Ys4+eJDkrrGdZcAewHXSlomaUxcPlrSHElLJL0qac8sxx0GHAucE/d/MC6fJekcSW8R\n",
       "ZphuLulcSTNjjNMkHZZSzsmSnkl5XiHpfyR9ECdpvLaabfeX9L6kryRdJ+kpSafk+t4Vm5JLXqI/\n",
       "xp+XJBqFS0RMWH4gcQla9QGtFz5Ln8eOZ8/LrmHgyA3si22ONuN5MyqSjtVlZ8arZpwE9AfmAv+W\n",
       "eEJiiETzhMNzqdoygkPpU2VZbadvqWcZkloCDwD/BDoA9xFuz6+sURFh5uct4mMFcC2Amf0ReAb4\n",
       "jZm1MbMRcZ+XgR1ieXcB98XjVGFmNwJ3ApfH/YekrD4GOBBob2argZnAnmbWFigHxkrqVM1LOwjY\n",
       "GdgeOFrSARlee8f4ev8AbAy8D+xOA9cmJakkkxcrs1WEE+ZolevopONxDS8mLDtJjAI+AJtIr8d3\n",
       "4Zd7bcI5m/2TA0/vbg/dcK49dIP3oSgyZsw342JCk9LNwLmEJqWzJDokGpwLWq6t4Uhbnvv0LfUv\n",
       "YzeghZmNNrPVZjYB1s56bmaLzOx+M/vWzL4GLgX2SSujShOTmd1pZovNrMLMrgTWJ3Qwzya9icqA\n",
       "MWb2qZl9F8scb2afx9/HEfp4VTfQ6mVmttTMPiH06fxBhm0GA++Y2QMx1jHA59WUWfRKMnkBsDL7\n",
       "EjgCuC525HUlJiYsP5K4nPBtZhwAA8uuoKz5N5y0XzO6v7i3jawYbmXmI+MWOTNWmnG3GbsDPyNc\n",
       "xD+S+JvEgITDa9pW8m2W5SsasYwuwKdpy2YTEwpJG0i6ITblLAGeAtqlzSpfpaZC0tmSpsemmMVA\n",
       "O6C2/Sk/SSvzREmvx2agxYSm0eqGCkhNQpYDG2bYpguhdjJV+vOSUrLJC4CV2evAmcD9Klf7pONx\n",
       "9RcTll0lriAMdHYnsAo4kj+1+ikj1ZeBF52LbCTw09iJ25UYM1424wRgADAfeEJiisQh3qSUgKWM\n",
       "YRIzqyybxIcs5ZpGLGMe0DVtWQ/WJiRnAf2AXcysHaHWRaytLUlPXPYCfg8MNbP2ZtYBWEL2DsDZ\n",
       "mmjWTjsi9QBuJNxYsnEs851qyszVZ0C3lOMo9XkpKsgOrZIGAVcDzYGbzezyupZlZXaHyrUzcKfK\n",
       "dUgcE8YVEYlmhGrVoYROnN8Q2ncPBd5mpFoT2np/SzhvTrAyy/0bnytaZswDRsbmwqHAhcDVEtcA\n",
       "/zBjSaIBNhE23x5RJ8FYhtOS1qxkBUu5xubbI41YxvPAKkkjCHesHQL8CHgirt+I0M9liaSNgbK0\n",
       "/ecT5uSq1IbwxWhh7OdyLtC2muPPB7asIcYNCcnMQqCZpBOhVh3RU5OtVI8QOhsPAf4F/BroXIty\n",
       "i07B1bxIak7oRDWI8K3q55K2rmexZxNO3JH1LMc1EolmEntKXE3LiZ/Tde9/06fvEWzZfg4dm59l\n",
       "xoWM1NuM1NHAu4R26B2tzC72xKXpMeM7M8YCuxAGrNyFMCHktRL9k42uabD59ojNsANtmg20GXZg\n",
       "bRKXfJRhZt8TugqcDHwJHA1MSNnkaqA1IXF4HniUqrUlo4Gj4p1IVwOT4+MDYBYh8ZlTTQi3AANi\n",
       "c9DETBuY2XTgr8ALhOagbYFnUzdJiym9Nid1/ZrfzWwhIXn/c3x9WwOvAt9VE29RK7hB6iTtDpSZ\n",
       "2aD4/FwAM7ssZZtaD5KkcnUi/DF/a2X2YB5DdnkSq/v3IPwTHgkspPWf32SrC/bmsJVbrNlwEjNp\n",
       "wdUM5mhCG/QIK7OnEwnaFSyJLoRvoMOANwhTEkxuyneZNbVB6poqSc0IfW2ONbOnko6nPoppkLqu\n",
       "VO3gNJd12zFrLc44fRRwk8pVXW9x14gkmkvsK3Ed4W9d2Ut+XzO2p+sfNq2SuEC4fbIlVwJ3Az/0\n",
       "xMVlYsZnZlxIuEvpLuBi4H2JYYkG5lwDiOO8tJe0PnB+XPxikjE1pELs85JTVZCkkSlPp5rZ1BoL\n",
       "LrOXVK7zgQdUrl2tzJbWLURXHxItCJ3ljiJU884FxgN7mzGjysbZbp/8ileszP7esJG6UmDGt8Dt\n",
       "EncQxr64X+I/ZmmdQ0uQpIHAwITDcI1jd0KS3hKYBhxWeXt2KSrEZqPdgJEpzUbnARWpnXbrW22p\n",
       "ct0AbAocxfUMoi0jaEkrVvItSxlTl7ZaVz2J9QgX0aGEyfhmEzrdTqhu9mZtrac4hr3XWTGWyTbD\n",
       "DmyYaF0pk7gemG1GnW8EKFbebOSKTbbzrRBrXl4F+krqSbj962fAz/N8jBHAVF7jVrqxR5VRHSfR\n",
       "W52EJzD1FxOWHxMSliHAR4SEZRczZmXdr1zrEUaVPJVd2YnHWMIBtFuzQW1vwXSuqvHA5fHhnCtC\n",
       "BVfzAiDpQNbeKn2LmY1KW1//bw/l6sIUPmI/1l9npX+rrzOJlsBPCU1ChxJ66o8n1LDMrnbfcm0J\n",
       "/Ipwt8BHwE3AfVzPQNrW/RZM51LFZst5wM41nZOlxmteXLHJdr4VZPJSk3z982hHvc5hGYZaHsdT\n",
       "Ns0G1rf8pkJifWA/Qg3LIcB0Qg3LRLOqo0uus2+51ifUypxKGDF1LHCTldn0Bg3aNWkSNwPTzbgy\n",
       "6VgakycvrtgUU7NR4/ma+RmX12ZI6yZKohVwAKGG5WDgLUINy/lm6wzRve7+4Y6vU4ETCSNM3gTc\n",
       "b2WWeYhw5/JrPHABNK3kJV8kFd+3XldSmnbNS+UU7Kl9Xv7FZ6zkQg7ndiuz7+t7jFIi0ZoweOBQ\n",
       "wkRgrxNqWO6PI51Wv3+5WhPGbzmVMKjcbcAtVmYzqtvPuXyLzZufA9vlkmyXCq81caWiSScvEBOY\n",
       "dvyODvQBWtKHRfSjHbA5oV18FmEOncpH5fPPrMxW5yOGQiaxAWE696GExOW/rE1YMtdcpZdRru0I\n",
       "CcuxhA7ZNwEPWZmtbJCgncuBxD+BV8y4NulYGosnL65UNPnkJesxwh0v3YFehEGueqU8ehJmAf2E\n",
       "dZOayscXVlaEby4gsSHhbp+jgP2BlwnV7PebsSCnMsq1IeFOsVMJ7+OtwK1WZrMaImbnakviUOAM\n",
       "M/ZNOpbG4smLKxWevNQ1htAE0oPMiU0vYAOqJjTpvy8upORGog0hYRlKuFvoBULC8oAZC2vcv1wC\n",
       "+gL7xsf+hDk7bgIetTJb1UChO1cnsd/W50A/M75IOp7GUAjXTufywZOXBqJytaFqYpP6ey/CSMLZ\n",
       "EpuPrcy+bvAYRVvC3UFHEcZjeY7QJPSgGYuq3TckK70JicrA+FgNPAlMBR6zMvusgUJ3Li8k7gae\n",
       "NOPGpGNpDMVw7XQuF568JCB+8Hcgc2LTMz6+IUtiA8yu6105Eu0I468MJSQcTxFqWCaZsbiGmHvF\n",
       "fSoTlmaEZKUyYfmokGqTnKuJxFHAqWYckHQsjaHYr53OVfLkpQDFRKETmRObXoQ+JF+SuSPxx8An\n",
       "qc00Eh1Ym7DsTUg27gMeMmNJNXH0YG2isi9hzozKROVJYGZ6sqJOGuzTLbhiEft3fQb0qqm2sRSU\n",
       "+rXTNR2evBQhlas5YabtnmSuvenENx3n88ZJ3/DOMRvxxbab0u6Td+j43sNse8/dbH/XDCuzigzl\n",
       "dqdqzcqGrE1UpgLvV1ezkvHW80nMZC6newLjCpXEBEIif1vSsTS0pn7tdKXDk5cSItEROAytHgr8\n",
       "P9p8+jr9H3iPvS9dwkbzu7I2yekAzGFtjU1zQrLSjpCkTCUkLO/WphlIfTWZ4zNUv/t0C66ASRwL\n",
       "HGvGwUnH0tD82ulKRdMeYbcESGwGHE7odLsL8BjW/GbgSFuyxddhDsoRVfcp1waEO6Uqk5lmwGhg\n",
       "WqYamZy1pFWW5a3rXKZzDe9h4G8Sbc1YmnQwzrma1Zi8SNrEzL5sjGBcbiQ6AUcQEpYfAo8CfweG\n",
       "mLG8pv2tzJYD78ZH/qwkcydin27BFTAzlko8TZjm4q6k43HO1SyXmpcXJb0B/AN41IqxnSkB0jed\n",
       "4Yzx8PcnoVkzQu2G0n7muix1XQ9ge+AR4BrgMbMCSQ6WMoZJ9E7r8/IhS7kmwaicy8UEwpcBT16c\n",
       "KwI19nmR1IwwaNkvgR8B44B/mNkHDR9e1pgKvt1WOuAU6Hcd9Lsfhr9DGNelIsPP2i5bAPzHLEst\n",
       "R8Li3UbDaUlrVrKCpVzjnXVdoZPYmNAHrKsZDT7GUlKK4drpXC5q1WFX0o+BsYS7UN4AzjOz5xso\n",
       "turiKNh/QKnLMGh7OuzZAm7qB6d+AM+ugqWjzT5rEgNhOVeMJB4DbjbjvqRjaSiFfO10rjZy6fPS\n",
       "ETgOOBGYD/wWeAjYgTC4Wc8GjK8IzbsJWi+C5n8NrT3NW8GqM2HexKQjc85Vazxh1vOSTV6cKxXN\n",
       "ctjmecIttEPMbLCZTTSz783sVUInUZci9AmSQbP2cPR0UIe42PsKOVfYHgAGSX53nHOFLpcOu/3N\n",
       "Mt8+a2aX5TmeEtGyL0z5BXx4P/Q+PDwvPD4arnNrmbFA4nXCpKIPJh2Pcy47H6SuifLRcJ1bl8Rv\n",
       "gF3NODHpWBqCXztdqcil2cgVCEmSeo2SVP+LT1tGVElcAA6lD20ZXu+ynSte9wMHS7RMOhDnXHY1\n",
       "Ji+S9sywbI+GCcdVr/cRMOi00BRVTz4arnPrMOMzwuCNP0k6FudcdrnUvGQaYOzafAfispO6DJP6\n",
       "T4OBl8L1bWHgKKn/tHBbdh35aLjOZVN515FzrkBl7bAraXfg/wGbSjqTcN8vQBu8uamRNcDt1z4a\n",
       "rnPZTATOl/i1GauSDsY5t67q7jZqSUhUmseflZYShtF2jcTMTOqTcvv1xt2p5+3XNt8eUSfBWB8N\n",
       "17lUZsyW+BjYB3gi6Xicc+vKZXqAHmY2u5HiyUlT7DEvDTgXVn6Qevu12fTLk47LuVIk8Qegpxn/\n",
       "m3Qs+dQUr52uNOWSvDyZYbGZ2Y8bJqSa+T+gc64hSfQBniXMdbQ66Xjyxa+drlTkMkjd71N+b0Xo\n",
       "yObtwM65kmXGTInPCf3+nkk6HudcVXUapE7SK2b2owaIJ9fj+7cH51yDkrgA6GjG6UnHki9+7XSl\n",
       "IpdxXjZOeXSUNAhoW5+DShoqaZqk1ZJ2Slt3nqQZkt6TtH99juOcc/UwHjhC8rsrnSs0uTQbvQZU\n",
       "Vs+sAmYBp9TzuG8DhwM3pC6UNAD4GTAA6Ao8LqlftrmVnHOuoZjxrsQyYBfgxaTjcc6tVWPyYmY9\n",
       "831QM3sPIMMo90OAu83se2CWpJn4hcM5l5wJhH5+fg1yroDk0mzUWtJZku6XNFHSGZIyDy1ff12A\n",
       "uSnP5xJqYJxzLgnjgSMlvJ+IcwUkl2aj2wkD040hDO96LHAHMLS6nSRNATpnWHW+mT1Uixgz9iiW\n",
       "NDLl6VQzm1qLMp1zLhdvARXAjoQm9KIiaSAwMOEwnMu7XJKXbcxsQMrz/0iaXtNOZrZfHeL5FOie\n",
       "8rxbXJap/JF1KN8553Jmhklrmo6KLnmJX+qmVj6XVJZYMM7lUS696F+L8xwBIGk34L95jCG1OnYS\n",
       "cIyklpJ6AX2Bl/N4LOecq63xwFHedORc4cil5mVn4DlJnxCacLYA3pf0NmGk3e1re1BJhxOaoToC\n",
       "/5L0upkdaGbTJY0DphPubDqtPvP3OOdcHrxKGKBzG+CdhGNxzpHj3EawzjcOq1xmZrMaJLLqY/KB\n",
       "lpxzjUbiSmCJGeVJx1Iffu10pSKX5OUOMzuhpmWNyf8BnXONSWIP4O9mbJd0LPXh105XKnLp87Jt\n",
       "6hNJLYAfNkw4zjlXkF4ANpHol3QgzrlqkhdJ50taBmwnaVnlA/iC0LHWOeeaBDMqgImEu46ccwnL\n",
       "pdnoMjM7t5HiyYlXfTrnGpvEvsAVZsVb8+zXTlcqckle9iHDQHFm9nRDBVUT/wd0zjU2iRbAZ8Cu\n",
       "ZnycdDx14ddOVypySV4eZm3y0oow19B/zezHDRxbdTH5P6BzrtFJ3Ah8YMYVScdSF37tdKWixuRl\n",
       "nR2k7sBoMzuiYULKKQb/B3TONTqJA4CRZuxe48YFyK+drlTkcrdRurnA1vkOxDnnisB/gH4S3ZIO\n",
       "xLmmrMYRdiVdk/K0GfAD8js9gHPOFQUzvpd4CDiCMEq4cy4BufR5OZm1fV5WAx+b2XMNHFe1vOrT\n",
       "OZcUiYOB35uxT9Kx1JZfO12pyCV5aQ30ISQwM83s28YIrDr+D+icS4rE+sDnwNZmfJ50PLXh105X\n",
       "KqobpG49SX8GPgH+CdwOzJX0F0nrNVaAzjlXSMz4DngUOCzpWJxrqqrrsPsXYGOgl5ntZGY7AVsC\n",
       "7aE4bxN0zrk8GY+PtutcYrI2G0maCfQzs4q05c2B982sTyPEl5FXfTrnkiSxATAP2NKML5OOJ1d+\n",
       "7XSlorqal4r0xAXAzFYD6yx3zrmmwozlwBRgSNKxONcUVZe8vCvppPSFkk4A3mu4kJxzriiMB45K\n",
       "OgjnmqLqmo26EWZRXcHacV1+CGwAHG5mcxslwsyxedWncy5REm2AT4EtzPgq6Xhy4ddOVyqqvVVa\n",
       "koAfA9sQbpWebmZPNFJsWfk/oHOuEEg8CNxnxtikY8mFXztdqaj13EaFwP8BnXOFQOJE4Aiz4rht\n",
       "2q+drlR48uKcc3Uk0QGYDXQ1Y1nS8dTEr52uVNRlYkbnnHOAGYuB54DBScfiXFPiyYtzztXPBHzA\n",
       "OucalTcbOedcPUh0BGYCXeL4LwXLr52uVHjNi3PO1YMZCwnDSRyQdCzONRWevDjnXP1505Fzjcib\n",
       "jZxzrp4kNgemA53jrNMFya+drlR4zYtzztWTGfOAd4CfJh2Lc02BJy/OOZcf3nTkXCNJJHmR9BdJ\n",
       "70p6U9JESe1S1p0naYak9yTtn0R8zjlXBxOBIRLrJR2Ic6UuqZqXfwPbmNkOwAfAeQCSBgA/AwYA\n",
       "g4DrJXntkHOu4Jkxh3DL9MCEQ3Gu5CWSGJjZFDOriE9fArrF34cAd5vZ92Y2i3Ah2CWBEJ1zri7G\n",
       "401HzjW4QqjV+CXwSPy9CzA3Zd1coGujR+Scc3UzAThconnSgThXylo0VMGSpgCdM6w638weitv8\n",
       "EVhpZndVU1TGe7kljUx5OtXMptYxVOecywszPpL4DNgTeCrpeCQNxJuxXAlqsOTFzParbr2kkwmT\n",
       "mf0kZfGnQPeU593iskzlj6xfhM451yDGA0dRAMlL/FI3tfK5pLLEgnEuj5K622gQ8HtgiJl9m7Jq\n",
       "EnCMpJaSegF9gZeTiNE55+poAnCEVBDN8s6VpAareanBNUBLYIokgBfM7DQzmy5pHGGkylXAaVaM\n",
       "QwA755osM96TWAzsCryQdDzOlSKfHsA55/JMohzYyIyzko4llV87Xanwak3nnMu/CcCREp4oONcA\n",
       "PHlxzrn8extYCfww6UCcK0WevDjnXJ6ZYfhcR841GE9enHOuYUwAjvKmI+fyz5MX55xrGP8F1gO2\n",
       "SzoQ50qNJy/OOdcAvOnIuYbjyYtzzjWcytF2nXN55MmLc841nJeA9hL9kw7EuVLiyYtzzjUQMyqA\n",
       "iXjTkXN55cmLc841LG86ci7PPHlxzrmG9SzQRaJ30oE4Vyo8eXHOuQZkxmrgfrzpyLm88eTFOeca\n",
       "nt8y7VweefLinHMNbyrQW2KLpANxrhR48uKccw3MjO+BScARScfiXCnw5MU55xqHNx05lycys6Rj\n",
       "qDVJZmY+2ZlzrmhIrA98DgwwY14yMfi105UGr3lxzrlGYMZ3wL+Aw5OOxbli58mLc841Hm86ci4P\n",
       "vNnIOecaiURrQtNRHzMWNP7x/drpSoPXvDjnXCMxYwXwGDAk6VicK2aevDjnXOOagM915Fy9eLOR\n",
       "c841Iok2wKdADzMWN+6x/drpSoPXvDjnXCMyYxnwH+CQpGNxrlh58uKcc41vPN505FydebORc841\n",
       "Mon2wBygmxlLG++4fu10pcFrXpxzrpGZ8RXwLHBQ0rE4V4w8eXHOuWSMxwesc65OEkleJF0s6U1J\n",
       "b0h6QlL3lHXnSZoh6T1J+ycRn3PONYIHgf0kNkw6EOeKTSJ9XiS1MbNl8ffhwA5m9itJA4C7gB8B\n",
       "XYHHgX5mVpG2v7fbOueKnsQU4O9mTGic4/m105WGRGpeKhOXaCNgYfx9CHC3mX1vZrOAmcAujRye\n",
       "c841Fp/ryLk6aJHUgSVdApwArGBtgtIFeDFls7mEGhjnnCtFDwCXSbQy49ukg3GuWDRYzYukKZLe\n",
       "zvA4BMDM/mhmWwD/AK6upqjiu5fbOedyYMbnwJvAfknH4lwxabCaFzPL9Z/xLuCR+PunQPeUdd3i\n",
       "snVIGpnydKqZTa1liM45Vwgqm44eynfBkgYCA/NdrnNJS6rDbl8zmxF/Hw7sYmYnpHTY3YW1HXb7\n",
       "WFqQ3unMOVcqJLoBbwGdzVjZsMfya6crDUn1eRklaStgNfAh8L8AZjZd0jhgOrAKOC09cXHOuVJi\n",
       "xlyJ94F9gceSjse5YuDTAzjnXMIkzgb6mTGsYY/j105XGjx5cc65hEn0Al4CupixquGO49dOVxp8\n",
       "egDnnEuYGR8DnwB7JR2Lc8XAkxfnnCsME4Cjkg7CuWLgzUbOOVcAJPoBU4FuZlTUsHkdj+HXTlca\n",
       "vObFOecKgBkfAF8Cuycdi3OFzpMX55wrHOPxpiPnauTJi3POFY4JwJES3rTjXDU8eXHOucIxjTBZ\n",
       "7Y+SDsS5/9/evYZaVtZxHP/+HBUr7SKVmQ7ZxaAxvGRYacGRMETCYZBQIyqD3pQ1lJSNA/kiQiso\n",
       "I/FFZUaIl8q01FLHGwrCmDDjWDPjBZxSM02sMMLy8u/FXhPbM2efc2Y8M89ac74fGM7aa2/W+Z0z\n",
       "7M3vPM9az+ozy4sk9UQVxWjq6JTWWaQ+s7xIUr84dSTNwfIiSf2yDlgCHNE6iNRXlhdJ6hGnjqS5\n",
       "WV4kqX+uAla0DiH1lSvsSlLPJOwB7F/FUwt7XD87tXuwvEjSIuFnp3YXThtJkqRBsbxIkqRBsbxI\n",
       "kqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRB\n",
       "sbxIkqRBaVpekpyV5MUk+4/tW5XkwSSbk3ykZT5JktQ/zcpLkqXACcCfxvYtA04FlgEnAhclGezo\n",
       "UJKp1hnmw5wLy5wLawg5h5BR2p20LAbfBb46bd9y4PKqeq6qtgAPAcfs6mALaKp1gHmaah1gnqZa\n",
       "B5inqdYB5mmqdYB5mmodYB6mWgeQFpMm5SXJcuDRqtow7ak3A4+OPX4UOGiXBZMkSb235846cJI1\n",
       "wJtmeGo1sAoYP58lsxyqFjKXJEkatlTt2m6Q5N3ALcC/u10HA48B7wPOAKiq87vX3gCcW1Vrpx3D\n",
       "QiNJO6CqZvtjURqEXV5etgmQPAwcXVVPdyfsXsboPJeDgJuBd1TrkJIkqTd22rTRdvh/MamqjUl+\n",
       "DoqXzhsAAAVBSURBVGwEngc+Z3GRJEnjmo+8SJIkbY8hr6HyjST3Jlmf5JZu3ZjeSfKdJJu6rL9K\n",
       "8prWmWaS5GNJ/pjkhSTvaZ1nuiQndgsXPpjk7NZ5ZpLkJ0meSHJf6yyTJFma5Lbu//oPSb7YOtNM\n",
       "kuyTZG33/t6Y5LzWmWaTZEmSdUmubZ1lkiRbkmzoct7dOo/0cgy2vADfrqojqupI4Brg3NaBJrgJ\n",
       "OKyqjgAeYHSlVR/dB6wA7mgdZLokS4ALGS1cuAw4Pcm72qaa0SWMMvbZc8CXquow4P3A5/v4u6yq\n",
       "Z4Hju/f34cDxST7YONZsVjKa7u7zUHYBU1V1VFUNef0sabjlpaqeGXu4L/BUqyyzqao1VfVi93At\n",
       "o6ureqeqNlfVA61zTHAM8FBVbamq54ArGC1o2CtVdSfw99Y5ZlNVf62q9d32v4BNjNZX6p2q2npF\n",
       "4t7AEuDphnEmSnIwcBLwY2Zf9qEP+p5PmpfBlheAJN9M8mfgU8D5rfPMw2eA37YOMUAHAY+MPXbx\n",
       "wgWQ5BDgKEaluneS7JFkPfAEcFtVbWydaYLvAV8BXpzrhY0VcHOSe5J8tnUY6eXow9VGE82y0N05\n",
       "VXVtVa0GVif5GqMPkDN2acDOXDm716wG/ltVl+3ScGPmk7On+jwUP0hJ9gV+CazsRmB6pxuxPLI7\n",
       "T+zGJFNVdXvjWC+R5KPAk1W1bgD3Nzquqh5P8gZgTZLN3WihNDi9Li9VdcI8X3oZDUc05sqZ5NOM\n",
       "hpU/vEsCTbAdv8++eQwYPyF7KS+9jYS2Q5K9gKuAS6vqmtZ55lJV/0xyPfBe4PbGcaY7Fjg5yUnA\n",
       "PsCrk/ysqj7ZONc2qurx7uvfklzNaDrW8qJBGuy0UZJDxx4uB9a1yjKbJCcyGlJe3p2EOAR9mxe/\n",
       "Bzg0ySFJ9mZ05/HfNM40SEkCXAxsrKoLWueZJMnrk7y2234FozvQ9+49XlXnVNXSqnorcBpwax+L\n",
       "S5JXJtmv234Vo9uz9PaqOGkugy0vwHlJ7uvmxKeAsxrnmeQHjE4oXtNdonhR60AzSbIiySOMrkC5\n",
       "PsnvWmfaqqqeB84EbmR0RceVVbWpbaptJbkcuAt4Z5JHkjSZxpzDccAnGF29s67718crpA4Ebu3e\n",
       "32uBa6vqlsaZ5qOvU5wHAHeO/T6vq6qbGmeSdpiL1EmSpEEZ8siLJElahCwvkiRpUCwvkiRpUCwv\n",
       "kiRpUCwvkiRpUCwvkiRpUCwv0jRJFny5/CRvSXL6Qh9XkhYjy4u0rZ2x+NFbgY/vhONK0qJjeZEm\n",
       "SDKV5PYkv0iyKcmlY89tSfKtJBuSrE3y9m7/T5OcMva6Z7rN84EPdSvarpz2fVYkubnbPjDJ/Une\n",
       "uPN/QkkaJsuLNLsjgZXAMuBtSY7t9hfwj6o6HLgQuGBs/0zOBu6sqqOq6vvjT1TV1cDjSc4Efgh8\n",
       "vaqeXOCfQ5J2G5YXaXZ3V9VfanQfjfXAIWPPXd59vQL4wBzHmetml18AVgHPVtWVOxJUkhYLy4s0\n",
       "u/+Mbb8A7DnhdVtHXJ6ne18l2QPYe57fZ2l3/AO6Oz9LkiawvEg77tSxr3d121uAo7vtk4G9uu1n\n",
       "gP1mOkiSPYGLgdOAzcCXd0JWSdptTPorUlrMasL2dK9Lci/wLLD1MugfAb9Osh64Adh62fW9wAvd\n",
       "/kumnfeyCrijqu5KsgH4fZLrqur+hfhhJGl3k9FUvqTtkeRh4Oiqerp1FklabJw2knaMrV+SGnHk\n",
       "RZIkDYojL5IkaVAsL5IkaVAsL5IkaVAsL5IkaVAsL5IkaVAsL5IkaVD+Bwn7vQevE1gaAAAAAElF\n",
       "TkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff4c76ff0f0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Read test data here\n",
    "a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
    "xt= a[:,0]\n",
    "yt= a[:,1]\n",
    "X = construct_poly(x,8)\n",
    "w = wRidge(X,y,1.0)\n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n)  \n",
    "    Xt = construct_poly(xt,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w),'g',label='regression training')\n",
    "    plt.plot(xt,Xt.dot(w),'b',label='regression test')\n",
    "    plt.title(\"MSE train: %f  MSE test: %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.ylabel('Output y') \n",
    "    plt.xlabel('Input x')  \n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####Conclusion:\n",
    "We get suitable solutions for n from 5 to 7 for the test regression. For higher values we get overfitting. For lower ones underfitting. The MSE of the test regression is always higher than the one of the training regression. That's evident because the regression function is fitted to the values of the training data and so doesn't provide an optimal fitting for the test data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum MSE: 0.6275\n"
     ]
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAk4AAAEACAYAAABF4/l0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecVNX5x/HPlyYooCiKIlZAjSXRxN6CxoYFjL3XaGLD\n",
       "JGo0JnGYaCy/GKNgSWJssXdFRY2JEnuLLWIDe0EsqICAgHx/f5wLDMvO7uzOzNbn/Xrd1zK3nHPu\n",
       "6Mw+e865z5FtQgghhBBC/To0dwNCCCGEEFqLCJxCCCGEEEoUgVMIIYQQQokicAohhBBCKFEETiGE\n",
       "EEIIJYrAKYQQQgihRGUFTpJWkPSQpLGSXpY0LNu/pKQHJL0h6Z+SlqhMc0MIIYQQmo/KyeMkaVlg\n",
       "WdsvSOoO/BfYFTgU+Mz2/0k6Gehl+5SKtDiEEEIIoZmU1eNk+2PbL2T/ngq8CiwPDAGuyk67ihRM\n",
       "hRBCCCG0ahWb4yRpZWA94Cmgj+2J2aGJQJ9K1RNCCCGE0FwqEjhlw3S3AsfbnlJ4zGksMNZ1CSGE\n",
       "EEKr16ncAiR1JgVNV9u+I9s9UdKytj+WtBzwSS3XRTAVQgiNYFvlXB/fvyGUprbPWlmBkyQBlwGv\n",
       "2D6/4NAo4GDgnOznHbVcXvaHvylIGm57eHO3oz7RzsqKdlZWa2hna2gjVC7oaQ3fvyE0p2KftXJ7\n",
       "nDYDDgBekvR8tu/XwNnATZIOB94B9iqznhBCCCGEZldW4GT7UYrPk9qmnLJDCCGEEFqayBxevzHN\n",
       "3YASjWnuBpRoTHM3oERjmrsBJRrT3A0o0ZjmbkAJxjR3A0IILV9ZCTDLqlhyjLGHEELDVOK7M75/\n",
       "Q6hfsc9J9DiFEEJoVyQdK+lZSTMkXVHC+UtKul3SVEnvSNq3AXW9I+kbSUvV2P+8pDmSVsxeby7p\n",
       "cUlfSvpc0qOS1q9RzjRJUwq2EdW43+yafSS9mt3zeEmbl3jsGkkTJE2W9Jak39Qot77jdZXdT9Jd\n",
       "2fszQdJISR1LuZ9KisAphBBCiyNpmKQzq1T8h8DpwOUlnn8RMANYBtgfuETSmiVea+AtYF6wJWkd\n",
       "oFt2DEk9gbuBC4BepBU48sA3NcrZ2XaPgm1YiW1o0P1K2pb0kNfBtrsDW2T3UOexzFnAKrZ7AoOB\n",
       "4yTtUM/x7UssewTwGbAcsC7wQ+DoEt+DionAKYQQQks0EthLUsVXnrB9u+07gc/rO1fSYsBuwO9s\n",
       "T7P9GHAncGADqrwGOKjg9cHAP4C5w0CrpWb5RiczbD9g+38NqKOohtxvJg/kbT+dXT/B9kclHMP2\n",
       "WNszCsqaTUEuxyLHPy2lbGAt4EbbM7PVSe7L9jWpCJxCCCG0ONmqE9dRR4Ai6W5JXxTZRpVQTSnz\n",
       "vFYDZtseX7DvRRr2C/tJoKekNbKhpb1JwdRcbwDfSrpS0g6SejWkvZIuknRRCe2o936z9v0AWEbS\n",
       "OEnvZ0NiXes6VqOMiyV9DYwFzrD9XB3H/2D7uRLLvh/YT1I3ScuTeqzuLeG+KyoCpxBCCAuQcCW2\n",
       "CjTlSuCQYgdt72y7V5FtSAnll9LG7sDkGvumAD1KuLbQ1aRep22BV0jDZ6kR9mRg86w9lwKfSLpT\n",
       "0jIF1wu4o0ZweHh2/TG2jymhDaXcbx+gM7B71qZ1SevQ/pY0VFns2PxK7KNJ79s2wBmSNqzj+OnZ\n",
       "8brqnWs4sDbpv8f7wDNZT1qTisAphBDCAmxUia0CTVkaWLTmL94KKqWNU4GeNfYtTgqeSmVS4LQ/\n",
       "Cw/TpRPs12wfansFUnDQFzi/RhlDawSHlzWgDdSss4jp2c+Rtifa/hw4D9ixnmMLyIYcxwA3UzC/\n",
       "q47j0+oqW5JIPU43A4sCvYElJZ1Twj1VVAROIYQQWpxsQvGGwBnAoUXOubfGU2aF2z0lVFNKD8wb\n",
       "QCdJAwr2fQ94uYRr51dkv0ea6DwYuK2ec18HriIFUJVU7/3a/gL4oPZD/rLIsbp0Br6u73gJZfcm\n",
       "DeVdaHuW7UmkHsmFgrZqi8AphBBCiyJpP2Br2yNJPQy71JxHA2B7cI2nzAq3neoov2NWXiego6RF\n",
       "ij3WbvtrUqDze0mLZo/H70LqQWqow7P7ml64U9Lqkk7I5u0gaQVSL8wTNZveiDobdL+ZK0hPuy2d\n",
       "zbf6Bempv2LH7srqWTpLJ7BYVuf2wJ6kyfT1Hq+rbNLTdBOAo7JrlyD13r3YmPekLLabZSOb+xdb\n",
       "bLHFFlvpWyW+O1vy9y+wMXB5jX0XAodUsI7hwJwa22kFx0cDpxS87gXcThq2ewfYpwF1vU0Klmru\n",
       "7wR8C6xIGpa7kdTjMjX7eQnQvUY500hDhHO3W7NjfwEuqeD9diKlYPiCFKycD3Qp4VhvUgb+L4Av\n",
       "gaeBIQXl1ne8aNnZ8Y2AR7LjnwI3AEtX8f/FWj8nkTk8hBBakUp8d8b3bwj1K/Y5iaG6EEIIIYQS\n",
       "ReAUQgghhFCiCJxCCCGEEEoUgVMIIVSBJEmrnJXlnwkhtBEROIUQQlX03w12OBr6/7i5WxJCqJwI\n",
       "nEIIoYKkvkdKa4yFQWfCxT1h0FnSGmOlvkc2d9tCCOXr1NwNCCGEtmXCpdBtEnT8U8pX2LErzP4l\n",
       "TKgzW3QIoXWIHqcQQqgg2wYZOiwBe70C6pXtbp6keSGEiooepxBCqLguA+GBQ+HN29Mcpy4Dm7tF\n",
       "IYTKiB6nEEKoMPuVs+3xt9l2+vlKk6/gHoqTtKSk2yVNlfSOpH3rOf9YSc9KmiHpigbW9Y6kbyQt\n",
       "VWP/85LmSFoxe725pMclfSnpc0mPSlq/RjnTaixkPKIa95tds4+kV7NrxkvarNT3opZrN8/2d5F0\n",
       "WdaGydl7sEPBdd+R9GD2HoyTtGuNcvtJuit7fyZIGlnPmntVEYFTCCGEFkfSMElnVqn4i4AZwDLA\n",
       "/sAlktas4/wPgdOByxtRl4G3SIv2AiBpHaBbdgxJPUmL6F5AWhdveSAPfFOjnJ294ELGw0psQ4Pu\n",
       "V9K2wNnAwba7A1uQ1sqDet6LIte+lR3uBLwHbGm7J/Bb4CZJK0rqRFrsd1T2HhwJXCOpsLd2BGmx\n",
       "3+WAdYEfAkeX+B5UTKxVF0IIrUh7Wasuy381DtjM9sQKlrsYMAlYy/b4bN9VwEe2f13PtacD/Wwf\n",
       "2oD63gb+Dgy1vWG279ysDWcAK5MCmgds96qnnMNtP1hq3dl1Db5fSY8Dl9ou2rtW7L0o5doa579I\n",
       "WoR4HPCE7R4Fx+4HnrJ9Wvb6deB42/dlr/8P6Gn7Z6XU1VCxVl0IIYRWI5tMfx1wYLFzJN0t6Ysi\n",
       "26gil60GzJ4bRGReBNYqoVmNDTafBHpKWiMbWtobuKbg+BvAt5KulLSDpGIBVK31S7pI0kVFrmnQ\n",
       "/Wbt+wGwTDZc9n42JNa1vrY04Nq55/fJ2je2SNs7AGsXvL4f2E9SN0nLA4OBe4tcWzUxOTyEEMIC\n",
       "lFdFhiKcK7tX60rS0M25tZZv79yIMrsDk2vsmwL0qOXchapsRH1zXQ0cBDwMvEIa8kqF2pOzeUAn\n",
       "A5cCy0oaDRxh+5PsNAF3SJpdUOaJti+zfUwd9Tb0fvsAnYHdgc2B2aQhtN9m27xml3EtkjoD1wJX\n",
       "2n4je/2JpJOA84GtgC2Bwh624cC/svvpmF17Z5H7qJoInEIIISygAgFPpSwNLCppQ9tPV6jMqUDP\n",
       "GvsWJwUT9Wns+2JS4PQIsArwj5pl2X4NOBRA0uqkHqnzgf0Kyhja0KE6Gn6/07OfI+cOkUo6j4WD\n",
       "n9rei5KuldSB9H7MAI4FsD0rmww+khRAPgPclJ0zd+j2fuBmYCNS4He5pHNsn1zH/VdcDNWFEEJo\n",
       "cbKnrTYkzQOqdU6RpHtrPGVWuN1TpOg3gE6SBhTs+x7wcgnNanSPk+33SJOkBwN1JkO1/TpwFQsO\n",
       "UzVWg+7X9hfAByWUu9B7Ucq1WQB0GSko3t32twXX/8/2INu9bQ8G+gNzA+bepGHAC23Psj2J1CO5\n",
       "YwltraiyAydJl0uaKOl/BfuGS/oge9RwgccNQwghhLpI2g/Y2vZIUg/DLrXNk7E9uMZTZoXbTrWV\n",
       "bftrUuDye0mLZkNku5B6QIq1p2NWfyego6RFGvkY/OHZfU0v3ClpdUknZPN2kLQC6Sm8J2o2paEV\n",
       "NuZ+gSuA4yQtnc23+gVwV9a2+t6LotdmLgHWAIbYLnxqEEnrSOqatfNE0tDfldnhz4AJwFFZG5YA\n",
       "DibN12patsvaSI8argf8r2BfDvhlPde53Lpjiy222NrbVonvzpb8/QtsDFxeY9+FwCEVrKMXcDtp\n",
       "GOsdYJ8ax0cDpxS8Hg7MqbGdVmJdb5OCpZr7OwHfAisCfYEbSb01U7OflwDda5QzjTTENne7NTv2\n",
       "F+CSCt5vJ1IKgy9Iwcr5QJdS3ot6rl0pO7/mfeybHf8/0hOAU4B7gFVrtHMj0nDnF8CnwA3A0lX8\n",
       "f7HWz0lF0hFIWhm4y/Y62escMNX2n+q4xm7hj8OGEFooqQv2zOZuRnNoL+kIQmhuzZGO4DhJLypl\n",
       "CV2iivWEENoLSUi/AaYh/R1p6eZuUgihfalW4HQJ6cmBdUlddUV7nkIIoSRSj0n0uv81Vj9+LV6e\n",
       "MYpdvjcHvYJ0DCnrcAghVF1VAifbnzhDypi6YW3nZZPI526DqtGWEELr91udset7rPDh7fx4iw0X\n",
       "uf+vr+x25k+Gdv/L2PV4Xu+w0vGGZ8nWw2prJA0q/K5s7vaE0N5Va47TcrYnZP/+BbCB7f1qXBNj\n",
       "7CGEOklsfiiXn/9/nLjeyIEDnv39Pi98QsdZW5ImuK7KrG4T9NpO3Q74b/fFLn73ei/mb0YLfkX2\n",
       "/dMWxRynEJpGsc9J2YGTpOtJC+31BiaSnqgbRBqmM+lJgJ+6xlpD8cENIdRGQsB2HZZ9+vTfdzlx\n",
       "7UM/f6zzHnvPmfHEitxHykI82jlPUl6dgXWxNuPLlXbv7qkb//bJqfrZC9/MGb2aR/98B876pDvP\n",
       "OedZzXtHlRWBUwhNo2qBU2PFBzeEUEhDjuzEVyv8ip4fDFu83/29rv/n+1r5qzmfHbsTv3hwVe5w\n",
       "bsGcLwtdv8zYzvT46NerL/LUSX/9/M/qO2tS52N25tsH+vMk8Fi2PeGcv2qSG6qSCJxCaBoROIUQ\n",
       "WhzltRgzFx3MpAHD6PnBpkxb+ptN3+7yn3/d/7+1us7mTsEJuGE9RhK9xJzf7MZtR/5NP/n2i55T\n",
       "xu6195znnuvLd4H1gTeZH0g9BrzrXDN9ETZCBE4hNI0InEIILYLyWg7YmTkdf4w7bMX7m8DbW73P\n",
       "Z2sMnz12/286MucvwInYV5VVj+jfg8l//A1nbn0853fowsyzz9vUI07ajjWBzUiLkG7GW3TidWYx\n",
       "h6lM41MmcZY/8l31FN9sInAKoWlE4BRCaBbKS8CawFBgCGZ1Pl7vHZ78+UqM3/4pvu7ze6OngdNJ\n",
       "C5rujv3fitUvtvgOr1w0kuP6bcITXy/K9J9ijwZQH+3IilzMzqw074J/YVbgJVZnFPOH92quLt9s\n",
       "InAKoWk0RwLMEEI7pbw6Ka9Byus8YDwwmm+6r8Q9I9/gjBmz+etzr/HiQVt5ap/BRq+TllfYGNig\n",
       "kkETgM0jr7Lmutvw7+P347quH7HcjZPV8wGkVenJsAWCJoBtEGOZTVo64lTgI+X1gvK6SHntp7xW\n",
       "yoLB0EpJOlbSs5JmSLqiluNLSrpd0lRJ70jat46ySj63yPXvSPpG0lI19j8vaY6kFbPXm0t6XNKX\n",
       "kj6X9Kik9QvKmFZjkeMRDWhDJd+P70h6MGvnOEm71jh+jaQJkiZLekspoW1J9ZTTzmxf4fszuyHv\n",
       "UaFIGhdCALLel54MowtdmckMJjPCE1PPTEnX59UD2J7Us7Qj6YnaUby+y5Fcf8cO0OEw0npZm9qM\n",
       "Sxfpu9m+O4CTsWdX+LYAsJkDXC3teuuDbP2rn3P+SSdzzsv9Pf3TN2tb8H42U53zqdl9dQG+Txre\n",
       "2wP4MzBLeT3K/HlSLzlXnba3V5KGAcva6b9DhX1I6uHcHuhWy/GLgBnAMqS1WO+R9KLtV8o8tzYG\n",
       "3iIt6nshpMVus3Y5e90TuBv4KXATsAhpndgZBWXsbPvBEuusqSLvh1Ii2juBi4EfkZ6wv0vSerbH\n",
       "ZaedBfzE9gxJqwP/kfSs7ftLqKfR7bTdvaCdiwEfk97LBouhuhBCCpr6cQFDGDBv5yjG8wHH1xU8\n",
       "Ka9+pJXWhwKbAo+TvjjvYrg7AycB+5BWYj/X5v35F2sfYCRwPPZ1lb+r4iT6rsXLf+7Yd+s9Xjry\n",
       "04V73q/hPo/z4FqvTb1N/Zk7Rypt/YCnSUHUo8CTznlKddrePobqJAkYB2xWM51NBes4Hehn+9CC\n",
       "fYuRFppdy/b4bN9VwEe2f13j+pLPraMNb5MSRQ+1vWG279ys3DOAlUmBwAO2e9VRxuFlBE5zyyn3\n",
       "/VgbeMJ2j4J99wNP2T6tlvpWB/4NDAFeb0A95bbzYOB3tgdQh2Kfk+hxCqEVKrd3aCE9GbZA0AQw\n",
       "hAFcw3GkldNTvSlo+C5z5yulpZVGA5cBeznnyRLfAc4k9Tr9FVjD5pP5jVcn4GxgN2Ab7Bcb3e5G\n",
       "svkI1t5bi296bLdbHjt3+h6fLTLv4CjeZDIji16bnsAbn21XAiivpYBNSMHUacB6ymsccO/cnqvQ\n",
       "MLYt6TrgQODc2s6RdDcpcK3NI7aH1FNNbcHjasDsub98My+Sek/KObcuTwIHSlqDFCzuTbqvM7Lj\n",
       "bwDfSroSuIEUiHxRo4yigbCkiwBsH1NPO8p9P2rTAVi7RnsuBg4m9Zwda/s5Ses1oJ5y23kw8I8S\n",
       "27+QCJxCaGWK9A71Vx/R6OCpC12L7O+WJZr8ISlQGgLMIfUqnQg8NjfBpMQPNJxTScHDBcAwltGm\n",
       "9OQfWisFeKtP4srX4AhgNrA+9qRGtbdC/NUdF2rpjm/pyr5nLE+ntVeb+WGnrSYtPv63MyY93KBy\n",
       "cv6cNJRyN4DyWoQ0vNev8q1uAlJlhiLK79W6EhhFkcDJ9s5lll/bfXYHaj4MMAXoUea59bkaOAh4\n",
       "GHiFNCyVGmlPVlpS6GTgUmBZSaOBI2x/Qgok7pBUOFx8ou3LsuvrC5jmVVXLvobc4+vAJ5JOAs4H\n",
       "tgK2BBboCbN9tKRjSN8rt0h6jhRElVpPo9spaaWsTYfSSBE4hdDalNg71CAz582VWFAPVgM+IX0h\n",
       "jgJ2Al4pzHsksSVpEvVapF9wB9pMqy3Am3EzP/rzdG7/xXT2xf62MU1NQzgrnwnvnOoKzDXwp9+O\n",
       "BkZLdFmG/57cn3NPncT9EyZpwAkDePNSGlFHlqzziXLb1mxazjDe0sCikja0/XQVyq/tPqcCPWvs\n",
       "W5z0S7icc+tiUuD0CKkX9x8122b7NbJf9tkQ1zWk4GS/7Pqh5Q7V1awzU/I92p6VTQYfSQryniHN\n",
       "I1ro+yX77I6RdDNpftdVpdZTZjsPJPVGvltLGSWJp+pCaG3q6B1qdJmTGcHdLPhF8k+msSS3AGs6\n",
       "542d85nOeaxztoQkBks8QhqmuxkYYHOBzTSg1gDv3T3p9Mvl6dHYoCnpvxvscDT0/3Hjy1iYzcz/\n",
       "+gen78f1yx/HyNHT6XbxW6wy/i2t+v1K1hNKI2kH0gLxZ1Ckd0DSvTWelCrc7imhmtqC4jeATpIK\n",
       "/9/9HvBymefW3RD7PdIk8cHAbfWc+zop0FirofXU14xa9jXoHm3/z/Yg271tDybNB6wr6O0MfN3A\n",
       "espp50Gk967RoscphNamWO/QTKY3pJhsvtL3gaEczRDeZglG8yGz+ZKpfMSXnF9z6E+iI2lu0qlA\n",
       "R9JcpltsFn6irMIBntT3SOh5PAzqBBf3hCPOktY4HSZfYH/0t8aUWRubSbD/3ktphzWO54Ibj+bi\n",
       "Z57RBvetxhv7L+6vvqxUPaE4SfsB69r+laQewO8l/cL2Av/vZ7+YG1N+R9Iv7E5AR0mLkObHfGv7\n",
       "a0m3ZXX+hPQZ2YU0h20BDTm3RIcDS9ienj2hNre9qwM7AzfY/lDSCqRemicLb6uRdVbs/cjKWoc0\n",
       "T6sDcDTQh7lzAaWlSU/b3UXqhdoG2BPYxva0+uopt52SNgX6kv7Qa7TocQqhtZnMCEYxfoF99Uxo\n",
       "nkt5LaK8dlBeFwPvA9eTHus9llVYyk+5n//rtf26tysMmiQ6SxwCjAVOIE2A/p7NDbUGTUDXb2r9\n",
       "q7DBAd58Ey6FWTno2DX9jujYFWaflvZX3ude6rXT/Pvv7c2Nu49nwEaz6PzJv7TNcKT43qwiSRuT\n",
       "fpH+CsD2FFK6in0qWM3vgGmk4aQDgOnAbwqOH036XHxCGhL7me1Xs/aNlnRKKec2lO23bD9XuCv7\n",
       "OYXU+/aUpKmkYeCXSJ/Fue6q0eN269wDki6RdEkdVVfy/TgQ+AiYSJrjtK3nL5tk4GfAB8DnpNQC\n",
       "B9p+pr56ym1n5iDgVttf1/Fe1CvSEYTQCmVP1R1HF7oxk+lMZmSxieHKa0nSE25DgW1Jwc+dwCjn\n",
       "/Fqd9YhuwGHAr0h/RZ4JPGQXCYrmX7jhrV24+6dr0PHz3Vhy3v5RvMkHDGvsJHZpwO6w7eXw+Qew\n",
       "5Arwr0Ps8XUOa1SCRMejuei0w7ns192Y/tVY1jp4D9/S+KcYy2pL+0hHEEJzK/Y5icAphDZIea3K\n",
       "/JQBPyA91TIKuNs5f1LXtQASPYGjgJ8DTwFn2TxVWuU6nJTk7ggtw6xSA7zSil7zFJj5Brx5e5rj\n",
       "1GWg/co5jS2voQZqXI+juOTa/bl25/+xzvPvstJuh/uyRk8ybYwInEJoGhE4hdCGKa8OwAakQGko\n",
       "0Js0j2AU8G/nPK2kckRvYBgpaPoncLbN/0prhLqQ0hBsBeyK6+7Nas2O0KVrDebeW7bgkdXuZfC1\n",
       "b7PK4TnnZ9V/ZfkicAqhaUTgFEIbo7y6AVuTAqVdSFlzR5GG4Z52znNKLkv0Jc2XOBS4BTjH5s3S\n",
       "G6O5Ey4/AQ7GLWdR3Go6U6fuO5h7/9KJ2Z3vYNeTf+fT651nVq4InEJoGhE4hdAGKK/epKdrhpCe\n",
       "TnmBbIkT5+atBVV6eWJV0kTLPUlPvpxn80EDC9kMuBH4C3AmLj1gawuGa7j68+aft+WBY59mw49u\n",
       "Y7d9rvQhj1ervgicQmgaETiF0IzKSdqovFZj/hDcd4EHSD1L92QZqxvRHtYGTgF2AC4BRth82sBC\n",
       "RFp09PfAIbiMJV/agNu0W+8eTLl5PZ7/4bXs/8ilHLHXy1674musReAUQtOIwCmEZjT/abAHDq3v\n",
       "KTDl1RHYiPmTuxdn/hDcQ8659jxOJbWDDUg5mDYhZR2+2F5omYJSCupKWol8I+DHuOG9XW3VvRq8\n",
       "6Qq8f6Pwcpdw1F8v4thj630KsQEicAqhaUTgFEIzmJ+0cfNOcOlqcMQb8OjsmkkbldeipFQBQ0hD\n",
       "cRPJUgYA/23IfKWF24BIa0L9Blgd+CNw2bwM3w0vcAXgVuBd4FDsqY1tW5sl6WG2OK4HUw5djxe+\n",
       "35hlW+ooOgKnEJpAsc9JZA4PoaomXArdJkHHPxUkbfwlTLhNefUhBUlDSSt4P0MKlM5wzm+XW3MW\n",
       "MO1E6mFaCjgbuNZmZhmF/pCUNPN84I+VDAjaFNtbwgjSFkJoQyJwCqGKbFsaYOiwBOz5Ciy5Ej+c\n",
       "tCtbcQLwHeB+4AbgYOf8RSXqzJZF2YMUMJmUtPJWm8avD5fmMx1H6rU6EPufFWhqCPWSdCXwvu3f\n",
       "NXdbQoAInEJoAl0GssVNt7L1pC14ebHpvN2jP5AD/uOcG9/7U4NEF9JyBycDnwK/Bu4te36N1A34\n",
       "K2li+ibYb5XZ1BAawtS+qOtCJI0BrrZ9WWMqqlSQJmll0oK9ndzOnjJtDyJwCqHahr86DdgU2Jt1\n",
       "vn7et0yt6PCWxKLAT4ATgVeBI4CHKzIhOf0CuC0rd1NcWiLN0HqV8wRoJcuoWWSJ57W0oeOYR9YG\n",
       "xWKVIVSR8tqDtM7bDs75OecqOUmYxSV+TfrLdhCwu832Nv+pUND0I9Lq61cDB0TQ1F703w12ODot\n",
       "adP0ZUhaT9JzkiZLugHoWnCsl6S7JX0iaZKkuyQtnx37A7AFcGG2yO2IbP8Fkt6T9JWkZyVtXqTe\n",
       "I4H9gF9l19+Z7e8r6daszrckHVdwzYZZmV9J+ljSudmhh7OfX2ZlbdTQ9yG0YLabZUtVN0/dscXW\n",
       "FBvD2ZLhfMJw1q1ouXhp8Bngz8BXg9eqaNtBhhMNEwxbNff7GFvN//64GmXAckfC6mPh8NdhjtPP\n",
       "1cfCckeWXm55ZQBdSE9rHg90BHYHZgK/z44vCfyYFEx1B24Cbi+4/iHgsBpl7g/0InUU/BKYAHQp\n",
       "Uv8Vc+vKXncA/gv8ljRCswrwJrBddvwJYP/s34sCG2X/XgmYA3Ro7v9fYmv8VuyzFj1OIVSB8lqL\n",
       "tATJfs75hYqUKfpJnA+8TlqLbkObA23GVqL8rJLFSE/N7QNshP1QxcoOLdyES2FWLj35Oe8J0NPS\n",
       "/iYrY2PSvKALbH9r+1bS06YA2J5k+3bbM5zSYJxJSrVRaIHhMdvX2v7C9hzb5wGLkNJyFFN4/QZA\n",
       "b9tn2J5t+23g76TPB6SgbqCk3ran2X6qljJCGxOBUwgVprz6AaOBE5zzv8ouTwyQuBR4CZgNrG3z\n",
       "M5vKTtKW+pP+gp4BbIH9XkXLDy2a7ayzscMSsNcroF7Z7pKHfStQRl/gwxr73iULRCQtKumvkt6R\n",
       "9BXwH2DxNKdqfjMKL5Z0oqRXJH0p6QtSQtneJbZnJaCvpC/mbqSHLpbJjh8OrAa8KulpSTuVWG5o\n",
       "xWJyeAgVpLyWAO4FLnLO15RVlliH9CW9LXAxMNCmUUuslFDZDsBVpOVTLqYBvyxDW9JlIDxwKLx5\n",
       "e5qf1GVgE5cxAVi+xr6VgPHZv08gBSob2v5E0rrAc6TAaqGn7yRtAZwEbG17bLZvEsV7hGr+f/8e\n",
       "8Lbt1Wo92R5PmheFpN2BWyQtWUs5oQ0pO3CSdDkpyd4nttfJ9i1JWvRzJeAdYC/bX5ZbVwgtmfJa\n",
       "BLidNM/ij40uR2xMysG0AXAe8FObKRVp5MKViRScHQvsgf1IVeoJrYL9ytkFL+tcGqhKZTwOzJY0\n",
       "jLSG4i5NhsJkAAAgAElEQVSkz8G/s+PdgenAV9nvmVyN6ycC/Qte9yD10n4mqQtpfcaeddQ/EVi1\n",
       "4PXTwBRJvwJGkobmvgN0tf2spAOA+21/CnxFCpjmkNKBzMnaEssRtTGVGKq7grRQaKFTgAeyKP3f\n",
       "2esQ2izl1YHUY/M58IuGPj0nIYkfSfyblBDzPmBVmz9WMWjqAdxCWuZlgwiaQnOzPQvYDTiE9Fna\n",
       "i7S8z1znA92Az0hB1r0s2LtzAbBH9sTd+aTP0X3AG6Q/4qeTepGKuQxYMxuWu80pB9POwLqkp1c/\n",
       "Bf7G/OBre+BlSVOAPwP72P7G6QnUPwCPZWVt2Ii3I7RQFVmrLkv2dVdBj9NrwA9tT5S0LDDG9ho1\n",
       "rrFjraTQRiivc4ENge0asgivRAfSF/OpwBLAWcB1NrOq0tD5Fa8G3AE8ChyH/U1V6wsVE2vVhdA0\n",
       "mnqtuj62J2b/ngj0qVI9ITQ75fULYDCwealBk0QnYE9SwDSL9NfpHS5nWZRSSTsDlwO/xf5bfaeH\n",
       "EEKYr+qTw21bUq3dWpKGF7wcY3tMtdsTQiUpr71JuWE2K2WtOYlFgINIy6JMIE1cvd9ugsmkUgfg\n",
       "d6TM4kOxn6h6naFskgaREpyGEFqAag7VDbL9saTlgIdiqC60NcprECkB3zbO+aU6zxWLkQKWE4H/\n",
       "AWfaNN2cImlxUgbwJUmTwD9usrpDRcVQXQhNo9jnpFp5nEYBB2f/Ppg0lyKENkN5rUN6cnTvEoKm\n",
       "pUnZh7cAhtoMbuKgaU3S00HvAVtH0BRCCI1Xdo+TpOtJmVt7k+YznQbcSfpLfEWKpCOIv3hCa6W8\n",
       "VgAeA37lnG+o81zRg5Se4F6bslZcbxRpN+CvwEnYVzZ5/aHioscphKZR7HNSkaG6xogPbmiNlFcv\n",
       "4BHgCuf8pzrPFV2Be0h5XI5qknlM8yvvSEpmeQCwO/azTVZ3qKoInEJoGk39VF0IbY7y6koadn6A\n",
       "lJiy+LmiI3AtMAk4pomDpl7AdaR8Nxtgf9JkdYcQQhsXa9WFUEB9tKMG6j6tpTEaqPvURzvCvASX\n",
       "VwMfk9agKxoISYiU9bgncECTpBiYX/l3gWeB14BtI2gKIYTKih6nEDLqox3pxwUMYcC8naPorz6C\n",
       "o9keWBrYwTnPqaeoM0iZhn9k03SJJaW9gQuBn2Nf22T1hlBFkq4E3rfd9HMEG0HS/sBBtrev5Lmh\n",
       "5Yg5TiFkNFD3cQALf4GN4nWGMJuU4LLONRclfg78DNjc5rPqtHShSjuRMo7vAfwY+4UmqTc0i/Y2\n",
       "x0nSFaTA6bQSzh0DXG37skbWdSWtKEgL1RVznEKoTxe61rq/OysDA0oImg4kJcNsyqCpN2ltOwPr\n",
       "Y3/eJPWGNkl9tCM9GUYXujKTGUxmhCd6dFOXUVuxJZ5X1Z4ASZ1sz65mHaHlizlOIcw1k9qXS/mM\n",
       "Z5zzB3VdKrET8Edge7vORUQrR1oPeIaUI2pwBE2hHPOGqg9ge/bihxzA9vTjgrnz/JqsDGk9Sc9J\n",
       "mizpBpj/B42kXpLulvRJtpDvXZKWz479gZQr7UJJUySNyPZfIOk9SV9JelbS5kXqPRLYD/hVdv2d\n",
       "2f53JP1K0kvAFEkdJZ0iaXzWxrGSdi0o5xBJjxS8niPpp5LeyBb8vbCOc7eT9LqkLyVdJOk/kg4v\n",
       "9b0LTSMCpxDmmswIRjF+gX338CGfcVZdl0lsDlxJSm75avUauEClBwL/BE7GPpn4KziUqyfDFpjf\n",
       "BzCEAfTkuKYqQ1IX0pOrVwG9gJuB3ZnfkyTgMlKOwBWB6aR5fdj+DSlVyDG2e9gell3zNPC9rLzr\n",
       "gJuzehbgtG7jtcA52fVDCw7vQ1qPcgnb3wLjgc1t9wTywDWS6lqTdSdgfeC7wF6SFpoSoNR7fDNp\n",
       "OaYlgdeBTahyL1pouAicQsh4okfzAcdzA//hQWZyOy/wLkfWNcwg8V3gNmB/m6eq3kipM9L5QI6U\n",
       "BfymqtcZ2odiQ9Vd6NaEZWwMdLJ9ge1vbd9K6lUFwPYk27fbnmF7KnAmKQFzoQWG9Wxfa/sL23Ns\n",
       "nwcsAqxeRxtqDgsaGGH7Q9vfZGXe4iwDv9NncBywUR1lnm17su33SQlx163lnB2Bl23fkbV1BOkp\n",
       "3tDCxBynEAodzZPAMqSs4BfUdarEqsC9wHE2/6x626RlSBn5p5HyM9W7qHAIJSs2VD2T6U1YRl/g\n",
       "wxr73iULZiQtCvwZ2J7UgwTQXdks3uz1Aj00kk4EDsvKNilNSO8S2zPX+zXKPAj4BbDy3DYAS9Vx\n",
       "fWEANA1YrJZz+gI1pwTUOUUgNI/ocQoho7y6kdZZvKeEoGlZ0lDZH2xurH7jtAEpP9OjwC4RNIWK\n",
       "q22oehRvMpmRTVjGBGD5GvtWYn4wdAKwGrCh7cVJvU1ifi9RzaBpC+AkYE/bS9juBXzFwr1KcxUb\n",
       "Fpu3X9JKwN+AY4AlszJfrqPMUn0E9CuoR4WvQ8sRPU4hAMqrI3ANaSHck+s8VywO3AdcbXNx9Run\n",
       "w4BzgCOxb696faFd8kSPVh/BNRxHF7oxk+lMZmRDnoirQBmPA7MlDSMlkd0F2AD4d3a8O2le01eS\n",
       "liQNWReaCPQveN0DmA18ls1rOoXU41TMRGDVetq4GCmQ+gzokPU+rV3PNYUKA71Co0kT24eSlmr6\n",
       "GbBsA8oNTSQCp9DuKS8BFwBLADvWleBSYm6v1COkteCq2DB1Ac4Htga2xG6aieeh3coCnLJSB5RT\n",
       "hu1ZSgtTX0pKJDsauLXglPNJE7w/Iw3pnQcMKTh+AXCVpKOAf5DSg9wHvAF8TRrmq+up18tIk8e/\n",
       "AB6yvVstbXxF0p+AJ4A5WT2PFp7Cgj1XNXuxCo/P+7ftzyTtCYwgTY6/ltTL3HRJdENJIgFmaPeU\n",
       "18mkx5C3dM5fFT1PdCJ9iX9NWkqlvgziZTRKywG3kH5BHISLtyu0L+0tAWZ7JakDaW7Vfrb/09zt\n",
       "aY+KfU5ijlNo15TXgcBRwOB6giaR5jUsAhxS5aBpE9KTRPeTMoFH0BRCO5DlcVpC0iLAqdnuJ5uz\n",
       "TWFhMVQX2i3ltS1wLrCVc/6ontPPAb4DbGMzszoNkoAjgdOBw7Dvrko9IYSWahPSUGQXYCyw69wU\n",
       "CKHliKG60C4pr/VIPTq7O+dH6jxXnAQcAmxhM6k6DVJXUiK/TUi9TG9UpZ7Q6sVQXQhNI9aqCyGj\n",
       "vFYB7gaOKiFoOpT02PHmVQya+pHmTr0PbIw9pSr1hBBCKFvMcQrtivLqTXrK5iznfGud54qhpMzE\n",
       "29lVSkQnbUlaEuJ2YM8ImkIIoWWLHqfQbiivRUmpBG53zhfWea74IemR6ME2lR82S/OZjgV+CxyM\n",
       "fV/F6wghhFBxETiFdkF5dQKuB95k/tMqtZ8r1iMttrmPzX8r3xh1A/5CWq9qE+y3Kl5HCPWQFIvH\n",
       "htAIETiFNi9LcHkhsCiwZz0JLgeQsvYeZfNg5RujlUiLAr8BbIr9dcXrCKEeMTE8hMaLOU6hPTiV\n",
       "tHL57s65aCoBieVI68/lbOqc/9Qo0tbAU6SMwPtF0BRCCK1P9DiFNk15HQL8BNjUOU8uep7oRUpP\n",
       "8HebSyvbCAn4OWkNvP2x/13PFSGEEFqoCJxCm6W8dgDOBgY55wlFzxOLAneRFhI9q7KNUBfSYqXr\n",
       "k1INvFPR8kMIITSpCJxCm6S8fkBafHNX5/xa0fNEZ+Am4G3gBHuhBTnLaIR6k/IzfQlshj21YmWH\n",
       "EEJoFjHHKbQ5ymtVUg/Skc758aLniQ7A5dnLwyq6/py0Jmk+0xOkTOARNIUQQhsQPU6hTVFeS5MS\n",
       "XJ7unO8oel5atPdPwCqkBJezKtcI7UDq7ToJ+6qKlRtCCKHZReAU2gzltRhpKZWbnfMl9Zz+a+BH\n",
       "wA9tplWmARIwDDiF1Mv0WEXKDSGE0GJUNXCS9A4wGfgWmGV7w2rWF9qvLMHlDcBrpGzcxc8VR5Ke\n",
       "tNvc5ovKNECdSbmiNiUltXynIuWGEEJoUard42RgkO3qLI4aAvMSXF4MdAZ+4pyLTvCW2B3IkXqa\n",
       "PqpMA7QkcAswjTQJvGjagxBCCK1bU0wOL5qhVgN1n/poxyZoQ2jbfgf8gJQVvOhcJYmtSakBdrIZ\n",
       "X5GapdVJk8CfA4ZG0BRCCG1btQMnA/+S9KykIxY6egDb048LIngKjaW8jgAOAXZyzlOKnifWJw3l\n",
       "7WnzQmUq17bAw8DZ2Cdif1uRckMIIbRY1R6q28z2BElLAw9Ies32I/OOPgT0YADfcIGkabbHVLk9\n",
       "oQ1RXodgcvy5391M/mAiuSLnidVJ6QmOsPlPZSrX0cBpwF7YlSkzhFpIGgQMauZmhBAyVQ2c7JSt\n",
       "2fankm4HNgTmB05bZT8/5UN/GUFTKJ3yOhD4A1f0PZvJO58BD/yTtHjugueJfqSlVE61ubP8itUJ\n",
       "OB/YmjSf6c2yywyhDtkflGPmvpZU5E+EEEJTqNpQnaRFJfXI/r0YsB3wv1pPnsn0arUjtD3Kaz+e\n",
       "WnQEp682nfcGHwsX94RBZ0lrjJX6HjnvPLEUKWi6yOaK8itWL+BeYADpybkImkIIoZ2p5hynPsAj\n",
       "kl4gTZ692/Y/FzprFG8ymZFVbEdoQ5TX3sCfWH/a5nw7+xTo2DU9f9CxK8w+DSZcCiAxN6fTPTZ/\n",
       "LL9iDSRlAR8L7Iz9VdllhhBCaHWqFjjZftv2utm2tu2FF0+9hvv4gGGe6NHVakdoO5TXnqRhsu38\n",
       "e48FGTosAXu9AuoF2LYlupDWiHsVOLn8irUV8CjwZ+yfY88uu8wQQgitUrNmDvc4D27O+kProbx2\n",
       "A0YC2zvnbMi3y0B44FB483bo/2PoMjBbf+5KYAZwZNmL9kpHAqcD+2I/WFZZIYQQWj25eK7A6lYs\n",
       "2XbRHE8hzKW8hgJ/A3Zwzs8XPS+tPzcC+C6wg13G3Lk0CfxcYDBpaG5co8sKoYLiuzOE5hVr1YUW\n",
       "TXntQgqadqwraMr8DtgcGFRm0LQ4KedTJ2Bj7MosyxJCCKHVa4rM4SE0ivLaEbgM2Nk5/7fOc8VR\n",
       "wEGknqbGT9yW+pMmgb8JDI6gKYQQQqEInEKLpLy2J81V2sU5P1PnuWIv4DfAdjYTG1+ptgQeAy7C\n",
       "PjYmgYcQQqgphupCi6O8tgWuBnZ1zk/Vea7YmTRpfFubtxpfqQ4Dzgb2x36g0eWEEEJo0yJwCi2K\n",
       "8toauBbYzTk/XvQ80Zn0tNsBwFCblxpXoTqSAqZdgS2xX2tUOSGEENqFCJxCi6G8BgE3Ars750eL\n",
       "nidWAa4HPgfWs/m0cRWqB3AdsBiwEfakRpUTQgih3Yg5TqFFUF5bAjcDeznnh4ueJ/YmZaK/Edi5\n",
       "jKBpZeBx4CNg+wiaQgghlCJ6nEKzU16bAbcA+zjnh2o9RywKXEBaJX6wTZ1P2dVd4bz6zgJG0lzJ\n",
       "zEIIIbQ60eMUmpXy2gS4HTjAOf+71nPEOsAzQFfg+2UGTQdl9R2KPSKCphBCCA0RPU6h2SivjYA7\n",
       "gYOcW3gB6CwT+M+A3wMn2Pyj8ZWpA/AHYC9gEPYrjS4rhBBCuxWBU2gWymsD4C7gUOd830LHRS/g\n",
       "78AqwGY2bzS+MnUnpTdYkjQJ/LNGlxVCCKFdi6G60OSU1/eBu4HDnfM9Cx0XmwHPA+8Dm5QZNK0I\n",
       "PApMAraNoCmEEEI5InAKTUp5rQvcC/zUOd+1wDHRUeK3wK3AsTY/t/mm8ZVpY9LyKf8AfoI9s/Et\n",
       "DyGEEGKoLjQh5fVd4D7gaOd8xwLHRF/gGlIw/wObD8urTPsB5wOHYd9dVlkhhBBCJnqcQpNQXmsD\n",
       "9wPDnPOtCxwTOwHPAQ8BPyoraJI6IJ1Omgj+owiaQgghVFL0OIWqU15rAv8EfuGcb5q3XyxCyqW0\n",
       "O7CnzSPlVaTFgKuAZUmTwD8pq7wQQgihhuhxClWlvNYAHgBOcs43zNsvBpIyd69CWjal3KBpeeBh\n",
       "4GtST1METSGEECouAqdQNcprdeBfwK+d87Xz9osDSEHT5cBuNuUtdyKtz/xlWA7BbvyE8hBCCKEO\n",
       "MVQXqkJ5DSQFTb9zzv8AkOgOXARsCGxj82LjCldPYGC2rQX8FDgC+84KND2EEEIoKgKnUHHKqz/w\n",
       "byDvnK8AkPg+cANpOG19m6/rLkSLAQOA1ZgfJM3dFgPGA+OybWvsl6tyMyGEEEIBNddSXZJsW81S\n",
       "eaga5bUKMAY40zn/NVs2ZRjwG2CYzQ3zT1Y3oD8LB0YDSVm+32R+cDQOeCP7OSHWmAvtVXx3htC8\n",
       "InAKFaO8VialFPijc75YondXpl+5Bq+tOIJhI7bg0cIhtoFAH+AdFg6MxgEfYM9p+rsIoWWL784Q\n",
       "mlcETqF8UqeLNmCTf63KTbu9ymMHvsSEiSyz0Qy6fn95PqQj376jhQOjccB72LObte0htDLx3RlC\n",
       "84rAqS7pEfczgeuBf2PPauYWNR+pI7AC83uL5s09Mqz4QU86zujEm6tO6vDg5RzWfzQ7rr8kk066\n",
       "jJ9c3a7ftxAqrFV8d4bQhkXgVBdpCeBgYB/SXJxbSUHUo21yGEnqAPSlRmCUbasAn7Fgj9G469bm\n",
       "q58M5e/TO3MJw30TcB0wAzjQ5uNmuIsQ2rRW8d0ZQhsWgVOppFVIAdQ+wFKknEHXA/9tVROVJZEy\n",
       "a9c2Ibs/MJkawVG2jceetkBReS1Hmgh+OcP9OvBX4DzgjzZtL7AMoQVodd+dIbQxVQucJO1AWmS1\n",
       "I/B32+fUON56P/zSmsC+2WbSY/bXY7/SrO2aKwVHvak9OBpI6hGqGRi9QQqOppRURV7LAg8xq+v1\n",
       "/GF6H2AwsJ/NkxW+mxBCgVb93RlCG1CVwElpPszrwDbAh8AzwL62Xy04p/V/+FOA8gNSALU38Dmp\n",
       "F+pG7LeboP5eLBwUzR1iM7U/yj8O+8uyqs1rGeAhPh/wL0aOG0T6b32kTdFy1Uc70pNhdKErM5nB\n",
       "ZEZ4okeX044Q2qM28d0ZQitWrcBpEyBne4fs9SkAts8uOKdtffjT/KAtSEN5e5ASNF4P3Iw9oYxy\n",
       "e1B7YDQQWITan1YbB3xejSFE5bU05kHe3O5trrl/E+BU4O82RetSH+1IPy5gCAPm7RzFeD7g+Aie\n",
       "QmiYNvfdGUIrU63AaQ9ge9tHZK8PADayfVzBOW33wy91JvW27QMMAZ4jBVG3Yn9Ry/mLUjxLdg8W\n",
       "zJJduE1syvlVyqs3czo+xIsHijsvM3TY26be4UkN1H0cwPYLHbiG+zzOg6vS2BDaqDb93RlCK1Ct\n",
       "JVdK+mUuaXjByzG2x1SlNU0tPX5/L3Bvlh17R1IQ9Sekh4EngJWZHxwtBbzF/IDoSeDq7N8ftYQn\n",
       "+JTXknzT/TGeP3wZ7jvvBujwS5vpJV3cha5F9nerZBtDaIskDQIGNXMzQgiZagVOH5Jy/sy1AvBB\n",
       "zZNsD69S/S2HPZ2UxuDWbHHaocD3gBeAm0nB0fvY3zZfI+umE/suifu+xCt79OL+cw+yO9zaoAJm\n",
       "MqPI/tICrxDasewPyjFzX0vKNVtjQghVC5yeBQZKWhn4iDRxet8q1dV62JNJPUlXN3dTSqWNRg5k\n",
       "g8Wf5YNNpvKvc9b0nE7vNriQyYxgFP1rzHF6k8mMrGRbQwghhGqrSuBke7akY4H7SekILit8oi60\n",
       "DlruhaEMufJmvlz5BZ4+elPP6tqo5VE80aPVR3ANx9GFbsxkOpMZGRPDQwghtDaRADMsRKIzi318\n",
       "Dvvtcgxdpo5m6dd2c64VJfkMoQ2L784Qmle1huqaVOQIqhyJVej6xY0ctN0AlnjnOhaZclgETSGE\n",
       "EELS6gOnIjmC+quPiOCpYST2ovPUC/nZelNY/L1bkX8aQVMIIYQwX6sfqoscQeWTWBQ4n85Tt+aX\n",
       "K3xBty9fAo5wrvnTIIQQFhRDdSE0rw7N3YCyRY6gskisAzzDIl9255ReH9Dty7FE0BRCCCHUqvUH\n",
       "TpEjqFEkJHEU8CCLfXwev+7Vm46z3wcOj6AphBBCqF3rD5xSjqDxC+yLHEF1kugF3AIcwTIvbcVJ\n",
       "y+0JfAYc4lzLTcQZQgghNLdWP8cJ5j1VFzmCSiCxGXAtcAeDTvsdg06/AZgK7O+cG5WnKYTQdGKO\n",
       "UwjNq00ETqF+Eh2BXwPHAkcwXPeTep1mAvs651nN2b4QQmniuzOE5tXq0xGE+kn0Ba4hDc3+gOH6\n",
       "FLgJmE0ETSGEEELJWv8cp1AniZ2A50iLhP6I4foEuB4QsE8ETSGEEELposepjZJYBDgL2B3Y0+YR\n",
       "5dWJNL9pEWB35zyzOdsYQgghtDYROLVBEgOBG4D3gPVsJmVB09VAd+DHzvmb5mxjCCGE0BrFUF0b\n",
       "I3EA8DhwObBbFjR1BK4ClgR2i6AphBBCaJzocWojJLoDFwEbAdvYvAiQBU1XAH2AXZxz7QlDQwgh\n",
       "hFCv6HFqAyS+T5oAPhv4QUHQ1AH4O9APGOKcI5t6CCGEUIYInFqxbNmU44H7gZzN4TZfw7yg6W/A\n",
       "KqSepmnN2NQQQgihTYihulZKojdpHtOywMY2b847loKmS4DVgcHO+evmaWUIIYTQtkSPUyskMQh4\n",
       "HngN2LxG0CTgQmBtYEfnPLVZGhlCCCG0QdHj1IpIdAJOA34CHGpz/wLHU9A0AlgP2N45T2n6VoYQ\n",
       "QghtVwROrYTECsB1wAzg+zYfL3A8BU1/Jj1Vt61zntz0rQwhhBDathiqawUkdgWeBe4Bti8SNP0R\n",
       "2BzYzjl/1fStDCGEENq+6HFqwSS6An8CdgSG2jy50DkpaDob2BrYxjl/2bStDCGEENqP6HGqg8RA\n",
       "iYcljpBYoonr/g7wFLA0admUYkHTH4DtScNzk5qyjSGEEEJ7E4FT3d4l9fhsD7wrcYvEUIku1aow\n",
       "y810GPAw6em4vW2K9SLlgZ1JPU2fV6wNfbSjBuo+raUxGqj71Ec7VqrsEEIIoTWT7eapWLJtNUvl\n",
       "jSDRC9gDOBD4DnAzadHcJ20q8iZKLA78hZRKYG+bV4qem9dpwF7AVs7500rUDylooh8XMIQB83aO\n",
       "YjwfcLwnenSl6gkhNE5r++4Moa2JHqcS2Xxhc6nNlsCGwIekNeDGSQyXCgKNRpDYkLRsypfAhvUE\n",
       "Tb8B9gF+VMmgCYCeDFsgaAIYwgB6clxF6wkhhBBaoQicGsHmbZs/kHqe9gV6AY9JPCFxtMRSpZYl\n",
       "0UHiJOBu4Fc2R9kUXVNOeZ1C6vXa2jlPLO9OatGFrkX2d6t4XSGEEEIrE4FTGWxs84zN8aSFdE8H\n",
       "tgDekrhTYo/sybhaSfQBRgO7AhvY3FpXfcrrJOAwUtD0cV3nNtpMZhTZHwsEhxBCaPeqEjhJGi7p\n",
       "A0nPZ9sO1ainJbGZZTPaZl9gBeB24CjgI4lLJbaU5r/fEtuSlk15Fvihzbt1la+8fgEcSZrT9FHV\n",
       "bmQyIxjF+AX2jeJNJjOyanWGEEIIrURVJodLygFTbJ9XxzntYoKjRD9gP9LwWg/gWqALaYjvIJsH\n",
       "6y0jr+OBYcAg5/x+FZub6uujHenJcXShGzOZzmRGxsTwEFqG9vLdGUJLVc3AaartP9VxTrv68EsI\n",
       "+C4pgFoaONGm3ondyusY4ARS0PRedVsZQmjp2tt3ZwgtTTUDp0OBr0hDUSfYC2a0jg9//ZTXz4BT\n",
       "SEHTO83cnBBCCxDfnSE0r0bPcZL0gKT/1bINAS4BVgHWBSaQkki2OpIkrXKWpCb/klJeRwCnkiaC\n",
       "v9PU9YcQQghhYY1eq872tqWcJ+nvwF1Fjg0veDnG9pjGtqc6+u8G2x4NDzwD3NZUtSqvw4Df/X97\n",
       "9x9rdV3Hcfz54tLdBY1s6Ux+NJzCFm6JVs4ltsMIJIYyWkU5ptmGbVYytZbKH3d3rTCbEdL8A7K2\n",
       "5tBaqQNNFBWX5kZZYCTQBYoFebUaJbkkIN798f0uL/fec+/33nPP+XzPua/Hdsf5nh/f72uXne/e\n",
       "9/P5fN9fsqLpj406rpmVj6QKUEkcw8xy9ZqqOy8ievLHtwAfjohr+7yntMPN0uQbYdJKmDMeNsyE\n",
       "Fd3wwkk4ujbi1fV1PXaXrie7/9zc6Ix99TyWmTWfMp87zcaCEY84DeFbkmYDAfwJ+EKdjlMnPRtg\n",
       "whFouwcEtHXAyVuhp66jTurScuCbZCNNLprMzMxKpi6FU0RcV4/9NkpEhHRhgN7D2QtP8caUKVx+\n",
       "ZDHz2AV01+OY6tK1wN1kt1H5Qz2OYWZmZrXxTX6rkGbdDse7+cqBLTx9zira2uZx9WvTgDfJ1mxt\n",
       "Bn4ZnXGy5mN1aRmwBpgfnfFKrfszs9ZV9nOnWatz4TQM6pKAS4Gr85/pwBayImpLdJ7ecqHgPj8J\n",
       "rAMWRGfsGr20ZtaKmvHcadZKXDjVQF2aAiwmK6I+StazajOwOTpj/2CfzT+/lKx1w1XRGS/XM6uZ\n",
       "tYZWOHeaNTMXTqNEXZoIfIysiFoM/BN4jKyQerHvlJ66tARYDyyMztjR4Lhm1qRa7dxp1mxcONWB\n",
       "ujQO+CBvj0a9j15TesCVwP3AouiM36TKaWbNp5XPnWbNwIVTA6hLU3m7iLoS+A9Z0fTrmved3ZD3\n",
       "Ztrp4DjHOMq9viGvWesaS+dOszJy4dRg6tIZwMTojCFv8Dvkvs7VIqaylmu48P9PbmI/h1np4sms\n",
       "NY3Vc6dZWbhwamKaoS0s56p+LzzAltgXH08QyczqzOdOs7RGfJNfK4F2Oqo8P6HBSczMzMYEF04N\n",
       "JknS+asl1f4X43GOVXn+rZr3bWZmZv24cGq4Cz4BC2+CC5bWvKuj3MsmTu8XtYkDHGVdzfs2MzOz\n",
       "frzGqUGkyTfCpJUwZzxsmAkruuGFk3B0bcSr60e83+yqui/TzgSO8xZHWeeF4Wata6ydO83Kpi43\n",
       "+bWB9GyACUeg7R4Q0NYBJ2+Fnodr2WteJLlQMjMzawBP1TVIRAQoYNxZ8OndoHfnT6cZ8jMzM7Nh\n",
       "84hTQ7XPgK03wIFHsjVO7TNSJzIzM7PivMbJzKyJ+Nxplpan6szMzMwKcuFkZmZmVpALJzMzM7OC\n",
       "XBOEmuAAAATaSURBVDiZmZmZFeTCyczMzKwgF05mZmZmBblwMjMzMyvIhZOZmZlZQS6czMzMzApy\n",
       "4WRmZmZWkAsnMzMzs4JcOJmZmZkVNOLCSdKnJL0i6b+SLu3z2h2S9knaK2lB7THNzMzM0qtlxGkX\n",
       "sBT4Re8nJc0ClgGzgIXAfZKadmRLUiV1hiKcc3Q55+hqhpzNkNHM0htxQRMReyOie4CXlgAPRsSJ\n",
       "iDgI7AcuG+lxSqCSOkBBldQBCqqkDlBQJXWAgiqpAxRUSR2ggErqAGZWfvUYCZoMHO61fRiYUofj\n",
       "mJmZmTXU+MFelLQVeO8AL90ZEZuHcZwYViozMzOzElJEbTWNpG3AbRHx23z7doCIuCvf3gJ0RsT2\n",
       "Pp9zMWVmNgIRodQZzMaqQUechqH3l3gTsFHSd8im6GYAv+r7AX/xzczMrNnU0o5gqaRDwOXA45Ke\n",
       "AIiI3cBPgN3AE8BNUeuwlpmZmVkJ1DxVZ2ZmZjZWJO2vJOnrkl6WtFPSM5KmpcxTjaRvS9qTZ31Y\n",
       "0rtSZxrIYE1Jy0DSwrwp6j5JX0udZyCSfiDpdUm7UmepRtI0Sdvy/+vfS7o5daaBSOqQtD3/fu+W\n",
       "tDp1psFIapO0Q9JwLnxpKEkHJf0uz9lvCYSZ1V/qxpR3R8TFETEbeBToTJynmqeAiyLiYqAbuCNx\n",
       "nmoGbEpaBpLagO+RNUWdBXxW0vvTphrQD8kyltkJ4JaIuIhsqvyLZfxdRsQxYG7+/f4AMFfSnMSx\n",
       "BrOSbIlBmYfhA6hExCUR0cz98cyaVtLCKSL+1WvzTODvqbIMJiK2RsSpfHM7MDVlnmoGaUpaBpcB\n",
       "+yPiYEScAB4ia5ZaKhHxPPCP1DkGExGvRcTO/PGbwB6y/mmlExH/zh+2A23AkYRxqpI0FVgEfJ/T\n",
       "L3Ypo7LnM2tpqUeckPQNSX8GrgfuSp2ngM8DP08doglNAQ712nZj1FEgaTpwCVlBXzqSxknaCbwO\n",
       "bMsvHimjNcBXgVNDvTGxAJ6W9JKkFanDmI1Fo9WOoKqhmmhGxCpgVd7/aQ1wQ70zDaRIs09Jq4Dj\n",
       "EbGxoeF6GcWmpI1W5umPpiTpTOCnwMp85Kl08pHa2fm6wCclVSLiucSxTiNpMfDXiNjRBPeruyIi\n",
       "eiSdA2yVtDcfJTWzBql74RQR8wu+dSMJR3KGyinpc2RD+fMaEqiKYfw+y+YvQO/F/9M4/dY8NgyS\n",
       "3gH8DHggIh5NnWcoEfGGpMeBDwHPJY7T10eAayQtAjqASZJ+FBHXJc7VT0T05P/+TdIjZFPgLpzM\n",
       "Gij1VXUzem0uAXakyjIYSQvJhvGX5Atem0HZ1kG8BMyQNF1SO7CMrFmqDZMkAfcDuyPiu6nzVCPp\n",
       "bEln5Y8nAPMp4Xc8Iu6MiGkRcT7wGeDZMhZNkiZKemf++AxgAdkFIWbWQKnXOK2WtCtfA1EBbkuc\n",
       "p5p1ZIvXt+aXAd+XOtBAqjUlLYOIOAl8CXiS7MqlH0fEnrSp+pP0IPAiMFPSIUlJpo6HcAWwnOwq\n",
       "tR35TxmvBDwPeDb/fm8HNkfEM4kzFVHWaeVzged7/T4fi4inEmcyG3PcANPMzMysoNQjTmZmZmZN\n",
       "w4WTmZmZWUEunMzMzMwKcuFkZmZmVpALJzMzM7OCXDiZmZmZFeTCyczMzKwgF05mZmZmBf0PM6oY\n",
       "ijHHSmwAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7ff4c776f710>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# optimal d = 6\n",
    "def plot():\n",
    "    X = construct_poly(x,6)  \n",
    "    Xt = construct_poly(xt,6) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 1.0'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    #plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "   \n",
    "    \n",
    "    w = wRidge(X,y,0.1)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 0.1'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    \n",
    "    \n",
    "    w = wRidge(X,y,10.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 10.0'+\"  MSE: %f\" % MSE(X.dot(w),y))\n",
    " \n",
    "  \n",
    "    \n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
    "plot()\n",
    "print(\"Minimum MSE: 0.6275\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hand in printed copy of completed notebook."
   ]
  }
 ],
 "metadata": {
  "annotations": {
   "author": "",
   "categories": [
    "intelligent-systems-1-2015"
   ],
   "date": "2015-04-30",
   "location": "Beginning of next lecture",
   "parent": "IS_SS2015",
   "submission_date": "2015-05-07",
   "subtitle": "Exercise Sheet 3, Linear Regression",
   "tags": [
    "IntelligenSystems",
    "Course"
   ],
   "title": "Intelligent Systems 1 - Summer Semester 2015"
  },
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.4.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}