{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "preamble": true
   },
   "source": [
    "(Defining latex commands: not to be shown...)\n",
    "$$\n",
    "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
    "\\DeclareMathOperator{\\minimize}{minimize}\n",
    "\\newcommand{\\real}{\\mathbb{R}}\n",
    "\\newcommand{\\normal}{\\mathcal{N}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Exercise Sheet 3\n",
    "Maximus Mutschler & Jan-Peter Hohloch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Gaussian Algebra (25 Points)\n",
    "\n",
    "Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
    "\n",
    "\\begin{equation}\n",
    "   \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Proof\n",
    "\\begin{align*}\n",
    "   f(x) &= \\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\cdot e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}}\\cdot \\frac{1}{\\sqrt{2\\pi s^2}}\\cdot e^{\\frac{-(x-m)^2}{2s^2}}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\alpha}\\\\\n",
    "   \\text{where }\\alpha &= \\frac{s^2(x-\\mu)^2+\\sigma^2(x-m)^2}{2\\sigma^2 s^2}\\\\\n",
    "   &=\\frac{(s^2+\\sigma^2)x^2-2(s^2\\mu+\\sigma^2 m)x+s^2\\mu^2+\\sigma^2m^2}{2\\sigma^2s^2}\\\\\n",
    "   &=\\frac{x^2-2\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}x+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}+\\underbrace{\\frac{\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}_\\beta\\\\\n",
    "   \\beta&= \\frac{\\frac{s^4\\mu^2+s^2\\sigma^2\\mu^2+s^2\\sigma^2m^2+\\sigma^4m^2-\\left(s^4\\mu^2+2s^2\\sigma^2\\mu m+\\sigma^4m^2\\right)}{\\left(s^2+\\sigma^2\\right)^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{s^2\\sigma^2\\left(\\mu^2-2\\mu m+m^2\\right)}{2\\sigma^2s^2\\left(s^2+\\sigma^2\\right)}\\\\\n",
    "   &=\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}\\\\\n",
    "   \\Rightarrow f(x)&=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-(\\alpha-\\beta)-\\beta}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}\\cdot e^{-\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}}\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi(s^2+\\sigma^2)}\\sqrt{2\\pi\\frac{\\sigma^2s^2}{s^2+\\sigma^2}}}{2\\pi s\\sigma}\\normal\\left[x;\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2},\\frac{\\sigma^2s^2}{s^2+\\sigma^2}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi\\cdot 2\\pi\\sigma^2s^2}}{2\\pi s\\sigma}\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\hspace{2cm}\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[m,\\mu,\\sigma^2+s^2\\right]\\hspace{2cm} q.e.d.\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
    "\n",
    "Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
    "\n",
    "\n",
    "Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\left(2\\pi\\right)^{-\\frac{k}{2}}\\left|\\Sigma\\right|^{-\\frac{1}{2}}\\cdot e^{-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)}\\text{, where: }k=n\\text{ number of measurments},\\ \\Sigma=\\sigma I,\\ \\mu=X\\vec w$\n",
    "\\begin{align*}\n",
    "    &\\frac{d}{d\\vec w} \\left(2\\pi\\right)^{-\\frac{k}{2}}\\left|\\Sigma\\right|^{-\\frac{1}{2}}\\cdot e^{-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)}\\\\ &=\\underbrace{\\left(2\\pi\\right)^{-\\frac{k}{2}}\\left|\\Sigma\\right|^{-\\frac{1}{2}}\\cdot e^{-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)}}_{>0}\\frac{d}{d\\vec w}\\left(-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)\\right)&\\overset{!}{=}&0\\\\\n",
    "   \\Leftrightarrow &\\frac{d}{d\\vec w}-\\frac{1}{2}\\left(\\vec{y}-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec{y}-\\mu\\right) &=&0\\\\\n",
    "   \\Leftrightarrow & \\frac{d}{d\\vec w}\\left(\\vec{y}-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec{y}-\\mu\\right) &=&0\\ (\\Sigma\\not=\\mathcal{O})\\\\\n",
    "   \\Leftrightarrow & \\frac{d}{d\\vec w}\\left(\\vec{y}-\\mu\\right)^T\\left(\\vec{y}-\\mu\\right) &=&0\\\\\n",
    "   \\Leftrightarrow & \\frac{d}{d\\vec w}\\left(\\left(\\vec{y}-X\\vec{w}\\right)^T\\right)\\left(\\vec{y}-X\\vec{w}\\right) + \\left(\\vec{y}-X\\vec{w}\\right)^T\\frac{d}{d\\vec w}\\left(\\vec{y}-X\\vec{w}\\right)&=&0\\\\\n",
    "   \\Leftrightarrow & -X\\left(\\vec{y}-X\\vec{w}\\right)-\\left(\\vec{y}-X\\vec{w}\\right)^TX^T &=&0\\\\\n",
    "   \\Leftrightarrow & -X\\left(\\vec{y}-X\\vec{w}\\right)-X\\left(\\vec{y}-X\\vec{w}\\right)&=&0\\\\\n",
    "   \\Leftrightarrow & X\\left(\\vec{y}-X\\vec{w}\\right) &=&0\\\\\n",
    "   \\Leftrightarrow & \\vec{y}-X\\vec{w} &=&0\\\\\n",
    "   \\Leftrightarrow & \\vec{y}&=& X\\vec{w}\\\\\n",
    "   \\Leftrightarrow & X^T\\vec{y} &=& X^TX\\vec{w}\\\\\n",
    "   \\Leftrightarrow & \\vec{w} &=& \\left(X^TX\\right)^{-1}X^T\\vec{y}\\hspace{1cm} q.e.d.\n",
    "\\end{align*}\\\\\n",
    "Warum lautet die Formel dann nicht einfach w=X^{-1}y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Linear regression (50 Points)\n",
    "\n",
    "In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
    "\n",
    "1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
    "2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Loading the required packages\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.html.widgets import interact\n",
    "import csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def wRidge(X,y,lamb):\n",
    "    # Change the following line and implement the ridge regression estimator wRidge\n",
    "    return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Read ex1_train.csv and assign the first column and \n",
    "# second column to variables x and y respectively.\n",
    "\n",
    "a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
    "x= a[:,0]\n",
    "y= a[:,1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4) Plot the training data with appropriate labels on each axes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x105203890>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEZCAYAAACTsIJzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAFmxJREFUeJzt3XuQZGd93vHvo1tWWIsEBmNZ7CIsQYJYQICREZcwQBzE\n",
       "xoHIGDCOMRi05Spgl4oJZQQJkkM53BLC7lJ2EgHiZm4yBiOQZGSsNUpUXAQISxFClsKyKwwSV7Eb\n",
       "dhNdfvnj9LJ9RnPpnpnu0z3z/VRNTZ/3dJ/+zZZ0nn7f97ynU1VIknTIEV0XIEmaLAaDJKnFYJAk\n",
       "tRgMkqQWg0GS1GIwSJJaDAapA0nek+QNXdchzcVg0KqVZHeSp4/hfc5P8v4hX1a9n0GOvyvJS4ev\n",
       "TFoag0Gr2cAn345kwOdN8t+gVchg0JqQ5MVJ/keStyb5YZL/neSsvv27krwxyReS3J7kE0nu09s3\n",
       "k2TvrOPtTvL03jHOBZ6fZF+Sr87z/o9O8pUkP0nyYWBd3777JPlUktt6tV2c5KTevj8Gngy8o3f8\n",
       "Hb327Un29Gq9OsmTVvifTGuYwaC15AzgBuDngbcA75q1/4XA7wEnAncCOxY4VgFVVZcB/xH4cFWt\n",
       "r6pHz35ikmOATwDvBe4DXAQ8h8M9gfRq2dj7OQC8g+YNXgdcCby8d/xtvdd8EXhU73gfBC7qvY+0\n",
       "bAaD1pJvVdW7qrlB2PuAE5P8Qm9fAe+rquur6qfAvweel2SQ4Z6w8LDQ44Gjqmp7Vd1VVR8DvnRo\n",
       "Z1X9sKo+XlUHq2o/TdA8ZY73oO81f1ZVP6qqu6vqbcA/Av7xALVKizIYtJZ899CD3skf4Li+/f3D\n",
       "RXuAo4H7rcD7/hLw7Vlt36J3sk9yryT/rTc8dTvwt8Dxs0KpNc+Q5N8muT7Jj5P8CDh+hWqVDAap\n",
       "z8ZZj+8Avg/8H+Beh3YkORK4f99zF5sc/g5w0qy2B/W97lXAQ4Ezqup4mt5Cfy9kdig8GXg18Nyq\n",
       "OqGq7gPczuCT2dKCDAapEeB3kjwsyb2A/wBc1Bt2uhFYl2RzkqOBf0czdHPId4GTFxh2ugq4M8m2\n",
       "JEcn+Q3gcX37j6OZV7g9yX2B82a9/lbglL7t9TRzIN9PckyS1wP3XsofLc3FYNBaMdelqzXr8fuB\n",
       "99B8wj8G2AZQVbcDLwPeCdwC7Kc97HRR7/cPklx9jzeuugP4DeDFwA+A5wEf63vK24FjaXonVwGX\n",
       "zqptO/CbvSuW3g5c1vu5EdhNEyp7FvzrpSGkyy/qSfJu4F8At1XVI3pt5wPnAN/rPe3c3pUf0sgk\n",
       "uQJ4f1W9u+tapK513WO4EDhrVlsBb6uqR/d+DAWNi2P0Eh0HQ1VdCfxojl3+D6ouuMJYovsew3y2\n",
       "JvlaknclOaHrYrT6VdVTHUaSGpMYDH8KPBg4nWYS8D93W44krS1HdV3AbFV126HHSd4JXDz7OUns\n",
       "8kvSElTVokP1ExcMSU6squ/0Ns8Grp3reYP8cZMqyflVdX7XdSyV9XfL+rszzbXD4B+qOw2GJB+i\n",
       "WeV5v97dK88DZpKcTjMR+E3g9zssUZLWnE6DoapeMEezE4CS1KFJnHxeC3Z1XcAy7eq6gGXa1XUB\n",
       "y7Sr6wKWaVfXBSzDrq4LGIdOVz4vVZKa5jkGSerCoOfOiZt8liStrGTTZtiwbfFnNgwGSVrFmlA4\n",
       "cztccOqgN5VwjkGSVrUN25pQGJzBIEmr2vp1w77CoSRJq8LhcfT162DfQdi7o+q6S7quq3v7Dg77\n",
       "CoNB0tRrj6MfsuWUZBOGw94dsOWUYYaTvFxV0liM8hN98szL4NJn3HPP5suqLnnmSrzHNGv+7Tdu\n",
       "hUvP8nJVSRNh9J/o5xtHP+7Y5R97+oeperVeMhX3SpK0Vsx1ZcwFp8LmrcAKnGDnG0fff2C5R16L\n",
       "w1RelSRpDEb7ib43jn5Tu+2cm2HPzuUfe75Q27h1+ceeTPYYJI3B6D7RQzNUkmyi6YEcd2xz3D07\n",
       "p2GYahIZDJLGYK4rY1bqE33j0Dj6Sh3vsNGG2iQyGCSN3Gg/0Y/a6ENt0ni5qiQt4vDlntMWam2D\n",
       "njsNBklaIwY9d3pVkiSpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1\n",
       "GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqSWToMhybuT3Jrk2r62+ya5\n",
       "PMmNST6T5IQua5SktabrHsOFwFmz2l4DXF5VDwU+29uWJI1Jp8FQVVcCP5rV/Czgvb3H7wX+1ViL\n",
       "kqQ1rusew1weUFW39h7fCjygy2Ikaa05qusCFlJVlaTm2pfk/L7NXVW1ayxFSdKUSDIDzAz9uqo5\n",
       "z7tjk+Rk4OKqekRv+wZgpqq+m+RE4Iqq+iezXlNVlbEXK0lTbNBz5yQOJX0SeFHv8YuAT3RYiySt\n",
       "OZ32GJJ8CHgKcD+a+YTXA38JfBTYCOwGnldVP571OnsMkjSkQc+dnQ8lLYXBIEnDm+ahJElShwwG\n",
       "SVKLwSBJajEYJEktE73ATdJhyabNsGEbrF8H+w7C3h1V113SdV1afQwGaQo0oXDmdrjg1MOtW05J\n",
       "NmE4aKU5lCRNhQ3b2qEAzfbGrd3Uo9XMYJCmwvp1c7cfd+x469BaYDBIU2Hfwbnb9x8Ybx1aCwwG\n",
       "aSrs3QFbbmq3nXMz7NnZTT1azbwlhjQlmgnojVub4aP9B2DPTieeNQzvlSRJavFeSZKkJTEYJEkt\n",
       "BoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqQWg0GS1GIw\n",
       "SJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWo7quoD5JNkN/AS4C7ijqs7o\n",
       "tiJJWhsmNhiAAmaq6oddFyJJa8mkDyWl6wIkaa2Z5GAo4K+TXJ1ky+ydyTMvSzZt7qAuSVrVJnko\n",
       "6YlV9Z0k9wcuT3JDVV15ePelz4AtpySbqLruks6qlKRVZmKDoaq+0/v9vSQfB84A+oLhfOCkU2Hf\n",
       "9iQ/rapdHZQpSRMryQwwM/TrqmqxA/98Vf1gaWUtTZJ7AUdW1b4kPwd8BvijqvpMb381I00Az/vb\n",
       "qo/OjLM+SZpGSaqqFp27HaTH8Pkk1wAXApfWYkmyMh4AfDwJNDX+2aFQuKf9B8ZQjyStGYP0GI4A\n",
       "/hnwEuBxwEeBC6vqxtGXN29NvR7DOTfD57c5xyBJixu0x7BoMMw66NOADwA/B1wDnFtVVy25yiVq\n",
       "guGZl8GenYaCJA1mxYIhyf2Afw38LnAr8E7gYuBRwJ9X1cnLrnZIg/5xkqTDVnKO4SqaXsKzq+qW\n",
       "vvark/zXpRYoSZpMA80xVNXdY6pnIPYYJGl4g547F135PGmhIEkarUm+JYYkqQOLBkOSJ83R9sTR\n",
       "lCNJ6togPYadc7S9Y6ULkSRNhnmvSkpyJvAE4P5J/oDDt8Bej0NQkrRqLXS56jE0IXBk7/chPwF+\n",
       "c5RFSZK6M8jlqg+qqm+NqZ6BeLmqJA1vJVc+XzFHc1XV05Za3HIZDJI0vJVc+fzqvsfrgOcAdy61\n",
       "MEnSZBvqJno/e1Hypap63AjqGfT97TFI0pBWrMeQ5L59m0cAvwLcexm1SZIm2CBDSV/h8Nel3Qns\n",
       "Bl46qoIkSd1a0lBS1xxKkqThreRQ0rHAy4An0fQcrgT+tKoOLrtKSdLEGeRy1YtoFrV9gGb1828D\n",
       "x1fVc0df3rw12WOQpCGt5DqG66vqtMXaxslgkKThrdj3MQBf6d036dCBHw98eTnFSZIm1yA9hhuA\n",
       "hwJ7aeYYNgLfoLlCqarqkaMuco6a7DFI0pBWcuXzMzh8Z9VDao42SdIqMEiP4f1V9cLF2sbJHoMk\n",
       "DW8l5xg2zTrwUcBjl1qYJGmyzRsMSV6bZB/wiCT7Dv0AtwGfHFuFkqSxGmQo6U1V9Zox1TMQh5Ik\n",
       "aXgruY7hKRy+V9LPVNXnll7e8hgMkjS8lQyGT3E4GNYBZwBf9ot6JGm6rNjlqlX167MOvAHYvoza\n",
       "JEkTbJCrkma7BXjYShciSZoMg9xddWff5hHA6XhLDElatQZZ+fxlDs8x3AV8sKr+5+hKkiR1aZDJ\n",
       "52OBU2nC4aZJ+B4GJ58laXjLXvmc5Ogkb6G5ed57gfcBtyR5a5KjV65USdIkWWjy+a3AfYEHV9Vj\n",
       "quoxwC8DJwD/aRzFSZLGb96hpCQ3AQ+tqrtntR8JfKOqTh1ZUclZwNuBI4F3VtWbZ+13KEmShrQS\n",
       "N9G7e3YoAFTVXcA92ldKL3jeAZwFnAa8IImXx0rSmCwUDF9P8qLZjUleCNwwupI4g2aSe3dV3QF8\n",
       "GHj2CN9Pa0SyaXPyzMuS5+1qfm/a3HVN0iRa6HLVlwN/keQlHF638FjgXsDZI6zpJJoJ70NuAX51\n",
       "hO+nNaAJgTO3wwV9Q6BbTkk2UXXdJd1VJk2eeYOhqm5J8qvA04CH01yu+umq+uyIa1r4+llpSTZs\n",
       "a4cCNNubtwIGg9RnwQVu1cxMf7b3My7fBjb0bW+g6TW0JDm/b3NXVe0abVmabuvXzd1+3LHjrUMa\n",
       "nyQzwMywrxtk5fO4XQ08JMnJwD8AzwdeMPtJVXX+WKvSlNs3z8LM/QfGW4c0Pr0PzLsObSc5b5DX\n",
       "LeUmeiNVVXcCrwD+Crge+EhVfb3bqjT99u6ALTe12865GfbsnPv50tq16C0xJpHrGLQUzQT0xq3N\n",
       "8NH+A7BnpxPPWktW7It6JpHBIEnDW4kFbpKkNWgSJ5870ww1bNjWXMGy7yDs3eFQg6S1xmDocQGU\n",
       "JDUcSvqZ+RZAbdzaTT2S1A2D4WdcACVJYDD0cQGUJIHB0McFUJIErmOYdVwXQElavVzgJklqcYGb\n",
       "JGlJDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMk\n",
       "qcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWo5qusCpEOSTZthwzZYvw72HYS9O6quu6TruqS1\n",
       "xmDQRGhC4cztcMGph1u3nJJswnCQxsuhJE2IDdvaoQDN9sat3dQjrV0GgybE+nVztx937HjrkGQw\n",
       "aELsOzh3+/4D461DksGgCbF3B2y5qd12zs2wZ2c39UhrV6qq6xqGlqSqKl3XoZXVTEBv3NoMH+0/\n",
       "AHt2OvEsrZxBz50TFwxJzgfOAb7Xazq3qi6b9RyDQZKGNOi5cxIvVy3gbVX1tq4LkaS1aFLnGOwN\n",
       "SFJHJjUYtib5WpJ3JTmh62IkaS3pZI4hyeXAL86x63XA5zk8v/AG4MSqeums1xfwR31Nu6pq1whK\n",
       "laSplWQGmOlrOm8qJ5/7JTkZuLiqHjGr3clnSRrSoOfOiRtKSnJi3+bZwLVd1SJJa9EkXpX05iSn\n",
       "01yd9E3g9zuuR5LWlIkeSpqPQ0mSNLypHUqSJHXLYJAktRgMkqQWg0GS1GIwSJJaDAZJUovBIElq\n",
       "MRgkSS2TuPJZE6r5hrUN22D9uuY7mvfu8BvWpNXHYNBAmlA4cztccOrh1i2nJJswHKTVxaEkDWjD\n",
       "tnYoQLO9cWs39UgaFYNBA1q/bu72444dbx2SRs1g0ID2HZy7ff+B8dYhadQMBg1o7w7YclO77Zyb\n",
       "Yc/ObuqRNCredlsDayagN25tho/2H4A9O514lqbHoOdOg0GS1gi/j0GStCQGgySpxWCQJLUYDJKk\n",
       "FoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWox\n",
       "GCRJLQaDJKnFYJAktXQSDEmem+R/JbkryWNm7Ts3yd8nuSHJP++iPklay7rqMVwLnA18rr8xyWnA\n",
       "84HTgLOAP0my6no1SWa6rmE5rL9b1t+daa59GJ2cdKvqhqq6cY5dzwY+VFV3VNVu4CbgjLEWNx4z\n",
       "XRewTDNdF7BMM10XsEwzXRewTDNdF7AMM10XMA6T9mn8l4Bb+rZvAU7qqBZJWpOOGtWBk1wO/OIc\n",
       "u15bVRcPcahaoZIkSQNIVXfn3SRXAK+qqq/0tl8DUFVv6m1fBpxXVV+Y9TrDQpKWoKqy2HNG1mMY\n",
       "Qn+RnwQ+mORtNENIDwG+OPsFg/xhkqSl6epy1bOT7AUeD3w6yaUAVXU98FHgeuBS4GXVZZdGktag\n",
       "ToeSJEmTZ9KuShpYkjck+VqSa5J8NsmGrmsaRpK3Jvl672/4iyTHd13TMBZapDipkpzVWzj590n+\n",
       "sOt6hpXk3UluTXJt17UMK8mGJFf0/pu5Lsm2rmsaRpJ1Sb7QO99cn+SNXde0FEmOTPLVJAteADS1\n",
       "wQC8paoeVVWnA58Azuu6oCF9Bnh4VT0KuBE4t+N6hjXnIsVJleRI4B00CydPA16Q5GHdVjW0C2nq\n",
       "n0Z3AP+mqh5OM4T88mn696+qg8BTe+ebRwJPTfKkjstailfSDNUvOFQ0tcFQVfv6No8Dvt9VLUtR\n",
       "VZdX1d29zS8AD+yynmEtsEhxUp0B3FRVu6vqDuDDNAsqp0ZVXQn8qOs6lqKqvltV1/Qe7we+TrNu\n",
       "aWpU1U97D48BjgR+2GE5Q0vyQGAz8E7aF/3cw9QGA0CSP06yB3gR8Kau61mGlwCXdF3EKncSsLdv\n",
       "28WTHUlyMvBomg9EUyPJEUmuAW4FruhdLDNN/gvwauDuxZ440cGQ5PIk187x8y8Bqup1VbUReA/N\n",
       "Hz1RFqu/95zXAf+vqj7YYalzGqT+KeJVFhMgyXHAnwOv7PUcpkZV3d0bSnog8E+n6b5JSX4duK2q\n",
       "vsoivQWYjHUM86qqXxvwqR9kAj9xL1Z/khfTdO2ePpaChjTEv/80+DbQf4HCBtq3X9GIJTka+Bjw\n",
       "gar6RNf1LFVV3Z7k08CvALs6LmdQTwCelWQzsA64d5L3VdXvzvXkie4xLCTJQ/o2nw18tataliLJ\n",
       "WTTdumf3Jram2TQsOLwaeEiSk5McQ3MX3092XNOakSTAu4Drq+rtXdczrCT3S3JC7/GxwK8xReec\n",
       "qnptVW2oqgcDvwX8zXyhAFMcDMAbe8Ma19Dc8fBVHdczrJ00k+aX9y4f+5OuCxrGfIsUJ1VV3Qm8\n",
       "AvgrmqsyPlJVX++2quEk+RBwFfDQJHuT/F7XNQ3hicDv0FzN89XezzRdYXUi8De9880XgIur6rMd\n",
       "17QcCw6tusBNktQyzT0GSdIIGAySpBaDQZLUYjBIkloMBklSi8EgSWoxGKRZkqz4rRqSPCjJC1b6\n",
       "uNIoGAzSPY1icc+Dgd8ewXGlFWcwSPNIMpNkV5KLel+q9IG+fbuTvDnJ3/W+wOWUXvt7kjyn73mH\n",
       "bg//JuDJvRW/r5z1Pmcn+eve4xOTfCPJL4z+L5TmZjBICzud5stNTgN+OckTeu0F/LiqHknzBUBv\n",
       "72ufyx8CV1bVo6tqe/+Oqvo48J0krwD+O/D6qrpthf8OaWAGg7SwL1bVP1Rz75hrgJP79n2o9/vD\n",
       "wJmLHGexGw1upfkWv4NV9ZGlFCqtFINBWtj/7Xt8F/Pfqv5QT+FOev9fJTmC5tu+BrGhd/wH9O5E\n",
       "KnXGYJCW7vl9v6/qPd4NPLb3+FnA0b3H+4D1cx0kyVE0t6T+LeAG4A9GUKs0sIn+oh6pIzXP49nu\n",
       "k+RrwEHg0KWoFwB/2bs982XAoUtfvwbc1Wu/cNY8w7nA56rqqiR/B3wpyaeq6hsr8cdIw/K229IS\n",
       "JPkm8NiqmqovhJcG4VCStDR+otKqZY9BktRij0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySp5f8D\n",
       "oJt+CXUlyycAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1057281d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the input data here\n",
    "plt.plot(x,y,'o')\n",
    "plt.title('Input data')\n",
    "plt.ylabel('Output y')\n",
    "plt.xlabel('Input x')   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
    "Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def construct_poly(x,d):\n",
    "    ## Implement a method which given an array of size N, \n",
    "    ## returns an array of dimension (N,d)\n",
    "    poly=np.empty((len(x),1), float)\n",
    "    xr = np.reshape(x,(len(x),1))\n",
    "\n",
    "    for i in range(d-1):\n",
    "        poly = np.append(poly, (xr**i), axis=1)\n",
    "\n",
    "    return poly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def MSE(y_predict,y_true):\n",
    "    ## Implement mean squared error for a given input y and its predictions.\n",
    "    return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHGW5/vHvTVjSCCFgHAj7SABxREE5gCLHsKghKovK\n",
       "5sImwhFIOP6MJwIqAQWNCSiJqPyURUEQkM0gBFmSA6JEUOBwCCoJiYYtDQk7HQjJc/6oGqczzGSm\n",
       "p7unuqvvz3X1lemq6qpn5oJ7at566i1FBGZmli9rZF2AmZnVnsPdzCyHHO5mZjnkcDczyyGHu5lZ\n",
       "DjnczcxyyOFuZpZDDncbVJIWSnpN0lu7Lb9f0kpJW6bvN5d0jaRnJD0v6SFJR6brtk63fanb6+Be\n",
       "jnmIpD9IekXSrD7qO7XbPl+VtELSRun6YZIuS+t6Jv16/bLPD5H0bUlPSHpR0l8kbZCuW0fS99N1\n",
       "SyWdL2nNss9uLmmGpCWSnpI0XdKQHmr8Zvr9793/n7y1Goe7DbYAHgMO71wgaUegkK7rdCnwD2BL\n",
       "YCPg88DibvvaICLWL3td3csxlwDnAt/ts7iIs8v3CUwGZkXE0nSTScAIoB3YBtg4XdbpDGB3YPeI\n",
       "GAZ8DliWrvsa8F6gA9gu/frrZZ+dBjwLjAR2Aj4EnFBen6RtgE8DT/b1vVhrc7hbFi4Djih7fyTw\n",
       "C0Bly3YBLomIUkSsjIgHImLmQA4WEbdHxK+Bpyr5nCSltf28bHEHcH1EvBwRLwLXp8uQtCFwMvDF\n",
       "iFiUHntuRLyWfvbjwPSIeD4iniUJ82O67fvKiHg9IhYDMzv3XeaHwERgeSXfi7Ueh7tl4R5gmKR3\n",
       "pMMOh5IEfvdtfiTp0M6hmh6ol+W1sifwNuCasmW3AJ+SNDwN808BN6XrdgTeAA5Oh1X+JmmVM+9u\n",
       "Na8BbF42rHML8BlJBUmbAfsBN//rg8mw07KIuBmzPjjcLSuXkpy9fxiYCzzRbf3BwF3AN4DH0jH5\n",
       "Xbpt86yk58pe29e4xiOBqyPi1bJl56f/LiEZQlkO/DhdtjmwAbAtsDXJ8MkkSfum62cCJ0saIWkT\n",
       "YDzJUNS66fpJwLuAF4FFwL0RcQNA+gvgLJK/DMz65HC3LARJuH+WnodkSIcuTomId5GMaz9AMgRS\n",
       "7q0RsWHZ62+1KlDSuiTh/PNuq34J/A1YDxhGcv2g86+OUvrvmRHxWkQ8BPwKGJsuPwu4P/1efg9c\n",
       "B7wREYvTIaBbgKtJwn4EsJGkyelnJwGXRsQ/y8uswbdqOeVwt0ykIfUYydDDtX1suwQ4B9g0HQoZ\n",
       "8GEr2PYgYElE/He35WOAC9JrAa8AF9AV3v+zuuNGxLKIGBcRm0fEKGApcF+6zQjgfcAPI2J5egH3\n",
       "krJ97w2MT4d7ngK2AK6S9NUKvidrIQ53y9IXgL0jotR9haTJkjokrZkOSXwJeDQinivfrD8HkbSG\n",
       "pKHAWsAaaUviWn18rPMviu7+B/iipKGSCsBxwIMAETGfZCjpNElrS9qB5HrCjWkdm6YvSdqdpFPm\n",
       "9HS/z5Jc8P1S2k45PK3hwXT9PiQXV99D0knzZHrsH/XnZ2Ctx+FumYmIxyLiL+WLyr4ukAxbPAfM\n",
       "JzlT3b/bLp7v1pP+n70c6gjgVZIg3JNk+OSCzpXpZ/coe78ZMJqew/0okjbGJ4DHScbWjyxbfziw\n",
       "FcmY/I3A1yOis7d+G+Bu4GXgYmBiRNyW/iwC+CTwCZKgfxR4Dfhyun5pRBTT12JgBfBc+teD2Zuo\n",
       "2od1SLoI+BhQjIgd02WTgGOBZ9LNThloG5uZmVWuFmfuF5OMQ5YL4NyI2Dl9OdjNzAZR1eEeEXeR\n",
       "/Oncna/km5llpJ5j7uMkPSjpwvTikJmZDZJ6hfuPSebe2ImkA+CcOh3HzMx6sGbfm1QuIoqdX0v6\n",
       "GTCj+zaSqruSa2bWoiKiz2HvuoS7pJER0TlJ00HAQz1t158CG5WkSRExKes6Bsr1Z8v1Z6eZa4f+\n",
       "nxhXHe6SriCZmnSEpEUkN2WMlrQTSdfMAuD4ao9jZmb9V3W4R8ThPSy+qNr9mpnZwPkO1YGbnXUB\n",
       "VZqddQFVmp11AVWanXUBVZqddQFVmJ11AYOh6jtUB3xgKZp5zN3MLAv9zc66XFA1M7Paam/X2LY2\n",
       "xvd3ew/LmJk1uPZ2je3o4LzJk/lofz/jcDcza3BtbYyfMIFRlXzGwzJmZhmSWIPkqV4b9fbaZZeO\n",
       "neDhivbrcDezhiAxZMMNJ5w9cuTdxxYKy3jttXj1n//83J0vvjjhLpJpTDpfT0ewPNtq30xiTWA4\n",
       "qwnpXl7DgVdInszV0+uJ5ctfeYbkcZP95nA3s0xJDAEOWXfdqyfvvPOVb/vGNx4fmq7a6OyzF425\n",
       "5551N3zppRNWACPT19skXmDVwH+K5OlUqyyL4E1P+epHPWsDG1J5SK8PvEDvIT0fuLeH5c/39cuq\n",
       "vX3hI1Oncl4lQzNuhTSzfuvs2CgUGFoqsaxYZNqCBXHTQPaVhvphJI8bXLLzzpusc+65i3fpvt3E\n",
       "icycMyf26/a5EXSF/Uhg027vRwKbAMt48y+BJax+GGQoyTTmvYX0kl6WvxDByoH8LPoj/dmP+9Of\n",
       "GONWSDOrmc6OjfKzx6lT2aa9XVQS8OnwxWHAN4AicBJwx7Bhi2f1tH2hQKH8fQQrgMXp64HVHEck\n",
       "Z+AjgZHDh39tzJZbXveJddahsGyZli1adOjvnn/+jKt5c0i/FFHRw9QHRfozvmnQ5pYxs9bQU8fG\n",
       "hAmMmjiRcUCf4Z6G+mdIztSfJnno+azOIN1tN5b19LlSqfKhFYB0v0uBpe3t2qqjgwNW/cV0ph5+\n",
       "+MybFixInmObN26FNLN+KRQY2svyQk/LO0msKXEk8AhwDMlEgh+K4I7yM+RikWlTpzKv/LNTpjC/\n",
       "WGR6tbX39ouprY1x1e67UfnM3cz6pVSq7MxaYi3gc8BpwCLgixG9z+uyYEHc1N4uJk5kXKFAoVSi\n",
       "VCwyfaBj+uUG+oupmTnczaxf0jPrbcrPgHs6s05D/fMkob4QOCaCO/tzjM5x5dpVnaj0F1MeuFvG\n",
       "zPqts2OjpzPrNNSPIAn1x4AzIrgry3o79XQxeMoU5s+dy/ha/GUwmPqbnQ53M6tK2hd+JHAq8ChJ\n",
       "qN+dbVVvtrpfTM3E4W5mdZWG+lEkof43klD/Q6ZFtQBP+WtmdSGxDnA0cAowFzg8gj9mW5V153A3\n",
       "s36T2AS4E5gHHBLBnIxLsl54WMbM+kViXZJH1N0YwZkZl9OyPOZuZjWTTkt7NcnshUc24u35rcJj\n",
       "7mZWS98lmazrMw725uBwN7PVkjgOOAB4fwSvZV2P9Y/D3cx6JfER4EzggxEszboe6z+Hu5n1SOJd\n",
       "wGXAJyNWndDLGp9nhTSzN0lbHm8E/jOC32ddj1XO4W5mq0hbHmcAF0Zwedb12MC4FdLM/iVtefw1\n",
       "8BJwlDtjGo9bIc1sICaTPEf0cAd7c3O4mxkAEscD++OWx1xwuJsZEh8FJuGWx9xwuJu1OIkdgUuB\n",
       "gyKYn3U9VhvuljFrYRIjSVoeT27EB2zYwDnczVqUxFuA3wA/jeCKrOux2nIrpFkLkhhC0vL4AnC0\n",
       "O2Oah1shzWx1JgPDgUMd7PlU9bCMpIskLZb0UNmyjSTdKunvkn4naXi1xzGz2pD4EvBx4FMRvJ51\n",
       "PVYftRhzvxgY023Z14BbI2I74Pb0vZllTGIM8E3gY255zLeqwz0i7gKe67Z4f+Dn6dc/Bw6s9jhm\n",
       "Vh2Jd5O0PH7aLY/5V69umY0jYnH69WJg4zodx8z6IW15nAGMd8tja6j7BdWICEk9XrCRNKns7eyI\n",
       "mF3vesxaTdryOAP4/255bD6SRgOjK/5cLVohJW0NzIiIHdP3fwVGR8TTkkYCsyLiHd0+41ZIszpL\n",
       "Wx6vAZ7HLY+50N/srNewzG+AI9OvjwSur9NxzGz1vgdsABznYG8tVZ+5S7oC+BDJk9EXk1yJvwG4\n",
       "CtgSWAgcEhHPd/ucz9zN6kjiBGA8ySyP3ZserEn1Nzt9h6pZDknsB1xEMsujO2NyxHeomrUoifeQ\n",
       "tiA72FuXJw4zyxGJTelqefxD1vVYdhzuZjlR1vJ4QQS/yroey5bH3M1yIG15vBZYChzjzpj88pi7\n",
       "WQNqb9fYtjbGFwoMLZVYViwybcGCuKkGu54KrA8c7GA3cLibDZr2do3t6OC8CRMY1bls6lS2aW8X\n",
       "1QS8xInAfiQtj57l0QCPuZsNmrY2xpcHO8CECYxqa2PcQPcpMRb4OjDWvexWzmfuZoOkUGBoL8sL\n",
       "A9lfWcvj/hE8Vk1tlj8+czcbJKUSy3pZXqp0X2UtjydF8Mdqa7P8cbibDZJikWlTpzKvfNmUKcwv\n",
       "FpleyX4k1gNuBH4SwZW1rNHyw62QZoMo7ZYZVyhQKJUoFYtMr+RiatryeB3wLPAFd8a0Hs8tY5ZD\n",
       "Ej8A3g2McWdMa3Kfu1nOSJwEfBT4gIPd+uJwN2sCEh8DTgP2cMuj9YfD3azBSewEXIJbHq0C7pYx\n",
       "a2ASm5G0PJ7olkerhMPdrEGlLY8zgB9FcFXW9VhzcbeMWQNKWx6vB4rAsW55tE5ZPyDbzKpzLrAu\n",
       "8B8OdhsIX1A1azAS44APk7Q8Ls+6HmtODnezBpK2PJ5KEuzPZ12PNS+Hu1mDKGt5/EQECzIux5qc\n",
       "x9zNGoDE5iSdMSdEcE/W9Vjzc7ibZays5fH8CK7Ouh7LB7dCmmUobXm8AXga+KI7Y6wvboU0aw7f\n",
       "B4YCX3KwWy35gqpZRiTGA/vilkerA4e7WQYkPg6cglserU4c7maDTGJn4GLc8mh15DF3s0EksQnJ\n",
       "BVS3PFpdOdzNBonEOsC1wEVuebR6cyuk2SCQEHAhMBz4dAQrMy7JmpSfoWrWWMYDu5BcQHWwW905\n",
       "3M3qTGJfks6Y90fwctb1WGtwuJvVkcQo4JfAYe6MscFU13CXtBB4EVgBLI+IXet5PLNGIjGMpDPm\n",
       "zAhmZV2PtZa6XlCVtAB4X0Qs7WGdL6habkmsQfKYvKeB4z21gNVKI11QdYBbKzoT2JCkM8bBboOu\n",
       "3n3uAdwm6T5JX+y+8n3v2/Du9naNrXMNZoNK4hDg88CnIng963qsNdU73PeIiJ2B/YATJe1ZvvKc\n",
       "c57/QEcH5zngLS/SqQXOBw6MoJh1Pda66josExFPpf8+I+k6YFfgrs71l1wCI0Yw6tVXOU/SqxEx\n",
       "u571mNWTRBvJOPtJEdyfdT2WD5JGA6Mr/Vzdwl3SusCQiHhJ0luAjwBnlG9z1FHJvwsX8sTixQ52\n",
       "a14SawPXAJdGcGXW9Vh+pCe9szvfSzq9P5+r55n7xsB1kjqP88uI+F1PG5ZKlOpYh1ldpVMLTAeW\n",
       "At/MuBwzoI7hHhELgJ362u6MM9pXPv74+pfVqw6zQfAl4IMkd6B6agFrCJneoTpxIjPnzp045OWX\n",
       "j980yzrMBkpiNHA6yZwxL2Zcjtm/ZD4rpMSewAVAh/uBrZlItAN/BD4XwW1Z12OtoZkekP17YG3g\n",
       "37IuxKy/JNYjmVrgOw52a0SZh3t6tn4JcHTGpZj1Szq1wC+A+4BpGZdj1qPMh2WSr9kCeADYPMKd\n",
       "M9bYJCaRtPbuFcFrGZdjLaaZhmWIYBHwZ+DArGsxWx2JTwFfAD7pYLdG1hDhnroYOCrrIsx6I/Fu\n",
       "4CfAQRE8nXU9ZqvTEMMyyXsKwBPAe9IzebOGITECuBc4LYLLs67HWldTDcsApGPtVwFHZF2LWTmJ\n",
       "tYCrgasc7NYsGubMPVnGbsBlwHbuebdGIXE+sDWwfwQrMi7HWlzTnbmn/gQsJ7mV2yxzEscB+wCf\n",
       "cbBbM2mocE/P1n1h1RpCevf0t0nO2F/Iuh6zSjTUsEyynJHAXJKe91cGvzIzkNgKuAc4OoKZWddj\n",
       "1qlZh2WI4CngbuDTWddirUniLSQP3TjHwW7NquHCPXUxno7AMpDOzX4x8BBwTsblmA1Yo4b7DKBD\n",
       "4u1ZF2It51RgK+A4d2xZM2vIcE+fGH8FcGTWtVjrkNif5MEbB0WwLOt6zKrRcBdUu9azM3Ad8HY/\n",
       "3cbqTaKD5DmVH49gTsblmPWqaS+odkqfHv88sFfWtVi+SWxEMjf7VxzslhcNG+4p97xbXUmsCVwJ\n",
       "3BDBL7Kux6xWGnZYJtmGEcA8YCvfRGL1IPEDYAfgYxG8kXU9Zn1p+mEZgAieBe4ADsm6FssfiWOA\n",
       "scBhDnbLm4YO95R73q3mJD4ATAYOiOC5rOsxq7VmCPebgXaJd2RdiOVD+ljHXwNHRfBI1vWY1UPD\n",
       "h3v65/JluOfdaiB9KMx1wLQIfpt1PWb10tAXVLu2pQP4HbClp121gUqnFvhl+vazvgPVmlEuLqh2\n",
       "iuBhkkfwfSTrWqyp/RewPXCsg93yrinCPeULqzZgEh8DTgYOjODVrOsxq7emGJZJtmdDYAHJdARL\n",
       "61eZ5Y3EDsCdJMF+d9b1mFUjV8MyAGm72s3A4VnXYs0jPSm4Afiag91aSdOEe+oSPDRj/SQxhGR2\n",
       "0ZkRXJh1PWaDqdnC/TZgY4kdsy7EmsJkYE3g/2VdiNlga6pwT9sgf4EnE7M+SBwBHAgc6qkFrBU1\n",
       "zQXVrs+xLfB7kgdoL699ZdbsJHYFfguMTttozXIjdxdUO0XwKPAoyYRPZquQ2BS4lqSX3cFuLavp\n",
       "wj3lnnd7E4mhJFML/CSCG7KuxyxLdQt3SWMk/VXSo5Im1nj3VwGjJdpqvF9rUunUAhcA/wDOyrgc\n",
       "s8zVJdwlDQF+CIwB3gkcLmmHWu0/gpdIepc/W6t9WtP7MvBu4GhPLWCWtInVw67AvIhYCCDpV8AB\n",
       "UNPpVS8Gpkn8wP8zt472do1ta2N8ocDQUollxSLTFi6MFcBXgd0jeCXrGs0aQb3CfTNgUdn7x4Hd\n",
       "anyMO4H1gfcCf67xvq0BtbdrbEcH502YwKjOZd/73lrvWLz4ug1KpYP2j+AfWdZn1kjqNeZe9zPp\n",
       "CFbiO1ZbSlsb48uDHeC//mv5VtttN+GJCO7Kqi6zRlSvM/cngC3K3m9Bcva+CkmTyt7OjojZFR7n\n",
       "58B9El+J4LVKi7TmUigwtKflw4c/9uxg12I2WCSNBkZX+rl6hft9wLaStgaeBA6lhwm/ImJSNQeJ\n",
       "YKHEg8D+wNXV7MsaX6nEsl6Wlwa7FrPBkp70zu58L+n0/nyuLsMyEfEGcBJwCzAXuDIi6vWsSve8\n",
       "t4hikWlTpzKvfNmUKcwvFpmeVU1mjarpph94835Yl2QY6F0RPFF9ZdaoJNYYNuzcyzfd9Mb9R4y4\n",
       "939ff/3lJcUi0xcsiJuyrs1ssPQ3O5s+3JN98VNgXgSTa7E/azwSbyGZNG4T4KAIihmXZJaJ3M4t\n",
       "04uLgaPTuxQtZyQ2B+4CXgb2drCb9a1eF1QH2x8BAbu3t2vD7je5+M/25pXO8HgtMB34nm9YM+uf\n",
       "XIR7BCFxyfrr//ibHR2MKu+FnjqVbdrbhQO++UgcRhLqx3oiMLPK5GLMPdkfm22//T4LfvKTO9bq\n",
       "vm7iRGbOmRP71epYVl8SawCnkzyUZf8IHsy2IrPG0Wpj7kTwxHrrLX2pp3WFAoXBrscGJu1++hXw\n",
       "EWA3B7vZwOQm3AFWrHh2UU/LfZNLc5DYjGTOoNeAvSJ4OuOSzJpWrsJ9yZJXJp15ZvvK8mW+yaU5\n",
       "SOwCzAGuAY6I6PluVDPrn9yMuXcaNuz8G7fe+ofbbbTRX58slSj5JpfGJ3EIcD5wXATXZV2PWSNr\n",
       "qZuYVt0vu5A8qWlUOnOkNaj0voRvAscAB0Zwf8YlmTW8lrugWubPwCvAnlkXYr2TKABXAPuRXDh1\n",
       "sJvVUO7CPb3J5WLgeCkfffx5I7Ep8N/ASnzh1KwuchfuqUuB7YBnJK6TOFFiO09PkD2J95JcOL0B\n",
       "+GyEO5nM6iF3Y+6rHoONgX2AD6evlcCtwG3A7Z6jZHBJfBr4MfAfEVyTdT1mzahlL6j2fjwEbA/s\n",
       "SxL0HwIW0hX2d0Xw6mDV00rSn/1pwHEkF07/knFJZk3L4d7n8VkT2JUk7PcFdgbupSvs/xLBiqzq\n",
       "y4v0wumFwCjggAieyrgks6bmcK+QxPrAv5Oc1e8LjARmkYZ9BPMzLK8pSYwErgcWAEd7fN2seg73\n",
       "KqUdHZ3j9fsCy0jO6G8F7ohgSYblNTyJnUkumv4M+Jan6jWrDYd7DaVjxu+ka7x+T+BRusL+bt8u\n",
       "n0iHu74AfBs4IcIPLjerJYd7HUmsDexGV9jvSPLAkM6wf7DV7o5Np+k9GPgWyTNtvxzBA9lWZZY/\n",
       "DvdBJLEBMJqusH8rcDtJ2N8WwcLMiquz9K+a/YCzgDeAU0m+Zw/DmNWBwz1DElvQ1YWzL/AiXWf1\n",
       "syJ4LsPyakZiT+Bskl9mXweuc6ib1ZfDvUGkwxXvouvC7B7AI3SF/R8jeC27CiuXXiw9G3gHMAm4\n",
       "zG2jZoPD4d6gJNYB3k9X2O8A3E1Xf/1DjXr2K7E9yZj6niTDMD9ttl9MZs3O4d4kJDYE9qIr7IeR\n",
       "jtUDt0bweIblASCxJcnUvAcA5wLTIngl26rMWpPDvUlJbE3Xhdl9gGfpOqufHcELg1hLG3AKcARw\n",
       "ATAlL9cLzJqVwz0H0vH6negK+92Bh+gK+3siWF6H424AfAU4EbgcOMvT8po1Bod7DqXztOxBV9iP\n",
       "Au6iK+znVjNen+7/JOCrwE3AGREsqLZuM6sdh3sLkBgB7E1X2K9DVxfO7RE82c/9rEVyV+k3gHuA\n",
       "b0Qwty5Fm1lVHO4tJr2Z6O10XZjdG3iSrrC/M4KXun1mCHAYcCYwHzgtgnurqaO9XWPb2hhfKDC0\n",
       "VGJZscg0P6DcrHYc7i0uDe730hX2/wbcT1fYv42knfFl4NQIZlV7zPZ2je3o4LwJExjVuWzqVOY9\n",
       "/DAnO+DNasPhbquQeAvwQbqeSrWC5AakGbXqq99tN82cPJmPdl8+cSIz58yJ/WpxDLNW19/s9AOk\n",
       "W0Tal35L+qqLQoGhvSwv1OuYZtazvD4g2zJQKvU87XGp5Id0mA02h7vVTLHItKlTmVe+bMoU5heL\n",
       "TM+qJrNW5TF3q6m0W2ZcoUChVKJULDLdF1PNaifTC6qSJgHHAs+ki06JiJkDKdDMzLpkfUE1gHMj\n",
       "4tw67d/MzFajnmPuPis3M8tIPcN9nKQHJV0oaXgdj2NmZt0MeMxd0q3AJj2sOo1kfpLO8fZvASMj\n",
       "4gvdPh/AGWWLZkfE7AEVY2aWU5JGkzyjudPpDXGHqqStgRkRsWO35b6gamZWof5mZ12GZSSNLHt7\n",
       "EMkc5GZmNkjq1S0zWdJOJF0zC4Dj63QcMzPrgW9iMjNrIpkOy5iZWbYc7mZmOeRwNzPLIYe7mVkO\n",
       "OdzNzHLI4W5mlkMOdzOzHHK4m5nlkB+Q3WLSJyWNLxQYWiqxrFhkmp+UZJY/DvcW0t6usR0dnDdh\n",
       "AqM6l02dyjbt7cIBb5YvHpZpIW1tjC8PdoAJExjV1sa4rGoys/pwuLeQQoGhvSwvDHYtZlZfDvcW\n",
       "UiqxrJflpcGuxczqy+HeQopFpk2dyrzyZVOmML9YZHpWNZlZfXjK3xaTdsuMKxQolEqUikWm+2Kq\n",
       "WfPob3Y63M3Mmojnczcza2EOdzOzHHK4m5nlkMPdzCyHHO5mZjnkcDczyyGHu5lZDjnczcxyyOFu\n",
       "ZpZDDnczsxxyuJuZ5ZDD3cwshxzuZmY55HA3M8shh7uZWQ453M3McsjhbmaWQw53M7MccribmeXQ\n",
       "gMNd0sGSHpa0QtJ7u607RdKjkv4q6SPVl2lmZpWo5sz9IeAg4M7yhZLeCRwKvBMYA/xIUu7+QpA0\n",
       "OusaquH6s+X6s9PMtVdiwKEbEX+NiL/3sOoA4IqIWB4RC4F5wK4DPU4DG511AVUanXUBVRqddQFV\n",
       "Gp11AVUanXUBVRiddQGDoR5n1JsCj5e9fxzYrA7HMTOzXqy5upWSbgU26WHVqRExo4LjREVVmZlZ\n",
       "VRRRXe5KmgV8JSL+kr7/GkBEfDd9PxM4PSLmdPucA9/MbAAiQn1ts9oz9wqUH+g3wOWSziUZjtkW\n",
       "+NNAijMzs4GpphXyIEmLgN2B30q6GSAi5gJXAXOBm4EToto/D8zMrCJVD8uYmVnjybT/XNK3JD0o\n",
       "6QFJt0vaIst6KiVpiqRH0u/hWkkbZF1TJVZ3I1qjkjQmvTnuUUkTs66nUpIukrRY0kNZ11IpSVtI\n",
       "mpX+N/O/ksZnXVMlJA2VNCfNm7mSvpN1TQMhaYik+yWttqkl65uLvhcR74mInYDrgdMzrqdSvwM6\n",
       "IuI9wN+BUzKup1I93ojWqCQNAX5IcnPcO4HDJe2QbVUVu5ik/ma0HPhyRHSQDMee2Ew//4hYBuyV\n",
       "5s27gb0kfTDjsgbiZJJh79UOu2Qa7hHxUtnb9YBns6plICLi1ohYmb6dA2yeZT2VWs2NaI1qV2Be\n",
       "RCyMiOXAr0hummsaEXEX8FzWdQxERDwdEQ+kX78MPEJyX0vTiIhX0y/XBoYASzMsp2KSNgfGAj9j\n",
       "1UaWN8n6zB1JZ0n6J3Ak8N2s66nCMcBNWReRc5sBi8re+wa5jEjaGtiZ5KSmaUhaQ9IDwGJgVtoA\n",
       "0ky+D3wVWNnXhnUPd0m3Snqoh9cnACLitIjYEriEpPCG0lf96TanAa9HxOUZltqj/tTfRHz1vwFI\n",
       "Wg/4NXByegbfNCJiZTossznw7800z4ykjwPFiLifPs7aoXZ97r2KiA/3c9PLacAz377ql3QUyZ9J\n",
       "+wxKQRWq4OffDJ4Ayi+6b8GqU11YnUlaC7gGuCwirs+6noGKiBck/RbYBZidcTn99QFgf0ljgaHA\n",
       "MEm/iIgjeto4626ZbcveHgDcn1UtAyFpDMmfSAekF2uaWTPcVHYfsK2krSWtTTL76G8yrqllSBJw\n",
       "ITA3In6QdT2VkjRC0vD06wLwYZoocyLi1IjYIiLagcOAO3oLdsh+zP076RDBAyQztX0l43oqNZ3k\n",
       "QvCtaWvSj7IuqBK93YjWqCLiDeAk4BaSboErI+KRbKuqjKQrgD8A20laJOnorGuqwB7A50i6TO5P\n",
       "X83U+TMSuCPNmznAjIi4PeOaqrHaYUrfxGRmlkNZn7mbmVkdONzNzHLI4W5mlkMOdzOzHHK4m5nl\n",
       "kMPdzCx2UQjRAAAAEklEQVSHHO5mZjnkcDczy6H/Az1pcRz7mHjbAAAAAElFTkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10687c0d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "##This function provides an interactive mode to change polynomial degree. \n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w))\n",
    "    plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "    plt.plot(x,y,'yo')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import scipy\n",
    "from scipy import linalg\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAi8AAAEZCAYAAABfH8VpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecVNX9//HXGxBBpSkKUgSkiNiiMZafDZOoiAUbxtgT\n",
       "I998TcBYYtREl9WvoolRwZJYYxQbAioaxaARe40dLKACooggCCgown5+f5yzMDvM7M7uzu6dmf08\n",
       "H4957M4t535m9u6dz5xz7jkyM5xzzjnnikWzpANwzjnnnKsNT16cc845V1Q8eXHOOedcUfHkxTnn\n",
       "nHNFxZMX55xzzhUVT16cc845V1Q8eXFIOk/STUnH4ZxzzuXCk5c8kjRL0neSNklb/rqkCklbxOfd\n",
       "JE2QtEDSV5LelnRSXNczbrss7TE0yzGnSjqlPnGb2SgzO7U+ZaTEs6+kJ+Pr+riGbY9Le43fxNe+\n",
       "Y1zfQtI1kuZJ+lLSJEld4rotMrxHFZLOiOsPkvSspMVx/5skbZRy7K6SHozlfiLpf1LW9Yvrvojr\n",
       "J0vqV4v3YGqMZfu05ffH5fvE5+0l3RrjWyrpfUl/SNm+QtLXaa/x7CzHvE3SxbnGWE3sledfna4N\n",
       "kp5I3z/Da1glaUxct5ukKfF9/kLSOEmdczhOX0nfSrojZdl6ksZL+jj1fU7b73JJC+PjspTl1Z5P\n",
       "cZs/SpotaYmkuyW1SVm3fvxbLol/zzPSj+2cyx9PXvLLgI+An1cukLQd0Dquq3QHMBvYAtgYOAGY\n",
       "n1ZWOzNrk/K4r5pjZiWpRe1eQr19DdwM/L6mDc3sztTXCJwGfGhmr8dNTgP2ArYHugCLgWvivnPS\n",
       "9t0OqAAmxH3bAhcBmwNbA12Bv6QcfizwIbAZcBBwqaSBcV074AGgH9AJeBl4sBbvgQHvAydWLlBI\n",
       "aHcHvmDt3+wqYAOgv5m1BQ4FZqaVtX3aeXBFLeKoD9V6B+k4oAVp56SZbZTyd+oMrADGxdXtgb8D\n",
       "PeJjGfCPHA53HeHvkn7+Pw0cD3yevi4mqEMI59P2wCGVSWtN51P8cnE88P8I52Jr4rkYjQR6E/6n\n",
       "9wXOkXRADq/DOVcXZuaPPD2Aj4E/Ai+nLLsCOJ9wIdwiLltG+FDKVEbPuG2zHI53CbCK8GGwDBgT\n",
       "l1cQPvhnEJIBgNHAHGAJ8CqwZ0o5I4E70o5/IiHBWgCcX4f34qfAx7Xc50nggpTnNwCXpzw/CHgv\n",
       "y75lwBPVlH048Fb8faP4GjumHev2LPtuHLfvUJvXAXwCKC77LXB9XLZ3XPY2MKSaciqALXM43jBg\n",
       "JfBdPA8ejMu7ED58vyAk1cNT9tklngdLCB/0V8Tlc+Jxl8XHrjm+5naEhG3X6s5f4CRgZjXl7AQs\n",
       "reFYxwD3xr/5HVm2WfM+pyx7HvhVyvNfAC/kcj4B44GzU57vTvi/axWffwr8NGV9OXB3bf9v/OEP\n",
       "f+T28JqX/HsRaCupv6TmwM8I3/LTt7le0s8Um5IyqPGbr5n9EXgG+I2Fb4wjUlYPAX4EDIjPXwZ2\n",
       "ADoAdwH3SWpZWVSG4vcg1Dz8BLhQ0lYAkvaUtLim2GpLUg9CLcvtKYv/DRwoaXNJGwDHAY9k2FeE\n",
       "ZOuf1RxiH+Cdyl3SfkKohdw2y757A/PMrDav+zNgOlD57fsEqr42COfBJZJOltQ3Szm5nAc3AncS\n",
       "Er02ZjYkNts8BLxOSGJ+AvxO0v5xt9HAVWbWDtgSqKzZ2yv+rKz5eyk2qSyW1K2aMC4lJGfpNYjp\n",
       "TmLd9yHV3qz9O61DUltCYnAGta8dGgC8mfL8LWCbDMfIdD4Z654v6wN9JXUg1PDVWLZzLj88eWkY\n",
       "dxAufvsRPsA+TVs/lJB0XAB8pNAnZue0bRbGD4zKx1bVHC/TRXyUmX1lZt/BmiaaxWZWYWZXEi68\n",
       "W1Wzf7mZfWdmbxEuyj+I5TxrZh2qiaWuTgSeNrPZlQvMbALhw/dTQg3BVkCmfh17Epp/xmcqWNJ+\n",
       "sfwLY7nLgOeAC2JfhZ2AIwhNAen7dgOuBc6sw2u6HThRUn+gvZm9mLZ+OCHp+C0wTdIMSYPStnkt\n",
       "7TzYr5rjpf4df0SoWfo/M1tlZh8TmvOOietXEj54O5rZcjN7KUMZwJomlQ5mNjfjQcO5uztVm1Ey\n",
       "bdeDkJxkTDJjH6ELqL7J8WLgZjP7jBqaTDPYiHAeVVoal6XLdD5NBn4lqYekdkBl36QNUspIL7sN\n",
       "zrkG4clL/hkheTmOtd8yq3wgxKTiPDPbltCn4g1CH4tUm8QPjMrH+zUcM90nqU8knS1pukJH2sWE\n",
       "av6O1ZT5ecrvy4ENq9k2H9apOZF0BeEDYON4/PuBRzPsexIw3syWp6+QtBshQTjSzFL7kxwH9CK8\n",
       "T9cRasc+Tdt3U0Ltz3Vmdm8tX48BE4EfA78hQ22DmX1robP0zsAmhH4g90lqn7LZjmnnwZQcj98D\n",
       "6JKa+ADnET6UAU4h1Ky9K+llSQfV8vUBEGt4rgd+Z2YVqasybH4C8ExqgppSTh9CrdoIM3suy7F+\n",
       "QKhBurqaY1Tna0JfqErt4rJ0mc6nW4G7gamE5r7/xOVzU8pIL3tZLeNzzuXIk5cGYGZzCH0MDiR8\n",
       "gFW37ZfAXwkfNHWp0cj27XPNckl7Eb7NDjWz9rHmZAl16JTZECTtQah2T685GQT8IyZ7Kwk1ILtI\n",
       "2jhl39bAUWT4Nq9w19KDwMlm9mTqulibcIiZbWZmuwObAi+l7NuBkLg8YGaj6vK6zGwFIdn6NSGh\n",
       "rW7bZcAoQpLWqy6HS3s+h9DnKDXxaWtmB8fjzTSzY81sU+ByYHx8L2tbm9EW+CFwr6R5hOZJgLnx\n",
       "75oqY9NerJGZAlxkZndWc6x9CH2y5sRjnQUcKenVHGOdRqxBjHYgrYkq2/lkwUgz62VmWxBqVOea\n",
       "2aexOXFeTWU75/LHk5eGcwrw4/gBVkW8XXMbhVuB2wD/C8xI61ORa2Ixn3CXQ3XaEDr2LpTUUtKF\n",
       "VP2WmIuc4lHQClgvPl0/pW9NNpXfdL9JW/4WcJKktpLWI3RC/tTMFqVscziwyMympsWxLaGq/7dm\n",
       "lqmfTH9JbeL7cTyhie/KuK4t8BjwrJmdn8vrrsb5wD4xoU2P4QJJO8cYWgGnE+6oSq1lq815sGXK\n",
       "85eBZZLOkdRaUnNJ21Y2T0o6PtYsQUhkjdDRdkH8WdM5BYRaRELiuUN8DI6rdmJtIoOkyrt0qtw1\n",
       "J6kroRbj2th3pzo3xte4AyFR+DvwL9b2K6q8ZblVfJr6O4TarzMldYnHPRO4Le0Y2c6nDpJ6x/N7\n",
       "AOELx0VpZf9J4fb3rYFfZSjbOZcnnrw0EDP7yMxeS12U8ntrQhPIYsLtut0Jt8mm+kpVx5z4XZZD\n",
       "jQaOkrRI0tVZtpkcHx8Aswh3SaR+mFpafJm+fRuEWhxJ1VWH70NoZvoX4XWtiMcm7v+OpNRbyVsR\n",
       "+gBl6gdxBuGD9EPCHTODCB8uqU4kc63GmYSmmFtT3sO3U9YfEMtdRLhb54BYC0Y8xs7AL1L2XVpD\n",
       "h9WMzGyemT2fZXUF4bbgBYQmq58AB6U1V7yZdh5cmaWsW4ABsYloYmzCOZjwIf9RPMaNrE1aDwDe\n",
       "iX/Lq4BjYh+n5YS72J6LZe2itWOgZHz9ZvZF5QNYSDhX5pvZ9ymbnQhMyJCg/opQ0zQy9b2uXCnp\n",
       "fEmPxOOsSDnWfEJzzYqUvxuExG85IVF6DPhGsVO8md1A6MT8NiExfihDwpTtfOpIOKe/JjRv3WJm\n",
       "N6esLyOcT7MJd5tdbmb/zvR+Oefqr/I2zsY/sNSd8G1lM8LF7kYzGxObBO4ltNnPAo6O3+6cc845\n",
       "5xJNXjoDnc3sDYWRT/8LHEYYe2Ghmf1ZYbTRDmZ2biJBOuecc67gJNZsZGafm9kb8fevgXcJo6Ae\n",
       "ytomhH8SEhrnnHPOOaBA+rxI6gnsSLjbo1Nsz4bQCbFTQmE555xzrgAlnrzEJqMJwOnxdtE1LLRp\n",
       "JdOu5ZxzzrmC1NiT9lURb3+dQJifpHKQtvmSOpvZ55I2J9xlkr6fJzTOOVcHZlav8Z38+usaW6Zz\n",
       "NrHkRZIIt3dON7PUW3wnEcb9uDz+TB95Fqj/P2BjkDTSzEYmHUdNPM788jjzqxjiLIYYIX+JRzFc\n",
       "f11pyHbOJlnzsgdhivm3JL0el50HXAaMk3QK8VbpZMJzzjnnXCFKLHkxs2fJ3ufmp40Zi3POOeeK\n",
       "R+Iddkvc1KQDyNHUpAPI0dSkA8jR1KQDyNHUpAPI0dSkA8jB1KQDcK4pSWyQuvqQZN7m6pxztZOP\n",
       "a6dff/MnTpp7k5n1TzqWTCS9A5xmZk/nc9taxpDxfPPkxTnnmghPXpqGOHbaR0CLOM9Z0cp2vnmz\n",
       "kXPOuZIjKa99OvNdXiPJmmRKat6YgeSbJy/OOVdgJEnqNSoOKeFyJGmWpHMkvQUsk9RM0m6Sno+z\n",
       "pL8haZ+U7XtJejrOGj9F0nWS7ojrekqqkPRLSbOBx+PyX0qaLmmRpMmVs5bHdVdJmi9piaS3JG0T\n",
       "lw+WNC0eZ66ks+LygZI+Sdl/a0lTY6zvSDokZd1tMb6HYzkvStoyy1tR2XTzVdx2N0knS3pO0pWS\n",
       "FgJlkraU9B9JCyUtkDRWUru09/PH8feRksZJ+mcs8x1JP6zjtjtJej2uGyfpXkkX1+Zv7cmLc84V\n",
       "nN5HwKDToPfhSUdShI4BDgTaA5sDDwMXmVkH4GxggqRN4rZ3AS8CGwMjCcN3pPel2BvoDwySNIQw\n",
       "pMfhQEfgGeBuAEkHAHsBfc2sHTAU+DKWcQswzMzaAtsA/0kPOg7a+hAwGdgUGA7cKalfymY/i3F2\n",
       "AGYCl2R5D/aKP9uZWVszezE+3wX4ENgMuJRQM3NJfJ+2BrrH8iulvxeHxNfbjjAm27W13VZSS+B+\n",
       "4Nb4Ou4mzGFYqz4sxVgN5pxzJUnqMgzang4DW8D1beHUUVL/i2HpaLPPbkw6vlyoPE8D4ZXVqV+N\n",
       "AWPM7FMASccDj5jZZAAze1zSq8BBkqYCOwP7mtkq4DlJk1i3qWWkma2I5f0aGGVm78fno4DzY+3L\n",
       "SqANsLWkVyq3iVYC20h628yWAK+zrt2ADc3ssvj8SUkPAz8HyuOyiWb2ajz2ncCVWd6HbO/dZ2Z2\n",
       "Xfz9W0Ii82F8vlDSVcCFWfYFeKbyvZQ0FvhdHbbdDWhuZtfE5/dLermacjLy5MU55wrGvJug9SJo\n",
       "/tfw+dO8Faw6E+ZNTDqyXNUx6cinT1J+7wEMTW1+IXzu/QfoAiwys2/T9u1eQ3mjJf01bZsuZvak\n",
       "pGuB64AekiYCZ8c5+44E/gRcFpu0zk2pDVlTRtqxAGbH5RASs/kp61YAG1E7VcqX1AkYDexJSLya\n",
       "AYuq2T/1+MuBVpKaZekUnHFbwuv5NENctTpvvNnIOecKRJiMVgbN2sPR00Ed4uLiuy00Oanv1RzC\n",
       "3HkdUh5tzOzPwDxgY0mtU7bfgnWllzcsrbwNKxMRM7vGzHYGBgD9gN/H5a+a2WGE5qAHgHEZjvMZ\n",
       "0D2tn1MP1v2gz0W28yV9+aXAamDb2NR1Ag2fF8wDuqYt2yJDbNXy5MU55wpKy74w5Rdw37bw+Mnh\n",
       "uaujscAhkvaX1FxSq9hJtquZzQZeBUZKWk/S7sDBVP8h+ndCM9EAAEntJA2Nv+8sadfYd2U5oVlm\n",
       "dSz7OEntzGw1sIyQMKR7Ke53TtxnYIznnri+NjUTC4AKoHcN220EfAMsldSVmGw1sBcI78tvJbWI\n",
       "/Yh+VNtCPHlxzrkCYjb9MrOZE83Mws/plycdU7Eys7nAEOB84AtCzclZrP3sOw7YndCx9mLgXkL/\n",
       "lDVFpJX3AGHS4HskLQHeBg6Iq9sCNxKaXWYBC4G/xHXHAx/HfYbF41Y5hpmtJHRyPZCQfFwLnGBm\n",
       "H6Rsl55YZUy0zGw5oSPuc/GuqF2z7F8O7AQsIXQWnpCtzNocv7pt4+s8AjgFWEx4Lx6m6vteIx+k\n",
       "zjnnmoh8XDtL+for6V5gupmV17ixyxtJLwHXm9k/M6zzQeqcc865SrGpp7fCeDAHAocS+qS4BiRp\n",
       "b0mdY7PRScC2hFvEc+Z3GznnnGuqOgMTgU0Id7z82szeTDakJmErQqflDQm3ah9lZvOr36UqbzZy\n",
       "zrkmwpuNXLHJdr55zYtzztWROmkwbRlBS1qxkm9Zyhibb48kHZdzpc6TF+ecqwN10mC6MZpD6bNm\n",
       "4SR6q5PwBMa5huUddp1z9aZOGqy+mqxtNFV9NVmdNDjpmBpcW06vkrgAHEof2jI8oYicazK85sW5\n",
       "JiafTR0ql3iIYfTiIg5kszUrJrOLhmgSO/Ea8DVhIKysP63M1hnjIdc4Va5mwPpA6/hoVc+fuW3b\n",
       "lfUzviktaZ1xuXMubzx5ca4JqW9Th8q1PmFQqz3jYw9asQH7pX1gD6IDD7MXO7GUMIrnhmk/q/yu\n",
       "ckFqUjOD5vSla5VypzBQJ2g+faigajLREviOMNfLt9X+NFYwYdPtOHLB02jNukVZ9qm+vE95gLUD\n",
       "lK21khU1vY/Oufrx5MW5pqQtIzI2dYxlOJCpVqM9YQTSymTlh8AHwLPAncBpLOZOYJ91jrWcT6zM\n",
       "RuQSlsrVktQE5w1uZWhanPuxPuOYQx9Oompy8Z2V5XbbpNTnSNjvMN6Z8n9mM+s12aGu1xgm0Tst\n",
       "EfyQpVxTzW7OrSFpL+AmM+ufdCzFxpMX55qSlrTKsrw1gMrVnbWJyp7AlsDLwHOE4cZftDJbmrqr\n",
       "xupbMqlFDURsNlpJGC4cjVO2ocJXW5l9lGu5a2JUl2HQ9nQY2AKubwunjpL6XwxLR5t9dmNtywOw\n",
       "+faIOgnGMpyWtGYlK1jKNd5Z1+XKzJ4B8p64SOoJfAS0yDLjc23Kug34xMwuqH9k+ePJi3NNyUoy\n",
       "Jxpt6KdyzSY0xTxDqFm5DXjDyuz7astcSv5rILLFWecmmXk3QetF0PyvYX675q1g1Zkwr161LzFR\n",
       "8WSlAElqYWarCrW8RlKy4/H43UbOFThJknqNklT/C9FSxvAwc6ose4yldOBe4KdAJyuzI63MrrIy\n",
       "e6XGxIX4AT6X0xnLZMbxFGOZzFxG1KsGIiREM6ssq0dCZGYGMmjWHo6eDuoQFxffKJ0uK0mzJJ0j\n",
       "6S1gWRz2fzdJz0taLOkNSfukbN9L0tOSlkqaIuk6SXfEdT0lVUj6paTZwONx+S8lTY8THk6WtEVK\n",
       "eVdJmi9piaS3JG0Tlw+WNC0eZ66ks+LygZI+Sdl/a0lTY6zvSDokZd1tMb6HYzkvStoyy1vxdPz5\n",
       "laRlcWLGWscuaRhwLGGm62WSHqzP3yevzKzoHsRrkT/80RQe0PtI+PUS6H1EvcoZSSdG8jdO4it2\n",
       "4l224zn68CibMTjp15gx3s0YTB8eZQBT8xEnbH0u9D4CUPi59R+Sfo2Nfy5hhVBGA76+WcBrQFfC\n",
       "HWhdCbM7D4rrfxqfbxKfvwD8mdAKsQdhduXb47qeQAWhBrKyc/gQYAZhePtmwB+B5+L2BwCvAm3j\n",
       "862AzvH3ecAe8fd2wI7x94GEJhmA9YCZwLkxnn2BpUC/uP62GPvOQHNgLHB3lvehR4y9Wcqyusb+\n",
       "D+CiQjtnvdnIuQKVr34aKldr4AzgTOAOetHbbrMvGyzwPMl3k4zZ9MtSntaruchlJ5GX2iyzOjV5\n",
       "GDDGzD4Nseh44BEzmxzKtMclvQocJGkqIRHY10Jz0HOSJrFuU8tIM1sRy/s1MMrM3o/PRwHnxxqM\n",
       "lUAbYGtJr1RuE60EtpH0tpktAV7PEPtuwIZmVnmePinpYeDnQOUs1xPN7NV47DuBK7O8D5neu7rG\n",
       "nq28RHmzkXMFa95N8H1Z6J+xpp/GhWF5zVSuZirXCcD7hNubd7MyO8PKCj9xccXLDOXjUY8QPkn5\n",
       "vQcwNDbDLJa0mFDD0hnoAiwys2+z7JutvNEpZVX+L3UxsyeBa4HrgPmSbpDUJq4/EhgMzIrNQrtl\n",
       "OE6XDMefHZdDSMxSJy9cQbg7L1d1jb0gefLiXIEyszr301C59iVUA/8G+LmV2VFWZjNr2M25UpD6\n",
       "/zEHuMPMOqQ82pjZnwlNORtLSh2jaAvWlV7esLTyNjSzFwHM7Boz2xkYAPQDfh+Xv2pmhwGbAg8Q\n",
       "ZlRO9xnQPa1vWw/g09q8+Awx1yv2LGUlLtHkRdKtsYPQ2ynLNo4dpz6Q9G9J7ZOM0blktewLU34B\n",
       "920Lj58cnmencvVXuSYBtwKXA7tbmT3XGJE6V4DGAodI2l9Sc0mtYifZrmY2m5Dgj5S0nqTdgYOp\n",
       "/sP674SmlgEAktpJGhp/31nSrpLWA5YTxiBaHcs+TlI7M1sNLANWZyj7pbjfOXGfgTGee+L62tRG\n",
       "LSD0eeldn9jjfvMJQyYUlKRrXv4BDEpbdi4wxcz6AU/E5841SWbTLzObOdHMLPycfnmm7VSuzVSu\n",
       "6wi3OT8NbG1ldm+ug7c5V4rMbC6ho+r5wBeE2oezWPvZdxxhEMYvgYuBewn9P9YUkVbeA4QvBfdI\n",
       "WgK8zdpRltsCNxJGbJ5F6Fz7l7jueODjuM+weNwqxzCzlcAhwIGE5ONa4AQz+yBlu/T/54z/32a2\n",
       "nDAu03OxmWiXesR+CzAgllMwfcWUQw10wwYQBtN5yMy2i8/fA/Yxs/mSOgNTLW30QUlmZgXXgci5\n",
       "xhY7454OnE0Y8fZiK7OFyUblClU+rp2lfP2VdC8w3czKa9zYNYps51sh3m3UycwqOyXNBzolGYxz\n",
       "hShORvhz4FJC1ffuVmYzko3KueIiaWfCqM4fE2ohDiX8T7kCV4jJyxpmZpK82tu5FCrXPsBfCW3a\n",
       "x1uZPZNwSM4Vq86E2+Y3Idzp82szezPZkFwuCjF5mS+ps5l9LmlzQjvlOiSNTHk61cymNkZwzjU2\n",
       "id2BeYzU+oQBtXYAzgPutbL6zVviSlvs9Dkw4TAKlpk9DDycdByu9gqxz8ufgS/N7HJJ5wLtzezc\n",
       "tH1Kts3VuUoSmwNXo9W702LFpvR6shm9/30b/R843a78JPPcP85Vw/u8uGKT7XxLNHmRdDewD9CR\n",
       "0L/lQuBBwj3wWxB6PR9tZl+l7ef/PK7gqJMG05YRtKQVK/mWpYypy/w+Es2A/wG7iH4PvcXQY3bg\n",
       "+9b38be3ZrCs6ymE5qIxwJ1mLM/363Cly5MXV2wKMnmpK//ncYVGnTSYboxOm1l5JnM5vTYJjMQO\n",
       "YDew0bz2HH9gGzq/9QrwByurHNIbAT8BRhBu8bwVuN6M2Xl9Qa4kefLiik228y3pcV6cKw1tGVEl\n",
       "cQE4lD60ZXguu0tsKPEXmq18kgPO2owzuy+j81vHWpkdVpm4AMQ5yR4341DCXCjrAa9JTJDYJyY3\n",
       "zjlX0jx5cS4fWtIqy/LWGZenkDiYZivfp8+jR3NGj2/Y/ao/0axiVyuzp6rbz4wPzTiTMIT4E8AN\n",
       "wOsSv5RqPq5zzhUrT16cy4eVZO5Au5IV2XaR6Kr1lk9iw/ljOfaQNhw/+HrafN7Pyuyu2txFZMbX\n",
       "ZlxPmJPkHOAIYLbEpRLda/lKnCsZkm6TdHHScbj88+TFuXxYyhgmUXXiw0l8yFKuSd9UorlafnMG\n",
       "633zPrtd/RNG9L2bPv/uY2V2uZVZ1mSnJmZUmPFvMw4mzJy7AfCGxDiJPb1JyTVBmYbUzyjO9nxK\n",
       "XQ+Ur0RJUk9JFZL887kahTjOi3NFx+bbI+okGMtwWtKalaxgKdekd9ZV8+93ZIPF49n4wy4MOv15\n",
       "ur3yGyuz9/IejzED+J3EBcBJhPlJvpEYA9xjlqWmyLl6CDMi97wUZp2fy+znDVVGepE5bldod6/4\n",
       "l43qWOwBWEyPEHbycfjDH7k+wDai4/R7aL3wew4cPosLm+/byMdvBjYI7BGw+WD/B9Y16ffFH437\n",
       "yMe1s7oyoPeR8Osl0PuIupdf9zKAHYHXgKWE2ZjvBi6O6zoQBqT7gjAB4UNA17juEmAVsIIw6/OY\n",
       "uHw0YTLHJYRpOPbMctxhhAkdv4v7PxiXdwEmxGN+BAxP2WeXWOYS4HPgirh8DmE4hGXxsWvS500h\n",
       "nrOJB5bPF+MPfxTigy2e+h82mP8N2971NcN2+l9G0izReLCtwMaALQK7B+z/gSnp98kfjfG3xxqi\n",
       "DNh8GGw1DU55Hyos/NxqGmw+LPdy61cG0BKYTZiotDlwZEwoLorrNwYOB1oBGxHGE7s/Zf8ngV+m\n",
       "lXlcTHqaAWcC84CWWY7/j8pjxefNgP8CfyK0cvQCPgT2j+tfAI6Lv29QmaQQOuBXQLLXiUJ5ZDtn\n",
       "vU3NuQaiH9y+nbq+MpOvN7+OgeV3cdSxm9oN//1b0kP6m/G+GSOAnoQL6O3AKxInSqyfZGyuWM27\n",
       "Cb4vg+atQmtH81aw6sKwvNHK2A1oYWajzWy1mU0AXqlcaWaLzOx+M/vWzL4mTMC4T1oZVZpqzOxO\n",
       "M1tsZhVmdiWwPrBVNTGk7v8joKOZ/Z+ZrTKzj4GbgWPi+pVAX0kdzWy5mb2UKQaXmScvzuWZDj9x\n",
       "A+1w+wN8MPhNNp22iEN/1dP+dd2p9emM2xDMWGrGaKAfUEb4ljlbojxOTeBcTszMQAbN2sPR00Ed\n",
       "4uKc+5HkoYwuwKdpy2YTkwFJG0i6QdIsSUuAp4B2oY/N2jBSd5Z0tqTpkr6StBhoRxgRPhc9gC6S\n",
       "Flc+CHOSbRbXn0L433tX0suSDsqxXId32HUub1Qu8dLwc/jv2RfRfNUyfnjTQfbMeY/CyUmHVi0z\n",
       "KoB/Af+S2Br4LTBN4lFgjBkvVVuAcwC07AtTfgEf3g+9Dw/PG7WMeUDXtGU9YM1dgGcRkoVdzOwL\n",
       "ST8g9I8RGe5KkrQX8Hvgx2Y2LS5bRPaakfQkaw7wsZn1y7ix2Uzg2FjukcB4SRtnKMdl4MmLc3mg\n",
       "X+yzO6/98z5mDO7EFs9dwftDzrfPdiq6i5AZ7wK/kfgj8AvgbokFhLmU7jNjZaIBuoJlNv2ylKcT\n",
       "EyjjeWCVpBHA34BDCE03T8T1GxE65C6JSUJZ2v7zgd4pz9sQOvEulNQSOBdoW83x5wNbpjx/GVgm\n",
       "6RzgGkIz0dZAKzN7VdLxwGNmtoDQadcIfV0WxJ+9gRm5v/ymxZuNnKsHnd9mEw0641Em3PUsC7ee\n",
       "zYZfdLUF/UsYAAAgAElEQVT3hpxnVtzfnsz4yoyrgL7A/xGqj2ZLlEl0TjQ45zIws+8JAzSeDHwJ\n",
       "HE2406fS1UBrYCEh0XmUqrUco4GjJC2SdDUwOT4+IEwSvIJQm5LNLcCA2EQ00cwqgIOBHxDuNFoA\n",
       "3MjaBOgA4B1Jy4CrgGPM7DszW064++m5WNYudXg7Sl7RTsxIHx6r66y9ztWXytWC2XudyzPnXcDn\n",
       "O3zDhgtPsM+3/1fScTUkiW0ITUrHEG45HWO2tkOkK3w+MaMrNqU3MePxHEA3RquTBicdiitOkiT1\n",
       "GpXWYa/m/c5r/xOmXjiHuyZdyFc9b+TrLpuXeuICYMY0M/6XUDX+BjBO4gWJn0u0TDg851wTUrzJ\n",
       "C9Rq1l7n1tX7CBh0WugYWDOVq4eOP/Bxbnn2Ef576gK+32AbW7D1cDO+a+hIC4kZi834K9AHuBw4\n",
       "FfhY4k/SmjspnHOuwRR38gI5zdrrXCqpyzCp/zQYeClc3xYGjpL6T5O6DMu4fbk20JndL+Phv73H\n",
       "feN2ZWm3/2FZtx/Y6pZNujOdGavNeMCMHwODgC2A9yVuk/hhwuE550pY8d9t1JJNkg7BFZt5N0Hr\n",
       "RdD8rymDYZ0J86rc3aByiYpmQ3nnuOt49OoNQRNZ2Wa4GYsSCrxgmfE2MEziPML4FRMlPiHcpXS/\n",
       "Gd8nGqBzrqQUd83LQ8xhe7qrXLsmHYorHrkMhqVy7cCC/i9w+5SbefhvX7Oi4/62fJPjPHGpnhlf\n",
       "mvFnwm2eVwKnEZqUzpfYNNnonHOlopjvNprMUq7hNFoA1wE7W5nNTzo2VxykAefCyg9SB8Mym365\n",
       "yrUJ37e6hBfOPI6n/yQqWoyiYr2/+PgmdSexAzCcMNfM/YS7lN5INqqmye82csUm2/lWtMlL6otR\n",
       "uS4CBgI/sTLz6mlXaypXC+DXzN7zIibcVcE3m73J6vWHmfFh0rGVComOwK+A3wAfE5qUHjBjVaKB\n",
       "NSGevLhiU3q3Slc1kjB1+BUJx+GKkMq1L8s3eZPxd5/LP/+ziqXdf8vq9X/qiUt+mbHQjMsIs+uO\n",
       "AUYAH0mcK3nfNedc7koieYmz9B4HDFa5Tkg6HlccVK6eGtlsPG+ceC9Xztmcd372MBXrbWXGPcU+\n",
       "Qm4hM2OVGePN2Bs4jDBL70yJmyW2Tzg8V0Ik3Sbp4qTjyJWk4yQ9lu9tS1FJNButWV6ubYEngQOs\n",
       "zF5r/MhcMVC5NgD+wPztTmf8PV+woP9KaDbMjOeTjq2piuPDnEro4DuDMFT7Q96klF9NrdlI0j+A\n",
       "T8zswhy2nQrcYWa31PFYt8VjXVCX/V1mpd5sBICV2TuE9vQJKleu05a7JkLlkg4/+SweHzWPq2ad\n",
       "zt9fW8GCAbdAsx09cUmWGV+YcQnQkzCp3tmE2pjfS2ycaHAuJ+qkweqrydpGU9VXk+sy+nk+yshU\n",
       "bI7bNeg3eUnFPzRJASmp5AXAymwcMA64O3bCdE2YhCS2V8f3b+Dqj77m8VGjeP+Qx1jSYzDWoqsZ\n",
       "l/sYJIXDjO/NuNeMPYCjgG2BDyVukNg24fBcFuqkwXRjNMdzAEezT12mb8lLGdKOkl6TtFTSPUCr\n",
       "lHUdJD0s6Ys4+eJDkrrGdZcAewHXSlomaUxcPlrSHElLJL0qac8sxx0GHAucE/d/MC6fJekcSW8R\n",
       "ZphuLulcSTNjjNMkHZZSzsmSnkl5XiHpfyR9ECdpvLaabfeX9L6kryRdJ+kpSafk+t4Vm5JLXqI/\n",
       "xp+XJBqFS0RMWH4gcQla9QGtFz5Ln8eOZ8/LrmHgyA3si22ONuN5MyqSjtVlZ8arZpwE9AfmAv+W\n",
       "eEJiiETzhMNzqdoygkPpU2VZbadvqWcZkloCDwD/BDoA9xFuz6+sURFh5uct4mMFcC2Amf0ReAb4\n",
       "jZm1MbMRcZ+XgR1ieXcB98XjVGFmNwJ3ApfH/YekrD4GOBBob2argZnAnmbWFigHxkrqVM1LOwjY\n",
       "GdgeOFrSARlee8f4ev8AbAy8D+xOA9cmJakkkxcrs1WEE+ZolevopONxDS8mLDtJjAI+AJtIr8d3\n",
       "4Zd7bcI5m/2TA0/vbg/dcK49dIP3oSgyZsw342JCk9LNwLmEJqWzJDokGpwLWq6t4Uhbnvv0LfUv\n",
       "YzeghZmNNrPVZjYB1s56bmaLzOx+M/vWzL4GLgX2SSujShOTmd1pZovNrMLMrgTWJ3Qwzya9icqA\n",
       "MWb2qZl9F8scb2afx9/HEfp4VTfQ6mVmttTMPiH06fxBhm0GA++Y2QMx1jHA59WUWfRKMnkBsDL7\n",
       "EjgCuC525HUlJiYsP5K4nPBtZhwAA8uuoKz5N5y0XzO6v7i3jawYbmXmI+MWOTNWmnG3GbsDPyNc\n",
       "xD+S+JvEgITDa9pW8m2W5SsasYwuwKdpy2YTEwpJG0i6ITblLAGeAtqlzSpfpaZC0tmSpsemmMVA\n",
       "O6C2/Sk/SSvzREmvx2agxYSm0eqGCkhNQpYDG2bYpguhdjJV+vOSUrLJC4CV2evAmcD9Klf7pONx\n",
       "9RcTll0lriAMdHYnsAo4kj+1+ikj1ZeBF52LbCTw09iJ25UYM1424wRgADAfeEJiisQh3qSUgKWM\n",
       "YRIzqyybxIcs5ZpGLGMe0DVtWQ/WJiRnAf2AXcysHaHWRaytLUlPXPYCfg8MNbP2ZtYBWEL2DsDZ\n",
       "mmjWTjsi9QBuJNxYsnEs851qyszVZ0C3lOMo9XkpKsgOrZIGAVcDzYGbzezyupZlZXaHyrUzcKfK\n",
       "dUgcE8YVEYlmhGrVoYROnN8Q2ncPBd5mpFoT2np/SzhvTrAyy/0bnytaZswDRsbmwqHAhcDVEtcA\n",
       "/zBjSaIBNhE23x5RJ8FYhtOS1qxkBUu5xubbI41YxvPAKkkjCHesHQL8CHgirt+I0M9liaSNgbK0\n",
       "/ecT5uSq1IbwxWhh7OdyLtC2muPPB7asIcYNCcnMQqCZpBOhVh3RU5OtVI8QOhsPAf4F/BroXIty\n",
       "i07B1bxIak7oRDWI8K3q55K2rmexZxNO3JH1LMc1EolmEntKXE3LiZ/Tde9/06fvEWzZfg4dm59l\n",
       "xoWM1NuM1NHAu4R26B2tzC72xKXpMeM7M8YCuxAGrNyFMCHktRL9k42uabD59ojNsANtmg20GXZg\n",
       "bRKXfJRhZt8TugqcDHwJHA1MSNnkaqA1IXF4HniUqrUlo4Gj4p1IVwOT4+MDYBYh8ZlTTQi3AANi\n",
       "c9DETBuY2XTgr8ALhOagbYFnUzdJiym9Nid1/ZrfzWwhIXn/c3x9WwOvAt9VE29RK7hB6iTtDpSZ\n",
       "2aD4/FwAM7ssZZtaD5KkcnUi/DF/a2X2YB5DdnkSq/v3IPwTHgkspPWf32SrC/bmsJVbrNlwEjNp\n",
       "wdUM5mhCG/QIK7OnEwnaFSyJLoRvoMOANwhTEkxuyneZNbVB6poqSc0IfW2ONbOnko6nPoppkLqu\n",
       "VO3gNJd12zFrLc44fRRwk8pVXW9x14gkmkvsK3Ed4W9d2Ut+XzO2p+sfNq2SuEC4fbIlVwJ3Az/0\n",
       "xMVlYsZnZlxIuEvpLuBi4H2JYYkG5lwDiOO8tJe0PnB+XPxikjE1pELs85JTVZCkkSlPp5rZ1BoL\n",
       "LrOXVK7zgQdUrl2tzJbWLURXHxItCJ3ljiJU884FxgN7mzGjysbZbp/8ileszP7esJG6UmDGt8Dt\n",
       "EncQxr64X+I/ZmmdQ0uQpIHAwITDcI1jd0KS3hKYBhxWeXt2KSrEZqPdgJEpzUbnARWpnXbrW22p\n",
       "ct0AbAocxfUMoi0jaEkrVvItSxlTl7ZaVz2J9QgX0aGEyfhmEzrdTqhu9mZtrac4hr3XWTGWyTbD\n",
       "DmyYaF0pk7gemG1GnW8EKFbebOSKTbbzrRBrXl4F+krqSbj962fAz/N8jBHAVF7jVrqxR5VRHSfR\n",
       "W52EJzD1FxOWHxMSliHAR4SEZRczZmXdr1zrEUaVPJVd2YnHWMIBtFuzQW1vwXSuqvHA5fHhnCtC\n",
       "BVfzAiDpQNbeKn2LmY1KW1//bw/l6sIUPmI/1l9npX+rrzOJlsBPCU1ChxJ66o8n1LDMrnbfcm0J\n",
       "/Ipwt8BHwE3AfVzPQNrW/RZM51LFZst5wM41nZOlxmteXLHJdr4VZPJSk3z982hHvc5hGYZaHsdT\n",
       "Ns0G1rf8pkJifWA/Qg3LIcB0Qg3LRLOqo0uus2+51ifUypxKGDF1LHCTldn0Bg3aNWkSNwPTzbgy\n",
       "6VgakycvrtgUU7NR4/ma+RmX12ZI6yZKohVwAKGG5WDgLUINy/lm6wzRve7+4Y6vU4ETCSNM3gTc\n",
       "b2WWeYhw5/JrPHABNK3kJV8kFd+3XldSmnbNS+UU7Kl9Xv7FZ6zkQg7ndiuz7+t7jFIi0ZoweOBQ\n",
       "wkRgrxNqWO6PI51Wv3+5WhPGbzmVMKjcbcAtVmYzqtvPuXyLzZufA9vlkmyXCq81caWiSScvEBOY\n",
       "dvyODvQBWtKHRfSjHbA5oV18FmEOncpH5fPPrMxW5yOGQiaxAWE696GExOW/rE1YMtdcpZdRru0I\n",
       "CcuxhA7ZNwEPWZmtbJCgncuBxD+BV8y4NulYGosnL65UNPnkJesxwh0v3YFehEGueqU8ehJmAf2E\n",
       "dZOayscXVlaEby4gsSHhbp+jgP2BlwnV7PebsSCnMsq1IeFOsVMJ7+OtwK1WZrMaImbnakviUOAM\n",
       "M/ZNOpbG4smLKxWevNQ1htAE0oPMiU0vYAOqJjTpvy8upORGog0hYRlKuFvoBULC8oAZC2vcv1wC\n",
       "+gL7xsf+hDk7bgIetTJb1UChO1cnsd/W50A/M75IOp7GUAjXTufywZOXBqJytaFqYpP6ey/CSMLZ\n",
       "EpuPrcy+bvAYRVvC3UFHEcZjeY7QJPSgGYuq3TckK70JicrA+FgNPAlMBR6zMvusgUJ3Li8k7gae\n",
       "NOPGpGNpDMVw7XQuF568JCB+8Hcgc2LTMz6+IUtiA8yu6105Eu0I468MJSQcTxFqWCaZsbiGmHvF\n",
       "fSoTlmaEZKUyYfmokGqTnKuJxFHAqWYckHQsjaHYr53OVfLkpQDFRKETmRObXoQ+JF+SuSPxx8An\n",
       "qc00Eh1Ym7DsTUg27gMeMmNJNXH0YG2isi9hzozKROVJYGZ6sqJOGuzTLbhiEft3fQb0qqm2sRSU\n",
       "+rXTNR2evBQhlas5YabtnmSuvenENx3n88ZJ3/DOMRvxxbab0u6Td+j43sNse8/dbH/XDCuzigzl\n",
       "dqdqzcqGrE1UpgLvV1ezkvHW80nMZC6newLjCpXEBEIif1vSsTS0pn7tdKXDk5cSItEROAytHgr8\n",
       "P9p8+jr9H3iPvS9dwkbzu7I2yekAzGFtjU1zQrLSjpCkTCUkLO/WphlIfTWZ4zNUv/t0C66ASRwL\n",
       "HGvGwUnH0tD82ulKRdMeYbcESGwGHE7odLsL8BjW/GbgSFuyxddhDsoRVfcp1waEO6Uqk5lmwGhg\n",
       "WqYamZy1pFWW5a3rXKZzDe9h4G8Sbc1YmnQwzrma1Zi8SNrEzL5sjGBcbiQ6AUcQEpYfAo8CfweG\n",
       "mLG8pv2tzJYD78ZH/qwkcydin27BFTAzlko8TZjm4q6k43HO1SyXmpcXJb0B/AN41IqxnSkB0jed\n",
       "4Yzx8PcnoVkzQu2G0n7muix1XQ9ge+AR4BrgMbMCSQ6WMoZJ9E7r8/IhS7kmwaicy8UEwpcBT16c\n",
       "KwI19nmR1IwwaNkvgR8B44B/mNkHDR9e1pgKvt1WOuAU6Hcd9Lsfhr9DGNelIsPP2i5bAPzHLEst\n",
       "R8Li3UbDaUlrVrKCpVzjnXVdoZPYmNAHrKsZDT7GUlKK4drpXC5q1WFX0o+BsYS7UN4AzjOz5xso\n",
       "turiKNh/QKnLMGh7OuzZAm7qB6d+AM+ugqWjzT5rEgNhOVeMJB4DbjbjvqRjaSiFfO10rjZy6fPS\n",
       "ETgOOBGYD/wWeAjYgTC4Wc8GjK8IzbsJWi+C5n8NrT3NW8GqM2HexKQjc85Vazxh1vOSTV6cKxXN\n",
       "ctjmecIttEPMbLCZTTSz783sVUInUZci9AmSQbP2cPR0UIe42PsKOVfYHgAGSX53nHOFLpcOu/3N\n",
       "Mt8+a2aX5TmeEtGyL0z5BXx4P/Q+PDwvPD4arnNrmbFA4nXCpKIPJh2Pcy47H6SuifLRcJ1bl8Rv\n",
       "gF3NODHpWBqCXztdqcil2cgVCEmSeo2SVP+LT1tGVElcAA6lD20ZXu+ynSte9wMHS7RMOhDnXHY1\n",
       "Ji+S9sywbI+GCcdVr/cRMOi00BRVTz4arnPrMOMzwuCNP0k6FudcdrnUvGQaYOzafAfispO6DJP6\n",
       "T4OBl8L1bWHgKKn/tHBbdh35aLjOZVN515FzrkBl7bAraXfg/wGbSjqTcN8vQBu8uamRNcDt1z4a\n",
       "rnPZTATOl/i1GauSDsY5t67q7jZqSUhUmseflZYShtF2jcTMTOqTcvv1xt2p5+3XNt8eUSfBWB8N\n",
       "17lUZsyW+BjYB3gi6Xicc+vKZXqAHmY2u5HiyUlT7DEvDTgXVn6Qevu12fTLk47LuVIk8Qegpxn/\n",
       "m3Qs+dQUr52uNOWSvDyZYbGZ2Y8bJqSa+T+gc64hSfQBniXMdbQ66Xjyxa+drlTkMkjd71N+b0Xo\n",
       "yObtwM65kmXGTInPCf3+nkk6HudcVXUapE7SK2b2owaIJ9fj+7cH51yDkrgA6GjG6UnHki9+7XSl\n",
       "IpdxXjZOeXSUNAhoW5+DShoqaZqk1ZJ2Slt3nqQZkt6TtH99juOcc/UwHjhC8rsrnSs0uTQbvQZU\n",
       "Vs+sAmYBp9TzuG8DhwM3pC6UNAD4GTAA6Ao8LqlftrmVnHOuoZjxrsQyYBfgxaTjcc6tVWPyYmY9\n",
       "831QM3sPIMMo90OAu83se2CWpJn4hcM5l5wJhH5+fg1yroDk0mzUWtJZku6XNFHSGZIyDy1ff12A\n",
       "uSnP5xJqYJxzLgnjgSMlvJ+IcwUkl2aj2wkD040hDO96LHAHMLS6nSRNATpnWHW+mT1Uixgz9iiW\n",
       "NDLl6VQzm1qLMp1zLhdvARXAjoQm9KIiaSAwMOEwnMu7XJKXbcxsQMrz/0iaXtNOZrZfHeL5FOie\n",
       "8rxbXJap/JF1KN8553Jmhklrmo6KLnmJX+qmVj6XVJZYMM7lUS696F+L8xwBIGk34L95jCG1OnYS\n",
       "cIyklpJ6AX2Bl/N4LOecq63xwFHedORc4cil5mVn4DlJnxCacLYA3pf0NmGk3e1re1BJhxOaoToC\n",
       "/5L0upkdaGbTJY0DphPubDqtPvP3OOdcHrxKGKBzG+CdhGNxzpHj3EawzjcOq1xmZrMaJLLqY/KB\n",
       "lpxzjUbiSmCJGeVJx1Iffu10pSKX5OUOMzuhpmWNyf8BnXONSWIP4O9mbJd0LPXh105XKnLp87Jt\n",
       "6hNJLYAfNkw4zjlXkF4ANpHol3QgzrlqkhdJ50taBmwnaVnlA/iC0LHWOeeaBDMqgImEu46ccwnL\n",
       "pdnoMjM7t5HiyYlXfTrnGpvEvsAVZsVb8+zXTlcqckle9iHDQHFm9nRDBVUT/wd0zjU2iRbAZ8Cu\n",
       "ZnycdDx14ddOVypySV4eZm3y0oow19B/zezHDRxbdTH5P6BzrtFJ3Ah8YMYVScdSF37tdKWixuRl\n",
       "nR2k7sBoMzuiYULKKQb/B3TONTqJA4CRZuxe48YFyK+drlTkcrdRurnA1vkOxDnnisB/gH4S3ZIO\n",
       "xLmmrMYRdiVdk/K0GfAD8js9gHPOFQUzvpd4CDiCMEq4cy4BufR5OZm1fV5WAx+b2XMNHFe1vOrT\n",
       "OZcUiYOB35uxT9Kx1JZfO12pyCV5aQ30ISQwM83s28YIrDr+D+icS4rE+sDnwNZmfJ50PLXh105X\n",
       "KqobpG49SX8GPgH+CdwOzJX0F0nrNVaAzjlXSMz4DngUOCzpWJxrqqrrsPsXYGOgl5ntZGY7AVsC\n",
       "7aE4bxN0zrk8GY+PtutcYrI2G0maCfQzs4q05c2B982sTyPEl5FXfTrnkiSxATAP2NKML5OOJ1d+\n",
       "7XSlorqal4r0xAXAzFYD6yx3zrmmwozlwBRgSNKxONcUVZe8vCvppPSFkk4A3mu4kJxzriiMB45K\n",
       "OgjnmqLqmo26EWZRXcHacV1+CGwAHG5mcxslwsyxedWncy5REm2AT4EtzPgq6Xhy4ddOVyqqvVVa\n",
       "koAfA9sQbpWebmZPNFJsWfk/oHOuEEg8CNxnxtikY8mFXztdqaj13EaFwP8BnXOFQOJE4Aiz4rht\n",
       "2q+drlR48uKcc3Uk0QGYDXQ1Y1nS8dTEr52uVNRlYkbnnHOAGYuB54DBScfiXFPiyYtzztXPBHzA\n",
       "OucalTcbOedcPUh0BGYCXeL4LwXLr52uVHjNi3PO1YMZCwnDSRyQdCzONRWevDjnXP1505Fzjcib\n",
       "jZxzrp4kNgemA53jrNMFya+drlR4zYtzztWTGfOAd4CfJh2Lc02BJy/OOZcf3nTkXCNJJHmR9BdJ\n",
       "70p6U9JESe1S1p0naYak9yTtn0R8zjlXBxOBIRLrJR2Ic6UuqZqXfwPbmNkOwAfAeQCSBgA/AwYA\n",
       "g4DrJXntkHOu4Jkxh3DL9MCEQ3Gu5CWSGJjZFDOriE9fArrF34cAd5vZ92Y2i3Ah2CWBEJ1zri7G\n",
       "401HzjW4QqjV+CXwSPy9CzA3Zd1coGujR+Scc3UzAThconnSgThXylo0VMGSpgCdM6w638weitv8\n",
       "EVhpZndVU1TGe7kljUx5OtXMptYxVOecywszPpL4DNgTeCrpeCQNxJuxXAlqsOTFzParbr2kkwmT\n",
       "mf0kZfGnQPeU593iskzlj6xfhM451yDGA0dRAMlL/FI3tfK5pLLEgnEuj5K622gQ8HtgiJl9m7Jq\n",
       "EnCMpJaSegF9gZeTiNE55+poAnCEVBDN8s6VpAareanBNUBLYIokgBfM7DQzmy5pHGGkylXAaVaM\n",
       "QwA755osM96TWAzsCryQdDzOlSKfHsA55/JMohzYyIyzko4llV87Xanwak3nnMu/CcCREp4oONcA\n",
       "PHlxzrn8extYCfww6UCcK0WevDjnXJ6ZYfhcR841GE9enHOuYUwAjvKmI+fyz5MX55xrGP8F1gO2\n",
       "SzoQ50qNJy/OOdcAvOnIuYbjyYtzzjWcytF2nXN55MmLc841nJeA9hL9kw7EuVLiyYtzzjUQMyqA\n",
       "iXjTkXN55cmLc841LG86ci7PPHlxzrmG9SzQRaJ30oE4Vyo8eXHOuQZkxmrgfrzpyLm88eTFOeca\n",
       "nt8y7VweefLinHMNbyrQW2KLpANxrhR48uKccw3MjO+BScARScfiXCnw5MU55xqHNx05lycys6Rj\n",
       "qDVJZmY+2ZlzrmhIrA98DgwwY14yMfi105UGr3lxzrlGYMZ3wL+Aw5OOxbli58mLc841Hm86ci4P\n",
       "vNnIOecaiURrQtNRHzMWNP7x/drpSoPXvDjnXCMxYwXwGDAk6VicK2aevDjnXOOagM915Fy9eLOR\n",
       "c841Iok2wKdADzMWN+6x/drpSoPXvDjnXCMyYxnwH+CQpGNxrlh58uKcc41vPN505FydebORc841\n",
       "Mon2wBygmxlLG++4fu10pcFrXpxzrpGZ8RXwLHBQ0rE4V4w8eXHOuWSMxwesc65OEkleJF0s6U1J\n",
       "b0h6QlL3lHXnSZoh6T1J+ycRn3PONYIHgf0kNkw6EOeKTSJ9XiS1MbNl8ffhwA5m9itJA4C7gB8B\n",
       "XYHHgX5mVpG2v7fbOueKnsQU4O9mTGic4/m105WGRGpeKhOXaCNgYfx9CHC3mX1vZrOAmcAujRye\n",
       "c841Fp/ryLk6aJHUgSVdApwArGBtgtIFeDFls7mEGhjnnCtFDwCXSbQy49ukg3GuWDRYzYukKZLe\n",
       "zvA4BMDM/mhmWwD/AK6upqjiu5fbOedyYMbnwJvAfknH4lwxabCaFzPL9Z/xLuCR+PunQPeUdd3i\n",
       "snVIGpnydKqZTa1liM45Vwgqm44eynfBkgYCA/NdrnNJS6rDbl8zmxF/Hw7sYmYnpHTY3YW1HXb7\n",
       "WFqQ3unMOVcqJLoBbwGdzVjZsMfya6crDUn1eRklaStgNfAh8L8AZjZd0jhgOrAKOC09cXHOuVJi\n",
       "xlyJ94F9gceSjse5YuDTAzjnXMIkzgb6mTGsYY/j105XGjx5cc65hEn0Al4CupixquGO49dOVxp8\n",
       "egDnnEuYGR8DnwB7JR2Lc8XAkxfnnCsME4Cjkg7CuWLgzUbOOVcAJPoBU4FuZlTUsHkdj+HXTlca\n",
       "vObFOecKgBkfAF8Cuycdi3OFzpMX55wrHOPxpiPnauTJi3POFY4JwJES3rTjXDU8eXHOucIxjTBZ\n",
       "7Y+SDsS5/9/evYZaVtZxHP/+HBUr7SKVmQ7ZxaAxvGRYacGRMETCYZBQIyqD3pQ1lJSNA/kiQiso\n",
       "I/FFZUaIl8q01FLHGwrCmDDjWDPjBZxSM02sMMLy8u/FXhPbM2efc2Y8M89ac74fGM7aa2/W+Z0z\n",
       "7M3vPM9az+ozy4sk9UQVxWjq6JTWWaQ+s7xIUr84dSTNwfIiSf2yDlgCHNE6iNRXlhdJ6hGnjqS5\n",
       "WV4kqX+uAla0DiH1lSvsSlLPJOwB7F/FUwt7XD87tXuwvEjSIuFnp3YXThtJkqRBsbxIkqRBsbxI\n",
       "kqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRBsbxIkqRB\n",
       "sbxIkqRBaVpekpyV5MUk+4/tW5XkwSSbk3ykZT5JktQ/zcpLkqXACcCfxvYtA04FlgEnAhclGezo\n",
       "UJKp1hnmw5wLy5wLawg5h5BR2p20LAbfBb46bd9y4PKqeq6qtgAPAcfs6mALaKp1gHmaah1gnqZa\n",
       "B5inqdYB5mmqdYB5mmodYB6mWgeQFpMm5SXJcuDRqtow7ak3A4+OPX4UOGiXBZMkSb235846cJI1\n",
       "wJtmeGo1sAoYP58lsxyqFjKXJEkatlTt2m6Q5N3ALcC/u10HA48B7wPOAKiq87vX3gCcW1Vrpx3D\n",
       "QiNJO6CqZvtjURqEXV5etgmQPAwcXVVPdyfsXsboPJeDgJuBd1TrkJIkqTd22rTRdvh/MamqjUl+\n",
       "DoqXzhsAAAVBSURBVGwEngc+Z3GRJEnjmo+8SJIkbY8hr6HyjST3Jlmf5JZu3ZjeSfKdJJu6rL9K\n",
       "8prWmWaS5GNJ/pjkhSTvaZ1nuiQndgsXPpjk7NZ5ZpLkJ0meSHJf6yyTJFma5Lbu//oPSb7YOtNM\n",
       "kuyTZG33/t6Y5LzWmWaTZEmSdUmubZ1lkiRbkmzoct7dOo/0cgy2vADfrqojqupI4Brg3NaBJrgJ\n",
       "OKyqjgAeYHSlVR/dB6wA7mgdZLokS4ALGS1cuAw4Pcm72qaa0SWMMvbZc8CXquow4P3A5/v4u6yq\n",
       "Z4Hju/f34cDxST7YONZsVjKa7u7zUHYBU1V1VFUNef0sabjlpaqeGXu4L/BUqyyzqao1VfVi93At\n",
       "o6ureqeqNlfVA61zTHAM8FBVbamq54ArGC1o2CtVdSfw99Y5ZlNVf62q9d32v4BNjNZX6p2q2npF\n",
       "4t7AEuDphnEmSnIwcBLwY2Zf9qEP+p5PmpfBlheAJN9M8mfgU8D5rfPMw2eA37YOMUAHAY+MPXbx\n",
       "wgWQ5BDgKEaluneS7JFkPfAEcFtVbWydaYLvAV8BXpzrhY0VcHOSe5J8tnUY6eXow9VGE82y0N05\n",
       "VXVtVa0GVif5GqMPkDN2acDOXDm716wG/ltVl+3ScGPmk7On+jwUP0hJ9gV+CazsRmB6pxuxPLI7\n",
       "T+zGJFNVdXvjWC+R5KPAk1W1bgD3Nzquqh5P8gZgTZLN3WihNDi9Li9VdcI8X3oZDUc05sqZ5NOM\n",
       "hpU/vEsCTbAdv8++eQwYPyF7KS+9jYS2Q5K9gKuAS6vqmtZ55lJV/0xyPfBe4PbGcaY7Fjg5yUnA\n",
       "PsCrk/ysqj7ZONc2qurx7uvfklzNaDrW8qJBGuy0UZJDxx4uB9a1yjKbJCcyGlJe3p2EOAR9mxe/\n",
       "Bzg0ySFJ9mZ05/HfNM40SEkCXAxsrKoLWueZJMnrk7y2234FozvQ9+49XlXnVNXSqnorcBpwax+L\n",
       "S5JXJtmv234Vo9uz9PaqOGkugy0vwHlJ7uvmxKeAsxrnmeQHjE4oXtNdonhR60AzSbIiySOMrkC5\n",
       "PsnvWmfaqqqeB84EbmR0RceVVbWpbaptJbkcuAt4Z5JHkjSZxpzDccAnGF29s67718crpA4Ebu3e\n",
       "32uBa6vqlsaZ5qOvU5wHAHeO/T6vq6qbGmeSdpiL1EmSpEEZ8siLJElahCwvkiRpUCwvkiRpUCwv\n",
       "kiRpUCwvkiRpUCwvkiRpUCwv0jRJFny5/CRvSXL6Qh9XkhYjy4u0rZ2x+NFbgY/vhONK0qJjeZEm\n",
       "SDKV5PYkv0iyKcmlY89tSfKtJBuSrE3y9m7/T5OcMva6Z7rN84EPdSvarpz2fVYkubnbPjDJ/Une\n",
       "uPN/QkkaJsuLNLsjgZXAMuBtSY7t9hfwj6o6HLgQuGBs/0zOBu6sqqOq6vvjT1TV1cDjSc4Efgh8\n",
       "vaqeXOCfQ5J2G5YXaXZ3V9VfanQfjfXAIWPPXd59vQL4wBzHmetml18AVgHPVtWVOxJUkhYLy4s0\n",
       "u/+Mbb8A7DnhdVtHXJ6ne18l2QPYe57fZ2l3/AO6Oz9LkiawvEg77tSxr3d121uAo7vtk4G9uu1n\n",
       "gP1mOkiSPYGLgdOAzcCXd0JWSdptTPorUlrMasL2dK9Lci/wLLD1MugfAb9Osh64Adh62fW9wAvd\n",
       "/kumnfeyCrijqu5KsgH4fZLrqur+hfhhJGl3k9FUvqTtkeRh4Oiqerp1FklabJw2knaMrV+SGnHk\n",
       "RZIkDYojL5IkaVAsL5IkaVAsL5IkaVAsL5IkaVAsL5IkaVAsL5IkaVD+Bwn7vQevE1gaAAAAAElF\n",
       "TkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105e7e1d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Read test data here\n",
    "a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
    "xt= a[:,0]\n",
    "yt= a[:,1]\n",
    "X = construct_poly(x,8)\n",
    "w = wRidge(X,y,1.0)\n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n)  \n",
    "    Xt = construct_poly(xt,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w),'g',label='regression training')\n",
    "    plt.plot(xt,Xt.dot(w),'b',label='regression test')\n",
    "    plt.title(\"MSE train: %f  MSE test: %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.ylabel('Output y') \n",
    "    plt.xlabel('Input x')  \n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####Conclusion:\n",
    "We get suitable solutions for n from 5 to 7 for the test regression. For higher values we get overfitting. For lower ones underfitting. The MSE of the test regression is always higher than the one of the training regression. That's evident because the regression function is fitted to the values of the training data and so doesn't provide an optimal fitting for the test data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum MSE: für lambda = 0.1\n"
     ]
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAk4AAAEACAYAAABF4/l0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecU1X6x/HPlw4KCqJgwQYo9o4NFVEEEbCgiB1WxQp2\n",
       "0Z8lRNe2u7oCll0r9i4KiigW7F0XCyhiAwRBEAUUpD2/P84dCCHJZCbJZMrzfr3yGnPvueecG8md\n",
       "Z06VmeGcc84550pXq9gVcM4555yrKjxwcs4555zLkgdOzjnnnHNZ8sDJOeeccy5LHjg555xzzmXJ\n",
       "AyfnnHPOuSzlFDhJaiXpNUlfSvpC0sDoeDNJYyVNkvSSpLXzU13nnHPOueJRLus4SWoJtDSz/0la\n",
       "E/gYOAzoB8w2s39IGgQ0NbNL8lJj55xzzrkiyanFycx+NrP/Rf+9AJgIbAj0BO6Lkt1HCKacc845\n",
       "56q0vI1xkrQpsBPwPtDCzGZGp2YCLfJVjnPOOedcseQlcIq66Z4CzjGz+YnnLPQF+r4uzjnnnKvy\n",
       "6uSagaS6hKDpATN7Jjo8U1JLM/tZ0vrArBTXeTDlnHPlYGbK5Xp//jpXunTfs5wCJ0kC7gYmmNnN\n",
       "CadGAicBN0Q/n0lxec5f/oogabCZDS52PUrj9cwvr2d+VYV6VoU6Qv6Cnqrw/HWuWDJ9z3Jtcdob\n",
       "OB74TNKn0bFLgeuBxyWdDPwA9M6xHOecc865osspcDKzt0g/TurAXPJ2zjnnnKtsfOXw0o0rdgWy\n",
       "NK7YFcjSuGJXIEvjil2BLI0rdgWyNK7YFcjCuGJXwDlX+eW0AGZOBUvmfezOOVc2+Xh2+vPXucwy\n",
       "fUe8xck551yNIulsSR9JWiTp3izSN5M0QtICST9IOqYMZf0g6S9J6yQd/1TSckkbR+87SHpH0m+S\n",
       "5kh6S9KuSfn8KWl+wmtoIe43uqaPpInRPU+WtHd0fJykhQl1mJhwzYKk+i1NrKOkjSSNiu5vhqRh\n",
       "kmpnKLNDdLyepLujz2Be9Nl1zeY+CiHn5Qicc865fIv2Pm1pZv9XgOx/Aq4GugANs0h/K7AIWI+w\n",
       "0PPzksab2YQsrjXgO+AY4BYASdtF5Vr0vgnwHHAa8DhQH9gH+Cspn+5m9moWZSYr0/1K6kyY5NXb\n",
       "zD6IlhUqaX0x4Cwzuyf5OjNbMyGPNYCfo/spMRSYDawPNAXGAmcCw0opsw4wBdjXzKZIOoQwAW07\n",
       "M/sx2w8hX7zFyTnnXGU0DOgtKe87T5jZCDN7FphTWtooADgCuMLM/jSzt4FngRPKUOSDwIkJ708C\n",
       "7mdlYLBFqJY9ZsEiMxtrZp+XoYy0ynK/kTgQN7MPoutnmNn0hPPZdPMeCcyMJpGV2AZ4zMwWR7uL\n",
       "vAhsXVqZ0eceN7Mp0fvnge+BnbO8n7zywMk551ylE+068TAZAhRJz0mam+Y1MotisgkAtgCWmtnk\n",
       "hGPjCUFAtt4DmkhqF3VNHU0IpkpMApZJGi6pq6SmZamvpFsl3ZpFPUq936h+uwDrSfpG0tSoS61B\n",
       "QrLrJP0SdSfulyarkuAw0YvAsZIaStoQ6AqMybLMxDq2IPx/+bK0+ykED5ycc86tQsLy8cpDVYYD\n",
       "fdOdNLPuZtY0zatnFvlnU8c1gXlJx+YDjbO4NtEDhFanzsAEQvdZqITZPKBDVJ87gVmSnpW0XsL1\n",
       "Ap5JCg5Pjq4/y8zOyqIO2dxvC6Au0Cuq046E7snLo/ODgM2ADYA7gFGSNk/MQNImwL7AfUl5Dwa2\n",
       "JXyeU4EPo5aw0spMzLsu8BAw3MwmZXE/eeeBk3POuVWYoXy88lCVdYFGktrnIa9UsqnjAqBJ0rG1\n",
       "CMFTtowQOB3H6t10IYHZV2bWz8xaEYKLDYCbk/I4NCk4vLsMdSC5zDQWRj+HmdlMM5sD3AR0i+r5\n",
       "gZn9YWZLzOx+4O2ScwlOAN5MHH8kSYQWpyeARkBzoJmkG4A/M5WZkEctwue4CDg725vONw+cnHPO\n",
       "VTrRrKn2wN+BfmnSvJA0iyvx9XwWxWTTAjMJqCOpTcKxHYAvsrh2ZUFhfM53wMHA06Wk/ZrQWrNt\n",
       "WcrIphqlJjCbC0zLsZwTWb21qTmhO+6WKOj6ldCi2M3MfiutzCjwupsQTPcys2U51rHcPHByzjlX\n",
       "qUg6FuhkZsMILRQ9Uo13MbODzaxxmtchGfKvHeVXB6gtqX7itPikMv4gBDpXSWoUTZHvQWj5KKuT\n",
       "o/tamHhQ0paSLojG/SCpFWEW3rvJVS9HmWW638i9wABJ60bjrc4jdMmtJamLpAaS6kg6jjD7b0xC\n",
       "WXsRWsueSMpzNjADOCOqz9qE1rfxmcpMuP52oB3Q08wSZxtWOA+cnHPOVRqS9gAONLOLAcxsPmGj\n",
       "+D55LOYKQvfQIMJ+qwuByxLqMFrSJQnpzyRM459FGNR9uplNpIzM7Dsz+yTxUPRzPqF17X1JCwgB\n",
       "02fABUlZjEpqVXsqqu9/JN2eoeiy3u/VwIeE1rYJwMfANYRxSFcTPodfgLMI3YeJA+dPBJ6KAs7E\n",
       "ezfC7MQehCDqG8JyC+eVUmbJmKn+hJa+nxPuP+v1tPLJVw53zrkqJB/PTn/+OpdZpu+Itzg555xz\n",
       "zmXJAyfnnHPOuSx54OScc845lyUPnJxzrgAkSdrsumgatXOumvDAyTnnCqL1EdD1TGh9eLFr4pzL\n",
       "Hw+cnHMuj6QN+kvtvoSO18JtTaDjdVK7L6UN+he7bs653NUpdgWcc656mXEnNPwVat8Y1ius3QCW\n",
       "ng8zMq4W7ZyrGrzFyTnn8igs9CeDWmtD7wmgptHh4iya55zLK29xcs65vKvXFsb2g29HhDFO9doW\n",
       "u0bOufzwFifnnMszswnXm01+2sws/JxwQ7Hr5FaS1EzSCEkLJP1Q2tYdks6W9JGkRZLuLWNZP0j6\n",
       "S9I6Scc/lbRc0sbR+w6S3pH0m6Q5kt6StGtSPn8mbbkytBD3G13TR9LE6JrJkvaWVE/S3VEe86J7\n",
       "6JpwTcbzSfm3jT7PB5KOJ5fbIeHcVpJejT6jbyQdls3955sHTs455yodSQMlXVug7G8FFgHrAccB\n",
       "t0vaOkP6nwh7qd1TjrIM+I6waS8AkrYj7H1n0fsmwHPAEKApsCEQJ+zllphP96SNjAdmWYcy3a+k\n",
       "zsD1wElmtiZhI9/vCb1UU4B9zawJcDnweLSXHFmcT67TB6zcry9dud9F5+oAzwIjo8+oP/CgpApv\n",
       "zfW96pxzrgqpKXvVRetffQPsbWYz85jvGsCvwDYlm9NKug+YbmaXlnLt1cBGZtavDOV9D9xF2Ay3\n",
       "fXTsX1Ed/g5sSghoxppZ01LyOdnMXs227Oi6Mt+vpHeAO82s1NY1SeOBwWY2ItvzkvoAhxM2821j\n",
       "ZieUVq6kbYF3zaxxwrEXgffN7MrS6llWvledc865KiUaTP8wcEK6NJKekzQ3zWtkmsu2AJaWBBGR\n",
       "8cA2WVSrvMHme0ATSe0k1QaOBh5MOD8JWCZpuKSuktIFUCnLl3SrpFvTXFOm+43qtwuwXtQdNlXS\n",
       "MEkNUqRtEeX/ZZq8Vjsfta7FgfMS76cs5SaoBWyb4XxB+OBw55xzq1BceemKsFjOrVrDCV0z/0qZ\n",
       "v1n3cuS5JjAv6dh8oHGKtKsVWY7ySjwAnAi8QWhp+WlFpmbzorE8g4A7gZaSRgOnmtmsKJmAZyQt\n",
       "TcjzQjO728zOylBuWe+3BVAX6AV0AJYSusguj16hMlJd4CFguJlNSs4kw/mrgbvMbLq0yr+z0sr9\n",
       "Gpgl6SLgZmB/YF+gTC1w+eCBk3POuVXkIeDJl3WBRpLam9kHecpzAdAk6dhahGCiNOX9XIwQOL0J\n",
       "bAbcn5yXmX0F9AOQtCWhRepm4NiEPA4ta1cdZb/fhdHPYSVdpJJuIiFwklQrup9FwNnJGaQ7L2lH\n",
       "4ABgp5JD2ZZrZkuiweDDCAHmh8DjURkVygMn55xzlU40G6stYRxQP8JA4uQ0LxBaJ1J5w8wOSXF8\n",
       "ElBHUpuE7qsdgC+yqFa5W5zMbIqk74CDgb+VkvbraBxSPlabL9P9mtlcSdPSZRaNPbubENR2M7Nl\n",
       "ZTi/H2FM15SQjDWB2pK2MrNdM5Ub1e1zoGNCWe8AZZrlmA85j3GSdI+kmZI+Tzg2WNK0aCpi2umI\n",
       "zjnnXDJJxwKdzGwY8ATQI9VYFzM7OGmWWeIrVdCEmf0BPA1cJalR1EXWg9BCkq4+taPy6xB+0deP\n",
       "xuSU1cnRfS1MPChpS0kXSNowet+KMAvv3eSqlLXA8twvIRgZIGndaLzVecCo6NztQDugp5n9leLa\n",
       "TOfvADYnBG47Av8Bnge6ZFEukraT1CC6jwsJ3XvDs/kc8ikfg8PvBZIDIwNuMrOdoteYPJTjnHOu\n",
       "mpO0B3CgmV0MYGbzgWeAPnks5kzCcgCzCF1ip5vZxIQ6jJZ0SUL6K4A/CV1ExxO6lS4ra6Fm9p2Z\n",
       "fZJ4KPo5H2gPvC9pASFg+gy4ICmLUVp1Haenovr+R9LtGYou6/1eTegKm0QYj/UxcE20rEB/QuDz\n",
       "c0I9jonyyXjezBaa2azoNZPQjbjQzOZkKjehXicA04GZhDFOnc1sSYb7Loi8LEcgaVNglJltF72P\n",
       "AQvM7MYM11T66bDOOVfZ5OPZ6c9f5zIr1nIEAySNV1hFdO0CluOcq0mko5G+RbqQMHPHOecqTKEC\n",
       "p9sJMwd2BGYAaVuenHMuK1IdpH8C1xHGPnQGPiFhSwbnnCu0gsyqS1h3Akl3kTC4K5GkwQlvx5nZ\n",
       "uELUxzlXxUnNgUcNlu/LG+e/xT6d67L4osXU3xJ4FGkscDFmvxS5pnknqSMJM4mcc8VVkBYnSesn\n",
       "vD0c+DxVOjMbnPAaV4i6OOeqOGlng4/eY/ff6rJkrbfY53pgwRLqjRXWuS/37kvYUuJLpP6ENWSq\n",
       "DTMbl/isLHZ9nKvp8rEcwSPAO8CW0RLpfwNukPSZwh41+xGa1Z1zrkwWqmG/hTQY15fhtffkvU2X\n",
       "UecfwDZmXApsCcy/j74fCJs5jQ27AX2Bd5B2ypSvc86Vl2/y65yrdI7WY03O4taRGzFt72N45OMP\n",
       "2P1yOg5+hY7xrQhr0OxB2P/qLR4eOZNJPWLA9nVZPGgRDZrUwq4FHgGuxCx5u4kqzWfVOVd4mb4j\n",
       "Hjg55yoNibW34YuL76D/hcCcC+tfdtK7l3ZfSgiWegD1CWMm3wK2IqwavRswmdlb/sA7F+zEDx1/\n",
       "7v7rV/83ip7HEFZpPh94nGI97PLMAyfnCs8DJ+dcpSaxPnDuXrx9+rPqUeetVnrnyL6/zllWiy7A\n",
       "N4RgaRQw3mKrPrQUVz3C3ld7Y+rA0gYH8uc6jfi17Y8nT/vspaGvzzmgwVJ+rAVnkWoz0hbqRhMG\n",
       "Uo8GLGYR8xhqM210Bdx2uXjg5FzheeDknKuUJDYHLqb5xD4XNj951qU/vLvZyYey5JmteJkQKD1n\n",
       "MZtRpjzjEj922JGpe/6dxj8fUGfz0fPPHT9njcvfpM6rm/HWBV249vumvG0xW6gW6sZGDKEnbVZk\n",
       "MJLJTOOcyho8eeDkXOFl/I6YWVFeoejilO0vf/mruC8azdqJrR8fS9eBf9Y/r/mce3bgj2mNmdv3\n",
       "UE5jMA3zVg62OdgTrDl96vZ79/nvuxvy9ZQmLDrkWBYxmHfZle8ZjK32asMLxf6M0t8TVhnyqMov\n",
       "4GzgI2ARcG+K882AEYQtQX4AjsmQV9Zp01z/A/AXsE7S8U+B5cDG0fsOhIlYvwFzCN3Vuybk8Sdh\n",
       "65aS19AifR4bEf7omUNYx3EYUDs6txXwanQP3wCHlaWcLM73ASZG5ycDHVLUr210nw+U8plYunMF\n",
       "WcfJOVf1FLrLSnGtxVc9B1J78WkMeH8DljaYsd1vv9/91q1/7td4MZMFfe99xublc6tzM74DjpLW\n",
       "3/eztx+5eU8emfMgx9028uGHB05vzOI2W1A71S6lNKKZ4lJyt6CrOJIGAi3N7P8KkP1PhH3RuhD2\n",
       "cEt2K+GX63qEbuDnJY03swk5pk3FgO8Im/reAmEz26heFr1vAjwHnAY8Thjrt09Ubkke3c3s1SzL\n",
       "TJbPz2MoMBtYH2gKjAXOlHQb8CxwG3AAYW2yUZJ2MrNvsiwn7XlJnYHrgd5m9kG0LFKqFqNbgQ9Y\n",
       "uU9gmXlXnXOOQnVZKa7WmHrwx3p9qTd/W6btuZh5G41gvS+utDs+Xh94jPAgux6z5bnfSYa6iFrA\n",
       "icA1azJ/3Nds+fMJG8wY8Gp/Vt+25WUWcSC/E/6qL3mNt1jFbyiarKZ01UkSoVVibwsbwhaijKuB\n",
       "jcysX8KxNQjrgm1jZpOjY/cB083s0qTrs06boQ7fA3cBh5pZ++jYv6J8/w5sSggUxppZ0wx5nJxD\n",
       "4FSST06fR3Tua+AcMxsTvf8H0JjwPX/XzBonpH0ReN/MriytnCzOvwPcaWZp//aS1IewtuQEoI2Z\n",
       "nZAhbdrviLc4OVcF5b11qAkDVwmaAHrShgcZAGSdr+KqTVgqoAdGD5Y22JCvey7j657z+XHfU5nX\n",
       "6gFDy4AzgBjQF7MXyl3vMjBjOTBc4skFNL5kQ6af0WbuwBc2eGpY5+m9Ev7KHsm3TGMgB/IloXtk\n",
       "b+BvwGaK6wNWBlLvWczmV0TdayIzM0kPAycA/0qVRtJzhP8/qbxpZj1LKSbVL8YtgKUlv5wj40m9\n",
       "entZ0mbyHnCCpHaEYPFown39PTo/CVgmaTjwKCHYmJuUR9pAWNKtAGZ2Vin1yPXzAHgROFbS64Su\n",
       "tYOBy9OkrQVsm2U5ac8rLHq7C/CspG+ABsAzwEVmtghWtNrFgf2B/mnqkxUPnJyrYtK0DrVWC1Hu\n",
       "4KkeDdIcT9Vsv2p94mpMaOLvAXTDNIPvDpzK65c3YeaUn1j/4kU0enw+63H0OnWZyxwOJTzk9sLs\n",
       "23LVNwdmLAAul7hz8sKh1/P1fgs2uPu0WZvUnrP+omX8sugPrpgwZ8Xn+CPwUHSfTYE9CcHUFcDO\n",
       "imsSCa1SFrPpFX0/BSHlpysi91at4cBI0gROZtY9x/xT3eeaQPLaX/MJrSa5pC3NA4QW0TcILSI/\n",
       "raik2TyFPRkHAXcCLSWNBk61sMWZgGckLU3I70Izuzu6vrSAaUVRKY6V9R4HAy9H19QGhpvZswob\n",
       "cs+SdBFwMyGA2Zcw5imbcjKdbwHUBXoRvp9LCd2Cl7MyaLsauMvMpivHf98eODlX1eSpdWgVi1eM\n",
       "lUg+vjDVYcW1CSvXVtoTeJeFa7/Ew6OmMrVDP+AnGp84jG0eOJWebF1yXaMn6XjXH7x3yiL2xOyP\n",
       "8lQ1dOFsei388H+Ww1gDM34EjpF67TV9aq9//8nc2e+y5zft+HoIUmPgrsTuQ4vZXMLnOzr6DOoD\n",
       "OxMe1McBtymueYQg6u3o50SLFbYLsiAqTzfeukAjSe3N7IMC5J/qPhcATZKOrUX4JZ1L2kyMEDi9\n",
       "CWwG3J9cNzP7CugHIGlL4EFCAHJsdP2huXbVJZcZyfoeo+7VF4EngN0JQc09km4ws0GSDiMMFh8E\n",
       "fEgYr1Xy7CmtnHTn58GK59Swkm5dSTcRBU6SdiSMqyrZUSCnf9/Vak8n52qEHFqH0prHUEYyeZVj\n",
       "I/mWeQwDUFy1FNfuiusaxfUZYQbOrsAdPHf7dgy297lh7iVM7dAGOMSMg2nxQKfkAG/qkdQ/dSMW\n",
       "ljdoClofAV3PhNaHlz+Plcx4B9jzN5revBVfdejEK58spMFpwNuEB27q62L2l8XsXYvZPy1mhxJ+\n",
       "yXcjtBjsSfiLtxC/7GsESV2B9oTuqn5p0rwgaX6a1/NZFJMq8J4E1JGU+G93B+CLHNNmrojZFMIg\n",
       "8YOBp0tJ+zVwH7BNWcsprRopjpXlHpsTWpNvMbMlZvYrodWwG4CZfW5mHc2suZkdDLRm5XektHLS\n",
       "nf/SzH4DpmW4r46EsWJTJM0ALgB6SfoowzVpeYuTc1VNGVuHsmEzbbRaCB5kAPVoyGIWsoQ7OZN6\n",
       "iutu4BDC9OJRhPFJ7zHYNiA8gO4EngT2NEsIvvIc4Ekb9Icm50DHOnBbEzj1Oqnd1TBviNn0O8qT\n",
       "Z4lo/NODEiNeo9OFa7Jg4A0Mevd8bnqpVhhnU+rWLdEMvK+i112wohvTlZGkY4Edzexihda/qySd\n",
       "VzJepUT0y7c8+dcmdO3UAWpLqk8YP7PMzP6Q9HRU5imEVsWSltVVlCVtlk4G1jazhZJW/H6OWpi6\n",
       "A4+a2U+SWhFm4b2XeFvlLDNvnwdhNt0M4AxJNxJanE4ijEVC0vaEAKgWcCahi204lP5ZZlGPe4EB\n",
       "ksYQuurOIzyvAP4LPJzwOV1ICKROL8/n5S1OzlU1pbQOlZfNtNEcz6n05jGOx+jHcML6Lp8Be1nM\n",
       "trGYXcJgm81gu5PwMFwKbGtG/1WCJqD2X6SegVbuAG/GnbAkBrUbhGdf7Qaw9MpwPD/M+MOM+HJq\n",
       "73AR//q1JT8v/5iddzaYgHQ0oSsi+/x88HiZSdoDONDMLgYws/mEgb598ljMFYR1jwYBxxO6ei5L\n",
       "OH8mYVr+LEKX2OlmNjGq32hJl2STtqzM7Dsz+yTxUPRzPqH17X1JC4B3Cd/LCxLSjkpqcXuq5ISk\n",
       "2yXdnqHovHweUdf5EYSAZjZhoPtfhCCGKO/pwEzCGKfOZqvMVC3ts8x0/mpC998kwhixj4Fronot\n",
       "NLNZ0WsmodtvoZnNyfCZpOXLEThXBUWz6la2Ds1jWHkGhiuuWqz8y60HsAnwAuEvtTEWs99XpBW7\n",
       "AJcSBnTeAtxixq+pM9bmj9Xn5XO2oPnMXgmDSKMZa+UdxC616QWd74E506BZK3i5r9nkjN0auZBo\n",
       "D9zcmZfWeZIj6zRh/rfA2aTYuqWi1JTlCJwrJt9yxTm3guJqRBgo2YPQ/D+PlXvBvWMxWzEzR0KE\n",
       "vwwvBdoRZjfdZUb6MUrSQYSBrn+v3Zxvl6+de4C3MuutL4HFk+DbEWGMU722ZhNuKG9+2ZWJgN51\n",
       "WPKPa7jstwu4sVVtlt8CXIdZubtHy18fD5ycKzQPnJyr4RTX+oQgqQdhoOQnRMGSxVJsfBsWi+xJ\n",
       "CJjWAm4AHjJjcfpCJEJT/0CgD2Zv5PcuikuiIXB+K6Zc+ARHzdyVj+rXZvmZFbUO1cp6eODkXKF5\n",
       "4ORcDaO4RJhx0oMQALUhTBMeSeiCS9nFJlGXMOh0EGGcw3XAM2Ysy1ygGgP3AK2AIzHLNMOlSpNY\n",
       "H7imO6MOe4ATljZh3hu1sPMwm1ox5Xvg5FyheeDkXA2guBoQutVKuuAWs7IL7s1M24VINCLM6LkQ\n",
       "+JYQML1slsV+TlJbwuDdt4EBmKXc/q26kdi5EX/cHCe2xUCGNqzHkquAoVhht2XxwMm5wvPAybki\n",
       "y9eijavlG9d6hKUCegKdgM8JrUqjgK9K26RWYm3gLEL32rvAdWa8n30F1J3Q0nQ5ZjktCVAVReOf\n",
       "jtiKCf8eTt8G2/H5rw1ZdDJmbxeuTA+cnCs0D5ycK7KVs8HG9stlFljUBbctK2fBbUXYfXwUMNpi\n",
       "Nju7+tCSMEX4FMKu6/8w48vsK6JahCnMpwC9MXu3DLdR7Ug0qMWygUfz2OW3cpbqsfjZNfjzXCy7\n",
       "/x9lK8sDJ+cKzQMn54pk5aKNHerAnVvAqZPgraVlWbRRcdUD9iO0KnUnLGJU0gX3usWy7xqT2By4\n",
       "iLCJ6EPAv6KtR8pyU2sRZs01BY7C7OcyXV+NSbRYj5nXXcHVR/fj3mV1WXJRPZbcQT5bGT1wcq7g\n",
       "PHByrkhCF93mveDAG+G/G8NpU+CV8+HbpzN12Smu5oRtCnoAnYGJrAyWviitC271erAdcAlhM97/\n",
       "AkPMmFWOG9oaGEFo5Tofs/Sz7GowiR325q07evN4y3MYumlWY8WyztsDJ+cKLdN3xLdcca6AzMyk\n",
       "Nga11obeE8KijVhy0BR1wbUjtCr1ALYj7Bo+CjjbYmHjyrKS2JsQMO1K2BD0TDN+z3xV2syOIARd\n",
       "F2E2vFx51BBmjJc67PE2HTbOZ9BUE0kaDkw1syuKXRfnwAMn5ypAvbYwtl/ioo0AiqsusA8rxyvV\n",
       "JwRK1wCvWcxS70lXimjAclfCGkwbAf8EepuVc6uTsI/V1cBxwMGYlWtjzJomCpjK1g3qUjFSbz67\n",
       "GknjgAfM7O7yFJSvIE3SpoQNe+uY2fJc8nKVjwdOzhWY2YTrS/5bcb0GNFBcjxCCm8mEYOlIYHxZ\n",
       "u+ASSdSO8rkEqA1cDzxuxtKMF2bOtBlhc8z6wK6Y/VLuvFyVkI8ZoAWYRZptt2Jla93z7tBqyDf5\n",
       "da4CKK4+imscoQWiD6EbbhuL2W4Ws6ssZv8rb9AkUV/iVOArwrICVwA7mPFwjkHT9oRNM78EOnvQ\n",
       "VFO0PgK6nhlaRys+D0k7SfpE0jxJjwINEs41lfScpFmSfpU0StKG0blrCC24t0Sb3A6Njg+RNEXS\n",
       "75I+ktQhTbn9gWOBi6Prn42ObyDpqajM7yQNSLimfZTn75J+lvSv6FTJqvm/RXntXtbPwVViZlaU\n",
       "F9FGyv7yV3V/MZhjGcxUBtOTwTTKW75YY7ALwH4CewFsXzDlJX/oY/CLwbHF/vz8lfz/HStEHrB+\n",
       "f9jySzj5a1hu4eeWX8L6/bPPN7c8gHqEPy7OIbSa9iIs5HpVdL4ZcDghmFoTeBwYkXD9a8DfkvI8\n",
       "jjADtBZwPjADqJem/HtLyore1wI+Bi4n9NBsRlgg9qDo/LvAcdF/NwJ2j/57E2A5UKvY/178Vb5X\n",
       "pu+Zd9U5V0CKqzNhUHYni9kXeclTNCe0LJ1BaLnqbsan+cgbqQ6hi+9w4EDMxuclX1cFzLgTGv4K\n",
       "tW8MPUy1G8DS82FGGdYdyzmPPQjjgoZE75+S9GHJSTP7lTCrEwBJ1xK+A4lW6R4zs4cS3t4k6XJg\n",
       "S8JisakkXr8b0NzM/h69/17SXYRW45cIQV1bSc0trNn1foo8XDXjXXXOFYji2pkwPujIfARNEq0k\n",
       "bgYmAS2Bvcw4Oo9BU3PCfnbbAbt50FSzmJmBEmaAqikpZoAWOI8NgJ+Sjv1IFIhIaiTpv5J+kPQ7\n",
       "8DqwVhhTtbIaiRdLulDSBEm/SZpL2LS6eZb12QTYQNLckhdh0sV60fmTgS2AiZI+kHRIlvm6Ksxb\n",
       "nJwrAMWcoik0AAAgAElEQVTVmrAi92kWszdKS58xL7ElYdPdwwjbm2xnttovl9xIOwNPEwK9KzDL\n",
       "vKmvq6ZSzwCtwDxmABsmHduEMIkC4AJCoNLezGZJ2hH4hBBYrTb7TtI+hAVfO5nZl9GxX0nfIpQc\n",
       "4E0BvjezLVImNptMGBeFpF7AkwoTKirbIHWXRzkHTpLuIeyVNcvMtouONQMeI/yD/wHobWa/5VqW\n",
       "c1VBtH/cGOBqi1n5t1cRuxD+ut0XuAVoY8av+anlKgWdCPwLOAOzp/Kev6syEmeAEgLpis7jHWCp\n",
       "pIHA7YRlOnYDXonOrwksBH6Pfs/Ekq6fCbROeN8YWArMllSPMOO0SYbyZwKbJ7z/AJgv6WJgGKFr\n",
       "biuggZl9JOl44EULEyd+JwRMy4Ffop+tgW+yv31XFeSjq+5ewrTqRJcAY6Mo/ZXovXPVnuJaE3ge\n",
       "eNRidnuZrxeS6CQxFngGeAvYzIyr8h40SXWRhhEGvu7vQZMrNjNbAhwB9AXmAL2BxH+XNwMNgdmE\n",
       "IOsFVm3dGQIcGc24u5nwB8wYQvf2D4Sga0qGKtwNbB11yz1tYQ2m7sCOhHWZfgHuYGXw1QX4QtJ8\n",
       "4N9AHzP7y8z+JKzH9naUV/tyfByuksrLlivRYl+jElqcvgL2M7OZkloC48ysXdI1Zr7kv6tGogUt\n",
       "RwHTgFPLsryARC3CquGXAmsDNwAPmlGYLU3C9/Jxwl/JJ+AtwlWGb7niXOEVY8uVFmYrtoiYCbQo\n",
       "UDnOVQrRlil3A0uA07MNmiTqAscQxjAtAq4DRphRuDFG0h7AE4T6XoWvbOycc1kr+OBwMzNJKX+J\n",
       "SBqc8HacmY0rdH2cK5DrgbbAARazUhedlGgE/A24kNAFcC7wslmBB5VKpxK6EE7BbGRBy3J5Iakj\n",
       "0LHI1XDORQoVOM2U1NLMfpa0PqTehd3MBheofOcqjOI6FzgU2Nti9mfGtGJt4EzCOkzvAX3MeK/w\n",
       "lVR9YChhZeV9MPu64GW6vIj+oBxX8l5S8oBo51wFKtQ6TiOBk6L/PokwyNW5akdx9SG0GnWxmM1J\n",
       "m060kLiesOpwO+AAMw6roKBpQ8Iv3ubA7h40Oedc+eUcOEl6hDC7YUtJUyX1I3RbdJY0CegUvXeu\n",
       "WlFcnQitON0sZj+mTSc2I2zb0BjYxYwTzfiyYiqpDoQp1SOBIzGbXyHlOudcNZWXWXXlKthndbgq\n",
       "THHtSNhyobfF0o/Nk2hJWFLg32bcWkHVg7CS8pnAlcBJmI2psLJdQfmsOucKrxiz6pyrthTXZoS1\n",
       "ms4qJWham7CGzP0VHDQ1JCweuBOwJ2bfVVjZzjlXzXng5FwCtVA3mjCQejRgMYuYx1CbaaNXnI9r\n",
       "XUIwdJ3F7Im0+YRZc88R9tK6uuAVX1nwJoQFA78B9sLsjwor2znnagAPnJyLqIW6sRFD6EmbFQdH\n",
       "0lothM200YprDUIw9KTF7Ja0+YS1mZ4AvgfOK/gSAysL7gQ8BPwT+DfF6od3Lo8kDQemmtkVxa5L\n",
       "NiQdB5xoZl3ymdZVHoWaVedc1dOEgasETQA9aUMTBkSrgj8BTCBsUZJStAL4cMI+VX8zo/CLS0pC\n",
       "Op8QNB2H2U0eNLlqZLXNe9ORNE7SyeUtSNJwSTm1EJvZQ9kGQmVJ6yoPb3FyrkQ9GqQ53hC4k/Dw\n",
       "7p9uVXAJEfbKagV0MWNJgWqaWOgawF2EHeP3wNLP7nOuNKV1VVdUHqmyzTJdQf9gkFTHrPQFbl31\n",
       "5i1OzpVYzKKUxxvRirAjem+LWaZgKEZYYLKHGQvzX8Ek0uaEpUD+Ajp40ORysaKr+ni60Jv9OJ4u\n",
       "bMQQtVC3Cs1D2knSJ5LmSXoUVv5BI6mppOckzYo28h2lsE4Zkq4hfP9ukTRf0tDo+BBJUyT9Lukj\n",
       "hSU6UpXbHzgWuDi6/tno+A+SLpb0GTBfUm1Jl0iaHNXxS0mHJeTTV9KbCe+XSzpN0qRow99bMqQ9\n",
       "SNLXkn6TdKuk13NpQXOF4YGTcyXmMZSRTF7l2AvMZivqAYdYLP1Aa4mBwHGElqbfC1tRQOoCvEvY\n",
       "qb0fZoUP1Fz1lqGruqLykFSPsGDyfUBTQvd4L1a2JJXsCblx9FoI3AJgZpcBbwJnmVljMxsYXfMB\n",
       "sEOU38PAE1E5qzCzOwjd3TdE1x+acLoPcDCwtpktAyYDHcysCRAHHpSUaU/WQ4Bdge2B3grf3+R7\n",
       "bx7d7yCgGfA1sCcFbkVzZeeBk3MRm2mjmcY5PMgYHud1RvA/tkS0Zl+L2ex010kcB1wEdDZjZrp0\n",
       "eRHGM10K3ENY0PJWH8/k8iJzV3VF5bEHUMfMhpjZMjN7Cviw5KSZ/WpmI8xskZktAK4F9kvKY5Vu\n",
       "vWgc0VwzW25mNwH1gS0z1CG5W9CAoWb2k5n9FeX5pJn9HP3344RZrLtnyPN6M5tnZlOB14AdU6Tp\n",
       "BnxhZs9EdR0K/JwhT1ckPsbJuQTRWIzRiqsj8DhwoMXs+3TpJQ4BbiRsofJDQSsnNQbuBTYC2mP2\n",
       "U0HLczVLuq7qxWXods49jw2A5H/XPxIFM5IaAf8GuhBakADWVLRaYfR+lT8kJF1I2FB7g+hcE8L2\n",
       "Q2UxNSnPE4HzgE1L6gCsk+H6xADoT2CNFGk2AKYlHUt+7yoBb3FyLoni2p4QNPWxmP0vbTqxD2EG\n",
       "3aEF30JF2oKwKfAcYD8PmlzepeqqHsm3zGNYBeYxA9gw6dgmrAyGLiBMhGhvZmsRWpvEylai5KBp\n",
       "H0Jr8FFmtraZNQV+Z/VWpRLpWm9XHFdYK+0O4CygWZTnFxnyzNZ0wh9FJeUo8b2rPLzFybkEimsT\n",
       "YDRwtsXs1bTpxI6EhSaPNeP9wlZK3Qldc5dhdmdBy3I1ls200WoheJAB1KMhi1nIPIaVZUZcHvJ4\n",
       "B1gqaSBh9fsewG7AK9H5NQnjmn6X1IwwISPRTKB1wvvGwFJgdjSu6RJCi1M6M4HNS6njGoRAajZQ\n",
       "K2p92raUaxIlBnqJRhMGth9K2JngdKBlGfJ1FcQDJ+ciimsd4EXgHxazx9OmE20JD7mzzBhbuAqp\n",
       "FnAFcArQE7P3ClaWc6zsqi5WHma2RNIRhOU//h7l81RCkpsJA7xnE7r0bgJ6JpwfAtwn6QzgfuB8\n",
       "wkr/k4A/CN18UzJU4W7C4PG5wGtmdkSKOk6QdCNhcsbyqJy3EpOwastVcitW4vkV/21msyUdRdg4\n",
       "/D7CQPWPCLNmXSXim/w6B0Srgr8MvG4xuyRtOrEhYebOdWYUrvVHWgt4gDCO4yiigajO+Sa/NYPC\n",
       "H05TgWPN7PVi16emyfQd8TFOrsZTXHWAxwh/lV6aNp1oRmiRuqPAQdPWhJlEPwIHeNDkXM0QreO0\n",
       "tqT6wP9Fh72luZLxwMnVaIpLwH8J3danZFgVfE1Ct8ELwA2Fq5B6ETYGvgazAZgtLlhZzrnKZk/C\n",
       "GlG/ENZ+OqxkCQRXeXhXnavRFNfVQFdgf4vZgpRpRH1gFGFq8MkF2bRXqg1cTVi5uBdmH+e9DFct\n",
       "eFedc4WX6Tvig8NdjaW4ziSsCLx3hqCpNmGs0R9A/wIFTc0IA17rArth9kvey3DOOZcX3lXnaiTF\n",
       "dQRwGdDFYjYrZZqwae9thIXtjjEj/5t7SjsQxjN9AXTxoMk55yo3b3FyNY7i2hf4DyFo+i5D0muA\n",
       "nYFOZmlWRM6pIjqGMPV4AGaP5j1/55xzeeeBk6tRFNd2hI00j7WYfZo2nbgAOBzY14z5+a2E6hAG\n",
       "mB9GmDX3WV7zdy4LknyPQ+fKwQMnV2Moro0JM+POtZi9nDad6AcMBDqYkd+uM2ld4FFgCWE80695\n",
       "zd+5LPjAcOfKz8c4uRpBcTUjrCB8k8XskbTpxGGEHdcPMlt1Y8/cK6FdCOOZ3gMO8aDJOeeqHl+O\n",
       "wFV7iqsRYVXwty1mF6VNJ/YnbO7b1Yz8LgcQ9rP6F3A6Zk/nNW9Xo/iz07ni8q46V61Fq4I/AnwH\n",
       "DEqbTuxKCJp65zVoCuszXQv0AjpiNiFveTvnnKtwHji5aitaFfw2oAFwlMVsecp0oh1hgctTzXgt\n",
       "fxVQY+BBwm7su2M2J295O+ecKwof4+SqsxhhOYEjLZZ66xKJjQn7z11qxjN5K1nahLBj+kzC+kwe\n",
       "NDnnXDXggZOrlhTXacDxwCEWs5TLCUisC7wEDDFjeP4K157Au8C9wGm+35xzzlUfPjjcVTuK6zBC\n",
       "F90+FrNvU6YRTYBXgRfNuCx/het44CagL2aj85avcxF/djpXXAUNnCT9AMwDlgFLzKx9wjn/8ru8\n",
       "U1wdgKeBgy2WeqNciQaE9ZwmAWfkZf85qRZhk95jgB6YfZlzns6l4M9O54qr0IPDDehovl6NqwCK\n",
       "axvgKeD4DEFTySy7X4Cz8hQ0rUHYCLg50B6z2Tnn6ZxzrlKqiDFOaf8yUluNUQt1q4A6uGpOcbUC\n",
       "XgAusJi9lDJN2LT3DqARcIIZy3IvWK0Ig8B/Azp70OScc9VboQMnA16W9JGkU1c7ezxd2IghHjy5\n",
       "XCiupoRVwYdazB5MmSYETf8EtgKOMCP3AdvS7oRVwB8CTsbsr5zzdM45V6kVuqtubzObobA/11hJ\n",
       "X5nZmyvOvgY0pg1/MUTSn2Y2rsD1cdWMLlRDxq0znu5znkTcmCHpIKArYdPeP3IvWMcAQwgB06ic\n",
       "83MuDUkdgY5FroZzLlLQwMnMZkQ/f5E0AmgPrAyc9o9+/sJP9psHTa5sFFd9JjZ5g/GHt+STV962\n",
       "5d+lHK8k0R/oT9i0N7fxdmEQeAw4ETgAs89zys+5UkR/UI4reS8pVrTKOOcK11UnqZHCyskoDJ49\n",
       "CEj9S2YxCwtVD1c9qW7LM/j35r/wZrd2LL2jLtbpWqndl9IG/VdJJ44iBDoHmTE9t0LVCHgMOJCw\n",
       "ErgHTc45V8MUssWpBTBCUkk5D5mlGLQ7km+Zx7AC1sNVM4qrLpdxEB82+YrRTVqC1oTaDWDp+TBj\n",
       "xQa6EgcBtwKdzZicW6HaEHgWmEBoaVqUU37OOeeqpIIFTmb2PbBjxkQPMoZ5DLOZvlCgy47iqgs8\n",
       "iqjN2GY3Qq07oPcEaNYKMIsWJpPYgzBo+wgzxudWqHYFRgC3AP+gWKvGOuecK7qibvJr39jBxSzf\n",
       "VS1R0PQwUB/oxZKG58HYfvDtCGh9ONRrCyCxDaF1qK9Zwpi6chWqowitVv0xy99eds4556ok33LF\n",
       "VQmKqw6hBakxcITFUneVSWxKmIBwiRkPlb9ACbgCOBk4FLP/lTsv5/LIn53OFVdRW5ycy0YUND0A\n",
       "rAUcliFoagGMBf6RY9DUELgH2IwwCPzncuflnHOuWqmIlcOdKzfFVRu4D1gHODxD0LQ28CLwkFkO\n",
       "kw2k9YHXgeVARw+anHPOJfLAyVVaUdB0L2GG5qEWs5TLVkg0AkYSuuji5S9QOwHvE8ZHHe8z55xz\n",
       "ziXzMU6uUoqCpruBjYHuFrM/U6YTdQkz3n4DTjRjefkK1BHAf4EzMHuyXHk4VwH82elccfkYJ1fp\n",
       "KK5awJ3AJmQOmtYhdOMB9CtX0BQGgV8KnAF0xezjclXaOedcjeBdda5SiYKmO4DWhKAp5b5yEvsB\n",
       "/wO+BnqZsaTshakBYdD54YRB4B40Oeecy8hbnFylEQVN/wG2BA5OFTRJ1CEsE9Af+JsZL5SvMLUA\n",
       "ngGmAPthqVu1nHPOuUTe4uQqhShoug3YGuhmMVuwWhrRCngV2BvYOYegaQfCIPAXgT4eNDnnnMuW\n",
       "B06u6BSXCNuZbE8ImuavlkYcDnwEjCZs2DujfIWpJ2Gtp0GYDfbtU5xzzpWFd9W5ooqCpqHAzsBB\n",
       "FrN5q5wXDYEbgYOBQ814r3wFScBFwEDgEMw+zKXezjnnaiYPnFzRREHTzUB7UgdNWwOPAhOBncz4\n",
       "rXwFqT5hqYHtgT0wm5ZLvZ1zztVc3lXniiIKmm4C9gK6WMx+X3FOSOJU4A1Ca1SfHIKmdYFXCHvc\n",
       "7eNBk3POuVx4i5OrcFHQ9E9gX+BAi9mKoCjaOuUOwsy6fcyYWP6CtC0wirA58JWYlW9xTOeccy7i\n",
       "LU6uQkVB0w3AAUBni9ncFefEnsCnwCxg9xyDpu7Aa8BlmF3uQZNzzrl88BYnV2GioOla4CDgAIvZ\n",
       "rwAStYBBwLnAaWY8U/5CJOA84EKgB2blG0zunHPOpeCBk6sQUdD0d+AQoJPFbA6AxAbA/UA9YFcz\n",
       "ppa/ENUDbgd2IQwCn5JrvZ1zzrlE3lXnCi4Kmq4CehJammYDSHQDPgbeBDrlGDQ1J6zPtA7QwYMm\n",
       "55xzheAtTq4ixIAjgP0tZr9I1AeuA44EjjbjjZxyl7YmDAJ/nDCmycczOeecKwgPnFxBKa4rgd6E\n",
       "oGmWRFvC2kxTCWszzSl7pqoLbAK0BbYBLgYuxOz+vFXcOeecS8EDJ1cwiusy4BhC0DRT4gTC2k2D\n",
       "gdvMSL/dSQiONiUER22Sfm4ETAcmA98APX0QuHPOuYqgYm3VJcnMTEUp3BWc4roUOAnYn8G2gLCB\n",
       "766ExSzHh0SqR+rgqA0hOPqJEByVBEglP3/A7K8KvB3nKg1/djpXXB44ubxTXBcDpwAdNxw8tVVT\n",
       "5j7Wgbe+uoFBrzZh/iasGhxNY9WgKDE4WlycO3Cu8vJnp3PF5YFTJmFNoLr+C7wUYS+4zYA2D2zP\n",
       "qUtqsd/RXzB+8dK1t23En83+ov7MJsz/lNWDox/9s3WubKrEs9O5aszHOGW2NfA60pOEbTverrEz\n",
       "tkJwtDmrjzdqA6wPTP16HZYsqsMGu0xu/p/Tlv77gM/YfuofrLHHt9b6m+JV3DnnnMsfb3EqjbQp\n",
       "0Ac4DmgCPAI8hNnnRaxVYUgNSB8ctQSmsHqr0WTgRw3mTGAgjz1xGROPvJGwqOWVZiyp8Ptwrhqr\n",
       "Ms9O56opD5zKQtqeEEAdA/xGaIV6pEotthiCo9asHGeUGCC1AH4kTXCE2dKUWcY1AOM87nx/NNPb\n",
       "Hw6cZMbLBb8X52qgKvnsdK4aKVjgJKkrcDNQG7jLzG5IOl91v/xSLaADIYjqBUwAHgaewKzs6xLl\n",
       "m9SQ0HKUair/esAPrBoUlfz3lHTBUdqi4jqLZXUu5bbPZzGn3c9AXzNm5etWnHOrqtLPTueqgYIE\n",
       "TpJqA18DBxKmlH8IHGNmExPSVI8vf5hS35UQRHUFXie0RI3C7M8CltuI9MHRuoTgKNVstallDY7S\n",
       "ViGuM1i8xlXc/j8xt821wM1mpBwDphbqRhMGUo8GLGYR8xhqM210PurhXE1SbZ6dzlVRhRoc3h6Y\n",
       "bGY/AEh6FDgUmJjpoiopzAobCYxEagIcBvwN+A/SKEIQ9Uq5gpUQHLUmdXDUHPielUHRp8ATrAyO\n",
       "luV2Y6VU7dK1zmZ5s+u5+51fmNvmKDM+Spu2hbqxEUPoSZsVB0fSWi2EB0/OOeeqkkK1OB0JdDGz\n",
       "U6P3xwO7m9mAhDTV+68mqSVwNHAsYZHHxwhB1AckfujSGqwaHCUGSOuwanCU2HpU8OAoHZ2yx2DW\n",
       "nnIZjzz7PNN3O8GM+RnTt9UYjqfLaiceZIx9YwcXrKLOVUPV/tnpXCVXqBanrKIxSYMT3o4zs3EF\n",
       "qU0xmP0MDAGGILUlBFAPALWQ3iHstdYGaAZ8x8qg6GPCXm6TgWnFCo5SkRDdT3uALaYew/O3XmI/\n",
       "7fbPrC6sR4M0xxvms37OVUeSOgIdi1wN51ykUIHTT0CrhPetCCtEr8LMBheo/MrF7BsgjnQVsAuw\n",
       "IyuDpWlVYW0oiWbsff1Y2j2zHZ+d0M0mHv5i1hcvZlGa4wvzVT/nqqvoD8pxJe8lxYpWGedcwbrq\n",
       "6hAGhx9A2Iz1A6rr4PAaQGIfdr7jGQ66uB4Lm+5lN39fpjWs0oxx+pZpDPQxTs6VjT87nSuuQi5H\n",
       "cDArlyO428yuSzrvX/5KTqI2cDk73XMu3c5aTt1FHSxm5RrgH82qG0A9GrKYhcxjmAdNzpWdPzud\n",
       "Ky5fANOlJNEKeJD2t6xD13OaU2t5J4vZhGLXy7mazp+dzhVXrWJXIB/UQt3UVmO0jcaprcaohboV\n",
       "u05VmcShwEd0umw6Bw9Yh1rLD/SgyTnnnKsGm/z6GkH5I9EA+CfQnT6HDqXdyAFAZ4vZF0WumnPO\n",
       "OVcpVP0WpyYMXCVoAuhJG5owIM0VLgWJrYD3gRact/GVUdDUxWLVcDNj55xzrpyqfIuTrxGUGwkR\n",
       "Vjq/Hvg/rqw9m1rL/0MImsYXt3bOOedc5VL1AydfI6jcJNYC/gtsA+zHYLUF7gAOtpj9r6iVc845\n",
       "5yqhqt9VN4+hjGTyKsdG8i3zGFakGlUJErsT9rebA7RnsNoQgqZuFrNPilo555xzrpKqFssR+BpB\n",
       "2ZOoBVwMnAecbsYIxdUduAc4xGL2YVEr6JzLyJcjcK64qkXg5LIj0ZKwX14D4DgzpiiubsBwoIfF\n",
       "7P1i1s85Vzp/djpXXFW/q85lRaIroWvuHWD/KGjqSgiaenrQ5JxzzpWu6g8OdxlJ1AOuBY4G+pjx\n",
       "OoDiOgi4HzjMYvZeEavonHPOVRkeOFVjEm2AR4GfgB3NmAOguA4EHgIOt5i9U8QqOuecc1WKd9VV\n",
       "UxLHA+8SuuIOSwiaDgAeAY6wmL1VvBo655xzVY+3OFUzEmsCtwK7AweasWIRS8W1P6EF6kiL2ZtF\n",
       "qqJzzjlXZXmLUzUisTPwCbAU2CUpaNoPeBzobTF7vUhVdM4556o0D5yqAQlJnAuMAa4042Qz/lhx\n",
       "Pq59gCeBPhaz14pVT+ecc66q8666Kk5iXeBeYF1gDzO+W+V8XB2Ap4FjLGavFKGKzjnnXLXhLU5V\n",
       "mEQnwtpMXwAdUgRNewEjgOMsZi8XoYrOOedcteItTlWQRF1gMNAX6GvG2NXSxLUH8AxwgsXspQqt\n",
       "oHPOOVdNeeBUxUhsCjwMzAN2NmPmamni2h0YCfS1mI2p2Bo655xz1Zd31VUhEkcCHwBPAd3SBE27\n",
       "AaOAv1nMNzp2zjnn8slbnKoAiUbAv4EDgEPM+DBlurh2BZ4HTrGYPVeBVXTOOedqBG9xquQktgU+\n",
       "BNYgdM2lC5p2JgRN/S1mIyuwis4551yN4YFTBhLrSMQkNi9C2ZI4HXgN+AdwghnzUqaNayfgBeAM\n",
       "i9kzFVhN55xzrkbxrrrM6gHrAO9LTATuA5404/dCFirRDLgT2AzY24xJadPGtQMhaDrLYvZ0Xspv\n",
       "oW40YSD1aMBiFjGPoTbTx0s555xzMrPiFCyZmakohZeRRD2gG3Ai0AkYDdwPjDVjWZ7L6gA8RFh/\n",
       "aZAZf6VNG9f2wEvAAIvZE3kpv4W6sRFD6EmbFQdHMplpnOPBk3PFV5Wenc5VRx44lZFEc+Bo4CRg\n",
       "I0KQc58ZX+SYb23g/4CzgFPMyDi4W3FtC4wFzrWYPZZL2avk21ZjOJ4uq514kDH2jR2cr3Kcc+VT\n",
       "VZ+dzlUXPsapjMyYbcatZrQnzHJbCoyR+FjinGgLlDKR2BB4GdifMAC8tKBpG0JL0wX5DJoAqEeD\n",
       "NMcb5rUc55xzrgrywCkHZkw041JgE2AQsCvwjcSzEr0k6peWh0RP4GNC4NTZjOkZ08e1NaGl6WKL\n",
       "2cM530SyxSxKc3xh3styzjnnqpiCBE6SBkuaJunT6NW1EOVUFmYsM+NlM04AWhHGJ50N/CRxm8Tu\n",
       "Eqs0rUs0kBgKDAWOMOOa0sZLKa52hKDpEovZgwW5mXkMZSSTVzk2km+Zx7CClOecc85VIQUZ4yQp\n",
       "Bsw3s5sypKn2/fQSmwAnEAaVLycMKH8AaAQ8CnwLnGrG3FLzimtL4BXgcovZ8ELVGVbMqhtAPRqy\n",
       "mIXMY5gPDHeucqgJz07nKrNCBk4LzOzGDGlqzJc/am3agzCg/ChCS98g4E4zSv0foLjaEtZzutJi\n",
       "dk8h6+qcq9xq0rPTucqokIFTP+B34CPgAjP7LSlNjfzySzQAmpgxK6v0cbUhBE1xi9ldBa2cc67S\n",
       "q6nPTucqi3IHTpLGAi1TnLoMeA/4JXp/NbC+mZ2cdH2l//Lr/9u7/1iv6jqO48+Xl25QRrRqCkJh\n",
       "cFnpltiKueSPr2sYshKxmdWcZZu2UcnUtRK3vn7LQjNT1NwS7dcYmDMlSEVJvFPWRllQ1IUBLpbk\n",
       "1X5Y0Q8ZkO/+OGfuAvd777n3++Nzvve+Htsd33PO957z2pd9z973cz7nfSTBzK/DvuWRoG+DappF\n",
       "VjTdENW4u93HN7Py6YRzp9lYNurO4RGxoMj7JN0DbKiz7foBi70R0TvaPK0x60JYsBQ2/RJoSlfu\n",
       "olTTO4DNwAoXTWbjl6QKUEkcw8xyrbpUNzUi+vPXVwHvi4hPHPOe0v7VJE27AiYvg/kTYNUcuHw3\n",
       "bDkCB1ZGPN/yIkY1zQR6gZujGt9u9fHMrHOU+dxpNh606ll1N0maCwTwB+AzLTpOi/SvgkkvQdct\n",
       "IKBrIhy5GvpbPuqkmt5OdnnuFhdNZmZm5dKSwikiLm3FftslIkKaHaApcNFOePP0fHVL5zmppreR\n",
       "FU0roxrum2RmZlYy7hxeV3cPp274MtUHpnLe2u3MOvQR1TS5VUdTTdPJ5jTdGdW4rVXHMTMzs9Hz\n",
       "Q36HoZqmAh8GLgDmA1uAdcD6qMYLTTrGKWRzmr4T1fhmM/ZpZmNTp5w7zcYqF04jkI84nUdWRC0E\n",
       "+siKqHVRjT2j3Oc0sqLp3qjGTU2KamZjVCeeO83GEhdOo6SaXgucQ1ZELQb+Rl5EAb+K6vAfbD6a\n",
       "1Qt8P6qxonVpzWys6PRzp1mnc+HUBKrpBGAesCT/mQT8hOxhv09FNQ4P8jsnkxVNq6MaN7QvrZl1\n",
       "srF07jTrRC6cmkw1CXgn2UjUBcBs4BGykaiNUY3/qKaTyO6euy+q8ZWGjpc9kPdKupnIIQ5ygNv9\n",
       "QF6zsWusnjvNOoULpxbL75Y7n6yIOotslGk2cH9U4/qG9n2SFjGdlZzP7FdXrmcv+1nm4slsbBov\n",
       "506zsnLh1EaqaQqwCOgGflBkHtSQ++vRRi7hg8dtWM3G2BPnNbJvMyun8XjuNCuTVnUOt0FENf4B\n",
       "rPNDj9IAAAXTSURBVGnaDruZWGf9pKYdw8zMzF7lBphtJknSqSskNf4X4yEO1ln/csP7NjMzs+O4\n",
       "cGq7WRfCwqUwa0nDuzrA7axn71Hr1vMsB/DjWszMzFrAc5zaRJp2BUxeBvMnwKo5cPlu2HIEDqyM\n",
       "eP7uUe83u6vu83QziUO8zAHu8MRws7FrvJ07zcrGc5zapn8VTHoJum4BAV0T4cjV0P9gI3vNiyQX\n",
       "SmZmZm3gS3VtEhEBCjhhCny0D/SmfHWaIT8zMzMbMY84tVV3D2y6DJ59KJvj1N2TOpGZmZkV5zlO\n",
       "ZmYdxOdOs7R8qc7MzMysIBdOZmZmZgW5cDIzMzMryIWTmZmZWUEunMzMzMwKcuFkZmZmVpALJzMz\n",
       "M7OCXDiZmZmZFeTCyczMzKwgF05mZmZmBblwMjMzMyvIhZOZmZlZQaMunCRdJOn3kv4n6T3HbLtW\n",
       "0h5JuySd23hMMzMzs/QaGXHaASwBnhq4UtJpwMXAacBC4C5JHTuyJamSOkMRztlcztlcnZCzEzKa\n",
       "WXqjLmgiYldE7B5k02JgbUQcjoh9wF5g3miPUwKV1AEKqqQOUFAldYCCKqkDFFRJHaCgSuoABVRS\n",
       "BzCz8mvFSNA0YP+A5f3AKS04jpmZmVlbTRhqo6RNwMmDbFoeERtGcJwYUSozMzOzElJEYzWNpCeB\n",
       "ayLi1/nylwAi4sZ8eSNQjYitx/yeiykzs1GICKXOYDZeDTniNAIDv8TrgTWSvkV2ia4H+MWxv+Av\n",
       "vpmZmXWaRtoRLJH0HHAW8LCkRwEiog+4H+gDHgWWRqPDWmZmZmYl0PClOjMzM7PxIml/JUlflfQb\n",
       "SdslPSFpRso89Ui6WdLOPOuDkt6YOtNghmpKWgaSFuZNUfdI+mLqPIOR9F1JL0rakTpLPZJmSHoy\n",
       "/7/+naQrU2cajKSJkrbm3+8+SStSZxqKpC5J2ySN5MaXtpK0T9Jv85zHTYEws9ZL3ZjyGxFxRkTM\n",
       "BdYB1cR56nkcOD0izgB2A9cmzlPPoE1Jy0BSF3AnWVPU04CPS3pX2lSD+h5ZxjI7DFwVEaeTXSr/\n",
       "bBk/y4g4CJyTf7/fDZwjaX7iWENZRjbFoMzD8AFUIuLMiOjk/nhmHStp4RQR/xqweCLw11RZhhIR\n",
       "myLilXxxKzA9ZZ56hmhKWgbzgL0RsS8iDgP3kTVLLZWIeBr4e+ocQ4mIFyJie/7638BOsv5ppRMR\n",
       "/81fdgNdwEsJ49QlaTqwCLiHo292KaOy5zMb01KPOCHpa5L+CHwSuDF1ngI+DTySOkQHOgV4bsCy\n",
       "G6M2gaSZwJlkBX3pSDpB0nbgReDJ/OaRMroV+ALwynBvTCyAn0l6RtLlqcOYjUfNakdQ13BNNCPi\n",
       "OuC6vP/TrcBlrc40mCLNPiVdBxyKiDVtDTdAE5uStluZL390JEknAg8Ay/KRp9LJR2rn5vMCH5NU\n",
       "iYjexLGOIulDwJ8jYlsHPK/u7Ijol/RWYJOkXfkoqZm1ScsLp4hYUPCta0g4kjNcTkmfIhvK/0Bb\n",
       "AtUxgs+zbP4EDJz8P4OjH81jIyDpNcCPgdURsS51nuFExD8lPQy8F+hNHOdY7wfOl7QImAhMlvTD\n",
       "iLg0ca7jRER//u9fJD1EdgnchZNZG6W+q65nwOJiYFuqLEORtJBsGH9xPuG1E5RtHsQzQI+kmZK6\n",
       "gYvJmqXaCEkScC/QFxG3pc5Tj6S3SJqSv54ELKCE3/GIWB4RMyLiVOBjwOYyFk2SXifpDfnr1wPn\n",
       "kt0QYmZtlHqO0wpJO/I5EBXgmsR56rmDbPL6pvw24LtSBxpMvaakZRARR4DPAY+R3bn0o4jYmTbV\n",
       "8SStBX4OzJH0nKQkl46HcTZwCdldatvynzLeCTgV2Jx/v7cCGyLiicSZiijrZeWTgKcHfJ4/jYjH\n",
       "E2cyG3fcANPMzMysoNQjTmZmZmYdw4WTmZmZWUEunMzMzMwKcuFkZmZmVpALJzMzM7OCXDiZmZmZ\n",
       "FeTCyczMzKwgF05mZmZmBf0f5H1MkKRxL/8AAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105f99f50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# optimal d = 6\n",
    "def plot():\n",
    "    X = construct_poly(x,6)  \n",
    "    Xt = construct_poly(xt,6) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 1.0'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    #plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "   \n",
    "    \n",
    "    w = wRidge(X,y,0.1)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 0.1'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    \n",
    "    \n",
    "    w = wRidge(X,y,10.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 10.0'+\"  MSE: %f\" % MSE(X.dot(w),y))\n",
    " \n",
    "  \n",
    "    \n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
    "plot()\n",
    "print('Minimum MSE: für lambda = 0.1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hand in printed copy of completed notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "annotations": {
   "author": "",
   "categories": [
    "intelligent-systems-1-2015"
   ],
   "date": "2015-04-30",
   "location": "Beginning of next lecture",
   "parent": "IS_SS2015",
   "submission_date": "2015-05-07",
   "subtitle": "Exercise Sheet 3, Linear Regression",
   "tags": [
    "IntelligenSystems",
    "Course"
   ],
   "title": "Intelligent Systems 1 - Summer Semester 2015"
  },
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}