{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "preamble": true
   },
   "source": [
    "(Defining latex commands: not to be shown...)\n",
    "$$\n",
    "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
    "\\DeclareMathOperator{\\minimize}{minimize}\n",
    "\\newcommand{\\real}{\\mathbb{R}}\n",
    "\\newcommand{\\normal}{\\mathcal{N}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#Exercise Sheet 3\n",
    "Maximus Mutschler & Jan-Peter Hohloch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Gaussian Algebra (25 Points)\n",
    "\n",
    "Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
    "\n",
    "\\begin{equation}\n",
    "   \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Proof\n",
    "\\begin{align*}\n",
    "   f(x) &= \\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\cdot e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}}\\cdot \\frac{1}{\\sqrt{2\\pi s^2}}\\cdot e^{\\frac{-(x-m)^2}{2s^2}}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\alpha}\\\\\n",
    "   \\text{where }\\alpha &= \\frac{s^2(x-\\mu)^2+\\sigma^2(x-m)^2}{2\\sigma^2 s^2}\\\\\n",
    "   &=\\frac{(s^2+\\sigma^2)x^2-2(s^2\\mu+\\sigma^2 m)x+s^2\\mu^2+\\sigma^2m^2}{2\\sigma^2s^2}\\\\\n",
    "   &=\\frac{x^2-2\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}x+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}+\\underbrace{\\frac{\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}_\\beta\\\\\n",
    "   \\beta&= \\frac{\\frac{s^4\\mu^2+s^2\\sigma^2\\mu^2+s^2\\sigma^2m^2+\\sigma^4m^2-\\left(s^4\\mu^2+2s^2\\sigma^2\\mu m+\\sigma^4m^2\\right)}{\\left(s^2+\\sigma^2\\right)^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{s^2\\sigma^2\\left(\\mu^2-2\\mu m+m^2\\right)}{2\\sigma^2s^2\\left(s^2+\\sigma^2\\right)}\\\\\n",
    "   &=\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}\\\\\n",
    "   \\Rightarrow f(x)&=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-(\\alpha-\\beta)-\\beta}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}\\cdot e^{-\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}}\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi(s^2+\\sigma^2)}\\sqrt{2\\pi\\frac{\\sigma^2s^2}{s^2+\\sigma^2}}}{2\\pi s\\sigma}\\normal\\left[x;\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2},\\frac{\\sigma^2s^2}{s^2+\\sigma^2}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi\\cdot 2\\pi\\sigma^2s^2}}{2\\pi s\\sigma}\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\hspace{2cm}\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[m,\\mu,\\sigma^2+s^2\\right]\\hspace{2cm} q.e.d.\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
    "\n",
    "Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
    "\n",
    "\n",
    "Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\left(2\\pi\\right)^{-\\frac{k}{2}}\\left|\\Sigma\\right|^{-\\frac{1}{2}}\\cdot e^{-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)}\\text{, where: }k=n\\text{ number of measurments},\\ \\Sigma=\\sigma I,\\ \\mu=X\\vec w$\n",
    "\\begin{align*}\n",
    "    &\\frac{d}{d\\vec w} \\left(2\\pi\\right)^{-\\frac{k}{2}}\\left|\\Sigma\\right|^{-\\frac{1}{2}}\\cdot e^{-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)}\\\\ &=\\underbrace{\\left(2\\pi\\right)^{-\\frac{k}{2}}\\left|\\Sigma\\right|^{-\\frac{1}{2}}\\cdot e^{-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)}}_{>0}\\frac{d}{d\\vec w}\\left(-\\frac{1}{2}\\left(\\vec y-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec y-\\mu\\right)\\right)&\\overset{!}{=}&0\\\\\n",
    "   \\Leftrightarrow &\\frac{d}{d\\vec w}-\\frac{1}{2}\\left(\\vec{y}-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec{y}-\\mu\\right) &=&0\\\\\n",
    "   \\Leftrightarrow & \\frac{d}{d\\vec w}\\left(\\vec{y}-\\mu\\right)^T\\Sigma^{-1}\\left(\\vec{y}-\\mu\\right) &=&0\\ (\\Sigma\\not=\\mathcal{O})\\\\\n",
    "   \\Leftrightarrow & \\frac{d}{d\\vec w}\\left(\\vec{y}-\\mu\\right)^T\\left(\\vec{y}-\\mu\\right) &=&0\\\\\n",
    "   \\Leftrightarrow & \\frac{d}{d\\vec w}\\left(\\left(\\vec{y}-X\\vec{w}\\right)^T\\right)\\left(\\vec{y}-X\\vec{w}\\right) + \\left(\\vec{y}-X\\vec{w}\\right)^T\\frac{d}{d\\vec w}\\left(\\vec{y}-X\\vec{w}\\right)&=&0\\\\\n",
    "   \\Leftrightarrow & -X\\left(\\vec{y}-X\\vec{w}\\right)-\\left(\\vec{y}-X\\vec{w}\\right)^TX^T &=&0\\\\\n",
    "   \\Leftrightarrow & -X\\left(\\vec{y}-X\\vec{w}\\right)-X\\left(\\vec{y}-X\\vec{w}\\right)&=&0\\\\\n",
    "   \\Leftrightarrow & X\\left(\\vec{y}-X\\vec{w}\\right) &=&0\\\\\n",
    "   \\Leftrightarrow & \\vec{y}-X\\vec{w} &=&0\\\\\n",
    "   \\Leftrightarrow & \\vec{y}&=& X\\vec{w}\\\\\n",
    "   \\Leftrightarrow & X^T\\vec{y} &=& X^TX\\vec{w}\\\\\n",
    "   \\Leftrightarrow & \\vec{w} &=& \\left(X^TX\\right)^{-1}X^T\\vec{y}\\hspace{1cm} q.e.d.\n",
    "\\end{align*}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Linear regression (50 Points)\n",
    "\n",
    "In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
    "\n",
    "1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
    "2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Loading the required packages\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.html.widgets import interact\n",
    "import csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def wRidge(X,y,lamb):\n",
    "    # Change the following line and implement the ridge regression estimator wRidge\n",
    "    return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Read ex1_train.csv and assign the first column and \n",
    "# second column to variables x and y respectively.\n",
    "\n",
    "a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
    "x= a[:,0]\n",
    "y= a[:,1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4) Plot the training data with appropriate labels on each axes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1057c4510>"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEZCAYAAACTsIJzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAFmxJREFUeJzt3XuQZGd93vHvo1tWWIsEBmNZ7CIsQYJYQICREZcwQBzE\n",
       "xoHIGDCOMRi05Spgl4oJZQQJkkM53BLC7lJ2EgHiZm4yBiOQZGSsNUpUXAQISxFClsKyKwwSV7Eb\n",
       "dhNdfvnj9LJ9RnPpnpnu0z3z/VRNTZ/3dJ/+zZZ0nn7f97ynU1VIknTIEV0XIEmaLAaDJKnFYJAk\n",
       "tRgMkqQWg0GS1GIwSJJaDAapA0nek+QNXdchzcVg0KqVZHeSp4/hfc5P8v4hX1a9n0GOvyvJS4ev\n",
       "TFoag0Gr2cAn345kwOdN8t+gVchg0JqQ5MVJ/keStyb5YZL/neSsvv27krwxyReS3J7kE0nu09s3\n",
       "k2TvrOPtTvL03jHOBZ6fZF+Sr87z/o9O8pUkP0nyYWBd3777JPlUktt6tV2c5KTevj8Gngy8o3f8\n",
       "Hb327Un29Gq9OsmTVvifTGuYwaC15AzgBuDngbcA75q1/4XA7wEnAncCOxY4VgFVVZcB/xH4cFWt\n",
       "r6pHz35ikmOATwDvBe4DXAQ8h8M9gfRq2dj7OQC8g+YNXgdcCby8d/xtvdd8EXhU73gfBC7qvY+0\n",
       "bAaD1pJvVdW7qrlB2PuAE5P8Qm9fAe+rquur6qfAvweel2SQ4Z6w8LDQ44Gjqmp7Vd1VVR8DvnRo\n",
       "Z1X9sKo+XlUHq2o/TdA8ZY73oO81f1ZVP6qqu6vqbcA/Av7xALVKizIYtJZ899CD3skf4Li+/f3D\n",
       "RXuAo4H7rcD7/hLw7Vlt36J3sk9yryT/rTc8dTvwt8Dxs0KpNc+Q5N8muT7Jj5P8CDh+hWqVDAap\n",
       "z8ZZj+8Avg/8H+Beh3YkORK4f99zF5sc/g5w0qy2B/W97lXAQ4Ezqup4mt5Cfy9kdig8GXg18Nyq\n",
       "OqGq7gPczuCT2dKCDAapEeB3kjwsyb2A/wBc1Bt2uhFYl2RzkqOBf0czdHPId4GTFxh2ugq4M8m2\n",
       "JEcn+Q3gcX37j6OZV7g9yX2B82a9/lbglL7t9TRzIN9PckyS1wP3XsofLc3FYNBaMdelqzXr8fuB\n",
       "99B8wj8G2AZQVbcDLwPeCdwC7Kc97HRR7/cPklx9jzeuugP4DeDFwA+A5wEf63vK24FjaXonVwGX\n",
       "zqptO/CbvSuW3g5c1vu5EdhNEyp7FvzrpSGkyy/qSfJu4F8At1XVI3pt5wPnAN/rPe3c3pUf0sgk\n",
       "uQJ4f1W9u+tapK513WO4EDhrVlsBb6uqR/d+DAWNi2P0Eh0HQ1VdCfxojl3+D6ouuMJYovsew3y2\n",
       "JvlaknclOaHrYrT6VdVTHUaSGpMYDH8KPBg4nWYS8D93W44krS1HdV3AbFV126HHSd4JXDz7OUns\n",
       "8kvSElTVokP1ExcMSU6squ/0Ns8Grp3reYP8cZMqyflVdX7XdSyV9XfL+rszzbXD4B+qOw2GJB+i\n",
       "WeV5v97dK88DZpKcTjMR+E3g9zssUZLWnE6DoapeMEezE4CS1KFJnHxeC3Z1XcAy7eq6gGXa1XUB\n",
       "y7Sr6wKWaVfXBSzDrq4LGIdOVz4vVZKa5jkGSerCoOfOiZt8liStrGTTZtiwbfFnNgwGSVrFmlA4\n",
       "cztccOqgN5VwjkGSVrUN25pQGJzBIEmr2vp1w77CoSRJq8LhcfT162DfQdi7o+q6S7quq3v7Dg77\n",
       "CoNB0tRrj6MfsuWUZBOGw94dsOWUYYaTvFxV0liM8hN98szL4NJn3HPP5suqLnnmSrzHNGv+7Tdu\n",
       "hUvP8nJVSRNh9J/o5xtHP+7Y5R97+oeperVeMhX3SpK0Vsx1ZcwFp8LmrcAKnGDnG0fff2C5R16L\n",
       "w1RelSRpDEb7ib43jn5Tu+2cm2HPzuUfe75Q27h1+ceeTPYYJI3B6D7RQzNUkmyi6YEcd2xz3D07\n",
       "p2GYahIZDJLGYK4rY1bqE33j0Dj6Sh3vsNGG2iQyGCSN3Gg/0Y/a6ENt0ni5qiQt4vDlntMWam2D\n",
       "njsNBklaIwY9d3pVkiSpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1\n",
       "GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqSWToMhybuT3Jrk2r62+ya5\n",
       "PMmNST6T5IQua5SktabrHsOFwFmz2l4DXF5VDwU+29uWJI1Jp8FQVVcCP5rV/Czgvb3H7wX+1ViL\n",
       "kqQ1rusew1weUFW39h7fCjygy2Ikaa05qusCFlJVlaTm2pfk/L7NXVW1ayxFSdKUSDIDzAz9uqo5\n",
       "z7tjk+Rk4OKqekRv+wZgpqq+m+RE4Iqq+iezXlNVlbEXK0lTbNBz5yQOJX0SeFHv8YuAT3RYiySt\n",
       "OZ32GJJ8CHgKcD+a+YTXA38JfBTYCOwGnldVP571OnsMkjSkQc+dnQ8lLYXBIEnDm+ahJElShwwG\n",
       "SVKLwSBJajEYJEktE73ATdJhyabNsGEbrF8H+w7C3h1V113SdV1afQwGaQo0oXDmdrjg1MOtW05J\n",
       "NmE4aKU5lCRNhQ3b2qEAzfbGrd3Uo9XMYJCmwvp1c7cfd+x469BaYDBIU2Hfwbnb9x8Ybx1aCwwG\n",
       "aSrs3QFbbmq3nXMz7NnZTT1azbwlhjQlmgnojVub4aP9B2DPTieeNQzvlSRJavFeSZKkJTEYJEkt\n",
       "BoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqQWg0GS1GIw\n",
       "SJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWo7quoD5JNkN/AS4C7ijqs7o\n",
       "tiJJWhsmNhiAAmaq6oddFyJJa8mkDyWl6wIkaa2Z5GAo4K+TXJ1ky+ydyTMvSzZt7qAuSVrVJnko\n",
       "6YlV9Z0k9wcuT3JDVV15ePelz4AtpySbqLruks6qlKRVZmKDoaq+0/v9vSQfB84A+oLhfOCkU2Hf\n",
       "9iQ/rapdHZQpSRMryQwwM/TrqmqxA/98Vf1gaWUtTZJ7AUdW1b4kPwd8BvijqvpMb381I00Az/vb\n",
       "qo/OjLM+SZpGSaqqFp27HaTH8Pkk1wAXApfWYkmyMh4AfDwJNDX+2aFQuKf9B8ZQjyStGYP0GI4A\n",
       "/hnwEuBxwEeBC6vqxtGXN29NvR7DOTfD57c5xyBJixu0x7BoMMw66NOADwA/B1wDnFtVVy25yiVq\n",
       "guGZl8GenYaCJA1mxYIhyf2Afw38LnAr8E7gYuBRwJ9X1cnLrnZIg/5xkqTDVnKO4SqaXsKzq+qW\n",
       "vvark/zXpRYoSZpMA80xVNXdY6pnIPYYJGl4g547F135PGmhIEkarUm+JYYkqQOLBkOSJ83R9sTR\n",
       "lCNJ6togPYadc7S9Y6ULkSRNhnmvSkpyJvAE4P5J/oDDt8Bej0NQkrRqLXS56jE0IXBk7/chPwF+\n",
       "c5RFSZK6M8jlqg+qqm+NqZ6BeLmqJA1vJVc+XzFHc1XV05Za3HIZDJI0vJVc+fzqvsfrgOcAdy61\n",
       "MEnSZBvqJno/e1Hypap63AjqGfT97TFI0pBWrMeQ5L59m0cAvwLcexm1SZIm2CBDSV/h8Nel3Qns\n",
       "Bl46qoIkSd1a0lBS1xxKkqThreRQ0rHAy4An0fQcrgT+tKoOLrtKSdLEGeRy1YtoFrV9gGb1828D\n",
       "x1fVc0df3rw12WOQpCGt5DqG66vqtMXaxslgkKThrdj3MQBf6d036dCBHw98eTnFSZIm1yA9hhuA\n",
       "hwJ7aeYYNgLfoLlCqarqkaMuco6a7DFI0pBWcuXzMzh8Z9VDao42SdIqMEiP4f1V9cLF2sbJHoMk\n",
       "DW8l5xg2zTrwUcBjl1qYJGmyzRsMSV6bZB/wiCT7Dv0AtwGfHFuFkqSxGmQo6U1V9Zox1TMQh5Ik\n",
       "aXgruY7hKRy+V9LPVNXnll7e8hgMkjS8lQyGT3E4GNYBZwBf9ot6JGm6rNjlqlX167MOvAHYvoza\n",
       "JEkTbJCrkma7BXjYShciSZoMg9xddWff5hHA6XhLDElatQZZ+fxlDs8x3AV8sKr+5+hKkiR1aZDJ\n",
       "52OBU2nC4aZJ+B4GJ58laXjLXvmc5Ogkb6G5ed57gfcBtyR5a5KjV65USdIkWWjy+a3AfYEHV9Vj\n",
       "quoxwC8DJwD/aRzFSZLGb96hpCQ3AQ+tqrtntR8JfKOqTh1ZUclZwNuBI4F3VtWbZ+13KEmShrQS\n",
       "N9G7e3YoAFTVXcA92ldKL3jeAZwFnAa8IImXx0rSmCwUDF9P8qLZjUleCNwwupI4g2aSe3dV3QF8\n",
       "GHj2CN9Pa0SyaXPyzMuS5+1qfm/a3HVN0iRa6HLVlwN/keQlHF638FjgXsDZI6zpJJoJ70NuAX51\n",
       "hO+nNaAJgTO3wwV9Q6BbTkk2UXXdJd1VJk2eeYOhqm5J8qvA04CH01yu+umq+uyIa1r4+llpSTZs\n",
       "a4cCNNubtwIGg9RnwQVu1cxMf7b3My7fBjb0bW+g6TW0JDm/b3NXVe0abVmabuvXzd1+3LHjrUMa\n",
       "nyQzwMywrxtk5fO4XQ08JMnJwD8AzwdeMPtJVXX+WKvSlNs3z8LM/QfGW4c0Pr0PzLsObSc5b5DX\n",
       "LeUmeiNVVXcCrwD+Crge+EhVfb3bqjT99u6ALTe12865GfbsnPv50tq16C0xJpHrGLQUzQT0xq3N\n",
       "8NH+A7BnpxPPWktW7It6JpHBIEnDW4kFbpKkNWgSJ5870ww1bNjWXMGy7yDs3eFQg6S1xmDocQGU\n",
       "JDUcSvqZ+RZAbdzaTT2S1A2D4WdcACVJYDD0cQGUJIHB0McFUJIErmOYdVwXQElavVzgJklqcYGb\n",
       "JGlJDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMk\n",
       "qcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWo5qusCpEOSTZthwzZYvw72HYS9O6quu6TruqS1\n",
       "xmDQRGhC4cztcMGph1u3nJJswnCQxsuhJE2IDdvaoQDN9sat3dQjrV0GgybE+nVztx937HjrkGQw\n",
       "aELsOzh3+/4D461DksGgCbF3B2y5qd12zs2wZ2c39UhrV6qq6xqGlqSqKl3XoZXVTEBv3NoMH+0/\n",
       "AHt2OvEsrZxBz50TFwxJzgfOAb7Xazq3qi6b9RyDQZKGNOi5cxIvVy3gbVX1tq4LkaS1aFLnGOwN\n",
       "SFJHJjUYtib5WpJ3JTmh62IkaS3pZI4hyeXAL86x63XA5zk8v/AG4MSqeums1xfwR31Nu6pq1whK\n",
       "laSplWQGmOlrOm8qJ5/7JTkZuLiqHjGr3clnSRrSoOfOiRtKSnJi3+bZwLVd1SJJa9EkXpX05iSn\n",
       "01yd9E3g9zuuR5LWlIkeSpqPQ0mSNLypHUqSJHXLYJAktRgMkqQWg0GS1GIwSJJaDAZJUovBIElq\n",
       "MRgkSS2TuPJZE6r5hrUN22D9uuY7mvfu8BvWpNXHYNBAmlA4cztccOrh1i2nJJswHKTVxaEkDWjD\n",
       "tnYoQLO9cWs39UgaFYNBA1q/bu72444dbx2SRs1g0ID2HZy7ff+B8dYhadQMBg1o7w7YclO77Zyb\n",
       "Yc/ObuqRNCredlsDayagN25tho/2H4A9O514lqbHoOdOg0GS1gi/j0GStCQGgySpxWCQJLUYDJKk\n",
       "FoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWox\n",
       "GCRJLQaDJKnFYJAktXQSDEmem+R/JbkryWNm7Ts3yd8nuSHJP++iPklay7rqMVwLnA18rr8xyWnA\n",
       "84HTgLOAP0my6no1SWa6rmE5rL9b1t+daa59GJ2cdKvqhqq6cY5dzwY+VFV3VNVu4CbgjLEWNx4z\n",
       "XRewTDNdF7BMM10XsEwzXRewTDNdF7AMM10XMA6T9mn8l4Bb+rZvAU7qqBZJWpOOGtWBk1wO/OIc\n",
       "u15bVRcPcahaoZIkSQNIVXfn3SRXAK+qqq/0tl8DUFVv6m1fBpxXVV+Y9TrDQpKWoKqy2HNG1mMY\n",
       "Qn+RnwQ+mORtNENIDwG+OPsFg/xhkqSl6epy1bOT7AUeD3w6yaUAVXU98FHgeuBS4GXVZZdGktag\n",
       "ToeSJEmTZ9KuShpYkjck+VqSa5J8NsmGrmsaRpK3Jvl672/4iyTHd13TMBZapDipkpzVWzj590n+\n",
       "sOt6hpXk3UluTXJt17UMK8mGJFf0/pu5Lsm2rmsaRpJ1Sb7QO99cn+SNXde0FEmOTPLVJAteADS1\n",
       "wQC8paoeVVWnA58Azuu6oCF9Bnh4VT0KuBE4t+N6hjXnIsVJleRI4B00CydPA16Q5GHdVjW0C2nq\n",
       "n0Z3AP+mqh5OM4T88mn696+qg8BTe+ebRwJPTfKkjstailfSDNUvOFQ0tcFQVfv6No8Dvt9VLUtR\n",
       "VZdX1d29zS8AD+yynmEtsEhxUp0B3FRVu6vqDuDDNAsqp0ZVXQn8qOs6lqKqvltV1/Qe7we+TrNu\n",
       "aWpU1U97D48BjgR+2GE5Q0vyQGAz8E7aF/3cw9QGA0CSP06yB3gR8Kau61mGlwCXdF3EKncSsLdv\n",
       "28WTHUlyMvBomg9EUyPJEUmuAW4FruhdLDNN/gvwauDuxZ440cGQ5PIk187x8y8Bqup1VbUReA/N\n",
       "Hz1RFqu/95zXAf+vqj7YYalzGqT+KeJVFhMgyXHAnwOv7PUcpkZV3d0bSnog8E+n6b5JSX4duK2q\n",
       "vsoivQWYjHUM86qqXxvwqR9kAj9xL1Z/khfTdO2ePpaChjTEv/80+DbQf4HCBtq3X9GIJTka+Bjw\n",
       "gar6RNf1LFVV3Z7k08CvALs6LmdQTwCelWQzsA64d5L3VdXvzvXkie4xLCTJQ/o2nw18tataliLJ\n",
       "WTTdumf3Jram2TQsOLwaeEiSk5McQ3MX3092XNOakSTAu4Drq+rtXdczrCT3S3JC7/GxwK8xReec\n",
       "qnptVW2oqgcDvwX8zXyhAFMcDMAbe8Ma19Dc8fBVHdczrJ00k+aX9y4f+5OuCxrGfIsUJ1VV3Qm8\n",
       "AvgrmqsyPlJVX++2quEk+RBwFfDQJHuT/F7XNQ3hicDv0FzN89XezzRdYXUi8De9880XgIur6rMd\n",
       "17QcCw6tusBNktQyzT0GSdIIGAySpBaDQZLUYjBIkloMBklSi8EgSWoxGKRZkqz4rRqSPCjJC1b6\n",
       "uNIoGAzSPY1icc+Dgd8ewXGlFWcwSPNIMpNkV5KLel+q9IG+fbuTvDnJ3/W+wOWUXvt7kjyn73mH\n",
       "bg//JuDJvRW/r5z1Pmcn+eve4xOTfCPJL4z+L5TmZjBICzud5stNTgN+OckTeu0F/LiqHknzBUBv\n",
       "72ufyx8CV1bVo6tqe/+Oqvo48J0krwD+O/D6qrpthf8OaWAGg7SwL1bVP1Rz75hrgJP79n2o9/vD\n",
       "wJmLHGexGw1upfkWv4NV9ZGlFCqtFINBWtj/7Xt8F/Pfqv5QT+FOev9fJTmC5tu+BrGhd/wH9O5E\n",
       "KnXGYJCW7vl9v6/qPd4NPLb3+FnA0b3H+4D1cx0kyVE0t6T+LeAG4A9GUKs0sIn+oh6pIzXP49nu\n",
       "k+RrwEHg0KWoFwB/2bs982XAoUtfvwbc1Wu/cNY8w7nA56rqqiR/B3wpyaeq6hsr8cdIw/K229IS\n",
       "JPkm8NiqmqovhJcG4VCStDR+otKqZY9BktRij0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySp5f8D\n",
       "oJt+CXUlyycAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1077d0750>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the input data here\n",
    "plt.plot(x,y,'o')\n",
    "plt.title('Input data')\n",
    "plt.ylabel('Output y')\n",
    "plt.xlabel('Input x')   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
    "Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def construct_poly(x,d):\n",
    "    ## Implement a method which given an array of size N, \n",
    "    ## returns an array of dimension (N,d)\n",
    "    poly=np.empty((len(x),1), float)\n",
    "    xr = np.reshape(x,(len(x),1))\n",
    "\n",
    "    for i in range(d-1):\n",
    "        poly = np.append(poly, (xr**i), axis=1)\n",
    "\n",
    "    return poly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def MSE(y_predict,y_true):\n",
    "    ## Implement mean squared error for a given input y and its predictions.\n",
    "    return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "minimal MSE for n=7\n"
     ]
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHFW9xvHvG7Y0yCLEQSRIRoIGAoKoILINIIsxsiiL\n",
       "yBJwuSqScNVBBK6XcAUVMnJNgigiqwKyI8i+RVAE5LKILEIgkUVIs68dZPndP6rGdIYZMr1Wd/X7\n",
       "eZ55mK6qrvpNHp63T59z6pQiAjMzy5cRWRdgZmb153A3M8shh7uZWQ453M3McsjhbmaWQw53M7Mc\n",
       "cribmeWQw92aStJcSa9JWmnA9jskvSXp/enr0ZLOl/SUpOcl3S1pUrpvTHrsSwN+dh3imrtJuknS\n",
       "K5KuX0R9hw4456uS3pS0Yrq/T9IDkl6UdJ+kvQe8fzFJR0p6PD3mdknLp/u+IOl+SS9IelrSBZLe\n",
       "V/be0ZIukfSMpCckzZS0WNn+nSTdk573Hkk7Vvavb53E4W7NFsDDwB79GyStCxTSff1+DfwDeD+w\n",
       "IrA3MG/AuZaPiGXLfs4d4prPAMcCP15kcRE/LD8ncDRwfUQ8mx7yMjAxIpYDJgHTJW1cdoojgE8A\n",
       "n0iP2QuYn+77E7B5RCwPrA68mtbVbwbwNLAKsD6wBbA/gKQu4Azg2+l5DwLOlDRqUX+TdSaHu2Xh\n",
       "N8A+Za8nAacDKtv2MeDUiChFxFsRcWdEXFHNxSLi2og4D3iikvdJUlrbaWXnmhoRD6S/3wrcCGyc\n",
       "Hv9u4EDgqxHxaHrMvRHxWvr7oxFR7D898OaAmsYDZ0fEvyJiHnBFug1gLPByRFyZnusy4BVgjUr+\n",
       "JuscDnfLws3AcpLGpd0Ou5ME/sBjjpe0e39XzSA0xPZ62Qx4D3D+oBeXCsDHgb+lm9YF3gB2TbtV\n",
       "/i5p/wHv2VTS88CLJN9KDi7bfSXwRUkFSasCnwYuT/f9FXhD0sS062cnkm8Ef63HH2r543C3rPya\n",
       "pPW+DXAv8PiA/buStIq/Dzyc9sl/bMAxT0t6ruznQ3WucRJwbkS8OsT+XwB3RsRV6evRwPLAmsAY\n",
       "YBdgqqRP9b8hIv4YESukx74OTCs731RgHZLgfxT4S0T8Ln3fy8DXgLNJQv0M4GsRUar9z7Q8crhb\n",
       "FoIk3Pdk8C4ZIuL5iDgkItYBVgbuBC4acJ6VIuLdZT9/r1eBkpYmCefThtg/DVgb2K1sc3/Q/k9E\n",
       "vBYRdwO/BSYMfH9E/JPkg2uf9HwiabmfCywNjAJWlHR0un8D4JfAZhGxBEl//EmS1qvxT7Wccrhb\n",
       "JiLiEZKB1U8DFyzi2GeAnwDvS/u1q75sBcfuDDwTEX8YuEPSEcB2wLZpi7rfUF0kQ113CZJBVUjC\n",
       "/KPAcRHxejqAeyoLPhi2Bm6OiNsBIuI24BbgU5gNwuFuWfoysNVgXQuSjpY0XtLikpYFvgE8GBHP\n",
       "lR82nItIGiFpJEmYjpC0lKQlFvG2/m8UA891CMlMn20G1EJEPETSlXSYpCUlrUUynvD79L17Slot\n",
       "/X114CgW9Oc/TTK4+o20T32FtIa70v13AZv1t9QlfYRkTKB/v9lCHO6WmYh4uL8l2r+p7PcCcCHw\n",
       "HPAQsBqww4BTPD9gTvp/DnGpfUhayMeTBGIJOKF/Z/reTcperwr0MEi4kwTyasDssut+r2z/HiTT\n",
       "HJ8hCfX/ioj+ufVrATdJehmYBfwZ+G76bxHA54DPkgT9g8BrwLfS/VcBxwAXSHoJOA84KiKuGeJv\n",
       "tg6nWh/WIelk4DNAMSLWTbdNBb4CPJUedki109jMzKxy9Wi5nwJsP2BbAMdGxEfSHwe7mVkT1Rzu\n",
       "EXEjyVfngRo9B9nMzIbQyD73yZLuknRSOjhkZmZN0qhw/znQTbI+xhMk09jMzKxJFm/EScvWz0DS\n",
       "r4BLBh4jqbaRXDOzDhURi+z2bki4S1olIvoXRNoZuHuw44ZTYKuSNDUipmZdR7Vcf7Zcf3bauXYY\n",
       "fsO45nCXdBbJrdCjJD0KHA70SFqfZNbMHJI1MczMrElqDveI2GOQzSfXel4zM6ue71Ct3qysC6jR\n",
       "rKwLqNGsrAuo0aysC6jRrKwLqIaEYJO7JZbMupZGq/kO1aovLEU797mbWXuQWALYHNiRZAmLZYF3\n",
       "kdxBP5fkiV8D//tIBC25nPJws7MhA6pmZlmSWI5kxdEd0v/OBn4HTATuARYD3key7v7q6X83JFnC\n",
       "eXVgNYnnGTz45wL/iKB8RdCG6+7WhK4upgz3eLfczSwXJEaThPmOJI8+/CNJoF8SwT8rPNcI4L0s\n",
       "CP/VWfiDYHWSBejmMsQHQATP1/L3lOvu1oTx45ne28vYLbfMcCqkmVmjJf3nfJgFgd4NXAacCOwS\n",
       "wUvVnjuCt4B/pj83DXHt97Bw2K9Jsr7+GGB1iWBB2A/2AfB0xPCeMdDVxZTeXsZW8jc43M2sbaT9\n",
       "55uxoP88SFrnBwF/jOD1ZtSRhnIx/fnLIHUKWIG3t/Y3LXu9lDRo6Pf/Pi/9kKFQYGSlNTrczaxl\n",
       "9PcrFwqMLJWYXywyY+7cuJFk5dkdSfrPHyYJ9B2Avw239dtMaU3PpT93DHZMOi4wsMvno2W/Ly/x\n",
       "CPCPddZZb41Kn8vicDezllDer9y/7cgj37fZvHkXq1Ta4QaSQD844m0PU29LEbxIcvf+oHfwSywN\n",
       "vB8Y89hjO3zmhz98bM9DD31m2I+Z9ICqmQ3bYC3rOXPisnqce6ONdMXRR7PdwO3f/e4S19x667+2\n",
       "qcc12ln6bz/51lvZ3gOqZlY3g7Ws+/pYo7tb1CPgR4wYPQYee9v2pZd+fVHPux2WRn4wNUNa62VN\n",
       "W1vGzDrDYDM2ensZe/DBTCaZpVIViWWAn40b173qYOFeKtV+M1GjP5hakZcfMLNhGWrGRqFAodpz\n",
       "SqwN3ArolVcemNTXx+zy/dOm8VCxyMxqz99vqA+mri4m13ruVuWWu5kNS6nE/CG2V9WyltgbOBY4\n",
       "GDjlkUeejO5uzT/4YCYXChRKJUrFIjPr0bJuxAdTq3O4m9mwFIvM6OtjjfIWcDUta4kCMJNkzvdW\n",
       "EQtmi/T3K9er5n71/mBqB54tY2bD1j9jo9qWtcSHgHNJpv99vZa7SCsxWJ/7tGk8dO+9TGm3Pvfh\n",
       "ZqfD3cyaQuKLwHTgMODEZt98VOsHU6twuJtZS0i7YX4KbAnsGlHhrZa2kOFmp2fLmFnDSKwJ/BlY\n",
       "HviYg715HO5m1hASuwF/Ak4A9khvt7cm8WwZM6sriZHAT4DtgO0juD3jkjqSW+5mVjcSa5C01lcG\n",
       "Pupgz47D3czqQuLzJP3rp5AMnL6QcUkdzd0yZlYTiaWAY4DPAhMiuC3jkgyHu5nVQKIbOJtkxa8N\n",
       "6vncUKuNu2XMrCoSOwG3AGcCn3ewtxa33M2sIhJLAj8GPgd8NoJbMi7JBuFwN7Nhk1gRuAR4hqQb\n",
       "5tmMS7IhuFvGzIZFYhRwLXAzsKODvbU53M1skSS6gOuBy4HeZi/6ZZVzuJvZO5JYBZgFnA8c5mBv\n",
       "D+5zN7MhSYwGrgNOi+CorOux4XPL3cwGJbE68AeStdcd7G3GLXczexuJD5AMnv40gulZ12OVc8vd\n",
       "zBaSrsE+C5jmYG9fbrmb2b9JjAOuAaZG8Kus67HqOdzNDACJdYCrgEMiOC3reqw2NXfLSDpZ0jxJ\n",
       "d5dtW1HS1ZIekHSVpBVqvY6ZNY7EesDVJHPYHew5UI8+91OA7Qds+x5wdUR8kGRQ5nt1uI6ZNYDE\n",
       "R4ErgSkRnJl1PVYfNYd7RNwIPDdg8w7w70//04Cdar2OmdWfxEbAZcDXIzg363qsfhrV575yRMxL\n",
       "f59H8sgtM2shEpsAFwL7RXBp1vVYfTV8QDUiQtKgtytLmlr2clZEzGp0PWYGEpsD5wF7R3Bl1vXY\n",
       "0CT1AD0Vvy+i9mUiJI0BLomIddPX9wM9EfGkpFWA6yNi3ID3RESo5oubWUUktgbOAr4QwXVZ12OV\n",
       "GW52NuompouBSenvk4CLGnQdM6uAxHYkwb6Lgz3fam65SzoL2AIYRdK//t/A74BzgPcDc4HdIuL5\n",
       "Ae9zy92siSQmAicDO0VwU9b1WHWGm5116ZaphsPdrHkkdgZ+QfJYvFuzrseql3W3jJm1CIndgJ8D\n",
       "n3awdw6Hu1mOSewJTAe2i+D2rOux5nG4m+WUxCTgGOBTEdyVdT3WXF44zCyHJL5KMrlhqwj+nnU9\n",
       "1nwOd7Mm6u7WhK4uphQKjCyVmF8sMmPOnLisnteQ2B84GNgygtn1PLe1D4e7WZN0d2vC+PFM7+1l\n",
       "bP+2vj7W6O4W9Qp4if8EDgR6IphTj3Nae3Kfu1mTdHUxpTzYAXp7GdvVxeR6nF/iIOAAYAsHu7nl\n",
       "btYkhQIjh9heqPXcEocB+5AE++O1ns/an8PdrElKJeYPsb1U7TklBEwFdiXpinmi2nNZvrhbxqxJ\n",
       "ikVm9PUtPMA5bRoPFYvMrOZ8abAfBeyMg90G8PIDZk2UzpaZXChQKJUoFYvMrGYwNQ32acDWwDYR\n",
       "PF33Yq0leW0Zs5xKg306sDHJnafPZlySNdFws9N97mZtRGIEcDywHsmdpy9kXJK1KIe7WZuQWAw4\n",
       "EViTpMX+YsYlWQtzuJu1AYnFgVOAVUlWd3w545KsxTnczVqcxBLA6cBKwMQIXs24JGsDDnezFiax\n",
       "JHAmUAB2iBh8rrzZQA53sxYlsRTJ4yoBPhfBa1nWY+3FNzGZtSCJkcCFwOvArg52q5TD3azFSCwN\n",
       "XAy8AHwhgn9lXJK1IYe7WQuReBdwKfAksHcEb2RckrUph7tZi5BYFrgceBjYz8FutXC4m7UAieWB\n",
       "q4B7gK9G8GbGJVmbc7ibZUzi3cA1wF+Ab0TwVsYlWQ443M0yJDEKuA64ATgwgmxW8rPccbibZUSi\n",
       "C7geuALodbBbPTnczTIgsQowCzgfONTBbvXmO1TNmkxiNElXzGkRHJV1PZZPDnezJpJYkWTw9FcR\n",
       "HJN1PZZffhKTWZOki4BdBdwWQW/W9Vh78mP2zFpI+mi8U4HlgF08j92q5cfsmbWW/wLWBnoc7NYM\n",
       "DnezBpP4IvBl4BMRvJJ1PdYZHO5mDSSxKfBTYKsInsy6Huscnudu1iASY4HzSFZ3/FvW9VhnaWjL\n",
       "XdJc4EXgTeD1iNiwkdczaxXplMdLgcMjuDLreqzzNLpbJoCeiHi2wdcxaxnp4/EuBC6O4ISs67HO\n",
       "1IxuGU93tI6RTnk8EXgGODjjcqyDNTrcA7hG0m2Svjpw50Yb6Yrubk1ocA1mzfR9YBywl5futSw1\n",
       "Otw3iYiPAJ8Gvilps/KdRx/NduPHM90Bb3kgsSfwJWCHCF7Nuh7rbA3tc4+IJ9L/PiXpQmBD4Mb+\n",
       "/aeeCqNGMfbVV5ku6dWImNXIeswaRWIz4H/xlEerM0k9QE+l72tYuEtaGlgsIl6StAywLXBE+TH7\n",
       "7pv8d+5cHp83z8Fu7UliTeBckq4YT3m0ukobvbP6X0s6fDjva2TLfWXgQkn91zkjIq4a7MBSiVID\n",
       "6zBrGImVSKY8/ncEg/7/bZaFhoV7RMwB1l/UcdOm8VCxyMxG1WHWKOmUxwuAiyL4Zdb1mJXLdFXI\n",
       "tdfe+Nknntj0pGefPea7mRRhVqV0yuPpwNLArp4ZY83SFkv+QkwEjgQ28GPGrJ1IHA58hmSVR8+M\n",
       "saYZbrhnvbbM5cBSwFYZ12E2bBJ7AfvhKY/WwjIN9/Sr7E/AT6Wx9pBOeTwWmOgpj9bKMn8Sk8RI\n",
       "YA6wjaeRWStLpzzeSLLK49VZ12OdqV26ZYhgPjAT+E7WtZgNJZ3yeBnwfQe7tYPMW+7J76wIzAbW\n",
       "ieCfmRRkNoR0yuPVwJ8jvBiYZastZsuUFygxA3glgkMyKchsEOmUx18DI4HdPOXRstaO4f4B4Fag\n",
       "O4KXMinKbACJqSQL323pmTHWCtqmz71fBA8D15GsqmeWuXTK4yQ85dHaUMu03JNtbAScDYyN4I1M\n",
       "CjMDJDYnef7plhHck3U9Zv3aruUOEMEtwCPA57OuxTqXxAeBc4A9HezWrloq3FN9wEHpQJZZU0mM\n",
       "Ilnl0VMera21Yrj/HlgW2CLrQqyzlD3Y+vwITsy6HrNatFSf+4J9/AfJINbEJpdlHSr9pvgbYElg\n",
       "d095tFbVdlMhF97HSGAuySPL7m1qYdaR0imP25MMoPrhMday2nJAtV+6JMHPgG9nXYvln8Q+JFMe\n",
       "d3SwW160ZMs92c8o4EFgLa++Z40isQXJ8097/C3R2kFbt9wBIngaOBM4IOtaLJ8kPkQy5fGLDnbL\n",
       "m5ZtuSfHMBb4MzAmgleaU5l1gvSb4c3AjyP4Vdb1mA1X27fcASKYDdxA8tQbs7pIB+wvAs5zsFte\n",
       "tXTLPTmOjYEzgDUjeLPxlVmeecqjtbtctNwBIvgz8ASwc9a1WC5MBdYA9nGwW561fLinvCSB1Syd\n",
       "8rg3yQ1ynvJoudYu4X4x8G5g06wLsfYk0UPSSJgYQTHjcswari3CPe1rPxbozboWaz/plMezgT08\n",
       "5dE6RcsPqC44nqWBOcDmEfy9cZVZnpRNefxRBCdlXY9ZrXIzoNovfRLOz/GSBDZMZVMez3WwW6dp\n",
       "m5Z78h66gL8DH3K/qb2TdPD9DGBx4AueGWN5kbuWO0Aa6GcD38y6Fmt5RwDdwCQHu3Witmq5J+/j\n",
       "Q8CNJEsS+KHF9jYSk4DDgU/4G57lTS5b7gDpYOpNJEu0mi0knfJ4DPAZB7t1srZruSfvZVPgFGCc\n",
       "lySwfhLjgD+QTHm8Lut6zBohty331J+AZ4Adsi7EWoPEe0iev3uIg92sTcM9ggCm4ZuajIWmPJ4T\n",
       "wclZ12PWCtqyWyZ5P4sBDwB7R3BT/SqzdiIxgmTK4wiS7hjPjLFcy7xbRtL2ku6X9KCkg+t9fi9J\n",
       "YKkjgDHAvg52swUa0nKXtBjJzUafAh4H/gLsERH3lR1TU8s9OQfLAHOBT0bwYC3nsvYjsS/wfWBj\n",
       "z4yxTjHc7Fy8QdffEJgdEXPTYn4L7Ajc905vqlQEr0j8AvgWsH89z22tqbtbE7q6mLL44iu9d9y4\n",
       "D6/13HNLTHnyySsd7GYDNCrcVwUeLXv9GLBRg651HHC/xOERPNWga1gL6O7WhPHjmd7by9hkstT1\n",
       "9PXR292tR+fMicuyrs+slTSqz71po7QRzAPOA77RrGtaNrq6mJIE+wK9vYzt6mJyVjWZtapGtdwf\n",
       "B1Yre70aSet9IZKmlr2cFRGzqrzescAsiWl+wk5+LbXUUsvCa2/bXihQyKAcs6aQ1AP0VPq+RoX7\n",
       "bcCaksYA/wR2B/YYeFBETK3HxSK4T+JWkkeo/bIe57TWIrHCWmt9fB3449v2lUr+QLf8Shu9s/pf\n",
       "Szp8OO9rSLdMRLwBHABcCdwLnF0+U6ZBpgHfSec9W46kD2q55LHHdp/V18fs8n3TpvFQscjMjEoz\n",
       "a1ltexPT28+HgFuAIyO4uF7ntWxJLAn8DngK2HfMGG3f1cXkQoFCqUSpWGSmB1Otkww3O3MT7sk5\n",
       "2Q04IILN63ley0Z6F/JZwBLArhG8kXFJZpnL/A7VjFwArCY1bNqlNUn6TewEYCWSZQUc7GYVyFXL\n",
       "HWCFFf7nF6NHX7zTqFH/d3+pxPxikRn+2t5e0mCfBmwKbBPBSxmXZNYysr5DNRPd3ZqwySba5qCD\n",
       "YmVgZYC+Ptbo7hYO+LZyKLAdsIWD3aw6ueqW6epiykEHxQfKt/kml/YicQCwH7BtBM9mXY9Zu8pV\n",
       "y71QYORg20eOHLFMs2uxyknsBRwMbB7BE1nXY9bOctVyL5WYP9j2efN6NpI4SlrorllrIRI7An3A\n",
       "dhHMyboes3aXq3AvFpkx2E0uzz8/YjLwLuBOifMktkgH7awFSGwFnAhMjODerOsxy4PczZZJl4Qd\n",
       "9CYXiWVJlig4AHiDZEXJMyJ4pd512PCk01YvIZnH/oes6zFrdR15E9Pwr42ArYDJJNPtTgOOj+Ch\n",
       "LOrpVBLrANcAX47g0qzrMWsHnXoT07BEEBFcG8FOwMdIWvE3S/xeYnuvT9N4EmsAVwDfcrCb1V9H\n",
       "ttwHI1EgWblyMrAM8DPg1AheyLSwHJJYFbgRODqCE7Kux6yduFumSmmXzSdJ+uW3A34LHOeBvvqQ\n",
       "WAm4ATg9gqOzrses3bhbpkppl82fItgDWAcoAtdKXCuxk5SvewOaSWI5kq6Yix3sZo3llvswpMvO\n",
       "fp6ky2ZV4HjgpAiezrSwNpJ2e11O8pD0/SOa9yhGszxxy72OIvhXBGdF8Engc8A44EGJkyU2yLi8\n",
       "liexBHAOyVO5DnCwmzWew71CEfxfBPsBawIPABdK/Elij7SFb2XSmUenAgImRfBmthWZdQZ3y9Qo\n",
       "7YP/LMkA7Fokz3A9wWuj/Htw+mfA2sCn/fBys9q5W6ZJIngjggsj2BrYBugC7pE4S+KTHb7MwZHA\n",
       "hsAODnaz5nLLvQEkVgAmkbTmXwJmAr/tpICT+C6wL8kKjx54NqsTz3NvAWl/87Yks2w+DpwM/DyC\n",
       "f2RaWANJLAUcQvLhtmkEj2dcklmuuFumBUTwVgRXRPAZkhujlgRul7hQYuu8ddlIbA7cAWwAbOZg\n",
       "N8uOW+5NJrEMsBdJl81iJCtTnh7By5kWVoP0rtNjSL6lTAEu8nRHs8Zwy71FRfBKup7Kh4H9SVan\n",
       "/IfEdIkPZltdZSQksQ9wD8nYwtrp4LKD3Sxjbrm3gPQJUV8HvkLSrTETuDyCtzIt7B2kH0Q/B1YA\n",
       "vhbBbRmXZNYRPKDahiRGAruRDMCuSDJH/JQInsu0sDLpgOn3SGr8AfCzCN7ItiqzzuFumTYUwfwI\n",
       "TieZG/5FkoHJhyVOkFg32+pAoge4C1gPWD+C6Q52s9bklnuLk1gZ+A+SbpsHSQZgL2pmqEqMInl4\n",
       "9VbA5Ah+16xrm9nC3HLPiQjmRfADYAzJapRTgDkSh0l0NfLa6YDpfiQDps+QDJg62M3agFvubUhi\n",
       "feCbwC4kD5eeGcFf6nyNccAvSJ5K9bUIbh/O+9IHlE8pFBhZKjG/WGRG/wPKzax2HlDtABIrAl8i\n",
       "CfoiySybcyN4rYZzjgQOBb4BHEFyR+2wVnLs7taE8eOZ3tvL2P5tfX3MvuceDnTAm9WHu2U6QATP\n",
       "RtAHjCVZpGsfkjnzP5AYXen5JLYG/kqyiuP6ERxXyRK9XV1MKQ92gN5exnZ1MbnSWsysNg73HIjg\n",
       "zQguiWBboIdk7vlfJc6R2HxRyxxIdEn8GjgJ+HYEu1SzdEChwMghthcqPZeZ1cbhnjMR3B/BZJIB\n",
       "2BtI1pe/U+KrEkuXHysxQuIrwN3Ak8D4CH5f7bVLJeYPsb1jVsM0axUO95yK4MUIjiN5gEgvMBF4\n",
       "RKJP4gMSawN/ILkrdtsIDorglVquWSwyo6+P2eXbpk3joWKRmbWc18wq5wHVDiLRTTJQul+66XCS\n",
       "p0bV7dF36WyZyYUChVKJUrHITA+mmtVPprNlJE0laRE+lW46JCKuqKZAq7+0e2aJCF7IuhYzq8xw\n",
       "s3PxBl0/gGMj4tgGnd9qEMGrWddgZo3VyD53t8rNzDLSyHCfLOkuSSdJWqGB1zEzswGq7nOXdDXw\n",
       "3kF2HQbczIL+9h8Aq0TElwe8P0jugOw3KyJmVVWMmVlOSeohuX+l3+EtsfyApDHAJRGx7oDtHlA1\n",
       "M6tQpssPSFql7OXOJDfJmJlZkzRqtszRktYnmTUzB/hag65jZmaD8E1MZmZtxKtCmpl1MIe7mVkO\n",
       "OdzNzHLI4W5mlkMOdzOzHHK4m5nlkMPdzCyHHO5mZjnUqDtUrUWlT0qaUigwslRifrHIDD8pySx/\n",
       "HO4dpLtbE8aPZ3pvL2P7t/X1sUZ3t3DAm+WLu2U6SFcXU8qDHaC3l7FdXUzOqiYzawyHewcpFBg5\n",
       "xPZCs2sxs8ZyuHeQUon5Q2wvNbsWM2ssh3sHKRaZ0dfH7PJt06bxULHIzKxqMrPG8JK/HSadLTO5\n",
       "UKBQKlEqFpnpwVSz9jHc7HS4m5m1Ea/nbmbWwRzuZmY55HA3M8shh7uZWQ453M3McsjhbmaWQw53\n",
       "M7MccribmeWQw93MLIcc7mZmOeRwNzPLIYe7mVkOOdzNzHLI4W5mlkMOdzOzHHK4m5nlkMPdzCyH\n",
       "HO5mZjnkcDczy6Gqw13SrpLukfSmpA0G7DtE0oOS7pe0be1lmplZJWppud8N7AzcUL5R0trA7sDa\n",
       "wPbA8ZJy9w1BUk/WNdTC9WfL9WennWuvRNWhGxH3R8QDg+zaETgrIl6PiLnAbGDDaq/TwnqyLqBG\n",
       "PVkXUKOerAuoUU/WBdSoJ+sCatCTdQHN0IgW9fuAx8pePwas2oDrmJnZEBZ/p52SrgbeO8iuQyPi\n",
       "kgquExVVZWZmNVFEbbkr6XrgOxFxe/r6ewAR8eP09RXA4RFxy4D3OfDNzKoQEVrUMe/Ycq9A+YUu\n",
       "Bs6UdCxJd8yawK3VFGdmZtWpZSrkzpIeBT4BXCrpcoCIuBc4B7gXuBzYP2r9emBmZhWpuVvGzMxa\n",
       "T6bzzyX9QNJdku6UdK2k1bKsp1KSpkm6L/0bLpC0fNY1VeKdbkRrVZK2T2+Oe1DSwVnXUylJJ0ua\n",
       "J+nurGuplKTVJF2f/j/zN0lTsq6pEpJGSrolzZt7Jf0o65qqIWkxSXdIesdJLVnfXHRMRKwXEesD\n",
       "FwGHZ1xPpa4CxkfEesADwCEZ11OpQW9Ea1WSFgOOI7k5bm1gD0lrZVtVxU4hqb8dvQ58KyLGk3TH\n",
       "frOd/v0jYj6wZZo3Hwa2lLRpxmVV40CSbu937HbJNNwj4qWyl+8Cns6qlmpExNUR8Vb68hZgdJb1\n",
       "VOodbkRrVRsCsyNibkS8DvyW5Ka5thERNwLPZV1HNSLiyYi4M/39ZeA+kvta2kZEvJr+uiSwGPBs\n",
       "huVUTNJoYALwKxaeyPI2WbfckXSUpEeAScCPs66nBl8CLsu6iJxbFXi07LVvkMuIpDHAR0gaNW1D\n",
       "0ghJdwLzgOvTCSDt5H+Bg4C3FnVgw8Nd0tWS7h7k57MAEXFYRLwfOJWk8JayqPrTYw4D/hURZ2ZY\n",
       "6qCGU38b8eh/C5D0LuA84MC0Bd82IuKttFtmNLB5O60zI2kiUIyIO1hEqx3qN899SBGxzTAPPZMW\n",
       "bPkuqn6XOaqqAAABOklEQVRJ+5J8Tdq6KQVVqIJ//3bwOFA+6L4aCy91YQ0maQngfOA3EXFR1vVU\n",
       "KyJekHQp8DFgVsblDNcngR0kTQBGAstJOj0i9hns4Kxny6xZ9nJH4I6saqmGpO1JviLtmA7WtLN2\n",
       "uKnsNmBNSWMkLUmy+ujFGdfUMSQJOAm4NyJ+mnU9lZI0StIK6e8FYBvaKHMi4tCIWC0iuoEvANcN\n",
       "FeyQfZ/7j9IugjtJVmr7Tsb1VGomyUDw1enUpOOzLqgSQ92I1qoi4g3gAOBKktkCZ0fEfdlWVRlJ\n",
       "ZwE3AR+U9Kik/bKuqQKbAHuRzDK5I/1pp5k/qwDXpXlzC3BJRFybcU21eMduSt/EZGaWQ1m33M3M\n",
       "rAEc7mZmOeRwNzPLIYe7mVkOOdzNzHLI4W5mlkMOdzOzHHK4m5nl0P8DGV+a6aOfOCUAAAAASUVO\n",
       "RK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105565ad0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "##This function provides an interactive mode to change polynomial degree. \n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w))\n",
    "    plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "    plt.plot(x,y,'yo')\n",
    "    print \"minimal MSE for n=7\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import scipy\n",
    "from scipy import linalg\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "minimal MSE for n=6\n"
     ]
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAi8AAAEZCAYAAABfH8VpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XeYnGX59vHvmYSQUBJCC4ReQgeFH1KUEgtCQu8ICSBo\n",
       "BISIdFCZjEiTVzA0FaQJSAcFhCgtgiggiiKE3ksIICXBJATI+f5x35tMJjub2d3ZnZ3d63Mcc+zO\n",
       "U+7nmtnZmWvuKtuEEEIIITSKXvUOIIQQQgihNSJ5CSGEEEJDieQlhBBCCA0lkpcQQgghNJRIXkII\n",
       "IYTQUCJ5CSGEEEJDieQlzJekEyVdXO84QgghBIjkpVNJelnSx5KWKNv+mKRZklbM95eXdJOkdyR9\n",
       "IOk/kg7I+1bOx04tu+1Z4ZoTJB3cnrhtn2772+0poySeL0u6Lz+ul6o4fiFJF5Y8F38u27+RpPvz\n",
       "c/CWpDEl++6T9LakKZKekvTtkn1flvS4pPclvSfpT5LWKSv7a5L+KekjSa+VPseStpD0d0kfSnqh\n",
       "tOwqHtOE/DfcoGz7LXn71vn+YpIulTQpP4ZnJB1fcvysHFvp6+CYCte8XNIp1cbYQuxNr7+q3zsk\n",
       "/bIsxhmSplQ4dglJD0p6Nz+3j0napeyYH+W/xwf5b7xOM+UMzde5smz7tyQ9l+O4U9KyJfsWk3SF\n",
       "pMn5VijZt2Iz/3OzJH2/2uchhFA7kbx0LgMvAt9o2iBpfaB/3tfkSuAVYEVgcWAUMLmsrIG2Fy25\n",
       "3dDCNSuS1Kd1D6HdPgJ+DRxb5fEXAYsBawGDgCObdkhaErgT+AXpeVoN+FPJuWOA5WwPAA4AzpO0\n",
       "Zt73JDDc9iBgMPAYcGlJ2esAVwMnAgOADYB/5H29gVuAi2wPBPYGzi5PRlpg4Blg/5LrLQFsDrzN\n",
       "nL/ZOcBCwFr5MewEPF9W1gZlr4P/V2UM7aVqD7R9SGmMwDXA9RUO/wg4CFg6P7djgeslLQIgaSfg\n",
       "EGBL0t/8b6T/l3IXAI9Q8vqXNAw4lfQ8Lg68lGNpcg7QD1gJ2AQYJenA/BheLXsM6wOzgJuqfR5C\n",
       "CLUTyUvnu4qSDy3Sh+pvmPvDYGPgctvTbc+y/S/b41t7IUmnkt7kz8/fFM/N22dJOkzSc6QPUSSN\n",
       "k/Rq/rb7qKQtSsoZ2/QNtuSb9/6SXsk1IidVG5Ptv9u+mvTBMb/41wJ2BEbb/q+Tx0oOOQoYb/sa\n",
       "25/Y/p/tp0uu9R/bn5Qc/xEwJe972/YbeXsv0gfRpJJjfwj80vYf89/gfdsv5n2DgSXIH5q2HwWe\n",
       "Atau9nkAfgvsLanp7/4N4GagNN6NgWtsf5iv84ztVn9YShoN7Ascl18Hv8/bhyjV8L0t6UVJR5Sc\n",
       "s0l+HXyoVKPVlBTdn39+kMvatJWxLAzsDlzR3H7bH+fH2VS7Mwt4F5iZD1kX+Ivtl23PIiWY5TVm\n",
       "+wDvA/cw9//VDsANtp/Kr4tTgK0krVKy/yzbM2y/AlxCSqSacwDwZ9uvtubxhxBqI5KXzvcQMEDS\n",
       "Wvkb/N6khKb8mAsl7a3clNSM+X7ztf0D4AHgu/kb45iS3TsDX2DOG/8jwOdItRu/BW6Q1LepqGaK\n",
       "/xKwBvBV4OSmGo3cnPL+/GKr0iakGqgf5yTpcUm7lezfFHg/NzNMlnSrpBVKC5B0u6TpwATgINuT\n",
       "SvatmGOdBmwPfKusbOVrvinpSkmD8r5JwOPAQZJ6S/oi6dv6X1rx2N4EJgLb5vujSElsqYeAUyUd\n",
       "KGlohXKqeR1cRPqQPzO/DnbOicFtpBqnIaS/45GSvp5PGweck2s/VgWaava2zD+bav4ebnoeJS0/\n",
       "v1hIicvbth9o6SBJjwPTgcuBXW03JS/3AJvnZqEFSEnEnSXnDQCKwPeZ97lx2bam97/1Si9dtr90\n",
       "X9M1RPoC0mwCFkLoeJG81MeVpDe/bUgfYG+U7d+TlHT8CHgxt/tvXHbMu/kDo+m2JpU19wF3uu0P\n",
       "bH8MYPvqXLswy/bZwILAmi2cX8zfkh8H/g18Ppfzl9wUUwvLkz48PgCWBQ4Hrih5rCuQPrzGkJrY\n",
       "ypsBsL0DsAjp+b68NBnMTQGDgCXzY7i05NQVgJHAbsBQUtPeefk8A6NJH5IzgD8DJ5XU5FTrN8D+\n",
       "uYZpMdsPle0/gpR0HA48mftqbFd2zD/LXgfbtHC90r/jF4Albf/E9qe2XyI15+2T988Ehkpa0vY0\n",
       "2w83UwYw53m0/XoVj7mpprFFtjcAFiU1G93U1Gxk+xFS0vAMKencnVQD1+QU4Ne232TepHs8sKek\n",
       "9SX1B07OxyxUsv94SYtIWp1U69K/mfC2AJYGbpzvow0hdIhIXjqfScnLfjTfZEROKk60vR6pieJf\n",
       "wO/Kylkif2A03Z6ZzzXLvVZ6R9IxkiYqdYJ8HxhI+lCv5K2S36cBC7dwbFtNJzWjNH3A3g/cx5za\n",
       "imnAzbb/kZOwIvBFSYuWFmL7M9s3Ag8Du5ZfxPb7wDHAjvmbe1PZl9l+3vb/gNOAEQCSlgNuB/a1\n",
       "vQCpKeN4SSNa8dhMaib6CvBdmvlAz80Xp9vemNRMdT2pRmyxksM2LHsd3FXl9VcChpQmPqT+PUvn\n",
       "/QeTataekvSIpO1b8dialRPHrakieQGwPdP2ecBU0vOEpMNJtUTLkxLsHwP3Suov6fN538+bLllW\n",
       "3j3kZIiU6L6Uy25KusaQktHnSH2afsu8Xywg/d/eaHtaNY8jhFB7kbzUQW4nfxEYTvoAa+nY/wI/\n",
       "I33QtKVGo1KH3dKOjFuSOtDuaXuxXBvxIa3olNlBHs8/m6v+L91frQWA/7WwbxbwcRVlfxF4vSlR\n",
       "sP0s8AfS37NqtqeTmjwOoflOp6XHTgVOJyWJq7R0bKUiyu6/CrxUlvgMyDVV5KRtX9tLAWcCN+ba\n",
       "ivYsQz+K3F+llef1ISWTANuR+gG9mWsJryA1da5DSoxWBl6VNAk4Gthd0qNNBdm+0PYatpch/e/1\n",
       "AZ7I+963PdL2srbXB3qTEt7Z8nOwB9FkFEJdRfJSPwcDX8kfYHORdKakdSX1ybUIhwLP5RqC2YdV\n",
       "eZ3JpFE4LVkU+JTUFNVX0smkETatUVU8SvqRkgVJWrCkb025P5M+ZE/Mz8WXgGHAH/P+y4BdJX0u\n",
       "93/4EfCA7amS1pQ0PH8jX0DSSFIH2D/lOHaVtIakXpKWAs4G7mhqRstlf1PSKpIWAk4g9RGBNFJp\n",
       "TaXh1pK0Gqmz57+re6rmchKwdXMdP5WGBG+c/yb9gO+ROqKW1rK15nWwasn9R4Cpko7Lz1FvSes1\n",
       "NU9KGpmfF0iJrEnJ3Tv55/xeU83Zn9SHpSJJmyr1m+qb4zqeNAKoqUntcWAvSUvnv90oUgLyHGlk\n",
       "2qqkvlufB35JSiq3zWUvmB+jci3QRcDPmzpES1pVaah2b0nDgW8DPykLcVfgPdsT2vD4Qwg1EslL\n",
       "ndh+0fY/SzeV/N6fVG39PvACqf/FTmVFNI32aLodSfPGAXsozWXy8wrHjM+3Z4GXSc01pR+mLouv\n",
       "uW/fhlSLI2lqhetA+nY8jfShskK+1uyRVJKekPQNANufkjoWjyD1e/kVMCrXdGD7PtKH/x+Y8+G8\n",
       "b1NRQCFvf4vUGXf7kiRhuXzdKcA/Sc/1AbMfjH0ZqXnj4ZLnZEzeN5GUUF5A+mCfQGpG+HULj7tZ\n",
       "tifZ/muF3bNISdQ7pOaLr+bHUNpc8e+y18HZFcq6BFgnNxHdnEfq7ED6kH8xX+Mi5iSt2wJP5L/l\n",
       "OcA+uY/TNNJw4wdzWZtozhwoFTvsStqc1DF4niH9ku6QdEK+uyBwPmmE0avAVsB2tj/K+39CSt4e\n",
       "J/3NvgfsbnuK0+i8t/NtMml02fRcewkpCbqa1FT0MPAgKeFt8n+53Cn5Me5r+6mycPdnPrVkIYSO\n",
       "p9T3sA4XTqNCfkNqYzdpzoxzJS0OXEdqk38Z2Mv2B3UJMoQQQghdTj2Tl2WAZWz/K48k+AewC/BN\n",
       "4F3bP81VxoNsn9BSWSGEEELoOerWbGT7Ldv/yr9/RJrkazlS80hTZ7grSAlNCCGEEALQRfq8SFoZ\n",
       "2JDUDj04t1dD6q8wuE5hhRBCCKELqnvykpuMbgK+l4eDzpYnA6tPu1YIIYQQuqTOXpRvLnl4603A\n",
       "lbabJmGbLGkZ228prfj6djPnRUITQghtYLtd8zfF+2/obM29ZuuWvEgSafjmRNulQ3hvJQ1ZPTP/\n",
       "LJ9ZFmj/P2BnkDTW9th6xzE/EWdtRZy11QhxNkKMULvEoxHef0P3UOk1W8+aly+R1o55XFLTSsEn\n",
       "AmcA10s6mDxUuj7hhRBCCKErqlvyYvsvVO5z87XOjCWEEEIIjaPuHXa7uQn1DqBKE+odQJUm1DuA\n",
       "Kk2odwBVmlDvAKo0od4BVGFCvQMIoSep2yR17SHJ0eYaQgitU4v3znj/rZ28KO7FtteqdyzNkfQE\n",
       "cJjt+2t5bCtjaPb1FslLCCH0EJG89Ax57rQXgT55HbOGVen1Fs1GIYQQuh1JNe3TWevyOknFJFNS\n",
       "784MpNYieQkhhNAtSHpZ0nGSHgemSuolaTNJf82roP9L0tYlx68i6X5JUyTdJekCSVfmfStLmiXp\n",
       "IEmvAHfn7QdJmijpPUnjJa1YUt45kiZL+lDS45LWzdtHSHoyX+d1SUfn7cMkvVZy/tqSJuRYn5C0\n",
       "Y8m+y3N8t+dyHpK0aoWnoqnp5oN87GaSDpT0oKSzJb0LFCStKuleSe9KekfSVZIGlj2fX8m/j5V0\n",
       "vaQrcplPSPq/Nh67kaTH8r7rJV0n6ZTW/K0jeQkhhNCd7AMMBxYDlgVuB35sexBwDHCTpCXysb8F\n",
       "HgIWB8aSpu8o70uxFbAWsJ2knUlTeuwKLAk8AFwDIGlbYEtgqO2BwJ7Af3MZlwCjbQ8A1gXuLQ86\n",
       "T9p6GzAeWAo4Arha0holh+2d4xwEPA+cWuE52DL/HGh7gO2H8v1NgBeApYHTSDUzp+bnaW1ghVx+\n",
       "k/LnYsf8eAeS5mQ7v7XHSuoL3AJcmh/HNaQ1DFvVh6URq8FCCCF0USrWaCK8Qpv61Rg41/YbAJJG\n",
       "AnfYHg9g+25JjwLbS5oAbAx82fanwIOSbmXeppaxtqfn8g4BTrf9TL5/OnBSrn2ZCSwKrC3p703H\n",
       "ZDOBdSX9x/aHwGPMazNgYdtn5Pv3Sbod+AZQzNtutv1ovvbVwNkVnodKz92bti/Iv88gJTIv5Pvv\n",
       "SjoHOLnCuQAPND2Xkq4CjmzDsZsBvW2fl+/fIumRFsppViQvIYQQaqaNSUctvVby+0rAnqXNL6TP\n",
       "vXuBIcB7tmeUnbvCfMobJ+lnZccMsX2fpPOBC4CVJN0MHJPX7Nsd+CFwRm7SOqGkNmR2GWXXAngl\n",
       "b4eUmE0u2TcdWITWmat8SYOBccAWpMSrF/BeC+eXXn8a0E9Srwqdgps9lvR43mgmrla9bqLZKIQQ\n",
       "QndSWvPzKmntvEElt0Vt/xSYBCwuqX/J8Ssyr/LyRpeVt3BTImL7PNsbA+sAawDH5u2P2t6F1Bz0\n",
       "O+D6Zq7zJrCCpNIP8ZWY94O+GpVqv8q3nwZ8BqyXm7pG0fF5wSRgubJtKzYTW4sieQkhhNBdXQXs\n",
       "KOnrknpL6pc7yS5n+xXgUWCspAUkbQ7sQMsfor8kNROtAyBpoKQ98+8bS9o0912ZRmqW+SyXvZ+k\n",
       "gbY/A6aSEoZyD+fzjsvnDMvxXJv3t6Zm4h1gFrDafI5bBPgfMEXScuRkq4P9jfS8HC6pT+5H9IXW\n",
       "FhLJSwghhG7J9uvAzsBJwNukmpOjmfPZtx+wOalj7SnAdaT+KbOLKCvvd6RFg6+V9CHwH2DbvHsA\n",
       "cBGp2eVl4F3grLxvJPBSPmd0vu5c17A9k9TJdTgp+TgfGGX72ZLjyhOrZhMt29NIHXEfzKOiNq1w\n",
       "fhHYCPiQ1Fn4pkpltub6LR2bH+duwMHA+6Tn4nbmft7nKyapCyGEHqIW753d+f1X0nXARNvF+R4c\n",
       "akbSw8CFtq9oZl9MUhdCCCE0yU09qynNBzMc2InUJyV0IElbSVomNxsdAKxHGiJetRhtFEIIoada\n",
       "BrgZWII04uUQ2/+ub0g9wpqkTssLk4Zq72F7csunzC2ajUIIoYeIZqPQaKLZKIQQGoQkSaucXjZs\n",
       "NoSQRfISQghdzmq7wXaHwWq71juSELqiSF5CCKGLkIaMltZ6EoadBhcOgGGnS2s9KQ0ZXe/YQuhK\n",
       "osNuCCF0GZMuhv7vQe+fpTnJeveDT4+CSTfXO7IQupKoeQkhhC7CtkGGXovBXhNBg/LmxhtZEUIH\n",
       "iuQlhBC6lL5D4a5vwg3rwd0HpvuhO5K0paSn6x1HI4qh0iGE0EPEUOmeQdLKwItAnworPremrMuB\n",
       "12z/qP2Rten6MVQ6hBBCzyCppn06a11eJ+m2SWYkLyGEELoFSS9LOk7S48DUPO3/ZpL+Kul9Sf+S\n",
       "tHXJ8atIul/SFEl3SbpA0pV538qSZkk6SNIrwN15+0GSJuYFD8dLWrGkvHMkTZb0oaTHJa2bt4+Q\n",
       "9GS+zuuSjs7bh0l6reT8tSVNyLE+IWnHkn2X5/huz+U8JGnVCk/F/fnnB5Km5oUZWx27pNHAvqSV\n",
       "rqdK+n17/j61FMlLCCGE7mQf0srMiwHLklYs/rHtQcAxwE2SlsjH/hZ4CFgcGEta/bm8L8VWwFrA\n",
       "dpJ2Bk4EdgWWBB4ArgGQtC2wJTDU9kBgT9Jq1QCXAKNtDwDWBe4tD1rSAqSVnccDSwFHAFdLWqPk\n",
       "sL1znIOA50krRzdny/xzoO1FbT/clthtXwRcDZyZy9m5wvU6XSQvIYQQakbCtbi18fIGzrX9hu2P\n",
       "ScnIHbbHA9i+G3gU2D7XOmwMnGz7U9sPArcyb1PLWNvTbc8ADgFOt/1M7ktyOvD5XNZMYFFgbUm9\n",
       "8jFv5TJmAutKGmD7Q9uPNRP7ZsDCts/I8dxHSry+UXLMzbYftf0ZKan4fIXnobnmorbGXqm8uork\n",
       "JYQQQs3YqBa3doTwWsnvKwF75maY9yW9D3yJtCDjEOC9nJQ0d26l8saVlNVUszIkJxvnAxcAkyX9\n",
       "StKief/uwAjg5dwstFkz1xnSzPVfydshJWalixdOBxZpppxK2hp7lxTJSwghhO6ktNbmVeBK24NK\n",
       "bova/ikwCVhcUv+S41dkXuXljS4rb2HbDwHYPs/2xsA6wBrAsXn7o7Z3ITUH/Y60onK5N4EVytaz\n",
       "Wgl4ozUPvpmY2xV7hbLqrq7Ji6RLcweh/5RsG5s7ND2Wb9vVM8YQQggN6ypgR0lfl9RbUr/cSXY5\n",
       "26+QmpDGSlpA0ubADrT8Yf1L4CRJ6wBIGihpz/z7xpI2zX1XpgEzgM9y2ftJGpibe6YCnzVT9sP5\n",
       "vOPyOcNyPNfm/a2pjXoHmAWs1p7Y83mTgUodg+um3jUvlwHlyYmBs21vmG/j6xBXCCGEBmf7dWBn\n",
       "4CTgbVLtw9HM+ezbD9ic1IRyCnAdqf/H7CLKyvsdcCZwraQPgf8A2+bdA4CLgPeAl4F3gbPyvpHA\n",
       "S/mc0fm6c13D9kxgR1Jn43dIzTijbD9bclx5YtVsomV7Gqkz74O5mWiTdsR+CbBOLqfLLFNR90nq\n",
       "lCbTuc32+vl+AfjI9s9aOCcmSQohhFaqxXtnd37/lXQdMNF2sd6xhKTS663eNS+VHCHp35IukbRY\n",
       "vYMJIYTQ/eTmktWU5oMZDuxE6pMSuriumLz8AliFNARsElCxBiaEEKolsZbEZRJTJM6SiC9GYRng\n",
       "PlI/lHOAQ2z/u74hhWp0uemObb/d9LukX5Mm7ZmHpLEldyfYntCxkYUQGpHEhqQ+D1sD5wKbAEcB\n",
       "z0j8GPiVzad1DLHD5E6fw+ocRpdl+3bSXCqhwXTFPi/L2p6Uf/8+8AXb+5ad023bXEMItSGxBSlp\n",
       "+Rzw/4CLbT4q2b8BqWZ3OdLMq3faXXNYaK1En5fQaCq93uqavEi6hvRtaEnScKwC6VvC50m9qF8C\n",
       "vmN7ctl58c8TQpiHhICvk5KW5UmjK66w+biF40eQkpvXgKNt/tPcsd1BJC+h0XTJ5KWt4p8nhFBK\n",
       "ohewCylp6QecBlxfbXOQxAKkIawnA78HTrZ5q+WzGk8kL6HRRPISQmgYKmplYBvSHBxTgNfLbm+6\n",
       "4Jk56fgGcALwEWlui9tsZrXpumIQ8APgQFKT0s9tprfnsXQlkbyERhPJSwihy1JRA4EvkxKWbUgr\n",
       "At9NWvl2IVIT0HL55/J80m8Z/jF6Og8e14+FJ09hkwse5POXP0SvWaUJzhsuuE2Jh8RqpCanjUkr\n",
       "8V7bXH8YDdYIBjCGvvRjJjOYwrme7Dvacs3OEMlLaDSRvIQQugwVtQCwKXOSlfWBh4A/AXcBj7vg\n",
       "eWpPJBYFvgM+igWmPcGGl17LiDEf0JTUzH1bjjQEtrzWZq6bC/6o/Dol19sSOBv4FDjK5m+z9w3W\n",
       "CJZnHDux+uwTbuV5Xud7XTWB6WnJi6TLgdds/6jesYS2ieQlhFA3Kkqkxd6akpWtSR3y78q3v7RU\n",
       "SyKxODAG+C6pRuYMmxbn41BRvUiDAZpLbEpvM2kpwflw+Tc455UdoNdpwIPACTYva6jGM3L29Opz\n",
       "XMV4P+fh1Twvna0HJi+XkZKXk6s4dgJpEcdL2nity6lBopRH4L4I9LHnTeB7mkqvty43z0sIofE0\n",
       "13zCYfwd+CpzEhaREpVrgW+7MGdOp4rlimVJc7IcBNwCfNHmuWpiyjU3b+fbP5stPyVVg5g3ofni\n",
       "7N8Hvr4CY3uLmQu/wT2nfoHHDnpaa9/zDwb2Htzs+np96T/vxp4hrYi88mnw8klu4zfjWpRRXmSV\n",
       "x3W1b/INkSDWSyQvIYR2abb55C6G8RKfsgr3kZqCzgKecaG6DyOJVYBjgX2AK4HP27xW69hzPO/l\n",
       "2+MV4ylqAH3/tzzDj1yeda9fj7vOHMlHW64CE+Y9eGb36eDbeqvtBtscBnf9HWjjIn5tL0PShqSF\n",
       "BFcH7qAkIZE0iPRa2oT02fcgaUbdNySdCmwJbCbp58BltsdIGgfsCgwEngOOtP2XZq47GtgXsKQj\n",
       "gXtt7yxpCHBeLvsj4Bzb5+VzNgEuBIYC04GrbB8D3J+L/UASwNdsP9ya56FHsN1wtxR2/eOIW9wa\n",
       "8cbSjGB1xrMOE1id8SzNiHaVtzrjGYvnuQ1lfKvLwmuDfwN+F3wqeOl6P18VYx2w8+Gst9y0uR7z\n",
       "Rjzf3uezQ2OuwXtnc2XAsqNhzSfh4GdgltPPNZ+EZUdXX277ygD6Aq8A3wN6A7uTmgR/nPcvTkpE\n",
       "+gGLANcDt5Scfx9wUFmZ+5Fq5nqRagAnAX0rXP+ypmvl+72AfwA/JCVLqwAvAF/P+/8G7Jd/XwjY\n",
       "NP++EjAL6FXv10tXuFV6zUbNSwg9SIVOpqtpsGhLJ1MV1ZclWLfZnQvQr+pyxP+R5mjZAhgHjGFp\n",
       "fZEB/Ebrds2RPP7wd+drqd4vcvlyp6AV1uYT/4+pi/0/fzC+y8TYeSZdDP3fg94/S60dvfvBp0fB\n",
       "pFbUnLS7jM1I/UTG5fs3Sfp7007b75GaHgGQdBpwb1kZczXV2L665O7Zkn4IrAkVJzIsPf8LwJK2\n",
       "f5Lvv5SXvNmHVBs5ExgqaUnb7wIPN1NGqCCSlxB6kgGMmStxAdiJ1bmKI0jV7FVTUauS+q8s2OwB\n",
       "VTSfSGxFSlrWJc1yO8pmWq2TrDnXq21/Cr/z2R3AHRJ9gUOBH+dlB8bazLdPT3dh29Lqhl6LwV4T\n",
       "YfEV8uaqn+MalDEEeKNs2yvkZEDSQqTFF7cl1aYALKLcI7QpjNKTJR1D6m81JO8bQOoEXo2VgCGS\n",
       "3i/Z1ps5zUIHAz8GnpL0ElC0/Ycqy+7xuuKq0iGEjtK3Qm1IKzuZqqg9SUObf8tLHMitPD/XAbfy\n",
       "AlM4r9lzhSSGSzxA6p9wA7C6zTibaUDlJGsAR7Qmznmtthtsdxistmv7ypmbzUybccBawMfARInj\n",
       "pOprnxpf36Fw1zfhhvXg7gPT/U4tYxJpeHyplZiTkBxNGvG2ie2BpBFvYk5NR3nisiWp39Wethez\n",
       "PQj4kMo1I+VJ1qvAS7YHldwG2N4BwPbztve1vRRpTqEbJfVvppzQjKh5CaEnmcmMCtur6mSqovqT\n",
       "5j3ZBhjhgh+lABosuIoj6Et/ZjKdKZxXXkMi0RvYjVTT0ps0hf+Nbm4K/xolWXOuPWQ0DPgeDOsD\n",
       "Fw6Ab58urXUKTBlnv3lRW8psjs17wPclfgH8FDhE4njS4+zWH0r2xDNK7raps247y/gr8KmkMcAv\n",
       "gB1JTTf35P2LkDrGfihpcdJaeqUmA6uV3F+UNL/Pu5L6kmZxHtDC9ScDq5bcfwSYKuk4UqfdmcDa\n",
       "QD/bj0oaCfzR9jukpMikvi7v5J+rQXUj63qiqHkJoSeZwrmtqSUppaLWItW2DAL+zwU/2rTPk32H\n",
       "n/NwP+lhfs7DSxMXiQUkDgSeJH37PRn4nM21zSYu0O4ka16TLoZPCqkfxez+FCen7bVn86zNLqSm\n",
       "gZOAv0hs0hHXContT0jJ8YHAf4G9gJtKDvk50B94l5To3MnctRzjgD0kvZdHHI3Pt2eBl0mJz6st\n",
       "hHAJsI6k9yXd7DRHyw6khYZfJCUlFzEnAdoWeELSVFJz1j62P7Y9jbTMxYO5rHjdNCMmqQuhh8lz\n",
       "srRYSzLPOUUdQOqTchLw62qGPEv0J/UXOI70DfI04L5qaiAq9Hl5gdcZ09Y+L9Lqu8M2l8J/X0/9\n",
       "Ke4+0H6+jcN5W3NdegOjSB9IE4AT7RY/BDswlp41SV1ofDHDbgih1VTUIsAFpOr3vV1wpVEWc84R\n",
       "A0idV48kjaA43abV81S0JclqOa51ToCZz8ILt6Q+L32H2hPPbGt5rb8+i5D6UBwO/JI0S/DUzrp+\n",
       "iiGSl9BYInkJIbSKitqANBfGX4EjXPD/WjxeLEmawv9Q0lDQM+yKQ0p7LInlSbVQXyM1oV1mNzdV\n",
       "b0dcO5KX0FgieQkhVCVPmf8d4BTg+y74qhaPF0NIfVm+CdwInGnzQocH2uAkNiZ1fl6MtOjj3R1/\n",
       "zUheQmOJtY1CCPOlohYDLiZNr/4lF/xsxWPFqsDxwJ7A5cAGNq93Rpzdgc2jEluTZn39lcRE4Fib\n",
       "p+scWgjrXyJhAAAgAElEQVRdXow2CqGLkyRpldOVFzrpsOsUtQlpAcPJwOaVEheJ9SSuIg0FfRtY\n",
       "0+aoSFxaL890fjOwDml6+gckzstNcCGECiJ5CaHL65iJ1ZqoqF4q6mjgduAYF3y4C55nqLLEFyRu\n",
       "Ae4GngBWtfmRzTsdEVdPYvOxzdmkeUAAnpI4Woqp4kNoTvR5CaGLmjOx2hZ94OI14NvPwl8+reXE\n",
       "aipqSVKTz5LAPi745bljQKSZSH9AWtPlLOCS2TPhhg4hsRawvc3Paltu9HkJjSX6vITQcGqx2F1l\n",
       "Kmor4Grgt8APXfAns/elpGV70rwuSwBnAFfbzKzFtUPLcr+X6PsSQgWRvITQRdVisbvm5NFEJ5CG\n",
       "NR/kgu+cvS9NqLYHKWkxaUjvTZ01lDeEWpJ0OfCa7R/VO5ZqSNoP2N/2trU8tjuK5CWELq1pobo5\n",
       "E6u1pzQV1Qs4F/gisLELfgMgr4o8ijR66B3gRODO7r4eT+j2TJULHUqaAFxp+5K2XKgWiZLtq0m1\n",
       "oTU9tjuK5CWELqwWi901UVELkPq3LA982QV/KLEQ8C3gGOAp4NvA/ZG0hNbKMyKPoS/9mMkMpnBu\n",
       "a2dErkUZzRVb5XEd+pqX1Md282t5hVaL0UYh9AB5NehbSIvCbcdYI3EiacG4YcDuNtva/DkSl9Ba\n",
       "s9eiGsm27MXWjGRblmecBmtEp5YhbSjpn5KmSLoW5qxOLmmQpNslvZ0XX7xN0nJ536nAlsD5kqZK\n",
       "OjdvHyfpVUkfSnpU0hYVrjsa2Bc4Lp//+7z9ZUnHSXqctMJ0b0knSHo+x/ikpF1KyjlQ0gMl92dJ\n",
       "+o6kZ/Mijee3cOzXJT0j6QNJF0j6s6SDq33uGk0kLyF0cypqIPBH4AMu/PdoxvoHwAukuUW+arOb\n",
       "zd/rGmRobAMYM9cimgA7sToDOKKzypDUF/gdcAVp5fMbgN2ZU6Mi0srPK+bbdOB8ANs/AB4Avmt7\n",
       "Udtj8jmPAJ/L5f0WuCFfZy62LyI14ZyZz9+5ZPc+wHBgMdufAc8DW9geABSBqyQNbuGhbQ9sDGwA\n",
       "7CVpnj4ukpbMj/d4YHHgGWBzOrg2qZ4ieQmhG1NRSwP3MXm95xn72bu8vcGTpGHRm9iMsnmyziGG\n",
       "7qDvnBqOsu39O7GMzYA+tsfZ/sz2TTAnKbf9nu1bbM+w/RGpM/rWZWXM1cRk+2rb79ueZftsYEHS\n",
       "lAGVlDdRGTjX9hu2P85l3mj7rfz79aQV1zdtocwzbE+x/RppIsPPN3PMCOAJ27/LsZ4LvNVCmQ0v\n",
       "kpcQuikVtSLvrP0Ql91nfvH4LtDrU2A9m0NsXqx3fKEbmck8kxrm7dM7sYwhwBtl214hJxSSFpL0\n",
       "q9yU8yHwZ2Bg2czVc9VUSDpG0sTcFPM+MBBaPfvxa2Vl7i/psdwM9D6wHmk6gkpKk5BpwMLNHDME\n",
       "5pnhulvPeB3JSwjdkDY9f0euu/EpfvXo0rwy7HbQUJtjbN6sd2yhG5rCudzK83Ntu5UXmMJ5nVjG\n",
       "JGC5sm0rMSchORpYA9jE9kBSrYuYU1tSnrhsCRwL7Gl7MduDgA+p3AG4UhPN7O2SVgIuAr4LLJ7L\n",
       "fKKFMqv1JqkjftN1VHq/O6rraCNJl5La8962vX7etjhwHelF9zKwl+0P6hZkCA1EYjMWeudMtOcW\n",
       "LP/QDXy60LdtptY7rtC9ebLv0GDBVRxBX/ozk+lM4bzWjBSqQRl/BT6VNAb4BbAj8AXgnrx/EVI/\n",
       "lw/z50yh7PzJwGol9xcFPgXezf1cTiB1eK9kMrDqfGJcmJTMvAv0krQ/qealWqXJVqk7SJ2Ndwb+\n",
       "ABwCLNOKchtOvWteLgO2K9t2AnCX7TVIL7oTOj2qEBqIhCS+KnEPfab/jq1O3YjvbLSPn955n0hc\n",
       "QmfxZN/h5zzcT3qYn/Pwtgxxbk8Ztj8BdgMOBP4L7AXcVHLIz4H+pMThr8CdzF1bMg7YI49E+jkw\n",
       "Pt+eJX2Rng682kIIlwDr5OagZqc1sD0R+BnwN1Jz0HrAX0oPKYupvDandP/s322/S1rd/af58a0N\n",
       "PAp83EK8Da3uaxtJWhm4raTm5Wlga9uTJS0DTLC9Vtk5sbZG6PEkegE7kGbDXYx1rxvPbqP2pfcn\n",
       "e7vg++ocXuiCYm2jnkFSL1Jfm31t/7ne8bRHI61tNNj25Pz7ZKClIWQh9DgSfUjfsk4CPgFO5Qf9\n",
       "F2aBGT8FtnfBMew5hB5G0tdJQ7unk/rqADxUv4g6VldMXmZLa7uo2aohSWNL7k6wPaFTggqhTiQW\n",
       "BPYnzeUwifQG9UfG6gjSDLlfccET6xhi6GIkDSNNQhi6v81Jc9H0BZ4Edmkant0dddVmo2G235K0\n",
       "LHBfNBuFnkxiYdK0/ccA/wFOs3kgL7B4MrAfsI0LfqWOYYYGEM1GodFUer3Vu8Nuc24FDsi/H0Ca\n",
       "MTGEHkliKeAfpKnLd7YZnhOXXqQOiLsAW0biEkLoSepa8yLpGtJY+yVJ/VtOBn4PXE+avvllmhkq\n",
       "HZl/6AkkFiXNqHmnzeyVavMCi5cAqwA7uhBTCYTqRM1LaDSVXm91bzZqi/jnCd2dRD/SfA3PAYc2\n",
       "LZaYF1i8jtRfbQ8XPK1+UYZGE8lLaDSRvITQICR6k2ofZwH72HwGoKIGkJpVJwEHuOCZ9YsyNKJa\n",
       "JS+1iieEajTKUOkQGpIGawQDGENf+jGTGUzh3NZO1CUh0uygA4AdShKXpUiTaj0CHOGCP6t1/CFU\n",
       "I744hq4gkpcQakCDNYLlGcdOrD57462spsGilQnMT0irxn7VTrNjqqgVgLuAG4EfudCA1aUhhFBD\n",
       "0WwUQg1oqMYzkm3n2XEV4/2ch1dVhjiStCbJFjbvAqioNYE/Aue64LNrGHLogeK9M3QXUfMSQi30\n",
       "pV+F7f2rOV1iFHAUcycuG5E67f7ABV9ao0hDCKHhRfISQi3MZEaF7dPnd6rE9sBZwJfttPCbitqK\n",
       "1Ex0iAtudpG3EELoqbriJHUhNJ4pnMutPD/Xtlt5gSmc19JpElsAl5MmoHsKQEVtT0pcvhGJSwgh\n",
       "zCv6vIRQI3m00RH0pT8zmc4Uzmups67EBsDdwEibPwGoqP2As4GdXPDDnRN56CnivTN0F5G8hFAH\n",
       "EqsCDwBH2VwHoKK+C5wAbOeCn6xnfKF7ivfO0F1En5cQOpnEMsCfgFNtrssLLP6QtJbXVi74pboG\n",
       "GEIIXVwkLyF0IomBwHjgSpsL8wKLPwO+Amzhgt+qa4AhhNAA5ttsJGkJ2//tpHiqElWfoRFJ9Ccl\n",
       "Lo8DYxir3sCvgaHADi74/XrGF7q/eO8M3UU1o40eknSDpBGS4kUfQhtI9AGuBd4AvsdYLUgaUbQM\n",
       "8PVIXEIIoXrVJC9rAhcD+wPPSzpd0hodG1YI3Uder+giYEHgQMZqYeAO4GPSqKL/1TO+EEJoNK0a\n",
       "bSTpK8BVwMLAv4ATbf+1g2JrKY6o+gwNQ+KnwJbA1xir/qQFFv8JHBYLLIbOFO+dobuYb4ddSUsC\n",
       "+5FqXiYDhwO3AZ8jVXuv3IHxhdDQJI4Ftge2ZKwGkUYZ/R44KRZYDCGEtqlmtNFfSbUtO9t+vWT7\n",
       "o5J+2TFhhdD4JL4JfBfYgrFagrQy9AUu+Kz6RhZCCI2tmtFGvWzP6qR4qhJVn6Grk9gZ+CWwNWO1\n",
       "EKmPy8ku+Nf1jSz0ZPHeGbqLmGE3hBqT2Bq4ARie+7jcTOrfcmN9Iws9Xbx3hu4ikpcQakhiQ+CP\n",
       "wD6MVT/gCmA/F/yn+kYWQrx3hu5jvkOlJW3RzLYvdUw4ITQuidWBPwCHMlaDgctIQ6EjcQkhhBqq\n",
       "ps/LY7Y3nN+2zhTfHkJXI7Es8CBwOmPVB/gBMNwF/6e+kYUwR7x3hu6i4mgjSZsDXwSWknQU0PSC\n",
       "X5TqJrcLoUeQGERqKvo1YzUAOIy0wOKL9Y0shBC6p5aGSvclJSq9888mU4A9OjKoEBqFxEKkeY/u\n",
       "4eQ+TwPjgM1dmGtagRBCCDVUTbPRSrZf6aR4qhJVn6ErkFgAuAV4n5MWPoe+0/4IbOeC/1Hn0EJo\n",
       "Vrx3hu6imknqLm9mPUbb/koHxBNCQ5DoBVwKwKHrn0jfaQ+ShkNH4hJCCB2smuTl2JLf+wG7A592\n",
       "TDghdH15ocWfAauw6bgdGfzEncClLviGOocWQgg9wnyTF9uPlm36i6S/d1A8ITSCE4Gv0nfK1gw/\n",
       "8nzgFeDHdY4phBB6jGoWZly85G4vYGNgQIdFNOe6L5M6B38GfGJ7k46+ZgjzIzEa+BawBScNPBRY\n",
       "A9g6FlkMIYTOU02z0T+BpjfmT4GXgYM7KqASBobZfq8TrhXCfEnsDhRI6xVtChwKbOaCp9U3shBC\n",
       "6FmqaTZauRPiqKRir3gN1XimcK4n+47ODCj0TBJfAX4BfJ2xWgS4iDQJ3Rv1jSyEEHqeapqN+pMm\n",
       "3dqCVBvyAPAL2zM6ODYDd0v6DPiV7Yvn2juSbbmV1TRYRAITOpLExsC1wJ6M1VvAw8B3XZinP1gI\n",
       "IYROUE2z0W9IfU/OJdWE7AtcCezZgXEBfMn2JElLAXdJetr2A7P33gcsyup8zDhJ02xP6OB4Qjcj\n",
       "SbDyafDySa4w4ZHEmqRJ6L7NWD1MeuVd5oKv78xYQ2gLScOAYXUOI4SaqyZ5Wdf2OiX375U0saMC\n",
       "amJ7Uv75jqRbgE1ItT7Jl/PPd3jDH0TiEtpitd1gm8Pgrr8DN5fvlVieNO3/SYzVraRE/jViZFFo\n",
       "EPlL3YSm+5IKdQsmhBqqZo2if+Z1jgCQtBnQoRNxSVpI0qL594WBrwPNL3A3k+kdGUvofqQho6W1\n",
       "noRhp8GFA2DY6dJaT0pDRs85hiVIicsFNpcBxwNrAwe64Fl1Cj2EEALV1bxsDDwo6TVSP5QVgWck\n",
       "/Yc00+4GHRDXYOCWPLNvH+Bq23+a56hbeYEpnNcB1w/d2qSLof970PtnqSW0dz/49CiYdDOAxMLA\n",
       "7cAfbM5SUTsDhwObxsiiEEKov2qSl22Zd9SPm9lWM7ZfAj7f4kF3M5MPOTE664bWsm1pdUOvxWCv\n",
       "ibD4CnmzJfoCNwFPAcerqM8BvwZGxMiiEELoGqpJXn5ie1TpBklXlm/rdF/jWmBdIKZkD23Qdyjc\n",
       "9U144RZYbVfoOzSvV3Q5MAMYzVgtDdwKHOGCY1bpEELoIqpZVfox2xuW3O8DPF7WibdTSTJjWZ00\n",
       "ZHUNF2Iiu9A+eb2ic4ENgO0YKwP3Ane54OjkGLqFWFU6dBcVO+xKOknSVGB9SVObbsDbpG+jdeWC\n",
       "XwBuAY6pdyyhW/gRaS6jnRirGaRJ6N4AinWNKoQQwjyqqXk5w/YJnRRPVZq+PaioFYHHgLVc8Dv1\n",
       "jis0JolDgaOBL9lMVlHHA3sBW0YH3dCdRM1L6C6qSV62Zs7aRrPZvr+jgpqf0n9AFXU+MMMFRw1M\n",
       "aDWJvYCzga1sXlRROwEXktYser2+0YVQW5G8hO6imuTlduYkL/1Ik8X9w/ZXOji2lmIqTV6GAE8A\n",
       "67ngN+sVU2g8EjsAlwDb2DyuojYA7gG2d8GP1De6EGovkpfQXcx3kjrbO9jeMd+2AdYDPuj40KqT\n",
       "E5bLgRPrHEpoEBILSJwB/BLYOScuTSOLxkTiEkIIXVs1M+yWe50002hXcgawX+4DE0JFEquQlplY\n",
       "H9jQ5iEVtSBpeYArXfA1dQ0whBDCfFWzqnTpDLa9SJPHdejyAK3lgt9WUb8CfgB8p97xhK5JYm/g\n",
       "POB04Oc2VlECfgW8BcSQ6BBCaADVTFL3D+b0efkM+K3tBzsupDb7f8AzKupMF/xivYMJXYfEQsA4\n",
       "0uq6w+25ku9jSHO7bBlrFoUQQmOopsNuf2B1UgLzvO0ZnRFYSyp1OlNRRWAlF3xg50cVuiKJ9YFr\n",
       "gX8Ch9lMnb2vqB1J/V42c8Gv1SnEEDpNdNgN3UVLk9QtIOmnwGvAFcBvgNclnSVpgc4KsJXOAbZX\n",
       "UWvWO5BQXxLK87fcC5xpM6oscVmfNNJot0hcQgihsbTUYfcsYHFgFdsb2d4IWBVYjNRE0+W44A+A\n",
       "nxN9F3o0iUHAjcC3SRPP/Wau/XNGFh3pgh+uQ4ghhBDaoWKzkaTngTXsufsBSOoNPGN79U6Ir1kt\n",
       "VX2qqEWB54GvuuAnOjeyUG8SXwKuBn4HHG/z8Vz708iie4AJLviHdQgxhLqJZqPQXbRU8zKrPHEB\n",
       "sP0Z0GU7NrrgqaRao7F1DiV0IoneEj8EbgIOtzmymcRFpD4ubwMn1yHMEEIINdBS8vKUpAPKN0oa\n",
       "BTzdcSHVxIXAF1XUhvM9MjQ8iSHAXcDXgP+zub3CoUeThvqPipFFIYTQuFpqNlqeNHHXdObM6/J/\n",
       "wELArnb91n2ppupTRY0BtnHBO3ZSWKEOJLYndby9ADjN5rNmjytqB9J8LjGyKPRY0WwUuosWh0pL\n",
       "EvAVYF3SUOmJtu/ppNgqqjJ56Qc8B+wRnTK7H4kFSZPN7Q6MtHmg4rFFrUcadbSTC36ok0IMocuJ\n",
       "5CV0F/Od56UrqvYfUEUdAuzqgrfthLBCJ5EYSpq75VXgYJv3Kh5b1FLAw8CPXPDVnRRiCF1SJC+h\n",
       "u2jL2kaN5FJgDRW1Rb0DCbUhMRL4K+lvu9t8Epe+pA6810TiEkII3Ue3rnkBUFHfBA4AvuxCAz7Y\n",
       "AIDEIqR+LZsA+9j8u8Xj08iiS0jzEu0RHXRDiJqX0H1095oXgCuBIaS+O6EBSWxEmt7/E2Dj+SUu\n",
       "2VHARsD+kbiEEEL30u2TFxf8KVAETsnfxkODyFP8fw8YD5xs8y2b/833vKK2Jw2L3skFf9TRcYYQ\n",
       "Quhc3T55ya4FBgLb1TuQUB2JJUlT+O8HbGZzbVXnFbUucBmwuwt+tQNDDCGEUCc9InlxwZ+RZtz9\n",
       "cdS+dH0Sw4DHgKeALWxerOq8opYEbgOOcsF/67gIQwgh1FOPSF6ym4AFgJ3qHUhonkQfiR8DvwW+\n",
       "ZXOczcyqzp0zsug6F3xVR8YZQgihvnpM8pI7bZ5Mqn3pMY+7UUisANwHbA5sZPPHqs9NtWkXAu8D\n",
       "P+iYCEMIIXQVPe1D/DbgY2CPegcS5pDYBXgUuB3Y1uatVhZxJLAxMDJGFoUQQvfXJZMXSdtJelrS\n",
       "c5KOr1W5eZ6Xk4GxKqp3rcoNbSPRT+IC4GxgZ5sz7datWK6ihgPHEiOLQgihx+hyyYuk3sD5pJFB\n",
       "6wDfkLR2DS/xR1LzwjdqWGZoJYm1SdP2L0VqJqq45pAGa4SGarzW1QQN1XgN1ggAFbUOcAUxsiiE\n",
       "EHqULpe8kGZQfd72y7Y/IQ1z3rlWhefalx8CBRXVp1blhurkuVsOBu4nJal723xQ8fjBGsHyjGMk\n",
       "27IXWzOSbVmecVpT+5CaAY+OkUUhhNCzdMXkZTngtZL7r+dtNeOC78vX2L+W5YaWSQwkjSQ6Etja\n",
       "5mKblpdsGMAYdmL1ubbtxOoswQXADS74yo6KN4QQQtfUFWseqlp/SNLYkrsTbE9o5XV+BFytoq5y\n",
       "wVUNxw1tJ7EJcA3wJ2ATm+lVndiXfs0XiIGTahVfCN2RpGHAsDqHEULNdcXk5Q1ghZL7K5BqX+Zi\n",
       "e2x7LuKCH1RRTwMHA79oT1mhMolepKn6jwEOs7mpVQXMZEaz29/lsRhZFELL8pe6CU33JRXqFkwI\n",
       "NdQVm40eBYZKWllSX2Bv0jTxHeFk4Acqqvlv96FdJAYDdwC7kGpbWpe4AEzhXG7l+bm23cYrfMA5\n",
       "NQkyhBBCw+lyyYvtT4HDSaOCJgLX2X6qQ65V8COk1Yq/0xHl92QS25Cm+H+U1L/llbaU48m+g9f5\n",
       "HtfzIPfyCTfyMK9xmCf7jpoGHEIIoWHIrqqLSZciybZrskaRivo8cCewugue74rFoWUSCwCnACOB\n",
       "/W3ubVd5afbcbwGnA991wde1P8oQeqZavneGUE9dsc9L57qQIayCMI/rKj3HFM6Nb/VtI7EKqVPu\n",
       "f4ENbd5pV3lpocWLgVWArVzwxPZHGUIIodF1uWajzjR7DpHhDGYEqzKSbVmB85smQQvVk9iLNOnc\n",
       "dcCONUhctgH+BTwPbBqJSwghhCY9utlIQzWekWw7z47beZkd2NwFt3aNnR5HYiHg58CXgX1s/tGu\n",
       "8opaEDiN1FH7QBd8d/ujDCFANBuF7qNH17xUnEOkDwsAE1XU2Spqmc4NqnFIrA/8HehPmuK/vYnL\n",
       "usAjwMrA5yJxCSGE0JyenbxUnkPkP8B6QG9SEnOOilq2M0PryvIU/4cC9wJn2oyymdrm8oqSijqc\n",
       "NB/FOGAPF/zf2kQbQgihu+nZzUZNfV5Kp5+/lRd4nTFNnXZz0nIccADwG+BMFzypvdduVBKDgF+T\n",
       "OtHuY/Nsu8orajBwKWmBxv1c8HPtjzKE0JxoNgrdRY9OXiAnMAM4gr70ZybTmcJ5zY02yknMscCB\n",
       "wJWkJObNWsTQKCS+BFwN/A443ubjdpVX1PakROgSoOiCP2l/lCGESiJ5Cd1Fj09eWn3t1AfmWOCb\n",
       "wFWkJOaNesTSWSR6AyeSJg/8ts1t7SqvqP7AWcAOwP4u+P72RxlCmJ9IXkJ3EclLW2NIzR3HAgeR\n",
       "Vko+wwXPswZTo5MYQkrSegH72bQrUcuTAv6WNAz6MBf8QfujDCFUoyu8d4ZQC5G8tFNOYo4hLfB4\n",
       "DSmJea2+UdWGxPakJp0LgVNtPmtzWUX1Ao4ETgC+74Kvrk2UIYRqdaX3zhDaI5KXGlFRSzMnibkO\n",
       "OL1RkxiJBUnT8e8OjLR5oF3lFbUccDmwEDDSBb/U7iBDCK3WFd87Q2iLnj1UuoZc8Nsu+DhgLWAq\n",
       "8C8V9QsVtWKdQ2sViaHAX0mjiTasQeKyK2nxy/uBrSNxCSGE0F5R89JB8ro8RwOjgRtINTFtWlm5\n",
       "s0iMBM4BxgIX2rT5xaGiFsllfYU0BPqhmgQZQmizRnjvDKEakbx0sJzEHAV8B7gROK2rJTESiwAX\n",
       "AJsCe9v8u13lFfU5UsL2IDDGBbd5ArsQQu000ntnCC2J5KWTqKglSEnMIcBNpCTm5boGBUhsBFwL\n",
       "PACMsflfu8or6qukjsvfc8HX1CDEEEKNNOJ7ZwjNieSlk+Uk5vvAocDNpCSm0/uBSAgYA/yQlLS0\n",
       "O9FQUfuSmor2cMHt6isTQqi9Rn7vDKFUJC91oqIWZ04S83vgVBf8YqdcWyxJmpJ/GeAbNi+0q7yi\n",
       "ROrfMwYY7oKfbH+UIYRa6w7vnSFAJC91l5OYI4HDgFtJSUy7kokWryeGkZY3uAb4oc3MdpWX5m/5\n",
       "GfA1UuLS7SbqC6G76E7vnaFni+Sli1BRg4Dvkabgv42UxDxfs/JFH+Bk4FvAN23+2O4yi1qQtFjl\n",
       "MsDOMVtuCF1bd3zvDD1TJC9djIpajJTEHAHcDvykvUmMxAqkKflnAKNs3qpBnANJCzS+C4xywTPa\n",
       "W2YIoWN15/fO0LNE8tJF5SRmTL79gZTEPNfqcsQuwK9IHWl/ajOrBrEtB9wJTCBN9d/mZQNCCJ2n\n",
       "J7x3hp4hkpcuLtdwNCUxd5KSmGfne57oR+qLMoLUKbcmk8SpqLVzHBcCZ7nQgC+gEHqonvTeGbq3\n",
       "SF46SJ5m/xJS59gbbNrVHyQnMUeQmpTGk5KYZypce23S3C3PAKPbe+2SGLYgzVFzjAu+shZlhhA6\n",
       "TyO8d4ZQjUheOohEX2A4MArYBriLlMjc2Z4RPipqAHOSmLuAU1zw0/maAr4JnAmcBPy6PVP8l113\n",
       "V1Lz00gX/KeKxw3WCAYwhr70YyYzmMK5nuw7ahFDCKF9GuG9M4RqRPLSCSQGAXuQEpm1SVPnXwk8\n",
       "1NbkIicxh5OGWd/N89uew1XjjwLWI03xP7EmwadrHfr/27v3GDvKOozj36cta6uAl4i12sZiWxOr\n",
       "IigaFS+LBq1Ei/WClhAUDUpUaBRvQOK6oAIaUajRCN5iKog3pCgWi7QRrVYJbUELQpEqSL2itkRq\n",
       "AR//mKGebvfsnt3u7szseT7JZufMmZ19dpuZ/vZ9531fisnsXu0+39D2uJk6mtlcwGLm7965ki3c\n",
       "xbIUMBHVa9q9M6KdFC8TTOJg4DiKQmYasAJYYTOqEUXq1wFsOv4c1px1MjM33cGRfcf685s2jEnW\n",
       "YvK5jwJvABYNN4meFmgVx/OKvd5YwSrf5leORaaIGL0m3zsjWk2rOkC3sbkD+JjEx4HDgeOBn0n8\n",
       "jqI15jKbv3dyLokp4JOBY9l/25tZumQ2sEr9WkPRnTTqmW7Vr/2Ai4CFwBHu81+H/aIeprfZP2O0\n",
       "OSIiIgaaUnWAbmVjm1/ZLANmA2cDLwJ+J3GFxOvLEUODkpgJXAW8BniOd8z6uvt8HjAPuAG4Vv26\n",
       "TP16+kizqV/7UyxZcBDw0o4KF4BdDD7Xyy7uG2mGiIiIdmpXvEj6iKS7JG0oPxZVnWm82dxvc5XN\n",
       "UmAOcDnFmkd3S1ws8eKilaUgcRSwAbgeeInN73efq8/3us+foChirgeuUb++2WkRo349DlgDbANe\n",
       "4z53vsr0di5k5YDur5XcznaWd3yOiIiIYdTumRdJfcAO2+cPcUxX9NtKzOb/z8ccAHwd6AGWAifY\n",
       "XDvsOfr1CIpC6H3AdcBZ7vNNbY6dB1xdfp+PjGYOl3K00Sn0MINd3Md2ludh3Yh66JZ7Z0x+dS1e\n",
       "7rX9qSGO6aoLsBwCfQhFEXMQ8D6bzrpyHjpHUcScTFHErKMoYja1vH84xcKQ/e7zF8Yqe0TUR7fd\n",
       "O2PyqmvxciLwL4puj9PsPRf8ywU4eurXw4F3AB8Afg6cRbGw4teAk9znKyqMFxHjKPfOmCwqKV4k\n",
       "rab4D3OgM4FfwO5WhbOBWbbfNuDra38BShLM/ThsPcN1qxDZXcS8naKImQoscZ/XVZsqIsZTE+6d\n",
       "EZ2oXctLK0lzgSttP2PAfgP9LbvW2l47ccmGJ81/HRz1ZVh9or3lu1XnaUf9mgE8zH0ekyUEIqI+\n",
       "JPUCvS27+lK8xGRQu+JF0izb28rt9wDPsX3cgGNq+9eD9IS3w4HL4IXT4OKnwEm3wk8fgO0X2Hdf\n",
       "VGNvMfYAAAgNSURBVHW+iOhedb53RoxEHSepO0/SoYCBOyiez2iQbRfDjHtg6qdAwNTp8MB7YVtt\n",
       "W18iIiKapHbFi+0Tqs6wL2xbmm+Y8ig4djM8Zk65u15NXBEREQ1Vu+JlcuhZAKtPhNsvh3lLitcR\n",
       "ERExFmr3zEsn0m8bETFyuXfGZFG75QEiIiIihpLiJSIiIholxUtEREQ0Sh7Y7WLlIoqn0sN0drGT\n",
       "7VyYRRQjIqLuUrx0Kc3U0czmAhYzf/fOlczTTJECJiIi6izdRt3qQE7do3ABWMx8DuSUihJFRER0\n",
       "JMVLt+phepv9MyY4SURExIikeGkQSZIOPqdYsXof7WJnm/337fO5IyIixlGKl0aZ91pY9M5i1t59\n",
       "tJ0LWcmWPfat5Ha2s3yfzx0RETGOMsNuA4zXStXlaKNT6GEGu7iP7SzPw7oRk1e33Ttj8spoo0YY\n",
       "n5Wqy0IlxUpERDRKuo0aoFiRWi0rVevRZKXqiIjoUml5aYysVB0REQF55iUiomvk3hmTRbqNIiIi\n",
       "olFSvERERESjpHiJiIiIRknxEhEREY2S4iUiIiIaJcVLRERENEqKl4iIiGiUFC8RERHRKCleIiIi\n",
       "olFSvERERESjpHiJiIiIRknxEhEREY1SSfEi6Q2SfiPpQUnPGvDe6ZJuk3SLpJdXkS8iIiLqq6qW\n",
       "l5uAJcBPWndKWgi8EVgILAI+J6mxrUOSeqvO0InkHFvJObaakLMJGSMmk0oKA9u32L51kLeOAS61\n",
       "fb/trcAW4LkTGm5s9VYdoEO9VQfoUG/VATrUW3WADvVWHaBDvVUH6EBv1QEiukndWjWeANzV8vou\n",
       "4IkVZYmIiIgamjZeJ5a0Gnj8IG+dYfvKEZzKYxQpIiIiJgHZ1dUGktYAp9m+oXz9IQDb55avVwF9\n",
       "ttcP+LoUNBERo2BbVWeI2Ffj1vIyAq0X0krgEknnU3QXLQB+OfALcvFFRER0r6qGSi+RdCfwPOAH\n",
       "kn4IYHsz8E1gM/BD4J2usmkoIiIiaqfSbqOIiIiIkarbaKOOSTpb0iZJGyX9WNKcqjMNRtInJd1c\n",
       "Zv2upEdWnWkwQ00cWAeSFpUTF94m6YNV5xmMpC9L+rOkm6rO0o6kOZLWlP/Wv5Z0atWZBiNpuqT1\n",
       "5fW9WdI5VWcaiqSpkjZIGslghAklaaukG8uce3XHRzRJY4sX4BO2n2n7UOB7QF/Vgdr4EfA0288E\n",
       "bgVOrzhPO4NOHFgHkqYCn6WYuHAhsFTSU6tNNaivUGSss/uB99h+GkW37bvq+Lu0vRM4sry+DwGO\n",
       "lPTCimMNZRlFd3edm7IN9No+zHaT58+KaG7xYntHy8v9gb9VlWUotlfb/m/5cj0wu8o87QwxcWAd\n",
       "PBfYYnur7fuBb1BMaFgrtq8D/lF1jqHY/pPtjeX2vcDNFPMr1Y7tf5ebPcBU4J4K47QlaTZwNPBF\n",
       "9hyAUEd1zxfRkcYWLwCSPibpD8CbgXOrztOBtwJXVR2igZ4I3NnyOpMXjgFJc4HDKIrq2pE0RdJG\n",
       "4M/AmvKB/jr6NPB+4L/DHVgxA9dIul7SSVWHidgXdRgq3dZwE93ZPhM4s5wf5tPAiRMasNTJhHyS\n",
       "zgR22b5kQsO1GMOJAydanZviG0nS/sC3gWVlC0ztlC2Wh5bPiV0tqdf22opj7UHSq4C/2N7QgPWN\n",
       "jrC9TdJBwGpJt5SthRGNU+vixfZRHR56CRW2aAyXU9JbKJqVXzYhgdoYwe+zbv4ItD6QPYc9l5GI\n",
       "EZC0H/AdYIXt71WdZzi2/yXpB8DhwNqK4wz0AmCxpKOB6cCBkr5m+4SKc+3F9rby818lXU7RHZvi\n",
       "JRqpsd1Gkha0vDwG2FBVlqFIWkTRpHxM+RBiE9StX/x6YIGkuZJ6KFYeX1lxpkaSJOBLwGbbn6k6\n",
       "TzuSHivpUeX2DOAoaniN2z7D9hzbBwNvAq6tY+Ei6eGSDii3HwG8nOIh/YhGamzxApwj6aayT7wX\n",
       "OK3iPO0sp3igeHU5RPFzVQcaTLuJA+vA9gPAu4GrKUZ0XGb75mpT7U3SpcA64CmS7pRUSTfmMI4A\n",
       "jqcYvbOh/KjjCKlZwLXl9b0euNL2jyvO1Im6dnHOBK5r+X1+3/aPKs4UMWqZpC4iIiIapcktLxER\n",
       "EdGFUrxEREREo6R4iYiIiEZJ8RIRERGNkuIlIiIiGiXFS0RERDRKipeIASSN+XT5kp4kaelYnzci\n",
       "ohuleInY23hMfnQwcNw4nDciouukeIloQ1KvpLWSviXpZkkrWt7bKuk8STdKWi9pXrn/q5Je13Lc\n",
       "jnLzXOBF5Yy2ywZ8nyWSrim3Z0n6raTHjf9PGBHRTCleIoZ2KLAMWAg8WdILyv0G/mn7EOCzwGda\n",
       "9g/mg8B1tg+zfUHrG7YvB7ZJejdwEfBh238Z458jImLSSPESMbRf2r7bxToaG4G5Le9dWn7+BvD8\n",
       "Yc4z3GKXpwCnAzttXzaaoBER3SLFS8TQ/tOy/SAwrc1xD7W4PEB5XUmaAvR0+H3mlOefWa78HBER\n",
       "baR4iRi9N7Z8XldubwWeXW4vBvYrt3cABwx2EknTgC8BbwJuAd47DlkjIiaNdn9FRnQzt9ke6NGS\n",
       "NgE7gYeGQV8MXCFpI7AKeGjY9SbgwXL/VwY893I68BPb6yTdCPxK0vdt/3YsfpiIiMlGRVd+RIyE\n",
       "pDuAZ9u+p+osERHdJt1GEaOTqj8ioiJpeYmIiIhGSctLRERENEqKl4iIiGiUFC8RERHRKCleIiIi\n",
       "olFSvERERESjpHiJiIiIRvkfO/rTcyrjvGcAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105544490>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Read test data here\n",
    "a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
    "xt= a[:,0]\n",
    "yt= a[:,1]\n",
    "X = construct_poly(x,8)\n",
    "w = wRidge(X,y,1.0)\n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n)  \n",
    "    Xt = construct_poly(xt,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w),'g',label='regression training')\n",
    "    plt.plot(xt,Xt.dot(w),'b',label='regression test')\n",
    "    plt.title(\"MSE train: %f  MSE test: %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.ylabel('Output y') \n",
    "    plt.xlabel('Input x')  \n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
    "    print \"minimal MSE for n=6\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#####Conclusion:\n",
    "We get suitable solutions for n from 5 to 7 for the test regression. For higher values we get overfitting. For lower ones underfitting. The MSE of the test regression is always higher than the one of the training regression. That's evident because the regression function is fitted to the values of the training data and so doesn't provide an optimal fitting for the test data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum MSE for lambda = 0.1\n"
     ]
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAk4AAAEACAYAAABF4/l0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecVNX5x/HPlyYooCiKIlZAjSXRxN6CxoYFjL3XaGLD\n",
       "JGo0JnGYaCy/GKNgSWJssXdFRY2JEnuLLWIDe0EsqICAgHx/f5wLDMvO7uzOzNbn/Xrd1zK3nHPu\n",
       "6Mw+e865z5FtQgghhBBC/To0dwNCCCGEEFqLCJxCCCGEEEoUgVMIIYQQQokicAohhBBCKFEETiGE\n",
       "EEIIJYrAKYQQQgihRGUFTpJWkPSQpLGSXpY0LNu/pKQHJL0h6Z+SlqhMc0MIIYQQmo/KyeMkaVlg\n",
       "WdsvSOoO/BfYFTgU+Mz2/0k6Gehl+5SKtDiEEEIIoZmU1eNk+2PbL2T/ngq8CiwPDAGuyk67ihRM\n",
       "hRBCCCG0ahWb4yRpZWA94Cmgj+2J2aGJQJ9K1RNCCCGE0FwqEjhlw3S3AsfbnlJ4zGksMNZ1CSGE\n",
       "EEKr16ncAiR1JgVNV9u+I9s9UdKytj+WtBzwSS3XRTAVQgiNYFvlXB/fvyGUprbPWlmBkyQBlwGv\n",
       "2D6/4NAo4GDgnOznHbVcXvaHvylIGm57eHO3oz7RzsqKdlZWa2hna2gjVC7oaQ3fvyE0p2KftXJ7\n",
       "nDYDDgBekvR8tu/XwNnATZIOB94B9iqznhBCCCGEZldW4GT7UYrPk9qmnLJDCCGEEFqayBxevzHN\n",
       "3YASjWnuBpRoTHM3oERjmrsBJRrT3A0o0ZjmbkAJxjR3A0IILV9ZCTDLqlhyjLGHEELDVOK7M75/\n",
       "Q6hfsc9J9DiFEEJoVyQdK+lZSTMkXVHC+UtKul3SVEnvSNq3AXW9I+kbSUvV2P+8pDmSVsxeby7p\n",
       "cUlfSvpc0qOS1q9RzjRJUwq2EdW43+yafSS9mt3zeEmbl3jsGkkTJE2W9Jak39Qot77jdZXdT9Jd\n",
       "2fszQdJISR1LuZ9KisAphBBCiyNpmKQzq1T8h8DpwOUlnn8RMANYBtgfuETSmiVea+AtYF6wJWkd\n",
       "oFt2DEk9gbuBC4BepBU48sA3NcrZ2XaPgm1YiW1o0P1K2pb0kNfBtrsDW2T3UOexzFnAKrZ7AoOB\n",
       "4yTtUM/x7UssewTwGbAcsC7wQ+DoEt+DionAKYQQQks0EthLUsVXnrB9u+07gc/rO1fSYsBuwO9s\n",
       "T7P9GHAncGADqrwGOKjg9cHAP4C5w0CrpWb5RiczbD9g+38NqKOohtxvJg/kbT+dXT/B9kclHMP2\n",
       "WNszCsqaTUEuxyLHPy2lbGAt4EbbM7PVSe7L9jWpCJxCCCG0ONmqE9dRR4Ai6W5JXxTZRpVQTSnz\n",
       "vFYDZtseX7DvRRr2C/tJoKekNbKhpb1JwdRcbwDfSrpS0g6SejWkvZIuknRRCe2o936z9v0AWEbS\n",
       "OEnvZ0NiXes6VqOMiyV9DYwFzrD9XB3H/2D7uRLLvh/YT1I3ScuTeqzuLeG+KyoCpxBCCAuQcCW2\n",
       "CjTlSuCQYgdt72y7V5FtSAnll9LG7sDkGvumAD1KuLbQ1aRep22BV0jDZ6kR9mRg86w9lwKfSLpT\n",
       "0jIF1wu4o0ZweHh2/TG2jymhDaXcbx+gM7B71qZ1SevQ/pY0VFns2PxK7KNJ79s2wBmSNqzj+OnZ\n",
       "8brqnWs4sDbpv8f7wDNZT1qTisAphBDCAmxUia0CTVkaWLTmL94KKqWNU4GeNfYtTgqeSmVS4LQ/\n",
       "Cw/TpRPs12wfansFUnDQFzi/RhlDawSHlzWgDdSss4jp2c+Rtifa/hw4D9ixnmMLyIYcxwA3UzC/\n",
       "q47j0+oqW5JIPU43A4sCvYElJZ1Twj1VVAROIYQQWpxsQvGGwBnAoUXOubfGU2aF2z0lVFNKD8wb\n",
       "QCdJAwr2fQ94uYRr51dkv0ea6DwYuK2ec18HriIFUJVU7/3a/gL4oPZD/rLIsbp0Br6u73gJZfcm\n",
       "DeVdaHuW7UmkHsmFgrZqi8AphBBCiyJpP2Br2yNJPQy71JxHA2B7cI2nzAq3neoov2NWXiego6RF\n",
       "ij3WbvtrUqDze0mLZo/H70LqQWqow7P7ml64U9Lqkk7I5u0gaQVSL8wTNZveiDobdL+ZK0hPuy2d\n",
       "zbf6Bempv2LH7srqWTpLJ7BYVuf2wJ6kyfT1Hq+rbNLTdBOAo7JrlyD13r3YmPekLLabZSOb+xdb\n",
       "bLHFFlvpWyW+O1vy9y+wMXB5jX0XAodUsI7hwJwa22kFx0cDpxS87gXcThq2ewfYpwF1vU0Klmru\n",
       "7wR8C6xIGpa7kdTjMjX7eQnQvUY500hDhHO3W7NjfwEuqeD9diKlYPiCFKycD3Qp4VhvUgb+L4Av\n",
       "gaeBIQXl1ne8aNnZ8Y2AR7LjnwI3AEtX8f/FWj8nkTk8hBBakUp8d8b3bwj1K/Y5iaG6EEIIIYQS\n",
       "ReAUQgghhFCiCJxCCCGEEEoUgVMIIVSBJEmrnJXlnwkhtBEROIUQQlX03w12OBr6/7i5WxJCqJwI\n",
       "nEIIoYKkvkdKa4yFQWfCxT1h0FnSGmOlvkc2d9tCCOXr1NwNCCGEtmXCpdBtEnT8U8pX2LErzP4l\n",
       "TKgzW3QIoXWIHqcQQqgg2wYZOiwBe70C6pXtbp6keSGEiooepxBCqLguA+GBQ+HN29Mcpy4Dm7tF\n",
       "IYTKiB6nEEKoMPuVs+3xt9l2+vlKk6/gHoqTtKSk2yVNlfSOpH3rOf9YSc9KmiHpigbW9Y6kbyQt\n",
       "VWP/85LmSFoxe725pMclfSnpc0mPSlq/RjnTaixkPKIa95tds4+kV7NrxkvarNT3opZrN8/2d5F0\n",
       "WdaGydl7sEPBdd+R9GD2HoyTtGuNcvtJuit7fyZIGlnPmntVEYFTCCGEFkfSMElnVqn4i4AZwDLA\n",
       "/sAlktas4/wPgdOByxtRl4G3SIv2AiBpHaBbdgxJPUmL6F5AWhdveSAPfFOjnJ294ELGw0psQ4Pu\n",
       "V9K2wNnAwba7A1uQ1sqDet6LIte+lR3uBLwHbGm7J/Bb4CZJK0rqRFrsd1T2HhwJXCOpsLd2BGmx\n",
       "3+WAdYEfAkeX+B5UTKxVF0IIrUh7Wasuy381DtjM9sQKlrsYMAlYy/b4bN9VwEe2f13PtacD/Wwf\n",
       "2oD63gb+Dgy1vWG279ysDWcAK5MCmgds96qnnMNtP1hq3dl1Db5fSY8Dl9ou2rtW7L0o5doa579I\n",
       "WoR4HPCE7R4Fx+4HnrJ9Wvb6deB42/dlr/8P6Gn7Z6XU1VCxVl0IIYRWI5tMfx1wYLFzJN0t6Ysi\n",
       "26gil60GzJ4bRGReBNYqoVmNDTafBHpKWiMbWtobuKbg+BvAt5KulLSDpGIBVK31S7pI0kVFrmnQ\n",
       "/Wbt+wGwTDZc9n42JNa1vrY04Nq55/fJ2je2SNs7AGsXvL4f2E9SN0nLA4OBe4tcWzUxOTyEEMIC\n",
       "lFdFhiKcK7tX60rS0M25tZZv79yIMrsDk2vsmwL0qOXchapsRH1zXQ0cBDwMvEIa8kqF2pOzeUAn\n",
       "A5cCy0oaDRxh+5PsNAF3SJpdUOaJti+zfUwd9Tb0fvsAnYHdgc2B2aQhtN9m27xml3EtkjoD1wJX\n",
       "2n4je/2JpJOA84GtgC2Bwh624cC/svvpmF17Z5H7qJoInEIIISygAgFPpSwNLCppQ9tPV6jMqUDP\n",
       "GvsWJwUT9Wns+2JS4PQIsArwj5pl2X4NOBRA0uqkHqnzgf0Kyhja0KE6Gn6/07OfI+cOkUo6j4WD\n",
       "n9rei5KuldSB9H7MAI4FsD0rmww+khRAPgPclJ0zd+j2fuBmYCNS4He5pHNsn1zH/VdcDNWFEEJo\n",
       "cbKnrTYkzQOqdU6RpHtrPGVWuN1TpOg3gE6SBhTs+x7wcgnNanSPk+33SJOkBwN1JkO1/TpwFQsO\n",
       "UzVWg+7X9hfAByWUu9B7Ucq1WQB0GSko3t32twXX/8/2INu9bQ8G+gNzA+bepGHAC23Psj2J1CO5\n",
       "YwltraiyAydJl0uaKOl/BfuGS/oge9RwgccNQwghhLpI2g/Y2vZIUg/DLrXNk7E9uMZTZoXbTrWV\n",
       "bftrUuDye0mLZkNku5B6QIq1p2NWfyego6RFGvkY/OHZfU0v3ClpdUknZPN2kLQC6Sm8J2o2paEV\n",
       "NuZ+gSuA4yQtnc23+gVwV9a2+t6LotdmLgHWAIbYLnxqEEnrSOqatfNE0tDfldnhz4AJwFFZG5YA\n",
       "DibN12patsvaSI8argf8r2BfDvhlPde53Lpjiy222NrbVonvzpb8/QtsDFxeY9+FwCEVrKMXcDtp\n",
       "GOsdYJ8ax0cDpxS8Hg7MqbGdVmJdb5OCpZr7OwHfAisCfYEbSb01U7OflwDda5QzjTTENne7NTv2\n",
       "F+CSCt5vJ1IKgy9Iwcr5QJdS3ot6rl0pO7/mfeybHf8/0hOAU4B7gFVrtHMj0nDnF8CnwA3A0lX8\n",
       "f7HWz0lF0hFIWhm4y/Y62escMNX2n+q4xm7hj8OGEFooqQv2zOZuRnNoL+kIQmhuzZGO4DhJLypl\n",
       "CV2iivWEENoLSUi/AaYh/R1p6eZuUgihfalW4HQJ6cmBdUlddUV7nkIIoSRSj0n0uv81Vj9+LV6e\n",
       "MYpdvjcHvYJ0DCnrcAghVF1VAifbnzhDypi6YW3nZZPI526DqtGWEELr91udset7rPDh7fx4iw0X\n",
       "uf+vr+x25k+Gdv/L2PV4Xu+w0vGGZ8nWw2prJA0q/K5s7vaE0N5Va47TcrYnZP/+BbCB7f1qXBNj\n",
       "7CGEOklsfiiXn/9/nLjeyIEDnv39Pi98QsdZW5ImuK7KrG4T9NpO3Q74b/fFLn73ei/mb0YLfkX2\n",
       "/dMWxRynEJpGsc9J2YGTpOtJC+31BiaSnqgbRBqmM+lJgJ+6xlpD8cENIdRGQsB2HZZ9+vTfdzlx\n",
       "7UM/f6zzHnvPmfHEitxHykI82jlPUl6dgXWxNuPLlXbv7qkb//bJqfrZC9/MGb2aR/98B876pDvP\n",
       "OedZzXtHlRWBUwhNo2qBU2PFBzeEUEhDjuzEVyv8ip4fDFu83/29rv/n+1r5qzmfHbsTv3hwVe5w\n",
       "bsGcLwtdv8zYzvT46NerL/LUSX/9/M/qO2tS52N25tsH+vMk8Fi2PeGcv2qSG6qSCJxCaBoROIUQ\n",
       "WhzltRgzFx3MpAHD6PnBpkxb+ptN3+7yn3/d/7+1us7mTsEJuGE9RhK9xJzf7MZtR/5NP/n2i55T\n",
       "xu6195znnuvLd4H1gTeZH0g9BrzrXDN9ETZCBE4hNI0InEIILYLyWg7YmTkdf4w7bMX7m8DbW73P\n",
       "Z2sMnz12/286MucvwInYV5VVj+jfg8l//A1nbn0853fowsyzz9vUI07ajjWBzUiLkG7GW3TidWYx\n",
       "h6lM41MmcZY/8l31FN9sInAKoWlE4BRCaBbKS8CawFBgCGZ1Pl7vHZ78+UqM3/4pvu7ze6OngdNJ\n",
       "C5rujv3fitUvtvgOr1w0kuP6bcITXy/K9J9ijwZQH+3IilzMzqw074J/YVbgJVZnFPOH92quLt9s\n",
       "InAKoWk0RwLMEEI7pbw6Ka9Byus8YDwwmm+6r8Q9I9/gjBmz+etzr/HiQVt5ap/BRq+TllfYGNig\n",
       "kkETgM0jr7Lmutvw7+P347quH7HcjZPV8wGkVenJsAWCJoBtEGOZTVo64lTgI+X1gvK6SHntp7xW\n",
       "yoLB0EpJOlbSs5JmSLqiluNLSrpd0lRJ70jat46ySj63yPXvSPpG0lI19j8vaY6kFbPXm0t6XNKX\n",
       "kj6X9Kik9QvKmFZjkeMRDWhDJd+P70h6MGvnOEm71jh+jaQJkiZLekspoW1J9ZTTzmxf4fszuyHv\n",
       "UaFIGhdCALLel54MowtdmckMJjPCE1PPTEnX59UD2J7Us7Qj6YnaUby+y5Fcf8cO0OEw0npZm9qM\n",
       "Sxfpu9m+O4CTsWdX+LYAsJkDXC3teuuDbP2rn3P+SSdzzsv9Pf3TN2tb8H42U53zqdl9dQG+Txre\n",
       "2wP4MzBLeT3K/HlSLzlXnba3V5KGAcva6b9DhX1I6uHcHuhWy/GLgBnAMqS1WO+R9KLtV8o8tzYG\n",
       "3iIt6nshpMVus3Y5e90TuBv4KXATsAhpndgZBWXsbPvBEuusqSLvh1Ii2juBi4EfkZ6wv0vSerbH\n",
       "ZaedBfzE9gxJqwP/kfSs7ftLqKfR7bTdvaCdiwEfk97LBouhuhBCCpr6cQFDGDBv5yjG8wHH1xU8\n",
       "Ka9+pJXWhwKbAo+TvjjvYrg7AycB+5BWYj/X5v35F2sfYCRwPPZ1lb+r4iT6rsXLf+7Yd+s9Xjry\n",
       "04V73q/hPo/z4FqvTb1N/Zk7Rypt/YCnSUHUo8CTznlKddrePobqJAkYB2xWM51NBes4Hehn+9CC\n",
       "fYuRFppdy/b4bN9VwEe2f13j+pLPraMNb5MSRQ+1vWG279ys3DOAlUmBwAO2e9VRxuFlBE5zyyn3\n",
       "/VgbeMJ2j4J99wNP2T6tlvpWB/4NDAFeb0A95bbzYOB3tgdQh2Kfk+hxCqEVKrd3aCE9GbZA0AQw\n",
       "hAFcw3GkldNTvSlo+C5z5yulpZVGA5cBeznnyRLfAc4k9Tr9FVjD5pP5jVcn4GxgN2Ab7Bcb3e5G\n",
       "svkI1t5bi296bLdbHjt3+h6fLTLv4CjeZDIji16bnsAbn21XAiivpYBNSMHUacB6ymsccO/cnqvQ\n",
       "MLYt6TrgQODc2s6RdDcpcK3NI7aH1FNNbcHjasDsub98My+Sek/KObcuTwIHSlqDFCzuTbqvM7Lj\n",
       "bwDfSroSuIEUiHxRo4yigbCkiwBsH1NPO8p9P2rTAVi7RnsuBg4m9Zwda/s5Ses1oJ5y23kw8I8S\n",
       "27+QCJxCaGWK9A71Vx/R6OCpC12L7O+WJZr8ISlQGgLMIfUqnQg8NjfBpMQPNJxTScHDBcAwltGm\n",
       "9OQfWisFeKtP4srX4AhgNrA+9qRGtbdC/NUdF2rpjm/pyr5nLE+ntVeb+WGnrSYtPv63MyY93KBy\n",
       "cv6cNJRyN4DyWoQ0vNev8q1uAlJlhiLK79W6EhhFkcDJ9s5lll/bfXYHaj4MMAXoUea59bkaOAh4\n",
       "GHiFNCyVGmlPVlpS6GTgUmBZSaOBI2x/Qgok7pBUOFx8ou3LsuvrC5jmVVXLvobc4+vAJ5JOAs4H\n",
       "tgK2BBboCbN9tKRjSN8rt0h6jhRElVpPo9spaaWsTYfSSBE4hdDalNg71CAz582VWFAPVgM+IX0h\n",
       "jgJ2Al4pzHsksSVpEvVapF9wB9pMqy3Am3EzP/rzdG7/xXT2xf62MU1NQzgrnwnvnOoKzDXwp9+O\n",
       "BkZLdFmG/57cn3NPncT9EyZpwAkDePNSGlFHlqzziXLb1mxazjDe0sCikja0/XQVyq/tPqcCPWvs\n",
       "W5z0S7icc+tiUuD0CKkX9x8122b7NbJf9tkQ1zWk4GS/7Pqh5Q7V1awzU/I92p6VTQYfSQryniHN\n",
       "I1ro+yX77I6RdDNpftdVpdZTZjsPJPVGvltLGSWJp+pCaG3q6B1qdJmTGcHdLPhF8k+msSS3AGs6\n",
       "542d85nOeaxztoQkBks8QhqmuxkYYHOBzTSg1gDv3T3p9Mvl6dHYoCnpvxvscDT0/3Hjy1iYzcz/\n",
       "+gen78f1yx/HyNHT6XbxW6wy/i2t+v1K1hNKI2kH0gLxZ1Ckd0DSvTWelCrc7imhmtqC4jeATpIK\n",
       "/9/9HvBymefW3RD7PdIk8cHAbfWc+zop0FirofXU14xa9jXoHm3/z/Yg271tDybNB6wr6O0MfN3A\n",
       "espp50Gk967RoscphNamWO/QTKY3pJhsvtL3gaEczRDeZglG8yGz+ZKpfMSXnF9z6E+iI2lu0qlA\n",
       "R9JcpltsFn6irMIBntT3SOh5PAzqBBf3hCPOktY4HSZfYH/0t8aUWRubSbD/3ktphzWO54Ibj+bi\n",
       "Z57RBvetxhv7L+6vvqxUPaE4SfsB69r+laQewO8l/cL2Av/vZ7+YG1N+R9Iv7E5AR0mLkObHfGv7\n",
       "a0m3ZXX+hPQZ2YU0h20BDTm3RIcDS9ienj2hNre9qwM7AzfY/lDSCqRemicLb6uRdVbs/cjKWoc0\n",
       "T6sDcDTQh7lzAaWlSU/b3UXqhdoG2BPYxva0+uopt52SNgX6kv7Qa7TocQqhtZnMCEYxfoF99Uxo\n",
       "nkt5LaK8dlBeFwPvA9eTHus9llVYyk+5n//rtf26tysMmiQ6SxwCjAVOIE2A/p7NDbUGTUDXb2r9\n",
       "q7DBAd58Ey6FWTno2DX9jujYFWaflvZX3ude6rXT/Pvv7c2Nu49nwEaz6PzJv7TNcKT43qwiSRuT\n",
       "fpH+CsD2FFK6in0qWM3vgGmk4aQDgOnAbwqOH036XHxCGhL7me1Xs/aNlnRKKec2lO23bD9XuCv7\n",
       "OYXU+/aUpKmkYeCXSJ/Fue6q0eN269wDki6RdEkdVVfy/TgQ+AiYSJrjtK3nL5tk4GfAB8DnpNQC\n",
       "B9p+pr56ym1n5iDgVttf1/Fe1CvSEYTQCmVP1R1HF7oxk+lMZmSxieHKa0nSE25DgW1Jwc+dwCjn\n",
       "/Fqd9YhuwGHAr0h/RZ4JPGQXCYrmX7jhrV24+6dr0PHz3Vhy3v5RvMkHDGvsJHZpwO6w7eXw+Qew\n",
       "5Arwr0Ps8XUOa1SCRMejuei0w7ns192Y/tVY1jp4D9/S+KcYy2pL+0hHEEJzK/Y5icAphDZIea3K\n",
       "/JQBPyA91TIKuNs5f1LXtQASPYGjgJ8DTwFn2TxVWuU6nJTk7ggtw6xSA7zSil7zFJj5Brx5e5rj\n",
       "1GWg/co5jS2voQZqXI+juOTa/bl25/+xzvPvstJuh/uyRk8ybYwInEJoGhE4hdCGKa8OwAakQGko\n",
       "0Js0j2AU8G/nPK2kckRvYBgpaPoncLbN/0prhLqQ0hBsBeyK6+7Nas2O0KVrDebeW7bgkdXuZfC1\n",
       "b7PK4TnnZ9V/ZfkicAqhaUTgFEIbo7y6AVuTAqVdSFlzR5GG4Z52znNKLkv0Jc2XOBS4BTjH5s3S\n",
       "G6O5Ey4/AQ7GLWdR3Go6U6fuO5h7/9KJ2Z3vYNeTf+fT651nVq4InEJoGhE4hdAGKK/epKdrhpCe\n",
       "TnmBbIkT5+atBVV6eWJV0kTLPUlPvpxn80EDC9kMuBH4C3AmLj1gawuGa7j68+aft+WBY59mw49u\n",
       "Y7d9rvQhj1ervgicQmgaETiF0IzKSdqovFZj/hDcd4EHSD1L92QZqxvRHtYGTgF2AC4BRth82sBC\n",
       "RFp09PfAIbiMJV/agNu0W+8eTLl5PZ7/4bXs/8ilHLHXy1674musReAUQtOIwCmEZjT/abAHDq3v\n",
       "KTDl1RHYiPmTuxdn/hDcQ8659jxOJbWDDUg5mDYhZR2+2F5omYJSCupKWol8I+DHuOG9XW3VvRq8\n",
       "6Qq8f6Pwcpdw1F8v4thj630KsQEicAqhaUTgFEIzmJ+0cfNOcOlqcMQb8OjsmkkbldeipFQBQ0hD\n",
       "cRPJUgYA/23IfKWF24BIa0L9Blgd+CNw2bwM3w0vcAXgVuBd4FDsqY1tW5sl6WG2OK4HUw5djxe+\n",
       "35hlW+ooOgKnEJpAsc9JZA4PoaomXArdJkHHPxUkbfwlTLhNefUhBUlDSSt4P0MKlM5wzm+XW3MW\n",
       "MO1E6mFaCjgbuNZmZhmF/pCUNPN84I+VDAjaFNtbwgjSFkJoQyJwCqGKbFsaYOiwBOz5Ciy5Ej+c\n",
       "tCtbcQLwHeB+4AbgYOf8RSXqzJZF2YMUMJmUtPJWm8avD5fmMx1H6rU6EPufFWhqCPWSdCXwvu3f\n",
       "NXdbQoAInEJoAl0GssVNt7L1pC14ebHpvN2jP5AD/uOcG9/7U4NEF9JyBycDnwK/Bu4te36N1A34\n",
       "K2li+ibYb5XZ1BAawtS+qOtCJI0BrrZ9WWMqqlSQJmll0oK9ndzOnjJtDyJwCqHahr86DdgU2Jt1\n",
       "vn7et0yt6PCWxKLAT4ATgVeBI4CHKzIhOf0CuC0rd1NcWiLN0HqV8wRoJcuoWWSJ57W0oeOYR9YG\n",
       "xWKVIVSR8tqDtM7bDs75OecqOUmYxSV+TfrLdhCwu832Nv+pUND0I9Lq61cDB0TQ1F703w12ODot\n",
       "adP0ZUhaT9JzkiZLugHoWnCsl6S7JX0iaZKkuyQtnx37A7AFcGG2yO2IbP8Fkt6T9JWkZyVtXqTe\n",
       "I4H9gF9l19+Z7e8r6daszrckHVdwzYZZmV9J+ljSudmhh7OfX2ZlbdTQ9yG0YLabZUtVN0/dscXW\n",
       "FBvD2ZLhfMJw1q1ouXhp8Bngz8BXg9eqaNtBhhMNEwxbNff7GFvN//64GmXAckfC6mPh8NdhjtPP\n",
       "1cfCckeWXm55ZQBdSE9rHg90BHYHZgK/z44vCfyYFEx1B24Cbi+4/iHgsBpl7g/0InUU/BKYAHQp\n",
       "Uv8Vc+vKXncA/gv8ljRCswrwJrBddvwJYP/s34sCG2X/XgmYA3Ro7v9fYmv8VuyzFj1OIVSB8lqL\n",
       "tATJfs75hYqUKfpJnA+8TlqLbkObA23GVqL8rJLFSE/N7QNshP1QxcoOLdyES2FWLj35Oe8J0NPS\n",
       "/iYrY2PSvKALbH9r+1bS06YA2J5k+3bbM5zSYJxJSrVRaIHhMdvX2v7C9hzb5wGLkNJyFFN4/QZA\n",
       "b9tn2J5t+23g76TPB6SgbqCk3ran2X6qljJCGxOBUwgVprz6AaOBE5zzv8ouTwyQuBR4CZgNrG3z\n",
       "M5vKTtKW+pP+gp4BbIH9XkXLDy2a7ayzscMSsNcroF7Z7pKHfStQRl/gwxr73iULRCQtKumvkt6R\n",
       "9BXwH2DxNKdqfjMKL5Z0oqRXJH0p6QtSQtneJbZnJaCvpC/mbqSHLpbJjh8OrAa8KulpSTuVWG5o\n",
       "xWJyeAgVpLyWAO4FLnLO15RVlliH9CW9LXAxMNCmUUuslFDZDsBVpOVTLqYBvyxDW9JlIDxwKLx5\n",
       "e5qf1GVgE5cxAVi+xr6VgPHZv08gBSob2v5E0rrAc6TAaqGn7yRtAZwEbG17bLZvEsV7hGr+f/8e\n",
       "8Lbt1Wo92R5PmheFpN2BWyQtWUs5oQ0pO3CSdDkpyd4nttfJ9i1JWvRzJeAdYC/bX5ZbVwgtmfJa\n",
       "BLidNM/ij40uR2xMysG0AXAe8FObKRVp5MKViRScHQvsgf1IVeoJrYL9ytkFL+tcGqhKZTwOzJY0\n",
       "jLSG4i5NhsJkAAAgAElEQVSkz8G/s+PdgenAV9nvmVyN6ycC/Qte9yD10n4mqQtpfcaeddQ/EVi1\n",
       "4PXTwBRJvwJGkobmvgN0tf2spAOA+21/CnxFCpjmkNKBzMnaEssRtTGVGKq7grRQaKFTgAeyKP3f\n",
       "2esQ2izl1YHUY/M58IuGPj0nIYkfSfyblBDzPmBVmz9WMWjqAdxCWuZlgwiaQnOzPQvYDTiE9Fna\n",
       "i7S8z1znA92Az0hB1r0s2LtzAbBH9sTd+aTP0X3AG6Q/4qeTepGKuQxYMxuWu80pB9POwLqkp1c/\n",
       "Bf7G/OBre+BlSVOAPwP72P7G6QnUPwCPZWVt2Ii3I7RQFVmrLkv2dVdBj9NrwA9tT5S0LDDG9ho1\n",
       "rrFjraTQRiivc4ENge0asgivRAfSF/OpwBLAWcB1NrOq0tD5Fa8G3AE8ChyH/U1V6wsVE2vVhdA0\n",
       "mnqtuj62J2b/ngj0qVI9ITQ75fULYDCwealBk0QnYE9SwDSL9NfpHS5nWZRSSTsDlwO/xf5bfaeH\n",
       "EEKYr+qTw21bUq3dWpKGF7wcY3tMtdsTQiUpr71JuWE2K2WtOYlFgINIy6JMIE1cvd9ugsmkUgfg\n",
       "d6TM4kOxn6h6naFskgaREpyGEFqAag7VDbL9saTlgIdiqC60NcprECkB3zbO+aU6zxWLkQKWE4H/\n",
       "AWfaNN2cImlxUgbwJUmTwD9usrpDRcVQXQhNo9jnpFp5nEYBB2f/Ppg0lyKENkN5rUN6cnTvEoKm\n",
       "pUnZh7cAhtoMbuKgaU3S00HvAVtH0BRCCI1Xdo+TpOtJmVt7k+YznQbcSfpLfEWKpCOIv3hCa6W8\n",
       "VgAeA37lnG+o81zRg5Se4F6bslZcbxRpN+CvwEnYVzZ5/aHioscphKZR7HNSkaG6xogPbmiNlFcv\n",
       "4BHgCuf8pzrPFV2Be0h5XI5qknlM8yvvSEpmeQCwO/azTVZ3qKoInEJoGk39VF0IbY7y6koadn6A\n",
       "lJiy+LmiI3AtMAk4pomDpl7AdaR8Nxtgf9JkdYcQQhsXa9WFUEB9tKMG6j6tpTEaqPvURzvCvASX\n",
       "VwMfk9agKxoISYiU9bgncECTpBiYX/l3gWeB14BtI2gKIYTKih6nEDLqox3pxwUMYcC8naPorz6C\n",
       "o9keWBrYwTnPqaeoM0iZhn9k03SJJaW9gQuBn2Nf22T1hlBFkq4E3rfd9HMEG0HS/sBBtrev5Lmh\n",
       "5Yg5TiFkNFD3cQALf4GN4nWGMJuU4LLONRclfg78DNjc5rPqtHShSjuRMo7vAfwY+4UmqTc0i/Y2\n",
       "x0nSFaTA6bQSzh0DXG37skbWdSWtKEgL1RVznEKoTxe61rq/OysDA0oImg4kJcNsyqCpN2ltOwPr\n",
       "Y3/eJPWGNkl9tCM9GUYXujKTGUxmhCd6dFOXUVuxJZ5X1Z4ASZ1sz65mHaHlizlOIcw1k9qXS/mM\n",
       "Z5zzB3VdKrET8Edge7vORUQrR1oPeIaUI2pwBE2hHPOGqg9ge/bihxzA9vTjgrnz/JqsDGk9Sc9J\n",
       "mizpBpj/B42kXpLulvRJtpDvXZKWz479gZQr7UJJUySNyPZfIOk9SV9JelbS5kXqPRLYD/hVdv2d\n",
       "2f53JP1K0kvAFEkdJZ0iaXzWxrGSdi0o5xBJjxS8niPpp5LeyBb8vbCOc7eT9LqkLyVdJOk/kg4v\n",
       "9b0LTSMCpxDmmswIRjF+gX338CGfcVZdl0lsDlxJSm75avUauEClBwL/BE7GPpn4KziUqyfDFpjf\n",
       "BzCEAfTkuKYqQ1IX0pOrVwG9gJuB3ZnfkyTgMlKOwBWB6aR5fdj+DSlVyDG2e9gell3zNPC9rLzr\n",
       "gJuzehbgtG7jtcA52fVDCw7vQ1qPcgnb3wLjgc1t9wTywDWS6lqTdSdgfeC7wF6SFpoSoNR7fDNp\n",
       "OaYlgdeBTahyL1pouAicQsh4okfzAcdzA//hQWZyOy/wLkfWNcwg8V3gNmB/m6eq3kipM9L5QI6U\n",
       "BfymqtcZ2odiQ9Vd6NaEZWwMdLJ9ge1vbd9K6lUFwPYk27fbnmF7KnAmKQFzoQWG9Wxfa/sL23Ns\n",
       "nwcsAqxeRxtqDgsaGGH7Q9vfZGXe4iwDv9NncBywUR1lnm17su33SQlx163lnB2Bl23fkbV1BOkp\n",
       "3tDCxBynEAodzZPAMqSs4BfUdarEqsC9wHE2/6x626RlSBn5p5HyM9W7qHAIJSs2VD2T6U1YRl/g\n",
       "wxr73iULZiQtCvwZ2J7UgwTQXdks3uz1Aj00kk4EDsvKNilNSO8S2zPX+zXKPAj4BbDy3DYAS9Vx\n",
       "fWEANA1YrJZz+gI1pwTUOUUgNI/ocQoho7y6kdZZvKeEoGlZ0lDZH2xurH7jtAEpP9OjwC4RNIWK\n",
       "q22oehRvMpmRTVjGBGD5GvtWYn4wdAKwGrCh7cVJvU1ifi9RzaBpC+AkYE/bS9juBXzFwr1KcxUb\n",
       "Fpu3X9JKwN+AY4AlszJfrqPMUn0E9CuoR4WvQ8sRPU4hAMqrI3ANaSHck+s8VywO3AdcbXNx9Run\n",
       "w4BzgCOxb696faFd8kSPVh/BNRxHF7oxk+lMZmRDnoirQBmPA7MlDSMlkd0F2AD4d3a8O2le01eS\n",
       "liQNWReaCPQveN0DmA18ls1rOoXU41TMRGDVetq4GCmQ+gzokPU+rV3PNYUKA71Co0kT24eSlmr6\n",
       "GbBsA8oNTSQCp9DuKS8BFwBLADvWleBSYm6v1COkteCq2DB1Ac4Htga2xG6aieeh3coCnLJSB5RT\n",
       "hu1ZSgtTX0pKJDsauLXglPNJE7w/Iw3pnQcMKTh+AXCVpKOAf5DSg9wHvAF8TRrmq+up18tIk8e/\n",
       "AB6yvVstbXxF0p+AJ4A5WT2PFp7Cgj1XNXuxCo/P+7ftzyTtCYwgTY6/ltTL3HRJdENJIgFmaPeU\n",
       "18mkx5C3dM5fFT1PdCJ9iX9NWkqlvgziZTRKywG3kH5BHISLtyu0L+0tAWZ7JakDaW7Vfrb/09zt\n",
       "aY+KfU5ijlNo15TXgcBRwOB6giaR5jUsAhxS5aBpE9KTRPeTMoFH0BRCO5DlcVpC0iLAqdnuJ5uz\n",
       "TWFhMVQX2i3ltS1wLrCVc/6ontPPAb4DbGMzszoNkoAjgdOBw7Dvrko9IYSWahPSUGQXYCyw69wU\n",
       "CKHliKG60C4pr/VIPTq7O+dH6jxXnAQcAmxhM6k6DVJXUiK/TUi9TG9UpZ7Q6sVQXQhNI9aqCyGj\n",
       "vFYB7gaOKiFoOpT02PHmVQya+pHmTr0PbIw9pSr1hBBCKFvMcQrtivLqTXrK5iznfGud54qhpMzE\n",
       "29lVSkQnbUlaEuJ2YM8ImkIIoWWLHqfQbiivRUmpBG53zhfWea74IemR6ME2lR82S/OZjgV+CxyM\n",
       "fV/F6wghhFBxETiFdkF5dQKuB95k/tMqtZ8r1iMttrmPzX8r3xh1A/5CWq9qE+y3Kl5HCPWQFIvH\n",
       "htAIETiFNi9LcHkhsCiwZz0JLgeQsvYeZfNg5RujlUiLAr8BbIr9dcXrCKEeMTE8hMaLOU6hPTiV\n",
       "tHL57s65aCoBieVI68/lbOqc/9Qo0tbAU6SMwPtF0BRCCK1P9DiFNk15HQL8BNjUOU8uep7oRUpP\n",
       "8HebSyvbCAn4OWkNvP2x/13PFSGEEFqoCJxCm6W8dgDOBgY55wlFzxOLAneRFhI9q7KNUBfSYqXr\n",
       "k1INvFPR8kMIITSpCJxCm6S8fkBafHNX5/xa0fNEZ+Am4G3gBHuhBTnLaIR6k/IzfQlshj21YmWH\n",
       "EEJoFjHHKbQ5ymtVUg/Skc758aLniQ7A5dnLwyq6/py0Jmk+0xOkTOARNIUQQhsQPU6hTVFeS5MS\n",
       "XJ7unO8oel5atPdPwCqkBJezKtcI7UDq7ToJ+6qKlRtCCKHZReAU2gzltRhpKZWbnfMl9Zz+a+BH\n",
       "wA9tplWmARIwDDiF1Mv0WEXKDSGE0GJUNXCS9A4wGfgWmGV7w2rWF9qvLMHlDcBrpGzcxc8VR5Ke\n",
       "tNvc5ovKNECdSbmiNiUltXynIuWGEEJoUard42RgkO3qLI4aAvMSXF4MdAZ+4pyLTvCW2B3IkXqa\n",
       "PqpMA7QkcAswjTQJvGjagxBCCK1bU0wOL5qhVgN1n/poxyZoQ2jbfgf8gJQVvOhcJYmtSakBdrIZ\n",
       "X5GapdVJk8CfA4ZG0BRCCG1btQMnA/+S9KykIxY6egDb048LIngKjaW8jgAOAXZyzlOKnifWJw3l\n",
       "7WnzQmUq17bAw8DZ2Cdif1uRckMIIbRY1R6q28z2BElLAw9Ies32I/OOPgT0YADfcIGkabbHVLk9\n",
       "oQ1RXodgcvy5391M/mAiuSLnidVJ6QmOsPlPZSrX0cBpwF7YlSkzhFpIGgQMauZmhBAyVQ2c7JSt\n",
       "2fankm4HNgTmB05bZT8/5UN/GUFTKJ3yOhD4A1f0PZvJO58BD/yTtHjugueJfqSlVE61ubP8itUJ\n",
       "OB/YmjSf6c2yywyhDtkflGPmvpZU5E+EEEJTqNpQnaRFJfXI/r0YsB3wv1pPnsn0arUjtD3Kaz+e\n",
       "WnQEp682nfcGHwsX94RBZ0lrjJX6HjnvPLEUKWi6yOaK8itWL+BeYADpybkImkIIoZ2p5hynPsAj\n",
       "kl4gTZ692/Y/FzprFG8ymZFVbEdoQ5TX3sCfWH/a5nw7+xTo2DU9f9CxK8w+DSZcCiAxN6fTPTZ/\n",
       "LL9iDSRlAR8L7Iz9VdllhhBCaHWqFjjZftv2utm2tu2FF0+9hvv4gGGe6NHVakdoO5TXnqRhsu38\n",
       "e48FGTosAXu9AuoF2LYlupDWiHsVOLn8irUV8CjwZ+yfY88uu8wQQgitUrNmDvc4D27O+kProbx2\n",
       "A0YC2zvnbMi3y0B44FB483bo/2PoMjBbf+5KYAZwZNmL9kpHAqcD+2I/WFZZIYQQWj25eK7A6lYs\n",
       "2XbRHE8hzKW8hgJ/A3Zwzs8XPS+tPzcC+C6wg13G3Lk0CfxcYDBpaG5co8sKoYLiuzOE5hVr1YUW\n",
       "TXntQgqadqwraMr8DtgcGFRm0LQ4KedTJ2Bj7MosyxJCCKHVa4rM4SE0ivLaEbgM2Nk5/7fOc8VR\n",
       "wEGknqbGT9yW+pMmgb8JDI6gKYQQQqEInEKLpLy2J81V2sU5P1PnuWIv4DfAdjYTG1+ptgQeAy7C\n",
       "PjYmgYcQQqgphupCi6O8tgWuBnZ1zk/Vea7YmTRpfFubtxpfqQ4Dzgb2x36g0eWEEEJo0yJwCi2K\n",
       "8toauBbYzTk/XvQ80Zn0tNsBwFCblxpXoTqSAqZdgS2xX2tUOSGEENqFCJxCi6G8BgE3Ars750eL\n",
       "nidWAa4HPgfWs/m0cRWqB3AdsBiwEfakRpUTQgih3Yg5TqFFUF5bAjcDeznnh4ueJ/YmZaK/Edi5\n",
       "jKBpZeBx4CNg+wiaQgghlCJ6nEKzU16bAbcA+zjnh2o9RywKXEBaJX6wTZ1P2dVd4bz6zgJG0lzJ\n",
       "zEIIIbQ60eMUmpXy2gS4HTjAOf+71nPEOsAzQFfg+2UGTQdl9R2KPSKCphBCCA0RPU6h2SivjYA7\n",
       "gYOcW3gB6CwT+M+A3wMn2Pyj8ZWpA/AHYC9gEPYrjS4rhBBCuxWBU2gWymsD4C7gUOd830LHRS/g\n",
       "78AqwGY2bzS+MnUnpTdYkjQJ/LNGlxVCCKFdi6G60OSU1/eBu4HDnfM9Cx0XmwHPA+8Dm5QZNK0I\n",
       "PApMAraNoCmEEEI5InAKTUp5rQvcC/zUOd+1wDHRUeK3wK3AsTY/t/mm8ZVpY9LyKf8AfoI9s/Et\n",
       "DyGEEGKoLjQh5fVd4D7gaOd8xwLHRF/gGlIw/wObD8urTPsB5wOHYd9dVlkhhBBCJnqcQpNQXmsD\n",
       "9wPDnPOtCxwTOwHPAQ8BPyoraJI6IJ1Omgj+owiaQgghVFL0OIWqU15rAv8EfuGcb5q3XyxCyqW0\n",
       "O7CnzSPlVaTFgKuAZUmTwD8pq7wQQgihhuhxClWlvNYAHgBOcs43zNsvBpIyd69CWjal3KBpeeBh\n",
       "4GtST1METSGEECouAqdQNcprdeBfwK+d87Xz9osDSEHT5cBuNuUtdyKtz/xlWA7BbvyE8hBCCKEO\n",
       "MVQXqkJ5DSQFTb9zzv8AkOgOXARsCGxj82LjCldPYGC2rQX8FDgC+84KND2EEEIoKgKnUHHKqz/w\n",
       "byDvnK8AkPg+cANpOG19m6/rLkSLAQOA1ZgfJM3dFgPGA+OybWvsl6tyMyGEEEIBNddSXZJsW81S\n",
       "eaga5bUKMAY40zn/NVs2ZRjwG2CYzQ3zT1Y3oD8LB0YDSVm+32R+cDQOeCP7OSHWmAvtVXx3htC8\n",
       "InAKFaO8VialFPijc75YondXpl+5Bq+tOIJhI7bg0cIhtoFAH+AdFg6MxgEfYM9p+rsIoWWL784Q\n",
       "mlcETqF8UqeLNmCTf63KTbu9ymMHvsSEiSyz0Qy6fn95PqQj376jhQOjccB72LObte0htDLx3RlC\n",
       "84rAqS7pEfczgeuBf2PPauYWNR+pI7AC83uL5s09Mqz4QU86zujEm6tO6vDg5RzWfzQ7rr8kk066\n",
       "jJ9c3a7ftxAqrFV8d4bQhkXgVBdpCeBgYB/SXJxbSUHUo21yGEnqAPSlRmCUbasAn7Fgj9G469bm\n",
       "q58M5e/TO3MJw30TcB0wAzjQ5uNmuIsQ2rRW8d0ZQhsWgVOppFVIAdQ+wFKknEHXA/9tVROVJZEy\n",
       "a9c2Ibs/MJkawVG2jceetkBReS1Hmgh+OcP9OvBX4DzgjzZtL7AMoQVodd+dIbQxVQucJO1AWmS1\n",
       "I/B32+fUON56P/zSmsC+2WbSY/bXY7/SrO2aKwVHvak9OBpI6hGqGRi9QQqOppRURV7LAg8xq+v1\n",
       "/GF6H2AwsJ/NkxW+mxBCgVb93RlCG1CVwElpPszrwDbAh8AzwL62Xy04p/V/+FOA8gNSALU38Dmp\n",
       "F+pG7LeboP5eLBwUzR1iM7U/yj8O+8uyqs1rGeAhPh/wL0aOG0T6b32kTdFy1Uc70pNhdKErM5nB\n",
       "ZEZ4okeX044Q2qM28d0ZQitWrcBpEyBne4fs9SkAts8uOKdtffjT/KAtSEN5e5ASNF4P3Iw9oYxy\n",
       "e1B7YDQQWITan1YbB3xejSFE5bU05kHe3O5trrl/E+BU4O82RetSH+1IPy5gCAPm7RzFeD7g+Aie\n",
       "QmiYNvfdGUIrU63AaQ9ge9tHZK8PADayfVzBOW33wy91JvW27QMMAZ4jBVG3Yn9Ry/mLUjxLdg8W\n",
       "zJJduE1syvlVyqs3czo+xIsHijsvM3TY26be4UkN1H0cwPYLHbiG+zzOg6vS2BDaqDb93RlCK1Ct\n",
       "JVdK+mUuaXjByzG2x1SlNU0tPX5/L3Bvlh17R1IQ9Sekh4EngJWZHxwtBbzF/IDoSeDq7N8ftYQn\n",
       "+JTXknzT/TGeP3wZ7jvvBujwS5vpJV3cha5F9nerZBtDaIskDQIGNXMzQgiZagVOH5Jy/sy1AvBB\n",
       "zZNsD69S/S2HPZ2UxuDWbHHaocD3gBeAm0nB0fvY3zZfI+umE/suifu+xCt79OL+cw+yO9zaoAJm\n",
       "MqPI/tICrxDasewPyjFzX0vKNVtjQghVC5yeBQZKWhn4iDRxet8q1dV62JNJPUlXN3dTSqWNRg5k\n",
       "g8Wf5YNNpvKvc9b0nE7vNriQyYxgFP1rzHF6k8mMrGRbQwghhGqrSuBke7akY4H7SekILit8oi60\n",
       "DlruhaEMufJmvlz5BZ4+elPP6tqo5VE80aPVR3ANx9GFbsxkOpMZGRPDQwghtDaRADMsRKIzi318\n",
       "Dvvtcgxdpo5m6dd2c64VJfkMoQ2L784Qmle1huqaVOQIqhyJVej6xY0ctN0AlnjnOhaZclgETSGE\n",
       "EELS6gOnIjmC+quPiOCpYST2ovPUC/nZelNY/L1bkX8aQVMIIYQwX6sfqoscQeWTWBQ4n85Tt+aX\n",
       "K3xBty9fAo5wrvnTIIQQFhRDdSE0rw7N3YCyRY6gskisAzzDIl9255ReH9Dty7FE0BRCCCHUqvUH\n",
       "TpEjqFEkJHEU8CCLfXwev+7Vm46z3wcOj6AphBBCqF3rD5xSjqDxC+yLHEF1kugF3AIcwTIvbcVJ\n",
       "y+0JfAYc4lzLTcQZQgghNLdWP8cJ5j1VFzmCSiCxGXAtcAeDTvsdg06/AZgK7O+cG5WnKYTQdGKO\n",
       "UwjNq00ETqF+Eh2BXwPHAkcwXPeTep1mAvs651nN2b4QQmniuzOE5tXq0xGE+kn0Ba4hDc3+gOH6\n",
       "FLgJmE0ETSGEEELJWv8cp1AniZ2A50iLhP6I4foEuB4QsE8ETSGEEELposepjZJYBDgL2B3Y0+YR\n",
       "5dWJNL9pEWB35zyzOdsYQgghtDYROLVBEgOBG4D3gPVsJmVB09VAd+DHzvmb5mxjCCGE0BrFUF0b\n",
       "I3EA8DhwObBbFjR1BK4ClgR2i6AphBBCaJzocWojJLoDFwEbAdvYvAiQBU1XAH2AXZxz7QlDQwgh\n",
       "hFCv6HFqAyS+T5oAPhv4QUHQ1AH4O9APGOKcI5t6CCGEUIYInFqxbNmU44H7gZzN4TZfw7yg6W/A\n",
       "KqSepmnN2NQQQgihTYihulZKojdpHtOywMY2b847loKmS4DVgcHO+evmaWUIIYTQtkSPUyskMQh4\n",
       "HngN2LxG0CTgQmBtYEfnPLVZGhlCCCG0QdHj1IpIdAJOA34CHGpz/wLHU9A0AlgP2N45T2n6VoYQ\n",
       "QghtVwROrYTECsB1wAzg+zYfL3A8BU1/Jj1Vt61zntz0rQwhhBDathiqawUkdgWeBe4Bti8SNP0R\n",
       "2BzYzjl/1fStDCGEENq+6HFqwSS6An8CdgSG2jy50DkpaDob2BrYxjl/2bStDCGEENqP6HGqg8RA\n",
       "iYcljpBYoonr/g7wFLA0admUYkHTH4DtScNzk5qyjSGEEEJ7E4FT3d4l9fhsD7wrcYvEUIku1aow\n",
       "y810GPAw6em4vW2K9SLlgZ1JPU2fV6wNfbSjBuo+raUxGqj71Ec7VqrsEEIIoTWT7eapWLJtNUvl\n",
       "jSDRC9gDOBD4DnAzadHcJ20q8iZKLA78hZRKYG+bV4qem9dpwF7AVs7500rUDylooh8XMIQB83aO\n",
       "YjwfcLwnenSl6gkhNE5r++4Moa2JHqcS2Xxhc6nNlsCGwIekNeDGSQyXCgKNRpDYkLRsypfAhvUE\n",
       "Tb8B9gF+VMmgCYCeDFsgaAIYwgB6clxF6wkhhBBaoQicGsHmbZs/kHqe9gV6AY9JPCFxtMRSpZYl\n",
       "0UHiJOBu4Fc2R9kUXVNOeZ1C6vXa2jlPLO9OatGFrkX2d6t4XSGEEEIrE4FTGWxs84zN8aSFdE8H\n",
       "tgDekrhTYo/sybhaSfQBRgO7AhvY3FpXfcrrJOAwUtD0cV3nNtpMZhTZHwsEhxBCaPeqEjhJGi7p\n",
       "A0nPZ9sO1ainJbGZZTPaZl9gBeB24CjgI4lLJbaU5r/fEtuSlk15Fvihzbt1la+8fgEcSZrT9FHV\n",
       "bmQyIxjF+AX2jeJNJjOyanWGEEIIrURVJodLygFTbJ9XxzntYoKjRD9gP9LwWg/gWqALaYjvIJsH\n",
       "6y0jr+OBYcAg5/x+FZub6uujHenJcXShGzOZzmRGxsTwEFqG9vLdGUJLVc3AaartP9VxTrv68EsI\n",
       "+C4pgFoaONGm3ondyusY4ARS0PRedVsZQmjp2tt3ZwgtTTUDp0OBr0hDUSfYC2a0jg9//ZTXz4BT\n",
       "SEHTO83cnBBCCxDfnSE0r0bPcZL0gKT/1bINAS4BVgHWBSaQkki2OpIkrXKWpCb/klJeRwCnkiaC\n",
       "v9PU9YcQQghhYY1eq872tqWcJ+nvwF1Fjg0veDnG9pjGtqc6+u8G2x4NDzwD3NZUtSqvw4Df/X97\n",
       "9x9rdV3Hcfz54tLdBY1s6Ux+NJzCFm6JVs4ltsMIJIYyWkU5ptmGbVYytZbKH3d3rTCbEdL8A7K2\n",
       "5tBaqQNNFBWX5kZZYCTQBYoFebUaJbkkIN798f0uL/fec+/33nPP+XzPua/Hdsf5nh/f72uXne/e\n",
       "9/P5fN9fsqLpj406rpmVj6QKUEkcw8xy9ZqqOy8ievLHtwAfjohr+7yntMPN0uQbYdJKmDMeNsyE\n",
       "Fd3wwkk4ujbi1fV1PXaXrie7/9zc6Ix99TyWmTWfMp87zcaCEY84DeFbkmYDAfwJ+EKdjlMnPRtg\n",
       "whFouwcEtHXAyVuhp66jTurScuCbZCNNLprMzMxKpi6FU0RcV4/9NkpEhHRhgN7D2QtP8caUKVx+\n",
       "ZDHz2AV01+OY6tK1wN1kt1H5Qz2OYWZmZrXxTX6rkGbdDse7+cqBLTx9zira2uZx9WvTgDfJ1mxt\n",
       "Bn4ZnXGy5mN1aRmwBpgfnfFKrfszs9ZV9nOnWatz4TQM6pKAS4Gr85/pwBayImpLdJ7ecqHgPj8J\n",
       "rAMWRGfsGr20ZtaKmvHcadZKXDjVQF2aAiwmK6I+StazajOwOTpj/2CfzT+/lKx1w1XRGS/XM6uZ\n",
       "tYZWOHeaNTMXTqNEXZoIfIysiFoM/BN4jKyQerHvlJ66tARYDyyMztjR4Lhm1qRa7dxp1mxcONWB\n",
       "ujQO+CBvj0a9j15TesCVwP3AouiM36TKaWbNp5XPnWbNwIVTA6hLU3m7iLoS+A9Z0fTrmved3ZD3\n",
       "Ztrp4DjHOMq9viGvWesaS+dOszJy4dRg6tIZwMTojCFv8Dvkvs7VIqaylmu48P9PbmI/h1np4sms\n",
       "NY3Vc6dZWbhwamKaoS0s56p+LzzAltgXH08QyczqzOdOs7RGfJNfK4F2Oqo8P6HBSczMzMYEF04N\n",
       "JknS+asl1f4X43GOVXn+rZr3bWZmZv24cGq4Cz4BC2+CC5bWvKuj3MsmTu8XtYkDHGVdzfs2MzOz\n",
       "frzGqUGkyTfCpJUwZzxsmAkruuGFk3B0bcSr60e83+yqui/TzgSO8xZHWeeF4Wata6ydO83Kpi43\n",
       "+bWB9GyACUeg7R4Q0NYBJ2+Fnodr2WteJLlQMjMzawBP1TVIRAQoYNxZ8OndoHfnT6cZ8jMzM7Nh\n",
       "84hTQ7XPgK03wIFHsjVO7TNSJzIzM7PivMbJzKyJ+Nxplpan6szMzMwKcuFkZmZmVpALJzMzM7OC\n",
       "XBOEmuAAAATaSURBVDiZmZmZFeTCyczMzKwgF05mZmZmBblwMjMzMyvIhZOZmZlZQS6czMzMzApy\n",
       "4WRmZmZWkAsnMzMzs4JcOJmZmZkVNOLCSdKnJL0i6b+SLu3z2h2S9knaK2lB7THNzMzM0qtlxGkX\n",
       "sBT4Re8nJc0ClgGzgIXAfZKadmRLUiV1hiKcc3Q55+hqhpzNkNHM0htxQRMReyOie4CXlgAPRsSJ\n",
       "iDgI7AcuG+lxSqCSOkBBldQBCqqkDlBQJXWAgiqpAxRUSR2ggErqAGZWfvUYCZoMHO61fRiYUofj\n",
       "mJmZmTXU+MFelLQVeO8AL90ZEZuHcZwYViozMzOzElJEbTWNpG3AbRHx23z7doCIuCvf3gJ0RsT2\n",
       "Pp9zMWVmNgIRodQZzMaqQUechqH3l3gTsFHSd8im6GYAv+r7AX/xzczMrNnU0o5gqaRDwOXA45Ke\n",
       "AIiI3cBPgN3AE8BNUeuwlpmZmVkJ1DxVZ2ZmZjZWJO2vJOnrkl6WtFPSM5KmpcxTjaRvS9qTZ31Y\n",
       "0rtSZxrIYE1Jy0DSwrwp6j5JX0udZyCSfiDpdUm7UmepRtI0Sdvy/+vfS7o5daaBSOqQtD3/fu+W\n",
       "tDp1psFIapO0Q9JwLnxpKEkHJf0uz9lvCYSZ1V/qxpR3R8TFETEbeBToTJynmqeAiyLiYqAbuCNx\n",
       "nmoGbEpaBpLagO+RNUWdBXxW0vvTphrQD8kyltkJ4JaIuIhsqvyLZfxdRsQxYG7+/f4AMFfSnMSx\n",
       "BrOSbIlBmYfhA6hExCUR0cz98cyaVtLCKSL+1WvzTODvqbIMJiK2RsSpfHM7MDVlnmoGaUpaBpcB\n",
       "+yPiYEScAB4ia5ZaKhHxPPCP1DkGExGvRcTO/PGbwB6y/mmlExH/zh+2A23AkYRxqpI0FVgEfJ/T\n",
       "L3Ypo7LnM2tpqUeckPQNSX8GrgfuSp2ngM8DP08doglNAQ712nZj1FEgaTpwCVlBXzqSxknaCbwO\n",
       "bMsvHimjNcBXgVNDvTGxAJ6W9JKkFanDmI1Fo9WOoKqhmmhGxCpgVd7/aQ1wQ70zDaRIs09Jq4Dj\n",
       "EbGxoeF6GcWmpI1W5umPpiTpTOCnwMp85Kl08pHa2fm6wCclVSLiucSxTiNpMfDXiNjRBPeruyIi\n",
       "eiSdA2yVtDcfJTWzBql74RQR8wu+dSMJR3KGyinpc2RD+fMaEqiKYfw+y+YvQO/F/9M4/dY8NgyS\n",
       "3gH8DHggIh5NnWcoEfGGpMeBDwHPJY7T10eAayQtAjqASZJ+FBHXJc7VT0T05P/+TdIjZFPgLpzM\n",
       "Gij1VXUzem0uAXakyjIYSQvJhvGX5Atem0HZ1kG8BMyQNF1SO7CMrFmqDZMkAfcDuyPiu6nzVCPp\n",
       "bEln5Y8nAPMp4Xc8Iu6MiGkRcT7wGeDZMhZNkiZKemf++AxgAdkFIWbWQKnXOK2WtCtfA1EBbkuc\n",
       "p5p1ZIvXt+aXAd+XOtBAqjUlLYOIOAl8CXiS7MqlH0fEnrSp+pP0IPAiMFPSIUlJpo6HcAWwnOwq\n",
       "tR35TxmvBDwPeDb/fm8HNkfEM4kzFVHWaeVzged7/T4fi4inEmcyG3PcANPMzMysoNQjTmZmZmZN\n",
       "w4WTmZmZWUEunMzMzMwKcuFkZmZmVpALJzMzM7OCXDiZmZmZFeTCyczMzKwgF05mZmZmBf0PM6oY\n",
       "ijHHSmwAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x106da2b10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# optimal d = 6\n",
    "def plot():\n",
    "    X = construct_poly(x,6)  \n",
    "    Xt = construct_poly(xt,6) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 1.0'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    \n",
    "    w = wRidge(X,y,0.1)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 0.1'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    \n",
    "    w = wRidge(X,y,10.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 10.0'+\"  MSE: %f\" % MSE(X.dot(w),y))\n",
    " \n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
    "plot()\n",
    "print('Minimum MSE for lambda = 0.1')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hand in printed copy of completed notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "annotations": {
   "author": "",
   "categories": [
    "intelligent-systems-1-2015"
   ],
   "date": "2015-04-30",
   "location": "Beginning of next lecture",
   "parent": "IS_SS2015",
   "submission_date": "2015-05-07",
   "subtitle": "Exercise Sheet 3, Linear Regression",
   "tags": [
    "IntelligenSystems",
    "Course"
   ],
   "title": "Intelligent Systems 1 - Summer Semester 2015"
  },
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}