{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "preamble": true
   },
   "source": [
    "(Defining latex commands: not to be shown...)\n",
    "$$\n",
    "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
    "\\DeclareMathOperator{\\minimize}{minimize}\n",
    "\\newcommand{\\real}{\\mathbb{R}}\n",
    "\\newcommand{\\normal}{\\mathcal{N}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Gaussian Algebra (25 Points)\n",
    "\n",
    "Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
    "\n",
    "\\begin{equation}\n",
    "   \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
    "\\end{equation}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Proof\n",
    "\\begin{align*}\n",
    "   f(x) &= \\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\cdot e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}}\\cdot \\frac{1}{\\sqrt{2\\pi s^2}}\\cdot e^{\\frac{-(x-m)^2}{2s^2}}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\alpha}\\\\\n",
    "   \\text{where }\\alpha &= \\frac{s^2(x-\\mu)^2+\\sigma^2(x-m)^2}{2\\sigma^2 s^2}\\\\\n",
    "   &=\\frac{(s^2+\\sigma^2)x^2-2(s^2\\mu+\\sigma^2 m)x+s^2\\mu^2+\\sigma^2m^2}{2\\sigma^2s^2}\\\\\n",
    "   &=\\frac{x^2-2\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}x+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}+\\underbrace{\\frac{\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}_\\beta\\\\\n",
    "   \\beta&= \\frac{\\frac{s^4\\mu^2+s^2\\sigma^2\\mu^2+s^2\\sigma^2m^2+\\sigma^4m^2-\\left(s^4\\mu^2+2s^2\\sigma^2\\mu m+\\sigma^4m^2\\right)}{\\left(s^2+\\sigma^2\\right)^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
    "   &=\\frac{s^2\\sigma^2\\left(\\mu^2-2\\mu m+m^2\\right)}{2\\sigma^2s^2\\left(s^2+\\sigma^2\\right)}\\\\\n",
    "   &=\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}\\\\\n",
    "   \\Rightarrow f(x)&=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-(\\alpha-\\beta)-\\beta}\\\\\n",
    "   &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}\\cdot e^{-\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}}\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi(s^2+\\sigma^2)}\\sqrt{2\\pi\\frac{\\sigma^2s^2}{s^2+\\sigma^2}}}{2\\pi s\\sigma}\\normal\\left[x;\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2},\\frac{\\sigma^2s^2}{s^2+\\sigma^2}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\frac{\\sqrt{2\\pi\\cdot 2\\pi\\sigma^2s^2}}{2\\pi s\\sigma}\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
    "   &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\hspace{2cm} q.e.d.\n",
    "\\end{align*}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
    "\n",
    "Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
    "\n",
    "\n",
    "Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#  Linear regression (50 Points)\n",
    "\n",
    "In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
    "\n",
    "1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
    "2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Loading the required packages\n",
    "%matplotlib inline\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from IPython.html.widgets import interact\n",
    "import csv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "def wRidge(X,y,lamb):\n",
    "  # Change the following line and implement the ridge regression estimator wRidge\n",
    "  #w = np.zeros(X.shape[-1])\n",
    "    #print y\n",
    "    #print np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)\n",
    "    return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Read ex1_train.csv and assign the first column and \n",
    "# second column to variables x and y respectively.\n",
    "\n",
    "\n",
    "a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
    "x= a[:,0]\n",
    "y= a[:,1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "4) Plot the training data with appropriate labels on each axes!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x105d55790>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEZCAYAAACTsIJzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAFmxJREFUeJzt3XuQZGd93vHvo1tWWIsEBmNZ7CIsQYJYQICREZcwQBzE\n",
       "xoHIGDCOMRi05Spgl4oJZQQJkkM53BLC7lJ2EgHiZm4yBiOQZGSsNUpUXAQISxFClsKyKwwSV7Eb\n",
       "dhNdfvnj9LJ9RnPpnpnu0z3z/VRNTZ/3dJ/+zZZ0nn7f97ynU1VIknTIEV0XIEmaLAaDJKnFYJAk\n",
       "tRgMkqQWg0GS1GIwSJJaDAapA0nek+QNXdchzcVg0KqVZHeSp4/hfc5P8v4hX1a9n0GOvyvJS4ev\n",
       "TFoag0Gr2cAn345kwOdN8t+gVchg0JqQ5MVJ/keStyb5YZL/neSsvv27krwxyReS3J7kE0nu09s3\n",
       "k2TvrOPtTvL03jHOBZ6fZF+Sr87z/o9O8pUkP0nyYWBd3777JPlUktt6tV2c5KTevj8Gngy8o3f8\n",
       "Hb327Un29Gq9OsmTVvifTGuYwaC15AzgBuDngbcA75q1/4XA7wEnAncCOxY4VgFVVZcB/xH4cFWt\n",
       "r6pHz35ikmOATwDvBe4DXAQ8h8M9gfRq2dj7OQC8g+YNXgdcCby8d/xtvdd8EXhU73gfBC7qvY+0\n",
       "bAaD1pJvVdW7qrlB2PuAE5P8Qm9fAe+rquur6qfAvweel2SQ4Z6w8LDQ44Gjqmp7Vd1VVR8DvnRo\n",
       "Z1X9sKo+XlUHq2o/TdA8ZY73oO81f1ZVP6qqu6vqbcA/Av7xALVKizIYtJZ899CD3skf4Li+/f3D\n",
       "RXuAo4H7rcD7/hLw7Vlt36J3sk9yryT/rTc8dTvwt8Dxs0KpNc+Q5N8muT7Jj5P8CDh+hWqVDAap\n",
       "z8ZZj+8Avg/8H+Beh3YkORK4f99zF5sc/g5w0qy2B/W97lXAQ4Ezqup4mt5Cfy9kdig8GXg18Nyq\n",
       "OqGq7gPczuCT2dKCDAapEeB3kjwsyb2A/wBc1Bt2uhFYl2RzkqOBf0czdHPId4GTFxh2ugq4M8m2\n",
       "JEcn+Q3gcX37j6OZV7g9yX2B82a9/lbglL7t9TRzIN9PckyS1wP3XsofLc3FYNBaMdelqzXr8fuB\n",
       "99B8wj8G2AZQVbcDLwPeCdwC7Kc97HRR7/cPklx9jzeuugP4DeDFwA+A5wEf63vK24FjaXonVwGX\n",
       "zqptO/CbvSuW3g5c1vu5EdhNEyp7FvzrpSGkyy/qSfJu4F8At1XVI3pt5wPnAN/rPe3c3pUf0sgk\n",
       "uQJ4f1W9u+tapK513WO4EDhrVlsBb6uqR/d+DAWNi2P0Eh0HQ1VdCfxojl3+D6ouuMJYovsew3y2\n",
       "JvlaknclOaHrYrT6VdVTHUaSGpMYDH8KPBg4nWYS8D93W44krS1HdV3AbFV126HHSd4JXDz7OUns\n",
       "8kvSElTVokP1ExcMSU6squ/0Ns8Grp3reYP8cZMqyflVdX7XdSyV9XfL+rszzbXD4B+qOw2GJB+i\n",
       "WeV5v97dK88DZpKcTjMR+E3g9zssUZLWnE6DoapeMEezE4CS1KFJnHxeC3Z1XcAy7eq6gGXa1XUB\n",
       "y7Sr6wKWaVfXBSzDrq4LGIdOVz4vVZKa5jkGSerCoOfOiZt8liStrGTTZtiwbfFnNgwGSVrFmlA4\n",
       "cztccOqgN5VwjkGSVrUN25pQGJzBIEmr2vp1w77CoSRJq8LhcfT162DfQdi7o+q6S7quq3v7Dg77\n",
       "CoNB0tRrj6MfsuWUZBOGw94dsOWUYYaTvFxV0liM8hN98szL4NJn3HPP5suqLnnmSrzHNGv+7Tdu\n",
       "hUvP8nJVSRNh9J/o5xtHP+7Y5R97+oeperVeMhX3SpK0Vsx1ZcwFp8LmrcAKnGDnG0fff2C5R16L\n",
       "w1RelSRpDEb7ib43jn5Tu+2cm2HPzuUfe75Q27h1+ceeTPYYJI3B6D7RQzNUkmyi6YEcd2xz3D07\n",
       "p2GYahIZDJLGYK4rY1bqE33j0Dj6Sh3vsNGG2iQyGCSN3Gg/0Y/a6ENt0ni5qiQt4vDlntMWam2D\n",
       "njsNBklaIwY9d3pVkiSpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1\n",
       "GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqSWToMhybuT3Jrk2r62+ya5\n",
       "PMmNST6T5IQua5SktabrHsOFwFmz2l4DXF5VDwU+29uWJI1Jp8FQVVcCP5rV/Czgvb3H7wX+1ViL\n",
       "kqQ1rusew1weUFW39h7fCjygy2Ikaa05qusCFlJVlaTm2pfk/L7NXVW1ayxFSdKUSDIDzAz9uqo5\n",
       "z7tjk+Rk4OKqekRv+wZgpqq+m+RE4Iqq+iezXlNVlbEXK0lTbNBz5yQOJX0SeFHv8YuAT3RYiySt\n",
       "OZ32GJJ8CHgKcD+a+YTXA38JfBTYCOwGnldVP571OnsMkjSkQc+dnQ8lLYXBIEnDm+ahJElShwwG\n",
       "SVKLwSBJajEYJEktE73ATdJhyabNsGEbrF8H+w7C3h1V113SdV1afQwGaQo0oXDmdrjg1MOtW05J\n",
       "NmE4aKU5lCRNhQ3b2qEAzfbGrd3Uo9XMYJCmwvp1c7cfd+x469BaYDBIU2Hfwbnb9x8Ybx1aCwwG\n",
       "aSrs3QFbbmq3nXMz7NnZTT1azbwlhjQlmgnojVub4aP9B2DPTieeNQzvlSRJavFeSZKkJTEYJEkt\n",
       "BoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWoxGCRJLQaDJKnFYJAktRgMkqQWg0GS1GIw\n",
       "SJJaDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWo7quoD5JNkN/AS4C7ijqs7o\n",
       "tiJJWhsmNhiAAmaq6oddFyJJa8mkDyWl6wIkaa2Z5GAo4K+TXJ1ky+ydyTMvSzZt7qAuSVrVJnko\n",
       "6YlV9Z0k9wcuT3JDVV15ePelz4AtpySbqLruks6qlKRVZmKDoaq+0/v9vSQfB84A+oLhfOCkU2Hf\n",
       "9iQ/rapdHZQpSRMryQwwM/TrqmqxA/98Vf1gaWUtTZJ7AUdW1b4kPwd8BvijqvpMb381I00Az/vb\n",
       "qo/OjLM+SZpGSaqqFp27HaTH8Pkk1wAXApfWYkmyMh4AfDwJNDX+2aFQuKf9B8ZQjyStGYP0GI4A\n",
       "/hnwEuBxwEeBC6vqxtGXN29NvR7DOTfD57c5xyBJixu0x7BoMMw66NOADwA/B1wDnFtVVy25yiVq\n",
       "guGZl8GenYaCJA1mxYIhyf2Afw38LnAr8E7gYuBRwJ9X1cnLrnZIg/5xkqTDVnKO4SqaXsKzq+qW\n",
       "vvark/zXpRYoSZpMA80xVNXdY6pnIPYYJGl4g547F135PGmhIEkarUm+JYYkqQOLBkOSJ83R9sTR\n",
       "lCNJ6togPYadc7S9Y6ULkSRNhnmvSkpyJvAE4P5J/oDDt8Bej0NQkrRqLXS56jE0IXBk7/chPwF+\n",
       "c5RFSZK6M8jlqg+qqm+NqZ6BeLmqJA1vJVc+XzFHc1XV05Za3HIZDJI0vJVc+fzqvsfrgOcAdy61\n",
       "MEnSZBvqJno/e1Hypap63AjqGfT97TFI0pBWrMeQ5L59m0cAvwLcexm1SZIm2CBDSV/h8Nel3Qns\n",
       "Bl46qoIkSd1a0lBS1xxKkqThreRQ0rHAy4An0fQcrgT+tKoOLrtKSdLEGeRy1YtoFrV9gGb1828D\n",
       "x1fVc0df3rw12WOQpCGt5DqG66vqtMXaxslgkKThrdj3MQBf6d036dCBHw98eTnFSZIm1yA9hhuA\n",
       "hwJ7aeYYNgLfoLlCqarqkaMuco6a7DFI0pBWcuXzMzh8Z9VDao42SdIqMEiP4f1V9cLF2sbJHoMk\n",
       "DW8l5xg2zTrwUcBjl1qYJGmyzRsMSV6bZB/wiCT7Dv0AtwGfHFuFkqSxGmQo6U1V9Zox1TMQh5Ik\n",
       "aXgruY7hKRy+V9LPVNXnll7e8hgMkjS8lQyGT3E4GNYBZwBf9ot6JGm6rNjlqlX167MOvAHYvoza\n",
       "JEkTbJCrkma7BXjYShciSZoMg9xddWff5hHA6XhLDElatQZZ+fxlDs8x3AV8sKr+5+hKkiR1aZDJ\n",
       "52OBU2nC4aZJ+B4GJ58laXjLXvmc5Ogkb6G5ed57gfcBtyR5a5KjV65USdIkWWjy+a3AfYEHV9Vj\n",
       "quoxwC8DJwD/aRzFSZLGb96hpCQ3AQ+tqrtntR8JfKOqTh1ZUclZwNuBI4F3VtWbZ+13KEmShrQS\n",
       "N9G7e3YoAFTVXcA92ldKL3jeAZwFnAa8IImXx0rSmCwUDF9P8qLZjUleCNwwupI4g2aSe3dV3QF8\n",
       "GHj2CN9Pa0SyaXPyzMuS5+1qfm/a3HVN0iRa6HLVlwN/keQlHF638FjgXsDZI6zpJJoJ70NuAX51\n",
       "hO+nNaAJgTO3wwV9Q6BbTkk2UXXdJd1VJk2eeYOhqm5J8qvA04CH01yu+umq+uyIa1r4+llpSTZs\n",
       "a4cCNNubtwIGg9RnwQVu1cxMf7b3My7fBjb0bW+g6TW0JDm/b3NXVe0abVmabuvXzd1+3LHjrUMa\n",
       "nyQzwMywrxtk5fO4XQ08JMnJwD8AzwdeMPtJVXX+WKvSlNs3z8LM/QfGW4c0Pr0PzLsObSc5b5DX\n",
       "LeUmeiNVVXcCrwD+Crge+EhVfb3bqjT99u6ALTe12865GfbsnPv50tq16C0xJpHrGLQUzQT0xq3N\n",
       "8NH+A7BnpxPPWktW7It6JpHBIEnDW4kFbpKkNWgSJ5870ww1bNjWXMGy7yDs3eFQg6S1xmDocQGU\n",
       "JDUcSvqZ+RZAbdzaTT2S1A2D4WdcACVJYDD0cQGUJIHB0McFUJIErmOYdVwXQElavVzgJklqcYGb\n",
       "JGlJDAZJUovBIElqMRgkSS0GgySpxWCQJLUYDJKkFoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMk\n",
       "qcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWo5qusCpEOSTZthwzZYvw72HYS9O6quu6TruqS1\n",
       "xmDQRGhC4cztcMGph1u3nJJswnCQxsuhJE2IDdvaoQDN9sat3dQjrV0GgybE+nVztx937HjrkGQw\n",
       "aELsOzh3+/4D461DksGgCbF3B2y5qd12zs2wZ2c39UhrV6qq6xqGlqSqKl3XoZXVTEBv3NoMH+0/\n",
       "AHt2OvEsrZxBz50TFwxJzgfOAb7Xazq3qi6b9RyDQZKGNOi5cxIvVy3gbVX1tq4LkaS1aFLnGOwN\n",
       "SFJHJjUYtib5WpJ3JTmh62IkaS3pZI4hyeXAL86x63XA5zk8v/AG4MSqeums1xfwR31Nu6pq1whK\n",
       "laSplWQGmOlrOm8qJ5/7JTkZuLiqHjGr3clnSRrSoOfOiRtKSnJi3+bZwLVd1SJJa9EkXpX05iSn\n",
       "01yd9E3g9zuuR5LWlIkeSpqPQ0mSNLypHUqSJHXLYJAktRgMkqQWg0GS1GIwSJJaDAZJUovBIElq\n",
       "MRgkSS2TuPJZE6r5hrUN22D9uuY7mvfu8BvWpNXHYNBAmlA4cztccOrh1i2nJJswHKTVxaEkDWjD\n",
       "tnYoQLO9cWs39UgaFYNBA1q/bu72444dbx2SRs1g0ID2HZy7ff+B8dYhadQMBg1o7w7YclO77Zyb\n",
       "Yc/ObuqRNCredlsDayagN25tho/2H4A9O514lqbHoOdOg0GS1gi/j0GStCQGgySpxWCQJLUYDJKk\n",
       "FoNBktRiMEiSWgwGSVKLwSBJajEYJEktBoMkqcVgkCS1GAySpBaDQZLUYjBIkloMBklSi8EgSWox\n",
       "GCRJLQaDJKnFYJAktXQSDEmem+R/JbkryWNm7Ts3yd8nuSHJP++iPklay7rqMVwLnA18rr8xyWnA\n",
       "84HTgLOAP0my6no1SWa6rmE5rL9b1t+daa59GJ2cdKvqhqq6cY5dzwY+VFV3VNVu4CbgjLEWNx4z\n",
       "XRewTDNdF7BMM10XsEwzXRewTDNdF7AMM10XMA6T9mn8l4Bb+rZvAU7qqBZJWpOOGtWBk1wO/OIc\n",
       "u15bVRcPcahaoZIkSQNIVXfn3SRXAK+qqq/0tl8DUFVv6m1fBpxXVV+Y9TrDQpKWoKqy2HNG1mMY\n",
       "Qn+RnwQ+mORtNENIDwG+OPsFg/xhkqSl6epy1bOT7AUeD3w6yaUAVXU98FHgeuBS4GXVZZdGktag\n",
       "ToeSJEmTZ9KuShpYkjck+VqSa5J8NsmGrmsaRpK3Jvl672/4iyTHd13TMBZapDipkpzVWzj590n+\n",
       "sOt6hpXk3UluTXJt17UMK8mGJFf0/pu5Lsm2rmsaRpJ1Sb7QO99cn+SNXde0FEmOTPLVJAteADS1\n",
       "wQC8paoeVVWnA58Azuu6oCF9Bnh4VT0KuBE4t+N6hjXnIsVJleRI4B00CydPA16Q5GHdVjW0C2nq\n",
       "n0Z3AP+mqh5OM4T88mn696+qg8BTe+ebRwJPTfKkjstailfSDNUvOFQ0tcFQVfv6No8Dvt9VLUtR\n",
       "VZdX1d29zS8AD+yynmEtsEhxUp0B3FRVu6vqDuDDNAsqp0ZVXQn8qOs6lqKqvltV1/Qe7we+TrNu\n",
       "aWpU1U97D48BjgR+2GE5Q0vyQGAz8E7aF/3cw9QGA0CSP06yB3gR8Kau61mGlwCXdF3EKncSsLdv\n",
       "28WTHUlyMvBomg9EUyPJEUmuAW4FruhdLDNN/gvwauDuxZ440cGQ5PIk187x8y8Bqup1VbUReA/N\n",
       "Hz1RFqu/95zXAf+vqj7YYalzGqT+KeJVFhMgyXHAnwOv7PUcpkZV3d0bSnog8E+n6b5JSX4duK2q\n",
       "vsoivQWYjHUM86qqXxvwqR9kAj9xL1Z/khfTdO2ePpaChjTEv/80+DbQf4HCBtq3X9GIJTka+Bjw\n",
       "gar6RNf1LFVV3Z7k08CvALs6LmdQTwCelWQzsA64d5L3VdXvzvXkie4xLCTJQ/o2nw18tataliLJ\n",
       "WTTdumf3Jram2TQsOLwaeEiSk5McQ3MX3092XNOakSTAu4Drq+rtXdczrCT3S3JC7/GxwK8xReec\n",
       "qnptVW2oqgcDvwX8zXyhAFMcDMAbe8Ma19Dc8fBVHdczrJ00k+aX9y4f+5OuCxrGfIsUJ1VV3Qm8\n",
       "AvgrmqsyPlJVX++2quEk+RBwFfDQJHuT/F7XNQ3hicDv0FzN89XezzRdYXUi8De9880XgIur6rMd\n",
       "17QcCw6tusBNktQyzT0GSdIIGAySpBaDQZLUYjBIkloMBklSi8EgSWoxGKRZkqz4rRqSPCjJC1b6\n",
       "uNIoGAzSPY1icc+Dgd8ewXGlFWcwSPNIMpNkV5KLel+q9IG+fbuTvDnJ3/W+wOWUXvt7kjyn73mH\n",
       "bg//JuDJvRW/r5z1Pmcn+eve4xOTfCPJL4z+L5TmZjBICzud5stNTgN+OckTeu0F/LiqHknzBUBv\n",
       "72ufyx8CV1bVo6tqe/+Oqvo48J0krwD+O/D6qrpthf8OaWAGg7SwL1bVP1Rz75hrgJP79n2o9/vD\n",
       "wJmLHGexGw1upfkWv4NV9ZGlFCqtFINBWtj/7Xt8F/Pfqv5QT+FOev9fJTmC5tu+BrGhd/wH9O5E\n",
       "KnXGYJCW7vl9v6/qPd4NPLb3+FnA0b3H+4D1cx0kyVE0t6T+LeAG4A9GUKs0sIn+oh6pIzXP49nu\n",
       "k+RrwEHg0KWoFwB/2bs982XAoUtfvwbc1Wu/cNY8w7nA56rqqiR/B3wpyaeq6hsr8cdIw/K229IS\n",
       "JPkm8NiqmqovhJcG4VCStDR+otKqZY9BktRij0GS1GIwSJJaDAZJUovBIElqMRgkSS0GgySp5f8D\n",
       "oJt+CXUlyycAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x106b1e090>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plot the input data here\n",
    "plt.plot(x,y,'o')\n",
    "plt.title('Input data')\n",
    "plt.ylabel('Output y')\n",
    "plt.xlabel('Input x')   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
    "Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def construct_poly(x,d):\n",
    "  ## Implement a method which given an array of size N, \n",
    "  ## returns an array of dimension (N,d)\n",
    "    poly=np.empty((len(x),1), float)\n",
    "    xr = np.reshape(x,(len(x),1))\n",
    "\n",
    "    for i in range(d-1):\n",
    "        poly = np.append(poly, (xr**i), axis=1)\n",
    "    #print poly \n",
    "    #print np.shape(poly)\n",
    "    return poly"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "def MSE(y_predict,y_true):\n",
    "  ## Implement mean squared error for a given input y and its predictions.\n",
    "  return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmYHGW5/vHvTVjSCCFgHAj7SABxREE5gCLHsKghKovK\n",
       "5sImwhFIOP6MJwIqAQWNCSiJqPyURUEQkM0gBFmSA6JEUOBwCCoJiYYtDQk7HQjJc/6oGqczzGSm\n",
       "p7unuqvvz3X1lemq6qpn5oJ7at566i1FBGZmli9rZF2AmZnVnsPdzCyHHO5mZjnkcDczyyGHu5lZ\n",
       "DjnczcxyyOFuZpZDDncbVJIWSnpN0lu7Lb9f0kpJW6bvN5d0jaRnJD0v6SFJR6brtk63fanb6+Be\n",
       "jnmIpD9IekXSrD7qO7XbPl+VtELSRun6YZIuS+t6Jv16/bLPD5H0bUlPSHpR0l8kbZCuW0fS99N1\n",
       "SyWdL2nNss9uLmmGpCWSnpI0XdKQHmr8Zvr9793/n7y1Goe7DbYAHgMO71wgaUegkK7rdCnwD2BL\n",
       "YCPg88DibvvaICLWL3td3csxlwDnAt/ts7iIs8v3CUwGZkXE0nSTScAIoB3YBtg4XdbpDGB3YPeI\n",
       "GAZ8DliWrvsa8F6gA9gu/frrZZ+dBjwLjAR2Aj4EnFBen6RtgE8DT/b1vVhrc7hbFi4Djih7fyTw\n",
       "C0Bly3YBLomIUkSsjIgHImLmQA4WEbdHxK+Bpyr5nCSltf28bHEHcH1EvBwRLwLXp8uQtCFwMvDF\n",
       "iFiUHntuRLyWfvbjwPSIeD4iniUJ82O67fvKiHg9IhYDMzv3XeaHwERgeSXfi7Ueh7tl4R5gmKR3\n",
       "pMMOh5IEfvdtfiTp0M6hmh6ol+W1sifwNuCasmW3AJ+SNDwN808BN6XrdgTeAA5Oh1X+JmmVM+9u\n",
       "Na8BbF42rHML8BlJBUmbAfsBN//rg8mw07KIuBmzPjjcLSuXkpy9fxiYCzzRbf3BwF3AN4DH0jH5\n",
       "Xbpt86yk58pe29e4xiOBqyPi1bJl56f/LiEZQlkO/DhdtjmwAbAtsDXJ8MkkSfum62cCJ0saIWkT\n",
       "YDzJUNS66fpJwLuAF4FFwL0RcQNA+gvgLJK/DMz65HC3LARJuH+WnodkSIcuTomId5GMaz9AMgRS\n",
       "7q0RsWHZ62+1KlDSuiTh/PNuq34J/A1YDxhGcv2g86+OUvrvmRHxWkQ8BPwKGJsuPwu4P/1efg9c\n",
       "B7wREYvTIaBbgKtJwn4EsJGkyelnJwGXRsQ/y8uswbdqOeVwt0ykIfUYydDDtX1suwQ4B9g0HQoZ\n",
       "8GEr2PYgYElE/He35WOAC9JrAa8AF9AV3v+zuuNGxLKIGBcRm0fEKGApcF+6zQjgfcAPI2J5egH3\n",
       "krJ97w2MT4d7ngK2AK6S9NUKvidrIQ53y9IXgL0jotR9haTJkjokrZkOSXwJeDQinivfrD8HkbSG\n",
       "pKHAWsAaaUviWn18rPMviu7+B/iipKGSCsBxwIMAETGfZCjpNElrS9qB5HrCjWkdm6YvSdqdpFPm\n",
       "9HS/z5Jc8P1S2k45PK3hwXT9PiQXV99D0knzZHrsH/XnZ2Ctx+FumYmIxyLiL+WLyr4ukAxbPAfM\n",
       "JzlT3b/bLp7v1pP+n70c6gjgVZIg3JNk+OSCzpXpZ/coe78ZMJqew/0okjbGJ4DHScbWjyxbfziw\n",
       "FcmY/I3A1yOis7d+G+Bu4GXgYmBiRNyW/iwC+CTwCZKgfxR4Dfhyun5pRBTT12JgBfBc+teD2Zuo\n",
       "2od1SLoI+BhQjIgd02WTgGOBZ9LNThloG5uZmVWuFmfuF5OMQ5YL4NyI2Dl9OdjNzAZR1eEeEXeR\n",
       "/Oncna/km5llpJ5j7uMkPSjpwvTikJmZDZJ6hfuPSebe2ImkA+CcOh3HzMx6sGbfm1QuIoqdX0v6\n",
       "GTCj+zaSqruSa2bWoiKiz2HvuoS7pJER0TlJ00HAQz1t158CG5WkSRExKes6Bsr1Z8v1Z6eZa4f+\n",
       "nxhXHe6SriCZmnSEpEUkN2WMlrQTSdfMAuD4ao9jZmb9V3W4R8ThPSy+qNr9mpnZwPkO1YGbnXUB\n",
       "VZqddQFVmp11AVWanXUBVZqddQFVmJ11AYOh6jtUB3xgKZp5zN3MLAv9zc66XFA1M7Paam/X2LY2\n",
       "xvd3ew/LmJk1uPZ2je3o4LzJk/lofz/jcDcza3BtbYyfMIFRlXzGwzJmZhmSWIPkqV4b9fbaZZeO\n",
       "neDhivbrcDezhiAxZMMNJ5w9cuTdxxYKy3jttXj1n//83J0vvjjhLpJpTDpfT0ewPNtq30xiTWA4\n",
       "qwnpXl7DgVdInszV0+uJ5ctfeYbkcZP95nA3s0xJDAEOWXfdqyfvvPOVb/vGNx4fmq7a6OyzF425\n",
       "5551N3zppRNWACPT19skXmDVwH+K5OlUqyyL4E1P+epHPWsDG1J5SK8PvEDvIT0fuLeH5c/39cuq\n",
       "vX3hI1Oncl4lQzNuhTSzfuvs2CgUGFoqsaxYZNqCBXHTQPaVhvphJI8bXLLzzpusc+65i3fpvt3E\n",
       "icycMyf26/a5EXSF/Uhg027vRwKbAMt48y+BJax+GGQoyTTmvYX0kl6WvxDByoH8LPoj/dmP+9Of\n",
       "GONWSDOrmc6OjfKzx6lT2aa9XVQS8OnwxWHAN4AicBJwx7Bhi2f1tH2hQKH8fQQrgMXp64HVHEck\n",
       "Z+AjgZHDh39tzJZbXveJddahsGyZli1adOjvnn/+jKt5c0i/FFHRw9QHRfozvmnQ5pYxs9bQU8fG\n",
       "hAmMmjiRcUCf4Z6G+mdIztSfJnno+azOIN1tN5b19LlSqfKhFYB0v0uBpe3t2qqjgwNW/cV0ph5+\n",
       "+MybFixInmObN26FNLN+KRQY2svyQk/LO0msKXEk8AhwDMlEgh+K4I7yM+RikWlTpzKv/LNTpjC/\n",
       "WGR6tbX39ouprY1x1e67UfnM3cz6pVSq7MxaYi3gc8BpwCLgixG9z+uyYEHc1N4uJk5kXKFAoVSi\n",
       "VCwyfaBj+uUG+oupmTnczaxf0jPrbcrPgHs6s05D/fMkob4QOCaCO/tzjM5x5dpVnaj0F1MeuFvG\n",
       "zPqts2OjpzPrNNSPIAn1x4AzIrgry3o79XQxeMoU5s+dy/ha/GUwmPqbnQ53M6tK2hd+JHAq8ChJ\n",
       "qN+dbVVvtrpfTM3E4W5mdZWG+lEkof43klD/Q6ZFtQBP+WtmdSGxDnA0cAowFzg8gj9mW5V153A3\n",
       "s36T2AS4E5gHHBLBnIxLsl54WMbM+kViXZJH1N0YwZkZl9OyPOZuZjWTTkt7NcnshUc24u35rcJj\n",
       "7mZWS98lmazrMw725uBwN7PVkjgOOAB4fwSvZV2P9Y/D3cx6JfER4EzggxEszboe6z+Hu5n1SOJd\n",
       "wGXAJyNWndDLGp9nhTSzN0lbHm8E/jOC32ddj1XO4W5mq0hbHmcAF0Zwedb12MC4FdLM/iVtefw1\n",
       "8BJwlDtjGo9bIc1sICaTPEf0cAd7c3O4mxkAEscD++OWx1xwuJsZEh8FJuGWx9xwuJu1OIkdgUuB\n",
       "gyKYn3U9VhvuljFrYRIjSVoeT27EB2zYwDnczVqUxFuA3wA/jeCKrOux2nIrpFkLkhhC0vL4AnC0\n",
       "O2Oah1shzWx1JgPDgUMd7PlU9bCMpIskLZb0UNmyjSTdKunvkn4naXi1xzGz2pD4EvBx4FMRvJ51\n",
       "PVYftRhzvxgY023Z14BbI2I74Pb0vZllTGIM8E3gY255zLeqwz0i7gKe67Z4f+Dn6dc/Bw6s9jhm\n",
       "Vh2Jd5O0PH7aLY/5V69umY0jYnH69WJg4zodx8z6IW15nAGMd8tja6j7BdWICEk9XrCRNKns7eyI\n",
       "mF3vesxaTdryOAP4/255bD6SRgOjK/5cLVohJW0NzIiIHdP3fwVGR8TTkkYCsyLiHd0+41ZIszpL\n",
       "Wx6vAZ7HLY+50N/srNewzG+AI9OvjwSur9NxzGz1vgdsABznYG8tVZ+5S7oC+BDJk9EXk1yJvwG4\n",
       "CtgSWAgcEhHPd/ucz9zN6kjiBGA8ySyP3ZserEn1Nzt9h6pZDknsB1xEMsujO2NyxHeomrUoifeQ\n",
       "tiA72FuXJw4zyxGJTelqefxD1vVYdhzuZjlR1vJ4QQS/yroey5bH3M1yIG15vBZYChzjzpj88pi7\n",
       "WQNqb9fYtjbGFwoMLZVYViwybcGCuKkGu54KrA8c7GA3cLibDZr2do3t6OC8CRMY1bls6lS2aW8X\n",
       "1QS8xInAfiQtj57l0QCPuZsNmrY2xpcHO8CECYxqa2PcQPcpMRb4OjDWvexWzmfuZoOkUGBoL8sL\n",
       "A9lfWcvj/hE8Vk1tlj8+czcbJKUSy3pZXqp0X2UtjydF8Mdqa7P8cbibDZJikWlTpzKvfNmUKcwv\n",
       "FpleyX4k1gNuBH4SwZW1rNHyw62QZoMo7ZYZVyhQKJUoFYtMr+RiatryeB3wLPAFd8a0Hs8tY5ZD\n",
       "Ej8A3g2McWdMa3Kfu1nOSJwEfBT4gIPd+uJwN2sCEh8DTgP2cMuj9YfD3azBSewEXIJbHq0C7pYx\n",
       "a2ASm5G0PJ7olkerhMPdrEGlLY8zgB9FcFXW9VhzcbeMWQNKWx6vB4rAsW55tE5ZPyDbzKpzLrAu\n",
       "8B8OdhsIX1A1azAS44APk7Q8Ls+6HmtODnezBpK2PJ5KEuzPZ12PNS+Hu1mDKGt5/EQECzIux5qc\n",
       "x9zNGoDE5iSdMSdEcE/W9Vjzc7ibZays5fH8CK7Ouh7LB7dCmmUobXm8AXga+KI7Y6wvboU0aw7f\n",
       "B4YCX3KwWy35gqpZRiTGA/vilkerA4e7WQYkPg6cglserU4c7maDTGJn4GLc8mh15DF3s0EksQnJ\n",
       "BVS3PFpdOdzNBonEOsC1wEVuebR6cyuk2SCQEHAhMBz4dAQrMy7JmpSfoWrWWMYDu5BcQHWwW905\n",
       "3M3qTGJfks6Y90fwctb1WGtwuJvVkcQo4JfAYe6MscFU13CXtBB4EVgBLI+IXet5PLNGIjGMpDPm\n",
       "zAhmZV2PtZa6XlCVtAB4X0Qs7WGdL6habkmsQfKYvKeB4z21gNVKI11QdYBbKzoT2JCkM8bBboOu\n",
       "3n3uAdwm6T5JX+y+8n3v2/Du9naNrXMNZoNK4hDg88CnIng963qsNdU73PeIiJ2B/YATJe1ZvvKc\n",
       "c57/QEcH5zngLS/SqQXOBw6MoJh1Pda66josExFPpf8+I+k6YFfgrs71l1wCI0Yw6tVXOU/SqxEx\n",
       "u571mNWTRBvJOPtJEdyfdT2WD5JGA6Mr/Vzdwl3SusCQiHhJ0luAjwBnlG9z1FHJvwsX8sTixQ52\n",
       "a14SawPXAJdGcGXW9Vh+pCe9szvfSzq9P5+r55n7xsB1kjqP88uI+F1PG5ZKlOpYh1ldpVMLTAeW\n",
       "At/MuBwzoI7hHhELgJ362u6MM9pXPv74+pfVqw6zQfAl4IMkd6B6agFrCJneoTpxIjPnzp045OWX\n",
       "j980yzrMBkpiNHA6yZwxL2Zcjtm/ZD4rpMSewAVAh/uBrZlItAN/BD4XwW1Z12OtoZkekP17YG3g\n",
       "37IuxKy/JNYjmVrgOw52a0SZh3t6tn4JcHTGpZj1Szq1wC+A+4BpGZdj1qPMh2WSr9kCeADYPMKd\n",
       "M9bYJCaRtPbuFcFrGZdjLaaZhmWIYBHwZ+DArGsxWx2JTwFfAD7pYLdG1hDhnroYOCrrIsx6I/Fu\n",
       "4CfAQRE8nXU9ZqvTEMMyyXsKwBPAe9IzebOGITECuBc4LYLLs67HWldTDcsApGPtVwFHZF2LWTmJ\n",
       "tYCrgasc7NYsGubMPVnGbsBlwHbuebdGIXE+sDWwfwQrMi7HWlzTnbmn/gQsJ7mV2yxzEscB+wCf\n",
       "cbBbM2mocE/P1n1h1RpCevf0t0nO2F/Iuh6zSjTUsEyynJHAXJKe91cGvzIzkNgKuAc4OoKZWddj\n",
       "1qlZh2WI4CngbuDTWddirUniLSQP3TjHwW7NquHCPXUxno7AMpDOzX4x8BBwTsblmA1Yo4b7DKBD\n",
       "4u1ZF2It51RgK+A4d2xZM2vIcE+fGH8FcGTWtVjrkNif5MEbB0WwLOt6zKrRcBdUu9azM3Ad8HY/\n",
       "3cbqTaKD5DmVH49gTsblmPWqaS+odkqfHv88sFfWtVi+SWxEMjf7VxzslhcNG+4p97xbXUmsCVwJ\n",
       "3BDBL7Kux6xWGnZYJtmGEcA8YCvfRGL1IPEDYAfgYxG8kXU9Zn1p+mEZgAieBe4ADsm6FssfiWOA\n",
       "scBhDnbLm4YO95R73q3mJD4ATAYOiOC5rOsxq7VmCPebgXaJd2RdiOVD+ljHXwNHRfBI1vWY1UPD\n",
       "h3v65/JluOfdaiB9KMx1wLQIfpt1PWb10tAXVLu2pQP4HbClp121gUqnFvhl+vazvgPVmlEuLqh2\n",
       "iuBhkkfwfSTrWqyp/RewPXCsg93yrinCPeULqzZgEh8DTgYOjODVrOsxq7emGJZJtmdDYAHJdARL\n",
       "61eZ5Y3EDsCdJMF+d9b1mFUjV8MyAGm72s3A4VnXYs0jPSm4Afiag91aSdOEe+oSPDRj/SQxhGR2\n",
       "0ZkRXJh1PWaDqdnC/TZgY4kdsy7EmsJkYE3g/2VdiNlga6pwT9sgf4EnE7M+SBwBHAgc6qkFrBU1\n",
       "zQXVrs+xLfB7kgdoL699ZdbsJHYFfguMTttozXIjdxdUO0XwKPAoyYRPZquQ2BS4lqSX3cFuLavp\n",
       "wj3lnnd7E4mhJFML/CSCG7KuxyxLdQt3SWMk/VXSo5Im1nj3VwGjJdpqvF9rUunUAhcA/wDOyrgc\n",
       "s8zVJdwlDQF+CIwB3gkcLmmHWu0/gpdIepc/W6t9WtP7MvBu4GhPLWCWtInVw67AvIhYCCDpV8AB\n",
       "UNPpVS8Gpkn8wP8zt472do1ta2N8ocDQUollxSLTFi6MFcBXgd0jeCXrGs0aQb3CfTNgUdn7x4Hd\n",
       "anyMO4H1gfcCf67xvq0BtbdrbEcH502YwKjOZd/73lrvWLz4ug1KpYP2j+AfWdZn1kjqNeZe9zPp\n",
       "CFbiO1ZbSlsb48uDHeC//mv5VtttN+GJCO7Kqi6zRlSvM/cngC3K3m9Bcva+CkmTyt7OjojZFR7n\n",
       "58B9El+J4LVKi7TmUigwtKflw4c/9uxg12I2WCSNBkZX+rl6hft9wLaStgaeBA6lhwm/ImJSNQeJ\n",
       "YKHEg8D+wNXV7MsaX6nEsl6Wlwa7FrPBkp70zu58L+n0/nyuLsMyEfEGcBJwCzAXuDIi6vWsSve8\n",
       "t4hikWlTpzKvfNmUKcwvFpmeVU1mjarpph94835Yl2QY6F0RPFF9ZdaoJNYYNuzcyzfd9Mb9R4y4\n",
       "939ff/3lJcUi0xcsiJuyrs1ssPQ3O5s+3JN98VNgXgSTa7E/azwSbyGZNG4T4KAIihmXZJaJ3M4t\n",
       "04uLgaPTuxQtZyQ2B+4CXgb2drCb9a1eF1QH2x8BAbu3t2vD7je5+M/25pXO8HgtMB34nm9YM+uf\n",
       "XIR7BCFxyfrr//ibHR2MKu+FnjqVbdrbhQO++UgcRhLqx3oiMLPK5GLMPdkfm22//T4LfvKTO9bq\n",
       "vm7iRGbOmRP71epYVl8SawCnkzyUZf8IHsy2IrPG0Wpj7kTwxHrrLX2pp3WFAoXBrscGJu1++hXw\n",
       "EWA3B7vZwOQm3AFWrHh2UU/LfZNLc5DYjGTOoNeAvSJ4OuOSzJpWrsJ9yZJXJp15ZvvK8mW+yaU5\n",
       "SOwCzAGuAY6I6PluVDPrn9yMuXcaNuz8G7fe+ofbbbTRX58slSj5JpfGJ3EIcD5wXATXZV2PWSNr\n",
       "qZuYVt0vu5A8qWlUOnOkNaj0voRvAscAB0Zwf8YlmTW8lrugWubPwCvAnlkXYr2TKABXAPuRXDh1\n",
       "sJvVUO7CPb3J5WLgeCkfffx5I7Ep8N/ASnzh1KwuchfuqUuB7YBnJK6TOFFiO09PkD2J95JcOL0B\n",
       "+GyEO5nM6iF3Y+6rHoONgX2AD6evlcCtwG3A7Z6jZHBJfBr4MfAfEVyTdT1mzahlL6j2fjwEbA/s\n",
       "SxL0HwIW0hX2d0Xw6mDV00rSn/1pwHEkF07/knFJZk3L4d7n8VkT2JUk7PcFdgbupSvs/xLBiqzq\n",
       "y4v0wumFwCjggAieyrgks6bmcK+QxPrAv5Oc1e8LjARmkYZ9BPMzLK8pSYwErgcWAEd7fN2seg73\n",
       "KqUdHZ3j9fsCy0jO6G8F7ohgSYblNTyJnUkumv4M+Jan6jWrDYd7DaVjxu+ka7x+T+BRusL+bt8u\n",
       "n0iHu74AfBs4IcIPLjerJYd7HUmsDexGV9jvSPLAkM6wf7DV7o5Np+k9GPgWyTNtvxzBA9lWZZY/\n",
       "DvdBJLEBMJqusH8rcDtJ2N8WwcLMiquz9K+a/YCzgDeAU0m+Zw/DmNWBwz1DElvQ1YWzL/AiXWf1\n",
       "syJ4LsPyakZiT+Bskl9mXweuc6ib1ZfDvUGkwxXvouvC7B7AI3SF/R8jeC27CiuXXiw9G3gHMAm4\n",
       "zG2jZoPD4d6gJNYB3k9X2O8A3E1Xf/1DjXr2K7E9yZj6niTDMD9ttl9MZs3O4d4kJDYE9qIr7IeR\n",
       "jtUDt0bweIblASCxJcnUvAcA5wLTIngl26rMWpPDvUlJbE3Xhdl9gGfpOqufHcELg1hLG3AKcARw\n",
       "ATAlL9cLzJqVwz0H0vH6negK+92Bh+gK+3siWF6H424AfAU4EbgcOMvT8po1Bod7DqXztOxBV9iP\n",
       "Au6iK+znVjNen+7/JOCrwE3AGREsqLZuM6sdh3sLkBgB7E1X2K9DVxfO7RE82c/9rEVyV+k3gHuA\n",
       "b0Qwty5Fm1lVHO4tJr2Z6O10XZjdG3iSrrC/M4KXun1mCHAYcCYwHzgtgnurqaO9XWPb2hhfKDC0\n",
       "VGJZscg0P6DcrHYc7i0uDe730hX2/wbcT1fYv42knfFl4NQIZlV7zPZ2je3o4LwJExjVuWzqVOY9\n",
       "/DAnO+DNasPhbquQeAvwQbqeSrWC5AakGbXqq99tN82cPJmPdl8+cSIz58yJ/WpxDLNW19/s9AOk\n",
       "W0Tal35L+qqLQoGhvSwv1OuYZtazvD4g2zJQKvU87XGp5Id0mA02h7vVTLHItKlTmVe+bMoU5heL\n",
       "TM+qJrNW5TF3q6m0W2ZcoUChVKJULDLdF1PNaifTC6qSJgHHAs+ki06JiJkDKdDMzLpkfUE1gHMj\n",
       "4tw67d/MzFajnmPuPis3M8tIPcN9nKQHJV0oaXgdj2NmZt0MeMxd0q3AJj2sOo1kfpLO8fZvASMj\n",
       "4gvdPh/AGWWLZkfE7AEVY2aWU5JGkzyjudPpDXGHqqStgRkRsWO35b6gamZWof5mZ12GZSSNLHt7\n",
       "EMkc5GZmNkjq1S0zWdJOJF0zC4Dj63QcMzPrgW9iMjNrIpkOy5iZWbYc7mZmOeRwNzPLIYe7mVkO\n",
       "OdzNzHLI4W5mlkMOdzOzHHK4m5nlkB+Q3WLSJyWNLxQYWiqxrFhkmp+UZJY/DvcW0t6usR0dnDdh\n",
       "AqM6l02dyjbt7cIBb5YvHpZpIW1tjC8PdoAJExjV1sa4rGoys/pwuLeQQoGhvSwvDHYtZlZfDvcW\n",
       "UiqxrJflpcGuxczqy+HeQopFpk2dyrzyZVOmML9YZHpWNZlZfXjK3xaTdsuMKxQolEqUikWm+2Kq\n",
       "WfPob3Y63M3Mmojnczcza2EOdzOzHHK4m5nlkMPdzCyHHO5mZjnkcDczyyGHu5lZDjnczcxyyOFu\n",
       "ZpZDDnczsxxyuJuZ5ZDD3cwshxzuZmY55HA3M8shh7uZWQ453M3McsjhbmaWQw53M7MccribmeXQ\n",
       "gMNd0sGSHpa0QtJ7u607RdKjkv4q6SPVl2lmZpWo5sz9IeAg4M7yhZLeCRwKvBMYA/xIUu7+QpA0\n",
       "OusaquH6s+X6s9PMtVdiwKEbEX+NiL/3sOoA4IqIWB4RC4F5wK4DPU4DG511AVUanXUBVRqddQFV\n",
       "Gp11AVUanXUBVRiddQGDoR5n1JsCj5e9fxzYrA7HMTOzXqy5upWSbgU26WHVqRExo4LjREVVmZlZ\n",
       "VRRRXe5KmgV8JSL+kr7/GkBEfDd9PxM4PSLmdPucA9/MbAAiQn1ts9oz9wqUH+g3wOWSziUZjtkW\n",
       "+NNAijMzs4GpphXyIEmLgN2B30q6GSAi5gJXAXOBm4EToto/D8zMrCJVD8uYmVnjybT/XNK3JD0o\n",
       "6QFJt0vaIst6KiVpiqRH0u/hWkkbZF1TJVZ3I1qjkjQmvTnuUUkTs66nUpIukrRY0kNZ11IpSVtI\n",
       "mpX+N/O/ksZnXVMlJA2VNCfNm7mSvpN1TQMhaYik+yWttqkl65uLvhcR74mInYDrgdMzrqdSvwM6\n",
       "IuI9wN+BUzKup1I93ojWqCQNAX5IcnPcO4HDJe2QbVUVu5ik/ma0HPhyRHSQDMee2Ew//4hYBuyV\n",
       "5s27gb0kfTDjsgbiZJJh79UOu2Qa7hHxUtnb9YBns6plICLi1ohYmb6dA2yeZT2VWs2NaI1qV2Be\n",
       "RCyMiOXAr0hummsaEXEX8FzWdQxERDwdEQ+kX78MPEJyX0vTiIhX0y/XBoYASzMsp2KSNgfGAj9j\n",
       "1UaWN8n6zB1JZ0n6J3Ak8N2s66nCMcBNWReRc5sBi8re+wa5jEjaGtiZ5KSmaUhaQ9IDwGJgVtoA\n",
       "0ky+D3wVWNnXhnUPd0m3Snqoh9cnACLitIjYEriEpPCG0lf96TanAa9HxOUZltqj/tTfRHz1vwFI\n",
       "Wg/4NXByegbfNCJiZTossznw7800z4ykjwPFiLifPs7aoXZ97r2KiA/3c9PLacAz377ql3QUyZ9J\n",
       "+wxKQRWq4OffDJ4Ayi+6b8GqU11YnUlaC7gGuCwirs+6noGKiBck/RbYBZidcTn99QFgf0ljgaHA\n",
       "MEm/iIgjeto4626ZbcveHgDcn1UtAyFpDMmfSAekF2uaWTPcVHYfsK2krSWtTTL76G8yrqllSBJw\n",
       "ITA3In6QdT2VkjRC0vD06wLwYZoocyLi1IjYIiLagcOAO3oLdsh+zP076RDBAyQztX0l43oqNZ3k\n",
       "QvCtaWvSj7IuqBK93YjWqCLiDeAk4BaSboErI+KRbKuqjKQrgD8A20laJOnorGuqwB7A50i6TO5P\n",
       "X83U+TMSuCPNmznAjIi4PeOaqrHaYUrfxGRmlkNZn7mbmVkdONzNzHLI4W5mlkMOdzOzHHK4m5nl\n",
       "kMPdzCx2UQjRAAAAEklEQVSHHO5mZjnkcDczy6H/Az1pcRz7mHjbAAAAAElFTkSuQmCC\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105561410>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "##This function provides an interactive mode to change polynomial degree. \n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w))\n",
    "    plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "    plt.plot(x,y,'yo')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAi8AAAEZCAYAAABfH8VpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XeYnGX59vHvmYSQUBJCC4ReQgeFH1KUEgtCQu8ICSBo\n",
       "BISIdFCZjEiTVzA0FaQJSAcFhCgtgiggiiKE3ksIICXBJATI+f5x35tMJjub2d3ZnZ3d63Mcc+zO\n",
       "U+7nmtnZmWvuKtuEEEIIITSKXvUOIIQQQgihNSJ5CSGEEEJDieQlhBBCCA0lkpcQQgghNJRIXkII\n",
       "IYTQUCJ5CSGEEEJDieQlzJekEyVdXO84QgghBIjkpVNJelnSx5KWKNv+mKRZklbM95eXdJOkdyR9\n",
       "IOk/kg7I+1bOx04tu+1Z4ZoTJB3cnrhtn2772+0poySeL0u6Lz+ul6o4fiFJF5Y8F38u27+RpPvz\n",
       "c/CWpDEl++6T9LakKZKekvTtkn1flvS4pPclvSfpT5LWKSv7a5L+KekjSa+VPseStpD0d0kfSnqh\n",
       "tOwqHtOE/DfcoGz7LXn71vn+YpIulTQpP4ZnJB1fcvysHFvp6+CYCte8XNIp1cbYQuxNr7+q3zsk\n",
       "/bIsxhmSplQ4dglJD0p6Nz+3j0napeyYH+W/xwf5b7xOM+UMzde5smz7tyQ9l+O4U9KyJfsWk3SF\n",
       "pMn5VijZt2Iz/3OzJH2/2uchhFA7kbx0LgMvAt9o2iBpfaB/3tfkSuAVYEVgcWAUMLmsrIG2Fy25\n",
       "3dDCNSuS1Kd1D6HdPgJ+DRxb5fEXAYsBawGDgCObdkhaErgT+AXpeVoN+FPJuWOA5WwPAA4AzpO0\n",
       "Zt73JDDc9iBgMPAYcGlJ2esAVwMnAgOADYB/5H29gVuAi2wPBPYGzi5PRlpg4Blg/5LrLQFsDrzN\n",
       "nL/ZOcBCwFr5MewEPF9W1gZlr4P/V2UM7aVqD7R9SGmMwDXA9RUO/wg4CFg6P7djgeslLQIgaSfg\n",
       "EGBL0t/8b6T/l3IXAI9Q8vqXNAw4lfQ8Lg68lGNpcg7QD1gJ2AQYJenA/BheLXsM6wOzgJuqfR5C\n",
       "CLUTyUvnu4qSDy3Sh+pvmPvDYGPgctvTbc+y/S/b41t7IUmnkt7kz8/fFM/N22dJOkzSc6QPUSSN\n",
       "k/Rq/rb7qKQtSsoZ2/QNtuSb9/6SXsk1IidVG5Ptv9u+mvTBMb/41wJ2BEbb/q+Tx0oOOQoYb/sa\n",
       "25/Y/p/tp0uu9R/bn5Qc/xEwJe972/YbeXsv0gfRpJJjfwj80vYf89/gfdsv5n2DgSXIH5q2HwWe\n",
       "Atau9nkAfgvsLanp7/4N4GagNN6NgWtsf5iv84ztVn9YShoN7Ascl18Hv8/bhyjV8L0t6UVJR5Sc\n",
       "s0l+HXyoVKPVlBTdn39+kMvatJWxLAzsDlzR3H7bH+fH2VS7Mwt4F5iZD1kX+Ivtl23PIiWY5TVm\n",
       "+wDvA/cw9//VDsANtp/Kr4tTgK0krVKy/yzbM2y/AlxCSqSacwDwZ9uvtubxhxBqI5KXzvcQMEDS\n",
       "Wvkb/N6khKb8mAsl7a3clNSM+X7ztf0D4AHgu/kb45iS3TsDX2DOG/8jwOdItRu/BW6Q1LepqGaK\n",
       "/xKwBvBV4OSmGo3cnPL+/GKr0iakGqgf5yTpcUm7lezfFHg/NzNMlnSrpBVKC5B0u6TpwATgINuT\n",
       "SvatmGOdBmwPfKusbOVrvinpSkmD8r5JwOPAQZJ6S/oi6dv6X1rx2N4EJgLb5vujSElsqYeAUyUd\n",
       "KGlohXKqeR1cRPqQPzO/DnbOicFtpBqnIaS/45GSvp5PGweck2s/VgWaava2zD+bav4ebnoeJS0/\n",
       "v1hIicvbth9o6SBJjwPTgcuBXW03JS/3AJvnZqEFSEnEnSXnDQCKwPeZ97lx2bam97/1Si9dtr90\n",
       "X9M1RPoC0mwCFkLoeJG81MeVpDe/bUgfYG+U7d+TlHT8CHgxt/tvXHbMu/kDo+m2JpU19wF3uu0P\n",
       "bH8MYPvqXLswy/bZwILAmi2cX8zfkh8H/g18Ppfzl9wUUwvLkz48PgCWBQ4Hrih5rCuQPrzGkJrY\n",
       "ypsBsL0DsAjp+b68NBnMTQGDgCXzY7i05NQVgJHAbsBQUtPeefk8A6NJH5IzgD8DJ5XU5FTrN8D+\n",
       "uYZpMdsPle0/gpR0HA48mftqbFd2zD/LXgfbtHC90r/jF4Albf/E9qe2XyI15+2T988Ehkpa0vY0\n",
       "2w83UwYw53m0/XoVj7mpprFFtjcAFiU1G93U1Gxk+xFS0vAMKencnVQD1+QU4Ne232TepHs8sKek\n",
       "9SX1B07OxyxUsv94SYtIWp1U69K/mfC2AJYGbpzvow0hdIhIXjqfScnLfjTfZEROKk60vR6pieJf\n",
       "wO/Kylkif2A03Z6ZzzXLvVZ6R9IxkiYqdYJ8HxhI+lCv5K2S36cBC7dwbFtNJzWjNH3A3g/cx5za\n",
       "imnAzbb/kZOwIvBFSYuWFmL7M9s3Ag8Du5ZfxPb7wDHAjvmbe1PZl9l+3vb/gNOAEQCSlgNuB/a1\n",
       "vQCpKeN4SSNa8dhMaib6CvBdmvlAz80Xp9vemNRMdT2pRmyxksM2LHsd3FXl9VcChpQmPqT+PUvn\n",
       "/QeTataekvSIpO1b8dialRPHrakieQGwPdP2ecBU0vOEpMNJtUTLkxLsHwP3Suov6fN538+bLllW\n",
       "3j3kZIiU6L6Uy25KusaQktHnSH2afsu8Xywg/d/eaHtaNY8jhFB7kbzUQW4nfxEYTvoAa+nY/wI/\n",
       "I33QtKVGo1KH3dKOjFuSOtDuaXuxXBvxIa3olNlBHs8/m6v+L91frQWA/7WwbxbwcRVlfxF4vSlR\n",
       "sP0s8AfS37NqtqeTmjwOoflOp6XHTgVOJyWJq7R0bKUiyu6/CrxUlvgMyDVV5KRtX9tLAWcCN+ba\n",
       "ivYsQz+K3F+llef1ISWTANuR+gG9mWsJryA1da5DSoxWBl6VNAk4Gthd0qNNBdm+0PYatpch/e/1\n",
       "AZ7I+963PdL2srbXB3qTEt7Z8nOwB9FkFEJdRfJSPwcDX8kfYHORdKakdSX1ybUIhwLP5RqC2YdV\n",
       "eZ3JpFE4LVkU+JTUFNVX0smkETatUVU8SvqRkgVJWrCkb025P5M+ZE/Mz8WXgGHAH/P+y4BdJX0u\n",
       "93/4EfCA7amS1pQ0PH8jX0DSSFIH2D/lOHaVtIakXpKWAs4G7mhqRstlf1PSKpIWAk4g9RGBNFJp\n",
       "TaXh1pK0Gqmz57+re6rmchKwdXMdP5WGBG+c/yb9gO+ROqKW1rK15nWwasn9R4Cpko7Lz1FvSes1\n",
       "NU9KGpmfF0iJrEnJ3Tv55/xeU83Zn9SHpSJJmyr1m+qb4zqeNAKoqUntcWAvSUvnv90oUgLyHGlk\n",
       "2qqkvlufB35JSiq3zWUvmB+jci3QRcDPmzpES1pVaah2b0nDgW8DPykLcVfgPdsT2vD4Qwg1EslL\n",
       "ndh+0fY/SzeV/N6fVG39PvACqf/FTmVFNI32aLodSfPGAXsozWXy8wrHjM+3Z4GXSc01pR+mLouv\n",
       "uW/fhlSLI2lqhetA+nY8jfShskK+1uyRVJKekPQNANufkjoWjyD1e/kVMCrXdGD7PtKH/x+Y8+G8\n",
       "b1NRQCFvf4vUGXf7kiRhuXzdKcA/Sc/1AbMfjH0ZqXnj4ZLnZEzeN5GUUF5A+mCfQGpG+HULj7tZ\n",
       "tifZ/muF3bNISdQ7pOaLr+bHUNpc8e+y18HZFcq6BFgnNxHdnEfq7ED6kH8xX+Mi5iSt2wJP5L/l\n",
       "OcA+uY/TNNJw4wdzWZtozhwoFTvsStqc1DF4niH9ku6QdEK+uyBwPmmE0avAVsB2tj/K+39CSt4e\n",
       "J/3NvgfsbnuK0+i8t/NtMml02fRcewkpCbqa1FT0MPAgKeFt8n+53Cn5Me5r+6mycPdnPrVkIYSO\n",
       "p9T3sA4XTqNCfkNqYzdpzoxzJS0OXEdqk38Z2Mv2B3UJMoQQQghdTj2Tl2WAZWz/K48k+AewC/BN\n",
       "4F3bP81VxoNsn9BSWSGEEELoOerWbGT7Ldv/yr9/RJrkazlS80hTZ7grSAlNCCGEEALQRfq8SFoZ\n",
       "2JDUDj04t1dD6q8wuE5hhRBCCKELqnvykpuMbgK+l4eDzpYnA6tPu1YIIYQQuqTOXpRvLnl4603A\n",
       "lbabJmGbLGkZ228prfj6djPnRUITQghtYLtd8zfF+2/obM29ZuuWvEgSafjmRNulQ3hvJQ1ZPTP/\n",
       "LJ9ZFmj/P2BnkDTW9th6xzE/EWdtRZy11QhxNkKMULvEoxHef0P3UOk1W8+aly+R1o55XFLTSsEn\n",
       "AmcA10s6mDxUuj7hhRBCCKErqlvyYvsvVO5z87XOjCWEEEIIjaPuHXa7uQn1DqBKE+odQJUm1DuA\n",
       "Kk2odwBVmlDvAKo0od4BVGFCvQMIoSep2yR17SHJ0eYaQgitU4v3znj/rZ28KO7FtteqdyzNkfQE\n",
       "cJjt+2t5bCtjaPb1FslLCCH0EJG89Ax57rQXgT55HbOGVen1Fs1GIYQQuh1JNe3TWevyOknFJFNS\n",
       "784MpNYieQkhhNAtSHpZ0nGSHgemSuolaTNJf82roP9L0tYlx68i6X5JUyTdJekCSVfmfStLmiXp\n",
       "IEmvAHfn7QdJmijpPUnjJa1YUt45kiZL+lDS45LWzdtHSHoyX+d1SUfn7cMkvVZy/tqSJuRYn5C0\n",
       "Y8m+y3N8t+dyHpK0aoWnoqnp5oN87GaSDpT0oKSzJb0LFCStKuleSe9KekfSVZIGlj2fX8m/j5V0\n",
       "vaQrcplPSPq/Nh67kaTH8r7rJV0n6ZTW/K0jeQkhhNCd7AMMBxYDlgVuB35sexBwDHCTpCXysb8F\n",
       "HgIWB8aSpu8o70uxFbAWsJ2knUlTeuwKLAk8AFwDIGlbYEtgqO2BwJ7Af3MZlwCjbQ8A1gXuLQ86\n",
       "T9p6GzAeWAo4Arha0holh+2d4xwEPA+cWuE52DL/HGh7gO2H8v1NgBeApYHTSDUzp+bnaW1ghVx+\n",
       "k/LnYsf8eAeS5mQ7v7XHSuoL3AJcmh/HNaQ1DFvVh6URq8FCCCF0USrWaCK8Qpv61Rg41/YbAJJG\n",
       "AnfYHg9g+25JjwLbS5oAbAx82fanwIOSbmXeppaxtqfn8g4BTrf9TL5/OnBSrn2ZCSwKrC3p703H\n",
       "ZDOBdSX9x/aHwGPMazNgYdtn5Pv3Sbod+AZQzNtutv1ovvbVwNkVnodKz92bti/Iv88gJTIv5Pvv\n",
       "SjoHOLnCuQAPND2Xkq4CjmzDsZsBvW2fl+/fIumRFsppViQvIYQQaqaNSUctvVby+0rAnqXNL6TP\n",
       "vXuBIcB7tmeUnbvCfMobJ+lnZccMsX2fpPOBC4CVJN0MHJPX7Nsd+CFwRm7SOqGkNmR2GWXXAngl\n",
       "b4eUmE0u2TcdWITWmat8SYOBccAWpMSrF/BeC+eXXn8a0E9Srwqdgps9lvR43mgmrla9bqLZKIQQ\n",
       "QndSWvPzKmntvEElt0Vt/xSYBCwuqX/J8Ssyr/LyRpeVt3BTImL7PNsbA+sAawDH5u2P2t6F1Bz0\n",
       "O+D6Zq7zJrCCpNIP8ZWY94O+GpVqv8q3nwZ8BqyXm7pG0fF5wSRgubJtKzYTW4sieQkhhNBdXQXs\n",
       "KOnrknpL6pc7yS5n+xXgUWCspAUkbQ7sQMsfor8kNROtAyBpoKQ98+8bS9o0912ZRmqW+SyXvZ+k\n",
       "gbY/A6aSEoZyD+fzjsvnDMvxXJv3t6Zm4h1gFrDafI5bBPgfMEXScuRkq4P9jfS8HC6pT+5H9IXW\n",
       "FhLJSwghhG7J9uvAzsBJwNukmpOjmfPZtx+wOalj7SnAdaT+KbOLKCvvd6RFg6+V9CHwH2DbvHsA\n",
       "cBGp2eVl4F3grLxvJPBSPmd0vu5c17A9k9TJdTgp+TgfGGX72ZLjyhOrZhMt29NIHXEfzKOiNq1w\n",
       "fhHYCPiQ1Fn4pkpltub6LR2bH+duwMHA+6Tn4nbmft7nKyapCyGEHqIW753d+f1X0nXARNvF+R4c\n",
       "akbSw8CFtq9oZl9MUhdCCCE0yU09qynNBzMc2InUJyV0IElbSVomNxsdAKxHGiJetRhtFEIIoada\n",
       "BrgZWII04uUQ2/+ub0g9wpqkTssLk4Zq72F7csunzC2ajUIIoYeIZqPQaKLZKIQQGoQkSaucXjZs\n",
       "NoSQRfISQghdzmq7wXaHwWq71juSELqiSF5CCKGLkIaMltZ6EoadBhcOgGGnS2s9KQ0ZXe/YQuhK\n",
       "osNuCCF0GZMuhv7vQe+fpTnJeveDT4+CSTfXO7IQupKoeQkhhC7CtkGGXovBXhNBg/LmxhtZEUIH\n",
       "iuQlhBC6lL5D4a5vwg3rwd0HpvuhO5K0paSn6x1HI4qh0iGE0EPEUOmeQdLKwItAnworPremrMuB\n",
       "12z/qP2Rten6MVQ6hBBCzyCppn06a11eJ+m2SWYkLyGEELoFSS9LOk7S48DUPO3/ZpL+Kul9Sf+S\n",
       "tHXJ8atIul/SFEl3SbpA0pV538qSZkk6SNIrwN15+0GSJuYFD8dLWrGkvHMkTZb0oaTHJa2bt4+Q\n",
       "9GS+zuuSjs7bh0l6reT8tSVNyLE+IWnHkn2X5/huz+U8JGnVCk/F/fnnB5Km5oUZWx27pNHAvqSV\n",
       "rqdK+n17/j61FMlLCCGE7mQf0srMiwHLklYs/rHtQcAxwE2SlsjH/hZ4CFgcGEta/bm8L8VWwFrA\n",
       "dpJ2Bk4EdgWWBB4ArgGQtC2wJTDU9kBgT9Jq1QCXAKNtDwDWBe4tD1rSAqSVnccDSwFHAFdLWqPk\n",
       "sL1znIOA50krRzdny/xzoO1FbT/clthtXwRcDZyZy9m5wvU6XSQvIYQQakbCtbi18fIGzrX9hu2P\n",
       "ScnIHbbHA9i+G3gU2D7XOmwMnGz7U9sPArcyb1PLWNvTbc8ADgFOt/1M7ktyOvD5XNZMYFFgbUm9\n",
       "8jFv5TJmAutKGmD7Q9uPNRP7ZsDCts/I8dxHSry+UXLMzbYftf0ZKan4fIXnobnmorbGXqm8uork\n",
       "JYQQQs3YqBa3doTwWsnvKwF75maY9yW9D3yJtCDjEOC9nJQ0d26l8saVlNVUszIkJxvnAxcAkyX9\n",
       "StKief/uwAjg5dwstFkz1xnSzPVfydshJWalixdOBxZpppxK2hp7lxTJSwghhO6ktNbmVeBK24NK\n",
       "bova/ikwCVhcUv+S41dkXuXljS4rb2HbDwHYPs/2xsA6wBrAsXn7o7Z3ITUH/Y60onK5N4EVytaz\n",
       "Wgl4ozUPvpmY2xV7hbLqrq7Ji6RLcweh/5RsG5s7ND2Wb9vVM8YQQggN6ypgR0lfl9RbUr/cSXY5\n",
       "26+QmpDGSlpA0ubADrT8Yf1L4CRJ6wBIGihpz/z7xpI2zX1XpgEzgM9y2ftJGpibe6YCnzVT9sP5\n",
       "vOPyOcNyPNfm/a2pjXoHmAWs1p7Y83mTgUodg+um3jUvlwHlyYmBs21vmG/j6xBXCCGEBmf7dWBn\n",
       "4CTgbVLtw9HM+ezbD9ic1IRyCnAdqf/H7CLKyvsdcCZwraQPgf8A2+bdA4CLgPeAl4F3gbPyvpHA\n",
       "S/mc0fm6c13D9kxgR1Jn43dIzTijbD9bclx5YtVsomV7Gqkz74O5mWiTdsR+CbBOLqfLLFNR90nq\n",
       "lCbTuc32+vl+AfjI9s9aOCcmSQohhFaqxXtnd37/lXQdMNF2sd6xhKTS663eNS+VHCHp35IukbRY\n",
       "vYMJIYTQ/eTmktWU5oMZDuxE6pMSuriumLz8AliFNARsElCxBiaEEKolsZbEZRJTJM6SiC9GYRng\n",
       "PlI/lHOAQ2z/u74hhWp0uemObb/d9LukX5Mm7ZmHpLEldyfYntCxkYUQGpHEhqQ+D1sD5wKbAEcB\n",
       "z0j8GPiVzad1DLHD5E6fw+ocRpdl+3bSXCqhwXTFPi/L2p6Uf/8+8AXb+5ad023bXEMItSGxBSlp\n",
       "+Rzw/4CLbT4q2b8BqWZ3OdLMq3faXXNYaK1En5fQaCq93uqavEi6hvRtaEnScKwC6VvC50m9qF8C\n",
       "vmN7ctl58c8TQpiHhICvk5KW5UmjK66w+biF40eQkpvXgKNt/tPcsd1BJC+h0XTJ5KWt4p8nhFBK\n",
       "ohewCylp6QecBlxfbXOQxAKkIawnA78HTrZ5q+WzGk8kL6HRRPISQmgYKmplYBvSHBxTgNfLbm+6\n",
       "4Jk56fgGcALwEWlui9tsZrXpumIQ8APgQFKT0s9tprfnsXQlkbyERhPJSwihy1JRA4EvkxKWbUgr\n",
       "At9NWvl2IVIT0HL55/J80m8Z/jF6Og8e14+FJ09hkwse5POXP0SvWaUJzhsuuE2Jh8RqpCanjUkr\n",
       "8V7bXH8YDdYIBjCGvvRjJjOYwrme7Dvacs3OEMlLaDSRvIQQugwVtQCwKXOSlfWBh4A/AXcBj7vg\n",
       "eWpPJBYFvgM+igWmPcGGl17LiDEf0JTUzH1bjjQEtrzWZq6bC/6o/Dol19sSOBv4FDjK5m+z9w3W\n",
       "CJZnHDux+uwTbuV5Xud7XTWB6WnJi6TLgdds/6jesYS2ieQlhFA3Kkqkxd6akpWtSR3y78q3v7RU\n",
       "SyKxODAG+C6pRuYMmxbn41BRvUiDAZpLbEpvM2kpwflw+Tc455UdoNdpwIPACTYva6jGM3L29Opz\n",
       "XMV4P+fh1Twvna0HJi+XkZKXk6s4dgJpEcdL2nity6lBopRH4L4I9LHnTeB7mkqvty43z0sIofE0\n",
       "13zCYfwd+CpzEhaREpVrgW+7MGdOp4rlimVJc7IcBNwCfNHmuWpiyjU3b+fbP5stPyVVg5g3ofni\n",
       "7N8Hvr4CY3uLmQu/wT2nfoHHDnpaa9/zDwb2Htzs+np96T/vxp4hrYi88mnw8klu4zfjWpRRXmSV\n",
       "x3W1b/INkSDWSyQvIYR2abb55C6G8RKfsgr3kZqCzgKecaG6DyOJVYBjgX2AK4HP27xW69hzPO/l\n",
       "2+MV4ylqAH3/tzzDj1yeda9fj7vOHMlHW64CE+Y9eGb36eDbeqvtBtscBnf9HWjjIn5tL0PShqSF\n",
       "BFcH7qAkIZE0iPRa2oT02fcgaUbdNySdCmwJbCbp58BltsdIGgfsCgwEngOOtP2XZq47GtgXsKQj\n",
       "gXtt7yxpCHBeLvsj4Bzb5+VzNgEuBIYC04GrbB8D3J+L/UASwNdsP9ya56FHsN1wtxR2/eOIW9wa\n",
       "8cbSjGB1xrMOE1id8SzNiHaVtzrjGYvnuQ1lfKvLwmuDfwN+F3wqeOl6P18VYx2w8+Gst9y0uR7z\n",
       "Rjzf3uezQ2OuwXtnc2XAsqNhzSfh4GdgltPPNZ+EZUdXX277ygD6Aq8A3wN6A7uTmgR/nPcvTkpE\n",
       "+gGLANcDt5Scfx9wUFmZ+5Fq5nqRagAnAX0rXP+ypmvl+72AfwA/JCVLqwAvAF/P+/8G7Jd/XwjY\n",
       "NP++EjAL6FXv10tXuFV6zUbNSwg9SIVOpqtpsGhLJ1MV1ZclWLfZnQvQr+pyxP+R5mjZAhgHjGFp\n",
       "fZEB/Ebrds2RPP7wd+drqd4vcvlyp6AV1uYT/4+pi/0/fzC+y8TYeSZdDP3fg94/S60dvfvBp0fB\n",
       "pFbUnLS7jM1I/UTG5fs3Sfp7007b75GaHgGQdBpwb1kZczXV2L665O7Zkn4IrAkVJzIsPf8LwJK2\n",
       "f5Lvv5SXvNmHVBs5ExgqaUnb7wIPN1NGqCCSlxB6kgGMmStxAdiJ1bmKI0jV7FVTUauS+q8s2OwB\n",
       "VTSfSGxFSlrWJc1yO8pmWq2TrDnXq21/Cr/z2R3AHRJ9gUOBH+dlB8bazLdPT3dh29Lqhl6LwV4T\n",
       "YfEV8uaqn+MalDEEeKNs2yvkZEDSQqTFF7cl1aYALKLcI7QpjNKTJR1D6m81JO8bQOoEXo2VgCGS\n",
       "3i/Z1ps5zUIHAz8GnpL0ElC0/Ycqy+7xuuKq0iGEjtK3Qm1IKzuZqqg9SUObf8tLHMitPD/XAbfy\n",
       "AlM4r9lzhSSGSzxA6p9wA7C6zTibaUDlJGsAR7Qmznmtthtsdxistmv7ypmbzUybccBawMfARInj\n",
       "pOprnxpf36Fw1zfhhvXg7gPT/U4tYxJpeHyplZiTkBxNGvG2ie2BpBFvYk5NR3nisiWp39Wethez\n",
       "PQj4kMo1I+VJ1qvAS7YHldwG2N4BwPbztve1vRRpTqEbJfVvppzQjKh5CaEnmcmMCtur6mSqovqT\n",
       "5j3ZBhjhgh+lABosuIoj6Et/ZjKdKZxXXkMi0RvYjVTT0ps0hf+Nbm4K/xolWXOuPWQ0DPgeDOsD\n",
       "Fw6Ab58urXUKTBlnv3lRW8psjs17wPclfgH8FDhE4njS4+zWH0r2xDNK7raps247y/gr8KmkMcAv\n",
       "gB1JTTf35P2LkDrGfihpcdJaeqUmA6uV3F+UNL/Pu5L6kmZxHtDC9ScDq5bcfwSYKuk4UqfdmcDa\n",
       "QD/bj0oaCfzR9jukpMikvi7v5J+rQXUj63qiqHkJoSeZwrmtqSUppaLWItW2DAL+zwU/2rTPk32H\n",
       "n/NwP+lhfs7DSxMXiQUkDgSeJH37PRn4nM21zSYu0O4ka16TLoZPCqkfxez+FCen7bVn86zNLqSm\n",
       "gZOAv0hs0hHXContT0jJ8YHAf4G9gJtKDvk50B94l5To3MnctRzjgD0kvZdHHI3Pt2eBl0mJz6st\n",
       "hHAJsI6k9yXd7DRHyw6khYZfJCUlFzEnAdoWeELSVFJz1j62P7Y9jbTMxYO5rHjdNCMmqQuhh8lz\n",
       "srRYSzLPOUUdQOqTchLw62qGPEv0J/UXOI70DfI04L5qaiAq9Hl5gdcZ09Y+L9Lqu8M2l8J/X0/9\n",
       "Ke4+0H6+jcN5W3NdegOjSB9IE4AT7RY/BDswlp41SV1ofDHDbgih1VTUIsAFpOr3vV1wpVEWc84R\n",
       "A0idV48kjaA43abV81S0JclqOa51ToCZz8ILt6Q+L32H2hPPbGt5rb8+i5D6UBwO/JI0S/DUzrp+\n",
       "iiGSl9BYInkJIbSKitqANBfGX4EjXPD/WjxeLEmawv9Q0lDQM+yKQ0p7LInlSbVQXyM1oV1mNzdV\n",
       "b0dcO5KX0FgieQkhVCVPmf8d4BTg+y74qhaPF0NIfVm+CdwInGnzQocH2uAkNiZ1fl6MtOjj3R1/\n",
       "zUheQmOJtY1CCPOlohYDLiZNr/4lF/xsxWPFqsDxwJ7A5cAGNq93Rpzdgc2jEluTZn39lcRE4Fib\n",
       "p+scWgjrXyJhAAAgAElEQVRdXow2CqGLkyRpldOVFzrpsOsUtQlpAcPJwOaVEheJ9SSuIg0FfRtY\n",
       "0+aoSFxaL890fjOwDml6+gckzstNcCGECiJ5CaHL65iJ1ZqoqF4q6mjgduAYF3y4C55nqLLEFyRu\n",
       "Ae4GngBWtfmRzTsdEVdPYvOxzdmkeUAAnpI4Woqp4kNoTvR5CaGLmjOx2hZ94OI14NvPwl8+reXE\n",
       "aipqSVKTz5LAPi745bljQKSZSH9AWtPlLOCS2TPhhg4hsRawvc3Paltu9HkJjSX6vITQcGqx2F1l\n",
       "Kmor4Grgt8APXfAns/elpGV70rwuSwBnAFfbzKzFtUPLcr+X6PsSQgWRvITQRdVisbvm5NFEJ5CG\n",
       "NR/kgu+cvS9NqLYHKWkxaUjvTZ01lDeEWpJ0OfCa7R/VO5ZqSNoP2N/2trU8tjuK5CWELq1pobo5\n",
       "E6u1pzQV1Qs4F/gisLELfgMgr4o8ijR66B3gRODO7r4eT+j2TJULHUqaAFxp+5K2XKgWiZLtq0m1\n",
       "oTU9tjuK5CWELqwWi901UVELkPq3LA982QV/KLEQ8C3gGOAp4NvA/ZG0hNbKMyKPoS/9mMkMpnBu\n",
       "a2dErkUZzRVb5XEd+pqX1Md282t5hVaL0UYh9AB5NehbSIvCbcdYI3EiacG4YcDuNtva/DkSl9Ba\n",
       "s9eiGsm27MXWjGRblmecBmtEp5YhbSjpn5KmSLoW5qxOLmmQpNslvZ0XX7xN0nJ536nAlsD5kqZK\n",
       "OjdvHyfpVUkfSnpU0hYVrjsa2Bc4Lp//+7z9ZUnHSXqctMJ0b0knSHo+x/ikpF1KyjlQ0gMl92dJ\n",
       "+o6kZ/Mijee3cOzXJT0j6QNJF0j6s6SDq33uGk0kLyF0cypqIPBH4AMu/PdoxvoHwAukuUW+arOb\n",
       "zd/rGmRobAMYM9cimgA7sToDOKKzypDUF/gdcAVp5fMbgN2ZU6Mi0srPK+bbdOB8ANs/AB4Avmt7\n",
       "Udtj8jmPAJ/L5f0WuCFfZy62LyI14ZyZz9+5ZPc+wHBgMdufAc8DW9geABSBqyQNbuGhbQ9sDGwA\n",
       "7CVpnj4ukpbMj/d4YHHgGWBzOrg2qZ4ieQmhG1NRSwP3MXm95xn72bu8vcGTpGHRm9iMsnmyziGG\n",
       "7qDvnBqOsu39O7GMzYA+tsfZ/sz2TTAnKbf9nu1bbM+w/RGpM/rWZWXM1cRk+2rb79ueZftsYEHS\n",
       "lAGVlDdRGTjX9hu2P85l3mj7rfz79aQV1zdtocwzbE+x/RppIsPPN3PMCOAJ27/LsZ4LvNVCmQ0v\n",
       "kpcQuikVtSLvrP0Ql91nfvH4LtDrU2A9m0NsXqx3fKEbmck8kxrm7dM7sYwhwBtl214hJxSSFpL0\n",
       "q9yU8yHwZ2Bg2czVc9VUSDpG0sTcFPM+MBBaPfvxa2Vl7i/psdwM9D6wHmk6gkpKk5BpwMLNHDME\n",
       "5pnhulvPeB3JSwjdkDY9f0euu/EpfvXo0rwy7HbQUJtjbN6sd2yhG5rCudzK83Ntu5UXmMJ5nVjG\n",
       "JGC5sm0rMSchORpYA9jE9kBSrYuYU1tSnrhsCRwL7Gl7MduDgA+p3AG4UhPN7O2SVgIuAr4LLJ7L\n",
       "fKKFMqv1JqkjftN1VHq/O6rraCNJl5La8962vX7etjhwHelF9zKwl+0P6hZkCA1EYjMWeudMtOcW\n",
       "LP/QDXy60LdtptY7rtC9ebLv0GDBVRxBX/ozk+lM4bzWjBSqQRl/BT6VNAb4BbAj8AXgnrx/EVI/\n",
       "lw/z50yh7PzJwGol9xcFPgXezf1cTiB1eK9kMrDqfGJcmJTMvAv0krQ/qealWqXJVqk7SJ2Ndwb+\n",
       "ABwCLNOKchtOvWteLgO2K9t2AnCX7TVIL7oTOj2qEBqIhCS+KnEPfab/jq1O3YjvbLSPn955n0hc\n",
       "QmfxZN/h5zzcT3qYn/Pwtgxxbk8Ztj8BdgMOBP4L7AXcVHLIz4H+pMThr8CdzF1bMg7YI49E+jkw\n",
       "Pt+eJX2Rng682kIIlwDr5OagZqc1sD0R+BnwN1Jz0HrAX0oPKYupvDandP/s322/S1rd/af58a0N\n",
       "PAp83EK8Da3uaxtJWhm4raTm5Wlga9uTJS0DTLC9Vtk5sbZG6PEkegE7kGbDXYx1rxvPbqP2pfcn\n",
       "e7vg++ocXuiCYm2jnkFSL1Jfm31t/7ne8bRHI61tNNj25Pz7ZKClIWQh9DgSfUjfsk4CPgFO5Qf9\n",
       "F2aBGT8FtnfBMew5hB5G0tdJQ7unk/rqADxUv4g6VldMXmZLa7uo2aohSWNL7k6wPaFTggqhTiQW\n",
       "BPYnzeUwifQG9UfG6gjSDLlfccET6xhi6GIkDSNNQhi6v81Jc9H0BZ4Edmkant0dddVmo2G235K0\n",
       "LHBfNBuFnkxiYdK0/ccA/wFOs3kgL7B4MrAfsI0LfqWOYYYGEM1GodFUer3Vu8Nuc24FDsi/H0Ca\n",
       "MTGEHkliKeAfpKnLd7YZnhOXXqQOiLsAW0biEkLoSepa8yLpGtJY+yVJ/VtOBn4PXE+avvllmhkq\n",
       "HZl/6AkkFiXNqHmnzeyVavMCi5cAqwA7uhBTCYTqRM1LaDSVXm91bzZqi/jnCd2dRD/SfA3PAYc2\n",
       "LZaYF1i8jtRfbQ8XPK1+UYZGE8lLaDSRvITQICR6k2ofZwH72HwGoKIGkJpVJwEHuOCZ9YsyNKJa\n",
       "JS+1iieEajTKUOkQGpIGawQDGENf+jGTGUzh3NZO1CUh0uygA4AdShKXpUiTaj0CHOGCP6t1/CFU\n",
       "I744hq4gkpcQakCDNYLlGcdOrD57462spsGilQnMT0irxn7VTrNjqqgVgLuAG4EfudCA1aUhhFBD\n",
       "0WwUQg1oqMYzkm3n2XEV4/2ch1dVhjiStCbJFjbvAqioNYE/Aue64LNrGHLogeK9M3QXUfMSQi30\n",
       "pV+F7f2rOV1iFHAUcycuG5E67f7ABV9ao0hDCKHhRfISQi3MZEaF7dPnd6rE9sBZwJfttPCbitqK\n",
       "1Ex0iAtudpG3EELoqbriJHUhNJ4pnMutPD/Xtlt5gSmc19JpElsAl5MmoHsKQEVtT0pcvhGJSwgh\n",
       "zCv6vIRQI3m00RH0pT8zmc4Uzmups67EBsDdwEibPwGoqP2As4GdXPDDnRN56CnivTN0F5G8hFAH\n",
       "EqsCDwBH2VwHoKK+C5wAbOeCn6xnfKF7ivfO0F1En5cQOpnEMsCfgFNtrssLLP6QtJbXVi74pboG\n",
       "GEIIXVwkLyF0IomBwHjgSpsL8wKLPwO+Amzhgt+qa4AhhNAA5ttsJGkJ2//tpHiqElWfoRFJ9Ccl\n",
       "Lo8DYxir3sCvgaHADi74/XrGF7q/eO8M3UU1o40eknSDpBGS4kUfQhtI9AGuBd4AvsdYLUgaUbQM\n",
       "8PVIXEIIoXrVJC9rAhcD+wPPSzpd0hodG1YI3Uder+giYEHgQMZqYeAO4GPSqKL/1TO+EEJoNK0a\n",
       "bSTpK8BVwMLAv4ATbf+1g2JrKY6o+gwNQ+KnwJbA1xir/qQFFv8JHBYLLIbOFO+dobuYb4ddSUsC\n",
       "+5FqXiYDhwO3AZ8jVXuv3IHxhdDQJI4Ftge2ZKwGkUYZ/R44KRZYDCGEtqlmtNFfSbUtO9t+vWT7\n",
       "o5J+2TFhhdD4JL4JfBfYgrFagrQy9AUu+Kz6RhZCCI2tmtFGvWzP6qR4qhJVn6Grk9gZ+CWwNWO1\n",
       "EKmPy8ku+Nf1jSz0ZPHeGbqLmGE3hBqT2Bq4ARie+7jcTOrfcmN9Iws9Xbx3hu4ikpcQakhiQ+CP\n",
       "wD6MVT/gCmA/F/yn+kYWQrx3hu5jvkOlJW3RzLYvdUw4ITQuidWBPwCHMlaDgctIQ6EjcQkhhBqq\n",
       "ps/LY7Y3nN+2zhTfHkJXI7Es8CBwOmPVB/gBMNwF/6e+kYUwR7x3hu6i4mgjSZsDXwSWknQU0PSC\n",
       "X5TqJrcLoUeQGERqKvo1YzUAOIy0wOKL9Y0shBC6p5aGSvclJSq9888mU4A9OjKoEBqFxEKkeY/u\n",
       "4eQ+TwPjgM1dmGtagRBCCDVUTbPRSrZf6aR4qhJVn6ErkFgAuAV4n5MWPoe+0/4IbOeC/1Hn0EJo\n",
       "Vrx3hu6imknqLm9mPUbb/koHxBNCQ5DoBVwKwKHrn0jfaQ+ShkNH4hJCCB2smuTl2JLf+wG7A592\n",
       "TDghdH15ocWfAauw6bgdGfzEncClLviGOocWQgg9wnyTF9uPlm36i6S/d1A8ITSCE4Gv0nfK1gw/\n",
       "8nzgFeDHdY4phBB6jGoWZly85G4vYGNgQIdFNOe6L5M6B38GfGJ7k46+ZgjzIzEa+BawBScNPBRY\n",
       "A9g6FlkMIYTOU02z0T+BpjfmT4GXgYM7KqASBobZfq8TrhXCfEnsDhRI6xVtChwKbOaCp9U3shBC\n",
       "6FmqaTZauRPiqKRir3gN1XimcK4n+47ODCj0TBJfAX4BfJ2xWgS4iDQJ3Rv1jSyEEHqeapqN+pMm\n",
       "3dqCVBvyAPAL2zM6ODYDd0v6DPiV7Yvn2juSbbmV1TRYRAITOpLExsC1wJ6M1VvAw8B3XZinP1gI\n",
       "IYROUE2z0W9IfU/OJdWE7AtcCezZgXEBfMn2JElLAXdJetr2A7P33gcsyup8zDhJ02xP6OB4Qjcj\n",
       "SbDyafDySa4w4ZHEmqRJ6L7NWD1MeuVd5oKv78xYQ2gLScOAYXUOI4SaqyZ5Wdf2OiX375U0saMC\n",
       "amJ7Uv75jqRbgE1ItT7Jl/PPd3jDH0TiEtpitd1gm8Pgrr8DN5fvlVieNO3/SYzVraRE/jViZFFo\n",
       "EPlL3YSm+5IKdQsmhBqqZo2if+Z1jgCQtBnQoRNxSVpI0qL594WBrwPNL3A3k+kdGUvofqQho6W1\n",
       "noRhp8GFA2DY6dJaT0pDRs85hiVIicsFNpcBxwNrAwe64Fl1Cj2EEALV1bxsDDwo6TVSP5QVgWck\n",
       "/Yc00+4GHRDXYOCWPLNvH+Bq23+a56hbeYEpnNcB1w/d2qSLof970PtnqSW0dz/49CiYdDOAxMLA\n",
       "7cAfbM5SUTsDhwObxsiiEEKov2qSl22Zd9SPm9lWM7ZfAj7f4kF3M5MPOTE664bWsm1pdUOvxWCv\n",
       "ibD4CnmzJfoCNwFPAcerqM8BvwZGxMiiEELoGqpJXn5ie1TpBklXlm/rdF/jWmBdIKZkD23Qdyjc\n",
       "9U144RZYbVfoOzSvV3Q5MAMYzVgtDdwKHOGCY1bpEELoIqpZVfox2xuW3O8DPF7WibdTSTJjWZ00\n",
       "ZHUNF2Iiu9A+eb2ic4ENgO0YKwP3Ane54OjkGLqFWFU6dBcVO+xKOknSVGB9SVObbsDbpG+jdeWC\n",
       "XwBuAY6pdyyhW/gRaS6jnRirGaRJ6N4AinWNKoQQwjyqqXk5w/YJnRRPVZq+PaioFYHHgLVc8Dv1\n",
       "jis0JolDgaOBL9lMVlHHA3sBW0YH3dCdRM1L6C6qSV62Zs7aRrPZvr+jgpqf0n9AFXU+MMMFRw1M\n",
       "aDWJvYCzga1sXlRROwEXktYser2+0YVQW5G8hO6imuTlduYkL/1Ik8X9w/ZXOji2lmIqTV6GAE8A\n",
       "67ngN+sVU2g8EjsAlwDb2DyuojYA7gG2d8GP1De6EGovkpfQXcx3kjrbO9jeMd+2AdYDPuj40KqT\n",
       "E5bLgRPrHEpoEBILSJwB/BLYOScuTSOLxkTiEkIIXVs1M+yWe50002hXcgawX+4DE0JFEquQlplY\n",
       "H9jQ5iEVtSBpeYArXfA1dQ0whBDCfFWzqnTpDLa9SJPHdejyAK3lgt9WUb8CfgB8p97xhK5JYm/g\n",
       "POB04Oc2VlECfgW8BcSQ6BBCaADVTFL3D+b0efkM+K3tBzsupDb7f8AzKupMF/xivYMJXYfEQsA4\n",
       "0uq6w+25ku9jSHO7bBlrFoUQQmOopsNuf2B1UgLzvO0ZnRFYSyp1OlNRRWAlF3xg50cVuiKJ9YFr\n",
       "gX8Ch9lMnb2vqB1J/V42c8Gv1SnEEDpNdNgN3UVLk9QtIOmnwGvAFcBvgNclnSVpgc4KsJXOAbZX\n",
       "UWvWO5BQXxLK87fcC5xpM6oscVmfNNJot0hcQgihsbTUYfcsYHFgFdsb2d4IWBVYjNRE0+W44A+A\n",
       "nxN9F3o0iUHAjcC3SRPP/Wau/XNGFh3pgh+uQ4ghhBDaoWKzkaTngTXsufsBSOoNPGN79U6Ir1kt\n",
       "VX2qqEWB54GvuuAnOjeyUG8SXwKuBn4HHG/z8Vz708iie4AJLviHdQgxhLqJZqPQXbRU8zKrPHEB\n",
       "sP0Z0GU7NrrgqaRao7F1DiV0IoneEj8EbgIOtzmymcRFpD4ubwMn1yHMEEIINdBS8vKUpAPKN0oa\n",
       "BTzdcSHVxIXAF1XUhvM9MjQ8iSHAXcDXgP+zub3CoUeThvqPipFFIYTQuFpqNlqeNHHXdObM6/J/\n",
       "wELArnb91n2ppupTRY0BtnHBO3ZSWKEOJLYndby9ADjN5rNmjytqB9J8LjGyKPRY0WwUuosWh0pL\n",
       "EvAVYF3SUOmJtu/ppNgqqjJ56Qc8B+wRnTK7H4kFSZPN7Q6MtHmg4rFFrUcadbSTC36ok0IMocuJ\n",
       "5CV0F/Od56UrqvYfUEUdAuzqgrfthLBCJ5EYSpq75VXgYJv3Kh5b1FLAw8CPXPDVnRRiCF1SJC+h\n",
       "u2jL2kaN5FJgDRW1Rb0DCbUhMRL4K+lvu9t8Epe+pA6810TiEkII3Ue3rnkBUFHfBA4AvuxCAz7Y\n",
       "AIDEIqR+LZsA+9j8u8Xj08iiS0jzEu0RHXRDiJqX0H1095oXgCuBIaS+O6EBSWxEmt7/E2Dj+SUu\n",
       "2VHARsD+kbiEEEL30u2TFxf8KVAETsnfxkODyFP8fw8YD5xs8y2b/833vKK2Jw2L3skFf9TRcYYQ\n",
       "Quhc3T55ya4FBgLb1TuQUB2JJUlT+O8HbGZzbVXnFbUucBmwuwt+tQNDDCGEUCc9InlxwZ+RZtz9\n",
       "cdS+dH0Sw4DHgKeALWxerOq8opYEbgOOcsF/67gIQwgh1FOPSF6ym4AFgJ3qHUhonkQfiR8DvwW+\n",
       "ZXOczcyqzp0zsug6F3xVR8YZQgihvnpM8pI7bZ5Mqn3pMY+7UUisANwHbA5sZPPHqs9NtWkXAu8D\n",
       "P+iYCEMIIXQVPe1D/DbgY2CPegcS5pDYBXgUuB3Y1uatVhZxJLAxMDJGFoUQQvfXJZMXSdtJelrS\n",
       "c5KOr1W5eZ6Xk4GxKqp3rcoNbSPRT+IC4GxgZ5sz7datWK6ihgPHEiOLQgihx+hyyYuk3sD5pJFB\n",
       "6wDfkLR2DS/xR1LzwjdqWGZoJYm1SdP2L0VqJqq45pAGa4SGarzW1QQN1XgN1ggAFbUOcAUxsiiE\n",
       "EHqULpe8kGZQfd72y7Y/IQ1z3rlWhefalx8CBRXVp1blhurkuVsOBu4nJal723xQ8fjBGsHyjGMk\n",
       "27IXWzOSbVmecVpT+5CaAY+OkUUhhNCzdMXkZTngtZL7r+dtNeOC78vX2L+W5YaWSQwkjSQ6Etja\n",
       "5mKblpdsGMAYdmL1ubbtxOoswQXADS74yo6KN4QQQtfUFWseqlp/SNLYkrsTbE9o5XV+BFytoq5y\n",
       "wVUNxw1tJ7EJcA3wJ2ATm+lVndiXfs0XiIGTahVfCN2RpGHAsDqHEULNdcXk5Q1ghZL7K5BqX+Zi\n",
       "e2x7LuKCH1RRTwMHA79oT1mhMolepKn6jwEOs7mpVQXMZEaz29/lsRhZFELL8pe6CU33JRXqFkwI\n",
       "NdQVm40eBYZKWllSX2Bv0jTxHeFk4Acqqvlv96FdJAYDdwC7kGpbWpe4AEzhXG7l+bm23cYrfMA5\n",
       "NQkyhBBCw+lyyYvtT4HDSaOCJgLX2X6qQ65V8COk1Yq/0xHl92QS25Cm+H+U1L/llbaU48m+g9f5\n",
       "HtfzIPfyCTfyMK9xmCf7jpoGHEIIoWHIrqqLSZciybZrskaRivo8cCewugue74rFoWUSCwCnACOB\n",
       "/W3ubVd5afbcbwGnA991wde1P8oQeqZavneGUE9dsc9L57qQIayCMI/rKj3HFM6Nb/VtI7EKqVPu\n",
       "f4ENbd5pV3lpocWLgVWArVzwxPZHGUIIodF1uWajzjR7DpHhDGYEqzKSbVmB85smQQvVk9iLNOnc\n",
       "dcCONUhctgH+BTwPbBqJSwghhCY9utlIQzWekWw7z47beZkd2NwFt3aNnR5HYiHg58CXgX1s/tGu\n",
       "8opaEDiN1FH7QBd8d/ujDCFANBuF7qNH17xUnEOkDwsAE1XU2Spqmc4NqnFIrA/8HehPmuK/vYnL\n",
       "usAjwMrA5yJxCSGE0JyenbxUnkPkP8B6QG9SEnOOilq2M0PryvIU/4cC9wJn2oyymdrm8oqSijqc\n",
       "NB/FOGAPF/zf2kQbQgihu+nZzUZNfV5Kp5+/lRd4nTFNnXZz0nIccADwG+BMFzypvdduVBKDgF+T\n",
       "OtHuY/Nsu8orajBwKWmBxv1c8HPtjzKE0JxoNgrdRY9OXiAnMAM4gr70ZybTmcJ5zY02yknMscCB\n",
       "wJWkJObNWsTQKCS+BFwN/A443ubjdpVX1PakROgSoOiCP2l/lCGESiJ5Cd1Fj09eWn3t1AfmWOCb\n",
       "wFWkJOaNesTSWSR6AyeSJg/8ts1t7SqvqP7AWcAOwP4u+P72RxlCmJ9IXkJ3EclLW2NIzR3HAgeR\n",
       "Vko+wwXPswZTo5MYQkrSegH72bQrUcuTAv6WNAz6MBf8QfujDCFUoyu8d4ZQC5G8tFNOYo4hLfB4\n",
       "DSmJea2+UdWGxPakJp0LgVNtPmtzWUX1Ao4ETgC+74Kvrk2UIYRqdaX3zhDaI5KXGlFRSzMnibkO\n",
       "OL1RkxiJBUnT8e8OjLR5oF3lFbUccDmwEDDSBb/U7iBDCK3WFd87Q2iLnj1UuoZc8Nsu+DhgLWAq\n",
       "8C8V9QsVtWKdQ2sViaHAX0mjiTasQeKyK2nxy/uBrSNxCSGE0F5R89JB8ro8RwOjgRtINTFtWlm5\n",
       "s0iMBM4BxgIX2rT5xaGiFsllfYU0BPqhmgQZQmizRnjvDKEakbx0sJzEHAV8B7gROK2rJTESiwAX\n",
       "AJsCe9v8u13lFfU5UsL2IDDGBbd5ArsQQu000ntnCC2J5KWTqKglSEnMIcBNpCTm5boGBUhsBFwL\n",
       "PACMsflfu8or6qukjsvfc8HX1CDEEEKNNOJ7ZwjNieSlk+Uk5vvAocDNpCSm0/uBSAgYA/yQlLS0\n",
       "O9FQUfuSmor2cMHt6isTQqi9Rn7vDKFUJC91oqIWZ04S83vgVBf8YqdcWyxJmpJ/GeAbNi+0q7yi\n",
       "ROrfMwYY7oKfbH+UIYRa6w7vnSFAJC91l5OYI4HDgFtJSUy7kokWryeGkZY3uAb4oc3MdpWX5m/5\n",
       "GfA1UuLS7SbqC6G76E7vnaFni+Sli1BRg4Dvkabgv42UxDxfs/JFH+Bk4FvAN23+2O4yi1qQtFjl\n",
       "MsDOMVtuCF1bd3zvDD1TJC9djIpajJTEHAHcDvykvUmMxAqkKflnAKNs3qpBnANJCzS+C4xywTPa\n",
       "W2YIoWN15/fO0LNE8tJF5SRmTL79gZTEPNfqcsQuwK9IHWl/ajOrBrEtB9wJTCBN9d/mZQNCCJ2n\n",
       "J7x3hp4hkpcuLtdwNCUxd5KSmGfne57oR+qLMoLUKbcmk8SpqLVzHBcCZ7nQgC+gEHqonvTeGbq3\n",
       "SF46SJ5m/xJS59gbbNrVHyQnMUeQmpTGk5KYZypce23S3C3PAKPbe+2SGLYgzVFzjAu+shZlhhA6\n",
       "TyO8d4ZQjUheOohEX2A4MArYBriLlMjc2Z4RPipqAHOSmLuAU1zw0/maAr4JnAmcBPy6PVP8l113\n",
       "V1Lz00gX/KeKxw3WCAYwhr70YyYzmMK5nuw7ahFDCKF9GuG9M4RqRPLSCSQGAXuQEpm1SVPnXwk8\n",
       "1NbkIicxh5OGWd/N89uew1XjjwLWI03xP7EmwadrHfr/27v3GDvKOozj36cta6uAl4i12sZiWxOr\n",
       "IigaFS+LBq1Ei/WClhAUDUpUaBRvQOK6oAIaUajRCN5iKog3pCgWi7QRrVYJbUELQpEqSL2itkRq\n",
       "AR//mKGebvfsnt3u7szseT7JZufMmZ19dpuZ/vZ9531fisnsXu0+39D2uJk6mtlcwGLm7965ki3c\n",
       "xbIUMBHVa9q9M6KdFC8TTOJg4DiKQmYasAJYYTOqEUXq1wFsOv4c1px1MjM33cGRfcf685s2jEnW\n",
       "YvK5jwJvABYNN4meFmgVx/OKvd5YwSrf5leORaaIGL0m3zsjWk2rOkC3sbkD+JjEx4HDgeOBn0n8\n",
       "jqI15jKbv3dyLokp4JOBY9l/25tZumQ2sEr9WkPRnTTqmW7Vr/2Ai4CFwBHu81+H/aIeprfZP2O0\n",
       "OSIiIgaaUnWAbmVjm1/ZLANmA2cDLwJ+J3GFxOvLEUODkpgJXAW8BniOd8z6uvt8HjAPuAG4Vv26\n",
       "TP16+kizqV/7UyxZcBDw0o4KF4BdDD7Xyy7uG2mGiIiIdmpXvEj6iKS7JG0oPxZVnWm82dxvc5XN\n",
       "UmAOcDnFmkd3S1ws8eKilaUgcRSwAbgeeInN73efq8/3us+foChirgeuUb++2WkRo349DlgDbANe\n",
       "4z53vsr0di5k5YDur5XcznaWd3yOiIiIYdTumRdJfcAO2+cPcUxX9NtKzOb/z8ccAHwd6AGWAifY\n",
       "XDvsOfr1CIpC6H3AdcBZ7vNNbY6dB1xdfp+PjGYOl3K00Sn0MINd3Md2ludh3Yh66JZ7Z0x+dS1e\n",
       "7rX9qSGO6aoLsBwCfQhFEXMQ8D6bzrpyHjpHUcScTFHErKMoYja1vH84xcKQ/e7zF8Yqe0TUR7fd\n",
       "O2PyqmvxciLwL4puj9PsPRf8ywU4eurXw4F3AB8Afg6cRbGw4teAk9znKyqMFxHjKPfOmCwqKV4k\n",
       "rab4D3OgM4FfwO5WhbOBWbbfNuDra38BShLM/ThsPcN1qxDZXcS8naKImQoscZ/XVZsqIsZTE+6d\n",
       "EZ2oXctLK0lzgSttP2PAfgP9LbvW2l47ccmGJ81/HRz1ZVh9or3lu1XnaUf9mgE8zH0ekyUEIqI+\n",
       "JPUCvS27+lK8xGRQu+JF0izb28rt9wDPsX3cgGNq+9eD9IS3w4HL4IXT4OKnwEm3wk8fgO0X2Hdf\n",
       "VGNvMfYAAAgNSURBVHW+iOhedb53RoxEHSepO0/SoYCBOyiez2iQbRfDjHtg6qdAwNTp8MB7YVtt\n",
       "W18iIiKapHbFi+0Tqs6wL2xbmm+Y8ig4djM8Zk65u15NXBEREQ1Vu+JlcuhZAKtPhNsvh3lLitcR\n",
       "ERExFmr3zEsn0m8bETFyuXfGZFG75QEiIiIihpLiJSIiIholxUtEREQ0Sh7Y7WLlIoqn0sN0drGT\n",
       "7VyYRRQjIqLuUrx0Kc3U0czmAhYzf/fOlczTTJECJiIi6izdRt3qQE7do3ABWMx8DuSUihJFRER0\n",
       "JMVLt+phepv9MyY4SURExIikeGkQSZIOPqdYsXof7WJnm/337fO5IyIixlGKl0aZ91pY9M5i1t59\n",
       "tJ0LWcmWPfat5Ha2s3yfzx0RETGOMsNuA4zXStXlaKNT6GEGu7iP7SzPw7oRk1e33Ttj8spoo0YY\n",
       "n5Wqy0IlxUpERDRKuo0aoFiRWi0rVevRZKXqiIjoUml5aYysVB0REQF55iUiomvk3hmTRbqNIiIi\n",
       "olFSvERERESjpHiJiIiIRknxEhEREY2S4iUiIiIaJcVLRERENEqKl4iIiGiUFC8RERHRKCleIiIi\n",
       "olFSvERERESjpHiJiIiIRknxEhEREY1SSfEi6Q2SfiPpQUnPGvDe6ZJuk3SLpJdXkS8iIiLqq6qW\n",
       "l5uAJcBPWndKWgi8EVgILAI+J6mxrUOSeqvO0InkHFvJObaakLMJGSMmk0oKA9u32L51kLeOAS61\n",
       "fb/trcAW4LkTGm5s9VYdoEO9VQfoUG/VATrUW3WADvVWHaBDvVUH6EBv1QEiukndWjWeANzV8vou\n",
       "4IkVZYmIiIgamjZeJ5a0Gnj8IG+dYfvKEZzKYxQpIiIiJgHZ1dUGktYAp9m+oXz9IQDb55avVwF9\n",
       "ttcP+LoUNBERo2BbVWeI2Ffj1vIyAq0X0krgEknnU3QXLQB+OfALcvFFRER0r6qGSi+RdCfwPOAH\n",
       "kn4IYHsz8E1gM/BD4J2usmkoIiIiaqfSbqOIiIiIkarbaKOOSTpb0iZJGyX9WNKcqjMNRtInJd1c\n",
       "Zv2upEdWnWkwQ00cWAeSFpUTF94m6YNV5xmMpC9L+rOkm6rO0o6kOZLWlP/Wv5Z0atWZBiNpuqT1\n",
       "5fW9WdI5VWcaiqSpkjZIGslghAklaaukG8uce3XHRzRJY4sX4BO2n2n7UOB7QF/Vgdr4EfA0288E\n",
       "bgVOrzhPO4NOHFgHkqYCn6WYuHAhsFTSU6tNNaivUGSss/uB99h+GkW37bvq+Lu0vRM4sry+DwGO\n",
       "lPTCimMNZRlFd3edm7IN9No+zHaT58+KaG7xYntHy8v9gb9VlWUotlfb/m/5cj0wu8o87QwxcWAd\n",
       "PBfYYnur7fuBb1BMaFgrtq8D/lF1jqHY/pPtjeX2vcDNFPMr1Y7tf5ebPcBU4J4K47QlaTZwNPBF\n",
       "9hyAUEd1zxfRkcYWLwCSPibpD8CbgXOrztOBtwJXVR2igZ4I3NnyOpMXjgFJc4HDKIrq2pE0RdJG\n",
       "4M/AmvKB/jr6NPB+4L/DHVgxA9dIul7SSVWHidgXdRgq3dZwE93ZPhM4s5wf5tPAiRMasNTJhHyS\n",
       "zgR22b5kQsO1GMOJAydanZviG0nS/sC3gWVlC0ztlC2Wh5bPiV0tqdf22opj7UHSq4C/2N7QgPWN\n",
       "jrC9TdJBwGpJt5SthRGNU+vixfZRHR56CRW2aAyXU9JbKJqVXzYhgdoYwe+zbv4ItD6QPYc9l5GI\n",
       "EZC0H/AdYIXt71WdZzi2/yXpB8DhwNqK4wz0AmCxpKOB6cCBkr5m+4SKc+3F9rby818lXU7RHZvi\n",
       "JRqpsd1Gkha0vDwG2FBVlqFIWkTRpHxM+RBiE9StX/x6YIGkuZJ6KFYeX1lxpkaSJOBLwGbbn6k6\n",
       "TzuSHivpUeX2DOAoaniN2z7D9hzbBwNvAq6tY+Ei6eGSDii3HwG8nOIh/YhGamzxApwj6aayT7wX\n",
       "OK3iPO0sp3igeHU5RPFzVQcaTLuJA+vA9gPAu4GrKUZ0XGb75mpT7U3SpcA64CmS7pRUSTfmMI4A\n",
       "jqcYvbOh/KjjCKlZwLXl9b0euNL2jyvO1Im6dnHOBK5r+X1+3/aPKs4UMWqZpC4iIiIapcktLxER\n",
       "EdGFUrxEREREo6R4iYiIiEZJ8RIRERGNkuIlIiIiGiXFS0RERDRKipeIASSN+XT5kp4kaelYnzci\n",
       "ohuleInY23hMfnQwcNw4nDciouukeIloQ1KvpLWSviXpZkkrWt7bKuk8STdKWi9pXrn/q5Je13Lc\n",
       "jnLzXOBF5Yy2ywZ8nyWSrim3Z0n6raTHjf9PGBHRTCleIoZ2KLAMWAg8WdILyv0G/mn7EOCzwGda\n",
       "9g/mg8B1tg+zfUHrG7YvB7ZJejdwEfBh238Z458jImLSSPESMbRf2r7bxToaG4G5Le9dWn7+BvD8\n",
       "Yc4z3GKXpwCnAzttXzaaoBER3SLFS8TQ/tOy/SAwrc1xD7W4PEB5XUmaAvR0+H3mlOefWa78HBER\n",
       "baR4iRi9N7Z8XldubwWeXW4vBvYrt3cABwx2EknTgC8BbwJuAd47DlkjIiaNdn9FRnQzt9ke6NGS\n",
       "NgE7gYeGQV8MXCFpI7AKeGjY9SbgwXL/VwY893I68BPb6yTdCPxK0vdt/3YsfpiIiMlGRVd+RIyE\n",
       "pDuAZ9u+p+osERHdJt1GEaOTqj8ioiJpeYmIiIhGSctLRERENEqKl4iIiGiUFC8RERHRKCleIiIi\n",
       "olFSvERERESjpHiJiIiIRvkfO/rTcyrjvGcAAAAASUVORK5CYII=\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x105f9f8d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "## Read test data here\n",
    "a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
    "xt= a[:,0]\n",
    "yt= a[:,1]\n",
    "X = construct_poly(x,8)\n",
    "w = wRidge(X,y,1.0)\n",
    "@interact(n=[1,16])\n",
    "def plot(n):\n",
    "    X = construct_poly(x,n)  \n",
    "    Xt = construct_poly(xt,n) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(x,X.dot(w),'g',label='regression training')\n",
    "    plt.plot(xt,Xt.dot(w),'b',label='regression test')\n",
    "    plt.title(\"MSE train: %f  MSE test: %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.ylabel('Output y') \n",
    "    plt.xlabel('Input x')  \n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conclusion:\n",
    "We get suitable solutions for n from 5 to 7 for the test regression. For higher values we get overfitting. For lower ones underfitting.\n",
    "The MSE of the test regression is always higher than the one of the training regression.\n",
    "That's evident because the regression function is fitted to the values of the training data and so doesn't provide an optimal fitting for the test data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum MSE: 0.6275\n"
     ]
    },
    {
     "data": {
      "image/png": [
       "iVBORw0KGgoAAAANSUhEUgAAAk4AAAEACAYAAABF4/l0AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
       "AAALEgAACxIB0t1+/AAAIABJREFUeJzs3XecVNX5x/HPl4UVFFYgKIpdQGyJJYoNFRsCItgLKkr8\n",
       "SQwqGntJHMZuokZAowmKotgroIhRI5bYgxo7RRELgiIKSIfn98e5C8OwMzs7dcvzfr3ua5lbzjl3\n",
       "2Ln7zKkyM5xzzjnnXPUalboAzjnnnHN1hQdOzjnnnHMZ8sDJOeeccy5DHjg555xzzmXIAyfnnHPO\n",
       "uQx54OScc845l6GcAidJm0h6UdJHkj6UNCja31rSc5ImSfqXpJb5Ka5zzjnnXOkol3mcJG0AbGBm\n",
       "70lqDvwXOAzoD/xgZn+RdBHQyswuzkuJnXPOOedKJKcaJzP7zszei/49H/gE2AjoDYyMThtJCKac\n",
       "c8455+q0vPVxkrQ5sBPwJtDWzGZGh2YCbfOVj3POOedcqeQlcIqa6R4DzjazeYnHLLQF+rouzjnn\n",
       "nKvzGueagKQmhKDpXjN7Mto9U9IGZvadpA2BWVVc58GUc85lwcyUy/X+/HUuM1V91nIKnCQJuBP4\n",
       "2MxuTjg0BjgZuD76+WQVl+f84S8GSYPNbHCpy1EdL2d+eTnzqy6Usy6UEfIX9NSF569zpZTqs5Zr\n",
       "jdNewInA/yS9G+27BLgOeFjSqcA04Jgc83HOOeecK7mcAicze5XU/aQOzCVt55xzzrnaxmcOr96E\n",
       "UhcgQxNKXYAMTSh1ATI0odQFyNCEUhcgQxNKXYAMTCh1AZxztV9OE2DmlLFk3sbunHM1k49npz9/\n",
       "nateqs+J1zg555xrUCSdKekdSYsk3ZXB+a0lPSFpvqRpko6vQV7TJC2W9Kuk/e9KWiFp0+h1F0mv\n",
       "SfpJ0mxJr0raJSmdBZLmJWxDC3G/0TXHSfokuucpkrpkeGyUpBmS5kr6XNJlSelWdzxd2htLGhu9\n",
       "PzMkDZNUlsn95JMHTs4552odSYMkXVOg5L8BrgRGZHj+rcAiYH3gBOA2SdtmeK0BnwMrgy1Jvwaa\n",
       "RceQVAE8BQwBWhFW4IgDi5PS6WVmLRK2QRmWoUb3K+kgwiCvk82sObB3dA9pj0WuBbYwswqgB3CW\n",
       "pO7VHD84w7SHAj8AGwI7AvsCAzN8D/LGAyfnnHO10TDgGEl5X3nCzJ4ws9HA7OrOlbQOcATwZzNb\n",
       "YGb/AUYDJ9Ugy1FAv4TXJwP3AJXNQFuFYtlDFiwys+fM7IMa5JFSTe43EgfiZvZWdP0MM/s2g2OY\n",
       "2UdmtighrWUkzOWY4vj3maQNbAc8ZGZLotVJxkf7isoDJ+ecc7VOtOrE/aQJUCQ9JWlOim1MBtlk\n",
       "0s9rK2CZmU1J2Pc+NfuD/QZQIWnrqGnpWEIwVWkSsFzS3ZK6S2pVk/JKulXSrRmUo9r7jcr3W2B9\n",
       "SZMlfRU1iTVNdywpjb9L+gX4CLjKzCamOX61mU3MMO1ngb6SmknaiFBj9UwG951XHjg555xbjYTl\n",
       "Y8tDUe4GTkl10Mx6mVmrFFvvDNLPpIzNgblJ++YBLTK4NtG9hFqng4CPCc1noRBmc4EuUXmGA7Mk\n",
       "jZa0fsL1Ap5MCg5Pja4/w8zOyKAMmdxvW6AJcGRUph0J69D+idBUmerYqkzMBhLetwOBqyR1TnP8\n",
       "yuh4unwrDQa2J/x/fAW8HdWkFZUHTs4551ZjhvKx5aEo6wFrJ//hzaNMyjgfqEjaty4heMqUEQKn\n",
       "E1izmS6cYPapmfU3s00IwUE74OakNPokBYd31qAMJOeZwsLo5zAzm2lms4GbgJ7VHFtN1OQ4AXiE\n",
       "hP5daY4vSJe2JBFqnB4B1gbaAK0lXZ/BPeWVB07OOedqnahDcWfgKqB/inOeSRpllrg9nUE2mdTA\n",
       "TAIaS+qQsG8H4MMMrl2Vkdl0QkfnHsDj1Zz7GTCSEEDlU7X3a2ZzgK+rPmQ/pTiWThPgl+qOZ5B2\n",
       "G0JT3i1mttTMfiTUSK4RtBWaB07OOedqFUl9gf3NbBihhuHQ5H40AGbWI2mUWeJ2SJr0y6L0GgNl\n",
       "ktZKNazdzH4hBDpXSFo7Gh5/KKEGqaZOje5rYeJOSZ0knRf120HSJoRamNeTi55FnjW638hdhNFu\n",
       "60X9rf5IGPWX6tjYKJ/1oukE1onyPBg4mtCZvtrj6dImjKabAfwhurYlofbu/Wzek5yYWUk2or5/\n",
       "vvnmm2++Zb7l49lZm5+/wO7AiKR9twCn5DGPwcCKpO3yhOPjgIsTXrcCniA0200DjqtBXl8QgqXk\n",
       "/Y2B5cCmhGa5hwg1LvOjn7cBzZPSWUBoIqzcHouO3Q7clsf7bUyYgmEOIVi5GSjP4Fgbwgz8c4Cf\n",
       "gLeA3gnpVnc8ZdrR8d2AV6Lj3wMPAusV8Hexys+JzxzunHN1SD6enf78da56qT4n3lTnnHPOOZch\n",
       "D5ycc8455zLkgZNzzjnnXIY8cHLOuQKQJGmLa6P5Z5xz9YQHTs45VxDtj4DuA6H94aUuiXMufzxw\n",
       "cs65PJLaDZC2/gi6XgN/r4Cu10pbfyS1G1Dqsjnncte41AVwzrn6ZcZwaPYjlN0Y5issawrLzoUZ\n",
       "aWeLds7VDV7j5JxzeWRmBjJo1BKO+RjUKtpdmknznHN55TVOzjmXd+Ud4bn+MPWJ0MepvGOpS+Sc\n",
       "yw+vcXLOuTwz+/g6symPm5mFnx8XfQV3l5qk1pKekDRf0jRJx1dz/pmS3pG0SNJdNcxrmqTFkn6V\n",
       "tP9dSSskbRq97iLpNUk/SZot6VVJuySlsyBpIeOhhbjf6JrjJH0SXTNF0l6ZvhdVXNsl2l8u6c6o\n",
       "DHOj96B7wnXbSPp39B5MlnRYUrobSxobvT8zJA2rZs29gvDAyTnnXK0jaZCkawqU/K3AImB94ATg\n",
       "Nknbpjn/G+BKYEQWeRnwOWHRXgAk/RpoFh1DUgVhEd0hhHXxNgLiwOKkdHrZ6gsZD8qwDDW6X0kH\n",
       "AdcBJ5tZc2Bvwlp5UM17keLaz6PDjYHpwD5mVgH8CXhY0qaSGhMW+x0TvQcDgFGSEmtrhxIW+90Q\n",
       "2BHYFxiY4XuQN75WnXPO1SENZa26aP6rycBeZjYzj+muA/wIbGdmU6J9I4FvzeySaq69EtjYzPrX\n",
       "IL8vgDuAPmbWOdp3Q1SGq4DNCQHNc2bWqpp0TjWzf2ead3Rdje9X0mvAcDNLWbuW6r3I5Nqk898n\n",
       "LEI8GXjdzFokHHsWeNPMLo9efwacbWbjo9d/ASrM7PRM8qopX6vOOedcnRF1pr8fOCnVOZKekjQn\n",
       "xTYmxWVbAcsqg4jI+8B2GRQr22DzDaBC0tZR09KxwKiE45OA5ZLultRdUqoAqsr8Jd0q6dYU19To\n",
       "fqPy/RZYP2ou+ypqEmtaXVlqcG3l+W2j8n2UouyNgO0TXj8L9JXUTNJGQA/gmRTXFox3DnfOObca\n",
       "xZWXpgiL5VyrdTeh6eaGKtM365VFms2BuUn75gEtqjh3jSyzyK/SvUA/4GXgY0KTV0jUbG7UD+gi\n",
       "YDiwgaRxwGlmNis6TcCTkpYlpHm+md1pZmekybem99sWaAIcCXQBlhGa0P4UbSuLncO1SGoC3Afc\n",
       "bWaTotezJF0A3AzsB+wDJNawDQaej+6nLLp2dIr7KBgPnJxzzq0mDwFPvqwHrC2ps5m9lac05wMV\n",
       "SfvWJQQT1cn2fTFC4PQKsAVwT3JaZvYp0B9AUidCjdTNQN+ENPrUtKmOmt/vwujnsMomUkk3sWbw\n",
       "U9V7kdG1khoR3o9FwJkAZrY06gw+jBBAvg08HJ1T2XT7LPAIsBsh8Bsh6XozuyjN/eedN9U555yr\n",
       "daLRVp0J/YCq7FMk6ZmkUWaJ29Mpkp4ENJbUIWHfDsCHGRQr6xonM5tO6CTdA0g7GaqZfQaMZPVm\n",
       "qmzV6H7NbA7wdQbprvFeZHJtFADdSQiKjzSz5QnXf2BmXc2sjZn1ANoDlQFzG0Iz4C1mttTMfiTU\n",
       "SPbMoKx5lXPgJGmEpJmSPkjYN1jS19FQw9WGGzrnnHPpSOoL7G9mwwg1DIdW1U/GzHokjTJL3A6p\n",
       "Km0z+4UQuFwhae2oiexQQg1IqvKURfk3BsokrZXlMPhTo/tamLhTUidJ50X9dpC0CWEU3uvJRalp\n",
       "htncL3AXcJak9aL+Vn8ExkZlq+69SHlt5DZga6C3mSWOGkTSryU1jcp5PqHp7+7o8A/ADOAPURla\n",
       "AicT+msVl5nltBGGGu4EfJCwLwacW811lmvevvnmm28NbcvHs7M2P3+B3YERSftuAU7JYx6tgCcI\n",
       "zVjTgOOSjo8DLk54PRhYkbRdnmFeXxCCpeT9jYHlwKZAO+AhQm3N/OjnbUDzpHQWEJrYKrfHomO3\n",
       "A7fl8X4bE6YwmEMIVm4GyjN5L6q5drPo/OT7OD46/hfCCMB5wNPAlknl3I3Q3DkH+B54EFivgL+L\n",
       "VX5O8jIdgaTNgbFm9uvodQyYb2Y3prnGrJYPh3XO1VJSM5K+tTcUDWU6AudKrRTTEZwl6X2FWUJb\n",
       "FjAf51xDITVCGmwwH+luwnBm55wrmkIFTrcRRg7sSKiqS1nz5JxzGZHW/Z42Ez5g+zM78dmKhzhm\n",
       "x+U0+gjpbMKsw845V3AFCZzMbJZFCDOmdq7qvKgTeeXWtRBlcc7VfefpxpOms8m3T3LYznu2GT5s\n",
       "8u+PO+24TQb9dwfeLZtExzNXoPeop88QSV0Tn5WlLo9zDV2h+jhtaGYzon//EdjVzPomXeNt7M65\n",
       "lCQEdO/PiBv/qnM73bD9RhOvO/zTpTRasS3wDrANK8rKNX2PJUe/sVHFPyaPXt58xaJnGxvnYZbJ\n",
       "cOo6yfs4OVccqT4nOQdOkh4gLLTXBphJGFHXldBMZ4SRAL+3pLWG/IPrnKuKRBmNFx5ZtumLV17D\n",
       "ZRsf//37ax1ztM1+Y1MeI8wi/aLFbLHiEmGUzl7M7njU2kvs4Es+nNb0jHeWrxi9jb18fjeGzF6b\n",
       "ly1mc0p7R/nlgZNzxVGwwClb/sF1ziXSsUdW8NPmV1Lxzcmt2o1f59Enf1m6yfxlsy48iNOe3Ibn\n",
       "LZb+YSXRiNaTT9uq4oVr/j7/yiYdlsxo8odDzZ7pyBfAq8B/ou3z6tKqzTxwcq44PHByztU6iqst\n",
       "i1sczrx2A2nxzfb8sM1P3b5Y8uzYf7+/Z/kKHgcuxGxpjdIU6wDn9eHJ8+/g/5YvXvvnD044ZtmL\n",
       "L23ONsBehDWu/pOwvWsxW5LnWysYD5ycKw4PnJxzJRc1r20N9GZF2eGsaLwDk3vAtH0/YcbOF9v0\n",
       "fVsRJjs8B7P7cspLtGvOvOsu5C+HX8BfVcbyG35ovuz6dufTlhBAhe0LOjKJRaxgHgv4gQVcb1Ps\n",
       "4ZxvtkA8cHKuODxwcs6VhOIqA/YA+gB9WNFoHabt9w2vn7MV0/b7F0vXudrQR8A1wNHAEZi9m7f8\n",
       "xc6d+PSWmzh3u315acE6LPg9MBYzU1v1ZBOGcShbrrzgeYzNmU4HnqUWNu954ORccZRiAkznXAOl\n",
       "uNZRXIcprrsIc7ndwvz1G/Hg429wxdJm3PP8B0zu1dmWrHOMoW+A8YSlm3bNZ9AEYMbEz9h6r0MY\n",
       "1+84Hlz+JZve/xPrvoTUkQoGrRY0ARyIeJdvgY+BXsDLwLeK61HF9UfF1VlxNclnGV1xSTpT0juS\n",
       "Fkm6q4rjrSU9IWm+pGmSjk+TVsbnprh+mqTFkn6VtP9dSSskbRq97iLpNUk/SZot6VVJuySksSBp\n",
       "keOhNShDPt+PbST9OyrnZEmHJR0fJWmGpLmSPpd0Wab55FLOaF/i+7OsJu9RIp80zjkHgNqqJxUM\n",
       "opymLGERcxlqM21cxtfHtQEh0OhDGGn7FjCaVy+6k+ev60tY4X4UsKMZ08NF2omwAOnDwGWYLcvv\n",
       "XQVmGDBaOvSZl9j3zDO5JX4J1/6vgy2YPYUVVVzAEovZEGDIaqP3wnYKsKXi+i+raqRer2+j90pN\n",
       "0iBgAzO7tADJfwNcCRwMNKvi+K3AImB9QkD/tKT3zezjHM+tigGfExb1vQXCYrdRuSx6XQE8Bfye\n",
       "8FlZi7BO7KKENHqZ2b8zzDNZXt4PhYloRwN/Bw4gjLAfK2knM5scnXYt8H9mtkhSJ+AlSe+Y2bMZ\n",
       "5JN1Oc2seUI51wG+I7yXNeZNdc65EDRtzBB602HlzjFM4WvOThU8RQHFNkBvQrC0NfAs4cH5DINt\n",
       "Q+BiQjD1D+BmM2atSkAnAn8DzsCK26dIovW2fHR9WbsDfvfBgJlr1ryPYrxNth4pr49rXULzY2Uw\n",
       "tSth8dTETudfFKJ5r6E01UkSMBnYK3k6mzzmcSWwsZn1T9i3DmGh2e3MbEq0byTwrZldknR9xuem\n",
       "KcMXhImi+5hZ52jfDVG6VwGbEwKB58ysVZo0Ts0hcKpMJ9f3Y3vgdTNrkbDvWeBNM7u8ivw6AS8Q\n",
       "niGf1SCfXMt5MvBnM+tAGqk+J17j5FwdlGvt0BoqGLRa0ATQmw6M4izCyukh37gaA3uyKlhaizC3\n",
       "0uXASxazJRK/Be4EugBDgfZm/LSq8GoC/JUQUO2P2QdZlztLZvwI252mltu91fTRpUMXHfVj05UH\n",
       "xzCVuQxLe33MfiY0L44HiJrudiAEUYcC1wGNFNdYi9mAQt1HfWZmJul+4CTghqrOkfQU4T2vyitm\n",
       "1ruabKoKHrcCllX+8Y28T6g9yeXcdN4ATpK0NSFYPJZwX1dFxycByyXdDTxICESSazhTBsKSbgUw\n",
       "szOqKUeu70dVGgHbJ5Xn78DJhOfHmWY2UaH2OdN8ci3nycA9GZZ/DR44OVfHpKgdaq+2IuvgqZym\n",
       "KfY3U1zNgW6EYOkQ4GtCrdIxwHuVtSoSe2swlxEekjcA/Vhf+1LBg9ouBHjbzeaeD2EAsIDQn6mk\n",
       "zVv20wvD1Vbf6O71B7ej6W86Lfm2fL8fW3/1p0WzXq1ROjFbSpjN/B1Wb97bPP+lLgIpPzVluddq\n",
       "3U0IzKsMnMysV47pV3WfzYG5SfvmAS1yPLc69wL9CH3qPiY0S4VCms2V1AW4CBgObCBpHHCamc0i\n",
       "BBJPSkps6j7fzO6Mrq8uYFqZVRX7anKPnwGzJF0A3AzsB+wDrFYTZmYDJZ1BaNJ/VNJEQhCVaT5Z\n",
       "l1PSZlGZ+pMlD5ycq2syrB2qkSUr+0ok57Ut8C3wJiFYutxiNr3ysIQ0mO7AZUA7Qk1LHzMWVxXg\n",
       "/fIIB9y0iMfOXcgJmC3PpqihCWfza2DapZaHvgZRsDlOoqwt/z1jc/527Y+M++5bbX/p9nw0hCzy\n",
       "iILJadFW99SeZrz1gLUldTaztwqQflX3OR+oSNq3LuGPcC7npmOEwOkVYAtCbchqZTOzT4n+2EdN\n",
       "XKMIwUnf6Po+uTbVJecZyfgezWxp1Bl8GCHIe5vQj2iN50v02Z0g6RFC/66RmeaTYzlPItRGfllF\n",
       "GhnxUXXO1TVpaoeyTnMuQ3mK6avte5b5VHAXsKnF7CCL2S2VQZNEI4kjCTUsNwC3AZ3MGG7GYqDK\n",
       "AG/a0TQ+byPWzTZoCtofAd0HQvvDs09jTWYs/6/9duhJjGp7Jrfcv5QmN0xly+n/02/2zGc+LjOS\n",
       "uhMWiL+KFLUDkp5JGimVuD2dQTZVBcWTgMaSEn93dwA+zPHc9AUxm07oJN6DMGAi3bmfEQKN7Wqa\n",
       "T3XFqGJfje7RzD4ws65m1sbMegDtCQNFUmkC/FLDfHIpZz/Ce5c1r3Fyrq5JVTu0hIU1SSbqr7QX\n",
       "0JuB9OFzmvE0X7KMn5nPd/zMkOSmP4kmhG+HlxC+yV0BjDWrYmhangM8qd0AqDgbujaGv1fAaddK\n",
       "W18Jc4eYffvPbNKsihnzoe//baj9rhjI3x86ndtfeV17vNSGH47uaJNn5ysfl5qkvsCOZnahpBbA\n",
       "FZL+aGar/e5Hf5izSb+M8Ae7MVAmaS1C/5jlZvaLpMejPP8P2JnQb22P5HRqcm6GTgVamtnCaIRa\n",
       "ZXk7EfoEPmhm30jahPA5fCPxtrLMM2/vR5TWrwn9tBoBA4G2hCZXJK1HGG03llALdSBh7rYDzWxB\n",
       "dfnkWk5JexJqxh/J9r0Cr3Fyru6Zy1DGMGW1fRl0aAZQXM0V15GKayRhOO7fCP0CjmJL2tpbtrlN\n",
       "tB1skh2cGDRJNJX4A+FbXX/gLGA3M0ZXGTQBTRdV+a2wxgHeKjOGw9IYlDUNfyPKmsKyy8P+/Jth\n",
       "G07/s125Rz/u6TaNzbduxZzvntYhNyL5c7OAJO1O+EN6IYCZzQOeBI7LYzZ/JvSzuwg4EVhIaG6u\n",
       "NJAw3H0WoUnsdDP7JCrfOEkXZ3JuTZnZ52Y2MXFX9HMeofbtTUnzgdeB/wHnJZw7NqnG7bHKA5Ju\n",
       "k3Rbmqzz+X6cRGjen0no43SQrVo2yYDTCf0kZxOmFjjJzN6uLp9cyxnpBzxmZr+keS+q5dMROFcH\n",
       "RaPqzqKcZixhIXMZlmbagA0JHbt7E+Z+eYPQX2mMxeyrtPmIFoS5Y84F/gtca8Zr1RdQezxezpjT\n",
       "tqbRj0fQeuX+MUzlawZl24ld6nAkHDQCZn8NrTeB508xm5K2WSMfJHQWQ8/txz1Xr8MvC9+i84CT\n",
       "bWRO31qzL0vDmI7AuVJL9TnxwMm5eiYa0bUdYbqA3kBHwrD50cD4aCh9+jREa0Kt0pmEeVauNeP9\n",
       "zAqgAYR+Kb/T+qzINMDLLOltL4Ylk2DqE6GPU3lHs4+vzza9mtpTrzU9lodGHMtDx37Arz+dyM5H\n",
       "XGTXf1as/MEDJ+eKxQMn5+qxqL9SF1YFS5Uz+I4GXo6Gy1efjtiAULt0KqF55HozJmVWCK1FGE3T\n",
       "BTgMs8yuq4PO1U1b7M0rj3fh1R3G0PuJl9j3pHus34Ji5O2Bk3PF4YGTc/WM4mpBWHqgN9AT+JJV\n",
       "wdL/ajJrtcTmwAWEDqejgBtWLouSWQIbAY8R5p45hdAvpd67Wef02oeX716LxS0e4PgrruZP10TL\n",
       "uxSMB07OFYcHTs7VA4qrHav6K3UhdBKt7K/0dY3TE1sTRsj1Av5JWBalZstbSHsDDxFqm67LZt6j\n",
       "Ok3SQxxzxT68fNE77DLnfvr2e8COf7aA2Xng5FwReODkXAllO2lj1F9pe1Y1wXUAnmFVf6XkmXIz\n",
       "LA87A5cSZtAdCtxqRs1m8Q5riQ0kLLdyMmbjsylLfTFRO1fModWDO/Lewfdy0ru3c/pRn9rW0/Kd\n",
       "jwdOzhWHB07OldCq0WDP9a9uFFjUX2lvVq0H14hVTXCvZNpfqepy0IUwfPc3hIkrh4d5i2qcUDPC\n",
       "pJc7A4djNjXbMtU3r2nPndbl50fLWL7ZUAbddxsDT8ln850HTs4VhwdOzpXAqkkbuzSG4VvBaZPg\n",
       "1WXJkzZG/ZW6s6q/0ueENbpGAx/UpL/SmmVAhLXmLgM2Aq4HRq6c4bvmCW5KmNl4CnAqOc6JUi9J\n",
       "eotdf7cWi/9vB/63Zz6bLz1wcq44Un1OfOZw5wpqxnBo9iOU3ZgwaeO5MONxxbURq/or7QX8hxAs\n",
       "XZJNf6VkEo2AwwhNck2Ba4GHzFiW9sL0ie4H3A/cCNzY4PozZcrMOsOdhM05V4944ORcAZmZSR0M\n",
       "GrWEoz9GrTZj3x+PoisXA1sSFuUdARybbX+lZNGyKMcROn3/QphTaUyqGb4zTFTAOVTO2Gv2fB6K\n",
       "6ly1JN0NfGVmfy51WZwDD5ycK4Lyjuz90OPsP6crHzT/hWktNgEuBF7Npb9SMommwClR2l8CZwPP\n",
       "59y/RlobGA5sA+yO2bSc0nOuZoyqF3Vdg6QJwL1mllVNX76CNEmbE5rbG5tZ9l9YXK3kgZNzhTb4\n",
       "k0XAbkAffjP/A3tsXl6btySas2pZlHeBEzNaFiWzxLcAngA+APbCLMt15lxdke0I0HynkZxkhufV\n",
       "tqZj70dWD/lilc4VkOI6mrAQZ3eLWY0mpaw2bdFa4nLCN9vOwCFm9Mpj0HQQYV27EUA/D5oaivZH\n",
       "QPeBYUmb4qchaSdJEyXNlfQgoX9e5bFWkp6SNEvSj5LGKky+iqSrCaNRb4kWuR0a7R8iabqknyW9\n",
       "I6lLinwHAH2BC6PrR0f720l6LMrzc0lnJVzTOUrzZ0nfSbohOvRy9POnKK3davo+uFrMzEqyhaxL\n",
       "k7dvvhVjYzD7MJhZDGaHvKaLbQB2PdhssBFgnfJadpDBhQbfGuxb6vfRt+T/f6wQacCGA6DTR3Dq\n",
       "Z7DCws9OH8GGAzJPN7c0gHJWNTOXAUcCS4ArouOtgcMJwVRz4GHgiYTrXwR+l5TmCUArQkXBucAM\n",
       "oDxF/ndV5hW9bkRY3PpPhBaaLYCpQLfo+OvACdG/1wZ2i/69GbACaFTq3xffst9Sfda8xsm5AlBc\n",
       "2wGPAMdbzDJbHLe6NMVmErcCHxMe0jub8Tsz8rfIrNScMAv4UcBumL2Ut7RdLTdjOCyNhZGfK0eA\n",
       "Xh72Fy2N3Qn9goaY2XIzewx4u/Kgmf1oZk+Y2SIzmw9cA+yblMZqzWNmdp+ZzTGzFWZ2E7AW0ClN\n",
       "GRKv3xVoY2ZXmdkyM/sCuIMw+AJCUNdRUhszW2Bmb1ZVBle/eODkXJ5F0wyMA/5oMXsh5/TE1hJ3\n",
       "AxOBecA2Zpxlxpe5pp2UUQdC09x8YB/Mvspr+q5WM7OosrFRSzjmY1CraHfGzct5SKMdYb3DRF8S\n",
       "BSKS1pb0D0nTJP0MvASsG/pUrSpG4sWSzpf0saSfJM0B1gXaZFiezYB2kuZUboTRqutHx08FtgI+\n",
       "kfSWpEMyTNfVYd453Lk8UlzrEpZEudVidn9OaYmdCHMw7UtYB66D1XRZlMwz60lophgM3E4N/li6\n",
       "+qS8IzzXH6Y+EfonlXcschozCJO0JtqMMNkqhP6CWwGdzWyWpB0JXyhEFaPvFNZRvADY38w+ivb9\n",
       "SOoaoeTf++nAF2a2VZUnm00h9ItC0pHAo5JaV5GOq0dyDpwkjQAOAWaZ2a+jfa0J1f2bAdOAY8zs\n",
       "p1zzcq42U1xrEUagvQT8Net0xF6EWb53IEw02d+yWRYls8waEYKzPwBHYPafguTj6gSzj69LeJl2\n",
       "aaACpfEasEzSIMKSPocSmssqa26bAwuBn6O/M7Gk62cC7RNetwCWAT9IKgcuBirS5D+TML9apbeA\n",
       "eZIuJHwDJPpPAAAgAElEQVR5WUKYlqOpmb0j6UTgWTP7HviZEDCtAL6PfrYHJmd++64uyEdT3V2E\n",
       "pSISXQw8F0XpL0Svnau3FFcjwmdhDnBOTUfPSUiim8RLwL2EpVa2NOOmAgZNFcBjhCVedvWgyZWa\n",
       "mS0FjiDMRzYbOIbwO1rpZqAZ8AMhyHqG1Wt3hgBHRSPubgbGR9skwpf4hYRapFTuBLaNmuUetzAH\n",
       "Uy9gR8Lo1e+Bf7Iq+DoY+FDSPOBvwHFmttjMFgBXA/+J0uqcxdvhaqm8rFUXTfY1NqHG6VPCaJyZ\n",
       "kjYAJpjZ1knXmPlaSa6eUFx/AfYEDrJY5sP2o2VR+hBqfdYmLIvyoOWyLEpmGXcCniTUjp2NWXbr\n",
       "1rmi87XqnCuOYq9V19bMZkb/ngm0LVA+zpWc4hpEaFLYK9OgSaIxq5ZFWUj4djraclkWJVNSH8JM\n",
       "4JdidkfB83POuXqk4J3DzcwkVVmtJWlwwssJZjah0OVxLp8U15GEJU72spj9WO35YVmUkwlrvk0H\n",
       "/gg8Z1aEzqShP9NgoD9wKKuGTrtaTFJXoGuJi+GcixQqcJopaQMz+07ShsCsqk4ys8EFyt+5glNc\n",
       "exM6sHazmKWdGiBaFmUAYVTQe8BJZhSvT5HUEhhF6JuxC6tqhF0tF32hnFD5WlJyh2jnXBEVah6n\n",
       "MYRv1UQ/nyxQPs6VhOLaFngUOMFi9l7ac8X6hNmH9yAsi3JIkYOm7Qijgz4HDvCgyTnnspdz53BJ\n",
       "DxDmmWlD6M90OWFE0MPApqSYjsA7J7q6SnG1I4zo+bPF7N6054oKwjIQT5txeTHKl1SAowi1Yudj\n",
       "NrLo+bu8887hzhVHqs9JXkbVZcM/uK4uUlwVwCvAAxaz69KeG/ozjQM+AwYWpR/TqszLgKuA44Ej\n",
       "Mftv0fJ2BeWBk3PFUexRdc7VO4qrnDCh36vA9WnPFWXAfYT5Zs4sctDUGrifsGDqroTJ+ZxzzuWB\n",
       "r1XnXAK1VU911HhtpwnqqPFqq56wcoLLEYS14galm+BSQoTmsQpCJ/DlxSh7lPkOhEVRPwS6edDk\n",
       "nHP55TVOzkXUVj3ZmCH0psPKnWNor7aCgewDbAEcaDGrLhC6ijDT8AFmFG9iSek4wrIQgzB7oGj5\n",
       "OldAku4GvjKzP5e6LJmQdALQz8wOzue5rvbwPk7ORdRR4zmRNR9go/mEPjQizNU0O20a4hzgdGBv\n",
       "M4pT2yM1Bq4jLFVxOGbvFyVfVxINrY+TpLsIgVO1gyskTQDuNbM7s8zrbupQkOYKy/s4OVedcppW\n",
       "ub8F7YGtMwiaTgTOBboUMWhqQ1hQexlhfqZqJ+F0LhW1VU8qGEQ5TVnCIuYy1GbauGKnUVWyGZ5X\n",
       "0JoASY3NrLDLIblaz/s4OVdpCYuq3D+bdyxmX6S7VKIncAPQ3SztIqL5I+0MvEOYo6mnB00uFyub\n",
       "qk/kYI5hX07kYDZmSGU/v6KlIe0kaaKkuZIehFVfaCS1kvSUpFnRQr5jJW0UHbsa2Bu4RdI8SUOj\n",
       "/UMkTZf0s6R3JHVJke8AoC9wYXT96Gj/NEkXSvofME9SmaSLJU2JyviRpMMS0jlF0isJr1dI+r2k\n",
       "SdGCv7ekObebpM8k/STpVkkvSTo10/fOFYcHTs5VmstQxjBltX1P8y3fc3W6yyT2BEYCh5nxcQFL\n",
       "mJjpScCzwAWYXYJV2+/KufQqGLRa/z6A3nSggrOKlYakcsKEySOBVsAjwJGsqkkScCdhjsBNCes8\n",
       "3gJgZpcRpgo5w8xamNmg6Jq3gB2i9O4HHonyWY2Z/ZMwEvb66Po+CYePA3oALS181qYAXcysAogD\n",
       "oySlW5P1EGAX4DfAMZLW6BKgUHv8CGE5ptaEaUz2oMC1aK7mPHByLmIzbRxfczYP8iIvsIgn+YAv\n",
       "OS1dM4PE9sAThNFzbxS8kFITpCGEiWb3w+yRgufpGoZUTdXlNCtiGrsDjc1siJktN7PHCKNEATCz\n",
       "H83sCTNbZGbzgWsIEzAnWq1Zz8zuM7M5ZrbCzG4C1gI6pSlDcrOgAUPN7BszWxyl+aiZfRf9+2Fg\n",
       "MrBbmjSvM7O5ZvYVYULcHas4pyfwoZk9GZV1KPBdmjRdiXjg5FyigbzKcbTiAK60d+031QRNmwPP\n",
       "AOeYMb7gZQvfaJ8HOhDmZ/qw4Hm6hiNVU/USFhYxjXbAN0n7viQKZiStLekfUfPZz8BLwLqSEoOd\n",
       "1WpoJJ0v6eOo+WsOsC5hpYua+CopzX6S3o2a3uYA2wO/SnN9YgC0AFininPaAV8n7Ut+7WoBD5yc\n",
       "i0QTXD4GvAFcm/bcsP7cv4C/mFH4of9SZ8I375eAQ0lawsi5nFXVVD2GqcxlWBHTmAFslLRvM1YF\n",
       "Q+cBWwGdzWxdQm2TWFVLlBw07Q1cABxtZi3NrBXwM2vWKlVK1Sy2cr+kzYB/AmcAraM0P0yTZqa+\n",
       "BTZOyEeJr13t4aPqnAMUl4A7CN8Gz6xmgssKQk3Tg2Y1+KOSdeF0KiGQG4CZL5jtCsJm2ji1FYzi\n",
       "LMppxhIWMpdhNRkRl4c0XgOWSRpEmET2UGBX4IXoeHNCv6afFWbIjyVdPxNon/C6BWHE6Q9Rv6aL\n",
       "CRPTpjIT2LKaMq5DCKR+ABpJ6keoccpUYqCXaByhY3sf4GnCtCYb1CBdVyQeODkXXA10BA5IN8Fl\n",
       "tP7ck4QOp8kP7fwKD/ohwH7APph9WtD8XIMXBTg5TR2QSxpmtlTSEcBwwkSy4wi1wJVuJnTw/oHQ\n",
       "pHcT0Dvh+BBgpKQ/APcQpgcZD0wCfgH+BmlHvd5J6Dw+B3jRzI6ooowfS7oReB1YEeXzauIprF5z\n",
       "lfwlLPH4yn+b2Q+SjgaGEjrH30cYNVu8SXRdRnwCTNfgKa6BwNmECS5/SHleWH/uYWA5cHxBl1KR\n",
       "NgQeBb4H+mE2t2B5uTqloU2A2VBJakToW9XXzF4qdXkaolSfE+/j5Bo0xXUY8CegRzVBU/HWn5P2\n",
       "JPRnGg8c4UGTcw1DNI9TS0lrAZdGuws/WtfViDfVuQZLce1JaBLobjH7vJrTC7/+XOgM+nvgCqA/\n",
       "Zk8XJB/nXG21B6Epshz4CDiscgoEV3t4U51rkBRXJ8IItVMsZmmnEijK+nNSU8JEfnsAh2E2uSD5\n",
       "uDrPm+qcKw5vqnMuorg2IDSDXZxB0FS5/ly3AgZNGxPNRwPs5kGTc87VXh44uQZFcbUgjNQZYTG7\n",
       "O+25xVh/TtqHMELvceAYwmzIzjnnainv4+QaDMXVhDBS7W1Cn6XU565af+7Qgqw/F/oznUnomH4S\n",
       "Zv/Kex7OOefyzgMn1yAkTHC5GDijmgkuC7v+nNQMuJ3Q2XwPrNqO6c7lnSRfPNa5LHjg5BqKK4Gt\n",
       "gf0tZstSnVTw9efCcg2PE1Y+3xOzX/Keh3PV8I7hzmXP+zi5ek9xnQ4cC/SyWOpApeDrz0n7A28C\n",
       "o4ATPGhyzrm6x2ucXL2muPoAlwN7W8xSjoor6PpzoT/TOcBFQF/M/p3X9J1zzhWNz+Pk6i3FtTsw\n",
       "FuhpMXs75Xlh/blxhOazgWYpV0jPohAqJ8w4/lugD2Zf5i1t1yD5s9O50vIaJ1cvKa6tCIvxnlxN\n",
       "0FRGWEzzB+DMPAdNbQgLlM4BuvhUA845V/d5HydX7yRMcHmZxSzlKu0FXX9O2pbQn+k1wnpzHjQ5\n",
       "51w94DVOrl5RXM2Bp4CRFrM7qzn9Sgqx/pzUHbgHOB+ze/KWrnPOuZLzwMnVG9EEl48A7xIWyk19\n",
       "rjgbOIqw/ty8/BRg5aSWlwKHY/afvKTrnHOu1iho4CRpGjAXWA4sNbPOhczPNVzRBJf/BFYAf6hm\n",
       "gssTgfOALnlbf05qAgwFuhAmtZyWl3Sdc87VKoWucTKgq5n9WOB8nIsD2wH7VTPBZU/gRmC/vK0/\n",
       "J7Um1HQtBPbCbG5e0nXOOVfrFKNzeMphs+qo8WqrnkUog6vHFNcAoC/VT3C5J3A30Cdv689JWwFv\n",
       "EJoH+3jQ5Jxz9VuhAycDnpf0jqTT1jh6IgezMUM8eHLZUly9CbVN3S1ms1Ket2r9uX55W39OOgB4\n",
       "Bbges/Mxy9+oPOecc7VSoZvq9jKzGZLWA56T9KmZvbLy6ItACzqwmCGSFpjZhAKXx9UjiqsHxh3c\n",
       "stE4Zn8zlViK8wqx/px0OjAYOAazl/KSpnNVkNQV6FriYjjnIgUNnMxsRvTze0lPAJ0J39CD/aKf\n",
       "3/ON/eRBk8uc4joYGMmDbf/G7EMvhufGEBbPXf28fK8/JzUGbgIOIvRnmppzms6lEX2hnFD5WlKK\n",
       "rwjOuWIoWFOdpLUltYj+vQ7QDfigypOXsLBQ5XD1j+I6kDfXfoSrO/7CZ71Ogb9XQNdrpa0/ktoN\n",
       "WHlevtefk1oCTwNbAbt70OSccw1PIfs4tQVekfQeYQblp8zsX2ucNYapzM3zoqqu3lJc+wMPsMuC\n",
       "Q1i6/AIoaxrGH5Q1hWWXw4zhsHL9uSeBtyBVI15NMlYH4HXgU6AXZj/nnKZzzrk6p2CBk5l9YWY7\n",
       "Rtv2ZnbtGieNYjxfM8hmpl4Ww7lKiqsr8BBwlF1hr4AMGrWEYz4GtQLMzCxh/bnZ5GP9OWlf4FVg\n",
       "CGZnY6mnO3DOOVe/lXTmcJtsPUqZv6s7FNfehLmSjrFYZWfs8o7wXH+Y+gS0PxzKOyatP9cr5/Xn\n",
       "pFOBa4C+mL2QU1rOOefqPFnqCZYLm7FkZpZyjifnKimuvQhTCRxvsfTBi8RVwMHA/jktpSKVAX8B\n",
       "DiU0zU3KOi3n8sifnc6Vlq9V52o1xbUHIWg6MYOgKT/rz0kVwAPAWsBumM3JOi3nnHP1SjFmDncu\n",
       "K4prN2A00M9iVQwsSDx31fpz3XJaf07aAngNmA708KDJOedcIg+cXK2kuHYFxgL9LWZpJ61MWH+u\n",
       "e07rz0ldCEHT7cBAzJZmnZZzzrl6yZvqXK2juH4LPAWcajF7Ou25Yh/C+nO9c1p/TjoZ+CtwEmbP\n",
       "Zp2Oc865es0DJ1erKK6dgHHAAIvZ2JTniUbABcC5QN+s15+TGhFGzR0F7IvZJ1ml45xzrkHwwMnV\n",
       "GoprB8JM33+wmI1OeZ7YALgHWBvYNevmOak5MApoSegEPjurdJxzzjUY3sfJ1QqK69fAeOAsi9ka\n",
       "a86tPE90AyYCbwBdcwiaNiVMavkD0M2DJuecc5nwGidXcoprO8JCvOdYzB6p8hxRDlwF9AVOMOPF\n",
       "7DPU7sBjwA3AzZRqMjPnnHN1jgdOrqQU17bAc8B5FrOHqjxHbEmYV2kWsKMZP2SfofoCNwP9sfQd\n",
       "z51zzrlk3lTnSkZxbU0Imi60mN1f5TniOEKz3P2EkXPZBU1SI6QrgauB/T1ocs45lw2vcXIlobi2\n",
       "Ap4HLrWYjVrjuFgHGArsTZifaWL2mWkdYCSwAaET+Kys03LOOdegeY2TKzrF1QF4AbjcYjZyjeNi\n",
       "B+AdQmD/2xyDpo2Al4FfgAM8aHLOOZcLD5xcUSmu9sC/gSssZiNWOyYkcSahJupqM07Occ25XYA3\n",
       "gYeAUzBbnH3JnXPOOW+qc0WkuLYgBE3XWMyGr3ZMtAZGABsDe5oxObfMdDTwd+A0zJ7MKS3nnHMu\n",
       "4jVOrigU1+aEoOkvFrPbVzsm9gbeA6aSa9AkCenPhKkGDvKgyTnnXD55jZMrOMW1KSFousliduvK\n",
       "/aIM+BNwOnCqGeNyy0jNCLVWWxA6gX+XU3rOOedcEg+cXEEprk2AF4GhFrNhK/eLjQnLnSwndAD/\n",
       "NreMtCHwJDAF6IrZopzSc84556rgTXWuYBTXRoSapr9bzG5euV/0Joya+xfQLQ9B006EuZ7GACd6\n",
       "0OScc65QvMbJFYTi2pAQNA23mN0IINEU+AvQGzjCjNeyS1xrA+2BjsB2wFnAQMwezUPRnXPOuZQ8\n",
       "cHJ5p7g2IDTPjbSY/QVAohPwIKED+E5mzEmfiNZiVXCUvLUBvgAmR1s3zN4ryM0455xzCVSq9U0l\n",
       "mZmpJJm7glFcbQlB0wMWsyslBJxCqGn6E/BPM8IvndSE0JF7K9YMjjYApgOTWBUgVW5fYba8eHfl\n",
       "XO3hz07nSssDJ5c3imt9QvPcoxazwRIV5Sy+fUs+3+V6LrqxN2PLWT042hj4hjUDo8nANMyWleRG\n",
       "nKvF/NnpXGl54ORyJ5Xd92t+88h2PHbgVD49420++4E2u8ylYrdNmd6ojOXfNsI+Y83g6AvMlpS0\n",
       "7M7VMf7sdK60PHBKJ6xzdiXwAPBig64BkRoB7aiiz5HBljPXoeznpszoOJtnH+T4DZ/g8L1bMG/w\n",
       "CE693Ue5OZc/deLZ6Vw95oFTOlIroD9wHLAZ8AghiHodsxWlLFpBSCL0LaqqQ3Z7YB6hpmhlv6OX\n",
       "N2Nmr77cOm8txnP1vBtZ2nwksC7Q14xpJbgL5+q1OvHsdK4e88ApU1IHQgB1PNCcsHDsA8B7lOpN\n",
       "zEYIjtpQdXDUEVhE1X2OpmA2d7Wk4moFPAe8RHzpM1jjkcDdwGAzlhblfpxrYOrcs9O5eqZggZOk\n",
       "7sDNQBlwh5ldn3S8bn74Q+CxPSGAOg5YQhhm/wBmn5WyaKuRWpM6ODKSao5WbmY/ZZR8XC2Bf7Gi\n",
       "7A2uWPILNDoJONmMF/J+L865lerss9O5eqIggZOkMuAz4EDCqKm3gePN7JOEc+r+hz8EUZ0JQdQx\n",
       "wExCLdSDmE0vQv4VVB0YbQU0oeqao8nA7FxqyRRXBfAv5q//KTfM6ASN5gCnmDEr5TVt1ZMKBlFO\n",
       "U5awiLkMtZmW29p0zjVA9eLZ6VwdVqjAaQ8gZmbdo9cXA5jZdQnn1K8PfwgW9yEEUUcQAscHgEcw\n",
       "m5lDus2BDlQdIDUnrM1W1VxHswrRhKi4WgDPMmOnufzjnZ2h0XXAzWak7POlturJxgyhNx1W7hzD\n",
       "FL7mbA+enKuZevfsdK6OKVTgdBRwsJmdFr0+EdjNzM5KOKf+fvilcqAboSmvF6HG7QHg8SqbwqRm\n",
       "rDlLduWkkK0Is21XVXP0bTH7VymuFqwoe5YpB7fkgTFNsLLjzXin2us6ajwncvAaB0Yx3iZbj4IU\n",
       "1rl6ql4/O52rAwq15EpGf8wlDU54OcHMJhSkNMUW5iZ6CngqWletFyGI+hvSBOB1YHNWBUnrA9NY\n",
       "1e/ov4R+U5OBr2vDCD7FtQ6LW7zIpF7tefyep7GygWbMrf5KoJymKfY3y2cZnauPJHUFupa4GM65\n",
       "SKECp2+ATRJebwJ8nXySmQ0uUP61h9kC4GHgYaR1gcOB3wAfAk8QgqPptXmOKF3Qdm2WbTaRL/fe\n",
       "lNF3nm4rGo+sUQJLqHoepyUszEf5nKvPoi+UEypfS4qVrDDOuYIFTu8AHSVtDnwLHEvo+9Owmf1M\n",
       "GK5fZ2inu9qx20bvMWeLMsbfvJMtL/+0xonMZShjaJ/Ux2kqcxmWz7I655xzhVaQwMnMlkk6E3iW\n",
       "MB3BnYkj6lzdoNZT96PP3eNY3Hwa7528sy34VVY1RDbTxqmtYBRnUU4zlrCQuQzzjuHOOefqGp8A\n",
       "061Booy1fvozxx5xCS2nv07rqQdYzJaXulzOOX92OldqhWqqKyqfIyh/JDai8YL7OKHXr2n7v/Gs\n",
       "Ne9ID5qcc865oM4HTinmCGqvtsKDp5qR6EXZojsY0Pln1vv438iOt1jt7bTunHPOFVujUhcgZxUM\n",
       "Wi1oAuhNByo4K8UVLonEWhI3U7b4Vs7ZYjLrf/QRsr4eNDnnnHOrq/M1Tj5HUG4ktgIepGzRdC5u\n",
       "9T+aLFoOHGcx80V6nXPOuSR1v8bJ5wjKmkQ/4D80mT+CPzVbRpNFAMdYzJaUuGjOOedcrVT3A6cw\n",
       "R9CU1fb5HEFpSbSQuBe4mLW/78ZlLfZBNAOO8qDJOeecS61eTEcQjarzOYIyIPFbwrp5L7Hlc+fR\n",
       "r9s/gJbA4RazqmvvnHO1hk9H4Fxp1YvAyVVPQsA5wCXAWQzWo8BIwjp5fSxm3rTpXB3gz07nSqvu\n",
       "dw531ZJYj7DUy6+A3Ris6cAIYEPgUA+anHPOuczU/T5OLi2J/YF3gQ+AvRmsL4E7gE0JQdOCUpbP\n",
       "Oeecq0u8xqmekmgMxIH+wMlmPKe4GgH/BLYEenrQ5JxzztWMB071kMRmwP3AfGAnM2ZGQdNtQCeg\n",
       "h8Xsl1KW0TnnnKuLvKmunpE4EngbeBLoEQVNAm4BtifUNM0vZRmdc865usprnOoJhXmY/gYcBPQy\n",
       "4y2AKGgaBuwEHGwxm1e6UjrnnHN1m9c41QMS2wFvARWEprnEoOlvQGegu8VsbulK6ZxzztV9HjjV\n",
       "YRKSGABMAG4CTjBjLqwMmm4AugDdLGY/l6ygzjnnXD3hTXV1lERLYDiwFbC3GZ+uPBaCpuuB/YAD\n",
       "LWY/laaUzjnnXP3iNU51kMQehLmZvgN2qyJougboBhxkMfuxNKV0zjnn6h+vcapDJMqAi4CzgQFm\n",
       "jF7teAiargQOAfa3mM0ufimdc865+ssDpzpCYkPgXqAc2MWMr6o4bTDQhxA0/VDE4jnnnHMNgjfV\n",
       "1QESPYCJwCvA/lUFTYrrcuAo4ACL2fdFLqJzzjnXIHiNUy0mUQ5cCxwNHGvGy1WeF9elwPHAfhaz\n",
       "WUUsonPOOdegeI1TGhLtJZ6T6CfRosh5dwD+A3QgzM2UKmi6CDiZ0Dz3XRGL6JxzzjU4Hjil9y1h\n",
       "yP/RwFcS90l0jxbQLRiJE4DXgZHAYWZU2clbcZ0P/B8haJqRt/zbqqc6ary20wR11Hi1Vc98pe2c\n",
       "c87VZTKz0mQsmZmpJJlnQWI94FjgJGAz4AFgFDDRjLy8iRLNCWvK7Q4cZ8Z7Kc+N64/AGUBXi9nX\n",
       "+cgfQtDExgyhNx1W7hzDFL7mbJtp4/KVj3MuO3Xt2elcfeM1Thky43szbjFjN2AfYB7wCPCRxCUS\n",
       "m+WSvsROwH+BFcBvqwmazgbOJPRpylvQBEAFg1YLmgB604EKzsprPs4551wd5IFTFsyYZMblQHvg\n",
       "NEIN1ESJCRKnRrN6ZyRaNuVs4F/AYDN+Z8YvKc+P6wzgHELzXFVTEuSmnKYp9jfLe17OOedcHeOB\n",
       "Uw7MMDP+Y8bpQDtgCNAT+FLiYYne0ci4Kkm0AUYDJwK7m/FAuvwU1+nABYSg6cu83UiiJSxKsX9h\n",
       "QfJzzjnn6pCCBE6SBkv6WtK70da9EPnUJmYsNuMJM44EtgCeJwQ530jcKrG7xMp+CRJdCcumfAbs\n",
       "ZcbUdOkrrgHApYSg6YtC3QdzGcoYpqy2bwxTmcuwguXpnHPO1REF6RwuKQbMM7Ob0pzTIDo4SmwB\n",
       "nEDoVN6I0KG8HOgP/M6M8dWmEdephFnB97OYTanm9JyprXpSwVmU04wlLGQuw7xjuHO1Q0N5djpX\n",
       "WxUycJpvZjemOadBffij2qZdCc1y6wF/NKPaeZcU1ynAVYSgaXJBC+mcq/Ua2rPTudqmkIFTf+Bn\n",
       "4B3gPDP7Kekc//BXQ3H1I8wcvr/F7LNSl8c5V3r+7HSutLIOnCQ9B2xQxaHLgDeAyvXSrgQ2NLNT\n",
       "k66v9R9+SYLNr4Fpl1qRJ7xSXCcAfyWsPfdJMfN2ztVedeHZ6Vx9lvUM2GZ2UCbnSboDGJvi2OCE\n",
       "lxPMbEK25SmM9kfAQQPhubeBx4uVq+I6DrgBONCDJucaNkldga4lLoZzLlKoproNzcISIJL+COxq\n",
       "Zn2Tzqm135qkdgOg4mzo0hiGbwWnTYJXl8HcIWbf/rOgecd1NDAU6GYx+6CQeTnn6p7a/Ox0riEo\n",
       "1Jpr10vaETDgC+D3BcqnQGYMh2Y/QtmNIKCsKSw7F2YUtNZJcR0JDAMO9qDJOeecq30KEjiZWb9C\n",
       "pFssZmZSBwP9ijbdV/DTRu3oPOcguvEmkN8lTiKK63DgVqC7xez9QuThnHPOudz4Ir8pSNteDEsm\n",
       "cfHUf/NCmz9D2X4cMnMz4EtCn62xwESL2Yqc84qrNzAc6GExm5hres65+qu2Pzudq+88cKoBxdUY\n",
       "2BM4NNoqgKcJQdTzFrMFWaTZCxgB9LSYvZPH4jrn6qG6+Ox0rj7xwCkHiqsjq4Ko3wIvE4Kopyxm\n",
       "32RwfQ9gJHCoxezNQpbVuf9v7+5D9azrOI6/P84Om5lpFJY6UGxCCvlASKR/HAl1STgWlAVSGdgf\n",
       "Vo6SKPWPMSKUokwN/8geoGRWlMrMfFg+kD2wkGZZU3SCpLWsMFqRMs1vf9xXebadc3adnZ37d9/n\n",
       "vF8wdl/Xfc51f7jHdfHd7/e7vpcWh8Vw7ZTGmYXTAZINOQJYzaCIWs1gUfz/pvS27jmllw05F/gO\n",
       "sKbW1y+HHFfSmFps105p3Fg4LYBuSu8MXhmNOpRXpvTuZTDdtxFYW+vr561ySho/i/naKY0DC6ch\n",
       "yIacwCtF1GnALgZF04PzPvbggbyXMsFydvECO7nOB/JKi9dSunZKo8jCaciyIa8DDqn1Ne+2Bjky\n",
       "53EM13I+b/7/zk1s5xnWWTxJi9NSvXZKo8LCaYxlVe7iQs7d642buKueqHc1iCRpgXntlNo6qHUA\n",
       "zcMEy2fYv2LISSRJWhIsnIYsSZLjrkoy//8x7uKFGfY/P+9jS5KkvVg4Dd3x74HVl8Dxa+d9qJ1c\n",
       "xya277ZvE0+yk+vnfWxJkrQX1zgNSXLUR+GwdXDmwXDjCXDx4/Czl2DntVV/+tp+H3dwV90nmGAF\n",
       "u3ienVzvwnBp8Vpq105p1CzIQ341nR03wornYNmXIMCy5fDSp2DHLfM5alckWShJkjQETtUNSVUV\n",
       "pOCgw+F92yBHdLvbDPlJkqQ5c8RpqCZWweaL4MlbB2ucJla1TiRJkvpzjZMkjRGvnVJbTtVJkiT1\n",
       "ZOEkSZLUk4WTJElSTxZOkiRJPVk4SZIk9WThJEmS1JOFkyRJUk8WTpIkST1ZOEmSJPVk4SRJktST\n",
       "hZMkSVJPFk6SJEk97XfhlOS9SX6f5D9JTtvjvcuTPJHksSTnzD+mJElSe/MZcXoEWAv8dOrOJCcC\n",
       "FwAnAquBG5KM7chWksnWGfow54FlzgNrHHKOQ0ZJ7e13QVNVj1XV49O8tQa4uaperKqngO3A6fv7\n",
       "OTOrpFcAAAQ3SURBVCNgsnWAniZbB+hpsnWAniZbB+hpsnWAniZbB+hhsnUASaNvIUaCjgKembL9\n",
       "DHD0AnyOJEnSUB0825tJNgNvnOatK6rq9jl8Ts0plSRJ0ghK1fxqmiT3A5dV1a+77c8CVNXV3fZd\n",
       "wPqq2rLH71lMSdJ+qKq0ziAtVbOOOM3B1JN4E7AxyZcZTNGtAn615y944kuSpHEzn3YEa5M8Dbwd\n",
       "uCPJnQBVtQ34PrANuBO4pOY7rCVJkjQC5j1VJ0mStFQ07a+U5HNJfpPk4ST3JlnZMs9MknwxyaNd\n",
       "1luSvLZ1punM1pR0FCRZ3TVFfSLJZ1rnmU6SbyZ5NskjrbPMJMnKJPd3/9a/S3Jp60zTSbI8yZbu\n",
       "/N6W5KrWmWaTZFmSrUnmcuPLUCV5Kslvu5x7LYGQtPBaN6b8QlWdXFWnALcB6xvnmck9wElVdTLw\n",
       "OHB54zwzmbYp6ShIsgz4KoOmqCcCH0jylrappvUtBhlH2YvAJ6vqJAZT5R8bxe+yql4AzurO77cC\n",
       "ZyU5s3Gs2axjsMRglIfhC5isqlOrapz740ljq2nhVFX/nLJ5KPC3VllmU1Wbq+rlbnMLcEzLPDOZ\n",
       "pSnpKDgd2F5VT1XVi8B3GTRLHSlV9SDw99Y5ZlNVf66qh7vX/wIeZdA/beRU1b+7lxPAMuC5hnFm\n",
       "lOQY4Dzg6+x+s8soGvV80qLWesSJJJ9P8gfgQ8DVrfP08BHgx61DjKGjgaenbNsY9QBIcixwKoOC\n",
       "fuQkOSjJw8CzwP3dzSOj6Brg08DL+/rBxgr4SZKHklzcOoy0FB2odgQz2lcTzaq6Eriy6/90DXDR\n",
       "QmeaTp9mn0muBHZV1cahhpviADYlHbZRnv4YS0kOBX4ArOtGnkZON1J7Srcu8O4kk1X1QONYu0ny\n",
       "buAvVbV1DJ5Xd0ZV7UjyBmBzkse6UVJJQ7LghVNVnd3zRzfScCRnXzmTfJjBUP47hxJoBnP4PkfN\n",
       "H4Gpi/9XsvujeTQHSV4F/BC4qapua51nX6rqH0nuAN4GPNA4zp7eAZyf5DxgOXBYkm9X1Qcb59pL\n",
       "Ve3o/v5rklsZTIFbOElD1PquulVTNtcAW1tlmU2S1QyG8dd0C17Hwaitg3gIWJXk2CQTwAUMmqVq\n",
       "jpIE+Aawraq+0jrPTJK8Psnh3esVwNmM4DleVVdU1cqqOg54P3DfKBZNSQ5J8pru9auBcxjcECJp\n",
       "iFqvcboqySPdGohJ4LLGeWZyPYPF65u724BvaB1oOjM1JR0FVfUS8HHgbgZ3Ln2vqh5tm2pvSW4G\n",
       "fgGckOTpJE2mjvfhDOBCBnepbe3+jOKdgG8C7uvO7y3A7VV1b+NMfYzqtPKRwINTvs8fVdU9jTNJ\n",
       "S44NMCVJknpqPeIkSZI0NiycJEmSerJwkiRJ6snCSZIkqScLJ0mSpJ4snCRJknqycJIkSerJwkmS\n",
       "JKmn/wJn2xIYUxs/NQAAAABJRU5ErkJggg==\n"
      ],
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1052c8e50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# optimal d = 6\n",
    "def plot():\n",
    "    X = construct_poly(x,6)  \n",
    "    Xt = construct_poly(xt,6) #n=d\n",
    "    w = wRidge(X,y,1.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 1.0'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    #plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
    "   \n",
    "    \n",
    "    w = wRidge(X,y,0.1)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 0.1'+ \"  MSE: %f\" % MSE(X.dot(w),y))\n",
    "    \n",
    "    \n",
    "    w = wRidge(X,y,10.0)\n",
    "    plt.plot(xt,Xt.dot(w),label='$\\lambda$ = 10.0'+\"  MSE: %f\" % MSE(X.dot(w),y))\n",
    " \n",
    "  \n",
    "    \n",
    "    plt.plot(xt,yt,'b*',label='data test')\n",
    "    plt.plot(x,y,'go',label='data trainig')\n",
    "    plt.legend(bbox_to_anchor=(1.05, 1), loc=2, borderaxespad=0.)\n",
    "plot()\n",
    "print \"Minimum MSE: 0.6275\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Hand in printed copy of completed notebook."
   ]
  }
 ],
 "metadata": {
  "annotations": {
   "author": "",
   "categories": [
    "intelligent-systems-1-2015"
   ],
   "date": "2015-04-30",
   "location": "Beginning of next lecture",
   "parent": "IS_SS2015",
   "submission_date": "2015-05-07",
   "subtitle": "Exercise Sheet 3, Linear Regression",
   "tags": [
    "IntelligenSystems",
    "Course"
   ],
   "title": "Intelligent Systems 1 - Summer Semester 2015"
  },
  "celltoolbar": "Edit Metadata",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}