diff --git a/is/UB3/ExerciseSheet3final.ipynb b/is/UB3/ExerciseSheet3final.ipynb new file mode 100644 index 0000000..4480cec --- /dev/null +++ b/is/UB3/ExerciseSheet3final.ipynb @@ -0,0 +1,1261 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "preamble": true + }, + "source": [ + "(Defining latex commands: not to be shown...)\n", + "$$\n", + "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n", + "\\DeclareMathOperator{\\minimize}{minimize}\n", + "\\newcommand{\\real}{\\mathbb{R}}\n", + "\\newcommand{\\normal}{\\mathcal{N}}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#Exercise Sheet 3\n", + "Maximus Mutschler & Jan-Peter Hohloch" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Gaussian Algebra (25 Points)\n", + "\n", + "Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n", + "\n", + "\\begin{equation}\n", + " \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n", + "\\end{equation}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Proof\n", + "\\begin{align*}\n", + " f(x) &= \\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\cdot e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}}\\cdot \\frac{1}{\\sqrt{2\\pi s^2}}\\cdot e^{\\frac{-(x-m)^2}{2s^2}}\\\\\n", + " &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\alpha}\\\\\n", + " \\text{where }\\alpha &= \\frac{s^2(x-\\mu)^2+\\sigma^2(x-m)^2}{2\\sigma^2 s^2}\\\\\n", + " &=\\frac{(s^2+\\sigma^2)x^2-2(s^2\\mu+\\sigma^2 m)x+s^2\\mu^2+\\sigma^2m^2}{2\\sigma^2s^2}\\\\\n", + " &=\\frac{x^2-2\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}x+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n", + " &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n", + " &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}+\\underbrace{\\frac{\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}_\\beta\\\\\n", + " \\beta&= \\frac{\\frac{s^4\\mu^2+s^2\\sigma^2\\mu^2+s^2\\sigma^2m^2+\\sigma^4m^2-\\left(s^4\\mu^2+2s^2\\sigma^2\\mu m+\\sigma^4m^2\\right)}{\\left(s^2+\\sigma^2\\right)^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n", + " &=\\frac{s^2\\sigma^2\\left(\\mu^2-2\\mu m+m^2\\right)}{2\\sigma^2s^2\\left(s^2+\\sigma^2\\right)}\\\\\n", + " &=\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}\\\\\n", + " \\Rightarrow f(x)&=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-(\\alpha-\\beta)-\\beta}\\\\\n", + " &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}\\cdot e^{-\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}}\\\\\n", + " &=\\frac{\\sqrt{2\\pi(s^2+\\sigma^2)}\\sqrt{2\\pi\\frac{\\sigma^2s^2}{s^2+\\sigma^2}}}{2\\pi s\\sigma}\\normal\\left[x;\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2},\\frac{\\sigma^2s^2}{s^2+\\sigma^2}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n", + " &=\\frac{\\sqrt{2\\pi\\cdot 2\\pi\\sigma^2s^2}}{2\\pi s\\sigma}\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n", + " &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\hspace{2cm}\\\\\n", + " &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[m,\\mu,\\sigma^2+s^2\\right]\\hspace{2cm} q.e.d.\n", + "\\end{align*}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n", + "\n", + "Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n", + "\n", + "\n", + "Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Linear regression (50 Points)\n", + "\n", + "In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n", + "\n", + "1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)!
\n", + "2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# Loading the required packages\n", + "%matplotlib inline\n", + "import numpy as np\n", + "import matplotlib.pyplot as plt\n", + "from IPython.html.widgets import interact\n", + "import csv" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": { + "collapsed": false, + "scrolled": false + }, + "outputs": [], + "source": [ + "def wRidge(X,y,lamb):\n", + " # Change the following line and implement the ridge regression estimator wRidge\n", + " return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# Read ex1_train.csv and assign the first column and \n", + "# second column to variables x and y respectively.\n", + "\n", + "a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n", + "x= a[:,0]\n", + "y= a[:,1]\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "4) Plot the training data with appropriate labels on each axes!" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "collapsed": false, + "scrolled": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "" + ] + }, + "execution_count": 7, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAG1tJREFUeJzt3X285nOdx/HX2wxRhFQMxtJETDGjZZasHCsaYmdnVXIT\n", + "IjfRNCuVu12mKCpJW9RaN7EMKy2LSiaczCqDjDGMCWW2QSYKyza2YT77x/dH5zrnzJzrOue6ru/v\n", + "d13v5+NxPeZc51w378PM73N97xURmJmZvWqV3AHMzKxcXBjMzKyGC4OZmdVwYTAzsxouDGZmVsOF\n", + "wczMamQtDJIulrRE0vw+35sh6XFJc4vb5JwZzcy6Te4WwyVA/wt/AOdExLbF7aYMuczMulbWwhAR\n", + "s4FnB/mR2p3FzMyS3C2GFZkmaZ6kiyStkzuMmVk3KWNh+DawGTAR+C3wtbxxzMy6y+jcAfqLiN+9\n", + "+rWkC4Eb+j9Gkjd4MjMbhogYsqu+dIVB0piI+G1xdyowf7DH1fPLlZWkGRExI3eO4XL+vJw/nypn\n", + "h/o/VGctDJKuBHYB3ixpMXAa0CNpIml20mPAURkjmpl1nayFISL2H+TbF7c9iJmZvaaMg8/doDd3\n", + "gBHqzR1ghHpzBxih3twBRqg3d4AR6M0doB1UxYN6JEWVxxjMzHKo99rpFoOZmdVwYTAzsxqlm65q\n", + "ZmbNI7Ee8I7iVhcXBjOzipNYDRjHnwtA39uqwC+LW32v58FnM7PykxCwPoNf/McCi/lzAeh7WxJB\n", + "pNeo79rpwmBmViISawCbM/DivwXwMoNf/H8VwZ+Gfm0XBjPrIMUn5rWBMcCGxZ/9b2/iz5+cFzLI\n", + "p+YyKH6Xjai98G9Z/LkB8GsGKQAR/H5k7+vCYGYVUFwk12Pwi33/7y0j7brc//Zk8eezwF+w8n72\n", + "vrdHIljawt9tTdIn/cE+/b/YJ0ffIrYogpdbk8eFwcxKRGIU8AngndRe7NcH/peBF/kBtwheHOZ7\n", + "952Z0/f2tuK1BysaT9TTyih+r01W8PpvAh4d7PUjeH44v8tIuDCYWWlIrAVcCawJXEPtxf+pCF7K\n", + "lGs06fyXwS7qawIP069YkFokW/Z53Djg9wxeXH4TwfL2/UYr58JgZqUgsQnpXJU7gU9GsCxzpLpI\n", + "rMPAbqCNgEUM7I4aVkum3VwYzCw7ie2B60gnMX69TAPA3ajea6cXuJlZS0h8EDgf+HgE1+fOY/Vz\n", + "YTCzpipmGZ1IGmh+fwRzM0eyBrkwmFnTFFszXAC8C9ghgiczR7Jh8O6qZtYUxZTQWaRFaLu4KFSX\n", + "C4OZjZjEO0izju4E9o3gfzNHshFwYTCzEZHYFbgdOCuCE8o0b9+Gx4XBzIZN4nDgKuAjEVyUO481\n", + "hwefzaxhEqsAZwFTgZ0jeDhzJGsiFwYza4jEG4DLSfsA7TDSHT+tfNyVZGZ1k9iINJ7wHLC7i0Jn\n", + "cmEws7pIvJs06+h7wGH1HAxj1eSuJDMbksQU4ELg6Ai+nzuPtZYLg5mtULG9xaeB44C9Irg7cyRr\n", + "AxcGMxuUxKrAecAkYMcIFmeOZG3iwmBmA0isSxpLWEqajvpC5kjWRh58NrMaEuOAnwPzgb9zUeg+\n", + "Lgxm9hqJnYE7gHMjOC6CV3JnsvZzV5KZASBxMHA2cFAEN+fOY/m4MJh1uWJ7iy8ABwA9ESzIHMky\n", + "c2Ew62ISawCXkg65/6sIns4cyUrAYwxmXUpiA6AXWAbs5qJgr3JhMOtCEtuQtrf4AWlM4aXMkaxE\n", + "shYGSRdLWiJpfp/vvUnSLEkPS7pZ0jo5M5p1Gom9gJ8AJ0bwhQgidyYrl9wthkuAyf2+dyIwKyK2\n", + "AG4p7ptZE0hMI+15NCWCq3LnsXJSRN4PC5I2BW6IiK2L+wuBXSJiiaQNgN6I2LLfcyIi1PawZhUl\n", + "MRo4F+gB9o5gUdZAlkW9184yzkpaPyKWFF8vAdbPGcas6iTWJh2/KWCnCJ7PHMlKroyF4TUREZIG\n", + "bdJImtHnbm9E9LYllFmFSGwK3Aj8FJgewct5E1k7SeohtRIbe15Ju5J6IuIpSWOA29yVZNY4iR2B\n", + "75POZv6mB5mt3mtn7sHnwVwPHFJ8fQhwXcYsZpUk8RHgP4EjIvhnFwVrRNYWg6QrgV2AN5PGE04l\n", + "/WW+GtgEWAR8OCKe6/c8txjMBlEcrPNPwOHAPhHcnzmSlUi9187sXUnD4cJgNpDE6qSpqJuTpqM+\n", + "lTmSlUyVu5LMrEESbyGt+1mNtBGei4INmwuDWcVJjAfmALcBH4lgaeZIVnGlnq5qZisnsTtwBfCZ\n", + "CC7Lncc6g1sMZhUlcTTwb8C+LgrWTG4xmFWMxCjSSWt7An8dwaOZI1mHcWEwqxCJtYCZwOuBHSN4\n", + "NnMk60DuSjKriOIIzhtIa34muyhYq7jFYFYdhwKrA0dGsDxzFutgXuBmVgES6wEPAntGMDd3Hqsm\n", + "r3w26yASFwIvRvAPubNYdVX5PAYz60NiJ9JJh+NzZ7Hu4MFnsxKTWBX4DvDpCP4ndx7rDi4MZuU2\n", + "HXgS+F7uINY9PMZgVlISY4G5wA5exGbN4N1VzarvG6ST11wUrK08+GxWQhIfALYGDsidxbqPC4NZ\n", + "yUi8HvgWaSHbS7nzWPdxV5JZ+ZwCzIlgVu4g1p3cYjArEYmtgCOBCbmzWPdyi8GsJCQEnA+cHsGT\n", + "ufNY93JhMCuPA4G1ScXBLBuvYzArAYl1gQXAlAjuyp3HOpM30TOrEInzAUXwidxZrHN5Ez2zipCY\n", + "BEzFm+RZSXiMwSwjidGkTfI+6xPZrCxcGMzyOgZ4DrgidxCzV3mMwSwTiQ2BecDOESzMncc6nzfR\n", + "Myu/c4ALXBSsbDz4bJaBxO7AJOCw3FnM+nOLwazNJFYnLWKbFsEfc+cx68+Fwaz9TgDmR/CD3EHM\n", + "BuPBZ7M2kng7cCewbQSLc+ex7uLBZ7OSKTbJOw84y0XBysyFwax9PgRsSDqy06y0PCvJrA0k3kia\n", + "nrpfBMty5zFbmdKOMUhaBPwP8AqwLCIm9fmZxxisUiTOBdaK4PDcWax7dcImegH0RMQfcgcxGwmJ\n", + "bYH9gXfmzmJWj7KPMbhVYJUmsQrwbeDkCJ7JncesHmUuDAH8RNI9ko7IHcZsmI4gdYdekjuIWb3K\n", + "3JW0U0T8VtJbgFmSFkbE7NyhzOolsT5wBrBbBMtz5zGrV2kLQ0T8tvjzaUnXkvaVea0wSJrR5+G9\n", + "EdHb1oBmQ/sqcGkE9+cOYt1JUg/Q0/DzyjgrSdLrgVER8YKkNwA3A5+PiJuLnwfEThH8LGtQsxWQ\n", + "6AEuA8ZH8GLmOGZA9Vc+rw/MlnQfMAe48dWi0Mdn2h/LbGgSq5E2yZvuomBVVMoWw1CKFsMzwHsi\n", + "eCR3HrO+JE4CdgL2iaB6/8CsY9XbYqhyYTgDWC+CY3LnMXuVxGbA3cD2ETyWO49ZX91QGDYAFgJb\n", + "RPB07kxmxSZ51wM/j+BLufOY9Vf1MYYhRbAEuAb4RO4sZoUpwNuBs3MHMRuJyrYYIkISWwG9wKYR\n", + "LM0cy7qYxJrAg8ChEdyWO4/ZYDq+xQAQwUPAXcBHc2exrncqcLuLgnWCSrcY0tfsAlwAbOXVpZaD\n", + "xLuAW4Gtiy5Os1LqihZD4XbS9tx75w5i3afPJnmnuShYp6h8YSjmiZ+NF7xZHocAryO1Ws06QuW7\n", + "ktJ9RgOPkk7HmpMvmXUTifVIA857RXBv7jxmQ+mmriQieBn4OnB87izWVc4CrnZRsE7TES2G9D3W\n", + "Ah4DJkXw6zzJrFtI7ARcTdok7/nceczq0VUtBoAIXgD+FfiH3Fmss0msShpwPt5FwTpRx7QY0vfZ\n", + "EHgAeHsEPivaWkLieGAysIc3ybMq6fi9klb0y0lcAjzivWqsFSTGAnNJO/s+nDuPWSO6uTC8C5gF\n", + "bBbBS+1NZp1O4vvAAxGcljuLWaO6bozhVRE8QPpEd2DuLNZZJPYCJgBn5s5i1kpDFgZJn5K0bjvC\n", + "NNHZwPHFqlSzEZN4PfAt4Fi3RK3T1XPhXB+4W9LVkiZLGrIZUgK3AS8Be+YOYh3jZOCeCH6cO4hZ\n", + "q9U1xiBpFWAP4FBgO9L87Ysi4lctTbfiPEP2k0kcABwRwa5timUdSmJLYDYwMYIncucxG66mjjFE\n", + "xHLgKWAJ8AqwLnCNpK+OKGVrfQ8YJ7Fd7iBWXcWpbOcDZ7goWLcYssUgaTpwMPB74ELg2ohYVrQi\n", + "HomIca2POSBTfVOuxKdJZ+/u34ZY1oEkDiRt0Lh9sfWKWWU1bbqqpM8DF0fEfw/ys/ERsWD4MYen\n", + "gcLwRtI2GX8ZwaKWB7OOIrEOsACY6s0ZrRN07TqGgY/lK8CqERzX4ljWYSTOA0ZFcHTuLGbN4MLw\n", + "2mPZGLgfGBfBs61NZp1CYnvgetImef57Yx2haxe49RfB48CNwJG5s1g1SIwCvgOc4KJg3ajjWwzp\n", + "8UwAfkjaJuNPrUtmnUBiGrAvsKs3ybNO4hZDHxHMI5205dlJtlISY4BTgWNcFKxbdUVhKJwNfKaY\n", + "l262IucAF0bQ9tl2ZmXRTYVhFhCkFdxmA0jsDuwInJ47i1lOXVMYim6Bs0mLlcxqSKwOnAdMi+CP\n", + "ufOY5dQ1haFwFbCVxMTcQax0PgcsiOCG3EHMcuuKWUm1z+VzwNYRfLTJsayiJN4O3Am8O4Lf5M5j\n", + "1ipe4LbC57IO8GtgQgSLm5vMqqaYjPAj4NYIvpI7j1krebrqCkTwHPBdYHrmKFYOHwQ2Br6eO4hZ\n", + "WXRdiyE9n01Ix3++LYLnm5fMqqTYZHEBsH8Es3PnMWs1txhWouhHvgk4IncWy+rzwCwXBbNapSwM\n", + "xRGiCyU9IumEFr3N14DpEqu16PWtxIqZaQeSZiOZWR+lKwySRpEOXZ8MjAf2l7RVs98ngnuBh4EP\n", + "N/u1rdwkVgG+DZwSwdO585iVTekKAzAJeDQiFkXEMtLagyktei9vk9GdPk5aBX9R7iBmZVTGwrAR\n", + "1Ewjfbz4XivcBKwK7Nai17eSkXgrcAZwdATLc+cxK6MyFoa2TZPyNhld6Rzg3yK4P3cQs7IanTvA\n", + "IJ4Axva5P5bUaqghaUafu70R0TvM95sJfFFiG18sOpvE3qRN8rbJncWsHST1AD0NP69s6xgkjQZ+\n", + "SereeRK4C9g/Ih7q85gRrWMY+J6cBLwjgkOb9ZpWLhJrAw8Ah0Rwa+48ZjlUeksMSXsC5wKjgIsi\n", + "4sx+P292YXgT8ChpD6UnmvW6Vh4SFwARwVG5s5jlUunCMJRmF4b0mnwDWBrBic18XctP4n3AxaTC\n", + "75Xu1rVcGBp+TTYD7iadC/1CM1/b8pFYE5gPHBvBD3PnMcvJW2I0KILHgFuAw3Nnsab6EnC7i4JZ\n", + "/dxiqHldtgeuAcZF8HKzX9/aS2Jn4N+Bd0Xwh9x5zHJzi2EYIrgbWETaitkqTGIN0srmY10UzBrj\n", + "wjDQ2cCpEhvnDmIjMgO4L4JrcwcxqxoXhoFuBK4AfiG55VBFRZfgocC0zFHMKsljDCt8DyaRCsRs\n", + "YLpnKlWDxOuAXwBfimBm7jxmZeIxhhGK4C5gW2A5MFdih8yRrD4nk870vjJ3ELOqcouhrvfj70n7\n", + "959H+iTqGUslJDEB+Akw0SvYzQZyi6GJIvgP4N3AzsBPJd6WOZL1IzGatLr5BBcFs5FxYahTcbF5\n", + "P2mdwxyJg33AT6l8Bvg9cEnuIGZV566kYb0/E0gD0w+SDnx5NlcWA4mtSJMEtotgUeY4ZqXlrqQW\n", + "imAesD2wBJgnNb7fuTWHlHbgBU5zUTBrDheGYYpgaQSfAo4ErpA4S2K13Lm60DTgZdLkADNrAncl\n", + "NYHEW0ifWjcCDoxgYeZIXUFiHDAH2DGCR3LnMSs7dyW1UQRPA1OAC4DZEkd7YLq1JFYBLgTOdFEw\n", + "ay63GJpMYkvSwPQTwMcj+F3mSB1J4ijgMOA9EbySO49ZFbjFkEnRjbQjacbSfRJ7Zo7UcSQ2Ac4A\n", + "DnNRMGs+txhaqJitdClwPfC5CJbmTVR9RRfdD4E7Ijgjdx6zKnGLoQQi6AUmAm8B7inWP9jIHAxs\n", + "AHw5dxCzTuUWQxsUn3IPAs4BzgTOjWB53lTVIzEGmAe8P4K5ufOYVU29104XhjaS2Ay4HPgjcKj3\n", + "9KlfUVyvBR6I4B9z5zGrIncllVAEjwG7ALcD9xa7tlp9PgxsAZyeO4hZp3OLIZPifIfLgZ+SDgJ6\n", + "MXOk0ioWEN4P/F0Ec3LnMasqtxhKLoI7SQcBQToIaFLOPCX3DWCmi4JZe4zOHaCbFceFHi6xL3CD\n", + "xDdJK3k9N78gMQWYBGyTO4tZt3BXUklIbAx8F1gd+GgxHtHVJNYFHgAOiOCnufOYVZ27kiomgseB\n", + "PUgzb+6SOMj7LfE14DoXBbP2couhhCQmAjNJc/Y/EcFzmSO1ncT7gX8Bti663MxshNxiqLAI7gP+\n", + "EniGdBDQLpkjtZXEWqSicKSLgln7ucVQchJ7kbaXvpR0StmfMkdqOYnzgDUiOCx3FrNO4pXPHUTi\n", + "raSDgMaQDgL6ZeZILSPxPtIg/NY+S9usudyV1EGKMx3+llQc/kviyE4cmJZ4L2ls5WAXBbN83GKo\n", + "GImtSAcB/QY4ojg9rvKKLcqvBvaP4JbMccw6klsMHSqCh4AdgF+SDgKanDnSiEn8Dako7OeiYJaf\n", + "WwwVJrEraVD6WuCECF7KHKlhxZjCTOBDXq9g1lqVbTFImiHpcUlzi1vlPxG3SgS3ARNIg9J3S2yd\n", + "OVJDJPYgFYV9XRTMyqN0hQEI4JyI2La43ZQ7UJkVg7T7AWcDt0ocJ5Xy/2uNogvscmBqBLNz5zGz\n", + "PyvrBaTru4kaEUFEcCnwV8CHgJskNswca4UkPgBcBkyJ4I7cecysVlkLwzRJ8yRdJGmd3GGqIoJf\n", + "A+8F7iAdBDQ1c6QBJPYBLgH2ieDnufOY2UBZBp8lzSId6N7fKcCd8NoUzNOBMRFxeL/nB/D5Pt/q\n", + "jYjeFkStLIkdSV01twLHleEgoGIL7QuAvSO4O3ces04nqQfo6fOt0yq/8lnSpsANEbF1v+97VlId\n", + "ij2H/hnYibRiOtvFuDjG9HxgrwjuzZXDrJtVeVbSmD53pwLzc2WpugheiOBjpJbYjRKnSIxqdw6J\n", + "D5KKwp4uCmblV7oWg6TLgImk2UmPAUdFxJJ+j3GLoUHFQUCXkU7t+2gE/92m990POBeYHMG8dryn\n", + "mQ3Om+jZAMU01uOBz5LGHa5o8fsdQDpsZ48It/zMcnNhsBWS2Ja039Jc4NhWHAQkcRDwFWD3CB5s\n", + "9uubWeMqO8ZgrRfBXGA74DnSfks7N/P1JQ4Bvgy8z0XBrHrcYuhyEnsD/wpcDMyIYNkIX+8w4Auk\n", + "orCwCRHNrEncYrC6RHAjabB/InCHxBbDfS2Jj5PWl/yNi4JZdbkwGBEsAfYmnZx2h8QRjR4EJHEU\n", + "cCqwawQPNz+lmbWLu5KshsR40sD0ItJBQM/U8ZxjgM8Bu0Xwq9YmNLPhcleSDUsEC0gHAT1KGpje\n", + "Y2WPl/gkafrrri4KZp3BLQZboeJktUuBa4CT+h8EJDEdmE4aU1jU/oRm1gi3GGzEIriVdBDQxsBd\n", + "fQ8Ckvg08Cmgx0XBrLO4xWBDKgaiDwG+CpwBvA44gtRSWJwzm5nVzyufrekkxpG28l6PNKbwROZI\n", + "ZtYAFwZriWJ31tUiWJo7i5k1pt5r5+h2hLHOEcEr4KJg1sk8+GxmZjVcGMzMrIYLg5mZ1XBhMDOz\n", + "Gi4MZmZWw4XBzMxquDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxq\n", + "uDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxqZCkMkj4k6UFJr0h6\n", + "d7+fnSTpEUkLJe2RI5+ZWTfL1WKYD0wFbu/7TUnjgf2A8cBk4HxJHdeqkdSTO8NIOH9ezp9PlbM3\n", + "IstFNyIWRsTDg/xoCnBlRCyLiEXAo8CktoZrj57cAUaoJ3eAEerJHWCEenIHGKGe3AFGoCd3gHYo\n", + "26fxDYHH+9x/HNgoUxYzs640ulUvLGkWsMEgPzo5Im5o4KWiSZHMzKwOish33ZV0G3B8RNxb3D8R\n", + "ICLOKu7fBJwWEXP6Pc/FwsxsGCJCQz2mZS2GBvQNeT0wU9I5pC6kzYG7+j+hnl/MzMyGJ9d01amS\n", + "FgM7AD+Q9COAiFgAXA0sAH4EHBM5mzRmZl0oa1eSmZmVT9lmJdVN0umS5km6T9ItksbmztQISV+V\n", + "9FDxO/yHpLVzZ2rEyhYplpWkycXCyUcknZA7T6MkXSxpiaT5ubM0StJYSbcVf2cekPSp3JkaIWl1\n", + "SXOK680CSWfmzjQckkZJmitppROAKlsYgK9ExISImAhcB5yWO1CDbgbeGRETgIeBkzLnadSgixTL\n", + "StIo4FukhZPjgf0lbZU3VcMuIeWvomXAcRHxTlIX8rFV+u8fES8BuxbXm22AXSX9deZYwzGd1FW/\n", + "0q6iyhaGiHihz901gWdyZRmOiJgVEcuLu3OAjXPmadRKFimW1STg0YhYFBHLgKtICyorIyJmA8/m\n", + "zjEcEfFURNxXfP0i8BBp3VJlRMQfiy9XA0YBf8gYp2GSNgb2Ai6kdtLPAJUtDACSvijpN8AhwFm5\n", + "84zAYcAPc4focBsBi/vc9+LJTCRtCmxL+kBUGZJWkXQfsAS4rZgsUyVfBz4LLB/qgaUuDJJmSZo/\n", + "yG0fgIg4JSI2Ab5L+qVLZaj8xWNOAf4UETMzRh1UPfkrxLMsSkDSmsA1wPSi5VAZEbG86EraGHhv\n", + "lfZNkrQ38LuImMsQrQUoxzqGFYqI3et86ExK+Il7qPySDiU17XZrS6AGNfDfvwqeAPpOUBhL7fYr\n", + "1mKSVgW+D1weEdflzjNcEfG8pB8A2wG9mePU6z3A30raC1gdeKOkyyLi4MEeXOoWw8pI2rzP3SnA\n", + "3FxZhkPSZFKzbkoxsFVlVVhweA+wuaRNJa1G2sX3+syZuoYkARcBCyLi3Nx5GiXpzZLWKb5eA9id\n", + "Cl1zIuLkiBgbEZsBHwFuXVFRgAoXBuDMolvjPtKOh8dnztOob5IGzWcV08fOzx2oEStapFhWEfEy\n", + "8Engx6RZGf8eEQ/lTdUYSVcCPwO2kLRY0sdyZ2rATsBBpNk8c4tblWZYjQFuLa43c4AbIuKWzJlG\n", + "YqVdq17gZmZmNarcYjAzsxZwYTAzsxouDGZmVsOFwczMargwmJlZDRcGMzOr4cJgZmY1XBjMzKyG\n", + "C4NZE0javjh06XWS3lAcRjM+dy6z4fDKZ7MmkXQ6aYOyNYDFEfHlzJHMhsWFwaxJit1D7wGWAjuG\n", + "/3FZRbkryax53gy8gbQ54hqZs5gNm1sMZk0i6XrS2SBvA8ZExLTMkcyGpdQH9ZhVhaSDgf+LiKsk\n", + "rQL8TFJPRPRmjmbWMLcYzMyshscYzMyshguDmZnVcGEwM7MaLgxmZlbDhcHMzGq4MJiZWQ0XBjMz\n", + "q+HCYGZmNf4fBBSJ6m7pJCYAAAAASUVORK5CYII=\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Plot the input data here\n", + "plt.plot(x,y)\n", + "plt.xlabel('x')\n", + "plt.ylabel('y') " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "5) Implement a function which constructs features upto a input polynomial degree $d$!
\n", + "Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._" + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def construct_poly(x,d):\n", + " ## Implement a method which given an array of size N, \n", + " ## returns an array of dimension (N,d)\n", + " poly=np.empty((len(x),1), float)\n", + " xr = np.reshape(x,(len(x),1))\n", + "\n", + " for i in range(d-1):\n", + " poly = np.append(poly, (xr**i), axis=1)\n", + " #print poly \n", + " #print np.shape(poly)\n", + " return poly" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable!
" + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def MSE(y_predict,y_true):\n", + " ## Implement mean squared error for a given input y and its predictions.\n", + " return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!
" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "collapsed": false, + "scrolled": true + }, + "outputs": [ + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecXFX9//HXOwm9hA6BUJQmvfwQQUAjTXpEepEO0hEL\n", + "Vbm5FAUFRCmC9I4g0gVEJIKoIF8CRELHQGihhF4T8vn9cW7IZrNtdmfmzsy+n4/HPLI7t312H/De\n", + "M+eee44iAjMzay0Dyi7AzMyqz+FuZtaCHO5mZi3I4W5m1oIc7mZmLcjhbmbWghzuZmYtyOFudSVp\n", + "rKRPJc3b7v1RkiZLWqz4fqik6yW9IekdSaMl7V5sW6LY9/12r+06ueb2kv4p6UNJ93RT3zHtzvmR\n", + "pM8lzVNsn1PSFUVdbxRfz9Hm+IGSTpT0sqT3JD0saXCxbSZJvy62TZB0tqRBbY4dKukWSW9JelXS\n", + "mZIGdlDjccXPv37Pf/PW3zjcrd4CeB7YacobklYCZim2TXE58AKwGDAP8D1gfLtzDY6IOdq8ruvk\n", + "mm8BpwMnd1tcxM/bnhM4BbgnIiYUu4wA5gO+BCwJLFi8N0UOrAWsFRFzArsCnxTbjgJWB1YAlim+\n", + "/mmbY38LvAkMAVYFvgkc2LY+SUsC2wKvdPezWP/mcLcyXAHs1ub73YHLALV5bw3gkoj4OCImR8Qj\n", + "EXFHby4WEXdHxB+BVys5TpKK2i5t8/YKwI0R8UFEvAfcWLyHpLmBw4B9I2Jcce0xEfFpcewWwJkR\n", + "8U5EvEkK873anfsPEfFZRIwH7phy7jbOAo4EJlbys1j/43C3MvwbmFPSV4puhx1Igd9+n3Mk7TCl\n", + "q6YD6uT9alkPmB+4vs17dwLbSJqrCPNtgD8X21YCJgHbFd0qT0mapuXdruYBwNA23Tp3AjtLmkXS\n", + "IsCmwO1fHJi6nT6JiNsx64bD3cpyOan1vhEwBni53fbtgPuAnwHPF33ya7Tb501Jb7d5LVvlGncH\n", + "rouIj9q8d3bx71ukLpSJwO+K94YCg4GlgSVI3ScjJG1YbL8DOEzSfJIWAg4ldUXNWmwfAawIvAeM\n", + "A/4TETcBFH8ATiJ9MjDrlsPdyhCkcN+FjrtkKLoujo6IFUn92o+QukDamjci5m7zeqpaBUqalRTO\n", + "l7bbdCXwFDA7MCfp/sGUTx0fF/8eHxGfRsRo4Bpgs+L9k4BRxc/yD+AGYFJEjC+6gO4EriOF/XzA\n", + "PJJOKY4dAVweES+2LbMKP6q1KIe7laIIqedJXQ9/6mbft4DTgIWLrpBeX7aCfbcG3oqIv7d7fxPg\n", + "vOJewIfAeUwN78e6um5EfBIRh0TE0IhYCpgAPFTsMx/w/4CzImJicQP3kjbnXh84tOjueRVYFLhW\n", + "0k8q+JmsH3G4W5n2BtaPiI/bb5B0iqQVJA0quiQOAJ6JiLfb7taTi0gaIGlmYAZgQDEkcYZuDpvy\n", + "iaK9x4B9Jc0saRZgP+BRgIh4jtSVdKykGSUtR7qfcGtRx8LFS5LWIo2UyYrzvkm64XtAMZxyrqKG\n", + "R4vtG5Burq5CGknzSnHtc3ryO7D+x+FupYmI5yPi4bZvtfl6FlK3xdvAc6SW6lbtTvFOuzHpP+jk\n", + "UrsBH5GCcD1S98l5UzYWx67T5vtFgGF0HO57kIYxvgy8ROpb373N9p2AxUl98rcCP42IKWPrlwTu\n", + "Bz4ALgaOjIi/Fr+LAL4LbEkK+meAT4HDi+0TIuL14jUe+Bx4u/j0YDYd9XWxDkkXAZsDr0fESsV7\n", + "I4B9gDeK3Y7u7TA2MzOrXDVa7heT+iHbCuD0iFiteDnYzczqqM/hHhH3kT46t+c7+WZmJalln/sh\n", + "kh6VdGFxc8jMzOqkVuH+O9LcG6uSRgCcVqPrmJlZBwZ1v0vlIuL1KV9LugC4pf0+kvp2J9fMrJ+K\n", + "iG67vWsS7pKGRMSUSZq2BkZ3tF9PCmxUkkZExIiy6+gt118u11+eZq4det4w7nO4S7qaNDXpfJLG\n", + "kR7KGCZpVdKomf8B3+/rdczMrOf6HO4RsVMHb1/U1/OamVnv+QnV3htZdgF9NLLsAvpoZNkF9NHI\n", + "sgvoo5FlF9AHI8suoB76/IRqry8sRTP3uZuZlaGn2emWu5lZC3K4m5m1oJoMhTQzs+pRrnmBZYtX\n", + "jzjczcxKJDEAmJN5n1yQ5f+4EvM8uyKzj1+GWd9cglnfXITZ3lgAzTKQdxb7kAnLTOzgmdCOz+sb\n", + "qmbWCCQGktbO3Zc05/6rpEVJXm33ei2CiWXV2RmJQcBcwDydvybPw1wvLMT8jw9h3mfnY+7n52Tu\n", + "52ZivieDOV8SHyz0Ge8u9j7vLfoW7yz+GhOWGscr/+85Xl/pBWLgBGAC6N6eZKfD3cxKVYT69sBx\n", + "pBlmTwUmAUPavBZu8/X8wLtMH/rT/SGIYLpVvnpQz4zA3HQZ0h2+5ijqmsAMH7zNkEc+Y8hDwUKP\n", + "DmK+J2dl8ItzMeub8wGTmDjLC3w+09PAGGb84FFm/GgM8Fxk8Vn39fUsOx3uZlY3yiVgMDCEiTMP\n", + "ZfTOW/Peotsw+MVJLHHPU8w1dgBiHmAcaSHyJ4t/nwLGRxZR/DGYj87Df8prIeATpv8j8BZpcfPO\n", + "Qnpm0h+ZCZ283gImoEkTWPnKQax01dwMGbUgs765GIqvkPrFFyKtEfxU+1dk8VaffocOdzOrlyK0\n", + "56XjkJ32vWAinw7+kDeWG8wnc3/AnOP+zgKPP4hiSsv7bdJShcu2e81AB2EJPBNZR+vwIlILvH0d\n", + "8zClhd3x6/2IqUs+KtfspKUV29ezDGnJxCl1tP1DNDaymNSHX2mnHO5mVnXKNZC0WPkKTBvgCwIf\n", + "0kUXCR/O/zq//896vLv4T4DXgBHAPW2DtJtrtx0x0vb15eIaHQX/y5F1H3LFz7VYJ+efB3i2o/NH\n", + "Fu/2pPZqcribWVUp1xzA1cDswB+ZNsBfiyw+6fC4dKNxF+CnpIXFc2BkT0O9B3UNIq0f0VEwzw48\n", + "TbvAJ30y+Eqb/ZYkdbd09AfixchicjVqrQaHu5lVjXItRhqD92/g4Mii29EqEjMAuwLHkvrQ84j6\n", + "zuuiXHMxfZfKIsBYpu/a+aCetfWWw93MqkK5vgrcSFpR7dfddXMUof49UqiPJYX6vbWus7/oaXb6\n", + "ISYz65RybQucA+wTWdzc5b4p1HcjhfrzwB4R3Ff7Kq0jDnczm04x+uUo0s3Tb0cWozrdN40L3x04\n", + "BngG+F4E99elUOuUw93MpqFcMwK/B1YE1oosXulwvxTqe5BC/Slglwj+Wa86rWsOdzP7QjHc8E+k\n", + "8d7fjCw+nG4fMROwJ3A0MAbYKYJ/1bVQ65bD3cwAUK5lgVtJ4X50R8P/JBYC7iWN+94+ggfqW6X1\n", + "lEfLmBnK9S3gGuCYyOLCDvcRs5KWqLs1guPrWJ614aGQZtYjyrU38HNgx8jing73SdPSXkd6CnX3\n", + "aj2AZJXzUEgz65JyDQBOBrYG1ossnu5i95NJk3Xt7GBvDg53s35IuWYDriDNm7JWVzMVSuwHDAfW\n", + "juDTOpVofeQ1VM36GeVahHRT9B1go26CfWPgeGDzCCbUqUSrAoe7WT+iXKuT5oe5Dtirq8UhJFYk\n", + "te63jeDZOpVoVeJuGbN+QrmGAxcA+0cW13e5bxryeCvwgwj+UY/6rLoc7mYtrphK4IfA4cBmkcV/\n", + "utw/DXm8BbgwgqvqUKLVgIdCmrUw5ZoBOBtYE9gyshjX5f5pyOMfgfdJE395ZEyD8VBIs35OueYm\n", + "9a1/TBrq+H4PDjuFNIJmJwd7c/MNVbMWpFxLAv8CRgPf6UmwS3wf2Ar4roc8Nj+33M1ajHKtR2qx\n", + "j4gszu3RMeLbpDVN1/WQx9bgcDdrIcq1G3AqsGtk8ZceHSNWAi4Hto7guVrWZ/XjcDdrAcVUAscD\n", + "OwPDIosxPTpODCENeTzMC2y0Foe7WZNTrlmAS0kLP38tsnijR8eJ2YCbgfMjuLqGJVoJfEPVrIkp\n", + "10KkaXgnAhtUEOwDSU+fPg6cVLMCrTQOd7MmpVwrk6YSuI3Ux/5JBYefAswF7Ochj62pz90yki4C\n", + "Ngdej4iVivfmAf4ALA6MBbaPiHf6ei0zS5RrM+AS4NDI4pqKjhUHAFsAX4+g07llrLlVo+V+MbBJ\n", + "u/eOAu6KiGWAu4vvzawKlOsQ0hwxw3sR7JsAx+FZHlteVaYfkLQEcEublvuTwDcjYryU+gQj4ivt\n", + "jvH0A2YVUK5BwBnAMGCLyGJsRceLlUmNre94ZEzzKnv6gQUjYnzx9XhgwRpdx6xfUK7BpDVOBawT\n", + "Wbxb0fFpyOMtwKEO9v6h5kMhIyIkdfjxQNKINt+OjIiRta7HrNko1xKkseh/Bw6LLCZVdHwa8ngL\n", + "8HsPeWw+koaRPq1VdlwNu2WGRcRrkoYA97hbxqxyyrU2cD1pDdMzI6vsf9hiyOP1pFWX9vTImOZX\n", + "drfMzcDupOFWuwM31ug6Zi1LuXYEfgvsGVnc1svT/BIYDGzvYO9f+txyl3Q18E3SyujjSXfibwKu\n", + "BRajk6GQbrmbdaxYXONnwN6kOdgf69V5xIHAoaSFrd+uYolWop5mpxfrMGsgyjUzaZjj0qShjq/1\n", + "6jxiU+Ai0iyPngyshfQ0O/2EqlmDUK75SUMVZyRN/tXbYF+FNNfMNg72/svhbtYAlGt54AHgHmDH\n", + "yOLjXp1HLMzUIY//rGKJ1mQ8K6RZyZRrI+BK4MeRxWW9Ps/UIY/nRVDRk6vWetznblYi5dqftALS\n", + "dpHFfb0+Txry+CdgArCXR8a0rrKHQppZF5RrIGnFpE2BdSOLZ/t4ylOBOYDtHOwGDnezulOuOYCr\n", + "gFmBtSOLPg1TlDiI9Edibc/yaFP4hqpZHRXL4d1CeiZkkyoE+2bAT4HNPJbd2nLL3ay+9gBmBvaL\n", + "LCb35URthjxuFcHzVajNWojD3axOlGte4OfAplUI9ilDHg+O4F/VqM9ai7tlzOrnFOCayGJUX04i\n", + "MTtplshzI/hDVSqzluOWu1kdKNc6pBXLlu/TedKQx6uAR4BfVKE0a1EOd7MaU64ZgHOBH0YW7/Xx\n", + "dKcBswPbesijdcXhblZ7hwGvANf15SQSBwPfxgtbWw/4CVWzGlKuRYFRwFp9eVBJYnPSbJHreGRM\n", + "/+YnVM0aw29IKyj1JdhXBS7BQx6tAg53sxpRrs2BlYCde30OsQhpyONBHvJolXC4m9WAcs0KnEV6\n", + "WOmTXp0jDXm8BTgngmurWZ+1Po9zN6uNY4EHIou7enNwMeTxalJ//cnVLMz6B7fczapMuZYD9gNW\n", + "6cNpTidNLLa/hzxabzjczaqoWNz6HOCEyOKVXp1DHAJsRBryOLGa9Vn/4XA3q65dgMGkgK9YMeTx\n", + "GFKwv1PNwqx/8Th3sypRrrmBMcDwyOLBio9PQx7vAraM4N/Vrs9aQ0+z0zdUzarnJODGXgb7UNLI\n", + "mAMd7FYN7pYxqwLlWhPYml5MDNZmyOPZEX2bosBsCnfLmPWRcg0CHgROjyyuqOjYNOTxJuA1YF+P\n", + "jLHuuFvGrH4OBN4BruzFsb8mrcx0gIPdqsndMmZ9oFwLAz8D1ousso/BEocCG+Ihj1YDDnezvjkd\n", + "+H1k8WQlB0lsARyNhzxajTjczXpJuTYC1gT2qug4sRpwMWnI4/9qUZuZ+9zNekG5ZiY9qHRIZPFR\n", + "j48TC5FuoHrIo9WUw92sd44ERkcWt/X0AImZgD8BF3nIo9Wau2XMKqRcSwGHAKv1+Bgh4HekIY/H\n", + "16g0sy843M0qUEwMdjZwcmQxroJDDwXWIN1AnVyT4szacLibVWY7YGHS8nk9IrEhaWTM2hF8UKvC\n", + "zNpyuJv1kHLNSRr6uENk0aNx6RJLkR5u2tEjY6yeahruksYC7wGfAxMjYs1aXs+sxo4H7ows7u/J\n", + "zhJzkkbGHB/BPTWtzKydWrfcAxgWERNqfB2zmlKu1YCdgBV6tL8YAFwB3E8v53Y364t6dMt4cjBr\n", + "aso1gDTS5ZjI4s0eHnY8MDewreeMsTLUepx7AH+V9JCkfdtvLBYnMGt0+5K6Fi/uyc4S2wPfA7aJ\n", + "4LNaFmbWmVq33NeJiFclzQ/cJenJiLivzfY9gcNqXINZrynXgsCJwAaRRbdDGIupBc4GNo7g9VrX\n", + "Z9aZmoZ7RLxa/PuGpBtI83C0Cfdj95F+9R5M/BwYGREja1mPWS/8Crg0snisux0lFgBuBA6OYFTN\n", + "K7N+QdIwYFilx9Us3CXNCgyMiPclzQZsDOTT7LTkek/x3EmPRHB9reow6y3lX/xP1e3qShIzAtcD\n", + "l0fwh9pWZv1J0egdOeV7SVlPjqtln/uCwH2SHgEeAG6NiL9Ms8emh00G9qhhDWa9olwzkka5HBZZ\n", + "dPngUTG1wJnABOC4OpRn1q2atdwj4n/QzQ3TeZ9ZnHmfXFr6ykIRvFarWsx64UfA86Rulu4cAKxL\n", + "egLVUwtYQyh3VkjFuWx8xCukkQVmDUG5vkQK90O6W11JYhiQAVtF8F4dyjPrkbKn/D2LpW4fyuyv\n", + "7lN8tDUrVTEx2G9Ji113OV2AxJeAa4BdIniuHvWZ9VSp4R5ZjGfApGv42m/mA75aZi1mheHAUsCp\n", + "Xe0kMTtpaoFfRPDXehRmVomyW+4gTudrZ83ETO/uU3Yp1r8p1+yk2R4PjCw6ffiomFrgMuAhUivf\n", + "rOGUHu6RxRPAv1j5ip0lZim7HuvXjgPujSy6m+TrOGAh4ABPLWCNqvRwB2DGD09k3VPEgM+2LrsU\n", + "65+Ua0XSsNwfd7mf2AbYG/huBJ/WoTSzXmmMcId7GfTJa6x8ZZf/Y5nVQpuJwbLIYnyn+4mVgXOB\n", + "rT101xpdQ4R7ZBEMmJix+gUrSSxadj3W7+wOzAT8vrMdJOYj3UA9LIKH6lWYWW81RLgDMMs71zDv\n", + "U5+w8uXHlF2K9R/KNS/wC2D/yOLzDvcRMwDXAddGcFU96zPrrYYJ98hiEhOWOp8VrtvVY96tjk4G\n", + "ro0sHu5inzOAjwA3PKxpNEy4AzDf0xmL3TcLq1+wXdmlWOtTrnWAzYCfdbqP2A/YANg5gg5b9maN\n", + "SNH109W1u7AUETFdC127bP43Pp9xwbjmhh4tZ2bWG8o1A/B/wM8ji2s63EesR5rpcd0Inq5nfWad\n", + "6Sw722usljvA6yv+kCVGLqfvfm9o2aVYSzsUGA8dT88rsThwLbCbg92aUcO13AG085bj0Of/jiv/\n", + "7O4ZqzrlWhQYBXw9spguuCVmA/4BXBnR9TQEZvXWvC13gCeHn8bQB7dSrpnLLsVa0hnA2Z0Eu0hr\n", + "pY4GTqt3YWbV0pjhPmqfc3h1dRi/otdXtapSrs2AVUjDHztyDLA4sJ+nFrBm1pDhHsFnjN7pNmZ7\n", + "/cfF04NmfaZcswJnAQdFFp9Mt11sRVp4Y+sIpttu1kwaNzgf2+UEPlxgTj4ftFnZpVjLOAZ4KLK4\n", + "s/0GiRWAC4FtInil7pWZVVnDhnt8PuMoHjz4VT6Z68Sya7Hmp1xfAb4PHD7dNjEPaWqBH0XwQL1r\n", + "M6uFhg13AB7b5TcollauNcouxZpXsbrSOcCJkcXL02wTg0jDIW+K4LIy6jOrhcYO94mzX879Rwxg\n", + "0kxHlV2KNbWdgbmBszvYdiowGTiyrhWZ1VhDh3sEbzJqz7uATZRriZLLsSakXHMBvyJNDDZpmm1i\n", + "L9L0AztGMKmj482aVUOHOwAfzX8+j+72LuBhkdYbJwE3RxbT9KVLfB04BRgewdulVGZWQw35hOq0\n", + "+zGIwS+8zGFfnokBk78UWfh/ROsR5foqcDOwfNv/boo1Ax4A9o3gtrLqM+uN5n5CtY0IJvHu4pfx\n", + "0trjgP3Krseag3INJK2adGS7YJ8FuAH4rYPdWlnDh3vhEm4/YwGCQ5VrxrKLsaZwIPA+cPmUN4qp\n", + "BS4EniZ1yZi1rKYI9wge59U1XuCDhV4Ddiq7HmtsyjUEOA44MLJp+h2PAJYF9vHUAtbqmiLcCxdz\n", + "1y8/AH5cjFs268zpwAWRxZgpb0hsTrop/50IPiqtMrM6aaZwv4bHdlmFyQMHABuXXYw1JuXaCFgb\n", + "OOGL98RywCXAdhGMK6k0s7pqmnBPw9UG3M5D338I+HHZ9VjjKaaIPhs4JLL4CEBibtLUAkdFcH+Z\n", + "9ZnVU9OEe+ES7jxtRWA55Vq17GKs4RwBjIksbgGQGAhcDdwRwYWlVmZWZ80W7n/l85kX4PUVrgV+\n", + "VHYx1jiUaynS0nmHtnn7FGAQ8MNSijIrUVOFe7H6/GVcfseMwObFcmnWzxU32M8CfhlZvAggsRvw\n", + "HWAHTy1g/VFThXvhEt4fuh2TB16GpySwZFtgKPBrAIk1SUvkDY/grTILMytL04V7BM8Az3DNDaOB\n", + "PZVrcNk1WXmUa05SqB8QWUyUWBj4E2ks++PlVmdWnqYL98LFPL3llsAdwL5lF2OlyoG7Iov7JGYm\n", + "TS1wbgQ3lVyXWalqFu6SNpH0pKRnJFV7ruxrgWE8v/5FwGGekqB/KkZM7QIcUUwtcB7wAmkmSLN+\n", + "rSbhLmkg6QbXJsDywE6SlqvW+SN4H7iJy+5emTRPyPbVOrc1h2Lh9N8Bx0YWb5CWz1sZ2NNTC5jV\n", + "ruW+JvBsRIyNiInANcDwKl/jYmBPJg84FU9J0B/tAwRwocS3gZ+Qphb4sNyyzBpDrcJ9EZjmMe+X\n", + "iveq6V5gDk785HVgBmCDKp/fGpRyLQCcCOzPiFgSuAzYPoIXyq3MrHHUKtxr/rE4gsnAJUyeYU/S\n", + "OpiekqD/OB24nBHxAmkxjuMiuK/kmswayqAanfdloO0DRouSWu/TkDSizbcjI2Jkhde5FHiIkT87\n", + "mmEnnKRcK0cWj1VarDUP5doCWJuX11iV1N13TwTnlVyWWc1IGgYMq/i4WiyzJ2kQ8BSpq+QV4EFg\n", + "p4h4os0+PVoqqvtrcTdwLiO0FLBsZLFHX89pjal4puG/wO6MiI2BtYCNIphYbmVm9VPqMnsRMQk4\n", + "GLgTGAP8oW2wV1m6sZqGwW2lXNXu27fG8Svgz4yIJYAdSFP4OtjNOtDwC2R3fx5mJXUDrcgIHQF8\n", + "HFkc1ecCraEo14YEF3H6uD/w/tDtgM0iGNPtgWYtpmUWyO5OsarOH4FdgTOAfZRrjnKrsmpSrtkJ\n", + "XcBNF7/I+0O/DqzpYDfrWtOHeyF1zYyIscDdwN7llmNV9dG8v+WJrefgkT2eA9aP4PWySzJrdK0S\n", + "7v8CRLrBdipwuHLVaiSQ1ZG2/P5+TB60O3edciawRwSfll2TWTNoiXAvHje/BNgzsvgPMJY0Daw1\n", + "Mc3x6vf48t3n8OTwX8aEpUZ4WgGznmuJcC9cBmxb3GA9FThOuYaWXJP1gsQAiZx1Tz6Lmd67O245\n", + "7+iyazJrNi0T7hG8DDwAbA3cClwJ/J9yuQXfRIo/ztewxD1bs+ZZnzHbG7uVXZNZM2r6oZDTnpPt\n", + "gf0i2BBAudYkhfx9wGGRxfvVvJ5Vl8QiwE3M8OFTHDPHqihOiiyuKrsus0bSb4ZCtnMzsKrE4gCR\n", + "xYPAasBkYJRyrVVmcdY5iTVIn7yu55jZn0XxHHB1yWWZNa2Warmn83I2MD6C46d5P9d3SfN/nw38\n", + "PLLwoskNovjEdTawHyP0PPBXYNXI4uVyKzNrPD3NzlYM9zVIKzUtVcwcOXVbmprgEmBW4HuRxfPV\n", + "vr71XLF60nHAXsB3GKHRpNb72ZHFRaUWZ9ag+mu3DMD/AR8C67XfULQEv016ovUB5drNi3yUQ2IW\n", + "UrfLpsDXIhhFmrb5LdJDaWbWBy3Xck/n5ofAGsBuEXTY/aJcq5Butj4O7B9ZvF2LWmx6EgsDNwLP\n", + "AntH8LFyLUe68b1GZDG2zPrMGll/brkDXA4sA7whcYPEQRLLFN0AAEQWjwJfBcYDjyrXsHJK7V8k\n", + "Vid1vdwE7FIE+0DgQiBzsJtVR0u23KdegwVJc8pvVLwmA3eRbtjdPWWOEuXahBQulwPHRRaf1bKu\n", + "/kpiW9JN7f0juP6L93P9APguMCyymNzZ8WbWj2+odn49BCwLbEgK+m+SpilIYb/TVk+w7C1nkdZ6\n", + "3SWyeLJetbW64nd/LLAfaRHrh7/YlmtJUkt+7cjimZJKNGsaDvdur88gYE1S2G8IrAaT/8PmB7/L\n", + "6ucPA45h4KRzIyvpF9QiihunFwJLAcMjePWLbbkGkGbxvDWyOK2kEs2aSn/vc+9WBJMi+GcEx0fw\n", + "DWBhGHAat53zAr9/6A3eWOFMXvz6yxoy6scSS5ZdbzOSGAKMJP139s22wV7YlzQs9Yw6l2bW8vpt\n", + "y707GjJqcTb5wZksOHpDbrz4Y54a/i6pr/4u4G8RvFVyiQ1NYjXSTdMLgBPaz+ioXIuRhq0Oiywe\n", + "L6FEs6bkbpkqUa5hBJfy3tB/cO6oUXw83zDSGPpnmBr290fwSZl1Noqiu2tv4ETgwAium26f9GzB\n", + "n4H7I4sT61yiWVNzt0yVRBYjEasy+KWBHDn/nozQscD8wOHAp8AJpCGXf5E4QmI1qf/9Xotpencg\n", + "LYi+I7BRR8Fe2A1YCDilXvWZ9TduufdQ0drcFTgd+AVwxpRhexKDgWFMHYkzL+lG4V+Bv0YwtoSS\n", + "66IYCbMpcBIwCTiG9DN3+B+Wcg0BHgW+HVmMqluhZi3C3TI1olxfAq4APgL26GhyK4lFmToKZ0Pg\n", + "PaZ24dwTQUs8DSuxHvBz0h+znwI3dLVaUvEH8gbgv5HFT+tTpVlrcbdMjUQW/yONkb8XeLiYbXLa\n", + "fYJxEVwcwS7AEGAb0qP2+wIvSDwgcZLEMImZ6ll/NRRdT7eTVr+6AFgpgj/1YBm87UlPDp9Q6xrN\n", + "+ju33PugmB/+CuDvpMVAPuj2mBTma5O6bzYElgPuZ+qTs6Mbda1QiWVJwbweqRvm/J4uWK1c8wOP\n", + "Ad+JLB6oXZVmrc0t9zqILP5NWgwE0mIga3Z7TPBpBCMjODaCrwGLA+cDSwPXA69JXCmxp0RDrAEr\n", + "sZjEBcA/gFGk6ZTP6mmwF34DXOVgN6sPt9yrRLm2Ac4BzgR+EVl83qvziCWYemN2A+BNprbqR0bw\n", + "blUK7lktCwBHk0a3nAf8qjf3C5RrOHAasHJk8VF1qzTrX3xDtQTKNZS0GMjMpMVA/ten86Uhlasy\n", + "NezXAkZbS1UMAAAJWElEQVQzNez/HcHEvlyjk+sOBn4EHARcBZwUwWu9OleuuYH/AjtHFn+vXpVm\n", + "/ZPDvSTFfCmHA0cV/15Zrflpinla1mFq2C9FmgN9StiP6Ut/fXH+g4GfkB4yyiPo2x+oXBcBH0cW\n", + "B/XlPGaWONxLplyrklq9jwIHRBbvVP0aYj5gfaaG/UxMHXJ5dwSv9PA8M5CeKv0Z8G/gZxGM6XN9\n", + "ub5N6s5ZKbJ4v6/nMzOHe0NQrlmAXwJbAbvVsluieJjoy0wdhbM+8ApTw/7eCN5vd8xA0tOkxwPP\n", + "AcdG8J+q1JNrDlIX0n6RxV+qcU4zc7g3FOXajDQe/FLSakM1XwykCO7VmRr2XyWNdJkS9vOThjN+\n", + "ABwTwT1VvX6us4FZIou9qnles/7O4d5glGsB0rzmQ0iLgTxV1+uL2YB1mboq1efACOCWao+rV64N\n", + "STeWV/LatGbV5XBvQMXj9/uTukGOBc5vtcVAlOsbwB+BHSOLv5Vdj1mrcbg3MOVaDrgSeBHYN7J4\n", + "o+SSqqJYZPxaYKfI4u6SyzFrSX5CtYFFFk+Qxqw/BTxSLNDd1JRrfVKw7+BgNyufW+4lU65vkW60\n", + "3gAcGVk03aIfRR/7VcB2flDJrLZKbblLGiHpJUmjilfTt0xrJbK4B1iFdKP1P8q1UsklVUS5NiYF\n", + "+zYOdrPGUZOWu6QMeD8iTu9iH7fc2yhutu4GnEqaI/03UxYDaVRFd9JlwNaRxf1l12PWHzRCn7uD\n", + "uwKRRUQWlwJfA7YD7lCuhUsuq1PKtTkp2Ic72M0aTy3D/RBJj0q6UNJcNbxOS4ksnge+QZrj/WHl\n", + "2rrkkqajXFsCFwNbRhb/KrseM5ter7tlJN1FWuS4vWNJ85NMGd53AjAkIvZud3wAeZu3RkbEyF4V\n", + "06KUa23SYiB/Aw7vyWIgtVZM3/t7YIvIoipTFZhZ5yQNI63RPEXWEOPcJS0B3BIRK7V7333uPVDM\n", + "0fJb0myQu5QZqMWSgucAm0UWD5dVh1l/VupDTJKGRMSrxdeHA1+NiJ17U6AlyrUdcBYp6E/u7WIg\n", + "fbj+tsX1N40sRtXz2mY2VdnhfhlpkYkA/gd8PyLG96ZAm6pYDOQyYBBpMZAX6nTdHYAzgE0ii0fr\n", + "cU0z65inH2hRxWIgPyItqHF4ZHFlja+3M2mJvI0ji9G1vJaZdc/h3uKUazXS/DSjgINqshhIrl1J\n", + "89FvFFk8Xu3zm1nlGmGcu9VQ0e+9BvAOaX6a9ap5fuXaHTgF2NDBbtZ83HJvAcq1BXA+cBEwIrLo\n", + "06LZyrUXaVriDSOLJ6tQoplViVvu/UhkcSvpBvaqwP3KtUxvz6Vc+5CeP1jfwW7WvNxybyHF/DQH\n", + "kML5GOCCShYDUa7vkx5CWz+yeLY2VZpZX/iGaj+mXMuTbraOJS0G8mYPjjkQOALYILJ4rrYVmllv\n", + "uVumH4ssxpAWA3mWdLN14672V66DSUMrv+VgN2sNbrm3uGKFpEtJ65oe3X4xEOU6DDiM1BUztv4V\n", + "mlkl3HI3AIpFqlcBhgIPtl0MRLl+CBwKDHOwm7UWt9z7ieJm6+7Ar4ATgZmAfUkt9nFl1mZmPecb\n", + "qtYh5VqSNI3wvKQ+9pdLLsnMKuBwt04p10Bgxsji47JrMbPKONzNzFqQb6iamfVjDnczsxbkcDcz\n", + "a0EOdzOzFuRwNzNrQQ53M7MW5HA3M2tBDnczsxbkcDcza0EOdzOzFuRwNzNrQQ53M7MW5HA3M2tB\n", + "DnczsxbkcDcza0EOdzOzFuRwNzNrQQ53M7MW5HA3M2tBDnczsxbkcDcza0EOdzOzFtTrcJe0naTH\n", + "JX0uafV2246W9IykJyVt3PcyzcysEn1puY8GtgbubfumpOWBHYDlgU2AcyS13CcEScPKrqEvXH+5\n", + "XH95mrn2SvQ6dCPiyYh4uoNNw4GrI2JiRIwFngXW7O11Gtiwsgvoo2FlF9BHw8ouoI+GlV1AHw0r\n", + "u4A+GFZ2AfVQixb1wsBLbb5/CVikBtcxM7NODOpqo6S7gIU62HRMRNxSwXWioqrMzKxPFNG33JV0\n", + "D/CjiHi4+P4ogIg4ufj+DiCLiAfaHefANzPrhYhQd/t02XKvQNsL3QxcJel0UnfM0sCDvSnOzMx6\n", + "py9DIbeWNA5YC7hN0u0AETEGuBYYA9wOHBh9/XhgZmYV6XO3jJmZNZ5Sx59LOkHSo5IekXS3pEXL\n", + "rKdSkn4l6YniZ/iTpMFl11SJrh5Ea1SSNikejntG0pFl11MpSRdJGi9pdNm1VErSopLuKf6b+a+k\n", + "Q8uuqRKSZpb0QJE3YyT9ouyaekPSQEmjJHU5qKXsh4t+GRGrRMSqwI1AVnI9lfoLsEJErAI8DRxd\n", + "cj2V6vBBtEYlaSBwFunhuOWBnSQtV25VFbuYVH8zmggcHhErkLpjD2qm339EfAJ8q8iblYFvSVq3\n", + "5LJ64zBSt3eX3S6lhntEvN/m29mBN8uqpTci4q6ImFx8+wAwtMx6KtXFg2iNak3g2YgYGxETgWtI\n", + "D801jYi4D3i77Dp6IyJei4hHiq8/AJ4gPdfSNCLio+LLGYGBwIQSy6mYpKHAZsAFTDuQZTplt9yR\n", + "dJKkF4HdgZPLrqcP9gL+XHYRLW4RYFyb7/2AXEkkLQGsRmrUNA1JAyQ9AowH7ikGgDSTXwM/ASZ3\n", + "t2PNw13SXZJGd/DaEiAijo2IxYBLSIU3lO7qL/Y5FvgsIq4qsdQO9aT+JuK7/w1A0uzAH4HDihZ8\n", + "04iIyUW3zFDgG800z4ykLYDXI2IU3bTaoXrj3DsVERv1cNeraMCWb3f1S9qD9DFpg7oUVKEKfv/N\n", + "4GWg7U33RZl2qgurMUkzANcDV0TEjWXX01sR8a6k24A1gJEll9NTXwe2krQZMDMwp6TLImK3jnYu\n", + "e7TM0m2+HQ6MKquW3pC0Cekj0vDiZk0za4aHyh4Clpa0hKQZSbOP3lxyTf2GJAEXAmMi4oyy66mU\n", + "pPkkzVV8PQuwEU2UORFxTEQsGhFfAnYE/tZZsEP5fe6/KLoIHiHN1Pajkuup1JmkG8F3FUOTzim7\n", + "oEp09iBao4qIScDBwJ2k0QJ/iIgnyq2qMpKuBv4JLCNpnKQ9y66pAusAu5JGmYwqXs008mcI8Lci\n", + "bx4AbomIu0uuqS+67Kb0Q0xmZi2o7Ja7mZnVgMPdzKwFOdzNzFqQw93MrAU53M3MWpDD3cysBTnc\n", + "zcxakMPdzKwF/X+jAwLESxPhCwAAAABJRU5ErkJggg==\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "##This function provides an interactive mode to change polynomial degree. \n", + "@interact(n=[1,16])\n", + "def plot(n):\n", + " X = construct_poly(x,n) #n=d\n", + " w = wRidge(X,y,1.0)\n", + " plt.plot(x,X.dot(w))\n", + " plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n", + " plt.plot(x,y)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude?
" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "import scipy\n", + "from scipy import linalg\n", + "import numpy as np" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "A = np.array([[10,1,0.1],[10,1,0.2],[10,1.1,0.7]])\n", + "a = np.array([[10,1,0.1]])\n", + "A = a.T.dot(a)\n", + "\n", + "B,rank = linalg.pinv(A,return_rank=True)\n", + "#print rank\n", + "#print B\n", + "#print np.allclose(A,A.dot(B.dot(A)))" + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecFPX9x/HXmyYI2I2goKCCxgoGwYBlwN5jjF2KvYu9\n", + "/wRiL7EmauydxITYEhtGxt5Qwd5QoiIiNkREpXx+f3znYO9u727vdvdmd+/zfDz2cbc7OzOf3b37\n", + "7Hc+853vV2aGc865ytEq7QCcc84Vlid255yrMJ7YnXOuwnhid865CuOJ3TnnKownduecqzCe2N0i\n", + "kk6XdEPacTjn8uOJvQgkTZX0s6Tlazz+mqSFklZN7neTNE7STEnfSXpD0vBkWY/kubNr3PaoY5+x\n", + "pIPyidvMLjCzQ/LZRkY8Jyev53tJH0k6KWPZipLGSpqWvO5nJPWvZ1vHS5qSbGuGpFskdc5YPlDS\n", + "S8nyyZIGZSyLsryPQzOW35p8VlXLvpekZNlmWd7/hZJ2S5YvIeny5HV8I+kvktrk+P5Ufb55/w8m\n", + "r+GcBp4zVdKPGa/jkYxlZ9R4jT9KWiBpuSzbyfmzk3Rz8hpXz3hsKUl3Jn/zM5PfMz/LhZJ+yIjl\n", + "+oxlwyVNlDRL0qeSLpLUOmN55nqzJc2XdFXu72Tl8MReHAZ8BOxT9YCk9YEOybIqdwD/A1YFlgOG\n", + "AjNqbGtpM+uccftHPfusU64Jp8CGAssA2wFHS9orebwT8CKwEbAscBvwH0kd69jO/UA/M1sKWJvw\n", + "fp0JkCSfB4GLgKWBi4EHJS2Tsf60Gu/hHRnLDLgoY9lSlly1Z2ZPZ64H7AT8AFQlxdOS17Au0Dv5\n", + "/axGvkdq5PObyoCdMl7PdosWmJ1f43VeBEwws2+ybCenz07SpsDq1P67HA2sAPQE1gBWSh7LtH5G\n", + "PIdmPN4BGAksDwwAtgQWNRjMrFPGa+gCzAXuaeB9qUxm5rcC34CPCYnnpYzHLgXOABYCqyaPzQY2\n", + "qGMbPZLntsphf+cB8wl/yLOBq5LHFwJHAh8AU5LHrgQ+AWYBE4FNM7YzGrijxv6HEb58ZgJn5PGe\n", + "XFkVVx3LZwF9c9jO8sB4YGRyfyfgrRrPeQ84MPk9Aj6tZ3u3AOfk+BpuAW7KuP8y8IeM+/sAn+S4\n", + "rU+S93d2chuQPH4g8DbwDeELZNWMdS4nfPHPAl4nfKEcCvwC/Jxs5/56/ia3zCEuERolQxvx2Vb7\n", + "7IA2wKvA+slrXD1j2aPA4Rn3jwIeybi/EFgjx/0eDzxQx7LhwIdN/Xst95u32IvnBWApSWsnh4t7\n", + "AXdmec41kvaqKs9k0WCLzszOBJ4GjrLQYjk2Y/GuwMbAOsn9l4ANCa2tu4F/SGpXtaksmx9EaI1u\n", + "CZwtaS0ILTJJ3zYUW/JcAZsDb9axvA/QDviwnm3sK2kW4QtmppldWc8uWxGSXpVfSfoiKQldJmnJ\n", + "Gs8/UtLXyWH+7+vYf0dgd0ILtdqiGvvtlllaqMdmyc+qI7IXJe0KnA7sRmjVPg2MTfa/bbJOLzNb\n", + "GtgD+NrMrgfuYvFRx6717PMuSV9KelTSBvXEtSIwLofXUNdndzzwpJm9kWWVR4HdJS0jaVnCe/pQ\n", + "jec8JWm6QplytXp2vwV1/E0REvvtubyGipT2N0sl3khaR4RW+/mEUsSjQGuqt9iXAS4g/HHOB14j\n", + "lBxgcYv52xq3terY5wTgoBqPLQSiBmL9hnDoC9lb7CtnPPdFYK8mvB9jktfWNsuypYA3gFNz3Naa\n", + "ybaOT+4vn7yGvYC2hH/oBcC1yfKVgLUzXtOTwHUZ2+tL+JJrBWwPfA8MzLLfoSRHPRmPnQM8Q0jC\n", + "XZL3ZwGwUg6vo+r9bZXx2MMkRxrJ/VbAHELpaTDhSGQANY7iyOGoA/gtsAShnHEaMJ3wpVLzeTcB\n", + "N+f4WdT67IDuhCPEzhl/g5kt9iUIR1wLktujmX8XwKaEFv/SwNXJ9ltn2feBhKOe5bIsW43w/7Ra\n", + "If+vy+mWegCVeCMk9iHJP+T/CK2u/ZI/2EWJvcY6yyf/oJ8l92v94zewzwmZSSF5rNZhLaEm+Tbw\n", + "HeGLYgEwOFk2mtqJvVV9+8ghrqOBKWR8QWQs60BItH9t5Db3Al7LuL854Ujka8JRyKPAmXWsO4DQ\n", + "4q9r29cCl2Z5/HFgVI3H2ifJ5zNCi/U04OccX0O29/dtQjkl84t8DrBJsvwYQvlsJvDXjOSZczkp\n", + "Y1/vEGrumY8tSSirbJHD+lk/O0JLf2jyu2r+DQL/BP6crN8xeb//Xsc+WiXvx7o1Hv8d8EXNxzOW\n", + "n0U4R5B6Lkjr5qWYIjKzTwj1yu2BfzXw3K+BPwErJ4eojd5dQ49L2gw4GdjDzJYxs2UJ/8hFOYEn\n", + "6UDgFEJt9/May5YA7iPUpA9r5KbbAj9W3TGzp8ysv5ktTzgnsDYh0delUX/3kroTDvurHdqb2U9m\n", + "doyZdTOzNQlHDhNz3Gy2z+sT4FAzWzbj1tHMXkj2d7WZ9SOU1XoTPsu6ttWU/e9GKO88Wd+KDXx2\n", + "Q4BLJE0Hqj7z5yXtnfy+HeHLYK6ZzSF8Qe1Q165q/ETSdsD1hC+lt+pYbxi1S2Ytiif24jsIGGJm\n", + "c2suSLprrSupTVKXPQL4wMwya9e5Jt0ZhF4G9elMOET9SlI7SWcTDqcbI6d4JO1HOKm7jZlNrbGs\n", + "LaHl9iMwIodtHSxpxeT3dQgt43EZy/tKaitpKcJJ6k/MbHyyLJK0moLuhB4f92Ws+wdJnSS1krQN\n", + "4cjqgRohDAWeNbOPa8S1cnKTpE0ILcVRDb45wUyS1mzGY9cBZySvEUlLK+neKqmfpAHJe/cj8BPh\n", + "aAvCZ786dZDUXdKg5DNvL+lkwhHiszWe2mBdOofPrhewAeE8Tp/ksZ1Y/J6/DhySxNGBcPJ3crLt\n", + "dST1kdRaUifgMsLR0DvJ8iGE8wm/N7OsX6CSBgIrA3X1HmsZ0j5kqMQbSSkmy+NtCP+MVTX2q4D3\n", + "CYebXxISylrJsh5U7zVRdTuujn1uQqjBfgNckTxWs77ZilBDnUVoTZ1MOKIYkiwfBdyesf8F1FGK\n", + "IZxkm13Pe/ARi3tqVN2uSZZtkcT2Q43lg7JtG7iZcOg9O3mNpwDKWH43obT0HaHstULGsuMJyWEO\n", + "oUV8BdAxY/lTyXqzCLX7PbO8lneAA7I8vlnyWc9JnrNPI/9OxiSf+7dA/+Sx/QnJb1YS743J40MI\n", + "CXA24UvhDmDJZFnVeYdvgX9l2c86ybo/AF8Ratwb1XjOKoTeNatnWf9aFp+zqPezy7Lughp/g70J\n", + "vX2+Tm4PkZRqCOcR3k22PYNwlJtZxnkiiTFzv/+psb/rgNvSzgFp35S8GU2StIBuB35FOLS73syu\n", + "SvoW/51wEmMq4Z/luybvyDnnXM7yTexdgC5mNik5dHqFcGLjAOArM7tY0qnAsmZ2WkEids45V6+8\n", + "auxm9oWZTUp+/4FwOLoKsAuLT17cRkj2zjnnmkHBTp5K6kHoE/wioR9v1aXxMwh9iZ1zzjWDgiT2\n", + "pAwzjnCZ9+zMZRZqPU2v9zjnnGuUvAeGSro/jSNc2FLVpWmGpC5m9oWkroQz/zXX82TvnHNNYGb1\n", + "djvOK7EnY4DcBLxtZldkLHqA0Cf2ouTnfVlWbzC4UiBptJmNTjuOhnicheVxFlY5xFkOMUJujeJ8\n", + "W+yDSPrdSnoteex04ELgHoXxwacCe+a5H+eccznKK7Gb2TPUXaffKp9tO+ecaxofUqBhcdoB5ChO\n", + "O4AcxWkHkKM47QByFKcdQI7itAPIQZx2AIWS1wVKee1YsnKosTvnXCnJJXd6i9055yqMJ3bnnKsw\n", + "ntidc67CeGJ3zrkK44ndOecqjCd255yrMJ7YnXOuwnhid865CuOJ3TnnKownduecqzCe2J1zrsJ4\n", + "YnfOuQrjid055yqMJ3bnnKswntidc67CeGJ3zrlikZZLY7ee2J1zrtCkDkh/Bh5BavYJhTyxO+dc\n", + "IUl9gInA8sA2pDBNnSd255wrBKkV0knAeOACYF/MvksjlDZp7NQ55yqKtApwG9Ae6I/Zx2mG4y12\n", + "55zLh7Q78CoQA1HaSR28xe6cc00jdQauBDYDdsHsxZQjWsRb7M4511jSJsBrwEKgbykldfAWu3PO\n", + "5U5qA5wBHAUcidm4lCPKyhO7c87lQuoJ3AnMBTbCbFrKEdXJSzHOOVcfSUjDgJeAcYS+6SWb1KEA\n", + "iV3SzZJmSHoj47HRkj6T9Fpy2y7f/TjnXLOTlgXGAqcCW2F2GWYLU46qQYVosd8C1EzcBlxmZn2T\n", + "2yMF2I9zzjUfKQImATOBfphNTjeg3OVdYzezpyX1yLKo2cdHcM65vEntgD8Cw4CDMHs45YgarZg1\n", + "9mMkTZZ0k6Rlirgf55wrDGlt4HlgHaBPOSZ1KF5ivxboCfQBpgN/KtJ+nHMuf+EE6eHAM8ANwK6Y\n", + "fZlyVE1WlO6OlvGGSLoReDDb8ySNzrgbm1lcjHicc64ux+vyDQ9kvb+145dVuvHZgI425520Y8qk\n", + "UOuPGrVOIUaUTGrsD5rZ+sn9rmY2Pfn9eGBjM9u3xjpmZl6Hd841O4kOwO/25a6TLuWkvk+yxaQT\n", + "+dPRn7PK82Y0+zC7jZFL7sw7sUsaC2wBrADMAEYRvl36EHrHfAwcZmYzGhucc84VioSAAcCIDvy4\n", + "x40c/MMuPNDxR5bc91f25WNpx5erZknsTeWJ3TnXHCRWBoYCI4BWv2fcI2PZZ9t2zJsMHI7Zt6kG\n", + "2Ei55E4fUsA5V3Ek2gO7AAcQWun/bM/cg+fQsX8r7AzgROCONGY3ag6e2J1zFSEptfQjtMz3Ioy+\n", + "eCuwu6FlCBNhdAQGYPZRSmE2Cx8rxjlX1iS6SJwEvEG4/H86sJEZW5txl6HtCEn+aWDzSk/q4C12\n", + "51wZklgC2IlQahkE/As4AnhmUa8WqRNwBTCY0C/9BQDFcTvCdTa9atyWsiga0LyvpDj85Klzriwk\n", + "pZa+hFLLPsCbhLGqxpkxJ/O53yy11G9nLrPM3c+st94HJx1xxGPfde68KosTeDdgGvA+8EHmzaJo\n", + "SnO9nqbyXjHOubIn8StgP0LrvDOhbn47E+L/EZJ0L6A30EtmvZafNWuT2UsuuUKbBQu+mtOhw2Rq\n", + "J/CPLYp+SeGlFIQndufcIopjAUsDXeq5rQh8Qe1k+KlF0YJmi1W0BXZENoLlf4lYf9bzbDf9Tfp9\n", + "a7Ra1PJeHfi2KsYe06fPHH3bbb9b+5NPvp/fqtV+g958s+Rb303hid25FkBx3AFYifoTdtXtZ0Li\n", + "znabAXydPK831evPywMfUT3ZVyX/zy2KmpxIki+cFYFevLHUEGYusRPtFm7Iaj/Op+tPbWht36Nq\n", + "XzRVv39oURRKMNJ+hHr6RUBZjJneVJ7YnatQiuPuhO57GwEdqDtZV0vcFkU/NnF/HYE1qX3CsTfQ\n", + "CfiQ6km36vZlVdJXHC+XZd1eGL35pVVrPu0Any1pLNREVp57H2vPfpqQvGfVHZiWAa4hXOm+H2av\n", + "NeX1lRNP7M5VIMXxZsDfgcsJIxHOyqfFXIB4liJ7wu9F6Hn3CbAy0JaqxD9fU3hqxSV5bKW+vNt5\n", + "I2a1e5BwInSCGbm1tqXNgduBfwOnYNakL61y44nduQqSlCwOB0YDwyyKHk03ooYlrfRVCX3Lv2Rw\n", + "tC6hV8v+wBTCidB7zKi7VQ5hWN0wHlU3YBVCF8Z9gYMx+0+Rwi9JPqSAcxVCcbwEcDUwEBhkUfRh\n", + "yiHlxKLoG4UUtA8hoXcltLI3N+N9AKQ2iO6EhN0t42fm7ysDc4DPktuHhIkwqg0u6AJvsTtX4hTH\n", + "XYF/Ek5uDrcomp1ySDnroi9GLM2sy/ow6bXdGffy7/nXd21YsArVk/iKhHlFp7E4cU+r9dNsbiov\n", + "osR4Kca5Mqc47g+MI9TSz7UoKo3eHqE0sjTVW9WLWtoLaNVtLh16t2VeO+DzJfjlI7In7s+ALzCb\n", + "n8KrKEue2J0rY4rj4cAlwCEWRfc3347VitCKrlkOqVkiMeBTarSu/8uQhf/HOSdOY5W3fqDT8K9t\n", + "+bI5wigHntidK0OK47bApcD2wO8sit4u3MbVjlDnzlbHrvrZFfieussiVaWR72tvnr0J5wLOAq4v\n", + "9dmIypGfPHWuzCiOVwDuIVxI1N+i6LvcV1ZH6j8B2Q1YjlCrr1kWmZjx2OeY/dSouMP455cDWwHb\n", + "mFHx/clLmSd250qE4nhD4D5CH/UzF13CH+rZy1J3wq762Z7aret3gf+yOInPwKygQwNIrAn8g3BB\n", + "0m/MqNWSd83LSzHOpUlqDay0w4UXDn+ib9/Tjrr33nF/uu66L6mduH+hvrJI+PlNc88IJLEH4crP\n", + "0cA1XnopPi/FOFcqpI0Jk75Xa3HPb9Wqy6mHHTbv9dVXb/PIKac8F02e/BOhvj2e6vXsH9IKPZtk\n", + "PPQ/Ec4DbGfGKymH5DJ4Yneu2KQNgYeAOwmJ+gXgs7FDhsze/8wzL1rYqlV7YM9o0qSZaYaZK4k1\n", + "COcBphJKL7mfB3DNwhO7c8UkLUvohz4Ss7sXPRzHvwbuBx4GTrIompdShI0isTtwLXAucLWXXkqT\n", + "19idK5bQH/xB4APMjlv0cBzvAtwInGpRdEta4TVGUnq5GNgF2MuMl1IOqcXyGrtz6TqbMOPPyQCK\n", + "41bAmcChwM4WRS+mGFvOJHoSeup8Tpgk+tuUQ3IN8MTuXDFIOwEHA/0wm6c4rprSrSuhf/r0NMPL\n", + "lcTvgOuBC4ArvPRSHjyxO1do0prAzcCumH2hOF6DUE9/HtjXoujnVOPLgUQ74ELg98AuZryQckiu\n", + "EVqlHYBzFSVc/XkvMBqz5xXHWwPPEfp6H1omSX014CnCjEkbeVIvP57YnSuUcIXojcAro4cPv1Zx\n", + "fBJh7PE9LIquSXOWo1xJ7Ay8RBgmeFczvkk5JNcEXopxrnBGAmudfcABW54zbNgdwK+BARZFn6Qc\n", + "V4Mk2gLnA3sBu5nxXMohuTzk3WKXdLOkGZLeyHhsOUnjJb0v6TGFCWedq1zSFsDpZx144NHnDBv2\n", + "OCBgszJJ6t2BJ4F1gL6e1MtfIUoxtwDb1XjsNGC8mfUmDEB0WgH241xpklYBxp5y6KEXnzd06Djg\n", + "bmB/i6KSn1xZYgfgZcLJ3Z3N+DrlkFwBFOQCJUk9gAfNbP3k/rvAFmY2Q1IXIDaztWus4xcoufIn\n", + "LbEQ4sNPOGHmDTvvPAAYalH0WNphNSQpvZwD7AfsY8YzKYfkcpTmBUor2eJJZmcAKxVpP86l6of2\n", + "7a/c96yzuj44cGBnYKBF0ZS0Y2qIRDfgb8APhF4vZTFGjctd0U+emplJynpYIGl0xt3YzOJix+Nc\n", + "oTy37rojj7nyyuFv9uw5Hmm/cphkWmI7woVSVwMXmFEac6i6OkmKgKhR6xSxFBOZ2ReSugITvBTj\n", + "KslRxx037N5NN72l09y5f/mge/fjSmaS6TpItAHGAMOB/cx4MuWQXBOlWYp5gPAHdFHy874i7ce5\n", + "Ztf9nnuO+XHIkCv2e/zxS6666qqS7xggsTLhhO48wjC7MxpYxZW5vFvsksYSJhBYgVBPP5twhv0e\n", + "YFXCmM17mtl3NdbzFrsrK4rjtq0WLrxs5a++OvDWCy8cu+Wrrx6cdkwNkdiacJHUtcB5ZhR0WjzX\n", + "/HLJnT5sr3M5UByvCNyz3kcfdf/vCSdM+9WsWVtiNj/tuOoi0RoYBRwEDDXjiZRDcgXiw/Y6VwCK\n", + "4z7AfZu89dYrTx977OptFi78Q4kn9S6E0guE0ssXacbjmp+PFeNcPRTHewPjN3399SueP/rozZKk\n", + "XrLdAyWGAK8CTwNbe1JvmbzF7lwWiuPWhLFT9tzgww93eXrkyJuBMzB7OeXQskpKL2cBhwHDzHg8\n", + "5ZBcirzG7lwNiuNlCaWMJTrOnbvnDzvscB3wDWaHphxaVhIrESbKbgvsa8bnKYfkiiiX3OmlGOcy\n", + "JNPXPQB8BGz7ww47HAh0B45JNbA6SETAK8CLwFae1B14Kca5mo4gjMx4jA0ePBg4HuiPWUlNkCGx\n", + "CnAUcAAwwoxHUw7JlRAvxTiXUByvSmj9bmaDB88ltIL3wWxCupEFEu2BXQnJvD9hMowxZkxLNTDX\n", + "rLy7o3M5UhyLcBHPlTZ48FTgGeDStJO6hICNCcl8T8IXz63A780o+WGBXTo8sTsX7AN0X2PatN0I\n", + "Cf4j4E9pBSPRFdgfGAG0IyTzvmaU/MQdLn1einEtnuJ4BeBNYBcbPLgPYYq7AZj90KxxiCWAnQmt\n", + "84HAOEJCf9aMkp8v1TUPL8U4l5srgLtt8GAB5wKbNVdST0otGxGS+d7AZEIy39OMOc0Rg6s8nthd\n", + "i6Y43h4YOPaPfxwCPAUcgtl7Rd9v6Hu+HyGhdyQk835mTC32vl3l81KMa7EUx52BN5f86adD5my/\n", + "/enAs5idVbT9iXbAjoRkvjlhOOtbgKd9wguXKx/d0bl6KI6vBjrZ4MFfA+sBO2JW8GFtJfoQToLu\n", + "C7xDSObjzCj5GZdc6fEau3N1UBwPBHafdPDBZwL/B/QrZFKXWJGQyA8AlgVuA35rRsnPierKnyd2\n", + "1+IojpcAbhz86quXbjhlysXANph9k/d2RVtge0IyHww8CJwAxF5qcc3JE7tric5oN2/eR4+feOLh\n", + "wAmYvZbPxiTWJ5Ra9gM+JJRahpvxfd6ROtcEnthdi6I4Xg+zI9844IBXW8FjmN3RpO2I5QmllhHA\n", + "rwjTz21mxgeFi9a5pvHE7lqMZIz1G4+6775ne0+btiKhTJL7+qINsB0hmW8N/Ac4DXjC5xJ1pcQT\n", + "u2tJjlnx22+XvPLqqzcGNsbsl1xWkliXkMz3J0zOfitwsBnf1b2Wc+nxxO5aBMVxz1YLFpz91MiR\n", + "C1ub7YZZveOWSyxLGD9mBNCNUGoZYsY7xY/Wufx4YncVT3Gs1gsW3HDK2LFz1/7004swezrr88L0\n", + "clsTerVsBzwCjALGm1Gyk1c7V5MndlfxtHDhsFW//HKjs2+//WHg6lrLxdrAcGAYMI1QajnCjLy7\n", + "QDqXBk/srqIpjlda8pdfrr7jvPOmt5837xCSS60llgb2IrTOewJ3ANua8WaK4TpXEJ7YXUVbZebM\n", + "u/eI49aD3nprB2E/o0Wllh2B8cB5wCNeanGVxBO7q1gb3nTT8HYdOmwx8O5PjxZ2AKHU8hXhAqJj\n", + "zfgq3QidKw4fBMxVpHX2fqTH13vM/eB3F77z9fUTzxBwF3CbGZPTjs1VHo3RUkDvjNtayc/5NsoG\n", + "FHRfPgiYa0kkWgERcMCgE6bt2+PlWT/fOvHEw4GHzMipz7pzddEYtQVWp3byXgtYCvgAeA94n3Dx\n", + "2uXJ782uqIld0lTge2ABMM/M+hdzf65lklid0KtlODBrWHTJF48NXN0Oeui5Xz9kS/wv5fBcGdEY\n", + "CehK9uS9KvAZIVm/D7wG/J2QzD+3UVYyA70Vu8VuQGQFGDnPuUwSnYA/EC4gWhe4G9jtzW492ux6\n", + "8HnPbzhlygnn3nSTJ3WXlcaoM9mTd2/gRxYn7/eAp5OfH9ko+zmVgBupOUoxXkd3BZGUWjYjJPPd\n", + "CP9wVwH/MeNnpOWO3+XIKa3MJj5yyilXpRiqKwFJ6aQn2ZP30oTSSVXyfpgw9+0HNsq+TSXgAmqO\n", + "FvvjkhYAfzWzG4q8P1eBJHoQerQMB+YSerWcbsYXGU9q9eQGG9x/4447LvHDkkv+LpVAXbNLSidd\n", + "yJ68VyNccFZV954M3JP8Pq2USieFVuzEPsjMpktaERgv6V3LuJxb0uiM58ZmFhc5HlcmJDoCvye0\n", + "zvsAfyNcUPSKGbW6cv3Utu3ow084oc+8Nm1GWhR9UXO5K64kwbYF2mXcat7PZVmuj3cE1iAk8J9Y\n", + "3PJ+H3gm+TmlXEon9ZEUEToF5Kyoid3Mpic/Z0q6F+hPOHwOJky4z6JoUjFjcOVDQsAgQjLfHXge\n", + "+CvwgBk/1bPizpcMHXrs1C5dXvu5XbsbmyPWYksSZRuKkyQLkVhrLmsLzEtuv2S51fV4fcuqHp+d\n", + "Zdlc4CPg/UoondQnafDGVfcljWponaIldklLAq3NbLakjsA2wJgaTzsN2LtYMbjyINGdUGoZAcwn\n", + "jNWyrhn1jsCYrNzrve7db/3jsGHMb9NmhEVROhdm1EFj1JMwsNgQYCUal3AX0LRE2NCyOU1Yp6HH\n", + "59molC6KcbUUs8W+EnCvpKr93GVmj9V4zpaK494WRan09XTpkehAOAE6AuhHqH3uD7yUrdRSx0ZW\n", + "XggPb3vxxTPnt2lzg0XRR8WKN1cao2UJSXxrYCugE/A4YaTIT2hE8qzkGrArrqIldjP7mFAbrc9f\n", + "gFOAg4sVhysdSallAGGslj2AlwknQnc1Y24jN7YCMH7kMcdM/l+XLqsCVxY43NzCGKN2wG8JiXxr\n", + "4NfAs4RxaP4CvOktWdfcUh1SgAkTViB0OdrQoujTVAJxzUJiJcIIij0Jyfx2Mz5r4saWAZ54vWfP\n", + "Zza8+eZ9gC0til4vWLD17TrUvtdlcSLflHDSbnxye64STti50pXLkAKpjxWjOL4UaGNRdFwqgbii\n", + "k4iAOwm189F5jaQodQIeBV7RE090Q3rbouisQsRZ5y7HqCuhrFJVXvmJxYl8go2yr4u5f+cylUti\n", + "Xxl4C+htUTQzlWBcUSQXFJ0BHAWMMOPRPDfYnjAGx9S248c/PL9Nm3OBPhZFdfeYacpuxqgjsAWL\n", + "E/kqwASSZG6jbEoh9+dcY5TFIGAWRZ8rju8BjgX+L+14XGFIrEhopXcA+pkxLc8NtgX+Aczsdccd\n", + "p8xv0+YNYM9CJHWNUWvgNywur/wGeIWQyA8GJtooW5DvfpxrLqm32AEUx2sALwKrWxR9n0pArmAk\n", + "NiOM3XIn8H95T2IhtU621wHYXRMmXAf8ZFF0VJM3OUZrsDiRDwY+Z3F55SkbZT/kFbNzRVIWLXYA\n", + "i6IpiuNHgSOAi9KOxzVNUno5FRgJHGDGwwXYaCvgRmB5YCdNmLA5IRmv16jNjNFyLO6GuDXhS2I8\n", + "cB9wtI0KF9M5VwlKosUOoDhen/CP1tOiqHFd31zqJFYAbicMrrS3Gfn3cgoXQVwF9AW21YQJBrwO\n", + "HGdR9O/j8awPAAAYUUlEQVR6Vx2jJYCBLE7kaxEuNa9qlb/l3RBdOSqLk6fVHovj+4HHLIr+kkpQ\n", + "rkkkBgFjCeO5nGnGvAJt+ALCFctDMJulOL4E6GZRtE+tp4ZuiOuzuPfKIOAdFifyF7wboqsE5ZjY\n", + "NyEMXL+mRVFhkoMrmqT0chJwInCQGfW2ohu58TOAfYEIs68Ux/0IPWLWtyj6UmPUClibcHHQYEJC\n", + "n8PiRP5EpY8h4lqmskvsAIrjJ4DbLIpuSyEslyOJ5YHbCLXvvcz4pIAbHwkcDWyO2XTFcVtswat8\n", + "9exDvD3qR0IyHwB8Sxgo7ClCN8TUhxRwrtjKNbFvBVwNrGtR5GNllCCJqiOrfxLGRS/cfKLSwQb/\n", + "d+pWHHjJpqwG/JYeB+9M57VW4I2TnwVeICTz522UzSjYfp0rE2XTK6aG/xKG6fwd8K+UY3EZkrFe\n", + "jif0fDnUjPsLst0xWhrof+5/OfSwDuy86UH89N4K3AA8z3K/nUL3vZbgl6/W8guDnMtNybXYARTH\n", + "vwPOAjYutWFYWyqJZQljvHQllF6mNmk7oTa+FqGcUnXrMWwS/7v6IVY7aRvOuKEf99go+0Jx3Ap4\n", + "EviHRZFPdecc5dtiB3gAOI/Qu6HmUL+umUn0J5Re7gf2bEzpRWO0FKEeXpXEBwCzSMopwF+/P58V\n", + "Ov/C7UB0/YM28frFqx9G+Bv1XlLONUJJttgBFMf7AwdbFEXNF5XLlJRejgXOBA4z4956nx9a472p\n", + "3hrvCbzK4kT+go2yzLlKNyOU3HbD7JlFD8dxN+A1ILIoequAL8u5slaWJ08XLY/jNoQhffe3KHq2\n", + "+SJzABLLADcRJgTe04xaPU6S1nh/FifxTajeGn8emGyjLHvXVakf8BCwH2bjFz0cxyIctU20KKo5\n", + "65ZzLVo5l2KwKJqvOL4YOB3YKe14WhKJfoTSy0PAvmb8nFwAlFkb3wRYndCqfh64ATiwWmu8/p2s\n", + "B/wbODgzqSf2IrT0d8//1TjX8pRsix1AcdyeMGHt9hZFk5snspYrKb0cBZzN8u+dyDFrT6d6bXw2\n", + "tVvjje/qKPUiTM57EmZjqy2K4+UJwzjvZlH0fNNfjXOVqaxLMYueF8cnAxtlu4zcFYbGSEzdfCM+\n", + "3PZaVnqzJ7/+19e0+bmqxp3Zbzz/gbKk1QgXFP0Rs5tqLY7j24FvLYpG5r0v5ypQWZdiMlwHfKQ4\n", + "7mVR9EHawVQCjVFnMmvjC1sPYrkpHen18FS6vnoBbX5+mqa2xuvdsboSJna+rI6kvi2wOY0cudE5\n", + "V13JJ3aLotmK42sIk14fknY85Sapjfcm1MSryiprAJMwnufZUz7jxWMXMHuV/c34e/ECCZNPA7dg\n", + "VmviacVxJ8KX+GEWRT4WunN5KPlSDCyqu34AbGBR1LQJkFuIWq3xkNB/oHptfBKjrT1wPWEgrT3M\n", + "KN7RkLQ08ATwGGanZ31KHF8BLGdRNKxocThXASqixr7o+XF8GSCLouOLGFZZSVrjvajeb3xNYBIZ\n", + "idxG2efV1hMbEqaZmwAcZ0bxxr+XOhIuMnsVOJYsf3DJqJ73AutZFPnE0M7Vo9IS+yrAG4RJr78q\n", + "XmSlS2PUidqt8TnUaI3XVRtPer0cQriqd6QZdxc3YLUndGn8hNCtsdagborjdoSkf65F0d+KGo9z\n", + "FaBSTp4CYFE0TXH8T8KVkGc3xz41RhsQ6sLvJ7cPMn6fYqOsaC3dpDW+JtVb471Y3Bq/BTi0Zmu8\n", + "zu2JTsBfCZNRbGrGe8WIO2OHbYF7gK+AQ7Il9cTpwMdQxPq+cy1M2bTYARTHaxK63zXLpNdJcu1K\n", + "OPnYK/lZ9fvqwAyyJ/2pNsoaNYFzHa3xH6ndGm/0LEAS6xNKL88Ax5rxY2O30cgdtgbuAjoCv8ey\n", + "X3mqOF6X0J+9r587cS43FVWKWbReHN8NTLIourgIYeUexxi1AValdsLvDawM/I/aCf994HPAqLs1\n", + "ntlvfFpeMYbSy4HAhcAJZtyRz/Zy3GnV5NOrATti9lPWp8Vxa8IXze0WRdcWPS7nKkSlJvYNgEcJ\n", + "rfaSnPQ6mUh5dRYn/MykvzTwE9lr4wWbk1OiI3AtsBGh18s7hdp2PTsVcCXwG2BbzOrstqg4PhbY\n", + "A9jCJ1RxLnepJnZJ2wFXAK2BG83sosYGV+e24/hB4GGLomvyj7R5Jd0RO+Y8pkpT9iHWJZReXgSO\n", + "NmNOsfZVY8fnAdsBW2L2XZ1Pi+PVgFeATS2K3m2W2JyrEKkldoUa63uECYanAS8D+5jZOxnPySex\n", + "/xYYC/TySa+rkxgBXAKcbMatzbjjM4D9gC0wq7PXUjJy48PA0xZF5zVXeM5VilxyZ6si7bs/8KGZ\n", + "TbVw4uxvwK6F2ngyONTHgI8fk5BYUuIWwhW6UTMn9WMJtfyt6kvqif0JJ6RTPUfiXCUrVmJfBfg0\n", + "4/5nyWOFdD5wWjJ9WoslsZLEnsBLhLJXfzOab2IK6UDgREL5pd5BwhTHvwIuBQ7yIy3niqdY/dhz\n", + "qu9IGp1xNzazuBH7eJxwAnJXqH9mn0oi8StgCyACBhNav08TvujGmuX23hcomL2Bc4DBmP2v3qeG\n", + "EsyfgTssiiY2R3jOVQJJEeH/Pfd1ilRj3wQYbWbbJfdPBxZmnkDNp8a+aBtxvBtwBtC/Uie9lliR\n", + "xYk8AroREnmc3CaZsSCFwHYhjDWzNWZvNPj0ON6P8Fn9xqIoaxdI51zD0rzydCLQS1IPQr/tvShO\n", + "Pfx+wuXxWxGuEC17EitQPZF3J/T3joERhETeqIufCk7aitBXfYcck3o34HLChCme1J0rsqIkdjOb\n", + "L+loQn/z1sBNmT1iCrafKFqoOL6Q0BIsy8SeJPLNWZzIV2NxIj8QeC31RJ5J2pTQI+n3mDVYUknO\n", + "gdwCXG1R9Eqxw3POleEFSrW2E8dtCVd37mtR9Fz+kRWXxHJUb5H3AJ5lcWnl1ZJK5Jmk3xC6Ku6P\n", + "2WM5rRLHRwHDgEEWRaX5upwrIxV55WnWbcXxkYTD/J0Lsb1CShJ5Zot8deA5wpC5MSGRl34PkTD5\n", + "9OPAYZjdn9Mqcdyb8FoHWRQVd9Ax51qIlpTYOxAmvd7Wouj1QmyzybGIZameyNcgDBlQlchfKYtE\n", + "nqmeyafrXCWO2xCORO6wKPpzEaNzrkVpMYkdQHF8CuHCqD2bc+wRiWWonsh7UT2RTyy7RJ5JWpUw\n", + "+fS5mN2Y82pxfBah5LStjwXjXOG0tMTemVAqmE24AKbeftVN3o9YmuqJvDdhRMbMRF7YSaDTEiaf\n", + "fgr4C2ZX5LxaHG8EPELo2vhpQ893zuWuRSV2WHT4fwpwPGECh5vy7d+eJPJNCRcDRcBahMG1qhL5\n", + "yxWTyDNJywNPAmMxy3lMF8Vxe8IAX+dbFN1VrPCca6laXGJftO04Xh+4DfgCOMSiKOdxzSWWonoi\n", + "/zUhkceEZP6yGQUbXrckhcmn/0s4Ajo92zylda4ax5cSumzuWakXjTmXphab2GFRN8gzgSMJY5nc\n", + "mS3RSHSmeiJfhzDuSkxI5C9VfCLPFCaffpQw6ccxjUzqWxD6uG/QUuelda7YWnRiX7SfUO+9DZgC\n", + "HM7gaA4wiMWJfF3CsMIxixN5y7w6Mkw+/QBhqOWD6pmntPaqcbwUMBk4xqLo30WK0LkWzxM7hEmc\n", + "l/0l4rj3z6DPd/24stdCnljpBRYn8hdbbCLPFCaf/idhdqd9MWvU+DOK45uAhRZFhxQjPOdc0CIT\n", + "u0QnYCCLRz9cn3AyL+agj75g30+OoxWTgSO9XJAIE6PcCXQmDBXQqJPBiuNdCLNlbWhRNLsIETrn\n", + "Ei0qsUv0B/4E9AVeZXGL/AUzFs2NmlzMdA5htp8jLYpazJC/WYXJp68HegI7YdaoeWQVxysSSjB7\n", + "WRQ9XYQInXMZWkRilxBwLOFE6fHAvWb82OB6cbwpcCuhD/oxFkXf5htL2QmTT18BbAxsU9/k01lX\n", + "D2OsjwM+tCg6pQgROudqSHNqvGaRXPX5T2AosIkZd+WS1AEsip4BNgS+Bd5QHO9YvEhL1jnAZoTh\n", + "dxuV1BNDCVfanl3QqJxzeSnbFrtEP+DvwEPASfl0SVQcDwZuJpRujrcomtXUbZWNMPnJUMLk0zMb\n", + "vXocr0o4d7G1RdGkQofnnMuuIlvsEpI4mpDQTzXjmHz7mVsUTQA2AOYBryuOty5AqKVLOgY4iDD5\n", + "dFOSetUY65d5Uneu9JRViz25vP8GYE1gTzM+LHhccbwtYXagfwMnWxQ1pURRuqQDgNGElvrUJm0i\n", + "jo8F9gY29zHWnWteFdVil9iIcOj/FTCwGEkdwKLoUUIXyfaE1vsWxdhPKqS9gHMJ85RObdIm4nht\n", + "4P+AYZ7UnStNJd9iT3q9HEY40Xe0GX8venBV+47jnYC/Ek7Qnm5RlNOJ2ZIk7Uw42slp8umsmwjD\n", + "NDxHGFztukKG55zLTdm32JNxXO4GjgAGNWdSB0gujd8AWAGYpDge2Jz7LxhpS+AmYOemJvXEGcDX\n", + "hC8751yJKtkWu8SGwD8IPVWOy7zIKA2K492BPwN3AGdbFJXHMATSIOBe4A+YPdXkzcRxP+A/wEaN\n", + "GS3TOVdYZdliT3q9HEIYMnaMGYelndQBLIrGEVrvqwOvKo43TjmkhkkbEZL60DyTegfCF9pIT+rO\n", + "lb6SarEn47xcR7hwaA8z3k0luHokV1vuBVxJqFn/0aKo9CbakNYljKl+OGb35bWpOL4c6GJRtE9B\n", + "YnPONVlZtdgl1gcmEkYXHFCKSR3Aosgsiv4G9CG04F9WHPdJOazqpDWBx4ATCpDUhwB7AEcVIjTn\n", + "XPGl3mJPer0cAFwEnGjG7akE1ARJ630YcClwNXCBRVG6E1eHyaefBM7H7Ia8NhXHSwOvA4dZFD1S\n", + "iPCcc/kp+UHAwDoB1wD9CKWXt1MJJk+K426Ei5pWAIZbFL2VTiDqQph8+prGTD5d5+bi+FZgrkXR\n", + "EfluyzlXGOVQinkZMKB/uSZ1AIuiz4DtCd0AY8XxqYrj1s0aRJh8ejxwe4GS+jDClIEn57st51zz\n", + "SrvFfoAZt6YSQJEojnsQBhTrAIywKHqv+DvVUoQTpU8ApzVmntKsm4vjQwkjNm6b2tGHcy6rki/F\n", + "NMfUeGlIBsk6AhgDnAdcaVGU8/yhjduZOgKPEGrhRxcgqZ9MiH1ri6IpBYjQOVdAqSV2SaOBg4Gq\n", + "kQNPN7NHajynYhN7FcXxmoRREA04oOCJUloCeBD4HDiwMZNP19pUOBF8DrA7Ial/VpggnXOFlGZi\n", + "HwXMNrPL8gmuEiS19mMJl+OPAq4rSOs9TD79D+AXwuTTTR6QKznCuIJQU9/WoqjRQ/k655pH2idP\n", + "Kz5p58KiaIFF0eWEmYqGAeMVx6vltdEw+fRtQBtg/zyTehvCUUVfYLAndefKXzET+zGSJku6SdIy\n", + "RdxPWbAoepfQIh4PTFQcH5yUPxonzFN6HdAF2AOzJl/1qjheArgHWInQUq/8maOcawGaXIqRNJ6Q\n", + "XGo6kzBBdFXL7xygq5kdVGP9FlGKyUZxvB6hxf0lcEjO9eyQ1C8H+tOEyadrxNCRMI7MLGC/khwW\n", + "wTlXS0n0ipHUA3jQzNav8bgReo1Uic0sLmowTSEp354mWTcbxjY/HTgaOAm4w6Ko/v1I5wA7AYMx\n", + "+y6PfS9DGKnxPeBQnzDDudIlKQKijIdGpXXytKuZTU9+Px7Y2Mz2rfGc0m+xS2sDdwEj8hzHvO5d\n", + "xHFfQuv9Y8Kl+1/UEcupwHCaOPl0xv5+BTxKGHbghKJ1w3TOFUWaJ08vkvS6pMnAFsDxRdpPsb1H\n", + "GAPmCaQTkQr+flkUvUYYUuENYLLieO9atXfpKOBQwuxH+ST17oQhBx4Ajvek7lxl8guUciH1BG4H\n", + "5hNa7/8rym7CGO+3AW8BR1oUzUQaAfwR2Lyp85Qm2+5FOHF7lUVRnd1QnXOlLe3ujpXD7GNCjesR\n", + "YCLS0OREZmF3E0UvAxsRyjKvHz1y5CXA+eQx+TSA4nh9IAbO9aTuXOXzFntjSX2AO4F3CJNYfF2M\n", + "3YwZPvyk27fZ5sI5HTo8MmO55YZZFH3TlO0ojgcQSi/HWhQ165yxzrnCK4leMXXuuFwTO4DUnjAG\n", + "zN7AQdQYLqEA2x8C/O3tVVfdfd3bbvsD4TL/wyyK/tOozcTxYODvhOEMGrWuc640eWIvNmkwcCvw\n", + "b+AUzOYUYJsDgfvImHxacbwF4erQmHDSs8ELiRTHuxDGiN/DoujJvONyzpUEr7EXm9kEwvysnYFX\n", + "kfrntb06Jp9OEvMGwM/AG4rjberdTBzvC1wP7OhJ3bmWx1vshSLtAfyZMCPU+Zg1boo8aR3CmOpH\n", + "YnZvnU+L460JLfGHgZMtimbXWH44cBY+lrpzFclb7M3J7B+EgbQGAs8g9c55XWkNwuTTJ9eX1AEs\n", + "isYTWu9tCf3eo0WbieNTgVOAzT2pO9dyeYu90EI3yCMJwyWcDVxb75AEUtVFQxdgdn2jdhXHOxKm\n", + "4/sX8COwC2Es9WlNC945V+r85GmapLWAO4CvCZNgTM/ynKrJp6/F7PIm7SaOlwWuAtYEdrYo+qrJ\n", + "MTvnSp4n9rSFyTDOBA4HjsJsXMay5Qi9XP6B2TmpxOecKzue2EuFNIDQen+eMJuSAY8TEvupxRg9\n", + "0jlXmTyxl5Iw6fSlwPbAF8CrhFa8J3XnXM48sZciaQdgE2B0PpNPO+daJk/szjlXYbwfu3POtUCe\n", + "2J1zrsJ4YnfOuQrjid055yqMJ3bnnKswntidc67CeGJ3zrkK44ndOecqjCd255yrMJ7YnXOuwnhi\n", + "d865CuOJ3TnnKownduecqzCe2J1zrsI0ObFL2kPSW5IWSNqoxrLTJX0g6V1J2+QfpnPOuVzl02J/\n", + "A9iNMBnzIpLWAfYC1gG2A66RVLZHBpKitGPIhcdZWB5nYZVDnOUQY66anHDN7F0zez/Lol2BsWY2\n", + "z8ymAh8C/Zu6nxIQpR1AjqK0A8hRlHYAOYrSDiBHUdoB5ChKO4AcRGkHUCjFaEmvDHyWcf8zYJUi\n", + "7Mc551wWbepbKGk80CXLojPM7MFG7McnbHbOuWaS95ynkiYAJ5rZq8n90wDM7MLk/iPAKDN7scZ6\n", + "nuydc64JGprztN4WeyNk7uQB4G5JlxFKML2AlxobmHPOuabJp7vjbpI+BTYB/iPpYQAzexu4B3gb\n", + "eBg40vI9LHDOOZezvEsxzjnnSkuq/cslnSNpsqRJkv4rqXua8dRF0iWS3kli/ZekpdOOKZv6Lhor\n", + "BZK2Sy5a+0DSqWnHk42kmyXNkPRG2rHURVJ3SROSz/pNScemHVM2ktpLejH5/35b0gVpx1QfSa0l\n", + "vSapMR1DmpWkqZJeT+KsVeKukvaFQxeb2YZm1ge4DxiVcjx1eQxY18w2BN4HTk85nrpkvWisFEhq\n", + "DfyZcNHaOsA+kn6dblRZ3UKIsZTNA443s3UJpdCjSvG9NLOfgMHJ//cGwGBJm6YcVn1GEkrIpVzG\n", + "MCAys75mVuf1QakmdjObnXG3E/BVWrHUx8zGm9nC5O6LQLc046lLPReNlYL+wIdmNtXM5gF/I1zM\n", + "VlLM7Gng27TjqI+ZfWFmk5LffwDeIVw/UnLM7Mfk13ZAa+CbFMOpk6RuwA7AjVTvDFKKGowv7RY7\n", + "ks6T9AkwHLgw7XhycCDwUNpBlKFVgE8z7vuFawUgqQfQl9DgKDmSWkmaBMwAJiSdK0rR5cDJwMKG\n", + "npgyAx6XNFHSIXU9qVDdHevU0EVOZnYmcGbS//1y4IBix5RNLhdjSToT+MXM7m7W4DIU8KKx5lbK\n", + "h7dlSVIn4J/AyKTlXnKSI90+yXmpRyVFZhanHFY1knYCvjSz18pgvJhBZjZd0orAeEnvJkeZ1RQ9\n", + "sZvZ1jk+9W5SbAk3FKekEYRDtS2bJaA6NOL9LDXTgMyT492pPvSEawRJbYFxwJ1mdl/a8TTEzGZJ\n", + "+g/QD4hTDqemgcAuknYA2gNLSbrdzIalHFctZjY9+TlT0r2EEmetxJ52r5heGXd3BV5LK5b6SNqO\n", + "cJi2a3JCqByUWp1wItBLUg9J7QgjgD6QckxlSZKAm4C3zeyKtOOpi6QVJC2T/N4B2JoS/B83szPM\n", + "rLuZ9QT2Bp4oxaQuaUlJnZPfOwLbEDpM1JJ2jf0CSW8kNbgIODHleOpyNeHk7vikm9E1aQeUTV0X\n", + "jZUCM5sPHA08Suh58HczeyfdqGqTNBZ4Dugt6VNJqZQGGzAI2J/Qy+S15FaKPXm6Ak8k/98vAg+a\n", + "2X9TjikXpVo2XAl4OuP9/LeZPZbtiX6BknPOVZi0W+zOOecKzBO7c85VGE/szjlXYTyxO+dchfHE\n", + "7pxzFcYTu3POVRhP7M45V2E8sTvnXIX5f4oqt2/zLhAlAAAAAElFTkSuQmCC\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "## Read test data here\n", + "a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n", + "xt= a[:,0]\n", + "yt= a[:,1]\n", + "X = construct_poly(x,8)\n", + "w = wRidge(X,y,1.0)\n", + "@interact(n=[1,16])\n", + "def plot(n):\n", + " X = construct_poly(x,n) \n", + " Xt = construct_poly(xt,n) #n=d\n", + " w = wRidge(X,y,1.0)\n", + " plt.plot(x,X.dot(w))\n", + " plt.plot(xt,Xt.dot(w))\n", + " plt.title(\"MSE train: %f test %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n", + " plt.plot(xt,yt)\n", + " plt.plot(x,y)\n", + "# was ist mit fehler kurven gemeint?" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!
" + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1.75123654094\n", + "0.781528050024\n", + "11.0428184273\n" + ] + }, + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VNXWx/HvCoSODWmiAtarSBPFipwICNgQFUVF7AXx\n", + "qtjbfYe5VsDeexfLtV0pClzlIDYQBUERxQKCIkhTiggk6/1jnySTMEkmmZrJ+jxPniQzZ87Zw6O/\n", + "7Nln77VFVTHGGJM9ctLdAGOMMYllwW6MMVnGgt0YY7KMBbsxxmQZC3ZjjMkyFuzGGJNlLNiNMSbL\n", + "WLCbpBKRBSLyt4g0KfX4TBEpEJGdg993FJHXReR3EVktInNE5IzguTbBsWtKfQ0o45onicjHIrJO\n", + "RCZHef4xEZknIvmF1yin/XVF5CkR+UNElojIsDKOGxy08ZyIx84QkRnBaxeJyAgRqRXx/I4iMkZE\n", + "VgTnvr/weRHJFZHXROSn4LzdS12vdnD8kuD1b4vIDuW9F1NzWLCbZFPgR+CUwgdEpD1QP3iu0PPA\n", + "QmBnYDvgdGBpqXNtraqNI77+U8Y1VwB3AbeX8fws4CLgi1JtiGY4sGvQrjzgahHpHXmAiGwLXA98\n", + "Vep89YFLgSbAAUAP4MqI5+8DlgMtgU5A96BdhT4ABgG/RWnnRUA3oAOwA7AKuL+C92JqCAt2kwov\n", + "AIMjfj8DeA6QiMf2A55R1b9UtUBVZ6nqu1W5mKq+p6qvAUvKeP4hVX0f2BDD6QYDN6nqH6o6D3gM\n", + "OLPUMbcB9+L+oERe5xFV/UhVN6vqr8CLwCERh7QDXlHVjaq6FHg3eAxV3aSq96nqR0B+lHa1Ayao\n", + "6u+q+jfwauFrjbFgN6nwKbCViPwjGGo4GRf2pY95SEROLhyeiULKeDwpgp54S+DLiIdnExGgItIV\n", + "2Bd4JIZTdsf16gtNAE4Vkfoi0groC7wTY/MmAn1FpKWINABOA8bH+FqT5SzYTao8j+v99gLmAr+U\n", + "en4AMBX4F/BjMAa/X6ljlovIqoivPZPc5kbB9z8iHvsTaAwQ/JF6ELhYKyi6JCJn4/4A3BHx8HBg\n", + "n+Cci4DPVPW/sTRMVV8HZuL+Hf8A9gRuiuW1JvtZsJtUUFywn0b0YRhUdbWqXqeq+wDNcePgb5U6\n", + "TxNV3Tbi69skt3tt8H2riMe2BtYEP18EzFbV6RHPb/GpQkSOA24F+qrqyuAxwfXY/wM0ALYHthOR\n", + "EbE0TETuwP2B2Q5oCLxJ7L19k+Us2E1KqOrPuJuofYE3Kjh2BXAnsEMwHFLly8bxWlR1FW6cvlPE\n", + "wx0pHk45HOgfzExZAhwM3Cki9xUeLCJ9cOPyR6vq1xHn2R7oAjwQjKevBJ4BjoyxeX2Ap4M/iBuB\n", + "B4CuIrJdZd+nyT4W7CaVzgEOV9W/Sj8RTAVsF0zjawwMAeYH4Vp0WCwXEZEcEakH5AI5wZTF3Ijn\n", + "c4Pnc4A6IlIv6EFH8xxwo4hsIyJ7AefiAhjcTdR/4MK+EzADN7xyQ3Cdw3E3TI9X1Rmlzrsc90dj\n", + "iIjUEpFtcJ9misbzg3bXC36N/BncWP8ZIrJV8N4uAn4p/ERgajYLdpMyqvqjqn4R+VDEz/Vxwwmr\n", + "gB+AnYBjS51idal57JeVcanBwHrgIdyUwL+ARyOenxQ8fyCuN70+OA4ROU1EIm9whoL2LAQmAyNU\n", + "dWLwfv5Q1WXB11JgI/CnqhYO1dyIGy55J6LN44LXKnA8cAwu5OcDfwOR8+S/Ddq2A27YZl3EjeVh\n", + "QEHQtmW4Hnz/Mv49TA0j8Wy0ISI74Xo0zXD/kz6mqvcFHwdfAVoDC4CTVHV1/M01xhhTkXiDvQXQ\n", + "QlVniUgj4HPgOOAsYLmqjhSRa4BtVfXahLTYGGNMueIailHV31R1VvDzWuAboBXuI/SzwWHP4sLe\n", + "GGNMCiRsjF1E2gCdgWlA82DMEdyy8OaJuo4xxpjyJSTYg2GY14FLI24cAUU3iWzHbGOMSZHa8Z4g\n", + "mGr1OvC8qhYuKFkqIi1U9TcRaYm7a1/6dRb2xhhTBapa7tTfuII9mPv7JDBXVe+JeOpt3JzcEcH3\n", + "0isIY2pcJhOR4ao6PN3tqCprf3pZ+9OnOrcdYusUx9tjPwRXVnS2iMwMHrsOVy711aA29QLgpDiv\n", + "Y4wxJkZxBbuqfkjZ4/Q94zm3McaYqrGVp1Xnp7sBcfLT3YA4+eluQJz8dDcgTn66GxAHP90NSLa4\n", + "FijFdWERrc5j7MYYkw6xZKf12I0xJstYsBtjTJaxYDfGmCxjwW6MMVnGgt0YY7KMBbsxxmQZC3Zj\n", + "jMkyFuzGGJNlLNiNMSbLWLAbY0yWsWA3xpgsY8FujDFZxoLdGGOyjAW7McZkGQt2Y4zJMhbsxhiT\n", + "QOL726W9DbbRhjHGxE98f3vgDmAfYH/1vKSEq220YYwxSSa+nyO+fzbwNbAKyEtWqMcqrs2sjTGm\n", + "JhPfbwc8DNQH+qrnfZHmJgEW7MYYU2ni+w2AG4HzgOHAI+p5+WltVAQLdmOMqQTx/T7Ag8BnQEf1\n", + "vF/T3KQtWLAbY0wMxPd3AO4G9gOGque9m+YmlclunhpjTDnE92uJ718MfAnMB/bJ5FAH67EbY0yZ\n", + "xPf3BR4F1gPd1fPmprlJMbFgN8aYUsT3GwM3AacA1wLPpHsKY2VYsBtjTEB8X4D+wL3AJKCdet7y\n", + "9Laq8uIOdhF5CjgKWKaq7YPHhgPnAr8Hh12nqhk9JmWMqdnE99sA9wO7AoPU86akt0VVl4ibp08D\n", + "fUo9psBdqto5+LJQN8ZkJPH9XPH9q4EZwKdAp+oc6pCAHruqThWRNlGesjowxpiMJr5/MO7m6C/A\n", + "Aep5P6S5SQmRzDH2f4rIYNxfwStUdXUSr2WMMTELKjDejhtGvhx4tTrdHK1IsuaxPwy0BToBS4A7\n", + "k3QdY0w2EdkakX8ismtSTu/7Ir5/Oq5g10Zgb/W8V7Ip1CFJPXZVXVb4s4g8AYyJdlxwk7WQr6p+\n", + "MtpjjKkGRHoATwFzgBAiE4HbUJ2TkNP7/h64Tud2QD/1vOmJOG+yiYgHeJV6TSLqsQdj7GMiZsW0\n", + "VNUlwc/DgP1V9dRSr7F67MYYEGkIjAD6AeeiOgGRrYAhwGW44dxbUf2kSqf3/Xq4uegXAzcDD6jn\n", + "bU5I29MgluyMO9hF5CWgO7A9sBQI4f66dMLNjvkJuEBVl1a2ccaYLCdyMPAMbjbKpaiuEiFHlYLg\n", + "+frAWcBVwELgVmASMQaX+H4PXC99DnCpet7ihL+HFEtJsFeVBbsxNZhIXSAMnAEMFfQtoCdwIe6G\n", + "5gTgIWCiKgWI5AIDcT3vv4DbgDdRLYh6et9vjru3dyhwsXre2CS/o5SxHZSMMZlHpDNueGXPEVx9\n", + "uKC74YprjcQFemvgbVzvfL4IVwu6NarPA+1xS/2vBuYicmYQ+u7Ubjej83E99F9xK0ezJtRjZT12\n", + "Y0xqiNQGrlW4ZBxHPXYsb++i5BwJvAk8AkxXRYsPR4CuuLH243CTMB4CPlW3TOZw4DpgD2DUns89\n", + "N+27nXa6J3j5hep5s1P11lLJhmKMMZlB5B+bqP3iQlo37MWknAW0zceF+XOqrJKwCG4T6L7AwcAy\n", + "XC/efY3+7yq+O/YU3FDNOlzAj1Zl7ZxddjnsjpNPfmL8AQfsdpLv//fa0aPP3mnZslVpeZ8pYMFu\n", + "jEmr/8iAnJ35+Y69+GbIDdyiDzL0v0rOI8AHDJdGuHH1vsHXZmA88AFuMsZuwO7BV2tgGcp8lv/j\n", + "L747ujWLD2zD/tt9ziDZg1zee+a22x4/Y+LEc3Fj9I8B91Bq0kY2sGA3xqSFCI16MfGfN3Pj9UDu\n", + "NYy418/d/05uaNwUOBIX5PvjZsOMB94BvtVQ9ECSsNQGdqYw6Bvu0pldLjmGus2b8O3dOfz6898U\n", + "1J5Po98+7rZo87IHxrNvu985pEAYnVvASFQXpOJ9p4IFuzEmpUToIBRceCGPnHEb19WelrvP6OOu\n", + "mDbmr3r5R+ACvQAX4uOByRrStZU6v+/Xxs1HvxF4ALid8GRh2T7nkrP5HBr/ugu7TPqW1h+ubbVh\n", + "XetLprHzeZ8jH+7M0rsOwvfb8hnFQzw/aUg3JvL9p4IFuzEm6USoDwwALtyRhW3ernP4xu3rLKxz\n", + "4sD8hdN3ZB9gGsVhPq+sXnmF1/H9rrhx+VXAEPW876K0ZS/cOPwg4CPqrXq039Etf7ti+t+X7LuE\n", + "/l814/cbDmfpe7vSHNgRWAx8T+R4vvv9Jw3ppqq0M9ks2I0xSSPCnsAF1FkzWPb6z8KhjW+s9X9f\n", + "LGn/ZGfWhD1e3ZDLOOB9DemauK7j+1sDtwAn4BYqvVhRbRcRGuJ2P7oI2Bp49GRefullTjkuOMe3\n", + "q+oxqsWVLNhYu2gcP3JMvxWwiJJhX/jzAg1p2lauWrAbYxJKhDpI/nG0mHU5u0xqR6fnlm9f/5um\n", + "o99gXcff0Df34p8XHsNrVe2Vl7iW283oJOAuYCxwrXpepWa7lJoy2Q8Y24Ilj/9Cq11z0GuA1bjF\n", + "TmMiFztJWOoCbSgO+sjgb0nJ0I8M/oXJDn0LdmNMQsgu7+9Ni1nDaTr3aHYfL9T9Yx21/37zgXfy\n", + "lw35jLNz4EXgRlQ3JOR6vr8L8CBuuOQC9byP4z6n0AQ4k2DKZF02PLKQ1muas+xyoC6ujO/LaPnB\n", + "HIT+LpTs4RcGfwvgIw1pj3jbW/b7sGA3xlSBhEVYt/1e/NplGPVXHkfTuU34c6dfEH2Z7b99dtVt\n", + "LNrmb+4BDgPORHVqla/l+3WB5riecAtgP1wPeyRwt3peQse6RcgBeuCGaboLBaPfpP+X/Xj7NNzM\n", + "m1HA01X5IyVhqQc005D+nMg2l7iGBbsxJlYSlgZAHuu3OxHRfmyutxULD13Bmh1fpt6qW/Stp105\n", + "7uLyuuOBq9AtZ7aI7+cATXBBHfnVMspjjXAFBH/D7d+wELhTPW9BMt+veyvsBJyH26N5/lWMfO82\n", + "rutai4IuwN3AI6j+mex2VIYFuzGmXBKW3YEjUfqitQ5j2T7r+HpAQxYfMIaFh92m+XVmFR8sDVc0\n", + "bnzn8q23Pva53r1H3Tpo0M+UHdbNgDW4oP4tylfk46vU86IW80oVEXJxZQuGAHsfxdgxLzCo6Tb8\n", + "cQhuJs69qC5PZxsLWbAbY0oIeuUehas9C3Ia8svBi/h2SBvWHbqaDgXv0n/xPLbevA0RQV1/w4Y2\n", + "Oao7bqxduyC/Vq3FBTk5v1J2UP8GLFPP+zsNbzFukVMmOzJr5mhO/XsvvjlI4DngTlQXpbd9FuzG\n", + "1EjBjJKtgBas+GQ/1i/qScHGA8ipvSv1dlxFrV02srnlNuTWacRWm6BAVlFbFyElg7rR+vXL773/\n", + "/iMP+OabXgKXt3vmmQqnGmaLYMrkQGDoTvy83WOcv+AIJnbMQd8ARqC6xTz61LTLgt2YrCK+X4ct\n", + "hz1Kjl+rtgRtiW6Cv1cKG1cquvkXChp+z9z2+XzeqgMr6uZTp+BlOq9+kqOW/BB1RyGRTsDzuKl8\n", + "FxCx5WVNEjllsgnLj7uV6xecyTM757Lpf+K27puZ2vZYsBuT8YIbjdsR243GxrjKhyWHPtYt3MyK\n", + "j3bkj9ntWL+oPZtWzyZ//Tjya4/nlvV1KMi9EDgetwL0YWBqZInckg1y5XWBS4ArgBdi3bEo2xVO\n", + "mWzMn0Mu4576VzGqQT02fJbL5pvimRlUuTZYsBuTNuL7Dag4qFvgpvqtoeKbjL8BK9XzCoJpdd1x\n", + "Y+VH4gL/XVxwT2K4bgJOxY0VbwM8CjytSvm9bpF/4MaSVwNno1rtt5JLhsIpkw1Yd/Fgnus5nOGb\n", + "GrD+x8as/RcwPpl/CC3YjUkw8f1aQFPKD+rCx3OpOKh/A5bGcqNRwrILxSVuDwNm44L8HWCWhrRA\n", + "hA64MB8ITMHN6JhUtIdomSeXHFwP/UbgX7hpftZLj4EIO+Wy8fyTeWXo//HvOg1Yv/BxzttnuA5P\n", + "yr+fBbsxVSC+3xE3cyRaT7sJsJLyg7rw8T/judEYrHA8jOIyt9tSHOSTNKQroWQRLtwCm8eBJ1WJ\n", + "rbct0hZ4GqiNW2z0fVXbXJOJkJtD/nEH8mnvj/SQc5N3HQt2YypFfL8LbkjjFdyemaUD/PeoNxoT\n", + "df2wtKF4eKU78BXF9cpnaiiinklhES4YDHyG652PUyW29okIbmHOrbhVnnehmp+gt2KSJJbsrJ2q\n", + "xhiT6cT3d8Ptq3m+et6bKbmm65V3ozjMm+D+sLwInKkhXVHieKEObiHNhUA73ArQrqr8WLkLyw7A\n", + "E7jxfQ/Vr+N7JyaTWI/dGEB8vznwEXCHet4jSb1WWFpTPFbuAXMprlf+RWSvvOg1Qlvc0vezg+Mf\n", + "Ad5SpXIbRbhe+kDgHtzsmFvQzKw7bqKzHrsxMRDfb4wL1ReTEeoSljrAoRSPlTcFJuCGe87RUPSl\n", + "6iLUDl5zIW4e9fOAp8q8qjVEtseFeTvgKFRnVOk8JuNZsJsaLVjw8zrwOTA8YecNy84U98rzgG9w\n", + "vfKzgBnReuVFrxVaAefgeuiLcb3zE1T5q+oNkmOD84wGTk9UeV2TmWwoxtRYwcKg53BzwE+I56Zo\n", + "0Cs/hOJeeQtcr3w8MFFD+nu5r3fzonvieud5wMvAo6rMKu91FTdMtobElNc1mcGGYowp30igLdCr\n", + "KqEuYdmR4pueecB3uCA/B9crr3CGiQhNcb34C3CLlB4GzlAlru3kgpNHltftGK28rslOFuymRhLf\n", + "vwIXyIeq562P6TVhycX1yguHWHYAJuKGcs6vqFdedB5Xe6Qbrnd+JPAmbpXo9DKX+VeGSENgBG4r\n", + "uHNRnRD3OU21YsFuahzx/VOBy4BD1PNWlntsWFpRHOQ9cPtavoPrYU+PpVdedC5hW+B0XKCDG/Me\n", + "qkql9vGs4CIHAc8C04AOqCbu3KbaiHuMXUSeAo4Clqlq++Cx7XB3/FsDC4CTVHV1qdfZGLtJOfH9\n", + "XsALQA/1vK+iHhOWvXGLfvri9tyciAvzCRrSpZW6nuud748L89iKcFWFSF3czd+zgItQfSNh5zYZ\n", + "JSUrT0WkG7AWeC4i2EcCy1V1pIhcA2yrqtdWtnHGJFLEqtLj1fOi3kSUsHQD3gAeA8bheuWVH38X\n", + "GgOnUNkiXFVh5XVrlJSVFBCRNsCYiGCfB3RX1aUi0gLwVfUflW2cMYkivr8rMBUYWtaqUglLHvAq\n", + "cIqG9H9Vuo7QETdMU7kiXFW7mJXXrYHSOSumuWrRR9aluGXLxqRFsKp0AvDvckL9CNwy/gEaUr9S\n", + "549ehKtDzEW4qqJked0u6d6uzWSWpN88VVUVkai9CBEZHvGrr1q5/6GMqUiwqnQc5awqlbAcCTwD\n", + "9NeQfhjzubcswjWCyhThqgorr1vjiIiHKz0Rs2QF+1IRaaGqv4lIS4g+rqiqw5N0fWMKV5W+Bsyk\n", + "jFWlEpZjccWwjtWQflrhOUsW4dobN098f1V+SlCzy7t4ZHndA628bs0QdHj9wt9FJFTRa3KS1Ja3\n", + "gTOCn88A3krSdYyJKlhV+hSwARgSrS66hOUE3LDJkRWFughtRbgV+BkX6o8AO6tyfdJDXUQQOQ+Y\n", + "jvv00d1C3ZQn7h67iLyEqxu9vYgsAv4PuB14VUTOIZjuGO91jKmkEZSzqlTCcjJwL9BHQ9E3Iw6K\n", + "cB2FC/L9ibcIV1VYeV1TBVYrxmQd8f3LcRtIHBptAZKEZRAwCuitIZ29xfNCC+B8ShbhejWuIlyV\n", + "ZeV1TRmsVoypccT3TwGGUcaqUgnLWcDNQA8N6dwtnnd7hr4DjAWOibsIV1VYeV0Tp2SNsRuTcsGq\n", + "0nuAvup5P2/xfFjOB/4NHF5GqB8CTAKGqXJBmkL9GNwm1Qtx0xgt1E2lWY/dZAXx/X1xtcZPiFYq\n", + "QMIyFLgayNPQljceRTgSV2NlkCqpL5pVXF63OzAQ1Q9S3gaTNazHbqq9YFXpWOAC9bwtAlHCMgy4\n", + "EvDKCPVTcdMIj0lTqPfA9dI34Ap3WaibuFiP3VRr4vvNcPVf/q2et0XhKwnL1bgbod01pFsOzwhD\n", + "geuAHqpELQqWNK687u24efFWXtckjAW7qbbE9xvhNpF4KdqqUgnLDbhVod01pL+UeM5VXfwXroxu\n", + "t5QsMCrZACuva5LGgt1USxF7lc4ESqzEk7AIbqXpANzwy5ISz7tt6Aq3iztUlUqV4o2Lldc1KWDB\n", + "bqqdYFXpk8DflFpVGoT6LcAxuBulJUJbhFzcePrOuMVGJfYJSG7Di8rr/oDbqi51f1BMjWLBbqqj\n", + "24FdgZ6Rq0qDUB+F2+koT0O6PPJFIjTAleVVoHfKFhyVLK97JfC8Fe4yyWTBbqoV8f1hwNFAt8i9\n", + "SoNQvwe3J2kPDWmJxUkibAOMwZW4OFuV1KzitPK6Jg1suqOpNoJVpZcDfdTzVhQ9HpYc4EHgAKBn\n", + "lFBvgdv04nPgjJSEukgOIpcBH+KGfnpbqJtUsR67qRbE93viinb1iFxVGoT6Y8BewBEa0j9LvE5o\n", + "i1tN+ixwc0L3GS2zsVZe16SX9dhNxhPf74xbVXqiet6cosfDUgsXoLvjCnqVDvX2uO3w7lLlpqSH\n", + "upXXNRnCeuwmo4nv74JbVXph5KpSCUttXC+8Oa6e+roSrxMOwu0DcJkqLyW/oVZe12QO67GbjBWs\n", + "Kp0A3By5qlTCkovrwTcBjokS6r2B/+LG05Mb6q6XfgpuPv103NCLhbpJK+uxm4wUrCodB7ysnvdw\n", + "0eNhqQO8AuQCx2lIN5R4nXAycB9wnCofJ7eRsj3wELAPVl7XZBDrsZuME7FX6Ze4Hbnc42Gpi1tt\n", + "CnB8lFAfAtwJ9EpBqBeW1/0ZK69rMoz12E1GiVhVugk3rq4AEpb6wBvAGuA0DRXvJhTUfbkBt0z/\n", + "MFV+TF4DrbyuyXzWYzeZ5jbcqtKTC1eVSlga4BYXrQJOLRXqOcBduH11D01yqBeW1/0bK69rMpj1\n", + "2E3GEN+/DDgWt1fpegAJSyPcrJifgbM0pPlFx7u6L0/g/hB0VyU5FRJLltc9D9V3k3IdYxLEeuwm\n", + "I4jvDwSuAHoXriqVsGyFq7X+PVuGen3ceHtT4IgkhvpBuBkv2+J66RbqJuNZsJu0E9/vgVtVemTh\n", + "qlIJyza4qY5zgPNLhfrWuMBfA/RTZf2WZ423UVIXkduAN4HrUB1kNdNNdWHBbtIqWFX6EnBS4apS\n", + "Cct2uDIA04GLNKQFRccLzQEfN9Z9elLqvrjyup/hyhR0RPX1Cl5hTEaxYDdpE7GqdIh63hQACcv2\n", + "wHu4ol2Xaai4vK0IbXAlAt4CLlGlYIuTxtUgyUHkStwflTuA/lYz3VRHdvPUpEWpVaWvA0hYmgH/\n", + "wy1Mur5UqLfDDb+MVOX+xDdIWuFKFNQDuqKa2q3yjEkg67GblAtWlY4lYlWphKUlbojlTbYM9QNw\n", + "vfhrkxTqxwNf4D4leBbqprqzHrtJKfH9XNyq0tkEq0olLK2A94HnNaQ3lzhe6IWrC3OmKuMS2xhp\n", + "hFtslAf0Q/XThJ7fmDSxHrtJGfF9odSqUgnLzrie8pNRQn0A8ALQPwmhvj+ul14L6GShbrJJUnvs\n", + "IrIA+BPIBzapatdkXs9kvNtxtdN7qOdtlrC0ASYD92lI7448UITzgRBujvqXCWuBSC3gamAYcDGq\n", + "rybs3MZkiGQPxSjgqZbcqszUPKVXlUpYdsUNv4zUkD5YdJyr+3ItcB5uNWniNqoQ2Rl4Hvffpe0/\n", + "arJWKoZiJAXXMBksYlVpH/W8FRKWPXE3Sm+JEuqjgFNxdV8SGeonAzOA8UAPC3WTzVLRY/+fiOQD\n", + "j6rq40m+nskwwarS+3DDLwslLHvj5onfqCF9uug4oTbwOLAnrqeemE95IlsB9wMHAn1R/Twh5zUm\n", + "gyU72A9R1SUi0hSYJCLzVHVq4ZMiMjziWF9V/SS3x6RQxKrSAep5cyQs7XFz16/WkL5QdJxQD3gZ\n", + "N4e8lyrrop6w0g2Qg4AXcX9I9kU1Mec1JoVExAO8Sr1GNfmbtgOISAhYq6p3Br+rqtowTZYS328L\n", + "fAhcop73uoSlE26B0WUa0peLjhO2wm1jtxQYrMrG+C8utXH12YcAF6L6VtznNCZDxJKdSRtjF5EG\n", + "ItI4+LkhcASuoJPJcuL7TXE981uCUN8v+H1oqVBvhpsV8w1wWoJCfRfgA+AQXC/dQt3UOMm8edoc\n", + "mCois4BpwFhVnZjE65kMIL6/I25e+kvqeQ9JWA7E3bA8X0PFxbREaI2r+zIOGKpKftQTxnxhEURO\n", + "x/239h+gD6q/xnVOY6qplA3FbHFhG4rJOuL7e+J65g+q542SsByCKxFwpoZ0fNFxwl7BcXeqcm/8\n", + "F5ZtgIeBDsCpqCZu3rsxGSatQzGmZhHf3w83hTEchHp3XKgPKhXqXXHDLzckKNQPw216vRzYz0Ld\n", + "GKsVYxJAfP9w3KyW89Tz/ith6YWbjTJQQ/p+0XFCD9wsmXNUGRPfRSUXGA6cDZyLamJLDhhTjVmw\n", + "m7iI75+AGwYZoJ43RcJyGm5z6eM1pB8WHSccDzwCnKhKfJtAi+yO+8OxHFfnxWqmGxPBxthNlYnv\n", + "nweEgaOYkjcLuBL4J9BXQ/p10XHCucC/gaNUmVn1C4rgeui3B9d9kHT9B2xMmsSSndZjN5UWVGks\n", + "qufClLwfgLuBHsDBGtLFRccKV+Pmk3dXZX7VLypNgMeA3XA107+u4BXG1Fh289RUivh+DnAnQT0X\n", + "puT9jBs37wx0Kwx1EUSEkcAZuLov8YR6D2AWsBA4wELdmPJZj93ELNgk40lgV+AwpuQpbjXpcqC3\n", + "hnQDFNV9eQTYBzhMlRVVu6DUBW4CTgPOwtZBGBMT67GbmIjvNwDeALYHejElrwFuhecc3OyXwlBv\n", + "A0wEdgZ6xhHqewGfAHsAHS3UjYmdBbupkPj+NrgFRX8A/ZiS1xr4CDcz5VINaX4w9HIu8FlwbF9V\n", + "1lb+YiKIXIj7o/EI0B/V5Ql6K8bUCDYUY8olvt8SN9wyGbicKXkHA68DV2pInwcQoRWu5G5zIE+V\n", + "r6p2MWmKG+ppBXRDdV7878CYmsd67KZM4vu74io0vgoMY0peP9xq0tM1pM8HvfTTgZnAp8CBcYR6\n", + "H9wN0rkqC4eqAAAUiUlEQVTAQRbqxlSd9dhNVOL7HXHFu25Sz3tEwjIE+BfQR0P6uQjNgUdxN1L7\n", + "qPJF1S4k9XDz0o8HBqE6OSFvwJgazILdbEF8vxtuuGUoU/JekyncDJwEHKoh/VGEAbhdiZ4CTlbl\n", + "76pdSNoDo4F5uBWktjeuMQlgK09NCeL7x+DGuU9lSt4UXK98H+AohmsB8CDQCThDlWlVu4gIboXq\n", + "v4CrgGdtBakxsbHqjqZSxPfPwN0EPZopeZ/gdjZqDuQxXA/CTW38FegcR6i3wA3xnIYbS3/GQt2Y\n", + "xLKhGAOA+P7lwKVAHlPyVuBmwXzFU1Ou5ufDHgS6AQPjKuAlcgyuLMDjwE2oboq/5caY0izYa7ig\n", + "7sutwHG4EgF1cHPUXya86UO09hfAGKBjlealA4g0AO4A+gID0OKqj8aYxLOhmBpMfL8WrgfdA+jG\n", + "lLxmwFT+2vYBhmsztPZjwNmqDI0j1DsDnwNb426QWqgbk2QW7DWU+H493Pz0NkAPpuR1Ad5l9qkP\n", + "MWLlpbhPcx1U+V/VLiA5iFyJKy9wM6qnofpHYlpvjCmPDcXUQOL7WwFv4Yp3Hc2UvJNRRvH66Ml8\n", + "dcoQ4HxVqrYjkSuv+w9cvfR6QFdUf0pQ040xMbBgr2HE95sB7wDTWT3rYr4cdhWb6l7KE9P+YmnH\n", + "TUB7VcqfTy6yFbB78LVHqe85wHxcwbBRqG5O3rsxxkRj89hrEPH91rihkVeYdVmYlXPvZ13TE3ni\n", + "E+XPnS9S5fXig6U+blVpYWhHBnhj4HvgO1yIz4/4+XebvmhM8sSSnRbsNYT4fjtcMa9RTMl7jLXN\n", + "xuYub3NQh5du/uSZv4c9tQ9fN6NkeDcHfqJkaBd+/xXVgrS8EWNqOAt2AyI5x9xyy7ET99vvqQvG\n", + "jv3vvx99aOOM5nVOb7Mqp17btX9tzqFgsWwZ3N8BP9swijGZx/Y8rSncEv0WRBnzfqdr190/bteu\n", + "9tMjR36170dzmt663za9F2zea0W/X1v03Y3XPkW1anVejDEZy3rs1YnIdmx5s7Lw+wZK9bx73Hln\n", + "28mdO1+hf9UewLBVJ3D0RRfzy/6vsbTjQJ083MbBjamGMn8oBrZFdXVaGpCpRBoRPbj3wH3Cijbm\n", + "PR/VVSVO4/tDget4s9UQ3pk/guMH7cqfra7Sh+bcl7o3Y4xJtOoQ7GuAL4BxwFhgXo2YUeFqkEfO\n", + "OIn8vg1lzzhZVtG/T1AiIIRyGsM6vUL+O/+k3zk51Pq7v4bzq7bYyBiTMdIa7OJ2xLkHqAU8oaoj\n", + "tmgcNATygKOBo4DNuIAfB0xB3QbJ1ZJIbdyqzmg97xbAAqL3vn+p6owT8f0c4D425uQxqOsa9hq5\n", + "Pd1vakStzX01pDPjfEfGmAyQtmAXkVrAt0BP4BfcBsenqOo3ZTbO3QDch+KQb4+rMDgOGIfqrwlv\n", + "aLxEcoAdid7zbg0sIfqMk4WJnnEivl8H5VmW1u3CeV2a0K/fTHZ7dyeEPhqylZ/GZIt0BvtBQEhV\n", + "+wS/XwugqrfH3Di3NL0PLuR7Awsp7s1/lrJ51O4PTuk53oXfdwVWEb3n/WOqPnGI7zfkr5xxfLvV\n", + "3tywxwKGtvmFhr+3BI7WkC5PRRuMMamRzumOrYBFEb8vBg6o1BlUVwAvAi8GwxoH4UL+SaAZIuNx\n", + "IT8xIcWlRLalOLRLB/hGSgb3K8H371GtWtXDBJF3PmjC+jrT+XzbHXi00QiuanQworWBwzWk69PZ\n", + "NmNMeiQr2GP6GCAiwyN+9VXVj3423QxMDb6uRaQNLuTPAp5EZAbFN2C/K/MGo5txshvRe991KNnj\n", + "HkfxjJOM3ItThnzfhe65PtObrGPy9D5ccvqdwExgiIZscZEx2UBEPMCr1GuSNBRzIDA8YijmOqAg\n", + "8gZqwuaxizQEDscF/dG4+dzjgE9xnxwi65xsC/xA9KGTpdVlRo4IgrfsCob8MIIvtnmPTT0vZatf\n", + "xwLPA2ENVY/3YYypvHSOsdfG3TztgdsjczoV3TxN0IWBDriQ74Ibl48M78XVvcaJCM3osvJlbvym\n", + "Gwsa3MIf+47H7U36fxrSx9PdPmNMcqVtjF1VN4vIxcAE3HTHJyNDPWncX6kvg6+sI8KJ7LfyMcJf\n", + "10U5hT/2XYf7dHK2hnRMuttnjMkMVlKgGhChCXA/PZd255p59amtxzMlry1wG9BfQ/pJmptojEmR\n", + "WLLTtsbLcCIcDczm/B+acv03OeRs6sGUvEOB/wO6W6gbY0qz6o4ZSoStgXtAu/PgF2PZe00vNq3x\n", + "+PjYS4GDgYM1pEvS3ExjTAayHnsGEuEIYA45uoF3po5l7zUH88fsnnx87G24GT6HWagbY8piwZ5B\n", + "RGgkwsPAEzTadAHvTWlAvYIuLHj2WGZd+hxuKueRGtI/09xUY0wGs2DPECJ0B2YD9Tjit66M+egi\n", + "oClfXn4WC58pnJc/SEO6Ma0NNcZkPBtjTzMRGgC3ACcBFzLZnwqMARbyyYl3snHFe8BdGtK709lO\n", + "Y0z1YcGeRiIcCDwLfA50YLKfC0wBfD444i100wTgEg3py+lspzGmerFgTwMR6gJh4EzgYlVeE9/f\n", + "FZgIPM2UHt9BwSvAyRrSyWlsqjGmGrJgTzERuuB66d8BHVVZKr7fERgP3MSUvLrAXUAvDWlWrqA1\n", + "xiSXBXuKiJAL3ABcBAwDRqui4vvdgNfQ/Ev4oGcX4BjgEA3pwjQ21xhTjVmwp4AI7XG99N+Azqr8\n", + "AiC+fwzwFJvXDuajY04D2gKHakhXpK+1xpjqzqY7JpEItUW4DngfeBA4KiLUBwOPs27hAD465nKg\n", + "EdDTQt0YEy/rsSeJCHvieunrgP1UKRpaEd+/HLiM36cMYO7we3FljS+2zTGMMYlgwZ5gIuQAl+LG\n", + "00PAw6oUAIjvC27Oen9+fPx0Fo1+FngKuMU2xzDGJIoFewKJsCvwNCDAgap8X/Sc79cCHgY6Mfua\n", + "S1k1/RXgBg3pk+lprTEmW9kYewKIICIMAaYBbwFeqVCvB7wKtGX64FGsmv4CcI6FujEmGazHHicR\n", + "dgaeBLYGuqlSYqco8f3GuLBfyUfHvM7mtfcBR2tIp6e+tcaYmsB67FUU9NLPxpUDmAwcHCXUmwGT\n", + "UZ3PB72+YvPaq3CbY1ioG2OSxnrsVSBCS+BxoBXQQ5XZWxzj+62BiWj+f/igV1PQY3ELj35LcXON\n", + "MTWM9dgrIeilnwrMAr4ADigj1NsBH7J5/eN80LM9aFvAs1A3xqSC9dhjJEIz4CFgL9xCoxlRj/P9\n", + "A4H/smFpiGkDBwM/AAOsjroxJlWsxx4DEY4HvsSFdJdyQr038DZ/zLmGaQMvAz4AzrBQN8akkvXY\n", + "yyHCdsD9wP7ACap8XOaxvj8QuJcl46/iu1G3ACM0pPenqKnGGFPEeuxlEOEo3FZ1y4FOFYT6RcAd\n", + "/PDw9Xw3ahQwzELdGJMu1mMvRYStcfXQDwcGqeKXeawrERACTmPO9SNZ+cmtuPH0KSlprDHGRGE9\n", + "9ggi9MT10jcDHSoI9RzcME0/ZpzzAis/uQpXndFC3RiTVqJpqj0lIqqqkpaLlyJCI2AkbpOL81R5\n", + "t9zjfb8O8CyqO/DpgNlsXHE40EdDuigFzTXG1GCxZGdae+wivCTCiUGwpqsN3XAzXhoC7WMI9YbA\n", + "22h+Iz46eikbV3QCulmoG2MyRVKCXUSGi8hiEZkZfPUp49DJwHnAryK8JcJgEbZNRpu2bCP1RbgL\n", + "eAUYpsoZqqwu9zW+vx3wP/L//p2pvRuRv74Wbm/SlaloszHGxCJZPXYF7lLVzsFX1F6wKo+p0hto\n", + "DbwO9AcWiDBBhAtEaJGMxolwADAT2AE3lv52ha/x/VbAVDau/oIP+3RA8+cCJ2lINySjjcYYU1XJ\n", + "HIqJefxclVWqPK9Kf1z9lceB7sA8EaaKcJkIreNukFBXhFuBt4H/U2WgKssrfJ3v7wF8xPqfx/FJ\n", + "/yOBl3E7HuXH2yZjjEm0pNw8FZEQcBbwBzADuEJVV5c6psIbACLUBXoCxwPHAgtxPfs3VPm2cm2i\n", + "M26ruh+BC1RZGtPrfL8LMJZVM55i9lXnANdoSJ+tzLWNMSZRYsrOqga7iEyCqEMlNwCfAr8Hv98E\n", + "tFTVcyrbuJLHUxvoBpyAG7JZDbyBC/ovVYn6RkTIBa4DLgauAF4o69gtXuv7ecAr/DrmcebfdT5w\n", + "uoaiDysZY0wqJDXYK9GINsAYVW1f6nEFwhEP+arqx3ZOcoCuuJ78CbhhnzeCr0+L9hgV2uF66cuB\n", + "c1VZHHO7ff944BF+ePh5Fr96CtBPQ/pZrK83xphEEBEP8CIeCqUl2EWkpaouCX4eBuyvqqeWOkaZ\n", + "PPl44C31vCo3QgQBOlAc8tsBbwIrgCHA9cATsfbSAcT3z0X138wNjWX51MI56t9X+EJjjEmytPXY\n", + "ReQ5oBNudsxPwAWqurTUMcrkyfNwY97/VM/7MTHXZk/cUM2eQFiVBTG/1pUIuAbVC5g59DPWfNMW\n", + "OEpDuiwRbTPGmHhlxFBMmRd2wV4XGAZcDdwLjFTPS8v0waBEwCi0oA+fDV7CX79swtV9WZuO9hhj\n", + "TDQZH+yFjQu2kbsH2Ae4WD1vQkrb4vu5wOMUbNqLT0+qxabVXwHnaUg3pbIdxhhTkWoT7EWP+f5R\n", + "uMJanwPD1PNivtlZ5Xb4fn3gVTavb8SnJ+5I/l+vAjdqKE3/MMYYU45qF+xQFLTXARcBtwH3qecl\n", + "pecsvr8N8DZ/r1jLtIEd0c23akgfTMa1jDEmEaplsBc97/u7Aw/glv1fpJ43NaHX9/0WwLus++kn\n", + "ZpxzCOiFGtI3EnkNY4xJtGod7FA0S+VE3MYX7wFXq+fFPUNFfH8XYBIrP5vJnKu7ASdqSBP6h8MY\n", + "Y5Kh2gd70bG+3xi3U9Hg4Ptj6nlVqtMivt8B1XdYMnYa8+/qAhypIf26KucyxphUy/hgZziXABOA\n", + "+bHcrBTfbw88DNQFhqjnzajUNX3/UFRf56cnPmfR6J2AvhrSpN+gNcaYRKkOwf4U0BvYiAv4d4H3\n", + "NaRrynydm28+GLgdV0LgBvW8VRVez/ePRvUp5t36Dcv+B65EQLn1140xJtNkfLCrqkhYBGiHC/g+\n", + "wIHAF7iQnwDM0pAWbPF6t+nFLcBxwDXA82WVJhDfH4wWjGL2VYtZ/cWPuGJeVkfdGFPtVItg3+Lx\n", + "sDTA1WLvgwv7bYGJuKCfVHp5v/j+/rjhmXXAUPW8r0o9PwzNv5LPL1jHuh/eBYZZHXVjTHVVLYN9\n", + "i+PC0gYX8L2Bw4EfKO7Nf6Ih3SS+Xwu4AFct8png+zrgFgo2DuSzM+uwYcl9wChbeGSMqc6yIthL\n", + "vCYsubihmsLe/G64fVMnABPoPnk9MBLIA2awee1eTB/chE2rrtCQPp/Yd2CMMamXdcG+xTnC0gzo\n", + "hQv5I3A7Nk2g7XmL2bbL4Xx5+X7krz9VQzoxAU02xpi0y/pgL3G+sOQAHSm+CdsKOFlD+kWirmGM\n", + "MelWo4LdGGNqgliyMydVjTHGGJMaFuzGGJNlLNiNMSbLWLAbY0yWsWA3xpgsY8FujDFZxoLdGGOy\n", + "jAW7McZkGQt2Y4zJMhbsxhiTZSzYjTEmy1iwG2NMlrFgN8aYLFPlYBeRASLytYjki8i+pZ67TkTm\n", + "i8g8ETki/mYaY4yJVTw99jlAf+CDyAdFZG/gZGBvXF30h0Qk6z4ZiIiX7jbEw9qfXtb+9KnObY9V\n", + "lQNXVeep6ndRnuoHvKSqm1R1AfA90LWq18lgXrobECcv3Q2Ik5fuBsTJS3cD4uSluwFx8NLdgGRL\n", + "Rk96B2BxxO+LcbsZGWOMSYHa5T0pIpOAFlGeul5Vx1TiOunZpskYY2qguLfGE5HJwBWqbm9REbkW\n", + "QFVvD35/Fwip6rRSr7OwN8aYKqhoa7xye+yVEHmRt4HRInIXbghmd2B6ZRtmjDGmauKZ7thfRBYB\n", + "BwLjROQdAFWdC7wKzAXeAS7SdO2YbYwxNVDcQzHGGGMyS1rnl4vIKBH5RkS+FJE3RGTrdLansspb\n", + "pJXJRKRPsHhsvohck+72VIaIPCUiS0VkTrrbUlkispOITA7+m/lKRC5Jd5sqQ0Tqicg0EZklInNF\n", + "5LZ0t6kqRKSWiMwUkcpMAMkIIrJARGYH7d9iiLtQuhcOTQTaqWpH4DvgujS3p7KiLtLKZCJSC3gA\n", + "t3hsb+AUEdkrva2qlKdxba+ONgHDVLUdbghzaHX6t1fVDUCeqnYCOgB5InJomptVFZfihoqr43CF\n", + "Ap6qdlbVMtcHpTXYVXWSqhYEv04DdkxneyqrnEVamawr8L2qLlDVTcDLuEVl1YKqTgVWpbsdVaGq\n", + "v6nqrODntcA3uHUf1Yaqrg9+rAPUAlamsTmVJiI7AkcCT1By0kd1UmG7091jj3Q2MD7djagBWgGL\n", + "In63BWRpICJtgM64Dk21ISI5IjILWApMDiZLVCd3A1cBBRUdmKEU+J+IzBCR88o6KFHTHcsUyyIn\n", + "EbkB2Kiqo5PdnspK4CKtTFEdP35mFRFpBLwGXBr03KuN4BN2p+B+2AQR8VTVT3OzYiIiRwPLVHVm\n", + "Na4Xc4iqLhGRpsAkEZkXfIotIenBrqq9ynteRM7EfTTqkey2VEVF7a+GfgF2ivh9J0qWgDBJJCK5\n", + "wOvAC6r6VrrbU1Wq+oeIjAP2A/w0NydWBwPHisiRQD1gKxF5TlUHp7ldMVPVJcH330XkTdzQ6hbB\n", + "nu5ZMX1wH4v6BTdmqrPqMl43A9hdRNqISB1cJc6309ymGkFEBHgSmKuq96S7PZUlItuLyDbBz/WB\n", + "XsDM9LYqdqp6varupKptgYHA+9Up1EWkgYg0Dn5uCByBm8CxhXSPsd8PNMJ9pJgpIg+luT2VUtYi\n", + "rUymqpuBi4EJuJkBr6jqN+ltVexE5CXgY2APEVkkImelu02VcAgwCDebZGbwVZ1m+LQE3g/G2KcB\n", + "Y1T1vTS3KR7VbViyOTA14t9/rKpOjHagLVAyxpgsk+4euzHGmASzYDfGmCxjwW6MMVnGgt0YY7KM\n", + "BbsxxmQZC3ZjjMkyFuzGGJNlLNiNMSbL/D9Nh9MMaaoMVgAAAABJRU5ErkJggg==\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "\n", + "def plot(n):\n", + " X = construct_poly(x,6) \n", + " Xt = construct_poly(xt,6) #n=d\n", + " w = wRidge(X,y,1.0)\n", + " plt.plot(xt,Xt.dot(w))\n", + " plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n", + " print(MSE(X.dot(w),y))\n", + " \n", + " w = wRidge(X,y,0.1)\n", + " plt.plot(xt,Xt.dot(w))\n", + " plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n", + " print(MSE(X.dot(w),y))\n", + " \n", + " w = wRidge(X,y,10.0)\n", + " plt.plot(xt,Xt.dot(w))\n", + " plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n", + " print(MSE(X.dot(w),y))\n", + " \n", + " plt.plot(xt,yt)\n", + " #plt.plot(x,y)\n", + "plot(1)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Hand in printed copy of completed notebook." + ] + } + ], + "metadata": { + "annotations": { + "author": "", + "categories": [ + "intelligent-systems-1-2015" + ], + "date": "2015-04-30", + "location": "Beginning of next lecture", + "parent": "IS_SS2015", + "submission_date": "2015-05-07", + "subtitle": "Exercise Sheet 3, Linear Regression", + "tags": [ + "IntelligenSystems", + "Course" + ], + "title": "Intelligent Systems 1 - Summer Semester 2015" + }, + "celltoolbar": "Edit Metadata", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}