{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"preamble": true
},
"source": [
"(Defining latex commands: not to be shown...)\n",
"$$\n",
"\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
"\\DeclareMathOperator{\\minimize}{minimize}\n",
"\\newcommand{\\real}{\\mathbb{R}}\n",
"\\newcommand{\\normal}{\\mathcal{N}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#Exercise Sheet 3\n",
"Maximus Mutschler & Jan-Peter Hohloch"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gaussian Algebra (25 Points)\n",
"\n",
"Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
"\n",
"\\begin{equation}\n",
" \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
"\\end{equation}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Proof\n",
"\\begin{align*}\n",
" f(x) &= \\frac{1}{\\sqrt{2\\pi\\sigma^2}}\\cdot e^{\\frac{-(x-\\mu)^2}{2\\sigma^2}}\\cdot \\frac{1}{\\sqrt{2\\pi s^2}}\\cdot e^{\\frac{-(x-m)^2}{2s^2}}\\\\\n",
" &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\alpha}\\\\\n",
" \\text{where }\\alpha &= \\frac{s^2(x-\\mu)^2+\\sigma^2(x-m)^2}{2\\sigma^2 s^2}\\\\\n",
" &=\\frac{(s^2+\\sigma^2)x^2-2(s^2\\mu+\\sigma^2 m)x+s^2\\mu^2+\\sigma^2m^2}{2\\sigma^2s^2}\\\\\n",
" &=\\frac{x^2-2\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}x+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
" &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2+\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
" &=\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}+\\underbrace{\\frac{\\frac{s^2\\mu^2+\\sigma^2m^2}{s^2+\\sigma^2}-\\left(\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}_\\beta\\\\\n",
" \\beta&= \\frac{\\frac{s^4\\mu^2+s^2\\sigma^2\\mu^2+s^2\\sigma^2m^2+\\sigma^4m^2-\\left(s^4\\mu^2+2s^2\\sigma^2\\mu m+\\sigma^4m^2\\right)}{\\left(s^2+\\sigma^2\\right)^2}}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}\\\\\n",
" &=\\frac{s^2\\sigma^2\\left(\\mu^2-2\\mu m+m^2\\right)}{2\\sigma^2s^2\\left(s^2+\\sigma^2\\right)}\\\\\n",
" &=\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}\\\\\n",
" \\Rightarrow f(x)&=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-(\\alpha-\\beta)-\\beta}\\\\\n",
" &=\\frac{1}{2\\pi s\\sigma}\\cdot e^{-\\frac{\\left(x-\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2}\\right)^2}{\\frac{2\\sigma^2s^2}{s^2+\\sigma^2}}}\\cdot e^{-\\frac{(\\mu-m)^2}{2(s^2+\\sigma^2)}}\\\\\n",
" &=\\frac{\\sqrt{2\\pi(s^2+\\sigma^2)}\\sqrt{2\\pi\\frac{\\sigma^2s^2}{s^2+\\sigma^2}}}{2\\pi s\\sigma}\\normal\\left[x;\\frac{s^2\\mu+\\sigma^2 m}{s^2+\\sigma^2},\\frac{\\sigma^2s^2}{s^2+\\sigma^2}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
" &=\\frac{\\sqrt{2\\pi\\cdot 2\\pi\\sigma^2s^2}}{2\\pi s\\sigma}\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\\\\n",
" &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[\\mu;m,\\sigma^2+s^2\\right]\\hspace{2cm}\\\\\n",
" &=\\normal\\left[x; \\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\left(\\frac \\mu{\\sigma^2}+\\frac m{s^2}\\right),\\left(\\frac 1{\\sigma^2}+\\frac 1{s^2}\\right)^{-1}\\right]\\normal\\left[m,\\mu,\\sigma^2+s^2\\right]\\hspace{2cm} q.e.d.\n",
"\\end{align*}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
"\n",
"Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
"\n",
"\n",
"Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Linear regression (50 Points)\n",
"\n",
"In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
"\n",
"1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
"2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Loading the required packages\n",
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from IPython.html.widgets import interact\n",
"import csv"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [],
"source": [
"def wRidge(X,y,lamb):\n",
" # Change the following line and implement the ridge regression estimator wRidge\n",
" return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Read ex1_train.csv and assign the first column and \n",
"# second column to variables x and y respectively.\n",
"\n",
"a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
"x= a[:,0]\n",
"y= a[:,1]\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4) Plot the training data with appropriate labels on each axes!"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.text.Text at 0x7f7e89472588>"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAG1tJREFUeJzt3X285nOdx/HX2wxRhFQMxtJETDGjZZasHCsaYmdnVXIT\n",
"IjfRNCuVu12mKCpJW9RaN7EMKy2LSiaczCqDjDGMCWW2QSYKyza2YT77x/dH5zrnzJzrOue6ru/v\n",
"d13v5+NxPeZc51w378PM73N97xURmJmZvWqV3AHMzKxcXBjMzKyGC4OZmdVwYTAzsxouDGZmVsOF\n",
"wczMamQtDJIulrRE0vw+35sh6XFJc4vb5JwZzcy6Te4WwyVA/wt/AOdExLbF7aYMuczMulbWwhAR\n",
"s4FnB/mR2p3FzMyS3C2GFZkmaZ6kiyStkzuMmVk3KWNh+DawGTAR+C3wtbxxzMy6y+jcAfqLiN+9\n",
"+rWkC4Eb+j9Gkjd4MjMbhogYsqu+dIVB0piI+G1xdyowf7DH1fPLlZWkGRExI3eO4XL+vJw/nypn\n",
"h/o/VGctDJKuBHYB3ixpMXAa0CNpIml20mPAURkjmpl1nayFISL2H+TbF7c9iJmZvaaMg8/doDd3\n",
"gBHqzR1ghHpzBxih3twBRqg3d4AR6M0doB1UxYN6JEWVxxjMzHKo99rpFoOZmdVwYTAzsxqlm65q\n",
"ZmbNI7Ee8I7iVhcXBjOzipNYDRjHnwtA39uqwC+LW32v58FnM7PykxCwPoNf/McCi/lzAeh7WxJB\n",
"pNeo79rpwmBmViISawCbM/DivwXwMoNf/H8VwZ+Gfm0XBjPrIMUn5rWBMcCGxZ/9b2/iz5+cFzLI\n",
"p+YyKH6Xjai98G9Z/LkB8GsGKQAR/H5k7+vCYGYVUFwk12Pwi33/7y0j7brc//Zk8eezwF+w8n72\n",
"vrdHIljawt9tTdIn/cE+/b/YJ0ffIrYogpdbk8eFwcxKRGIU8AngndRe7NcH/peBF/kBtwheHOZ7\n",
"952Z0/f2tuK1BysaT9TTyih+r01W8PpvAh4d7PUjeH44v8tIuDCYWWlIrAVcCawJXEPtxf+pCF7K\n",
"lGs06fyXwS7qawIP069YkFokW/Z53Djg9wxeXH4TwfL2/UYr58JgZqUgsQnpXJU7gU9GsCxzpLpI\n",
"rMPAbqCNgEUM7I4aVkum3VwYzCw7ie2B60gnMX69TAPA3ajea6cXuJlZS0h8EDgf+HgE1+fOY/Vz\n",
"YTCzpipmGZ1IGmh+fwRzM0eyBrkwmFnTFFszXAC8C9ghgiczR7Jh8O6qZtYUxZTQWaRFaLu4KFSX\n",
"C4OZjZjEO0izju4E9o3gfzNHshFwYTCzEZHYFbgdOCuCE8o0b9+Gx4XBzIZN4nDgKuAjEVyUO481\n",
"hwefzaxhEqsAZwFTgZ0jeDhzJGsiFwYza4jEG4DLSfsA7TDSHT+tfNyVZGZ1k9iINJ7wHLC7i0Jn\n",
"cmEws7pIvJs06+h7wGH1HAxj1eSuJDMbksQU4ELg6Ai+nzuPtZYLg5mtULG9xaeB44C9Irg7cyRr\n",
"AxcGMxuUxKrAecAkYMcIFmeOZG3iwmBmA0isSxpLWEqajvpC5kjWRh58NrMaEuOAnwPzgb9zUeg+\n",
"Lgxm9hqJnYE7gHMjOC6CV3JnsvZzV5KZASBxMHA2cFAEN+fOY/m4MJh1uWJ7iy8ABwA9ESzIHMky\n",
"c2Ew62ISawCXkg65/6sIns4cyUrAYwxmXUpiA6AXWAbs5qJgr3JhMOtCEtuQtrf4AWlM4aXMkaxE\n",
"shYGSRdLWiJpfp/vvUnSLEkPS7pZ0jo5M5p1Gom9gJ8AJ0bwhQgidyYrl9wthkuAyf2+dyIwKyK2\n",
"AG4p7ptZE0hMI+15NCWCq3LnsXJSRN4PC5I2BW6IiK2L+wuBXSJiiaQNgN6I2LLfcyIi1PawZhUl\n",
"MRo4F+gB9o5gUdZAlkW9184yzkpaPyKWFF8vAdbPGcas6iTWJh2/KWCnCJ7PHMlKroyF4TUREZIG\n",
"bdJImtHnbm9E9LYllFmFSGwK3Aj8FJgewct5E1k7SeohtRIbe15Ju5J6IuIpSWOA29yVZNY4iR2B\n",
"75POZv6mB5mt3mtn7sHnwVwPHFJ8fQhwXcYsZpUk8RHgP4EjIvhnFwVrRNYWg6QrgV2AN5PGE04l\n",
"/WW+GtgEWAR8OCKe6/c8txjMBlEcrPNPwOHAPhHcnzmSlUi9187sXUnD4cJgNpDE6qSpqJuTpqM+\n",
"lTmSlUyVu5LMrEESbyGt+1mNtBGei4INmwuDWcVJjAfmALcBH4lgaeZIVnGlnq5qZisnsTtwBfCZ\n",
"CC7Lncc6g1sMZhUlcTTwb8C+LgrWTG4xmFWMxCjSSWt7An8dwaOZI1mHcWEwqxCJtYCZwOuBHSN4\n",
"NnMk60DuSjKriOIIzhtIa34muyhYq7jFYFYdhwKrA0dGsDxzFutgXuBmVgES6wEPAntGMDd3Hqsm\n",
"r3w26yASFwIvRvAPubNYdVX5PAYz60NiJ9JJh+NzZ7Hu4MFnsxKTWBX4DvDpCP4ndx7rDi4MZuU2\n",
"HXgS+F7uINY9PMZgVlISY4G5wA5exGbN4N1VzarvG6ST11wUrK08+GxWQhIfALYGDsidxbqPC4NZ\n",
"yUi8HvgWaSHbS7nzWPdxV5JZ+ZwCzIlgVu4g1p3cYjArEYmtgCOBCbmzWPdyi8GsJCQEnA+cHsGT\n",
"ufNY93JhMCuPA4G1ScXBLBuvYzArAYl1gQXAlAjuyp3HOpM30TOrEInzAUXwidxZrHN5Ez2zipCY\n",
"BEzFm+RZSXiMwSwjidGkTfI+6xPZrCxcGMzyOgZ4DrgidxCzV3mMwSwTiQ2BecDOESzMncc6nzfR\n",
"Myu/c4ALXBSsbDz4bJaBxO7AJOCw3FnM+nOLwazNJFYnLWKbFsEfc+cx68+Fwaz9TgDmR/CD3EHM\n",
"BuPBZ7M2kng7cCewbQSLc+ex7uLBZ7OSKTbJOw84y0XBysyFwax9PgRsSDqy06y0PCvJrA0k3kia\n",
"nrpfBMty5zFbmdKOMUhaBPwP8AqwLCIm9fmZxxisUiTOBdaK4PDcWax7dcImegH0RMQfcgcxGwmJ\n",
"bYH9gXfmzmJWj7KPMbhVYJUmsQrwbeDkCJ7JncesHmUuDAH8RNI9ko7IHcZsmI4gdYdekjuIWb3K\n",
"3JW0U0T8VtJbgFmSFkbE7NyhzOolsT5wBrBbBMtz5zGrV2kLQ0T8tvjzaUnXkvaVea0wSJrR5+G9\n",
"EdHb1oBmQ/sqcGkE9+cOYt1JUg/Q0/DzyjgrSdLrgVER8YKkNwA3A5+PiJuLnwfEThH8LGtQsxWQ\n",
"6AEuA8ZH8GLmOGZA9Vc+rw/MlnQfMAe48dWi0Mdn2h/LbGgSq5E2yZvuomBVVMoWw1CKFsMzwHsi\n",
"eCR3HrO+JE4CdgL2iaB6/8CsY9XbYqhyYTgDWC+CY3LnMXuVxGbA3cD2ETyWO49ZX91QGDYAFgJb\n",
"RPB07kxmxSZ51wM/j+BLufOY9Vf1MYYhRbAEuAb4RO4sZoUpwNuBs3MHMRuJyrYYIkISWwG9wKYR\n",
"LM0cy7qYxJrAg8ChEdyWO4/ZYDq+xQAQwUPAXcBHc2exrncqcLuLgnWCSrcY0tfsAlwAbOXVpZaD\n",
"xLuAW4Gtiy5Os1LqihZD4XbS9tx75w5i3afPJnmnuShYp6h8YSjmiZ+NF7xZHocAryO1Ws06QuW7\n",
"ktJ9RgOPkk7HmpMvmXUTifVIA857RXBv7jxmQ+mmriQieBn4OnB87izWVc4CrnZRsE7TES2G9D3W\n",
"Ah4DJkXw6zzJrFtI7ARcTdok7/nceczq0VUtBoAIXgD+FfiH3Fmss0msShpwPt5FwTpRx7QY0vfZ\n",
"EHgAeHsEPivaWkLieGAysIc3ybMq6fi9klb0y0lcAjzivWqsFSTGAnNJO/s+nDuPWSO6uTC8C5gF\n",
"bBbBS+1NZp1O4vvAAxGcljuLWaO6bozhVRE8QPpEd2DuLNZZJPYCJgBn5s5i1kpDFgZJn5K0bjvC\n",
"NNHZwPHFqlSzEZN4PfAt4Fi3RK3T1XPhXB+4W9LVkiZLGrIZUgK3AS8Be+YOYh3jZOCeCH6cO4hZ\n",
"q9U1xiBpFWAP4FBgO9L87Ysi4lctTbfiPEP2k0kcABwRwa5timUdSmJLYDYwMYIncucxG66mjjFE\n",
"xHLgKWAJ8AqwLnCNpK+OKGVrfQ8YJ7Fd7iBWXcWpbOcDZ7goWLcYssUgaTpwMPB74ELg2ohYVrQi\n",
"HomIca2POSBTfVOuxKdJZ+/u34ZY1oEkDiRt0Lh9sfWKWWU1bbqqpM8DF0fEfw/ys/ERsWD4MYen\n",
"gcLwRtI2GX8ZwaKWB7OOIrEOsACY6s0ZrRN07TqGgY/lK8CqERzX4ljWYSTOA0ZFcHTuLGbN4MLw\n",
"2mPZGLgfGBfBs61NZp1CYnvgetImef57Yx2haxe49RfB48CNwJG5s1g1SIwCvgOc4KJg3ajjWwzp\n",
"8UwAfkjaJuNPrUtmnUBiGrAvsKs3ybNO4hZDHxHMI5205dlJtlISY4BTgWNcFKxbdUVhKJwNfKaY\n",
"l262IucAF0bQ9tl2ZmXRTYVhFhCkFdxmA0jsDuwInJ47i1lOXVMYim6Bs0mLlcxqSKwOnAdMi+CP\n",
"ufOY5dQ1haFwFbCVxMTcQax0PgcsiOCG3EHMcuuKWUm1z+VzwNYRfLTJsayiJN4O3Am8O4Lf5M5j\n",
"1ipe4LbC57IO8GtgQgSLm5vMqqaYjPAj4NYIvpI7j1krebrqCkTwHPBdYHrmKFYOHwQ2Br6eO4hZ\n",
"WXRdiyE9n01Ix3++LYLnm5fMqqTYZHEBsH8Es3PnMWs1txhWouhHvgk4IncWy+rzwCwXBbNapSwM\n",
"xRGiCyU9IumEFr3N14DpEqu16PWtxIqZaQeSZiOZWR+lKwySRpEOXZ8MjAf2l7RVs98ngnuBh4EP\n",
"N/u1rdwkVgG+DZwSwdO585iVTekKAzAJeDQiFkXEMtLagyktei9vk9GdPk5aBX9R7iBmZVTGwrAR\n",
"1Ewjfbz4XivcBKwK7Nai17eSkXgrcAZwdATLc+cxK6MyFoa2TZPyNhld6Rzg3yK4P3cQs7IanTvA\n",
"IJ4Axva5P5bUaqghaUafu70R0TvM95sJfFFiG18sOpvE3qRN8rbJncWsHST1AD0NP69s6xgkjQZ+\n",
"SereeRK4C9g/Ih7q85gRrWMY+J6cBLwjgkOb9ZpWLhJrAw8Ah0Rwa+48ZjlUeksMSXsC5wKjgIsi\n",
"4sx+P292YXgT8ChpD6UnmvW6Vh4SFwARwVG5s5jlUunCMJRmF4b0mnwDWBrBic18XctP4n3AxaTC\n",
"75Xu1rVcGBp+TTYD7iadC/1CM1/b8pFYE5gPHBvBD3PnMcvJW2I0KILHgFuAw3Nnsab6EnC7i4JZ\n",
"/dxiqHldtgeuAcZF8HKzX9/aS2Jn4N+Bd0Xwh9x5zHJzi2EYIrgbWETaitkqTGIN0srmY10UzBrj\n",
"wjDQ2cCpEhvnDmIjMgO4L4JrcwcxqxoXhoFuBK4AfiG55VBFRZfgocC0zFHMKsljDCt8DyaRCsRs\n",
"YLpnKlWDxOuAXwBfimBm7jxmZeIxhhGK4C5gW2A5MFdih8yRrD4nk870vjJ3ELOqcouhrvfj70n7\n",
"959H+iTqGUslJDEB+Akw0SvYzQZyi6GJIvgP4N3AzsBPJd6WOZL1IzGatLr5BBcFs5FxYahTcbF5\n",
"P2mdwxyJg33AT6l8Bvg9cEnuIGZV566kYb0/E0gD0w+SDnx5NlcWA4mtSJMEtotgUeY4ZqXlrqQW\n",
"imAesD2wBJgnNb7fuTWHlHbgBU5zUTBrDheGYYpgaQSfAo4ErpA4S2K13Lm60DTgZdLkADNrAncl\n",
"NYHEW0ifWjcCDoxgYeZIXUFiHDAH2DGCR3LnMSs7dyW1UQRPA1OAC4DZEkd7YLq1JFYBLgTOdFEw\n",
"ay63GJpMYkvSwPQTwMcj+F3mSB1J4ijgMOA9EbySO49ZFbjFkEnRjbQjacbSfRJ7Zo7UcSQ2Ac4A\n",
"DnNRMGs+txhaqJitdClwPfC5CJbmTVR9RRfdD4E7Ijgjdx6zKnGLoQQi6AUmAm8B7inWP9jIHAxs\n",
"AHw5dxCzTuUWQxsUn3IPAs4BzgTOjWB53lTVIzEGmAe8P4K5ufOYVU29104XhjaS2Ay4HPgjcKj3\n",
"9KlfUVyvBR6I4B9z5zGrIncllVAEjwG7ALcD9xa7tlp9PgxsAZyeO4hZp3OLIZPifIfLgZ+SDgJ6\n",
"MXOk0ioWEN4P/F0Ec3LnMasqtxhKLoI7SQcBQToIaFLOPCX3DWCmi4JZe4zOHaCbFceFHi6xL3CD\n",
"xDdJK3k9N78gMQWYBGyTO4tZt3BXUklIbAx8F1gd+GgxHtHVJNYFHgAOiOCnufOYVZ27kiomgseB\n",
"PUgzb+6SOMj7LfE14DoXBbP2couhhCQmAjNJc/Y/EcFzmSO1ncT7gX8Bti663MxshNxiqLAI7gP+\n",
"EniGdBDQLpkjtZXEWqSicKSLgln7ucVQchJ7kbaXvpR0StmfMkdqOYnzgDUiOCx3FrNO4pXPHUTi\n",
"raSDgMaQDgL6ZeZILSPxPtIg/NY+S9usudyV1EGKMx3+llQc/kviyE4cmJZ4L2ls5WAXBbN83GKo\n",
"GImtSAcB/QY4ojg9rvKKLcqvBvaP4JbMccw6klsMHSqCh4AdgF+SDgKanDnSiEn8Dako7OeiYJaf\n",
"WwwVJrEraVD6WuCECF7KHKlhxZjCTOBDXq9g1lqVbTFImiHpcUlzi1vlPxG3SgS3ARNIg9J3S2yd\n",
"OVJDJPYgFYV9XRTMyqN0hQEI4JyI2La43ZQ7UJkVg7T7AWcDt0ocJ5Xy/2uNogvscmBqBLNz5zGz\n",
"PyvrBaTru4kaEUFEcCnwV8CHgJskNswca4UkPgBcBkyJ4I7cecysVlkLwzRJ8yRdJGmd3GGqIoJf\n",
"A+8F7iAdBDQ1c6QBJPYBLgH2ieDnufOY2UBZBp8lzSId6N7fKcCd8NoUzNOBMRFxeL/nB/D5Pt/q\n",
"jYjeFkStLIkdSV01twLHleEgoGIL7QuAvSO4O3ces04nqQfo6fOt0yq/8lnSpsANEbF1v+97VlId\n",
"ij2H/hnYibRiOtvFuDjG9HxgrwjuzZXDrJtVeVbSmD53pwLzc2WpugheiOBjpJbYjRKnSIxqdw6J\n",
"D5KKwp4uCmblV7oWg6TLgImk2UmPAUdFxJJ+j3GLoUHFQUCXkU7t+2gE/92m990POBeYHMG8dryn\n",
"mQ3Om+jZAMU01uOBz5LGHa5o8fsdQDpsZ48It/zMcnNhsBWS2Ja039Jc4NhWHAQkcRDwFWD3CB5s\n",
"9uubWeMqO8ZgrRfBXGA74DnSfks7N/P1JQ4Bvgy8z0XBrHrcYuhyEnsD/wpcDMyIYNkIX+8w4Auk\n",
"orCwCRHNrEncYrC6RHAjabB/InCHxBbDfS2Jj5PWl/yNi4JZdbkwGBEsAfYmnZx2h8QRjR4EJHEU\n",
"cCqwawQPNz+lmbWLu5KshsR40sD0ItJBQM/U8ZxjgM8Bu0Xwq9YmNLPhcleSDUsEC0gHAT1KGpje\n",
"Y2WPl/gkafrrri4KZp3BLQZboeJktUuBa4CT+h8EJDEdmE4aU1jU/oRm1gi3GGzEIriVdBDQxsBd\n",
"fQ8Ckvg08Cmgx0XBrLO4xWBDKgaiDwG+CpwBvA44gtRSWJwzm5nVzyufrekkxpG28l6PNKbwROZI\n",
"ZtYAFwZriWJ31tUiWJo7i5k1pt5r5+h2hLHOEcEr4KJg1sk8+GxmZjVcGMzMrIYLg5mZ1XBhMDOz\n",
"Gi4MZmZWw4XBzMxquDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxq\n",
"uDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxqZCkMkj4k6UFJr0h6\n",
"d7+fnSTpEUkLJe2RI5+ZWTfL1WKYD0wFbu/7TUnjgf2A8cBk4HxJHdeqkdSTO8NIOH9ezp9PlbM3\n",
"IstFNyIWRsTDg/xoCnBlRCyLiEXAo8CktoZrj57cAUaoJ3eAEerJHWCEenIHGKGe3AFGoCd3gHYo\n",
"26fxDYHH+9x/HNgoUxYzs640ulUvLGkWsMEgPzo5Im5o4KWiSZHMzKwOish33ZV0G3B8RNxb3D8R\n",
"ICLOKu7fBJwWEXP6Pc/FwsxsGCJCQz2mZS2GBvQNeT0wU9I5pC6kzYG7+j+hnl/MzMyGJ9d01amS\n",
"FgM7AD+Q9COAiFgAXA0sAH4EHBM5mzRmZl0oa1eSmZmVT9lmJdVN0umS5km6T9ItksbmztQISV+V\n",
"9FDxO/yHpLVzZ2rEyhYplpWkycXCyUcknZA7T6MkXSxpiaT5ubM0StJYSbcVf2cekPSp3JkaIWl1\n",
"SXOK680CSWfmzjQckkZJmitppROAKlsYgK9ExISImAhcB5yWO1CDbgbeGRETgIeBkzLnadSgixTL\n",
"StIo4FukhZPjgf0lbZU3VcMuIeWvomXAcRHxTlIX8rFV+u8fES8BuxbXm22AXSX9deZYwzGd1FW/\n",
"0q6iyhaGiHihz901gWdyZRmOiJgVEcuLu3OAjXPmadRKFimW1STg0YhYFBHLgKtICyorIyJmA8/m\n",
"zjEcEfFURNxXfP0i8BBp3VJlRMQfiy9XA0YBf8gYp2GSNgb2Ai6kdtLPAJUtDACSvijpN8AhwFm5\n",
"84zAYcAPc4focBsBi/vc9+LJTCRtCmxL+kBUGZJWkXQfsAS4rZgsUyVfBz4LLB/qgaUuDJJmSZo/\n",
"yG0fgIg4JSI2Ab5L+qVLZaj8xWNOAf4UETMzRh1UPfkrxLMsSkDSmsA1wPSi5VAZEbG86EraGHhv\n",
"lfZNkrQ38LuImMsQrQUoxzqGFYqI3et86ExK+Il7qPySDiU17XZrS6AGNfDfvwqeAPpOUBhL7fYr\n",
"1mKSVgW+D1weEdflzjNcEfG8pB8A2wG9mePU6z3A30raC1gdeKOkyyLi4MEeXOoWw8pI2rzP3SnA\n",
"3FxZhkPSZFKzbkoxsFVlVVhweA+wuaRNJa1G2sX3+syZuoYkARcBCyLi3Nx5GiXpzZLWKb5eA9id\n",
"Cl1zIuLkiBgbEZsBHwFuXVFRgAoXBuDMolvjPtKOh8dnztOob5IGzWcV08fOzx2oEStapFhWEfEy\n",
"8Engx6RZGf8eEQ/lTdUYSVcCPwO2kLRY0sdyZ2rATsBBpNk8c4tblWZYjQFuLa43c4AbIuKWzJlG\n",
"YqVdq17gZmZmNarcYjAzsxZwYTAzsxouDGZmVsOFwczMargwmJlZDRcGMzOr4cJgZmY1XBjMzKyG\n",
"C4NZE0javjh06XWS3lAcRjM+dy6z4fDKZ7MmkXQ6aYOyNYDFEfHlzJHMhsWFwaxJit1D7wGWAjuG\n",
"/3FZRbkryax53gy8gbQ54hqZs5gNm1sMZk0i6XrS2SBvA8ZExLTMkcyGpdQH9ZhVhaSDgf+LiKsk\n",
"rQL8TFJPRPRmjmbWMLcYzMyshscYzMyshguDmZnVcGEwM7MaLgxmZlbDhcHMzGq4MJiZWQ0XBjMz\n",
"q+HCYGZmNf4fBBSJ6m7pJCYAAAAASUVORK5CYII=\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x7f7e8951d7b8>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot the input data here\n",
"plt.plot(x,y)\n",
"plt.xlabel('x')\n",
"plt.ylabel('y') "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
"Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def construct_poly(x,d):\n",
" ## Implement a method which given an array of size N, \n",
" ## returns an array of dimension (N,d)\n",
" poly=np.empty((len(x),1), float)\n",
" xr = np.reshape(x,(len(x),1))\n",
"\n",
" for i in range(d-1):\n",
" poly = np.append(poly, (xr**i), axis=1)\n",
" #print poly \n",
" #print np.shape(poly)\n",
" return poly"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def MSE(y_predict,y_true):\n",
" ## Implement mean squared error for a given input y and its predictions.\n",
" return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecXFX9//HXOwm9hA6BUJQmvfwQQUAjTXpEepEO0hEL\n",
"Vbm5FAUFRCmC9I4g0gVEJIKoIF8CRELHQGihhF4T8vn9cW7IZrNtdmfmzsy+n4/HPLI7t312H/De\n",
"M+eee44iAjMzay0Dyi7AzMyqz+FuZtaCHO5mZi3I4W5m1oIc7mZmLcjhbmbWghzuZmYtyOFudSVp\n",
"rKRPJc3b7v1RkiZLWqz4fqik6yW9IekdSaMl7V5sW6LY9/12r+06ueb2kv4p6UNJ93RT3zHtzvmR\n",
"pM8lzVNsn1PSFUVdbxRfz9Hm+IGSTpT0sqT3JD0saXCxbSZJvy62TZB0tqRBbY4dKukWSW9JelXS\n",
"mZIGdlDjccXPv37Pf/PW3zjcrd4CeB7YacobklYCZim2TXE58AKwGDAP8D1gfLtzDY6IOdq8ruvk\n",
"mm8BpwMnd1tcxM/bnhM4BbgnIiYUu4wA5gO+BCwJLFi8N0UOrAWsFRFzArsCnxTbjgJWB1YAlim+\n",
"/mmbY38LvAkMAVYFvgkc2LY+SUsC2wKvdPezWP/mcLcyXAHs1ub73YHLALV5bw3gkoj4OCImR8Qj\n",
"EXFHby4WEXdHxB+BVys5TpKK2i5t8/YKwI0R8UFEvAfcWLyHpLmBw4B9I2Jcce0xEfFpcewWwJkR\n",
"8U5EvEkK873anfsPEfFZRIwH7phy7jbOAo4EJlbys1j/43C3MvwbmFPSV4puhx1Igd9+n3Mk7TCl\n",
"q6YD6uT9alkPmB+4vs17dwLbSJqrCPNtgD8X21YCJgHbFd0qT0mapuXdruYBwNA23Tp3AjtLmkXS\n",
"IsCmwO1fHJi6nT6JiNsx64bD3cpyOan1vhEwBni53fbtgPuAnwHPF33ya7Tb501Jb7d5LVvlGncH\n",
"rouIj9q8d3bx71ukLpSJwO+K94YCg4GlgSVI3ScjJG1YbL8DOEzSfJIWAg4ldUXNWmwfAawIvAeM\n",
"A/4TETcBFH8ATiJ9MjDrlsPdyhCkcN+FjrtkKLoujo6IFUn92o+QukDamjci5m7zeqpaBUqalRTO\n",
"l7bbdCXwFDA7MCfp/sGUTx0fF/8eHxGfRsRo4Bpgs+L9k4BRxc/yD+AGYFJEjC+6gO4EriOF/XzA\n",
"PJJOKY4dAVweES+2LbMKP6q1KIe7laIIqedJXQ9/6mbft4DTgIWLrpBeX7aCfbcG3oqIv7d7fxPg\n",
"vOJewIfAeUwN78e6um5EfBIRh0TE0IhYCpgAPFTsMx/w/4CzImJicQP3kjbnXh84tOjueRVYFLhW\n",
"0k8q+JmsH3G4W5n2BtaPiI/bb5B0iqQVJA0quiQOAJ6JiLfb7taTi0gaIGlmYAZgQDEkcYZuDpvy\n",
"iaK9x4B9Jc0saRZgP+BRgIh4jtSVdKykGSUtR7qfcGtRx8LFS5LWIo2UyYrzvkm64XtAMZxyrqKG\n",
"R4vtG5Burq5CGknzSnHtc3ryO7D+x+FupYmI5yPi4bZvtfl6FlK3xdvAc6SW6lbtTvFOuzHpP+jk\n",
"UrsBH5GCcD1S98l5UzYWx67T5vtFgGF0HO57kIYxvgy8ROpb373N9p2AxUl98rcCP42IKWPrlwTu\n",
"Bz4ALgaOjIi/Fr+LAL4LbEkK+meAT4HDi+0TIuL14jUe+Bx4u/j0YDYd9XWxDkkXAZsDr0fESsV7\n",
"I4B9gDeK3Y7u7TA2MzOrXDVa7heT+iHbCuD0iFiteDnYzczqqM/hHhH3kT46t+c7+WZmJalln/sh\n",
"kh6VdGFxc8jMzOqkVuH+O9LcG6uSRgCcVqPrmJlZBwZ1v0vlIuL1KV9LugC4pf0+kvp2J9fMrJ+K\n",
"iG67vWsS7pKGRMSUSZq2BkZ3tF9PCmxUkkZExIiy6+gt118u11+eZq4det4w7nO4S7qaNDXpfJLG\n",
"kR7KGCZpVdKomf8B3+/rdczMrOf6HO4RsVMHb1/U1/OamVnv+QnV3htZdgF9NLLsAvpoZNkF9NHI\n",
"sgvoo5FlF9AHI8suoB76/IRqry8sRTP3uZuZlaGn2emWu5lZC3K4m5m1oJoMhTQzs+pRrnmBZYtX\n",
"jzjczcxKJDEAmJN5n1yQ5f+4EvM8uyKzj1+GWd9cglnfXITZ3lgAzTKQdxb7kAnLTOzgmdCOz+sb\n",
"qmbWCCQGktbO3Zc05/6rpEVJXm33ei2CiWXV2RmJQcBcwDydvybPw1wvLMT8jw9h3mfnY+7n52Tu\n",
"52ZivieDOV8SHyz0Ge8u9j7vLfoW7yz+GhOWGscr/+85Xl/pBWLgBGAC6N6eZKfD3cxKVYT69sBx\n",
"pBlmTwUmAUPavBZu8/X8wLtMH/rT/SGIYLpVvnpQz4zA3HQZ0h2+5ijqmsAMH7zNkEc+Y8hDwUKP\n",
"DmK+J2dl8ItzMeub8wGTmDjLC3w+09PAGGb84FFm/GgM8Fxk8Vn39fUsOx3uZlY3yiVgMDCEiTMP\n",
"ZfTOW/Peotsw+MVJLHHPU8w1dgBiHmAcaSHyJ4t/nwLGRxZR/DGYj87Df8prIeATpv8j8BZpcfPO\n",
"Qnpm0h+ZCZ283gImoEkTWPnKQax01dwMGbUgs765GIqvkPrFFyKtEfxU+1dk8VaffocOdzOrlyK0\n",
"56XjkJ32vWAinw7+kDeWG8wnc3/AnOP+zgKPP4hiSsv7bdJShcu2e81AB2EJPBNZR+vwIlILvH0d\n",
"8zClhd3x6/2IqUs+KtfspKUV29ezDGnJxCl1tP1DNDaymNSHX2mnHO5mVnXKNZC0WPkKTBvgCwIf\n",
"0kUXCR/O/zq//896vLv4T4DXgBHAPW2DtJtrtx0x0vb15eIaHQX/y5F1H3LFz7VYJ+efB3i2o/NH\n",
"Fu/2pPZqcribWVUp1xzA1cDswB+ZNsBfiyw+6fC4dKNxF+CnpIXFc2BkT0O9B3UNIq0f0VEwzw48\n",
"TbvAJ30y+Eqb/ZYkdbd09AfixchicjVqrQaHu5lVjXItRhqD92/g4Mii29EqEjMAuwLHkvrQ84j6\n",
"zuuiXHMxfZfKIsBYpu/a+aCetfWWw93MqkK5vgrcSFpR7dfddXMUof49UqiPJYX6vbWus7/oaXb6\n",
"ISYz65RybQucA+wTWdzc5b4p1HcjhfrzwB4R3Ff7Kq0jDnczm04x+uUo0s3Tb0cWozrdN40L3x04\n",
"BngG+F4E99elUOuUw93MpqFcMwK/B1YE1oosXulwvxTqe5BC/Slglwj+Wa86rWsOdzP7QjHc8E+k\n",
"8d7fjCw+nG4fMROwJ3A0MAbYKYJ/1bVQ65bD3cwAUK5lgVtJ4X50R8P/JBYC7iWN+94+ggfqW6X1\n",
"lEfLmBnK9S3gGuCYyOLCDvcRs5KWqLs1guPrWJ614aGQZtYjyrU38HNgx8jing73SdPSXkd6CnX3\n",
"aj2AZJXzUEgz65JyDQBOBrYG1ossnu5i95NJk3Xt7GBvDg53s35IuWYDriDNm7JWVzMVSuwHDAfW\n",
"juDTOpVofeQ1VM36GeVahHRT9B1go26CfWPgeGDzCCbUqUSrAoe7WT+iXKuT5oe5Dtirq8UhJFYk\n",
"te63jeDZOpVoVeJuGbN+QrmGAxcA+0cW13e5bxryeCvwgwj+UY/6rLoc7mYtrphK4IfA4cBmkcV/\n",
"utw/DXm8BbgwgqvqUKLVgIdCmrUw5ZoBOBtYE9gyshjX5f5pyOMfgfdJE395ZEyD8VBIs35OueYm\n",
"9a1/TBrq+H4PDjuFNIJmJwd7c/MNVbMWpFxLAv8CRgPf6UmwS3wf2Ar4roc8Nj+33M1ajHKtR2qx\n",
"j4gszu3RMeLbpDVN1/WQx9bgcDdrIcq1G3AqsGtk8ZceHSNWAi4Hto7guVrWZ/XjcDdrAcVUAscD\n",
"OwPDIosxPTpODCENeTzMC2y0Foe7WZNTrlmAS0kLP38tsnijR8eJ2YCbgfMjuLqGJVoJfEPVrIkp\n",
"10KkaXgnAhtUEOwDSU+fPg6cVLMCrTQOd7MmpVwrk6YSuI3Ux/5JBYefAswF7Ochj62pz90yki4C\n",
"Ngdej4iVivfmAf4ALA6MBbaPiHf6ei0zS5RrM+AS4NDI4pqKjhUHAFsAX4+g07llrLlVo+V+MbBJ\n",
"u/eOAu6KiGWAu4vvzawKlOsQ0hwxw3sR7JsAx+FZHlteVaYfkLQEcEublvuTwDcjYryU+gQj4ivt\n",
"jvH0A2YVUK5BwBnAMGCLyGJsRceLlUmNre94ZEzzKnv6gQUjYnzx9XhgwRpdx6xfUK7BpDVOBawT\n",
"Wbxb0fFpyOMtwKEO9v6h5kMhIyIkdfjxQNKINt+OjIiRta7HrNko1xKkseh/Bw6LLCZVdHwa8ngL\n",
"8HsPeWw+koaRPq1VdlwNu2WGRcRrkoYA97hbxqxyyrU2cD1pDdMzI6vsf9hiyOP1pFWX9vTImOZX\n",
"drfMzcDupOFWuwM31ug6Zi1LuXYEfgvsGVnc1svT/BIYDGzvYO9f+txyl3Q18E3SyujjSXfibwKu\n",
"BRajk6GQbrmbdaxYXONnwN6kOdgf69V5xIHAoaSFrd+uYolWop5mpxfrMGsgyjUzaZjj0qShjq/1\n",
"6jxiU+Ai0iyPngyshfQ0O/2EqlmDUK75SUMVZyRN/tXbYF+FNNfMNg72/svhbtYAlGt54AHgHmDH\n",
"yOLjXp1HLMzUIY//rGKJ1mQ8K6RZyZRrI+BK4MeRxWW9Ps/UIY/nRVDRk6vWetznblYi5dqftALS\n",
"dpHFfb0+Txry+CdgArCXR8a0rrKHQppZF5RrIGnFpE2BdSOLZ/t4ylOBOYDtHOwGDnezulOuOYCr\n",
"gFmBtSOLPg1TlDiI9Edibc/yaFP4hqpZHRXL4d1CeiZkkyoE+2bAT4HNPJbd2nLL3ay+9gBmBvaL\n",
"LCb35URthjxuFcHzVajNWojD3axOlGte4OfAplUI9ilDHg+O4F/VqM9ai7tlzOrnFOCayGJUX04i\n",
"MTtplshzI/hDVSqzluOWu1kdKNc6pBXLlu/TedKQx6uAR4BfVKE0a1EOd7MaU64ZgHOBH0YW7/Xx\n",
"dKcBswPbesijdcXhblZ7hwGvANf15SQSBwPfxgtbWw/4CVWzGlKuRYFRwFp9eVBJYnPSbJHreGRM\n",
"/+YnVM0aw29IKyj1JdhXBS7BQx6tAg53sxpRrs2BlYCde30OsQhpyONBHvJolXC4m9WAcs0KnEV6\n",
"WOmTXp0jDXm8BTgngmurWZ+1Po9zN6uNY4EHIou7enNwMeTxalJ//cnVLMz6B7fczapMuZYD9gNW\n",
"6cNpTidNLLa/hzxabzjczaqoWNz6HOCEyOKVXp1DHAJsRBryOLGa9Vn/4XA3q65dgMGkgK9YMeTx\n",
"GFKwv1PNwqx/8Th3sypRrrmBMcDwyOLBio9PQx7vAraM4N/Vrs9aQ0+z0zdUzarnJODGXgb7UNLI\n",
"mAMd7FYN7pYxqwLlWhPYml5MDNZmyOPZEX2bosBsCnfLmPWRcg0CHgROjyyuqOjYNOTxJuA1YF+P\n",
"jLHuuFvGrH4OBN4BruzFsb8mrcx0gIPdqsndMmZ9oFwLAz8D1ousso/BEocCG+Ihj1YDDnezvjkd\n",
"+H1k8WQlB0lsARyNhzxajTjczXpJuTYC1gT2qug4sRpwMWnI4/9qUZuZ+9zNekG5ZiY9qHRIZPFR\n",
"j48TC5FuoHrIo9WUw92sd44ERkcWt/X0AImZgD8BF3nIo9Wau2XMKqRcSwGHAKv1+Bgh4HekIY/H\n",
"16g0sy843M0qUEwMdjZwcmQxroJDDwXWIN1AnVyT4szacLibVWY7YGHS8nk9IrEhaWTM2hF8UKvC\n",
"zNpyuJv1kHLNSRr6uENk0aNx6RJLkR5u2tEjY6yeahruksYC7wGfAxMjYs1aXs+sxo4H7ows7u/J\n",
"zhJzkkbGHB/BPTWtzKydWrfcAxgWERNqfB2zmlKu1YCdgBV6tL8YAFwB3E8v53Y364t6dMt4cjBr\n",
"aso1gDTS5ZjI4s0eHnY8MDewreeMsTLUepx7AH+V9JCkfdtvLBYnMGt0+5K6Fi/uyc4S2wPfA7aJ\n",
"4LNaFmbWmVq33NeJiFclzQ/cJenJiLivzfY9gcNqXINZrynXgsCJwAaRRbdDGIupBc4GNo7g9VrX\n",
"Z9aZmoZ7RLxa/PuGpBtI83C0Cfdj95F+9R5M/BwYGREja1mPWS/8Crg0snisux0lFgBuBA6OYFTN\n",
"K7N+QdIwYFilx9Us3CXNCgyMiPclzQZsDOTT7LTkek/x3EmPRHB9reow6y3lX/xP1e3qShIzAtcD\n",
"l0fwh9pWZv1J0egdOeV7SVlPjqtln/uCwH2SHgEeAG6NiL9Ms8emh00G9qhhDWa9olwzkka5HBZZ\n",
"dPngUTG1wJnABOC4OpRn1q2atdwj4n/QzQ3TeZ9ZnHmfXFr6ykIRvFarWsx64UfA86Rulu4cAKxL\n",
"egLVUwtYQyh3VkjFuWx8xCukkQVmDUG5vkQK90O6W11JYhiQAVtF8F4dyjPrkbKn/D2LpW4fyuyv\n",
"7lN8tDUrVTEx2G9Ji113OV2AxJeAa4BdIniuHvWZ9VSp4R5ZjGfApGv42m/mA75aZi1mheHAUsCp\n",
"Xe0kMTtpaoFfRPDXehRmVomyW+4gTudrZ83ETO/uU3Yp1r8p1+yk2R4PjCw6ffiomFrgMuAhUivf\n",
"rOGUHu6RxRPAv1j5ip0lZim7HuvXjgPujSy6m+TrOGAh4ABPLWCNqvRwB2DGD09k3VPEgM+2LrsU\n",
"65+Ua0XSsNwfd7mf2AbYG/huBJ/WoTSzXmmMcId7GfTJa6x8ZZf/Y5nVQpuJwbLIYnyn+4mVgXOB\n",
"rT101xpdQ4R7ZBEMmJix+gUrSSxadj3W7+wOzAT8vrMdJOYj3UA9LIKH6lWYWW81RLgDMMs71zDv\n",
"U5+w8uXHlF2K9R/KNS/wC2D/yOLzDvcRMwDXAddGcFU96zPrrYYJ98hiEhOWOp8VrtvVY96tjk4G\n",
"ro0sHu5inzOAjwA3PKxpNEy4AzDf0xmL3TcLq1+wXdmlWOtTrnWAzYCfdbqP2A/YANg5gg5b9maN\n",
"SNH109W1u7AUETFdC127bP43Pp9xwbjmhh4tZ2bWG8o1A/B/wM8ji2s63EesR5rpcd0Inq5nfWad\n",
"6Sw722usljvA6yv+kCVGLqfvfm9o2aVYSzsUGA8dT88rsThwLbCbg92aUcO13AG085bj0Of/jiv/\n",
"7O4ZqzrlWhQYBXw9spguuCVmA/4BXBnR9TQEZvXWvC13gCeHn8bQB7dSrpnLLsVa0hnA2Z0Eu0hr\n",
"pY4GTqt3YWbV0pjhPmqfc3h1dRi/otdXtapSrs2AVUjDHztyDLA4sJ+nFrBm1pDhHsFnjN7pNmZ7\n",
"/cfF04NmfaZcswJnAQdFFp9Mt11sRVp4Y+sIpttu1kwaNzgf2+UEPlxgTj4ftFnZpVjLOAZ4KLK4\n",
"s/0GiRWAC4FtInil7pWZVVnDhnt8PuMoHjz4VT6Z68Sya7Hmp1xfAb4PHD7dNjEPaWqBH0XwQL1r\n",
"M6uFhg13AB7b5TcollauNcouxZpXsbrSOcCJkcXL02wTg0jDIW+K4LIy6jOrhcYO94mzX879Rwxg\n",
"0kxHlV2KNbWdgbmBszvYdiowGTiyrhWZ1VhDh3sEbzJqz7uATZRriZLLsSakXHMBvyJNDDZpmm1i\n",
"L9L0AztGMKmj482aVUOHOwAfzX8+j+72LuBhkdYbJwE3RxbT9KVLfB04BRgewdulVGZWQw35hOq0\n",
"+zGIwS+8zGFfnokBk78UWfh/ROsR5foqcDOwfNv/boo1Ax4A9o3gtrLqM+uN5n5CtY0IJvHu4pfx\n",
"0trjgP3Krseag3INJK2adGS7YJ8FuAH4rYPdWlnDh3vhEm4/YwGCQ5VrxrKLsaZwIPA+cPmUN4qp\n",
"BS4EniZ1yZi1rKYI9wge59U1XuCDhV4Ddiq7HmtsyjUEOA44MLJp+h2PAJYF9vHUAtbqmiLcCxdz\n",
"1y8/AH5cjFs268zpwAWRxZgpb0hsTrop/50IPiqtMrM6aaZwv4bHdlmFyQMHABuXXYw1JuXaCFgb\n",
"OOGL98RywCXAdhGMK6k0s7pqmnBPw9UG3M5D338I+HHZ9VjjKaaIPhs4JLL4CEBibtLUAkdFcH+Z\n",
"9ZnVU9OEe+ES7jxtRWA55Vq17GKs4RwBjIksbgGQGAhcDdwRwYWlVmZWZ80W7n/l85kX4PUVrgV+\n",
"VHYx1jiUaynS0nmHtnn7FGAQ8MNSijIrUVOFe7H6/GVcfseMwObFcmnWzxU32M8CfhlZvAggsRvw\n",
"HWAHTy1g/VFThXvhEt4fuh2TB16GpySwZFtgKPBrAIk1SUvkDY/grTILMytL04V7BM8Az3DNDaOB\n",
"PZVrcNk1WXmUa05SqB8QWUyUWBj4E2ks++PlVmdWnqYL98LFPL3llsAdwL5lF2OlyoG7Iov7JGYm\n",
"TS1wbgQ3lVyXWalqFu6SNpH0pKRnJFV7ruxrgWE8v/5FwGGekqB/KkZM7QIcUUwtcB7wAmkmSLN+\n",
"rSbhLmkg6QbXJsDywE6SlqvW+SN4H7iJy+5emTRPyPbVOrc1h2Lh9N8Bx0YWb5CWz1sZ2NNTC5jV\n",
"ruW+JvBsRIyNiInANcDwKl/jYmBPJg84FU9J0B/tAwRwocS3gZ+Qphb4sNyyzBpDrcJ9EZjmMe+X\n",
"iveq6V5gDk785HVgBmCDKp/fGpRyLQCcCOzPiFgSuAzYPoIXyq3MrHHUKtxr/rE4gsnAJUyeYU/S\n",
"OpiekqD/OB24nBHxAmkxjuMiuK/kmswayqAanfdloO0DRouSWu/TkDSizbcjI2Jkhde5FHiIkT87\n",
"mmEnnKRcK0cWj1VarDUP5doCWJuX11iV1N13TwTnlVyWWc1IGgYMq/i4WiyzJ2kQ8BSpq+QV4EFg\n",
"p4h4os0+PVoqqvtrcTdwLiO0FLBsZLFHX89pjal4puG/wO6MiI2BtYCNIphYbmVm9VPqMnsRMQk4\n",
"GLgTGAP8oW2wV1m6sZqGwW2lXNXu27fG8Svgz4yIJYAdSFP4OtjNOtDwC2R3fx5mJXUDrcgIHQF8\n",
"HFkc1ecCraEo14YEF3H6uD/w/tDtgM0iGNPtgWYtpmUWyO5OsarOH4FdgTOAfZRrjnKrsmpSrtkJ\n",
"XcBNF7/I+0O/DqzpYDfrWtOHeyF1zYyIscDdwN7llmNV9dG8v+WJrefgkT2eA9aP4PWySzJrdK0S\n",
"7v8CRLrBdipwuHLVaiSQ1ZG2/P5+TB60O3edciawRwSfll2TWTNoiXAvHje/BNgzsvgPMJY0Daw1\n",
"Mc3x6vf48t3n8OTwX8aEpUZ4WgGznmuJcC9cBmxb3GA9FThOuYaWXJP1gsQAiZx1Tz6Lmd67O245\n",
"7+iyazJrNi0T7hG8DDwAbA3cClwJ/J9yuQXfRIo/ztewxD1bs+ZZnzHbG7uVXZNZM2r6oZDTnpPt\n",
"gf0i2BBAudYkhfx9wGGRxfvVvJ5Vl8QiwE3M8OFTHDPHqihOiiyuKrsus0bSb4ZCtnMzsKrE4gCR\n",
"xYPAasBkYJRyrVVmcdY5iTVIn7yu55jZn0XxHHB1yWWZNa2Warmn83I2MD6C46d5P9d3SfN/nw38\n",
"PLLwoskNovjEdTawHyP0PPBXYNXI4uVyKzNrPD3NzlYM9zVIKzUtVcwcOXVbmprgEmBW4HuRxfPV\n",
"vr71XLF60nHAXsB3GKHRpNb72ZHFRaUWZ9ag+mu3DMD/AR8C67XfULQEv016ovUB5drNi3yUQ2IW\n",
"UrfLpsDXIhhFmrb5LdJDaWbWBy3Xck/n5ofAGsBuEXTY/aJcq5Butj4O7B9ZvF2LWmx6EgsDNwLP\n",
"AntH8LFyLUe68b1GZDG2zPrMGll/brkDXA4sA7whcYPEQRLLFN0AAEQWjwJfBcYDjyrXsHJK7V8k\n",
"Vid1vdwE7FIE+0DgQiBzsJtVR0u23KdegwVJc8pvVLwmA3eRbtjdPWWOEuXahBQulwPHRRaf1bKu\n",
"/kpiW9JN7f0juP6L93P9APguMCyymNzZ8WbWj2+odn49BCwLbEgK+m+SpilIYb/TVk+w7C1nkdZ6\n",
"3SWyeLJetbW64nd/LLAfaRHrh7/YlmtJUkt+7cjimZJKNGsaDvdur88gYE1S2G8IrAaT/8PmB7/L\n",
"6ucPA45h4KRzIyvpF9QiihunFwJLAcMjePWLbbkGkGbxvDWyOK2kEs2aSn/vc+9WBJMi+GcEx0fw\n",
"DWBhGHAat53zAr9/6A3eWOFMXvz6yxoy6scSS5ZdbzOSGAKMJP139s22wV7YlzQs9Yw6l2bW8vpt\n",
"y707GjJqcTb5wZksOHpDbrz4Y54a/i6pr/4u4G8RvFVyiQ1NYjXSTdMLgBPaz+ioXIuRhq0Oiywe\n",
"L6FEs6bkbpkqUa5hBJfy3tB/cO6oUXw83zDSGPpnmBr290fwSZl1Noqiu2tv4ETgwAium26f9GzB\n",
"n4H7I4sT61yiWVNzt0yVRBYjEasy+KWBHDn/nozQscD8wOHAp8AJpCGXf5E4QmI1qf/9Xotpencg\n",
"LYi+I7BRR8Fe2A1YCDilXvWZ9TduufdQ0drcFTgd+AVwxpRhexKDgWFMHYkzL+lG4V+Bv0YwtoSS\n",
"66IYCbMpcBIwCTiG9DN3+B+Wcg0BHgW+HVmMqluhZi3C3TI1olxfAq4APgL26GhyK4lFmToKZ0Pg\n",
"PaZ24dwTQUs8DSuxHvBz0h+znwI3dLVaUvEH8gbgv5HFT+tTpVlrcbdMjUQW/yONkb8XeLiYbXLa\n",
"fYJxEVwcwS7AEGAb0qP2+wIvSDwgcZLEMImZ6ll/NRRdT7eTVr+6AFgpgj/1YBm87UlPDp9Q6xrN\n",
"+ju33PugmB/+CuDvpMVAPuj2mBTma5O6bzYElgPuZ+qTs6Mbda1QiWVJwbweqRvm/J4uWK1c8wOP\n",
"Ad+JLB6oXZVmrc0t9zqILP5NWgwE0mIga3Z7TPBpBCMjODaCrwGLA+cDSwPXA69JXCmxp0RDrAEr\n",
"sZjEBcA/gFGk6ZTP6mmwF34DXOVgN6sPt9yrRLm2Ac4BzgR+EVl83qvziCWYemN2A+BNprbqR0bw\n",
"blUK7lktCwBHk0a3nAf8qjf3C5RrOHAasHJk8VF1qzTrX3xDtQTKNZS0GMjMpMVA/ten86Uhlasy\n",
"NezXAkZbS1UMAAAJWElEQVQzNez/HcHEvlyjk+sOBn4EHARcBZwUwWu9OleuuYH/AjtHFn+vXpVm\n",
"/ZPDvSTFfCmHA0cV/15Zrflpinla1mFq2C9FmgN9StiP6Ut/fXH+g4GfkB4yyiPo2x+oXBcBH0cW\n",
"B/XlPGaWONxLplyrklq9jwIHRBbvVP0aYj5gfaaG/UxMHXJ5dwSv9PA8M5CeKv0Z8G/gZxGM6XN9\n",
"ub5N6s5ZKbJ4v6/nMzOHe0NQrlmAXwJbAbvVsluieJjoy0wdhbM+8ApTw/7eCN5vd8xA0tOkxwPP\n",
"AcdG8J+q1JNrDlIX0n6RxV+qcU4zc7g3FOXajDQe/FLSakM1XwykCO7VmRr2XyWNdJkS9vOThjN+\n",
"ABwTwT1VvX6us4FZIou9qnles/7O4d5glGsB0rzmQ0iLgTxV1+uL2YB1mboq1efACOCWao+rV64N\n",
"STeWV/LatGbV5XBvQMXj9/uTukGOBc5vtcVAlOsbwB+BHSOLv5Vdj1mrcbg3MOVaDrgSeBHYN7J4\n",
"o+SSqqJYZPxaYKfI4u6SyzFrSX5CtYFFFk+Qxqw/BTxSLNDd1JRrfVKw7+BgNyufW+4lU65vkW60\n",
"3gAcGVk03aIfRR/7VcB2flDJrLZKbblLGiHpJUmjilfTt0xrJbK4B1iFdKP1P8q1UsklVUS5NiYF\n",
"+zYOdrPGUZOWu6QMeD8iTu9iH7fc2yhutu4GnEqaI/03UxYDaVRFd9JlwNaRxf1l12PWHzRCn7uD\n",
"uwKRRUQWlwJfA7YD7lCuhUsuq1PKtTkp2Ic72M0aTy3D/RBJj0q6UNJcNbxOS4ksnge+QZrj/WHl\n",
"2rrkkqajXFsCFwNbRhb/KrseM5ter7tlJN1FWuS4vWNJ85NMGd53AjAkIvZud3wAeZu3RkbEyF4V\n",
"06KUa23SYiB/Aw7vyWIgtVZM3/t7YIvIoipTFZhZ5yQNI63RPEXWEOPcJS0B3BIRK7V7333uPVDM\n",
"0fJb0myQu5QZqMWSgucAm0UWD5dVh1l/VupDTJKGRMSrxdeHA1+NiJ17U6AlyrUdcBYp6E/u7WIg\n",
"fbj+tsX1N40sRtXz2mY2VdnhfhlpkYkA/gd8PyLG96ZAm6pYDOQyYBBpMZAX6nTdHYAzgE0ii0fr\n",
"cU0z65inH2hRxWIgPyItqHF4ZHFlja+3M2mJvI0ji9G1vJaZdc/h3uKUazXS/DSjgINqshhIrl1J\n",
"89FvFFk8Xu3zm1nlGmGcu9VQ0e+9BvAOaX6a9ap5fuXaHTgF2NDBbtZ83HJvAcq1BXA+cBEwIrLo\n",
"06LZyrUXaVriDSOLJ6tQoplViVvu/UhkcSvpBvaqwP3KtUxvz6Vc+5CeP1jfwW7WvNxybyHF/DQH\n",
"kML5GOCCShYDUa7vkx5CWz+yeLY2VZpZX/iGaj+mXMuTbraOJS0G8mYPjjkQOALYILJ4rrYVmllv\n",
"uVumH4ssxpAWA3mWdLN14672V66DSUMrv+VgN2sNbrm3uGKFpEtJ65oe3X4xEOU6DDiM1BUztv4V\n",
"mlkl3HI3AIpFqlcBhgIPtl0MRLl+CBwKDHOwm7UWt9z7ieJm6+7Ar4ATgZmAfUkt9nFl1mZmPecb\n",
"qtYh5VqSNI3wvKQ+9pdLLsnMKuBwt04p10Bgxsji47JrMbPKONzNzFqQb6iamfVjDnczsxbkcDcz\n",
"a0EOdzOzFuRwNzNrQQ53M7MW5HA3M2tBDnczsxbkcDcza0EOdzOzFuRwNzNrQQ53M7MW5HA3M2tB\n",
"DnczsxbkcDcza0EOdzOzFuRwNzNrQQ53M7MW5HA3M2tBDnczsxbkcDcza0EOdzOzFtTrcJe0naTH\n",
"JX0uafV2246W9IykJyVt3PcyzcysEn1puY8GtgbubfumpOWBHYDlgU2AcyS13CcEScPKrqEvXH+5\n",
"XH95mrn2SvQ6dCPiyYh4uoNNw4GrI2JiRIwFngXW7O11Gtiwsgvoo2FlF9BHw8ouoI+GlV1AHw0r\n",
"u4A+GFZ2AfVQixb1wsBLbb5/CVikBtcxM7NODOpqo6S7gIU62HRMRNxSwXWioqrMzKxPFNG33JV0\n",
"D/CjiHi4+P4ogIg4ufj+DiCLiAfaHefANzPrhYhQd/t02XKvQNsL3QxcJel0UnfM0sCDvSnOzMx6\n",
"py9DIbeWNA5YC7hN0u0AETEGuBYYA9wOHBh9/XhgZmYV6XO3jJmZNZ5Sx59LOkHSo5IekXS3pEXL\n",
"rKdSkn4l6YniZ/iTpMFl11SJrh5Ea1SSNikejntG0pFl11MpSRdJGi9pdNm1VErSopLuKf6b+a+k\n",
"Q8uuqRKSZpb0QJE3YyT9ouyaekPSQEmjJHU5qKXsh4t+GRGrRMSqwI1AVnI9lfoLsEJErAI8DRxd\n",
"cj2V6vBBtEYlaSBwFunhuOWBnSQtV25VFbuYVH8zmggcHhErkLpjD2qm339EfAJ8q8iblYFvSVq3\n",
"5LJ64zBSt3eX3S6lhntEvN/m29mBN8uqpTci4q6ImFx8+wAwtMx6KtXFg2iNak3g2YgYGxETgWtI\n",
"D801jYi4D3i77Dp6IyJei4hHiq8/AJ4gPdfSNCLio+LLGYGBwIQSy6mYpKHAZsAFTDuQZTplt9yR\n",
"dJKkF4HdgZPLrqcP9gL+XHYRLW4RYFyb7/2AXEkkLQGsRmrUNA1JAyQ9AowH7ikGgDSTXwM/ASZ3\n",
"t2PNw13SXZJGd/DaEiAijo2IxYBLSIU3lO7qL/Y5FvgsIq4qsdQO9aT+JuK7/w1A0uzAH4HDihZ8\n",
"04iIyUW3zFDgG800z4ykLYDXI2IU3bTaoXrj3DsVERv1cNeraMCWb3f1S9qD9DFpg7oUVKEKfv/N\n",
"4GWg7U33RZl2qgurMUkzANcDV0TEjWXX01sR8a6k24A1gJEll9NTXwe2krQZMDMwp6TLImK3jnYu\n",
"e7TM0m2+HQ6MKquW3pC0Cekj0vDiZk0za4aHyh4Clpa0hKQZSbOP3lxyTf2GJAEXAmMi4oyy66mU\n",
"pPkkzVV8PQuwEU2UORFxTEQsGhFfAnYE/tZZsEP5fe6/KLoIHiHN1Pajkuup1JmkG8F3FUOTzim7\n",
"oEp09iBao4qIScDBwJ2k0QJ/iIgnyq2qMpKuBv4JLCNpnKQ9y66pAusAu5JGmYwqXs008mcI8Lci\n",
"bx4AbomIu0uuqS+67Kb0Q0xmZi2o7Ja7mZnVgMPdzKwFOdzNzFqQw93MrAU53M3MWpDD3cysBTnc\n",
"zcxakMPdzKwF/X+jAwLESxPhCwAAAABJRU5ErkJggg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x7f7e890946a0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"##This function provides an interactive mode to change polynomial degree. \n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" X = construct_poly(x,n) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(x,X.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" plt.plot(x,y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import scipy\n",
"from scipy import linalg\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"A = np.array([[10,1,0.1],[10,1,0.2],[10,1.1,0.7]])\n",
"a = np.array([[10,1,0.1]])\n",
"A = a.T.dot(a)\n",
"\n",
"B,rank = linalg.pinv(A,return_rank=True)\n",
"#print rank\n",
"#print B\n",
"#print np.allclose(A,A.dot(B.dot(A)))"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecFPX9x/HXmyYI2I2goKCCxgoGwYBlwN5jjF2KvYu9\n",
"/wRiL7EmauydxITYEhtGxt5Qwd5QoiIiNkREpXx+f3znYO9u727vdvdmd+/zfDz2cbc7OzOf3b37\n",
"7Hc+853vV2aGc865ytEq7QCcc84Vlid255yrMJ7YnXOuwnhid865CuOJ3TnnKownduecqzCe2N0i\n",
"kk6XdEPacTjn8uOJvQgkTZX0s6Tlazz+mqSFklZN7neTNE7STEnfSXpD0vBkWY/kubNr3PaoY5+x\n",
"pIPyidvMLjCzQ/LZRkY8Jyev53tJH0k6KWPZipLGSpqWvO5nJPWvZ1vHS5qSbGuGpFskdc5YPlDS\n",
"S8nyyZIGZSyLsryPQzOW35p8VlXLvpekZNlmWd7/hZJ2S5YvIeny5HV8I+kvktrk+P5Ufb55/w8m\n",
"r+GcBp4zVdKPGa/jkYxlZ9R4jT9KWiBpuSzbyfmzk3Rz8hpXz3hsKUl3Jn/zM5PfMz/LhZJ+yIjl\n",
"+oxlwyVNlDRL0qeSLpLUOmN55nqzJc2XdFXu72Tl8MReHAZ8BOxT9YCk9YEOybIqdwD/A1YFlgOG\n",
"AjNqbGtpM+uccftHPfusU64Jp8CGAssA2wFHS9orebwT8CKwEbAscBvwH0kd69jO/UA/M1sKWJvw\n",
"fp0JkCSfB4GLgKWBi4EHJS2Tsf60Gu/hHRnLDLgoY9lSlly1Z2ZPZ64H7AT8AFQlxdOS17Au0Dv5\n",
"/axGvkdq5PObyoCdMl7PdosWmJ1f43VeBEwws2+ybCenz07SpsDq1P67HA2sAPQE1gBWSh7LtH5G\n",
"PIdmPN4BGAksDwwAtgQWNRjMrFPGa+gCzAXuaeB9qUxm5rcC34CPCYnnpYzHLgXOABYCqyaPzQY2\n",
"qGMbPZLntsphf+cB8wl/yLOBq5LHFwJHAh8AU5LHrgQ+AWYBE4FNM7YzGrijxv6HEb58ZgJn5PGe\n",
"XFkVVx3LZwF9c9jO8sB4YGRyfyfgrRrPeQ84MPk9Aj6tZ3u3AOfk+BpuAW7KuP8y8IeM+/sAn+S4\n",
"rU+S93d2chuQPH4g8DbwDeELZNWMdS4nfPHPAl4nfKEcCvwC/Jxs5/56/ia3zCEuERolQxvx2Vb7\n",
"7IA2wKvA+slrXD1j2aPA4Rn3jwIeybi/EFgjx/0eDzxQx7LhwIdN/Xst95u32IvnBWApSWsnh4t7\n",
"AXdmec41kvaqKs9k0WCLzszOBJ4GjrLQYjk2Y/GuwMbAOsn9l4ANCa2tu4F/SGpXtaksmx9EaI1u\n",
"CZwtaS0ILTJJ3zYUW/JcAZsDb9axvA/QDviwnm3sK2kW4QtmppldWc8uWxGSXpVfSfoiKQldJmnJ\n",
"Gs8/UtLXyWH+7+vYf0dgd0ILtdqiGvvtlllaqMdmyc+qI7IXJe0KnA7sRmjVPg2MTfa/bbJOLzNb\n",
"GtgD+NrMrgfuYvFRx6717PMuSV9KelTSBvXEtSIwLofXUNdndzzwpJm9kWWVR4HdJS0jaVnCe/pQ\n",
"jec8JWm6QplytXp2vwV1/E0REvvtubyGipT2N0sl3khaR4RW+/mEUsSjQGuqt9iXAS4g/HHOB14j\n",
"lBxgcYv52xq3terY5wTgoBqPLQSiBmL9hnDoC9lb7CtnPPdFYK8mvB9jktfWNsuypYA3gFNz3Naa\n",
"ybaOT+4vn7yGvYC2hH/oBcC1yfKVgLUzXtOTwHUZ2+tL+JJrBWwPfA8MzLLfoSRHPRmPnQM8Q0jC\n",
"XZL3ZwGwUg6vo+r9bZXx2MMkRxrJ/VbAHELpaTDhSGQANY7iyOGoA/gtsAShnHEaMJ3wpVLzeTcB\n",
"N+f4WdT67IDuhCPEzhl/g5kt9iUIR1wLktujmX8XwKaEFv/SwNXJ9ltn2feBhKOe5bIsW43w/7Ra\n",
"If+vy+mWegCVeCMk9iHJP+T/CK2u/ZI/2EWJvcY6yyf/oJ8l92v94zewzwmZSSF5rNZhLaEm+Tbw\n",
"HeGLYgEwOFk2mtqJvVV9+8ghrqOBKWR8QWQs60BItH9t5Db3Al7LuL854Ujka8JRyKPAmXWsO4DQ\n",
"4q9r29cCl2Z5/HFgVI3H2ifJ5zNCi/U04OccX0O29/dtQjkl84t8DrBJsvwYQvlsJvDXjOSZczkp\n",
"Y1/vEGrumY8tSSirbJHD+lk/O0JLf2jyu2r+DQL/BP6crN8xeb//Xsc+WiXvx7o1Hv8d8EXNxzOW\n",
"n0U4R5B6Lkjr5qWYIjKzTwj1yu2BfzXw3K+BPwErJ4eojd5dQ49L2gw4GdjDzJYxs2UJ/8hFOYEn\n",
"6UDgFEJt9/May5YA7iPUpA9r5KbbAj9W3TGzp8ysv5ktTzgnsDYh0delUX/3kroTDvurHdqb2U9m\n",
"doyZdTOzNQlHDhNz3Gy2z+sT4FAzWzbj1tHMXkj2d7WZ9SOU1XoTPsu6ttWU/e9GKO88Wd+KDXx2\n",
"Q4BLJE0Hqj7z5yXtnfy+HeHLYK6ZzSF8Qe1Q165q/ETSdsD1hC+lt+pYbxi1S2Ytiif24jsIGGJm\n",
"c2suSLprrSupTVKXPQL4wMwya9e5Jt0ZhF4G9elMOET9SlI7SWcTDqcbI6d4JO1HOKm7jZlNrbGs\n",
"LaHl9iMwIodtHSxpxeT3dQgt43EZy/tKaitpKcJJ6k/MbHyyLJK0moLuhB4f92Ws+wdJnSS1krQN\n",
"4cjqgRohDAWeNbOPa8S1cnKTpE0ILcVRDb45wUyS1mzGY9cBZySvEUlLK+neKqmfpAHJe/cj8BPh\n",
"aAvCZ786dZDUXdKg5DNvL+lkwhHiszWe2mBdOofPrhewAeE8Tp/ksZ1Y/J6/DhySxNGBcPJ3crLt\n",
"dST1kdRaUifgMsLR0DvJ8iGE8wm/N7OsX6CSBgIrA3X1HmsZ0j5kqMQbSSkmy+NtCP+MVTX2q4D3\n",
"CYebXxISylrJsh5U7zVRdTuujn1uQqjBfgNckTxWs77ZilBDnUVoTZ1MOKIYkiwfBdyesf8F1FGK\n",
"IZxkm13Pe/ARi3tqVN2uSZZtkcT2Q43lg7JtG7iZcOg9O3mNpwDKWH43obT0HaHstULGsuMJyWEO\n",
"oUV8BdAxY/lTyXqzCLX7PbO8lneAA7I8vlnyWc9JnrNPI/9OxiSf+7dA/+Sx/QnJb1YS743J40MI\n",
"CXA24UvhDmDJZFnVeYdvgX9l2c86ybo/AF8Ratwb1XjOKoTeNatnWf9aFp+zqPezy7Lughp/g70J\n",
"vX2+Tm4PkZRqCOcR3k22PYNwlJtZxnkiiTFzv/+psb/rgNvSzgFp35S8GU2StIBuB35FOLS73syu\n",
"SvoW/51wEmMq4Z/luybvyDnnXM7yTexdgC5mNik5dHqFcGLjAOArM7tY0qnAsmZ2WkEids45V6+8\n",
"auxm9oWZTUp+/4FwOLoKsAuLT17cRkj2zjnnmkHBTp5K6kHoE/wioR9v1aXxMwh9iZ1zzjWDgiT2\n",
"pAwzjnCZ9+zMZRZqPU2v9zjnnGuUvAeGSro/jSNc2FLVpWmGpC5m9oWkroQz/zXX82TvnHNNYGb1\n",
"djvOK7EnY4DcBLxtZldkLHqA0Cf2ouTnfVlWbzC4UiBptJmNTjuOhnicheVxFlY5xFkOMUJujeJ8\n",
"W+yDSPrdSnoteex04ELgHoXxwacCe+a5H+eccznKK7Gb2TPUXaffKp9tO+ecaxofUqBhcdoB5ChO\n",
"O4AcxWkHkKM47QByFKcdQI7itAPIQZx2AIWS1wVKee1YsnKosTvnXCnJJXd6i9055yqMJ3bnnKsw\n",
"ntidc67CeGJ3zrkK44ndOecqjCd255yrMJ7YnXOuwnhid865CuOJ3TnnKownduecqzCe2J1zrsJ4\n",
"YnfOuQrjid055yqMJ3bnnKswntidc67CeGJ3zrlikZZLY7ee2J1zrtCkDkh/Bh5BavYJhTyxO+dc\n",
"IUl9gInA8sA2pDBNnSd255wrBKkV0knAeOACYF/MvksjlDZp7NQ55yqKtApwG9Ae6I/Zx2mG4y12\n",
"55zLh7Q78CoQA1HaSR28xe6cc00jdQauBDYDdsHsxZQjWsRb7M4511jSJsBrwEKgbykldfAWu3PO\n",
"5U5qA5wBHAUcidm4lCPKyhO7c87lQuoJ3AnMBTbCbFrKEdXJSzHOOVcfSUjDgJeAcYS+6SWb1KEA\n",
"iV3SzZJmSHoj47HRkj6T9Fpy2y7f/TjnXLOTlgXGAqcCW2F2GWYLU46qQYVosd8C1EzcBlxmZn2T\n",
"2yMF2I9zzjUfKQImATOBfphNTjeg3OVdYzezpyX1yLKo2cdHcM65vEntgD8Cw4CDMHs45YgarZg1\n",
"9mMkTZZ0k6Rlirgf55wrDGlt4HlgHaBPOSZ1KF5ivxboCfQBpgN/KtJ+nHMuf+EE6eHAM8ANwK6Y\n",
"fZlyVE1WlO6OlvGGSLoReDDb8ySNzrgbm1lcjHicc64ux+vyDQ9kvb+145dVuvHZgI425520Y8qk\n",
"UOuPGrVOIUaUTGrsD5rZ+sn9rmY2Pfn9eGBjM9u3xjpmZl6Hd841O4kOwO/25a6TLuWkvk+yxaQT\n",
"+dPRn7PK82Y0+zC7jZFL7sw7sUsaC2wBrADMAEYRvl36EHrHfAwcZmYzGhucc84VioSAAcCIDvy4\n",
"x40c/MMuPNDxR5bc91f25WNpx5erZknsTeWJ3TnXHCRWBoYCI4BWv2fcI2PZZ9t2zJsMHI7Zt6kG\n",
"2Ei55E4fUsA5V3Ek2gO7AAcQWun/bM/cg+fQsX8r7AzgROCONGY3ag6e2J1zFSEptfQjtMz3Ioy+\n",
"eCuwu6FlCBNhdAQGYPZRSmE2Cx8rxjlX1iS6SJwEvEG4/H86sJEZW5txl6HtCEn+aWDzSk/q4C12\n",
"51wZklgC2IlQahkE/As4AnhmUa8WqRNwBTCY0C/9BQDFcTvCdTa9atyWsiga0LyvpDj85Klzriwk\n",
"pZa+hFLLPsCbhLGqxpkxJ/O53yy11G9nLrPM3c+st94HJx1xxGPfde68KosTeDdgGvA+8EHmzaJo\n",
"SnO9nqbyXjHOubIn8StgP0LrvDOhbn47E+L/EZJ0L6A30EtmvZafNWuT2UsuuUKbBQu+mtOhw2Rq\n",
"J/CPLYp+SeGlFIQndufcIopjAUsDXeq5rQh8Qe1k+KlF0YJmi1W0BXZENoLlf4lYf9bzbDf9Tfp9\n",
"a7Ra1PJeHfi2KsYe06fPHH3bbb9b+5NPvp/fqtV+g958s+Rb303hid25FkBx3AFYifoTdtXtZ0Li\n",
"znabAXydPK831evPywMfUT3ZVyX/zy2KmpxIki+cFYFevLHUEGYusRPtFm7Iaj/Op+tPbWht36Nq\n",
"XzRVv39oURRKMNJ+hHr6RUBZjJneVJ7YnatQiuPuhO57GwEdqDtZV0vcFkU/NnF/HYE1qX3CsTfQ\n",
"CfiQ6km36vZlVdJXHC+XZd1eGL35pVVrPu0Any1pLNREVp57H2vPfpqQvGfVHZiWAa4hXOm+H2av\n",
"NeX1lRNP7M5VIMXxZsDfgcsJIxHOyqfFXIB4liJ7wu9F6Hn3CbAy0JaqxD9fU3hqxSV5bKW+vNt5\n",
"I2a1e5BwInSCGbm1tqXNgduBfwOnYNakL61y44nduQqSlCwOB0YDwyyKHk03ooYlrfRVCX3Lv2Rw\n",
"tC6hV8v+wBTCidB7zKi7VQ5hWN0wHlU3YBVCF8Z9gYMx+0+Rwi9JPqSAcxVCcbwEcDUwEBhkUfRh\n",
"yiHlxKLoG4UUtA8hoXcltLI3N+N9AKQ2iO6EhN0t42fm7ysDc4DPktuHhIkwqg0u6AJvsTtX4hTH\n",
"XYF/Ek5uDrcomp1ySDnroi9GLM2sy/ow6bXdGffy7/nXd21YsArVk/iKhHlFp7E4cU+r9dNsbiov\n",
"osR4Kca5Mqc47g+MI9TSz7UoKo3eHqE0sjTVW9WLWtoLaNVtLh16t2VeO+DzJfjlI7In7s+ALzCb\n",
"n8KrKEue2J0rY4rj4cAlwCEWRfc3347VitCKrlkOqVkiMeBTarSu/8uQhf/HOSdOY5W3fqDT8K9t\n",
"+bI5wigHntidK0OK47bApcD2wO8sit4u3MbVjlDnzlbHrvrZFfieussiVaWR72tvnr0J5wLOAq4v\n",
"9dmIypGfPHWuzCiOVwDuIVxI1N+i6LvcV1ZH6j8B2Q1YjlCrr1kWmZjx2OeY/dSouMP455cDWwHb\n",
"mFHx/clLmSd250qE4nhD4D5CH/UzF13CH+rZy1J3wq762Z7aret3gf+yOInPwKygQwNIrAn8g3BB\n",
"0m/MqNWSd83LSzHOpUlqDay0w4UXDn+ib9/Tjrr33nF/uu66L6mduH+hvrJI+PlNc88IJLEH4crP\n",
"0cA1XnopPi/FOFcqpI0Jk75Xa3HPb9Wqy6mHHTbv9dVXb/PIKac8F02e/BOhvj2e6vXsH9IKPZtk\n",
"PPQ/Ec4DbGfGKymH5DJ4Yneu2KQNgYeAOwmJ+gXgs7FDhsze/8wzL1rYqlV7YM9o0qSZaYaZK4k1\n",
"COcBphJKL7mfB3DNwhO7c8UkLUvohz4Ss7sXPRzHvwbuBx4GTrIompdShI0isTtwLXAucLWXXkqT\n",
"19idK5bQH/xB4APMjlv0cBzvAtwInGpRdEta4TVGUnq5GNgF2MuMl1IOqcXyGrtz6TqbMOPPyQCK\n",
"41bAmcChwM4WRS+mGFvOJHoSeup8Tpgk+tuUQ3IN8MTuXDFIOwEHA/0wm6c4rprSrSuhf/r0NMPL\n",
"lcTvgOuBC4ArvPRSHjyxO1do0prAzcCumH2hOF6DUE9/HtjXoujnVOPLgUQ74ELg98AuZryQckiu\n",
"EVqlHYBzFSVc/XkvMBqz5xXHWwPPEfp6H1omSX014CnCjEkbeVIvP57YnSuUcIXojcAro4cPv1Zx\n",
"fBJh7PE9LIquSXOWo1xJ7Ay8RBgmeFczvkk5JNcEXopxrnBGAmudfcABW54zbNgdwK+BARZFn6Qc\n",
"V4Mk2gLnA3sBu5nxXMohuTzk3WKXdLOkGZLeyHhsOUnjJb0v6TGFCWedq1zSFsDpZx144NHnDBv2\n",
"OCBgszJJ6t2BJ4F1gL6e1MtfIUoxtwDb1XjsNGC8mfUmDEB0WgH241xpklYBxp5y6KEXnzd06Djg\n",
"bmB/i6KSn1xZYgfgZcLJ3Z3N+DrlkFwBFOQCJUk9gAfNbP3k/rvAFmY2Q1IXIDaztWus4xcoufIn\n",
"LbEQ4sNPOGHmDTvvPAAYalH0WNphNSQpvZwD7AfsY8YzKYfkcpTmBUor2eJJZmcAKxVpP86l6of2\n",
"7a/c96yzuj44cGBnYKBF0ZS0Y2qIRDfgb8APhF4vZTFGjctd0U+emplJynpYIGl0xt3YzOJix+Nc\n",
"oTy37rojj7nyyuFv9uw5Hmm/cphkWmI7woVSVwMXmFEac6i6OkmKgKhR6xSxFBOZ2ReSugITvBTj\n",
"KslRxx037N5NN72l09y5f/mge/fjSmaS6TpItAHGAMOB/cx4MuWQXBOlWYp5gPAHdFHy874i7ce5\n",
"Ztf9nnuO+XHIkCv2e/zxS6666qqS7xggsTLhhO48wjC7MxpYxZW5vFvsksYSJhBYgVBPP5twhv0e\n",
"YFXCmM17mtl3NdbzFrsrK4rjtq0WLrxs5a++OvDWCy8cu+Wrrx6cdkwNkdiacJHUtcB5ZhR0WjzX\n",
"/HLJnT5sr3M5UByvCNyz3kcfdf/vCSdM+9WsWVtiNj/tuOoi0RoYBRwEDDXjiZRDcgXiw/Y6VwCK\n",
"4z7AfZu89dYrTx977OptFi78Q4kn9S6E0guE0ssXacbjmp+PFeNcPRTHewPjN3399SueP/rozZKk\n",
"XrLdAyWGAK8CTwNbe1JvmbzF7lwWiuPWhLFT9tzgww93eXrkyJuBMzB7OeXQskpKL2cBhwHDzHg8\n",
"5ZBcirzG7lwNiuNlCaWMJTrOnbvnDzvscB3wDWaHphxaVhIrESbKbgvsa8bnKYfkiiiX3OmlGOcy\n",
"JNPXPQB8BGz7ww47HAh0B45JNbA6SETAK8CLwFae1B14Kca5mo4gjMx4jA0ePBg4HuiPWUlNkCGx\n",
"CnAUcAAwwoxHUw7JlRAvxTiXUByvSmj9bmaDB88ltIL3wWxCupEFEu2BXQnJvD9hMowxZkxLNTDX\n",
"rLy7o3M5UhyLcBHPlTZ48FTgGeDStJO6hICNCcl8T8IXz63A780o+WGBXTo8sTsX7AN0X2PatN0I\n",
"Cf4j4E9pBSPRFdgfGAG0IyTzvmaU/MQdLn1einEtnuJ4BeBNYBcbPLgPYYq7AZj90KxxiCWAnQmt\n",
"84HAOEJCf9aMkp8v1TUPL8U4l5srgLtt8GAB5wKbNVdST0otGxGS+d7AZEIy39OMOc0Rg6s8nthd\n",
"i6Y43h4YOPaPfxwCPAUcgtl7Rd9v6Hu+HyGhdyQk835mTC32vl3l81KMa7EUx52BN5f86adD5my/\n",
"/enAs5idVbT9iXbAjoRkvjlhOOtbgKd9wguXKx/d0bl6KI6vBjrZ4MFfA+sBO2JW8GFtJfoQToLu\n",
"C7xDSObjzCj5GZdc6fEau3N1UBwPBHafdPDBZwL/B/QrZFKXWJGQyA8AlgVuA35rRsnPierKnyd2\n",
"1+IojpcAbhz86quXbjhlysXANph9k/d2RVtge0IyHww8CJwAxF5qcc3JE7tric5oN2/eR4+feOLh\n",
"wAmYvZbPxiTWJ5Ra9gM+JJRahpvxfd6ROtcEnthdi6I4Xg+zI9844IBXW8FjmN3RpO2I5QmllhHA\n",
"rwjTz21mxgeFi9a5pvHE7lqMZIz1G4+6775ne0+btiKhTJL7+qINsB0hmW8N/Ac4DXjC5xJ1pcQT\n",
"u2tJjlnx22+XvPLqqzcGNsbsl1xWkliXkMz3J0zOfitwsBnf1b2Wc+nxxO5aBMVxz1YLFpz91MiR\n",
"C1ub7YZZveOWSyxLGD9mBNCNUGoZYsY7xY/Wufx4YncVT3Gs1gsW3HDK2LFz1/7004swezrr88L0\n",
"clsTerVsBzwCjALGm1Gyk1c7V5MndlfxtHDhsFW//HKjs2+//WHg6lrLxdrAcGAYMI1QajnCjLy7\n",
"QDqXBk/srqIpjlda8pdfrr7jvPOmt5837xCSS60llgb2IrTOewJ3ANua8WaK4TpXEJ7YXUVbZebM\n",
"u/eI49aD3nprB2E/o0Wllh2B8cB5wCNeanGVxBO7q1gb3nTT8HYdOmwx8O5PjxZ2AKHU8hXhAqJj\n",
"zfgq3QidKw4fBMxVpHX2fqTH13vM/eB3F77z9fUTzxBwF3CbGZPTjs1VHo3RUkDvjNtayc/5NsoG\n",
"FHRfPgiYa0kkWgERcMCgE6bt2+PlWT/fOvHEw4GHzMipz7pzddEYtQVWp3byXgtYCvgAeA94n3Dx\n",
"2uXJ782uqIld0lTge2ABMM/M+hdzf65lklid0KtlODBrWHTJF48NXN0Oeui5Xz9kS/wv5fBcGdEY\n",
"CehK9uS9KvAZIVm/D7wG/J2QzD+3UVYyA70Vu8VuQGQFGDnPuUwSnYA/EC4gWhe4G9jtzW492ux6\n",
"8HnPbzhlygnn3nSTJ3WXlcaoM9mTd2/gRxYn7/eAp5OfH9ko+zmVgBupOUoxXkd3BZGUWjYjJPPd\n",
"CP9wVwH/MeNnpOWO3+XIKa3MJj5yyilXpRiqKwFJ6aQn2ZP30oTSSVXyfpgw9+0HNsq+TSXgAmqO\n",
"FvvjkhYAfzWzG4q8P1eBJHoQerQMB+YSerWcbsYXGU9q9eQGG9x/4447LvHDkkv+LpVAXbNLSidd\n",
"yJ68VyNccFZV954M3JP8Pq2USieFVuzEPsjMpktaERgv6V3LuJxb0uiM58ZmFhc5HlcmJDoCvye0\n",
"zvsAfyNcUPSKGbW6cv3Utu3ow084oc+8Nm1GWhR9UXO5K64kwbYF2mXcat7PZVmuj3cE1iAk8J9Y\n",
"3PJ+H3gm+TmlXEon9ZEUEToF5Kyoid3Mpic/Z0q6F+hPOHwOJky4z6JoUjFjcOVDQsAgQjLfHXge\n",
"+CvwgBk/1bPizpcMHXrs1C5dXvu5XbsbmyPWYksSZRuKkyQLkVhrLmsLzEtuv2S51fV4fcuqHp+d\n",
"Zdlc4CPg/UoondQnafDGVfcljWponaIldklLAq3NbLakjsA2wJgaTzsN2LtYMbjyINGdUGoZAcwn\n",
"jNWyrhn1jsCYrNzrve7db/3jsGHMb9NmhEVROhdm1EFj1JMwsNgQYCUal3AX0LRE2NCyOU1Yp6HH\n",
"59molC6KcbUUs8W+EnCvpKr93GVmj9V4zpaK494WRan09XTpkehAOAE6AuhHqH3uD7yUrdRSx0ZW\n",
"XggPb3vxxTPnt2lzg0XRR8WKN1cao2UJSXxrYCugE/A4YaTIT2hE8qzkGrArrqIldjP7mFAbrc9f\n",
"gFOAg4sVhysdSallAGGslj2AlwknQnc1Y24jN7YCMH7kMcdM/l+XLqsCVxY43NzCGKN2wG8JiXxr\n",
"4NfAs4RxaP4CvOktWdfcUh1SgAkTViB0OdrQoujTVAJxzUJiJcIIij0Jyfx2Mz5r4saWAZ54vWfP\n",
"Zza8+eZ9gC0til4vWLD17TrUvtdlcSLflHDSbnxye64STti50pXLkAKpjxWjOL4UaGNRdFwqgbii\n",
"k4iAOwm189F5jaQodQIeBV7RE090Q3rbouisQsRZ5y7HqCuhrFJVXvmJxYl8go2yr4u5f+cylUti\n",
"Xxl4C+htUTQzlWBcUSQXFJ0BHAWMMOPRPDfYnjAGx9S248c/PL9Nm3OBPhZFdfeYacpuxqgjsAWL\n",
"E/kqwASSZG6jbEoh9+dcY5TFIGAWRZ8rju8BjgX+L+14XGFIrEhopXcA+pkxLc8NtgX+Aczsdccd\n",
"p8xv0+YNYM9CJHWNUWvgNywur/wGeIWQyA8GJtooW5DvfpxrLqm32AEUx2sALwKrWxR9n0pArmAk\n",
"NiOM3XIn8H95T2IhtU621wHYXRMmXAf8ZFF0VJM3OUZrsDiRDwY+Z3F55SkbZT/kFbNzRVIWLXYA\n",
"i6IpiuNHgSOAi9KOxzVNUno5FRgJHGDGwwXYaCvgRmB5YCdNmLA5IRmv16jNjNFyLO6GuDXhS2I8\n",
"cB9wtI0KF9M5VwlKosUOoDhen/CP1tOiqHFd31zqJFYAbicMrrS3Gfn3cgoXQVwF9AW21YQJBrwO\n",
"HGdR9O/j8awPAAAYUUlEQVR6Vx2jJYCBLE7kaxEuNa9qlb/l3RBdOSqLk6fVHovj+4HHLIr+kkpQ\n",
"rkkkBgFjCeO5nGnGvAJt+ALCFctDMJulOL4E6GZRtE+tp4ZuiOuzuPfKIOAdFifyF7wboqsE5ZjY\n",
"NyEMXL+mRVFhkoMrmqT0chJwInCQGfW2ohu58TOAfYEIs68Ux/0IPWLWtyj6UmPUClibcHHQYEJC\n",
"n8PiRP5EpY8h4lqmskvsAIrjJ4DbLIpuSyEslyOJ5YHbCLXvvcz4pIAbHwkcDWyO2XTFcVtswat8\n",
"9exDvD3qR0IyHwB8Sxgo7ClCN8TUhxRwrtjKNbFvBVwNrGtR5GNllCCJqiOrfxLGRS/cfKLSwQb/\n",
"d+pWHHjJpqwG/JYeB+9M57VW4I2TnwVeICTz522UzSjYfp0rE2XTK6aG/xKG6fwd8K+UY3EZkrFe\n",
"jif0fDnUjPsLst0xWhrof+5/OfSwDuy86UH89N4K3AA8z3K/nUL3vZbgl6/W8guDnMtNybXYARTH\n",
"vwPOAjYutWFYWyqJZQljvHQllF6mNmk7oTa+FqGcUnXrMWwS/7v6IVY7aRvOuKEf99go+0Jx3Ap4\n",
"EviHRZFPdecc5dtiB3gAOI/Qu6HmUL+umUn0J5Re7gf2bEzpRWO0FKEeXpXEBwCzSMopwF+/P58V\n",
"Ov/C7UB0/YM28frFqx9G+Bv1XlLONUJJttgBFMf7AwdbFEXNF5XLlJRejgXOBA4z4956nx9a472p\n",
"3hrvCbzK4kT+go2yzLlKNyOU3HbD7JlFD8dxN+A1ILIoequAL8u5slaWJ08XLY/jNoQhffe3KHq2\n",
"+SJzABLLADcRJgTe04xaPU6S1nh/FifxTajeGn8emGyjLHvXVakf8BCwH2bjFz0cxyIctU20KKo5\n",
"65ZzLVo5l2KwKJqvOL4YOB3YKe14WhKJfoTSy0PAvmb8nFwAlFkb3wRYndCqfh64ATiwWmu8/p2s\n",
"B/wbODgzqSf2IrT0d8//1TjX8pRsix1AcdyeMGHt9hZFk5snspYrKb0cBZzN8u+dyDFrT6d6bXw2\n",
"tVvjje/qKPUiTM57EmZjqy2K4+UJwzjvZlH0fNNfjXOVqaxLMYueF8cnAxtlu4zcFYbGSEzdfCM+\n",
"3PZaVnqzJ7/+19e0+bmqxp3Zbzz/gbKk1QgXFP0Rs5tqLY7j24FvLYpG5r0v5ypQWZdiMlwHfKQ4\n",
"7mVR9EHawVQCjVFnMmvjC1sPYrkpHen18FS6vnoBbX5+mqa2xuvdsboSJna+rI6kvi2wOY0cudE5\n",
"V13JJ3aLotmK42sIk14fknY85Sapjfcm1MSryiprAJMwnufZUz7jxWMXMHuV/c34e/ECCZNPA7dg\n",
"VmviacVxJ8KX+GEWRT4WunN5KPlSDCyqu34AbGBR1LQJkFuIWq3xkNB/oHptfBKjrT1wPWEgrT3M\n",
"KN7RkLQ08ATwGGanZ31KHF8BLGdRNKxocThXASqixr7o+XF8GSCLouOLGFZZSVrjvajeb3xNYBIZ\n",
"idxG2efV1hMbEqaZmwAcZ0bxxr+XOhIuMnsVOJYsf3DJqJ73AutZFPnE0M7Vo9IS+yrAG4RJr78q\n",
"XmSlS2PUidqt8TnUaI3XVRtPer0cQriqd6QZdxc3YLUndGn8hNCtsdagborjdoSkf65F0d+KGo9z\n",
"FaBSTp4CYFE0TXH8T8KVkGc3xz41RhsQ6sLvJ7cPMn6fYqOsaC3dpDW+JtVb471Y3Bq/BTi0Zmu8\n",
"zu2JTsBfCZNRbGrGe8WIO2OHbYF7gK+AQ7Il9cTpwMdQxPq+cy1M2bTYARTHaxK63zXLpNdJcu1K\n",
"OPnYK/lZ9fvqwAyyJ/2pNsoaNYFzHa3xH6ndGm/0LEAS6xNKL88Ax5rxY2O30cgdtgbuAjoCv8ey\n",
"X3mqOF6X0J+9r587cS43FVWKWbReHN8NTLIourgIYeUexxi1AValdsLvDawM/I/aCf994HPAqLs1\n",
"ntlvfFpeMYbSy4HAhcAJZtyRz/Zy3GnV5NOrATti9lPWp8Vxa8IXze0WRdcWPS7nKkSlJvYNgEcJ\n",
"rfaSnPQ6mUh5dRYn/MykvzTwE9lr4wWbk1OiI3AtsBGh18s7hdp2PTsVcCXwG2BbzOrstqg4PhbY\n",
"A9jCJ1RxLnepJnZJ2wFXAK2BG83sosYGV+e24/hB4GGLomvyj7R5Jd0RO+Y8pkpT9iHWJZReXgSO\n",
"NmNOsfZVY8fnAdsBW2L2XZ1Pi+PVgFeATS2K3m2W2JyrEKkldoUa63uECYanAS8D+5jZOxnPySex\n",
"/xYYC/TySa+rkxgBXAKcbMatzbjjM4D9gC0wq7PXUjJy48PA0xZF5zVXeM5VilxyZ6si7bs/8KGZ\n",
"TbVw4uxvwK6F2ngyONTHgI8fk5BYUuIWwhW6UTMn9WMJtfyt6kvqif0JJ6RTPUfiXCUrVmJfBfg0\n",
"4/5nyWOFdD5wWjJ9WoslsZLEnsBLhLJXfzOab2IK6UDgREL5pd5BwhTHvwIuBQ7yIy3niqdY/dhz\n",
"qu9IGp1xNzazuBH7eJxwAnJXqH9mn0oi8StgCyACBhNav08TvujGmuX23hcomL2Bc4DBmP2v3qeG\n",
"EsyfgTssiiY2R3jOVQJJEeH/Pfd1ilRj3wQYbWbbJfdPBxZmnkDNp8a+aBtxvBtwBtC/Uie9lliR\n",
"xYk8AroREnmc3CaZsSCFwHYhjDWzNWZvNPj0ON6P8Fn9xqIoaxdI51zD0rzydCLQS1IPQr/tvShO\n",
"Pfx+wuXxWxGuEC17EitQPZF3J/T3joERhETeqIufCk7aitBXfYcck3o34HLChCme1J0rsqIkdjOb\n",
"L+loQn/z1sBNmT1iCrafKFqoOL6Q0BIsy8SeJPLNWZzIV2NxIj8QeC31RJ5J2pTQI+n3mDVYUknO\n",
"gdwCXG1R9Eqxw3POleEFSrW2E8dtCVd37mtR9Fz+kRWXxHJUb5H3AJ5lcWnl1ZJK5Jmk3xC6Ku6P\n",
"2WM5rRLHRwHDgEEWRaX5upwrIxV55WnWbcXxkYTD/J0Lsb1CShJ5Zot8deA5wpC5MSGRl34PkTD5\n",
"9OPAYZjdn9Mqcdyb8FoHWRQVd9Ax51qIlpTYOxAmvd7Wouj1QmyzybGIZameyNcgDBlQlchfKYtE\n",
"nqmeyafrXCWO2xCORO6wKPpzEaNzrkVpMYkdQHF8CuHCqD2bc+wRiWWonsh7UT2RTyy7RJ5JWpUw\n",
"+fS5mN2Y82pxfBah5LStjwXjXOG0tMTemVAqmE24AKbeftVN3o9YmuqJvDdhRMbMRF7YSaDTEiaf\n",
"fgr4C2ZX5LxaHG8EPELo2vhpQ893zuWuRSV2WHT4fwpwPGECh5vy7d+eJPJNCRcDRcBahMG1qhL5\n",
"yxWTyDNJywNPAmMxy3lMF8Vxe8IAX+dbFN1VrPCca6laXGJftO04Xh+4DfgCOMSiKOdxzSWWonoi\n",
"/zUhkceEZP6yGQUbXrckhcmn/0s4Ajo92zylda4ax5cSumzuWakXjTmXphab2GFRN8gzgSMJY5nc\n",
"mS3RSHSmeiJfhzDuSkxI5C9VfCLPFCaffpQw6ccxjUzqWxD6uG/QUuelda7YWnRiX7SfUO+9DZgC\n",
"HM7gaA4wiMWJfF3CsMIxixN5y7w6Mkw+/QBhqOWD6pmntPaqcbwUMBk4xqLo30WK0LkWzxM7hEmc\n",
"l/0l4rj3z6DPd/24stdCnljpBRYn8hdbbCLPFCaf/idhdqd9MWvU+DOK45uAhRZFhxQjPOdc0CIT\n",
"u0QnYCCLRz9cn3AyL+agj75g30+OoxWTgSO9XJAIE6PcCXQmDBXQqJPBiuNdCLNlbWhRNLsIETrn\n",
"Ei0qsUv0B/4E9AVeZXGL/AUzFs2NmlzMdA5htp8jLYpazJC/WYXJp68HegI7YdaoeWQVxysSSjB7\n",
"WRQ9XYQInXMZWkRilxBwLOFE6fHAvWb82OB6cbwpcCuhD/oxFkXf5htL2QmTT18BbAxsU9/k01lX\n",
"D2OsjwM+tCg6pQgROudqSHNqvGaRXPX5T2AosIkZd+WS1AEsip4BNgS+Bd5QHO9YvEhL1jnAZoTh\n",
"dxuV1BNDCVfanl3QqJxzeSnbFrtEP+DvwEPASfl0SVQcDwZuJpRujrcomtXUbZWNMPnJUMLk0zMb\n",
"vXocr0o4d7G1RdGkQofnnMuuIlvsEpI4mpDQTzXjmHz7mVsUTQA2AOYBryuOty5AqKVLOgY4iDD5\n",
"dFOSetUY65d5Uneu9JRViz25vP8GYE1gTzM+LHhccbwtYXagfwMnWxQ1pURRuqQDgNGElvrUJm0i\n",
"jo8F9gY29zHWnWteFdVil9iIcOj/FTCwGEkdwKLoUUIXyfaE1vsWxdhPKqS9gHMJ85RObdIm4nht\n",
"4P+AYZ7UnStNJd9iT3q9HEY40Xe0GX8venBV+47jnYC/Ek7Qnm5RlNOJ2ZIk7Uw42slp8umsmwjD\n",
"NDxHGFztukKG55zLTdm32JNxXO4GjgAGNWdSB0gujd8AWAGYpDge2Jz7LxhpS+AmYOemJvXEGcDX\n",
"hC8751yJKtkWu8SGwD8IPVWOy7zIKA2K492BPwN3AGdbFJXHMATSIOBe4A+YPdXkzcRxP+A/wEaN\n",
"GS3TOVdYZdliT3q9HEIYMnaMGYelndQBLIrGEVrvqwOvKo43TjmkhkkbEZL60DyTegfCF9pIT+rO\n",
"lb6SarEn47xcR7hwaA8z3k0luHokV1vuBVxJqFn/0aKo9CbakNYljKl+OGb35bWpOL4c6GJRtE9B\n",
"YnPONVlZtdgl1gcmEkYXHFCKSR3Aosgsiv4G9CG04F9WHPdJOazqpDWBx4ATCpDUhwB7AEcVIjTn\n",
"XPGl3mJPer0cAFwEnGjG7akE1ARJ630YcClwNXCBRVG6E1eHyaefBM7H7Ia8NhXHSwOvA4dZFD1S\n",
"iPCcc/kp+UHAwDoB1wD9CKWXt1MJJk+K426Ei5pWAIZbFL2VTiDqQph8+prGTD5d5+bi+FZgrkXR\n",
"EfluyzlXGOVQinkZMKB/uSZ1AIuiz4DtCd0AY8XxqYrj1s0aRJh8ejxwe4GS+jDClIEn57st51zz\n",
"SrvFfoAZt6YSQJEojnsQBhTrAIywKHqv+DvVUoQTpU8ApzVmntKsm4vjQwkjNm6b2tGHcy6rki/F\n",
"NMfUeGlIBsk6AhgDnAdcaVGU8/yhjduZOgKPEGrhRxcgqZ9MiH1ri6IpBYjQOVdAqSV2SaOBg4Gq\n",
"kQNPN7NHajynYhN7FcXxmoRREA04oOCJUloCeBD4HDiwMZNP19pUOBF8DrA7Ial/VpggnXOFlGZi\n",
"HwXMNrPL8gmuEiS19mMJl+OPAq4rSOs9TD79D+AXwuTTTR6QKznCuIJQU9/WoqjRQ/k655pH2idP\n",
"Kz5p58KiaIFF0eWEmYqGAeMVx6vltdEw+fRtQBtg/zyTehvCUUVfYLAndefKXzET+zGSJku6SdIy\n",
"RdxPWbAoepfQIh4PTFQcH5yUPxonzFN6HdAF2AOzJl/1qjheArgHWInQUq/8maOcawGaXIqRNJ6Q\n",
"XGo6kzBBdFXL7xygq5kdVGP9FlGKyUZxvB6hxf0lcEjO9eyQ1C8H+tOEyadrxNCRMI7MLGC/khwW\n",
"wTlXS0n0ipHUA3jQzNav8bgReo1Uic0sLmowTSEp354mWTcbxjY/HTgaOAm4w6Ko/v1I5wA7AYMx\n",
"+y6PfS9DGKnxPeBQnzDDudIlKQKijIdGpXXytKuZTU9+Px7Y2Mz2rfGc0m+xS2sDdwEj8hzHvO5d\n",
"xHFfQuv9Y8Kl+1/UEcupwHCaOPl0xv5+BTxKGHbghKJ1w3TOFUWaJ08vkvS6pMnAFsDxRdpPsb1H\n",
"GAPmCaQTkQr+flkUvUYYUuENYLLieO9atXfpKOBQwuxH+ST17oQhBx4Ajvek7lxl8guUciH1BG4H\n",
"5hNa7/8rym7CGO+3AW8BR1oUzUQaAfwR2Lyp85Qm2+5FOHF7lUVRnd1QnXOlLe3ujpXD7GNCjesR\n",
"YCLS0OREZmF3E0UvAxsRyjKvHz1y5CXA+eQx+TSA4nh9IAbO9aTuXOXzFntjSX2AO4F3CJNYfF2M\n",
"3YwZPvyk27fZ5sI5HTo8MmO55YZZFH3TlO0ojgcQSi/HWhQ165yxzrnCK4leMXXuuFwTO4DUnjAG\n",
"zN7AQdQYLqEA2x8C/O3tVVfdfd3bbvsD4TL/wyyK/tOozcTxYODvhOEMGrWuc640eWIvNmkwcCvw\n",
"b+AUzOYUYJsDgfvImHxacbwF4erQmHDSs8ELiRTHuxDGiN/DoujJvONyzpUEr7EXm9kEwvysnYFX\n",
"kfrntb06Jp9OEvMGwM/AG4rjberdTBzvC1wP7OhJ3bmWx1vshSLtAfyZMCPU+Zg1boo8aR3CmOpH\n",
"YnZvnU+L460JLfGHgZMtimbXWH44cBY+lrpzFclb7M3J7B+EgbQGAs8g9c55XWkNwuTTJ9eX1AEs\n",
"isYTWu9tCf3eo0WbieNTgVOAzT2pO9dyeYu90EI3yCMJwyWcDVxb75AEUtVFQxdgdn2jdhXHOxKm\n",
"4/sX8COwC2Es9WlNC945V+r85GmapLWAO4CvCZNgTM/ynKrJp6/F7PIm7SaOlwWuAtYEdrYo+qrJ\n",
"MTvnSp4n9rSFyTDOBA4HjsJsXMay5Qi9XP6B2TmpxOecKzue2EuFNIDQen+eMJuSAY8TEvupxRg9\n",
"0jlXmTyxl5Iw6fSlwPbAF8CrhFa8J3XnXM48sZciaQdgE2B0PpNPO+daJk/szjlXYbwfu3POtUCe\n",
"2J1zrsJ4YnfOuQrjid055yqMJ3bnnKswntidc67CeGJ3zrkK44ndOecqjCd255yrMJ7YnXOuwnhi\n",
"d865CuOJ3TnnKownduecqzCe2J1zrsI0ObFL2kPSW5IWSNqoxrLTJX0g6V1J2+QfpnPOuVzl02J/\n",
"A9iNMBnzIpLWAfYC1gG2A66RVLZHBpKitGPIhcdZWB5nYZVDnOUQY66anHDN7F0zez/Lol2BsWY2\n",
"z8ymAh8C/Zu6nxIQpR1AjqK0A8hRlHYAOYrSDiBHUdoB5ChKO4AcRGkHUCjFaEmvDHyWcf8zYJUi\n",
"7Mc551wWbepbKGk80CXLojPM7MFG7McnbHbOuWaS95ynkiYAJ5rZq8n90wDM7MLk/iPAKDN7scZ6\n",
"nuydc64JGprztN4WeyNk7uQB4G5JlxFKML2AlxobmHPOuabJp7vjbpI+BTYB/iPpYQAzexu4B3gb\n",
"eBg40vI9LHDOOZezvEsxzjnnSkuq/cslnSNpsqRJkv4rqXua8dRF0iWS3kli/ZekpdOOKZv6Lhor\n",
"BZK2Sy5a+0DSqWnHk42kmyXNkPRG2rHURVJ3SROSz/pNScemHVM2ktpLejH5/35b0gVpx1QfSa0l\n",
"vSapMR1DmpWkqZJeT+KsVeKukvaFQxeb2YZm1ge4DxiVcjx1eQxY18w2BN4HTk85nrpkvWisFEhq\n",
"DfyZcNHaOsA+kn6dblRZ3UKIsZTNA443s3UJpdCjSvG9NLOfgMHJ//cGwGBJm6YcVn1GEkrIpVzG\n",
"MCAys75mVuf1QakmdjObnXG3E/BVWrHUx8zGm9nC5O6LQLc046lLPReNlYL+wIdmNtXM5gF/I1zM\n",
"VlLM7Gng27TjqI+ZfWFmk5LffwDeIVw/UnLM7Mfk13ZAa+CbFMOpk6RuwA7AjVTvDFKKGowv7RY7\n",
"ks6T9AkwHLgw7XhycCDwUNpBlKFVgE8z7vuFawUgqQfQl9DgKDmSWkmaBMwAJiSdK0rR5cDJwMKG\n",
"npgyAx6XNFHSIXU9qVDdHevU0EVOZnYmcGbS//1y4IBix5RNLhdjSToT+MXM7m7W4DIU8KKx5lbK\n",
"h7dlSVIn4J/AyKTlXnKSI90+yXmpRyVFZhanHFY1knYCvjSz18pgvJhBZjZd0orAeEnvJkeZ1RQ9\n",
"sZvZ1jk+9W5SbAk3FKekEYRDtS2bJaA6NOL9LDXTgMyT492pPvSEawRJbYFxwJ1mdl/a8TTEzGZJ\n",
"+g/QD4hTDqemgcAuknYA2gNLSbrdzIalHFctZjY9+TlT0r2EEmetxJ52r5heGXd3BV5LK5b6SNqO\n",
"cJi2a3JCqByUWp1wItBLUg9J7QgjgD6QckxlSZKAm4C3zeyKtOOpi6QVJC2T/N4B2JoS/B83szPM\n",
"rLuZ9QT2Bp4oxaQuaUlJnZPfOwLbEDpM1JJ2jf0CSW8kNbgIODHleOpyNeHk7vikm9E1aQeUTV0X\n",
"jZUCM5sPHA08Suh58HczeyfdqGqTNBZ4Dugt6VNJqZQGGzAI2J/Qy+S15FaKPXm6Ak8k/98vAg+a\n",
"2X9TjikXpVo2XAl4OuP9/LeZPZbtiX6BknPOVZi0W+zOOecKzBO7c85VGE/szjlXYTyxO+dchfHE\n",
"7pxzFcYTu3POVRhP7M45V2E8sTvnXIX5f4oqt2/zLhAlAAAAAElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x7f7e86dcfe48>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"## Read test data here\n",
"a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
"xt= a[:,0]\n",
"yt= a[:,1]\n",
"X = construct_poly(x,8)\n",
"w = wRidge(X,y,1.0)\n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" X = construct_poly(x,n) \n",
" Xt = construct_poly(xt,n) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(x,X.dot(w))\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE train: %f test %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n",
" plt.plot(xt,yt)\n",
" plt.plot(x,y)\n",
"# was ist mit fehler kurven gemeint?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1.75123654094\n",
"0.781528050024\n",
"11.0428184273\n"
]
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4VNXWx/HvCoSODWmiAtarSBPFipwICNgQFUVF7AXx\n",
"qtjbfYe5VsDeexfLtV0pClzlIDYQBUERxQKCIkhTiggk6/1jnySTMEkmmZrJ+jxPniQzZ87Zw6O/\n",
"7Nln77VFVTHGGJM9ctLdAGOMMYllwW6MMVnGgt0YY7KMBbsxxmQZC3ZjjMkyFuzGGJNlLNiNMSbL\n",
"WLCbpBKRBSLyt4g0KfX4TBEpEJGdg993FJHXReR3EVktInNE5IzguTbBsWtKfQ0o45onicjHIrJO\n",
"RCZHef4xEZknIvmF1yin/XVF5CkR+UNElojIsDKOGxy08ZyIx84QkRnBaxeJyAgRqRXx/I4iMkZE\n",
"VgTnvr/weRHJFZHXROSn4LzdS12vdnD8kuD1b4vIDuW9F1NzWLCbZFPgR+CUwgdEpD1QP3iu0PPA\n",
"QmBnYDvgdGBpqXNtraqNI77+U8Y1VwB3AbeX8fws4CLgi1JtiGY4sGvQrjzgahHpHXmAiGwLXA98\n",
"Vep89YFLgSbAAUAP4MqI5+8DlgMtgU5A96BdhT4ABgG/RWnnRUA3oAOwA7AKuL+C92JqCAt2kwov\n",
"AIMjfj8DeA6QiMf2A55R1b9UtUBVZ6nqu1W5mKq+p6qvAUvKeP4hVX0f2BDD6QYDN6nqH6o6D3gM\n",
"OLPUMbcB9+L+oERe5xFV/UhVN6vqr8CLwCERh7QDXlHVjaq6FHg3eAxV3aSq96nqR0B+lHa1Ayao\n",
"6u+q+jfwauFrjbFgN6nwKbCViPwjGGo4GRf2pY95SEROLhyeiULKeDwpgp54S+DLiIdnExGgItIV\n",
"2Bd4JIZTdsf16gtNAE4Vkfoi0groC7wTY/MmAn1FpKWINABOA8bH+FqT5SzYTao8j+v99gLmAr+U\n",
"en4AMBX4F/BjMAa/X6ljlovIqoivPZPc5kbB9z8iHvsTaAwQ/JF6ELhYKyi6JCJn4/4A3BHx8HBg\n",
"n+Cci4DPVPW/sTRMVV8HZuL+Hf8A9gRuiuW1JvtZsJtUUFywn0b0YRhUdbWqXqeq+wDNcePgb5U6\n",
"TxNV3Tbi69skt3tt8H2riMe2BtYEP18EzFbV6RHPb/GpQkSOA24F+qrqyuAxwfXY/wM0ALYHthOR\n",
"EbE0TETuwP2B2Q5oCLxJ7L19k+Us2E1KqOrPuJuofYE3Kjh2BXAnsEMwHFLly8bxWlR1FW6cvlPE\n",
"wx0pHk45HOgfzExZAhwM3Cki9xUeLCJ9cOPyR6vq1xHn2R7oAjwQjKevBJ4BjoyxeX2Ap4M/iBuB\n",
"B4CuIrJdZd+nyT4W7CaVzgEOV9W/Sj8RTAVsF0zjawwMAeYH4Vp0WCwXEZEcEakH5AI5wZTF3Ijn\n",
"c4Pnc4A6IlIv6EFH8xxwo4hsIyJ7AefiAhjcTdR/4MK+EzADN7xyQ3Cdw3E3TI9X1Rmlzrsc90dj\n",
"iIjUEpFtcJ9misbzg3bXC36N/BncWP8ZIrJV8N4uAn4p/ERgajYLdpMyqvqjqn4R+VDEz/Vxwwmr\n",
"gB+AnYBjS51idal57JeVcanBwHrgIdyUwL+ARyOenxQ8fyCuN70+OA4ROU1EIm9whoL2LAQmAyNU\n",
"dWLwfv5Q1WXB11JgI/CnqhYO1dyIGy55J6LN44LXKnA8cAwu5OcDfwOR8+S/Ddq2A27YZl3EjeVh\n",
"QEHQtmW4Hnz/Mv49TA0j8Wy0ISI74Xo0zXD/kz6mqvcFHwdfAVoDC4CTVHV1/M01xhhTkXiDvQXQ\n",
"QlVniUgj4HPgOOAsYLmqjhSRa4BtVfXahLTYGGNMueIailHV31R1VvDzWuAboBXuI/SzwWHP4sLe\n",
"GGNMCiRsjF1E2gCdgWlA82DMEdyy8OaJuo4xxpjyJSTYg2GY14FLI24cAUU3iWzHbGOMSZHa8Z4g\n",
"mGr1OvC8qhYuKFkqIi1U9TcRaYm7a1/6dRb2xhhTBapa7tTfuII9mPv7JDBXVe+JeOpt3JzcEcH3\n",
"0isIY2pcJhOR4ao6PN3tqCprf3pZ+9OnOrcdYusUx9tjPwRXVnS2iMwMHrsOVy711aA29QLgpDiv\n",
"Y4wxJkZxBbuqfkjZ4/Q94zm3McaYqrGVp1Xnp7sBcfLT3YA4+eluQJz8dDcgTn66GxAHP90NSLa4\n",
"FijFdWERrc5j7MYYkw6xZKf12I0xJstYsBtjTJaxYDfGmCxjwW6MMVnGgt0YY7KMBbsxxmQZC3Zj\n",
"jMkyFuzGGJNlLNiNMSbLWLAbY0yWsWA3xpgsY8FujDFZxoLdGGOyjAW7McZkGQt2Y4zJMhbsxhiT\n",
"QOL726W9DbbRhjHGxE98f3vgDmAfYH/1vKSEq220YYwxSSa+nyO+fzbwNbAKyEtWqMcqrs2sjTGm\n",
"JhPfbwc8DNQH+qrnfZHmJgEW7MYYU2ni+w2AG4HzgOHAI+p5+WltVAQLdmOMqQTx/T7Ag8BnQEf1\n",
"vF/T3KQtWLAbY0wMxPd3AO4G9gOGque9m+YmlclunhpjTDnE92uJ718MfAnMB/bJ5FAH67EbY0yZ\n",
"xPf3BR4F1gPd1fPmprlJMbFgN8aYUsT3GwM3AacA1wLPpHsKY2VYsBtjTEB8X4D+wL3AJKCdet7y\n",
"9Laq8uIOdhF5CjgKWKaq7YPHhgPnAr8Hh12nqhk9JmWMqdnE99sA9wO7AoPU86akt0VVl4ibp08D\n",
"fUo9psBdqto5+LJQN8ZkJPH9XPH9q4EZwKdAp+oc6pCAHruqThWRNlGesjowxpiMJr5/MO7m6C/A\n",
"Aep5P6S5SQmRzDH2f4rIYNxfwStUdXUSr2WMMTELKjDejhtGvhx4tTrdHK1IsuaxPwy0BToBS4A7\n",
"k3QdY0w2EdkakX8ismtSTu/7Ir5/Oq5g10Zgb/W8V7Ip1CFJPXZVXVb4s4g8AYyJdlxwk7WQr6p+\n",
"MtpjjKkGRHoATwFzgBAiE4HbUJ2TkNP7/h64Tud2QD/1vOmJOG+yiYgHeJV6TSLqsQdj7GMiZsW0\n",
"VNUlwc/DgP1V9dRSr7F67MYYEGkIjAD6AeeiOgGRrYAhwGW44dxbUf2kSqf3/Xq4uegXAzcDD6jn\n",
"bU5I29MgluyMO9hF5CWgO7A9sBQI4f66dMLNjvkJuEBVl1a2ccaYLCdyMPAMbjbKpaiuEiFHlYLg\n",
"+frAWcBVwELgVmASMQaX+H4PXC99DnCpet7ihL+HFEtJsFeVBbsxNZhIXSAMnAEMFfQtoCdwIe6G\n",
"5gTgIWCiKgWI5AIDcT3vv4DbgDdRLYh6et9vjru3dyhwsXre2CS/o5SxHZSMMZlHpDNueGXPEVx9\n",
"uKC74YprjcQFemvgbVzvfL4IVwu6NarPA+1xS/2vBuYicmYQ+u7Ubjej83E99F9xK0ezJtRjZT12\n",
"Y0xqiNQGrlW4ZBxHPXYsb++i5BwJvAk8AkxXRYsPR4CuuLH243CTMB4CPlW3TOZw4DpgD2DUns89\n",
"N+27nXa6J3j5hep5s1P11lLJhmKMMZlB5B+bqP3iQlo37MWknAW0zceF+XOqrJKwCG4T6L7AwcAy\n",
"XC/efY3+7yq+O/YU3FDNOlzAj1Zl7ZxddjnsjpNPfmL8AQfsdpLv//fa0aPP3mnZslVpeZ8pYMFu\n",
"jEmr/8iAnJ35+Y69+GbIDdyiDzL0v0rOI8AHDJdGuHH1vsHXZmA88AFuMsZuwO7BV2tgGcp8lv/j\n",
"L747ujWLD2zD/tt9ziDZg1zee+a22x4/Y+LEc3Fj9I8B91Bq0kY2sGA3xqSFCI16MfGfN3Pj9UDu\n",
"NYy418/d/05uaNwUOBIX5PvjZsOMB94BvtVQ9ECSsNQGdqYw6Bvu0pldLjmGus2b8O3dOfz6898U\n",
"1J5Po98+7rZo87IHxrNvu985pEAYnVvASFQXpOJ9p4IFuzEmpUToIBRceCGPnHEb19WelrvP6OOu\n",
"mDbmr3r5R+ACvQAX4uOByRrStZU6v+/Xxs1HvxF4ALid8GRh2T7nkrP5HBr/ugu7TPqW1h+ubbVh\n",
"XetLprHzeZ8jH+7M0rsOwvfb8hnFQzw/aUg3JvL9p4IFuzEm6USoDwwALtyRhW3ernP4xu3rLKxz\n",
"4sD8hdN3ZB9gGsVhPq+sXnmF1/H9rrhx+VXAEPW876K0ZS/cOPwg4CPqrXq039Etf7ti+t+X7LuE\n",
"/l814/cbDmfpe7vSHNgRWAx8T+R4vvv9Jw3ppqq0M9ks2I0xSSPCnsAF1FkzWPb6z8KhjW+s9X9f\n",
"LGn/ZGfWhD1e3ZDLOOB9DemauK7j+1sDtwAn4BYqvVhRbRcRGuJ2P7oI2Bp49GRefullTjkuOMe3\n",
"q+oxqsWVLNhYu2gcP3JMvxWwiJJhX/jzAg1p2lauWrAbYxJKhDpI/nG0mHU5u0xqR6fnlm9f/5um\n",
"o99gXcff0Df34p8XHsNrVe2Vl7iW283oJOAuYCxwrXpepWa7lJoy2Q8Y24Ilj/9Cq11z0GuA1bjF\n",
"TmMiFztJWOoCbSgO+sjgb0nJ0I8M/oXJDn0LdmNMQsgu7+9Ni1nDaTr3aHYfL9T9Yx21/37zgXfy\n",
"lw35jLNz4EXgRlQ3JOR6vr8L8CBuuOQC9byP4z6n0AQ4k2DKZF02PLKQ1muas+xyoC6ujO/LaPnB\n",
"HIT+LpTs4RcGfwvgIw1pj3jbW/b7sGA3xlSBhEVYt/1e/NplGPVXHkfTuU34c6dfEH2Z7b99dtVt\n",
"LNrmb+4BDgPORHVqla/l+3WB5riecAtgP1wPeyRwt3peQse6RcgBeuCGaboLBaPfpP+X/Xj7NNzM\n",
"m1HA01X5IyVhqQc005D+nMg2l7iGBbsxJlYSlgZAHuu3OxHRfmyutxULD13Bmh1fpt6qW/Stp105\n",
"7uLyuuOBq9AtZ7aI7+cATXBBHfnVMspjjXAFBH/D7d+wELhTPW9BMt+veyvsBJyH26N5/lWMfO82\n",
"rutai4IuwN3AI6j+mex2VIYFuzGmXBKW3YEjUfqitQ5j2T7r+HpAQxYfMIaFh92m+XVmFR8sDVc0\n",
"bnzn8q23Pva53r1H3Tpo0M+UHdbNgDW4oP4tylfk46vU86IW80oVEXJxZQuGAHsfxdgxLzCo6Tb8\n",
"cQhuJs69qC5PZxsLWbAbY0oIeuUehas9C3Ia8svBi/h2SBvWHbqaDgXv0n/xPLbevA0RQV1/w4Y2\n",
"Oao7bqxduyC/Vq3FBTk5v1J2UP8GLFPP+zsNbzFukVMmOzJr5mhO/XsvvjlI4DngTlQXpbd9FuzG\n",
"1EjBjJKtgBas+GQ/1i/qScHGA8ipvSv1dlxFrV02srnlNuTWacRWm6BAVlFbFyElg7rR+vXL773/\n",
"/iMP+OabXgKXt3vmmQqnGmaLYMrkQGDoTvy83WOcv+AIJnbMQd8ARqC6xTz61LTLgt2YrCK+X4ct\n",
"hz1Kjl+rtgRtiW6Cv1cKG1cquvkXChp+z9z2+XzeqgMr6uZTp+BlOq9+kqOW/BB1RyGRTsDzuKl8\n",
"FxCx5WVNEjllsgnLj7uV6xecyTM757Lpf+K27puZ2vZYsBuT8YIbjdsR243GxrjKhyWHPtYt3MyK\n",
"j3bkj9ntWL+oPZtWzyZ//Tjya4/nlvV1KMi9EDgetwL0YWBqZInckg1y5XWBS4ArgBdi3bEo2xVO\n",
"mWzMn0Mu4576VzGqQT02fJbL5pvimRlUuTZYsBuTNuL7Dag4qFvgpvqtoeKbjL8BK9XzCoJpdd1x\n",
"Y+VH4gL/XVxwT2K4bgJOxY0VbwM8CjytSvm9bpF/4MaSVwNno1rtt5JLhsIpkw1Yd/Fgnus5nOGb\n",
"GrD+x8as/RcwPpl/CC3YjUkw8f1aQFPKD+rCx3OpOKh/A5bGcqNRwrILxSVuDwNm44L8HWCWhrRA\n",
"hA64MB8ITMHN6JhUtIdomSeXHFwP/UbgX7hpftZLj4EIO+Wy8fyTeWXo//HvOg1Yv/BxzttnuA5P\n",
"yr+fBbsxVSC+3xE3cyRaT7sJsJLyg7rw8T/judEYrHA8jOIyt9tSHOSTNKQroWQRLtwCm8eBJ1WJ\n",
"rbct0hZ4GqiNW2z0fVXbXJOJkJtD/nEH8mnvj/SQc5N3HQt2YypFfL8LbkjjFdyemaUD/PeoNxoT\n",
"df2wtKF4eKU78BXF9cpnaiiinklhES4YDHyG652PUyW29okIbmHOrbhVnnehmp+gt2KSJJbsrJ2q\n",
"xhiT6cT3d8Ptq3m+et6bKbmm65V3ozjMm+D+sLwInKkhXVHieKEObiHNhUA73ArQrqr8WLkLyw7A\n",
"E7jxfQ/Vr+N7JyaTWI/dGEB8vznwEXCHet4jSb1WWFpTPFbuAXMprlf+RWSvvOg1Qlvc0vezg+Mf\n",
"Ad5SpXIbRbhe+kDgHtzsmFvQzKw7bqKzHrsxMRDfb4wL1ReTEeoSljrAoRSPlTcFJuCGe87RUPSl\n",
"6iLUDl5zIW4e9fOAp8q8qjVEtseFeTvgKFRnVOk8JuNZsJsaLVjw8zrwOTA8YecNy84U98rzgG9w\n",
"vfKzgBnReuVFrxVaAefgeuiLcb3zE1T5q+oNkmOD84wGTk9UeV2TmWwoxtRYwcKg53BzwE+I56Zo\n",
"0Cs/hOJeeQtcr3w8MFFD+nu5r3fzonvieud5wMvAo6rMKu91FTdMtobElNc1mcGGYowp30igLdCr\n",
"KqEuYdmR4pueecB3uCA/B9crr3CGiQhNcb34C3CLlB4GzlAlru3kgpNHltftGK28rslOFuymRhLf\n",
"vwIXyIeq562P6TVhycX1yguHWHYAJuKGcs6vqFdedB5Xe6Qbrnd+JPAmbpXo9DKX+VeGSENgBG4r\n",
"uHNRnRD3OU21YsFuahzx/VOBy4BD1PNWlntsWFpRHOQ9cPtavoPrYU+PpVdedC5hW+B0XKCDG/Me\n",
"qkql9vGs4CIHAc8C04AOqCbu3KbaiHuMXUSeAo4Clqlq++Cx7XB3/FsDC4CTVHV1qdfZGLtJOfH9\n",
"XsALQA/1vK+iHhOWvXGLfvri9tyciAvzCRrSpZW6nuud748L89iKcFWFSF3czd+zgItQfSNh5zYZ\n",
"JSUrT0WkG7AWeC4i2EcCy1V1pIhcA2yrqtdWtnHGJFLEqtLj1fOi3kSUsHQD3gAeA8bheuWVH38X\n",
"GgOnUNkiXFVh5XVrlJSVFBCRNsCYiGCfB3RX1aUi0gLwVfUflW2cMYkivr8rMBUYWtaqUglLHvAq\n",
"cIqG9H9Vuo7QETdMU7kiXFW7mJXXrYHSOSumuWrRR9aluGXLxqRFsKp0AvDvckL9CNwy/gEaUr9S\n",
"549ehKtDzEW4qqJked0u6d6uzWSWpN88VVUVkai9CBEZHvGrr1q5/6GMqUiwqnQc5awqlbAcCTwD\n",
"9NeQfhjzubcswjWCyhThqgorr1vjiIiHKz0Rs2QF+1IRaaGqv4lIS4g+rqiqw5N0fWMKV5W+Bsyk\n",
"jFWlEpZjccWwjtWQflrhOUsW4dobN098f1V+SlCzy7t4ZHndA628bs0QdHj9wt9FJFTRa3KS1Ja3\n",
"gTOCn88A3krSdYyJKlhV+hSwARgSrS66hOUE3LDJkRWFughtRbgV+BkX6o8AO6tyfdJDXUQQOQ+Y\n",
"jvv00d1C3ZQn7h67iLyEqxu9vYgsAv4PuB14VUTOIZjuGO91jKmkEZSzqlTCcjJwL9BHQ9E3Iw6K\n",
"cB2FC/L9ibcIV1VYeV1TBVYrxmQd8f3LcRtIHBptAZKEZRAwCuitIZ29xfNCC+B8ShbhejWuIlyV\n",
"ZeV1TRmsVoypccT3TwGGUcaqUgnLWcDNQA8N6dwtnnd7hr4DjAWOibsIV1VYeV0Tp2SNsRuTcsGq\n",
"0nuAvup5P2/xfFjOB/4NHF5GqB8CTAKGqXJBmkL9GNwm1Qtx0xgt1E2lWY/dZAXx/X1xtcZPiFYq\n",
"QMIyFLgayNPQljceRTgSV2NlkCqpL5pVXF63OzAQ1Q9S3gaTNazHbqq9YFXpWOAC9bwtAlHCMgy4\n",
"EvDKCPVTcdMIj0lTqPfA9dI34Ap3WaibuFiP3VRr4vvNcPVf/q2et0XhKwnL1bgbod01pFsOzwhD\n",
"geuAHqpELQqWNK687u24efFWXtckjAW7qbbE9xvhNpF4KdqqUgnLDbhVod01pL+UeM5VXfwXroxu\n",
"t5QsMCrZACuva5LGgt1USxF7lc4ESqzEk7AIbqXpANzwy5ISz7tt6Aq3iztUlUqV4o2Lldc1KWDB\n",
"bqqdYFXpk8DflFpVGoT6LcAxuBulJUJbhFzcePrOuMVGJfYJSG7Di8rr/oDbqi51f1BMjWLBbqqj\n",
"24FdgZ6Rq0qDUB+F2+koT0O6PPJFIjTAleVVoHfKFhyVLK97JfC8Fe4yyWTBbqoV8f1hwNFAt8i9\n",
"SoNQvwe3J2kPDWmJxUkibAOMwZW4OFuV1KzitPK6Jg1suqOpNoJVpZcDfdTzVhQ9HpYc4EHgAKBn\n",
"lFBvgdv04nPgjJSEukgOIpcBH+KGfnpbqJtUsR67qRbE93viinb1iFxVGoT6Y8BewBEa0j9LvE5o\n",
"i1tN+ixwc0L3GS2zsVZe16SX9dhNxhPf74xbVXqiet6cosfDUgsXoLvjCnqVDvX2uO3w7lLlpqSH\n",
"upXXNRnCeuwmo4nv74JbVXph5KpSCUttXC+8Oa6e+roSrxMOwu0DcJkqLyW/oVZe12QO67GbjBWs\n",
"Kp0A3By5qlTCkovrwTcBjokS6r2B/+LG05Mb6q6XfgpuPv103NCLhbpJK+uxm4wUrCodB7ysnvdw\n",
"0eNhqQO8AuQCx2lIN5R4nXAycB9wnCofJ7eRsj3wELAPVl7XZBDrsZuME7FX6Ze4Hbnc42Gpi1tt\n",
"CnB8lFAfAtwJ9EpBqBeW1/0ZK69rMoz12E1GiVhVugk3rq4AEpb6wBvAGuA0DRXvJhTUfbkBt0z/\n",
"MFV+TF4DrbyuyXzWYzeZ5jbcqtKTC1eVSlga4BYXrQJOLRXqOcBduH11D01yqBeW1/0bK69rMpj1\n",
"2E3GEN+/DDgWt1fpegAJSyPcrJifgbM0pPlFx7u6L0/g/hB0VyU5FRJLltc9D9V3k3IdYxLEeuwm\n",
"I4jvDwSuAHoXriqVsGyFq7X+PVuGen3ceHtT4IgkhvpBuBkv2+J66RbqJuNZsJu0E9/vgVtVemTh\n",
"qlIJyza4qY5zgPNLhfrWuMBfA/RTZf2WZ423UVIXkduAN4HrUB1kNdNNdWHBbtIqWFX6EnBS4apS\n",
"Cct2uDIA04GLNKQFRccLzQEfN9Z9elLqvrjyup/hyhR0RPX1Cl5hTEaxYDdpE7GqdIh63hQACcv2\n",
"wHu4ol2Xaai4vK0IbXAlAt4CLlGlYIuTxtUgyUHkStwflTuA/lYz3VRHdvPUpEWpVaWvA0hYmgH/\n",
"wy1Mur5UqLfDDb+MVOX+xDdIWuFKFNQDuqKa2q3yjEkg67GblAtWlY4lYlWphKUlbojlTbYM9QNw\n",
"vfhrkxTqxwNf4D4leBbqprqzHrtJKfH9XNyq0tkEq0olLK2A94HnNaQ3lzhe6IWrC3OmKuMS2xhp\n",
"hFtslAf0Q/XThJ7fmDSxHrtJGfF9odSqUgnLzrie8pNRQn0A8ALQPwmhvj+ul14L6GShbrJJUnvs\n",
"IrIA+BPIBzapatdkXs9kvNtxtdN7qOdtlrC0ASYD92lI7448UITzgRBujvqXCWuBSC3gamAYcDGq\n",
"rybs3MZkiGQPxSjgqZbcqszUPKVXlUpYdsUNv4zUkD5YdJyr+3ItcB5uNWniNqoQ2Rl4Hvffpe0/\n",
"arJWKoZiJAXXMBksYlVpH/W8FRKWPXE3Sm+JEuqjgFNxdV8SGeonAzOA8UAPC3WTzVLRY/+fiOQD\n",
"j6rq40m+nskwwarS+3DDLwslLHvj5onfqCF9uug4oTbwOLAnrqeemE95IlsB9wMHAn1R/Twh5zUm\n",
"gyU72A9R1SUi0hSYJCLzVHVq4ZMiMjziWF9V/SS3x6RQxKrSAep5cyQs7XFz16/WkL5QdJxQD3gZ\n",
"N4e8lyrrop6w0g2Qg4AXcX9I9kU1Mec1JoVExAO8Sr1GNfmbtgOISAhYq6p3Br+rqtowTZYS328L\n",
"fAhcop73uoSlE26B0WUa0peLjhO2wm1jtxQYrMrG+C8utXH12YcAF6L6VtznNCZDxJKdSRtjF5EG\n",
"ItI4+LkhcASuoJPJcuL7TXE981uCUN8v+H1oqVBvhpsV8w1wWoJCfRfgA+AQXC/dQt3UOMm8edoc\n",
"mCois4BpwFhVnZjE65kMIL6/I25e+kvqeQ9JWA7E3bA8X0PFxbREaI2r+zIOGKpKftQTxnxhEURO\n",
"x/239h+gD6q/xnVOY6qplA3FbHFhG4rJOuL7e+J65g+q542SsByCKxFwpoZ0fNFxwl7BcXeqcm/8\n",
"F5ZtgIeBDsCpqCZu3rsxGSatQzGmZhHf3w83hTEchHp3XKgPKhXqXXHDLzckKNQPw216vRzYz0Ld\n",
"GKsVYxJAfP9w3KyW89Tz/ith6YWbjTJQQ/p+0XFCD9wsmXNUGRPfRSUXGA6cDZyLamJLDhhTjVmw\n",
"m7iI75+AGwYZoJ43RcJyGm5z6eM1pB8WHSccDzwCnKhKfJtAi+yO+8OxHFfnxWqmGxPBxthNlYnv\n",
"nweEgaOYkjcLuBL4J9BXQ/p10XHCucC/gaNUmVn1C4rgeui3B9d9kHT9B2xMmsSSndZjN5UWVGks\n",
"qufClLwfgLuBHsDBGtLFRccKV+Pmk3dXZX7VLypNgMeA3XA107+u4BXG1Fh289RUivh+DnAnQT0X\n",
"puT9jBs37wx0Kwx1EUSEkcAZuLov8YR6D2AWsBA4wELdmPJZj93ELNgk40lgV+AwpuQpbjXpcqC3\n",
"hnQDFNV9eQTYBzhMlRVVu6DUBW4CTgPOwtZBGBMT67GbmIjvNwDeALYHejElrwFuhecc3OyXwlBv\n",
"A0wEdgZ6xhHqewGfAHsAHS3UjYmdBbupkPj+NrgFRX8A/ZiS1xr4CDcz5VINaX4w9HIu8FlwbF9V\n",
"1lb+YiKIXIj7o/EI0B/V5Ql6K8bUCDYUY8olvt8SN9wyGbicKXkHA68DV2pInwcQoRWu5G5zIE+V\n",
"r6p2MWmKG+ppBXRDdV7878CYmsd67KZM4vu74io0vgoMY0peP9xq0tM1pM8HvfTTgZnAp8CBcYR6\n",
"H9wN0rkqC4eqAAAUiUlEQVTAQRbqxlSd9dhNVOL7HXHFu25Sz3tEwjIE+BfQR0P6uQjNgUdxN1L7\n",
"qPJF1S4k9XDz0o8HBqE6OSFvwJgazILdbEF8vxtuuGUoU/JekyncDJwEHKoh/VGEAbhdiZ4CTlbl\n",
"76pdSNoDo4F5uBWktjeuMQlgK09NCeL7x+DGuU9lSt4UXK98H+AohmsB8CDQCThDlWlVu4gIboXq\n",
"v4CrgGdtBakxsbHqjqZSxPfPwN0EPZopeZ/gdjZqDuQxXA/CTW38FegcR6i3wA3xnIYbS3/GQt2Y\n",
"xLKhGAOA+P7lwKVAHlPyVuBmwXzFU1Ou5ufDHgS6AQPjKuAlcgyuLMDjwE2oboq/5caY0izYa7ig\n",
"7sutwHG4EgF1cHPUXya86UO09hfAGKBjlealA4g0AO4A+gID0OKqj8aYxLOhmBpMfL8WrgfdA+jG\n",
"lLxmwFT+2vYBhmsztPZjwNmqDI0j1DsDnwNb426QWqgbk2QW7DWU+H493Pz0NkAPpuR1Ad5l9qkP\n",
"MWLlpbhPcx1U+V/VLiA5iFyJKy9wM6qnofpHYlpvjCmPDcXUQOL7WwFv4Yp3Hc2UvJNRRvH66Ml8\n",
"dcoQ4HxVqrYjkSuv+w9cvfR6QFdUf0pQ040xMbBgr2HE95sB7wDTWT3rYr4cdhWb6l7KE9P+YmnH\n",
"TUB7VcqfTy6yFbB78LVHqe85wHxcwbBRqG5O3rsxxkRj89hrEPH91rihkVeYdVmYlXPvZ13TE3ni\n",
"E+XPnS9S5fXig6U+blVpYWhHBnhj4HvgO1yIz4/4+XebvmhM8sSSnRbsNYT4fjtcMa9RTMl7jLXN\n",
"xuYub3NQh5du/uSZv4c9tQ9fN6NkeDcHfqJkaBd+/xXVgrS8EWNqOAt2AyI5x9xyy7ET99vvqQvG\n",
"jv3vvx99aOOM5nVOb7Mqp17btX9tzqFgsWwZ3N8BP9swijGZx/Y8rSncEv0WRBnzfqdr190/bteu\n",
"9tMjR36170dzmt663za9F2zea0W/X1v03Y3XPkW1anVejDEZy3rs1YnIdmx5s7Lw+wZK9bx73Hln\n",
"28mdO1+hf9UewLBVJ3D0RRfzy/6vsbTjQJ083MbBjamGMn8oBrZFdXVaGpCpRBoRPbj3wH3Cijbm\n",
"PR/VVSVO4/tDget4s9UQ3pk/guMH7cqfra7Sh+bcl7o3Y4xJtOoQ7GuAL4BxwFhgXo2YUeFqkEfO\n",
"OIn8vg1lzzhZVtG/T1AiIIRyGsM6vUL+O/+k3zk51Pq7v4bzq7bYyBiTMdIa7OJ2xLkHqAU8oaoj\n",
"tmgcNATygKOBo4DNuIAfB0xB3QbJ1ZJIbdyqzmg97xbAAqL3vn+p6owT8f0c4D425uQxqOsa9hq5\n",
"Pd1vakStzX01pDPjfEfGmAyQtmAXkVrAt0BP4BfcBsenqOo3ZTbO3QDch+KQb4+rMDgOGIfqrwlv\n",
"aLxEcoAdid7zbg0sIfqMk4WJnnEivl8H5VmW1u3CeV2a0K/fTHZ7dyeEPhqylZ/GZIt0BvtBQEhV\n",
"+wS/XwugqrfH3Di3NL0PLuR7Awsp7s1/lrJ51O4PTuk53oXfdwVWEb3n/WOqPnGI7zfkr5xxfLvV\n",
"3tywxwKGtvmFhr+3BI7WkC5PRRuMMamRzumOrYBFEb8vBg6o1BlUVwAvAi8GwxoH4UL+SaAZIuNx\n",
"IT8xIcWlRLalOLRLB/hGSgb3K8H371GtWtXDBJF3PmjC+jrT+XzbHXi00QiuanQworWBwzWk69PZ\n",
"NmNMeiQr2GP6GCAiwyN+9VXVj3423QxMDb6uRaQNLuTPAp5EZAbFN2C/K/MGo5txshvRe991KNnj\n",
"HkfxjJOM3ItThnzfhe65PtObrGPy9D5ccvqdwExgiIZscZEx2UBEPMCr1GuSNBRzIDA8YijmOqAg\n",
"8gZqwuaxizQEDscF/dG4+dzjgE9xnxwi65xsC/xA9KGTpdVlRo4IgrfsCob8MIIvtnmPTT0vZatf\n",
"xwLPA2ENVY/3YYypvHSOsdfG3TztgdsjczoV3TxN0IWBDriQ74Ibl48M78XVvcaJCM3osvJlbvym\n",
"Gwsa3MIf+47H7U36fxrSx9PdPmNMcqVtjF1VN4vIxcAE3HTHJyNDPWncX6kvg6+sI8KJ7LfyMcJf\n",
"10U5hT/2XYf7dHK2hnRMuttnjMkMVlKgGhChCXA/PZd255p59amtxzMlry1wG9BfQ/pJmptojEmR\n",
"WLLTtsbLcCIcDczm/B+acv03OeRs6sGUvEOB/wO6W6gbY0qz6o4ZSoStgXtAu/PgF2PZe00vNq3x\n",
"+PjYS4GDgYM1pEvS3ExjTAayHnsGEuEIYA45uoF3po5l7zUH88fsnnx87G24GT6HWagbY8piwZ5B\n",
"RGgkwsPAEzTadAHvTWlAvYIuLHj2WGZd+hxuKueRGtI/09xUY0wGs2DPECJ0B2YD9Tjit66M+egi\n",
"oClfXn4WC58pnJc/SEO6Ma0NNcZkPBtjTzMRGgC3ACcBFzLZnwqMARbyyYl3snHFe8BdGtK709lO\n",
"Y0z1YcGeRiIcCDwLfA50YLKfC0wBfD444i100wTgEg3py+lspzGmerFgTwMR6gJh4EzgYlVeE9/f\n",
"FZgIPM2UHt9BwSvAyRrSyWlsqjGmGrJgTzERuuB66d8BHVVZKr7fERgP3MSUvLrAXUAvDWlWrqA1\n",
"xiSXBXuKiJAL3ABcBAwDRqui4vvdgNfQ/Ev4oGcX4BjgEA3pwjQ21xhTjVmwp4AI7XG99N+Azqr8\n",
"AiC+fwzwFJvXDuajY04D2gKHakhXpK+1xpjqzqY7JpEItUW4DngfeBA4KiLUBwOPs27hAD465nKg\n",
"EdDTQt0YEy/rsSeJCHvieunrgP1UKRpaEd+/HLiM36cMYO7we3FljS+2zTGMMYlgwZ5gIuQAl+LG\n",
"00PAw6oUAIjvC27Oen9+fPx0Fo1+FngKuMU2xzDGJIoFewKJsCvwNCDAgap8X/Sc79cCHgY6Mfua\n",
"S1k1/RXgBg3pk+lprTEmW9kYewKIICIMAaYBbwFeqVCvB7wKtGX64FGsmv4CcI6FujEmGazHHicR\n",
"dgaeBLYGuqlSYqco8f3GuLBfyUfHvM7mtfcBR2tIp6e+tcaYmsB67FUU9NLPxpUDmAwcHCXUmwGT\n",
"UZ3PB72+YvPaq3CbY1ioG2OSxnrsVSBCS+BxoBXQQ5XZWxzj+62BiWj+f/igV1PQY3ELj35LcXON\n",
"MTWM9dgrIeilnwrMAr4ADigj1NsBH7J5/eN80LM9aFvAs1A3xqSC9dhjJEIz4CFgL9xCoxlRj/P9\n",
"A4H/smFpiGkDBwM/AAOsjroxJlWsxx4DEY4HvsSFdJdyQr038DZ/zLmGaQMvAz4AzrBQN8akkvXY\n",
"yyHCdsD9wP7ACap8XOaxvj8QuJcl46/iu1G3ACM0pPenqKnGGFPEeuxlEOEo3FZ1y4FOFYT6RcAd\n",
"/PDw9Xw3ahQwzELdGJMu1mMvRYStcfXQDwcGqeKXeawrERACTmPO9SNZ+cmtuPH0KSlprDHGRGE9\n",
"9ggi9MT10jcDHSoI9RzcME0/ZpzzAis/uQpXndFC3RiTVqJpqj0lIqqqkpaLlyJCI2AkbpOL81R5\n",
"t9zjfb8O8CyqO/DpgNlsXHE40EdDuigFzTXG1GCxZGdae+wivCTCiUGwpqsN3XAzXhoC7WMI9YbA\n",
"22h+Iz46eikbV3QCulmoG2MyRVKCXUSGi8hiEZkZfPUp49DJwHnAryK8JcJgEbZNRpu2bCP1RbgL\n",
"eAUYpsoZqqwu9zW+vx3wP/L//p2pvRuRv74Wbm/SlaloszHGxCJZPXYF7lLVzsFX1F6wKo+p0hto\n",
"DbwO9AcWiDBBhAtEaJGMxolwADAT2AE3lv52ha/x/VbAVDau/oIP+3RA8+cCJ2lINySjjcYYU1XJ\n",
"HIqJefxclVWqPK9Kf1z9lceB7sA8EaaKcJkIreNukFBXhFuBt4H/U2WgKssrfJ3v7wF8xPqfx/FJ\n",
"/yOBl3E7HuXH2yZjjEm0pNw8FZEQcBbwBzADuEJVV5c6psIbACLUBXoCxwPHAgtxPfs3VPm2cm2i\n",
"M26ruh+BC1RZGtPrfL8LMJZVM55i9lXnANdoSJ+tzLWNMSZRYsrOqga7iEyCqEMlNwCfAr8Hv98E\n",
"tFTVcyrbuJLHUxvoBpyAG7JZDbyBC/ovVYn6RkTIBa4DLgauAF4o69gtXuv7ecAr/DrmcebfdT5w\n",
"uoaiDysZY0wqJDXYK9GINsAYVW1f6nEFwhEP+arqx3ZOcoCuuJ78CbhhnzeCr0+L9hgV2uF66cuB\n",
"c1VZHHO7ff944BF+ePh5Fr96CtBPQ/pZrK83xphEEBEP8CIeCqUl2EWkpaouCX4eBuyvqqeWOkaZ\n",
"PPl44C31vCo3QgQBOlAc8tsBbwIrgCHA9cATsfbSAcT3z0X138wNjWX51MI56t9X+EJjjEmytPXY\n",
"ReQ5oBNudsxPwAWqurTUMcrkyfNwY97/VM/7MTHXZk/cUM2eQFiVBTG/1pUIuAbVC5g59DPWfNMW\n",
"OEpDuiwRbTPGmHhlxFBMmRd2wV4XGAZcDdwLjFTPS8v0waBEwCi0oA+fDV7CX79swtV9WZuO9hhj\n",
"TDQZH+yFjQu2kbsH2Ae4WD1vQkrb4vu5wOMUbNqLT0+qxabVXwHnaUg3pbIdxhhTkWoT7EWP+f5R\n",
"uMJanwPD1PNivtlZ5Xb4fn3gVTavb8SnJ+5I/l+vAjdqKE3/MMYYU45qF+xQFLTXARcBtwH3qecl\n",
"pecsvr8N8DZ/r1jLtIEd0c23akgfTMa1jDEmEaplsBc97/u7Aw/glv1fpJ43NaHX9/0WwLus++kn\n",
"ZpxzCOiFGtI3EnkNY4xJtGod7FA0S+VE3MYX7wFXq+fFPUNFfH8XYBIrP5vJnKu7ASdqSBP6h8MY\n",
"Y5Kh2gd70bG+3xi3U9Hg4Ptj6nlVqtMivt8B1XdYMnYa8+/qAhypIf26KucyxphUy/hgZziXABOA\n",
"+bHcrBTfbw88DNQFhqjnzajUNX3/UFRf56cnPmfR6J2AvhrSpN+gNcaYRKkOwf4U0BvYiAv4d4H3\n",
"NaRrynydm28+GLgdV0LgBvW8VRVez/ePRvUp5t36Dcv+B65EQLn1140xJtNkfLCrqkhYBGiHC/g+\n",
"wIHAF7iQnwDM0pAWbPF6t+nFLcBxwDXA82WVJhDfH4wWjGL2VYtZ/cWPuGJeVkfdGFPtVItg3+Lx\n",
"sDTA1WLvgwv7bYGJuKCfVHp5v/j+/rjhmXXAUPW8r0o9PwzNv5LPL1jHuh/eBYZZHXVjTHVVLYN9\n",
"i+PC0gYX8L2Bw4EfKO7Nf6Ih3SS+Xwu4AFct8png+zrgFgo2DuSzM+uwYcl9wChbeGSMqc6yIthL\n",
"vCYsubihmsLe/G64fVMnABPoPnk9MBLIA2awee1eTB/chE2rrtCQPp/Yd2CMMamXdcG+xTnC0gzo\n",
"hQv5I3A7Nk2g7XmL2bbL4Xx5+X7krz9VQzoxAU02xpi0y/pgL3G+sOQAHSm+CdsKOFlD+kWirmGM\n",
"MelWo4LdGGNqgliyMydVjTHGGJMaFuzGGJNlLNiNMSbLWLAbY0yWsWA3xpgsY8FujDFZxoLdGGOy\n",
"jAW7McZkGQt2Y4zJMhbsxhiTZSzYjTEmy1iwG2NMlrFgN8aYLFPlYBeRASLytYjki8i+pZ67TkTm\n",
"i8g8ETki/mYaY4yJVTw99jlAf+CDyAdFZG/gZGBvXF30h0Qk6z4ZiIiX7jbEw9qfXtb+9KnObY9V\n",
"lQNXVeep6ndRnuoHvKSqm1R1AfA90LWq18lgXrobECcv3Q2Ik5fuBsTJS3cD4uSluwFx8NLdgGRL\n",
"Rk96B2BxxO+LcbsZGWOMSYHa5T0pIpOAFlGeul5Vx1TiOunZpskYY2qguLfGE5HJwBWqbm9REbkW\n",
"QFVvD35/Fwip6rRSr7OwN8aYKqhoa7xye+yVEHmRt4HRInIXbghmd2B6ZRtmjDGmauKZ7thfRBYB\n",
"BwLjROQdAFWdC7wKzAXeAS7SdO2YbYwxNVDcQzHGGGMyS1rnl4vIKBH5RkS+FJE3RGTrdLansspb\n",
"pJXJRKRPsHhsvohck+72VIaIPCUiS0VkTrrbUlkispOITA7+m/lKRC5Jd5sqQ0Tqicg0EZklInNF\n",
"5LZ0t6kqRKSWiMwUkcpMAMkIIrJARGYH7d9iiLtQuhcOTQTaqWpH4DvgujS3p7KiLtLKZCJSC3gA\n",
"t3hsb+AUEdkrva2qlKdxba+ONgHDVLUdbghzaHX6t1fVDUCeqnYCOgB5InJomptVFZfihoqr43CF\n",
"Ap6qdlbVMtcHpTXYVXWSqhYEv04DdkxneyqrnEVamawr8L2qLlDVTcDLuEVl1YKqTgVWpbsdVaGq\n",
"v6nqrODntcA3uHUf1Yaqrg9+rAPUAlamsTmVJiI7AkcCT1By0kd1UmG7091jj3Q2MD7djagBWgGL\n",
"In63BWRpICJtgM64Dk21ISI5IjILWApMDiZLVCd3A1cBBRUdmKEU+J+IzBCR88o6KFHTHcsUyyIn\n",
"EbkB2Kiqo5PdnspK4CKtTFEdP35mFRFpBLwGXBr03KuN4BN2p+B+2AQR8VTVT3OzYiIiRwPLVHVm\n",
"Na4Xc4iqLhGRpsAkEZkXfIotIenBrqq9ynteRM7EfTTqkey2VEVF7a+GfgF2ivh9J0qWgDBJJCK5\n",
"wOvAC6r6VrrbU1Wq+oeIjAP2A/w0NydWBwPHisiRQD1gKxF5TlUHp7ldMVPVJcH330XkTdzQ6hbB\n",
"nu5ZMX1wH4v6BTdmqrPqMl43A9hdRNqISB1cJc6309ymGkFEBHgSmKuq96S7PZUlItuLyDbBz/WB\n",
"XsDM9LYqdqp6varupKptgYHA+9Up1EWkgYg0Dn5uCByBm8CxhXSPsd8PNMJ9pJgpIg+luT2VUtYi\n",
"rUymqpuBi4EJuJkBr6jqN+ltVexE5CXgY2APEVkkImelu02VcAgwCDebZGbwVZ1m+LQE3g/G2KcB\n",
"Y1T1vTS3KR7VbViyOTA14t9/rKpOjHagLVAyxpgsk+4euzHGmASzYDfGmCxjwW6MMVnGgt0YY7KM\n",
"BbsxxmQZC3ZjjMkyFuzGGJNlLNiNMSbL/D9Nh9MMaaoMVgAAAABJRU5ErkJggg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x7f7e89284f28>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"def plot(n):\n",
" X = construct_poly(x,6) \n",
" Xt = construct_poly(xt,6) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" print(MSE(X.dot(w),y))\n",
" \n",
" w = wRidge(X,y,0.1)\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" print(MSE(X.dot(w),y))\n",
" \n",
" w = wRidge(X,y,10.0)\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" print(MSE(X.dot(w),y))\n",
" \n",
" plt.plot(xt,yt)\n",
" #plt.plot(x,y)\n",
"plot(1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Hand in printed copy of completed notebook."
]
}
],
"metadata": {
"annotations": {
"author": "",
"categories": [
"intelligent-systems-1-2015"
],
"date": "2015-04-30",
"location": "Beginning of next lecture",
"parent": "IS_SS2015",
"submission_date": "2015-05-07",
"subtitle": "Exercise Sheet 3, Linear Regression",
"tags": [
"IntelligenSystems",
"Course"
],
"title": "Intelligent Systems 1 - Summer Semester 2015"
},
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.4.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
Jan-Peter Hohloch