{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"preamble": true
},
"source": [
"(Defining latex commands: not to be shown...)\n",
"$$\n",
"\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
"\\DeclareMathOperator{\\minimize}{minimize}\n",
"\\newcommand{\\real}{\\mathbb{R}}\n",
"\\newcommand{\\normal}{\\mathcal{N}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gaussian Algebra (25 Points)\n",
"\n",
"Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
"\n",
"\\begin{equation}\n",
" \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
"\\end{equation}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
"\n",
"Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
"\n",
"\n",
"Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Linear regression (50 Points)\n",
"\n",
"In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
"\n",
"1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
"2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
":0: FutureWarning: IPython widgets are experimental and may change in the future.\n"
]
}
],
"source": [
"# Loading the required packages\n",
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from IPython.html.widgets import interact\n",
"import csv"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [],
"source": [
"def wRidge(X,y,lamb):\n",
" # Change the following line and implement the ridge regression estimator wRidge\n",
" #w = np.zeros(X.shape[-1])\n",
" #print y\n",
" #print np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)\n",
" return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Read ex1_train.csv and assign the first column and \n",
"# second column to variables x and y respectively.\n",
"\n",
"\n",
"a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
"x= a[:,0]\n",
"y= a[:,1]\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4) Plot the training data with appropriate labels on each axes!"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.text.Text at 0x105161310>"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAG1tJREFUeJzt3X285nOdx/HX2wxRhFQMxtJETDGjZZasHCsaYmdnVXIT\n",
"IjfRNCuVu12mKCpJW9RaN7EMKy2LSiaczCqDjDGMCWW2QSYKyza2YT77x/dH5zrnzJzrOue6ru/v\n",
"d13v5+NxPeZc51w378PM73N97xURmJmZvWqV3AHMzKxcXBjMzKyGC4OZmdVwYTAzsxouDGZmVsOF\n",
"wczMamQtDJIulrRE0vw+35sh6XFJc4vb5JwZzcy6Te4WwyVA/wt/AOdExLbF7aYMuczMulbWwhAR\n",
"s4FnB/mR2p3FzMyS3C2GFZkmaZ6kiyStkzuMmVk3KWNh+DawGTAR+C3wtbxxzMy6y+jcAfqLiN+9\n",
"+rWkC4Eb+j9Gkjd4MjMbhogYsqu+dIVB0piI+G1xdyowf7DH1fPLlZWkGRExI3eO4XL+vJw/nypn\n",
"h/o/VGctDJKuBHYB3ixpMXAa0CNpIml20mPAURkjmpl1nayFISL2H+TbF7c9iJmZvaaMg8/doDd3\n",
"gBHqzR1ghHpzBxih3twBRqg3d4AR6M0doB1UxYN6JEWVxxjMzHKo99rpFoOZmdVwYTAzsxqlm65q\n",
"ZmbNI7Ee8I7iVhcXBjOzipNYDRjHnwtA39uqwC+LW32v58FnM7PykxCwPoNf/McCi/lzAeh7WxJB\n",
"pNeo79rpwmBmViISawCbM/DivwXwMoNf/H8VwZ+Gfm0XBjPrIMUn5rWBMcCGxZ/9b2/iz5+cFzLI\n",
"p+YyKH6Xjai98G9Z/LkB8GsGKQAR/H5k7+vCYGYVUFwk12Pwi33/7y0j7brc//Zk8eezwF+w8n72\n",
"vrdHIljawt9tTdIn/cE+/b/YJ0ffIrYogpdbk8eFwcxKRGIU8AngndRe7NcH/peBF/kBtwheHOZ7\n",
"952Z0/f2tuK1BysaT9TTyih+r01W8PpvAh4d7PUjeH44v8tIuDCYWWlIrAVcCawJXEPtxf+pCF7K\n",
"lGs06fyXwS7qawIP069YkFokW/Z53Djg9wxeXH4TwfL2/UYr58JgZqUgsQnpXJU7gU9GsCxzpLpI\n",
"rMPAbqCNgEUM7I4aVkum3VwYzCw7ie2B60gnMX69TAPA3ajea6cXuJlZS0h8EDgf+HgE1+fOY/Vz\n",
"YTCzpipmGZ1IGmh+fwRzM0eyBrkwmFnTFFszXAC8C9ghgiczR7Jh8O6qZtYUxZTQWaRFaLu4KFSX\n",
"C4OZjZjEO0izju4E9o3gfzNHshFwYTCzEZHYFbgdOCuCE8o0b9+Gx4XBzIZN4nDgKuAjEVyUO481\n",
"hwefzaxhEqsAZwFTgZ0jeDhzJGsiFwYza4jEG4DLSfsA7TDSHT+tfNyVZGZ1k9iINJ7wHLC7i0Jn\n",
"cmEws7pIvJs06+h7wGH1HAxj1eSuJDMbksQU4ELg6Ai+nzuPtZYLg5mtULG9xaeB44C9Irg7cyRr\n",
"AxcGMxuUxKrAecAkYMcIFmeOZG3iwmBmA0isSxpLWEqajvpC5kjWRh58NrMaEuOAnwPzgb9zUeg+\n",
"Lgxm9hqJnYE7gHMjOC6CV3JnsvZzV5KZASBxMHA2cFAEN+fOY/m4MJh1uWJ7iy8ABwA9ESzIHMky\n",
"c2Ew62ISawCXkg65/6sIns4cyUrAYwxmXUpiA6AXWAbs5qJgr3JhMOtCEtuQtrf4AWlM4aXMkaxE\n",
"shYGSRdLWiJpfp/vvUnSLEkPS7pZ0jo5M5p1Gom9gJ8AJ0bwhQgidyYrl9wthkuAyf2+dyIwKyK2\n",
"AG4p7ptZE0hMI+15NCWCq3LnsXJSRN4PC5I2BW6IiK2L+wuBXSJiiaQNgN6I2LLfcyIi1PawZhUl\n",
"MRo4F+gB9o5gUdZAlkW9184yzkpaPyKWFF8vAdbPGcas6iTWJh2/KWCnCJ7PHMlKroyF4TUREZIG\n",
"bdJImtHnbm9E9LYllFmFSGwK3Aj8FJgewct5E1k7SeohtRIbe15Ju5J6IuIpSWOA29yVZNY4iR2B\n",
"75POZv6mB5mt3mtn7sHnwVwPHFJ8fQhwXcYsZpUk8RHgP4EjIvhnFwVrRNYWg6QrgV2AN5PGE04l\n",
"/WW+GtgEWAR8OCKe6/c8txjMBlEcrPNPwOHAPhHcnzmSlUi9187sXUnD4cJgNpDE6qSpqJuTpqM+\n",
"lTmSlUyVu5LMrEESbyGt+1mNtBGei4INmwuDWcVJjAfmALcBH4lgaeZIVnGlnq5qZisnsTtwBfCZ\n",
"CC7Lncc6g1sMZhUlcTTwb8C+LgrWTG4xmFWMxCjSSWt7An8dwaOZI1mHcWEwqxCJtYCZwOuBHSN4\n",
"NnMk60DuSjKriOIIzhtIa34muyhYq7jFYFYdhwKrA0dGsDxzFutgXuBmVgES6wEPAntGMDd3Hqsm\n",
"r3w26yASFwIvRvAPubNYdVX5PAYz60NiJ9JJh+NzZ7Hu4MFnsxKTWBX4DvDpCP4ndx7rDi4MZuU2\n",
"HXgS+F7uINY9PMZgVlISY4G5wA5exGbN4N1VzarvG6ST11wUrK08+GxWQhIfALYGDsidxbqPC4NZ\n",
"yUi8HvgWaSHbS7nzWPdxV5JZ+ZwCzIlgVu4g1p3cYjArEYmtgCOBCbmzWPdyi8GsJCQEnA+cHsGT\n",
"ufNY93JhMCuPA4G1ScXBLBuvYzArAYl1gQXAlAjuyp3HOpM30TOrEInzAUXwidxZrHN5Ez2zipCY\n",
"BEzFm+RZSXiMwSwjidGkTfI+6xPZrCxcGMzyOgZ4DrgidxCzV3mMwSwTiQ2BecDOESzMncc6nzfR\n",
"Myu/c4ALXBSsbDz4bJaBxO7AJOCw3FnM+nOLwazNJFYnLWKbFsEfc+cx68+Fwaz9TgDmR/CD3EHM\n",
"BuPBZ7M2kng7cCewbQSLc+ex7uLBZ7OSKTbJOw84y0XBysyFwax9PgRsSDqy06y0PCvJrA0k3kia\n",
"nrpfBMty5zFbmdKOMUhaBPwP8AqwLCIm9fmZxxisUiTOBdaK4PDcWax7dcImegH0RMQfcgcxGwmJ\n",
"bYH9gXfmzmJWj7KPMbhVYJUmsQrwbeDkCJ7JncesHmUuDAH8RNI9ko7IHcZsmI4gdYdekjuIWb3K\n",
"3JW0U0T8VtJbgFmSFkbE7NyhzOolsT5wBrBbBMtz5zGrV2kLQ0T8tvjzaUnXkvaVea0wSJrR5+G9\n",
"EdHb1oBmQ/sqcGkE9+cOYt1JUg/Q0/DzyjgrSdLrgVER8YKkNwA3A5+PiJuLnwfEThH8LGtQsxWQ\n",
"6AEuA8ZH8GLmOGZA9Vc+rw/MlnQfMAe48dWi0Mdn2h/LbGgSq5E2yZvuomBVVMoWw1CKFsMzwHsi\n",
"eCR3HrO+JE4CdgL2iaB6/8CsY9XbYqhyYTgDWC+CY3LnMXuVxGbA3cD2ETyWO49ZX91QGDYAFgJb\n",
"RPB07kxmxSZ51wM/j+BLufOY9Vf1MYYhRbAEuAb4RO4sZoUpwNuBs3MHMRuJyrYYIkISWwG9wKYR\n",
"LM0cy7qYxJrAg8ChEdyWO4/ZYDq+xQAQwUPAXcBHc2exrncqcLuLgnWCSrcY0tfsAlwAbOXVpZaD\n",
"xLuAW4Gtiy5Os1LqihZD4XbS9tx75w5i3afPJnmnuShYp6h8YSjmiZ+NF7xZHocAryO1Ws06QuW7\n",
"ktJ9RgOPkk7HmpMvmXUTifVIA857RXBv7jxmQ+mmriQieBn4OnB87izWVc4CrnZRsE7TES2G9D3W\n",
"Ah4DJkXw6zzJrFtI7ARcTdok7/nceczq0VUtBoAIXgD+FfiH3Fmss0msShpwPt5FwTpRx7QY0vfZ\n",
"EHgAeHsEPivaWkLieGAysIc3ybMq6fi9klb0y0lcAjzivWqsFSTGAnNJO/s+nDuPWSO6uTC8C5gF\n",
"bBbBS+1NZp1O4vvAAxGcljuLWaO6bozhVRE8QPpEd2DuLNZZJPYCJgBn5s5i1kpDFgZJn5K0bjvC\n",
"NNHZwPHFqlSzEZN4PfAt4Fi3RK3T1XPhXB+4W9LVkiZLGrIZUgK3AS8Be+YOYh3jZOCeCH6cO4hZ\n",
"q9U1xiBpFWAP4FBgO9L87Ysi4lctTbfiPEP2k0kcABwRwa5timUdSmJLYDYwMYIncucxG66mjjFE\n",
"xHLgKWAJ8AqwLnCNpK+OKGVrfQ8YJ7Fd7iBWXcWpbOcDZ7goWLcYssUgaTpwMPB74ELg2ohYVrQi\n",
"HomIca2POSBTfVOuxKdJZ+/u34ZY1oEkDiRt0Lh9sfWKWWU1bbqqpM8DF0fEfw/ys/ERsWD4MYen\n",
"gcLwRtI2GX8ZwaKWB7OOIrEOsACY6s0ZrRN07TqGgY/lK8CqERzX4ljWYSTOA0ZFcHTuLGbN4MLw\n",
"2mPZGLgfGBfBs61NZp1CYnvgetImef57Yx2haxe49RfB48CNwJG5s1g1SIwCvgOc4KJg3ajjWwzp\n",
"8UwAfkjaJuNPrUtmnUBiGrAvsKs3ybNO4hZDHxHMI5205dlJtlISY4BTgWNcFKxbdUVhKJwNfKaY\n",
"l262IucAF0bQ9tl2ZmXRTYVhFhCkFdxmA0jsDuwInJ47i1lOXVMYim6Bs0mLlcxqSKwOnAdMi+CP\n",
"ufOY5dQ1haFwFbCVxMTcQax0PgcsiOCG3EHMcuuKWUm1z+VzwNYRfLTJsayiJN4O3Am8O4Lf5M5j\n",
"1ipe4LbC57IO8GtgQgSLm5vMqqaYjPAj4NYIvpI7j1krebrqCkTwHPBdYHrmKFYOHwQ2Br6eO4hZ\n",
"WXRdiyE9n01Ix3++LYLnm5fMqqTYZHEBsH8Es3PnMWs1txhWouhHvgk4IncWy+rzwCwXBbNapSwM\n",
"xRGiCyU9IumEFr3N14DpEqu16PWtxIqZaQeSZiOZWR+lKwySRpEOXZ8MjAf2l7RVs98ngnuBh4EP\n",
"N/u1rdwkVgG+DZwSwdO585iVTekKAzAJeDQiFkXEMtLagyktei9vk9GdPk5aBX9R7iBmZVTGwrAR\n",
"1Ewjfbz4XivcBKwK7Nai17eSkXgrcAZwdATLc+cxK6MyFoa2TZPyNhld6Rzg3yK4P3cQs7IanTvA\n",
"IJ4Axva5P5bUaqghaUafu70R0TvM95sJfFFiG18sOpvE3qRN8rbJncWsHST1AD0NP69s6xgkjQZ+\n",
"SereeRK4C9g/Ih7q85gRrWMY+J6cBLwjgkOb9ZpWLhJrAw8Ah0Rwa+48ZjlUeksMSXsC5wKjgIsi\n",
"4sx+P292YXgT8ChpD6UnmvW6Vh4SFwARwVG5s5jlUunCMJRmF4b0mnwDWBrBic18XctP4n3AxaTC\n",
"75Xu1rVcGBp+TTYD7iadC/1CM1/b8pFYE5gPHBvBD3PnMcvJW2I0KILHgFuAw3Nnsab6EnC7i4JZ\n",
"/dxiqHldtgeuAcZF8HKzX9/aS2Jn4N+Bd0Xwh9x5zHJzi2EYIrgbWETaitkqTGIN0srmY10UzBrj\n",
"wjDQ2cCpEhvnDmIjMgO4L4JrcwcxqxoXhoFuBK4AfiG55VBFRZfgocC0zFHMKsljDCt8DyaRCsRs\n",
"YLpnKlWDxOuAXwBfimBm7jxmZeIxhhGK4C5gW2A5MFdih8yRrD4nk870vjJ3ELOqcouhrvfj70n7\n",
"959H+iTqGUslJDEB+Akw0SvYzQZyi6GJIvgP4N3AzsBPJd6WOZL1IzGatLr5BBcFs5FxYahTcbF5\n",
"P2mdwxyJg33AT6l8Bvg9cEnuIGZV566kYb0/E0gD0w+SDnx5NlcWA4mtSJMEtotgUeY4ZqXlrqQW\n",
"imAesD2wBJgnNb7fuTWHlHbgBU5zUTBrDheGYYpgaQSfAo4ErpA4S2K13Lm60DTgZdLkADNrAncl\n",
"NYHEW0ifWjcCDoxgYeZIXUFiHDAH2DGCR3LnMSs7dyW1UQRPA1OAC4DZEkd7YLq1JFYBLgTOdFEw\n",
"ay63GJpMYkvSwPQTwMcj+F3mSB1J4ijgMOA9EbySO49ZFbjFkEnRjbQjacbSfRJ7Zo7UcSQ2Ac4A\n",
"DnNRMGs+txhaqJitdClwPfC5CJbmTVR9RRfdD4E7Ijgjdx6zKnGLoQQi6AUmAm8B7inWP9jIHAxs\n",
"AHw5dxCzTuUWQxsUn3IPAs4BzgTOjWB53lTVIzEGmAe8P4K5ufOYVU29104XhjaS2Ay4HPgjcKj3\n",
"9KlfUVyvBR6I4B9z5zGrIncllVAEjwG7ALcD9xa7tlp9PgxsAZyeO4hZp3OLIZPifIfLgZ+SDgJ6\n",
"MXOk0ioWEN4P/F0Ec3LnMasqtxhKLoI7SQcBQToIaFLOPCX3DWCmi4JZe4zOHaCbFceFHi6xL3CD\n",
"xDdJK3k9N78gMQWYBGyTO4tZt3BXUklIbAx8F1gd+GgxHtHVJNYFHgAOiOCnufOYVZ27kiomgseB\n",
"PUgzb+6SOMj7LfE14DoXBbP2couhhCQmAjNJc/Y/EcFzmSO1ncT7gX8Bti663MxshNxiqLAI7gP+\n",
"EniGdBDQLpkjtZXEWqSicKSLgln7ucVQchJ7kbaXvpR0StmfMkdqOYnzgDUiOCx3FrNO4pXPHUTi\n",
"raSDgMaQDgL6ZeZILSPxPtIg/NY+S9usudyV1EGKMx3+llQc/kviyE4cmJZ4L2ls5WAXBbN83GKo\n",
"GImtSAcB/QY4ojg9rvKKLcqvBvaP4JbMccw6klsMHSqCh4AdgF+SDgKanDnSiEn8Dako7OeiYJaf\n",
"WwwVJrEraVD6WuCECF7KHKlhxZjCTOBDXq9g1lqVbTFImiHpcUlzi1vlPxG3SgS3ARNIg9J3S2yd\n",
"OVJDJPYgFYV9XRTMyqN0hQEI4JyI2La43ZQ7UJkVg7T7AWcDt0ocJ5Xy/2uNogvscmBqBLNz5zGz\n",
"PyvrBaTru4kaEUFEcCnwV8CHgJskNswca4UkPgBcBkyJ4I7cecysVlkLwzRJ8yRdJGmd3GGqIoJf\n",
"A+8F7iAdBDQ1c6QBJPYBLgH2ieDnufOY2UBZBp8lzSId6N7fKcCd8NoUzNOBMRFxeL/nB/D5Pt/q\n",
"jYjeFkStLIkdSV01twLHleEgoGIL7QuAvSO4O3ces04nqQfo6fOt0yq/8lnSpsANEbF1v+97VlId\n",
"ij2H/hnYibRiOtvFuDjG9HxgrwjuzZXDrJtVeVbSmD53pwLzc2WpugheiOBjpJbYjRKnSIxqdw6J\n",
"D5KKwp4uCmblV7oWg6TLgImk2UmPAUdFxJJ+j3GLoUHFQUCXkU7t+2gE/92m990POBeYHMG8dryn\n",
"mQ3Om+jZAMU01uOBz5LGHa5o8fsdQDpsZ48It/zMcnNhsBWS2Ja039Jc4NhWHAQkcRDwFWD3CB5s\n",
"9uubWeMqO8ZgrRfBXGA74DnSfks7N/P1JQ4Bvgy8z0XBrHrcYuhyEnsD/wpcDMyIYNkIX+8w4Auk\n",
"orCwCRHNrEncYrC6RHAjabB/InCHxBbDfS2Jj5PWl/yNi4JZdbkwGBEsAfYmnZx2h8QRjR4EJHEU\n",
"cCqwawQPNz+lmbWLu5KshsR40sD0ItJBQM/U8ZxjgM8Bu0Xwq9YmNLPhcleSDUsEC0gHAT1KGpje\n",
"Y2WPl/gkafrrri4KZp3BLQZboeJktUuBa4CT+h8EJDEdmE4aU1jU/oRm1gi3GGzEIriVdBDQxsBd\n",
"fQ8Ckvg08Cmgx0XBrLO4xWBDKgaiDwG+CpwBvA44gtRSWJwzm5nVzyufrekkxpG28l6PNKbwROZI\n",
"ZtYAFwZriWJ31tUiWJo7i5k1pt5r5+h2hLHOEcEr4KJg1sk8+GxmZjVcGMzMrIYLg5mZ1XBhMDOz\n",
"Gi4MZmZWw4XBzMxquDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxq\n",
"uDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxqZCkMkj4k6UFJr0h6\n",
"d7+fnSTpEUkLJe2RI5+ZWTfL1WKYD0wFbu/7TUnjgf2A8cBk4HxJHdeqkdSTO8NIOH9ezp9PlbM3\n",
"IstFNyIWRsTDg/xoCnBlRCyLiEXAo8CktoZrj57cAUaoJ3eAEerJHWCEenIHGKGe3AFGoCd3gHYo\n",
"26fxDYHH+9x/HNgoUxYzs640ulUvLGkWsMEgPzo5Im5o4KWiSZHMzKwOish33ZV0G3B8RNxb3D8R\n",
"ICLOKu7fBJwWEXP6Pc/FwsxsGCJCQz2mZS2GBvQNeT0wU9I5pC6kzYG7+j+hnl/MzMyGJ9d01amS\n",
"FgM7AD+Q9COAiFgAXA0sAH4EHBM5mzRmZl0oa1eSmZmVT9lmJdVN0umS5km6T9ItksbmztQISV+V\n",
"9FDxO/yHpLVzZ2rEyhYplpWkycXCyUcknZA7T6MkXSxpiaT5ubM0StJYSbcVf2cekPSp3JkaIWl1\n",
"SXOK680CSWfmzjQckkZJmitppROAKlsYgK9ExISImAhcB5yWO1CDbgbeGRETgIeBkzLnadSgixTL\n",
"StIo4FukhZPjgf0lbZU3VcMuIeWvomXAcRHxTlIX8rFV+u8fES8BuxbXm22AXSX9deZYwzGd1FW/\n",
"0q6iyhaGiHihz901gWdyZRmOiJgVEcuLu3OAjXPmadRKFimW1STg0YhYFBHLgKtICyorIyJmA8/m\n",
"zjEcEfFURNxXfP0i8BBp3VJlRMQfiy9XA0YBf8gYp2GSNgb2Ai6kdtLPAJUtDACSvijpN8AhwFm5\n",
"84zAYcAPc4focBsBi/vc9+LJTCRtCmxL+kBUGZJWkXQfsAS4rZgsUyVfBz4LLB/qgaUuDJJmSZo/\n",
"yG0fgIg4JSI2Ab5L+qVLZaj8xWNOAf4UETMzRh1UPfkrxLMsSkDSmsA1wPSi5VAZEbG86EraGHhv\n",
"lfZNkrQ38LuImMsQrQUoxzqGFYqI3et86ExK+Il7qPySDiU17XZrS6AGNfDfvwqeAPpOUBhL7fYr\n",
"1mKSVgW+D1weEdflzjNcEfG8pB8A2wG9mePU6z3A30raC1gdeKOkyyLi4MEeXOoWw8pI2rzP3SnA\n",
"3FxZhkPSZFKzbkoxsFVlVVhweA+wuaRNJa1G2sX3+syZuoYkARcBCyLi3Nx5GiXpzZLWKb5eA9id\n",
"Cl1zIuLkiBgbEZsBHwFuXVFRgAoXBuDMolvjPtKOh8dnztOob5IGzWcV08fOzx2oEStapFhWEfEy\n",
"8Engx6RZGf8eEQ/lTdUYSVcCPwO2kLRY0sdyZ2rATsBBpNk8c4tblWZYjQFuLa43c4AbIuKWzJlG\n",
"YqVdq17gZmZmNarcYjAzsxZwYTAzsxouDGZmVsOFwczMargwmJlZDRcGMzOr4cJgZmY1XBjMzKyG\n",
"C4NZE0javjh06XWS3lAcRjM+dy6z4fDKZ7MmkXQ6aYOyNYDFEfHlzJHMhsWFwaxJit1D7wGWAjuG\n",
"/3FZRbkryax53gy8gbQ54hqZs5gNm1sMZk0i6XrS2SBvA8ZExLTMkcyGpdQH9ZhVhaSDgf+LiKsk\n",
"rQL8TFJPRPRmjmbWMLcYzMyshscYzMyshguDmZnVcGEwM7MaLgxmZlbDhcHMzGq4MJiZWQ0XBjMz\n",
"q+HCYGZmNf4fBBSJ6m7pJCYAAAAASUVORK5CYII=\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x10511c210>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot the input data here\n",
"plt.plot(x,y)\n",
"plt.xlabel('x')\n",
"plt.ylabel('y') "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
"Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def construct_poly(x,d):\n",
" ## Implement a method which given an array of size N, \n",
" ## returns an array of dimension (N,d)\n",
" poly=np.empty((len(x),1), float)\n",
" xr = np.reshape(x,(len(x),1))\n",
"\n",
" for i in range(d-1):\n",
" poly = np.append(poly, (xr**i), axis=1)\n",
" #print poly \n",
" #print np.shape(poly)\n",
" return poly"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def MSE(y_predict,y_true):\n",
" ## Implement mean squared error for a given input y and its predictions.\n",
" return (np.linalg.norm(y_predict-y_true)**2)/y_predict.shape[0]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecHVX5x/HPN40E6U1CR6nBSBERRDT0gBRBigkdBAQp\n",
"Ih2UyQgIghQLqEjvIE2KCAESRGkqAYKh5SclgPTeE/L8/jgT2Gx2k93bZu/d7/v12lf23pk58+wq\n",
"z5575jnnKCIwM7PW0qfsAMzMrPac3M3MWpCTu5lZC3JyNzNrQU7uZmYtyMndzKwFObmbmbUgJ3dr\n",
"KElPS/pI0vzt3h8naaqkJYrXi0m6WtIrkt6UNF7SLsWxpYpz32n3tW0n99xO0t2S3pM0pgsx7iVp\n",
"oqS3JP1T0tptjv1S0hOS3pb0qKSd2hybX9I/JL1aXDtO0nfaHP+epMeKY69KukbSIm2OryjpjuLn\n",
"fbLttcXx2SWd2eZ3cuesf+PWWzm5W6MF8F9gxLQ3JA0FBhXHprkIeAZYApgP2Al4qV1bc0fEnG2+\n",
"/tTJPV8DTgVOnFVwklYBTgG2jYi5gXOAayWpOOVdYLOImAvYBfiVpLXaHNsdWKi4dhRwpaQ5iuP/\n",
"AL5ZHFsSeL+IC0n9gD8D1wPzAnsBF0tatk14ZwHzACsU5/xoVj+P9V5O7laGi4Gd27zeBbgQUJv3\n",
"VgfOj4gPImJqRDwYEX+t5GYRcXtEXAX8rwunDwEmRMS44vVFwALAQkVboyLiieL7+4G7gLWK1x9F\n",
"xOMRMVVSH2Aq8CrwcXF8UkS8XLQr4JM2Ma0ADI6I0yMZQ/pjsBOApBWAzYG9IuK14pxpMZrNwMnd\n",
"ynAvMJekFST1BbYnJfz255wpaftpQzUdUCfvV+MuYGlJaxSx7Q6Mi4j2nxqQNAj4KvBIu/cfBj4A\n",
"zge2ioiP2xz7hqQ3gbdJn0oOn0ksfYCViu/XIH2S+VkxLPOwpK0r/BmtF3Byt7JcROq9bwhMAJ5v\n",
"d3xbUqL9KfDfYvx69XbnvCrpjTZfy1cbVERMAn5C6jV/WNx/705O/z3wYETc2q6NLwNzkoZlrm4z\n",
"LENE/D0i5gEWAyYDJxeHHgdelnSopP6SNgK+CcxeHF8M+BLwJjAY2A+4oOjRm83Ayd3KEKTkvgMd\n",
"D8kQEW9GxJER8SXg88CDwHXt2pk/IuZt8/V4tYFJ2gI4GFgxIvqThkVulDS43Xknk4ZwtuvwB4z4\n",
"OCJ+A7wDrN/B8RdIfzh2Ll5PBr4DfJs0VHMQcCXwXHHJB6Q/BsdFxJSI+BswBtioqh/YWpaTu5Ui\n",
"Ip4lPVjdBLhmFue+RnrIuYikeau5bRfO2Ri4KSImFve+hZRspz00RVJenLdRRLw7i/b6Ae91cqw/\n",
"6aEqxb3GR8SwiFggIjYBvgjcXxx+eNrt27XhZV2tQ07uVqY9gPUi4oP2ByT9QtJKkvpJmhPYB3gy\n",
"It5oe1pXbiKpj6SBpGTaR9Jskvp3cvpDwLclLa1kQ2A5inF1SUeSKn02bBcLkr5WjKkPkDRI0uHA\n",
"QNLzAyTtIGnx4vslgeOBq9tcP1TSwKLk8RDSJ5bzi8N3As8CRxa/k7WBYcAtXfkdWO/j5G6liYj/\n",
"RsQDbd9q8/0g4FrgDeD/gMWBLdo18Wa7OvfOSgN3JvWQzwTWIQ1x/GHaweLaabXsZ5NKEv8GvAWc\n",
"TqpQeaI4fnwRy8Q29z2iODYb8FtShcyzpDHz4W169ysCd0t6FxgL3AMc1ibOnYAXSCWf65L+gEwu\n",
"fldTgC2BTUnj7n8AdmoTl9l0VO1mHZLOJY0TvhwRQ4v3RgHfB14pTjuy0jI2MzPrvlr03M8Dhrd7\n",
"L4BTI2LV4suJ3cysgapO7hFxF+mjc3v1qEE2M7MuqOeY+/6SHpJ0jqR56ngfMzNrp17J/XfA0sAq\n",
"pDKyU+p0HzMz60C/ejTaZv0MJJ0N3ND+HEmuzzUzq0BEzHLYuy7JXdLgiJi2INJWwPiOzutKgD2V\n",
"pFERMarsOCrl+Mvl+Dtqk0HAccABpBLR0yNqP0mrBX73XfqdVJ3cJV0GfAtYQNIkIAOGFUunBvAU\n",
"na/NYWaGxNdJlXcPAF8jLUmxtMRBEXxSanBNqurkHhEjOnj73GrbNbPW16a3PhL4YURaikJibdLs\n",
"3WslRkR0uoSDdcIzVCs3tuwAqjS27ACqNLbsAKo0tuwAqjS22gaK3vqDwCLA0GmJHSCCt0izcV8D\n",
"xkosXO392hhbw7Z6rKpnqFZ8YymaeczdzCrTWW+9k3NFWoJ5d+DbEUxoTJQ9V1dzp3vuZtYwM+ut\n",
"dySCiOBY4BhgjMS6DQizJbjnbmZ1153e+kzaWBe4HDg0ggtrHGLT6GrurEsppJnZNO0qYYZG8Gol\n",
"7UR82nO/SWIp4Nh6lEr2RFrt3FXp98FIBrwzw8YvnXFyN7O6qEVvvb0IJkisRZoYubTE3hF8PKvr\n",
"moFyDSBt0LI8kwetxNuLfRNiKLO/thDDp/Th3c+/xicDHuMfXWzPwzJmVmvteuv7V9pbn0n7nwMu\n",
"I+0xu00Eb9ay/XpRLpE2YVl+uq9gedASfDjPW7wyBF5YfR7eGfwMH849lo/mvpKpfW+P/2w3Fbqe\n",
"O53czaxm6tFbn8m9+pI2U1kX2DSCZ+t1r+5SrkHAsrRP4mlXrynA40wZ8H9M+vonPL7l53lq3S/x\n",
"6gp9+WS2m4G/Ard19gfLyd3MGqorvfWi5zo3MJhUMTO4g6/5gEnA48Bjxb+PAy9FNn3CKkolf0Ta\n",
"1HyLCNru7FVXxc+yKNMn7xWKfxcm7RH8+KdfU/s+zlWXBxO2+TppD4w1gX+SkvnNwCNdeYbg5G5m\n",
"DZF661OPY/ZXd2DFa37B5vtMYPpk3T6JTyatFtv+64Xi3zeAJZmx19uftsnys68nGRWbkLYe3C2C\n",
"G2v68+Wag9Tj7qgX/m6bONr+IXo6spgiMRewPimZT9vU6Obi644I3ul2PE7uZlZrytWXtFn5SsBg\n",
"PpxrWSZ/bjlmf6UPfT55G8W0BD3t3xm+Ivt0T9nu3nt+ZkywywNfAP7Hewu8yIRtvkz/965nlYvO\n",
"JiXZ59v39mfycy3RSfvzARPp4A9LZPHWdO2kTxJfBjYhJfOvAHfzWe/88WorfJzczaymlGtO0kPM\n",
"Ofj4c3/mzmOGMWnNtRn0xtGM+M4FkcWHJcXVj7R/xPK8tszXee5r+7LoP99h/if6I+YAnmD6pPw8\n",
"6ZPBtCGU5UlVKq/R8SeDZyOLqZ3eX8wLbMhnvfP3+CyZ31nrdXGc3M2sZpRrCVL54b2c9PJFvL/g\n",
"OdSpEqZaEvMD1wEv8t0R+zP08vY98kWBp2k3tNPVTxQSfYDVSIl8E2Ao8DdSQv9rBBNr+gPNcH8n\n",
"dzOrAeX6KilZnsLPPhrN1AF3AHvXsxKmWhIDSQ93lwS2jOCVKttbENiIlNA3Bl7ls975XRE07FOL\n",
"k7uZVU25tgHOBL7PqLiVVN1xSgTnlxpYFxQ97OOAbUmlkk9249q+wBp81jtfHhhDSua3RPB0zQPu\n",
"cmxO7mZWoaLM7wjSw9MtI4txEr8iVbts30zT/iX2BI4FvhvR+fzOYlnhjUnJfEPS2Py0uvN/9JSZ\n",
"sE7uZlaRYhr8WcCXgC0iixckppUarhzBG6UGWAGJ4cBFwH4RXFG81x9Yi89650sBt/FZ7/z5cqKd\n",
"OSd3M+u2otzwGuB1YMfI4j2JhUjL9I6I4M5SA6yCxMrAjaTnB4sA65EmGk3rnd8bwZTyIuwaJ3cz\n",
"6xblWp6U/K4BjowsphZ129cD4yM4qtQAa0BiMeBQ4N+k3vlLJYfUbU7uZtZlyjVtrfSjIotzPn1f\n",
"7AvsBqzdU8acezsndzPrEuXaA/g58L3IYsyn74uVSPuNfr07lSZWX96sw8xmSrn6ACcCWwHrRBZP\n",
"fHpMzAZcChzhxN6cnNzNeiHl+hxwMWndlDUji9fanXICaT2Vcxsdm9WGk7tZL6Nci5Iekj4MbB9Z\n",
"TDeWLrERaeLPys1Uz27T61N2AGbWOMq1GnAv8Cdg9w4S+4Kkafu7RPB6CSFajfiBqlkvoVxbAmcD\n",
"P4gsrp7heCp7vA54LILDGx2fdY0fqJoZ8OlSAj8GDgI2jSz+2cmpewGLkYZkrMk5uZu1MOXqD5xB\n",
"WgRrrchiUofniRVJi2x9w/XsrcHJ3axFKde8pLH1D0iljh1u6dam7PHoCB5vYIhWR36gataClOuL\n",
"wD3AeOA7nSX2wnGkzSv+2IDQrEHcczdrMcq1DqnHPiqy+P1MzxUbACOAVVz22Fqc3M1aiHLtDPyS\n",
"tKLjrTM9N21Hdz6wW0/bKs+q5+Ru1gKKpQR+BowEhkUWE2Z6fip7/CNweQSjGxCiNZiTu1mTU65B\n",
"wAWkjZ+/Fll0Zb/Q7wNLk4ZkrAX5gapZE1OuhUkrN04G1u9KYpdYnrQK5MgIPqpvhFYWJ3ezJqVc\n",
"XyYtJXATaYz9w1leIwYAlwDHRPBonUO0ElU9LCPpXODbwMsRMbR4bz7gCmBJUonVdhHxZrX3MrNE\n",
"uTYlPQw9ILK4vBuX/gx4AZhpFY01v1r03M8jbTDb1hHA6IhYDri9eG1mNaBc+5PWiNmyO4ldYl1g\n",
"J2APlz22vposHCZpKeCGNj33x4BvRcRLUhoTjIgV2l3jhcPMukG5+gGnA8OAzSKLp7t8rZiPtMn1\n",
"nhHcUpcArSHKXjjs8xExbePZl4DP1+k+Zr2Ccs1N2uNUwNqRxVtdvjaVPZ4FXO3E3nvUvRQyIkJS\n",
"hx8PJI1q83JsRIytdzxmzUa5lgJuBO4EDowspnSzid2AZYEdaxuZNYKkYaRPa927ro7DMsMi4kVJ\n",
"g4ExHpYx6z7lWgu4mrTX6W8i695/sBLLAncDwyL4Tx1CtAYre1jmemAX4BfFv9fV6T5mLUu5vgf8\n",
"Gtgtsrip29eL/qSyx9yJvfepuucu6TLgW8ACpPH1Y4A/A1cCS9BJKaR77mYdKzbX+CmwB7B5ZPFw\n",
"Re2I44FVgM1cHdM6upo7vc2eWQ+iXANJZY7LkkodX6yoHfEt4DLSao8v1zBEK1lXc6dnqJr1EMq1\n",
"IGleyADS4l+VJvZ5gQtJ9exO7L2Uk7tZD6BcQ4D7gDHA9yKLDypqJ5U9/h74cwQ31zBEazJeFdKs\n",
"ZMq1IenB5yGRxYVVNrczMIRUyGC9mMfczUqkXD8ARgHbRhZ3VdWW+CJpIbH1Ihhfg/CsByq7FNLM\n",
"ZkK5+pJ2TNoE+EZkMbGq9j4rezzOid3Ayd2s4ZRrTuBSYHZgrcjijRo0+1PgDVJdvJmTu1kjFdvh\n",
"3QBMBPaJLCZX3aZYB9gTWNX17DaNk7tZY+0KDAT2iiymVtuYxDzARaTVHisqnbTW5AeqZg2iXPMD\n",
"/wE2iSzGVd1eKnu8BHgjgh9W2541Bz9QNet5fgFcXovEXtiBtLzAV2rUnrUQJ3ezBlCutUk7lg2p\n",
"SXtiaeA0YMMIKprwZK3NM1TN6ky5+pNmjf44sni76vZEP9JwzAkRPFhte9aanNzN6u9A0qbUf6pR\n",
"e0cD75K23DPrkB+omtWRci0OjAPWrHaiEoDE14FrgNUieKHa9qz5eFVIs57hV6QdlGqR2OcCLgb2\n",
"dmK3WfEDVbM6Ua5vA0OBkTVq8gzg1gj+XKP2rIU5uZvVgXLNDvyWNFnpw6rbEyOBrwKrVduW9Q5O\n",
"7mb1cTRwX2QxutqGJJYiPTzdOIL3q23Pegcnd7MaU64Vgb2AlatuK5U9XgycFEGtJj9ZL+AHqmY1\n",
"VGxufSZwbGRRi4eeRwIfAafWoC3rRdxzN6utHYC5SQm+KhJrAvuRyh6rXmTMehcnd7MaUa55gZOB\n",
"LSOLKVW1JeYkzUL9QQTP1yI+6108LGNWO8cD10UW99egrd8Ad0RwbQ3asl7IPXezGlCuNYCtqMHC\n",
"YBLbA2vhskergpO7WZWUqx9pYbBDq90yT2IJUq990wjeq0V81jt5WMasevsCb5LGyCsm0Ze0q9Ip\n",
"EfyrFoFZ7+Weu1kVlGsR0ubU60RW9Sp8hwNTgV9WHZj1ek7uZtU5FTgrsnismkYk1iAtDbx6BJ/U\n",
"JDLr1ZzczSqkXBsCawC7V9WOmIM0pLNfBJNqEZuZ13M3q4ByDQTGAz+KLG6qqi1xDqCI6v5IWO/g\n",
"DbLN6utwYHwNEvs2wDeBVWsSlVnByd2sm5RrGWB/qkzIEouT1mjfPIJ3axGb2TQuhTTrhmJhsDOA\n",
"EyOLisfHi7LHC4FfRVCLGa1m03FyN+uebYFFSNvnVeMQ0n9/v6g6IrMO+IGqWRcp11zABGD7yOIf\n",
"FbcjvgLcTCp7fLZW8Vnv0CMeqEp6Gngb+ASYHBFr1PN+ZnX2M+CWKhP7EsDVpLJHJ3arm3o/UA1g\n",
"WES8Xuf7mNWVcq0KjABWqrgNMRi4nTTOfmWtYjPrSCPG3D30Yk1NufoAvwOOiixeragNsSBwG3Be\n",
"BKfVMj6zjtQ7uQdwm6R/Sdqzzvcyq5c9SUOL51VyscS8wK3AtRH8vJaBmXWm3sMya0fE/yQtCIyW\n",
"9FhE3FXne5rVjHJ9HjgOWD+y6PZWdxJzAX8FxpAWGDNriLom94j4X/HvK5KuJa3D8Wlyl5Y4HSa9\n",
"WbwcGxFj6xmPWQVOBi6ILB7u7oUSnwNuBB4ADo6gnNI0a2qShgHDun1dvUohJc0O9I2IdyR9jvSx\n",
"NI+IW4vjwZcuuTTGj9yhLgGYVUm5hpEmGg2JLLo1g1RiIHAD8Dywuze4tlrpailkPcfcPw/cJelB\n",
"4D7gxmmJ/VNDL9+m6N2Y9SjKNQA4EziwgsQ+ALgKeA3Yw4ndylC3YZmIeApYZaYnLXknLHfjQbDZ\n",
"cfWKw6xCBwP/Ba7rzkUS/YBLgSnATl6b3cpS7vIDry9zJUP+dJDkcknrOZRraVJy3787uysV68Wc\n",
"D8wBbB/B5PpEaDZr5Sb3BR89hBWum4el7vhOqXGYFYqFwX4NnBpZPNXl61IH5ffAosDWEXxUpxDN\n",
"uqTU5B7Hvf8Sry13N0Oucu2v9RRbAsvQjX1Mi8T+K9Ls1S0ieL9OsZl1WfmrQvb9+EBWump5LX7P\n",
"CmWHYr2bcs1BStL7RhYfd+malNhPBNYGNo3gnTqGaNZlpSf3+N1DD/D2ov9lyFXVLqFqVq1jgL9F\n",
"FmO6cc1PgU2BjSJ4c1YnmzVK6ckdgPcWOprlr19fCzw2R9mhWO+kXF8CdiWts961a8ShwEhggwhe\n",
"q1NoZhXpGcl9mVuvJPq8w8oXeeMCa7g2C4NlkcVLXbpG7Af8gJTYu3SNWSP1iOQeWQQvrfwbvjh6\n",
"V5dFWgl2AWYDzurKyRJ7AIeREvtz9QzMrFI9IrkDMO///Yw5n+/Puj/Zp+xQrPdQrvmBE4AfRBaz\n",
"nHAkMZK0accGEXS5VNKs0XpMco8//HsKz65zDYvffXTZsVivciJwZWTxwKxOlNgaOBXYOIIn6h6Z\n",
"WRV6THIH4O1FDmThhwZro0O/WXYo1vqUa21Spcssl+KV2JQ0Lr9JBI/UOzazavW4DbK1/Tb30u+D\n",
"/nHJTV8pIy7rHZSrP/Bv4OeRxeUzPVesD1wObB7BvY2Iz6wzPWFVyMq8sNoBLH73qtp6p8XKDsVa\n",
"2gHAS8AVMztJ4hukxL6NE7s1kx7XcwfQDps9w9S+D8Rlf96q0XFZ61OuxYFxwNcji07HziW+CtwE\n",
"7BjBrZ2dZ9ZIzdtzB3hqvZzF79lMBy82sOxQrCWdDpwxi8S+MmkXpT2c2K0Z9czkfs+Pz+PllT7k\n",
"xZW9zrvVlHJtCqxMKn/s+BwxhLTv6X4R3NCo2MxqqUcm9wiCicPPYqFH9ipmD5pVTblmB34L/DCy\n",
"+LDDc8QypC0hD4vgT42Mz6yWem7iHLfbMXw4z+w8841dyw7FWsZRwL8ii1s6OiixJHAb8LMILmpo\n",
"ZGY11mOTe7y30Hs8utVfmf3VUWXHYs1PuVYA9gYO6vC4WBS4HTgtomvLEJj1ZD02uQPw8I4/YrZ3\n",
"FtUBy36j7FCseRW7K50JHBdZPD/DcbEQqcd+dgReetpaQo9O7vH6MhMZP3ICUwadWnYs1tRGAvMC\n",
"Z7Q/IDEfMBr4UwQnNjows3rp0ckdgIdHHsFcz66mn862VNmhWPNRrnmAk0kLg02Z7piYG7iFlNyz\n",
"EsIzq5uen9xfWuUvPDLyDV5ftst7Wpq1cTxwfWRxX9s3JeYA/gLcDxwaQTmz+czqpMcn9wiC8SNO\n",
"Zu5ntlCuecuOx5qHcn0V2Bo4crr3xSDgz8DjwP5O7NaKenxyB+DZdc7gic2m8sZSh5cdijUH5eoL\n",
"/B44PLJ449P3xWzA1cDLwJ4RTC0pRLO6aorkHsF7PLzDZQx644fKNaDseKwp7Au8A5/Vq0v0By4D\n",
"PgR2jmCWm3OYNaumSO4APLlZzv9Wm43359u17FCsZ1OuwcAxwL6RpZXxJPoCFwADgRERTC4xRLO6\n",
"a5rkHsHTPLDHP4k+WVG3bNaZU4GzI4sJABJ9gD8CCwPfjeCjMoMza4SmSe4APLL9T/hg/vmZ2mej\n",
"skOxnkm5NgTWAo4FKDZc/zWwPLBFBB+UGJ5ZwzRXco9+Y7n3gFd4f4Gflx2K9TzKNZA0UWn/yOL9\n",
"IrGfBHwN2DSCd0sN0KyBmiq5p7LIHY5HnwxRrlXKjsd6nMOACZHFtGV6RwEbkTa0fqu0qMxK0FTJ\n",
"HYCP5r6Qew+ayvvzekahfUq5liFtnXcAgMQRwHbAhhG8XmZsZmVouuQewfs8tPPZ9PtoeLFdmvVy\n",
"xQP23wInRRbPShwA7AlsEMHL5UZnVo6mS+4AvL34aYzbHSYPOqTsUKxH2AZYDDhNYi/gYGC9CGZY\n",
"AdKst+iRG2R36foFJ9zM3l9Zh/4fLhpZeDy1l1KuuYAJwAhGxVKk7fOGRTCx1MDM6qS5N8juileH\n",
"nMTE4VMI7Vl2KFaqHBjNqFiYVBmzkRO7WR2Tu6Thkh6T9KSkeqwJM5a7D3mFKbMd5iUJeqeiYmoH\n",
"zvn7aNKY+yYRTCg5LLMeoS7JXVJf0n9sw4EhwAhJK9byHhEEk9Y+iVeGiFQVYb1IsXH673hi04uZ\n",
"tPbpwOYRPFh2XGY9Rb167msAEyPi6YiYDFwObFmH+1zC2Kw/U2Y7yksS9Drf54N55uSy63cCto7g\n",
"/rIDMutJ6pXcFwUmtXn9XPFeTUXwPk9sdhYfzLcQsH6t27eeSbkW4pP+J3LB7YsQfUdE8PeyYzLr\n",
"aeqV3BtYgtPnTMaMGsgn/bzWe2/xzsLn8a+9Z+PF1XaK4LaywzHrifrVqd3ngbYTjBYn9d6nI2lU\n",
"m5djI2Jsd28UwdMasMNtDP/xMOX6cmTxcHfbsOahbUYcwmIDN+bug3eK4Kay4zGrN0nDgGHdvq4e\n",
"de6S+pG2MFsfeIG0T+WIiHi0zTlV1blPfz+Gsd5PrmCd42+OUbFrLdq0nkeDH9ickZtfx/gRR8et\n",
"vzyx7HjMylBqnXtETAH2I+0sPwG4om1ir4M7ufeAV5naf2vlqvnYvpVPYjO+9psrmTz7TU7sZrNW\n",
"tzr3iLg5IpaPiGUi4oR63Sfdi+D9hU5nwndfBfav572s8SS2Z5mbL2LoZW8x/8Sdyo7HrBk07fID\n",
"M7bH7Mw7cRL7Ly/6TF0ysninVm1beST2YMDbx3HYgp/Q7+O9Iou/lB2TWZlaf/mBdiJ4nzeWOZvn\n",
"13gZ2KPseKx6EgcCx7D/CqPp9/HtTuxmXdcyyb1wJrecujChHytXvSqBrM4kJPETYD92HH4wc/5v\n",
"A+CgsuMyayYtldwjeIbn1rqDtxb/kLQMrDWZYmu8E4Ht+cKtG7HMLT8HfhhZeMMNs25oqeRe+DWj\n",
"TxpIcIxyLVZ2MNZ1En2AM4H1gGHsvPEPgAcji2vLjcys+bRicr+T/2z7Js+teS/wb+VyD74JSPQD\n",
"LgBWAtZnlL4A7Iqrn8wq0nLJPYKAPr/mnHuW4oXVRgInKNe5yjVn2bFZxyRmA64EFgSGM0ofAecB\n",
"B0UWL5UanFmTarnkXrgEeJaz/n0VJ718O+8t8DlgnHKtWXZgNj2J2YHrSesRbRnB+8BRwH+By8qM\n",
"zayZtUyde8f3YFHSTNk9Wf13E9nkwOXoM+V0FD+PLKbU8942axJzAzcCTwG7RzBFuVYGbgNWiSy8\n",
"B6pZO13NnS2d3D+7F3MAuzD3M4ewzYj5mPe/LwKbxckvPtmI+9uMJOYnLU9xH7B/BFOL8tX7gDMi\n",
"i3NLDdCsh+p1k5hmJoJ3IziDt5ZchvPH7Mq/9+xHn8mPaZMDL9WgN+cuO77eRmIwcCeph75fBFOL\n",
"Q4cAr5HG282sCr2i597h/df/yQ58+aIzeGH1Qfz1tD/y9hInR/BMWfH0FhJLkZL6uRH8/NP3c60I\n",
"3AWsHlk8XU50Zj2fh2W6EkOuQby34Jn0mfJdrrwyeGqDvwKneMu2+pBYHhgN/DKCX3/6fq6+pMR+\n",
"SWRxRlnxmTUDD8t0QWTxQZz08m4MemM7dt7oPXYcPj99P7pS4u8SW0v0LTvGViGxMjAGGNU2sRf2\n",
"B6YAv2t4YGYtqlf33NtSrgWBcwgW5ZqLLmT8jiNIddenA+dF8G65ETYviTWBP5MenF453bFcXyQ9\n",
"RF0rsvADbrNZ8LBMBZRLwF7AcYR+yrEfPczU/gcD3wLOBn4bMeN2gdY5iXVJE5R2bb8tnnL1AW4H\n",
"bowsTikjPrNm42GZCkQWEVn8AVgHxZ4cM+AIRmkf4GvAIOBhiYskVi030uYg8W3gCmC7TvY73ROY\n",
"nfTpyMxqyMm9A5HFY8BawH+ABxml5SI4EPgiMB64QWKMxGbFYlfWjsS2wLnA5hGMmeF4riWA44Dd\n",
"I4tPGh2fWavzsMwsKNcw0oJW1wOHRRYfSPQHtgMOJvU8TwMuKqbO93oSewGjgOERPDzD8TT89Rfg\n",
"H5HFcQ0Oz6ypeVimRiKLscAqpIer/1KulSOYHMElwFeAHwDfBp6WOFZi4fKiLZdEX4lfkiYjfauj\n",
"xF7YGVgY+EXDgjPrZdxz76Kit7kjcCpwAnB6ZDH10+OphvtHwPeAa4HTIhhfRqxlKJZ4uASYG9g6\n",
"gg4311CuwcBDwMaRxbgGhmjWElwtUyfKtTRwMfA+sGv7xa0kFgD2Ji1Y9jDpj8GtaSni1iSxGHAD\n",
"MA74QQQjRSn2AAALN0lEQVQfd3he+gN5LfBIZPGTBoZo1jI8LFMnkcVTpNLIvwEPKNfW0x0PXo3g\n",
"eGAp0pK1JwPjJXaXGNjoeOtN4ivAvcDlwB6dJfbCdsBywLGNiM2sN3PPvQrF+vAXkxbBOjCymGGi\n",
"U7En6AbAj0lj92cCv4/glUbGWg8SWwFnAXtHcM1Mz02TxB4GvhNZ3NeI+MxakXvuDRBZ3Auf1ryP\n",
"U641ZjgniAhGR7AJKckvCTwh8QeJFRoYbs1ISOIw4DfAJrNK7IVfAZc6sZs1hnvuNaJc3yX1yn8D\n",
"nDCz2m2JzwP7AvsA9wOnAGObYVxeYgBpDZjVSDXss5yxq1xbkn7GL0cWLhc1q4IfqJZAuRYDzgcG\n",
"AjsV4/Odny8GATsBBwEfkh6+XjGLcevSSMwHXA28A4zsyno7yjUv8AgwMrK4s84hmrU8D8uUILJ4\n",
"DtiIVBFyv3LtWFSIdHx+8EEEZwErAUcDuwJPSRxRJNIeQ2JZ0oPTB4CturGQ2inAdU7sZo3lnnud\n",
"KNcqwKWkmu59Ios3u3SdWIXUk9+cVDf+qwgm1i3QrsU0jLRGzE+LP0Zduy7XxsAfgKGRxTt1Cs+s\n",
"V3HPvWSRxYOkGayvAg8p17e6dF3wYAS7AENJwx/3SFwj8Y2i8qahJHYjreq4QzcT+5ykxL6XE7tZ\n",
"47nn3gDKtSlpyeALgCyy6PKYusTnSMM1BwGvk4Y5ro5gSh1CbXvfPsDPgW2BzSJ4tFvX5zoDGBRZ\n",
"7F6P+Mx6Kz9Q7WGUayHgHGAwsENk8Xi3rk+7Qm1OqpdfEvg1cHYEb9U8VjE7cBGwEGl8/dVuXZ9r\n",
"A9KD5aGRxRu1js+sN/OwTA8TWbwMbEFK8H9Xrr1m9rB1huuDTyK4LoJvAtsAq5Mevp4qsWSt4pRY\n",
"hDT79n1ggwoS+zdJzxp2dmI3K4977iVQrhVJD0ufBfaMLCqarSqxBHAAsBtwG3BqBBVPEioe5l5P\n",
"mnV6fHfr7ovlka8ERkQWt1cah5l1zj33HiyyeBRYE3gceFC5hlfUTvBsBIcASwP3AFdUurm3xObA\n",
"aOCQCI6rILGvR0rs2zuxm5XPPfeSKde6pAet1wKHRxYfVtyW6AdsRdpEpEubexcVOAeR1mDfqpKe\n",
"fzHGfimwrevZzeqr1J67pFGSnpM0rviqqGfaG0QWY4CVSQ9a/6lcQytuK5gSwZ8iWJO09vy3SJuI\n",
"/KJYlnc6xY5SvycN66xVYWLfiJTYv+vEbtZz1KXnLikD3omIU2dyjnvubRQPV3cGfkkqQfxV281A\n",
"Km5XfIE0Lr8zaWu7UyN4QGIe4E/AZOB7EbxdQczDgQuBrSKLf1Qbq5nNWqmlkEVyfzciTpnJOU7u\n",
"HVCuL5CWEX6XtBnICzVpNyXzPUmJfiJpm7vRwI8rqZlXrm8D5wFbRhb31CJGM5u1npDcdwPeAv4F\n",
"HBwx/fR7J/fOKVc/0loz+5CWLri2Zm2noZhtASK4tML4NieVdG7uJXzNGqvuyV3SaOhwM+ijSQtM\n",
"TSvvOxYYHBF7tA8QyNu8NTYixlYUTItSrrVIvfg7gIM62gyk0Yrle88CNoss/ll2PGatTtIwYFib\n",
"t7IeMUNV0lLADRExtN377rl3QbFGy6+BtUkzW0tLqMWWgmcCm0YWD5QVh1lvVvawzOCI+F/x/UHA\n",
"VyNiZCUBWqJc2wK/JSX6E2e2GUid7r9Ncf9NIotxjby3mX2m7OR+IWm/0ACeAvaOiJcqCdA+U2wG\n",
"ciHQj7QZyDMNuu/2pJr54ZHFQ424p5l1zAuHtSjl6kOapHQoaRz+kjrfbyRpJcqNIovx9byXmc2a\n",
"k3uLU65VSevTjAN+2NXNQLp5jx2Bk4ANI4v/1Lp9M+s+ry3T4opx79WBN0nr06xTy/aVaxfgF8AG\n",
"Tuxmzcc99xagXJsBfwTOBUZFFpOrbG934GekxP5YDUI0sxpxz70XiSxuJD3AXgX4h3ItV2lbyvV9\n",
"0vyD9ZzYzZqXe+4tpFifZh9Scj4KODuyrv8PrFx7kyahrRdZlLopt5l1zA9UezHlGkJ62Po0aTOQ\n",
"We6mpFz7AocB60cW/1ffCM2sUh6W6cUiiwmkzUAmkh62bjSz85VrP1Jp5bpO7GatwT33FlfskHQB\n",
"cBVwZPvNQJTrQOBA0lDM042P0My6wz13AyCyuIO0GchiwP1tNwNRrh+TlgAe5sRu1lrcc+8lioet\n",
"uwAnA8cBs5HWd18vsphUZmxm1nV+oGodUq4vkpYRnp80xv58ySGZWTc4uVunlKsvMCCy+KDsWMys\n",
"e5zczcxakB+ompn1Yk7uZmYtyMndzKwFObmbmbUgJ3czsxbk5G5m1oKc3M3MWpCTu5lZC3JyNzNr\n",
"QU7uZmYtyMndzKwFObmbmbUgJ3czsxbk5G5m1oKc3M3MWpCTu5lZC3JyNzNrQU7uZmYtyMndzKwF\n",
"ObmbmbUgJ3czsxbk5G5m1oIqTu6StpX0H0mfSFqt3bEjJT0p6TFJG1UfppmZdUc1PffxwFbA39q+\n",
"KWkIsD0wBBgOnCmp5T4hSBpWdgzVcPzlcvzlaebYu6PipBsRj0XEEx0c2hK4LCImR8TTwERgjUrv\n",
"04MNKzuAKg0rO4AqDSs7gCoNKzuAKg0rO4AqDCs7gEaoR496EeC5Nq+fAxatw33MzKwT/WZ2UNJo\n",
"YOEODh0VETd04z7RrajMzKwqiqgu70oaAxwcEQ8Ur48AiIgTi9d/BbKIuK/ddU74ZmYViAjN6pyZ\n",
"9ty7oe2NrgculXQqaThmWeD+SoIzM7PKVFMKuZWkScCawE2SbgaIiAnAlcAE4GZg36j244GZmXVL\n",
"1cMyZmbW85Rafy7pWEkPSXpQ0u2SFi8znu6SdLKkR4uf4RpJc5cdU3fMbCJaTyVpeDE57klJh5cd\n",
"T3dJOlfSS5LGlx1Ld0laXNKY4v8zj0g6oOyYukPSQEn3FflmgqQTyo6pEpL6ShonaaZFLWVPLjop\n",
"IlaOiFWA64Cs5Hi661ZgpYhYGXgCOLLkeLqrw4loPZWkvsBvSZPjhgAjJK1YblTddh4p/mY0GTgo\n",
"IlYiDcf+sJl+/xHxIbBukW++DKwr6Rslh1WJA0nD3jMddik1uUfEO21ezgG8WlYslYiI0RExtXh5\n",
"H7BYmfF010wmovVUawATI+LpiJgMXE6aNNc0IuIu4I2y46hERLwYEQ8W378LPEqa19I0IuL94tsB\n",
"QF/g9RLD6TZJiwGbAmczfSHLDMruuSPpeEnPArsAJ5YdTxV2B/5SdhAtblFgUpvXniBXEklLAauS\n",
"OjVNQ1IfSQ8CLwFjigKQZnIacCgwdVYn1j25SxotaXwHX5sDRMTREbEEcD4p8B5lVvEX5xwNfBwR\n",
"l5YYaoe6En8T8dP/HkDSHMBVwIFFD75pRMTUYlhmMeCbzbTOjKTNgJcjYhyz6LVD7ercOxURG3bx\n",
"1EvpgT3fWcUvaVfSx6T1GxJQN3Xj998MngfaPnRfnOmXurA6k9QfuBq4OCKuKzueSkXEW5JuAlYH\n",
"xpYcTld9HdhC0qbAQGAuSRdGxM4dnVx2tcyybV5uCYwrK5ZKSBpO+oi0ZfGwppk1w6SyfwHLSlpK\n",
"0gDS6qPXlxxTryFJwDnAhIg4vex4ukvSApLmKb4fBGxIE+WciDgqIhaPiKWB7wF3dJbYofwx9xOK\n",
"IYIHSSu1HVxyPN31G9KD4NFFadKZZQfUHZ1NROupImIKsB9wC6la4IqIeLTcqLpH0mXA3cBykiZJ\n",
"2q3smLphbWBHUpXJuOKrmSp/BgN3FPnmPuCGiLi95JiqMdNhSk9iMjNrQWX33M3MrA6c3M3MWpCT\n",
"u5lZC3JyNzNrQU7uZmYtyMndzKwFObmbmbUgJ3czsxb0/6B5+DGF3R6VAAAAAElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x10586f5d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"##This function provides an interactive mode to change polynomial degree. \n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" X = construct_poly(x,n) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(x,X.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" plt.plot(x,y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
]
},
{
"cell_type": "code",
"execution_count": 308,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 1.01010000e+04, 1.01010000e+03, 1.01010000e+02],\n",
" [ 1.01010000e+03, 1.01010000e+02, 1.01010000e+01],\n",
" [ 1.01010000e+02, 1.01010000e+01, 1.01010000e+00]])"
]
},
"execution_count": 308,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"A.T.dot(A)"
]
},
{
"cell_type": "code",
"execution_count": 309,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import scipy\n",
"from scipy import linalg\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 310,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1\n",
"[[ 9.80101960e-03 9.80101960e-04 9.80101960e-05]\n",
" [ 9.80101960e-04 9.80101960e-05 9.80101960e-06]\n",
" [ 9.80101960e-05 9.80101960e-06 9.80101960e-07]]\n",
"True\n"
]
}
],
"source": [
"A = np.array([[10,1,0.1],[10,1,0.2],[10,1.1,0.7]])\n",
"a = np.array([[10,1,0.1]])\n",
"A = a.T.dot(a)\n",
"\n",
"B,rank = linalg.pinv(A,return_rank=True)\n",
"print rank\n",
"print B\n",
"print np.allclose(A,A.dot(B.dot(A)))"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd4FOX2wPHvCYQOIlKliAj2q1iwITLYG6jYRUERLFgQ\n",
"uPayWXu5oohXvYoi4hUVsaGCgDKAP71WimADlV6V3kvO7493AktI2SS7mc3mfJ4nT7I7s++c3U1O\n",
"3j3zzvuKqmKMMSZ9ZIQdgDHGmMSyxG6MMWnGErsxxqQZS+zGGJNmLLEbY0yascRujDFpxhK7iYuI\n",
"3CkiL4UdhzGmcJbYS5mIzBaRTSKyR677J4tItog0C243EZERIrJMRFaKyI8i0i3Y1jzYd02urwvz\n",
"OaYvIleXJG5VfURVe5akjZh4OojI+OB5/VnIvl1yPcd1wXM/LNieJSJbYravFpHmMY9/IHjttohI\n",
"JFfbZ4nIFyKyQkQWichLIlIjZvvjIjI3aHO+iPQXkYrBtnZ5vP7ZInJenK9Bid+ToB1PROYVsk8f\n",
"Efk9eB5LRGSwiNSMo+37gud0Ysx9FUVkYPB6/S0iH4rInjHbjxORb4JjTRWRtrnavExE5ojIWhF5\n",
"T0R2j9n2avC3EfteSsz240XkWxFZFTyfhPw+piNL7KVPgT+AS3PuEJF/AFWDbTmGAnOAZkAd4Apg\n",
"Sa62dlPVmjFfwws4Zr5yklUpWgsMAm4tbEdV/W/scwR6Ab+r6uScXYBhMfvUUtXZMU3MDI7zMbu+\n",
"DrWA+4FGwAFAY+CJmO0vAweqai3gKOBUoEcQ16RccZ0dPK/Rcb4GpXll4AfAkcHz2B/3O3V3QQ8Q\n",
"kX2AC4CFuTb1AtoBhwB7AiuAgcFj6gAjgceA3YDHgZEiUjvYfhDwAtAFaACsB56LaVuBx3K9lxo8\n",
"tgLwHvCiqu4GXAz0F5FDiv5ypD9L7OF4Hegac7sb8BogMfcdCbyqqhtUNVtVp6hqvEljOxF5CPeH\n",
"+GzQC3omuD9bRHqJyEzg1+C+AUEPdZWIfCcix8e0kyUiQ4Ofcz4xdA16X8tE5K54Y1LVb1X1v0CB\n",
"vfV8XIl7rbaHxs6vW+5jvRa8bmty76eqw1R1jKpuVNWVwEtA25jtv6rq2pjjZAOLCohruKpuKOwJ\n",
"FPCe7C8iY4Oe8C+xn8BE5EwRmRHz6aGviFQDRgF7xvRwG+bxGvyhqiuCmxmFPI8czwK3A1ty3X8Q\n",
"8KmqLlPVTcDbwX0AxwGLVXWEOv8FlgGdg+1dgA9V9QtVXQfcC3QWkeqxL08+8TQA9sB1eFDV74Cf\n",
"cf+QTS6W2MPxP6BW8IdcAdf7eD2PfZ4TkYslKM/kId+ElkNV7wYmATcEvaCbYzafA7QBDgxufwMc\n",
"CuwOvAEMF5FKOU3l0XxbYF/gJOA+EdkPtn9kXpHH/iUiInvhEmJsYlegY5AMp4vIdSU4RHtgeq5j\n",
"3iEia4B5wEeq+kEecVUHzgeGxHOQvN6ToI2xuN+DesAluPd//+BhLwPXBL3ug4DxqroeOB1YGNPD\n",
"XZzXMYMSyCpcol2mqgPyiy/4h7JRVUflsXkMcIaINAr+sXQBPing6WawI/EfBEyNeR3+ADbhfody\n",
"9Arey+9EpHPMvguBaUB3EakgIscBewFfFHDscssSe3iG4nrtpwA/AQtybb8Q98d/L/CHuBr8kbn2\n",
"+UtcfTjna78CjpfXP4FHVHVl0PPKKXusCD4h9AcqA/sV8Pioqm5S1Wm4P9jWQTtfqOrueexfUl2B\n",
"iao6J+a+t3HlhbpAT9w/mEuK2rCInBK0f1/s/ar6aFBqOQLoEptsYnTGJcuJRT1szM9nA3+q6pCc\n",
"T2jAu8BFwfbNwEEiUktVV8WUogr95x48jzeCEsa+wAEi0ifPgFzt/SGgdz7tjAAm435fV+F+Px4I\n",
"Nn8FNAo6I5nizgm1AKoF26sHj4m1Gsip9z8DtMT9Y7sXeDVI4DmuAaLARmACcJeq5v67MVhiD4vi\n",
"EnsX8i7DECTcO1X1YNzH0CnA+7na2UNVd4/5+rWQY+a200k3EfmniPwk7qTmClydtG4Bbcb2Dtfj\n",
"/nCTqSu5esWq+rOqLg4++n8FDMDVhuMmIscA/wXOV9VZee0TJNLncOc6cst5D4sq9j3ZCzg69h81\n",
"cBnuvQf3ieBMYLa4E6/HFON4BM/vUXYuBcbKAoaq6tyY+2JPYP4Ll4jr4N7v93DlIFT1b+BcoB/u\n",
"d+M0YBwwP3j4WtzvVKzdcGUyVHVyTMdiFO496RwctzHwEXCZqmbiev+3i8iZRXwJygVL7CEJ/nD+\n",
"AM7A9cwK2vdv4ElcLbU4PeH8TtRtv19E2uFOMl6oqrWDHvcq4uwRJlswuqIR8E4JmtnldRA3uuYD\n",
"4EpVHV/I4zOBdbke3xRXwilqYs8dy1xgQq5/1DVV9QZwNWVVPRfXm30f90klz+cUh0zcP+K8nAjc\n",
"LG7UyyKgKfC2iOSc6D4dGBx0PDbjavFHBSdOUdWJqnqUqu6B++exP67EBzADV+oDtp+grQT8FkfM\n",
"xwHzVXVscJzfcCfEz4j3SZcnltjDdTVwYl4n3ETkMRE5SNzwsprA9cDMmJNgEH/SXQLsU8g+NYGt\n",
"uPJOJRG5DzdqpCjiikecKrgEIyJSOaaWn59uwDvBSbfYts4Rkd2DNo8CbsYl6pztFYNjVQAyRaSK\n",
"iGQE2w7GjWK5UVU/ydWuiMi1IlI7pu1e7PpP+Arg/1S1qCeCc78nHwH7isjlQRkjU0TaBOdhMsUN\n",
"+9xNVbfherjbYtrZQ0Tyfa9EpIeI1At+PhC4AxiRz+4n4XrDh+JKawtxJZB/B9unAd1EpJaIZOJe\n",
"kwWqujxo/7Ag3lrAv4C5OckY1wPvGJyDqY4r4YzIeU9F5AIRqSEiGSJyKsHJ1uCxM4D9xA2VleCf\n",
"wtnE1OxNDFW1r1L8wo0EOTGP+yvi/libBbefwfVk1gBLcb/g+wXbmuNGNqzJ9XVLPsc8BjfyZTnw\n",
"dHBfNtAiZp8M3Am6Vbg/5ltxnyhODLZHgNdijr8NyIh5/Hige/BzO2BNAa+BFxw/O2gnG/g8Zvt0\n",
"4NKY21Vww+o65NHWG8BfwfP/GZekY7e/GnOsnK+uwbbBuH9msa/hjzGvxyjgb1wd+Mec55er/Z+B\n",
"q4rxe5DXe7IvLsEvDZ7TONywwswgluXB+/M1cFxMWy8H+y8HGuZxrFdwpZE1wTFvAyS/17ug31dc\n",
"aeht3EnYFcBE3FDK2PdjZfA1DKibq71LccN41+LKOLVjtk0MHrcKV8e/KNdju+LOR63GlREfCfvv\n",
"OVW/JHjBiiX4GPoaUB/3kfBFVX0m+Fj2Fq5uODt4g1YW+0DGGGPiVtLE3hDXQ5gi7oq973EnT64C\n",
"/lLVx0XkdmB3Vb0jIREbY4wpUIlq7OpGI0wJfl6L+1jaGOjEjtELQ3DJ3hhjTClI2MlTcfNzHIar\n",
"/zVQ1ZzL35ewY8iWMcaYJEtIYg/KMCOA3qq6JnabulqPrZhtjDGlpMSTPwVDnkbgLmrIuYBmiYg0\n",
"VNXFItIId5Y/9+Ms2RtjTDGoaoFDi0uU2EVEcEOtflLVp2M2fYgbd/xY8D33FZNxBZcKRCRLVbPC\n",
"jqMwFmdiWZyJVRbiLAsxQnyd4pL22NsClwPTRCRn7oo7cZcsvy1uvunZ7JjvwhhjTJKVKLGr6hfk\n",
"X6c/uSRtG2OMKR6bUqBwftgBxMkPO4A4+WEHECc/7ADi5IcdQJz8sAOIgx92AIlSoguUSnRgES0L\n",
"NXZjjEkl8eRO67EbY0yascRujDFpxhK7McakGUvsxhiTZiyxG2NMmrHEbowxacYSuzHGpBlL7MYY\n",
"k2YssRtjTJqxxG6MMWnGErsxxqQZS+zGGJNmLLEbY0yascRujDFpxhK7MabMkahUCjuGVGbzsRtj\n",
"ygyJyr7AfcDFwHfAcGCERnROqIHlR6QOqssT22ThudMSuzEm5UlU9sYl9I7AAOA5oA1wAXAu8Ds7\n",
"kvyfYcW5nUhV4AngKOBoEphoLbEbY8o0iUpT4B5cAv830F8jujLXPpmAB1wInAfMBt4B3tGI/l6a\n",
"8bqApDXwX2AacD26c7wlb94SuzGmDJKoNALuAroALwJPaET/juNxFYH2uH8EnYEFuJ78OxrRmcmL\n",
"GBDJAPoCtwN9gP8msqe+4zCW2I0xZYhEpT4uMV4FvAo8phFdUsy2KgAn4JL8+cBiXE9+uEb014QE\n",
"vP1g0hgYAlQBrkCTVw6yxG6MKRMkKnsAtwLXAG8AD2tEFyaw/QpAW1y55nzgb1xPfrhG9OeSNS7n\n",
"42r+A4FHUd1asmgLO5wldmNMCpOo1MaVL27A9aYf0ojOTfIxM4DjcD35C4BVBOUaYIZG4kyKIjVx\n",
"J3LbAZej+nVSAt7lsJbYjTEpSKJSC+gdfI0EHtCI/hFCHBnA0bie/AXAOoJyDfBjvkle5BjgdcAH\n",
"bkF1bWnE6w5tid0Yk0IkKtWBG4F+wBjgfo3ob+FG5UhUBDc8MSfJb2ZHT36KRlQRqYg7qXsD0AvV\n",
"EaUepyV2Y0wqkKhUBa4DbgMmAVka0Z/CjSp/QZI/EpfgLwSyj57P2E+HcmytTfwl0A3VBaHEZond\n",
"GBMmiUploAeul/stENGITg03qqLJ8kTaz+GeNgu489F2rH/keFZnZ2wv13wXd00+QUolsYvIK8BZ\n",
"wFJV/UdwXxbuzVwW7Hanqo4uanDGmLIpuGjoKtzFRT8C92lEvw83qmIQ2R14HvgHcJlkMQ04lB09\n",
"+coEF0MBX5dGki+txN4OWAu8FpPYI8AaVe1fkuCMMWVLcIHQ5bjL/3/HJfSvwo2qmEQ83Nj0D4Hb\n",
"UN2w02ZXrjkYl+AvBKoDI3A9+f9pRLOTE1YplWJEpDkwMldiX6uqT5YkOGNM2RCME78YiOAuBLpP\n",
"Izoh3KiKSaQScD/QFbga1VFxPSwqB7GjJ1+bIMlrRL9IbHiF586KiTxgLjeJSFfcDGz9NMHzJRhj\n",
"whcMF+wMRIHVuNEin5V23TlhRPbHzfOyAGiN6tJ4H6oRnQHMAKISlQNwSb4nkNDEHo9k9djrs6O+\n",
"/gDQSFWvzvUY67EbU0YFZYhOuIS+FbgXGF2GE7oA1wIP4s4L/CcZ87wkQmg9do35Lycig3AXIOwi\n",
"OMmaw1dVPxnxGGMSI0jop+NKFZVwtfQPy2xCBxCpB7wMNAGOR/WXkCPaibhav1ekxySpx95IVRcF\n",
"P/cB2qjqZbkeYz12Y8qIIKGfiPsEXhtXSx+RrBOEpUbkDFxSfw24D9XNIUdUqFLpsYvIMNw0mXVF\n",
"ZB7uDffEzUmswJ+4jzjGmDJIotIOl9D3BLKAtzSi20INqqTcQhiP4RbpuIw0qxbYBUrGmDxJVI7B\n",
"lVxaBt9f10jRZy4UoSawTpXU6N2LHIo7QToDuA7VFSFHVCRhj4oxxpRBEpUjcIn8H7iTia9qJP4S\n",
"hQgCtMYtY9cRN9ZbRJiPW91oNjAn1/eFqiR1uttgIYzeuKtg+wFDU/UEaUlZj90YA4BE5RDcKJej\n",
"gEeAlzSim+J6rFAVV4M/O/jaiBs0MRI33K8i0AxoDuwVfI/9uR5uiGFOop/Nzsl/nipbiv/kZE/c\n",
"xUbVcVPslvpMkolic8UYYwolUTkQVzs/AVd3fkEjO19lmefjhIa4JN4R6ABMYUcy/1WVuJOLCJWB\n",
"puxI9Lm/NwKWkHdvfzYwV5W8/wmJdMZNC/Bv4OFkL4SRbJbYjTH5kqi0wg12OBV4EnhWI7ou3/1d\n",
"ieVQdpRYWuGm3h0JjFKl0DVJix2rkAk0ZteEn/NzE9yqSNuTfmPmLxrHyWfszZ8HLqLRFc119gQA\n",
"8f1KwN5B/LFftdTzjk7Wc0gUS+zGmF1IVPbGXVDUCbcC0ACN6Oo89xWq4HrjHXG98y3s6JVPUiUl\n",
"hgeKUAHXq28O7HVJ5uvtbq73xGUfNT9xc/9m12VvrC91aLphG83WK7tvzmRdxTWsrbiATRm/ky3T\n",
"2W3Lt9TfNE097/dwn0nhLLEbY7aTqDQF7sbNZfJvoL9Gdp3qQ4QGuBlbO+Lq5tPYkcx/KUqJJZnE\n",
"9zNwPfVWwL5AK1FttceqVcesqVatbsVt2/5aV7XqVOA3spnFn9WXMqbhZj5pmMnazCbsWvJZqEqr\n",
"cJ5N/CyxG2OQqDTCjQTpQuV6Q2jV7y32OLoK0HCnr3UVWvB3pdYsrlKVtZl/Ivole60fSYt13wPz\n",
"1PNKfey6+L7geuI55ZJ9Y35uAawAZgIzmy9atCxryJBz9587d/XWjIwubadPj7v3HZSZdlMl5ee0\n",
"ssRuTDkgvl8VaEDuRL11XXM2zD8e1eZUbbSOirUqI7IJN/viji9lMZ82bMGkuhdRe/NL3PD7V1Tb\n",
"1oKd6897AH8QJNHg67fg+0L1vGInkiB518t1vJwE3hK3Dmlex52lnufOCYh0AZ7Gnfztj5bxK2IL\n",
"YIndmDQlvt8UN3zvcKAqsYl62/oV/P2/fVg9/TCkwkTqHPs8ux8+HViinrd+p3aEWrgRI4cBl6gy\n",
"LZ/jVccl2dwnHPcFagCz2Dnp5nwtzUn64vt18nhszs9bcz0up61Z6nmr8n8hpDbwHG7cfBdUJ8f1\n",
"ApZhltiNSUPi++2At4CngJeAVep5KlGpDfTBLRb9DvCQRnRuvu0IRwNvAGOBvqqsz2/fQuKpRd4J\n",
"vxVu/Ppc3HQEmeya+GcCM9Xzij6iRuQE3BwvH+EWwihW/GWNJXZj0khQsrgON+a8q3repwASlZq4\n",
"Kyp745LcAxrJ/wIcETJwi0r3Aa5X5d0kxlwHd2HSImJ670VvSASoiztZ2hg3UucyoAeqHycm2rLB\n",
"phQwJk2I71cGBgLHAW3V82ZJVKrjFrboh+t1t9WI/lZgO8KewFBc77mNKvn26BNBPW85sLzAnUQq\n",
"4k6QNmZH4m6S6+c9cbX2+cHXLNxCGEuSFXtZZondmBQnvt8IV1pZAhzLhA5bZQK3ALcDk4AOGtGf\n",
"Cm1HOBsYhKupP5T0uVncQatScMJujDtxugw3pUBO4l6Au5J1wfbbWvjVsMaxxG5MChPfPwq3duZL\n",
"TL7hcVb/dDVwJ27JydM1olMLbcNdZPQ4cA5wgWoClmpzpZHd2DVJ507c1YGF7EjWOb3tCexI4ovL\n",
"+mX+qcYSuzEpSny/G/AE2zZcxxdn7gH8iptq9lyN6HdxtSEcALyJO0nZWpXCp6h1syDWI+/edex9\n",
"CswjtlcNPwAfxtz3V7rOoJjK7OSpMSlGfD8T+BeqZ/Dzg4NZ9nkP3Bjy+zSiX8XVhrvgpgfwMO7i\n",
"pEGqKCKV2LmenVfiboRbmDq2LBL7Pac0kuc0BCa5bFSMMWWM+H5dVN9m05I6fH9NdbauWYhL6BMK\n",
"f7BUBxr/zP77D+aqu2uzslk3hoxvzMJq7EjidXC1+tz17NjvC1HdmJQnaErMErsxZYh89mlryP6U\n",
"xaMzmPXsLMi+F/hMswDYnYJPQDYBqqyn6t9TaF13GxVmH8U3H1Vm8xx2TuJL0DK+rF05Z4ndmFQn\n",
"UmFDRRp40WsenXL4WV1O/fTfyyNvjRl/xCI2y86JezMFlEV+4oBFBzP9eiXjeqCnKiNDekYmyWwc\n",
"uzGpQqQNbtH37T1uhcZbMjIa3dHz6oz5LTvIsMhdM877ZsYkybuevTb/pmmGGw65BThclYVJfz4m\n",
"pVmP3Zhkc4snjwNeJ0jWj7el/otH1er+R7usllqt6Ryk4ol66rlLi9405+PGpfcHnlDFyixpznrs\n",
"xoRNZHfcOPTeqL4hUWkH3E/1FnvTekBlKlQbjGT0U88r0nqeIlTEzWZ4BtBRla8TH7wpqyyxG5Ms\n",
"bjz468BHksXPRGUM0JK9r/mQppcchMjt6nmDi94slXGTd9XClV7yn/3QlEuW2I1JnvuAmvvczL+B\n",
"L5AKWRw/qgEZmVcDHdXzitzLFqEG8B6wCjg73wWcTblmid2YZBA5G+gxpQHH/lGHEVSq259jh5+M\n",
"u/jnKPW8RUVvkjrAJ7irT68tlbleTJlkJ0+NSTSRlsCXwDmSxUVU3+cgjnhpT0S+Am5UzytyL1uE\n",
"RsAY4FPg1lRZd9SUvnhyZ0ZpBWNMueCu/nwPyJIsGlDnmEs44sXWiDwHXFPMpL4P8AUwDEvqJg5W\n",
"ijEmUdyMh4OA7yvfmzmKZhdNY69um5CMzup5E4vXJP8ARgMPqvJ8IsM16csSuzGJ0xvYr2uvs0/f\n",
"fMgxP1Fz/5VkZLZVzyvWYhYiHAN8ANyiyrCERmrSWolLMSLyiogsEZEfY+6rIyJjReQ3ERkjbsFZ\n",
"Y9KXSHvgznu6d79xaMcuP5JRaQOSsV8JkvopwEjgKkvqpqgSUWMfDJye6747gLGqui/wWXDbmPQk\n",
"0hgYdts11zz+0GUXfcSijyvyU9Yheup5xVsc2l1N+l+gsyqfJDRWUy4kZFSMiDQHRqrqP4LbvwDt\n",
"VXWJiDQEfFXdP9djbFSMKftEKmeDf13fvsteOvvMY5l+d0WWf32aRvSb4jVHd+BB4CxVJic2WJMO\n",
"wpxSoIHuWGR2CdAgSccxJlRrq1QZcNk99zQaedyxtfiu+0LWzxlcgqTeD7gJ8FQpcFFqYwqS9JOn\n",
"qqoikufHAhHJirnpq6qf7HiMSZQvDzqo900DBnSbvvfeY/nq/HlsWdkYGFDUdoLVjh4EOgPtVJmX\n",
"6FhN2SUiHuAV6TFJLMV4qrpYRBoB460UY9LJDbfc0vW9448fXGPDhn/P/LO7j259EjhcI1r4mqIx\n",
"RKgAPAu0Ac5QZVky4jXpI8wLlD4EugU/dwPeT9JxjCl1Td9++6Y3Tzxx8AUTJz4x84+uT6FbXwAu\n",
"LkZSr4SbJOwA4ERL6iZRStxjF5FhuAUE6uLq6ffhxt6+DTQDZgMXqerKXI+zHrspU8T3MzOys/vv\n",
"+ddf3V999NFhJ58zuRfuitA3NKJPF6ktoRpucYytwMWqbEhCyCYN2dJ4xiSI+H494O2D//ij6Wd9\n",
"+y6ov2rVSZLFv4DmwHkaif8PSYTauDHqs4HuqhRpLnZTvtlcMcYkgPh+a+DbY2bMWD65Z8/M+qtW\n",
"XSBZdATOAa4qYlJvAIwHJgPdLKmbZLDEbkwBxPcvAcYeP23a01/deGO7itnZF0gWNYD/UMS6ugh7\n",
"AZNwpcreqmQnJ2pT3tlcMcbkQXy/AvAwcNEhs2Z1mtS79yvAXZLFVFxd/eGijFcXYX/ctLtPqhZ9\n",
"SKQxRWE9dmNyEd/fHfgIaFN9w4Y2U3v27AdMQnUQ8DiwkCKMVxfhCFz55V5L6qY0WI/dmBji+xm4\n",
"4brTgFvWnnlmH6Ap0EWich6urn54vHV1ETzcCLFrVG3YrykdltiN2dn1gAA3aYcOHYA+wFGSxZ64\n",
"uvrZ8dbVRegEvIwbzvh5sgI2Jjcb7mhMQHy/GfA90E47dNgAfA1cKln8H0Ucry7CFcATQCdVijV3\n",
"jDF5seGOxsRJfF+A54EB2qHDbGAE8C9Ux1PEuroIN+FOvJ5oSd2EwUoxxjiXAk33WbDgPFyC/wN4\n",
"sih19WAyr3uBK3CTec1ObsjG5M1KMabcE9+vC0wHOmmHDq1xS9wdLVnUw5Vjzi5saKMIGUB/oANw\n",
"miqLkxy2KafCnI/dmLLkaeAN7dAhZ/rcdpLFZuAt4hivLkJF3CLWrYD2qqwsaH9jks1q7KZcE98/\n",
"Azhu2P33PwMMB3qi+itx1tVFqIKbzKsBcKoldZMKrBRjyi3x/ZrA9GobN/Zcd8YZdwL/h+o9QV29\n",
"P4XMry5CTdz0AMuAK1TZXCqBm3LNZnc0pgDi+wOBGtqhw9/AwcBZkkUz4qiri1AXGAX8APRSZVtp\n",
"xGyMDXc0Jh/i+8cB50/p0WMibkm6yySLCsRRVxehCTAR+Ay4zpK6STWW2E25I75fGRjU4Ycf/nXo\n",
"778/DpyP6nLiqKuL0Ao3Q+OrqtyhSjgfeY0pgI2KMeXRXZW2bPljXL9+1wF9UZ0cz3h1EQ4FPgGy\n",
"VHmpFOM1pkisx27KFfH9g1Ht9eNVV2VmwBhUh0pU9qaQ+dVFaIubdvcWS+om1VliN+VGMMf6oBve\n",
"f///9l2woAbQV6JSCVdXfyi/uroIp+MWZO+qyvDSi9iY4rHEbsqTm+qtWFFtwMCBbYALUd2Mq6sv\n",
"AJ7J6wEiXAwMAc5R5dPSC9WY4rMauykXxPf3zti27b6JvXtnV1A9D9WFhdXVRbgGiACnqDKttGM2\n",
"prgssZu0J74vFbZte+m2YcM27D9v3mOoToqpq+c5v7oIdwDX4qYImFXaMRtTEpbYTdqT7OyuzZYu\n",
"Pfy+114bBQwsrK4uwgO4se3Hq7KglMM1psSsxm7Smvh+g6qbNw8c+tBDy6ps2dITd6l1vnV1EW4B\n",
"LsT11C2pmzLJeuwmrTVetuyNC32/QtsZM85EdX1BdXURLgX64Xrqf4URrzGJYIndpK1DX365W6Wq\n",
"Vdv3HT78fFR/L6iuLsIpuOl7T1ZlThjxGpMoVooxaenUJ56ou3j33V+8/c03X2u6dOkHBdXVRTgC\n",
"eAO4UJUfw4jXmESyxG7SUobq+ONmzFh87ciRPYK78qyri9ASGAn0VGViKYdpTFIkNbGLyGwRmSYi\n",
"k0XEFvU1peLhLl0enLrPPvsfNHv2Cahmx9TVu8fW1UVoCHyKm/vl/bDiNSbRkjofu4j8CRyhbua8\n",
"3NtsPnaTcL82bdrmrEcf/arlggV9R9922zNBXX2X+dVFqAVMAN5T5f6w4jWmqFJlPnZL3qZ0iNR5\n",
"oVOnMRmq3wVJPc+6ugiVgfeA/wEPhBStMUmT7FExCowTkW3Af1TVZsUzySGSMeGQQz4YdNZZlddW\n",
"q3ZucO8T5Kqri5ABvAasBG60+dRNOkp2Ym+rqotEpB4wVkR+UdVJORtFJCtmX19V/STHY9LUxszM\n",
"rOv69m29pWLF3up5iyUqnYFOxIxXF0FwQxobAqfZykemLBARD/CK9JjSWvNURCLAWlV9MritPDbl\n",
"JL3t0M9LJQCTvkQ6PnDFFUMf7tJl2sbKldszocPeuDJL7rr6ncClwAmqrAwrXGNKItQau4hUE5Ga\n",
"wc/VgVMh1xjhmlsHJuv4ppwQafVr06av3t+1KxsrV76SCR3yq6t3B64BTrekbtJdMk+eNgAmicgU\n",
"3KiEj1QdwKaGAAAcYElEQVR1zE577LnhAHlm8mFJjMGkM5E9s2HUaY8/vmxrxYoPqOf9gRuvPp+d\n",
"6+odgYdxSX1hSNEaU2qSVmNX1T+B1gXu9EW96ey17j/AUcmKw6QpkbrA2N433TR1TsOGzYAB+dTV\n",
"jwNeAc5S5dfwAjam9IR75elXdXqx1/oj5bnvW4YahylbRGoDY6btvfdnz3bufAJwNRM6NANeIGbd\n",
"UhEOxA1rvEIVu0DOlBuhJnb9ot4XfLnHn2zN+E+YcZgyRKQG8DHwxaEvv9wE+A8TOvxKrrq6CE2B\n",
"0cA/VRkdWrzGhCD8uWK+2qMPLdZ6MvTrhmGHYlKcSBXgA+CXzLFjJyJyIPAguerqItTBJfVnVBka\n",
"VrjGhCX0xK5+/Q/5ts5S/q78XNixmBQmkgkMB5a1Gjr0tq0VKz4D9GBCh064unp3jaiKUA34EBit\n",
"yr9CjNiY0ISe2AH4su7d7LO2k4z8olbYoZgUJFIBeB13JfMVs5o0eRx4L6irPwN01oiuEKEi8CYw\n",
"G7g1rHCNCVtqJPZxDQYzbbfVzKoxIOxQTIoRyQAGAXsAF8n48ScAp/C/S3/FTRlwskZ0cnBV6QtA\n",
"ZaC7KtmhxWxMyFIisaui/F/dh2mxtov4ftWw4zEpQkSAAUAr4BwZPz4D+A9/DhrHpsV9gPYa0enB\n",
"3g8AhwDnq7I5nICNSQ0pkdgBGNXoaX6ruZkpuz0UdigmZTwMHAecheo6VKOs/mkjc/97LNBOIzoL\n",
"QIQbgYtwY9XXhhivMSkhZRK7Klv5ou5Amq2/Xnw/M+x4TMhE7gI6Aqehuko+H9OGbetvYMZ92cAJ\n",
"GtH5bjcuAu7ATeq1LMSIjUkZKZPYAfiwcRbzq2Uwdbc7wg7FhEikN3AVcAqqf8nrN1Zh8/JP+fPl\n",
"eWz++wSN6DK3Gx2AZ3E99T/DDNmYVJJSiV2VTUyq+yqNNv5TfD+lYjOlRKQH0Bc4GdVFEpXKSMZ3\n",
"bP47m2WfH64RXel24zDcRUkXqTI1zJCNSTWplzw/bfhPVmZWY3qt68IOxZQykUuBKC6pz5GoVKfG\n",
"vuNocForKlQ/Vu9auc7tRgvgI6CXKn6IERuTklIusevqzDX49d6jzuao+L4tq1deiJwDPIWrqc+U\n",
"qOwG8in737kvGZVv17OunOl2oz5uAeqHVHknzJCNSVUpl9gBGNXoRrZk7M7PNS8OOxRTCkROAV4C\n",
"zkZ1ukSlHjCeFtduodpef5BRcaDbjZrAJ8AwVexKZWPykZKJXVdUWsr4+uOovvXxsGMxSSbSDngD\n",
"6IzqdxKVJsBEarScRJOLDkakh3reNhEqASOAH4BImCEbk+pSMrEDMKrh9VTKbiwvf3t62KGYJBE5\n",
"EpesL0P1C4nKPsBEkMEc8VILRJ5Vz5sRLEA9GFiPq6vbAtTGFCBlE7suqfInnzX4hgx9OuxYTBKI\n",
"HIw7AdoD1bESlYOBCcDjtP98LrA38EgwVcC/gGbApapsDS1mY8qIlE3sAIxrcB27bWklQ78+OuxQ\n",
"TAKJtMKdAO2D6ocSlTbAOOA22o8fDjwN9FTP2wz8E7debidVNoQWszFlSEondv2z+lTGNfiJTRl2\n",
"oixdiOyFS+L3oTpMotIet3BGT43oG7iRMW+p530lQlfgRtxapSvCC9qYsiWlEzsAnzW4ifqbWss7\n",
"Xx4QdiimhEQa4ZJ6f1RflqicCbwDXKIRHSm+fxpwAnC3CGfiFtA4XZX54QVtTNmT8oldf63p81n9\n",
"OSyv9HzYsZgSCBafBgajOkCicjHuhGhHjejn4vs1cNPuXksH7yBgCHCeKj+HF7QxZVPKJ3YAPm/Q\n",
"jz03tJOPJjUNOxRTDCK74WrqI1F9WKLSA1dyOUUj+r9grweBSXTwZuOWv7tSla9CideYMq5sJPbp\n",
"u73PhHrLWFD132GHYopIpDruoqIvgbskKn2Ae3BzqU8DEN8/BriYgS0fw/0DuEOVj8MK2Ziyrkwk\n",
"dlWU8fXvocmGM2XUxLphx2PitGPx6V+HH0hvySICXIebdtdNEeD7lYBBLKt0N+82eRN4QZVXQ4vZ\n",
"mDRQJhI7AN/XeYUv91jL3Gr9ww7FxMEtPv028Je/Fz0vuogngXNxSX1uzJ53so05XHTslcDnwGOl\n",
"H6wx6aXMJHZVshlf/xEab7hEfN8WvU5lbvHpoYAMOZRuHa7iReAYoINGdMn23Xz/IJQbueooAVkI\n",
"9LGrSo0puTKT2AH4qu5T/LD7FmbWeDDsUEw+3OLTLwH1nm3DZVeex1BgL9yJ0u1j0cX3K6AM4s2m\n",
"vzCvWiWgmy1AbUxilKnErspmxtd/lgYbe9qi1ynILT79NLDfU8dwyU1n8RZQCThbI5p7LdIbWFRl\n",
"T15qUQ3orMqm0g7XmHSVtMQuIqeLyC8iMlNEbk9Yw379+/mlVga/V7fl81LPg0DbJ4/lkr6n8w7w\n",
"N3ChRnRj7E7i+3uxKeNh7jkYVM5UZXUo0RqTppKS2MXVWJ8FTgcOBC4VkYRcOarKOj6rP4Q9Nve1\n",
"Ra9TiFt8+tyH2nHpP0/jXWAG0E0jumWn3XxfWFr5fd5sms2fNU5UZUme7Rljii1ZPfajgFmqOltV\n",
"twBvAuckrPUxDe9gbrUqzK52Q8LaNMUncjPQ/cF2dLnnJN4DPgNu0IjuWjP/co8HWVfhYD5ofLIq\n",
"v5d2qMaUB8lK7I2BeTG35wf3JYQqyxnb4H1qbr3XFr0OmUh3oN+D7bjy3pMYAQzViN6hEd1ldIuc\n",
"uNTjwNV34Ne/WZdX+qb0gzWmfKiYpHbjGrImIlkxN31V9eM+wtgGvem4cC4bKlwOvFak6ExiiFwC\n",
"PPDgCfS490SGAY9oRPOciVMysw8ksnQUC6t8rEOa27w/xsRJRDzAK9JjdNeOVSICOQbIUtXTg9t3\n",
"Atmq+ljMPqqqJVqsWs6fP4aL5h1Eg01N1PNs/HNpEukEvPhgO3rfexJPA7dqRF/Pe1f25/RFX9Lr\n",
"9/XU3NpSPW9jXvsZYwoXT+5MVhnjO6CViDQXkUrAxcCHCT/KqIY3simjAYsrd0x42yZ/IicDgx5s\n",
"xz33nsRA4PoCkvq+NNg4nltmVqTm1nMsqRuTfElJ7Kq6FbdAwqfAT8Bbqprw6Vd1fcXfGN3oO7LF\n",
"phkoLSLHA8MeOZ7H7j2JR4AuGtH3896Vloh+xtOTl1M5+wn1vO9LN1hjyqeklGLiOnACSjEAssem\n",
"Njz/w1dkZp+k57WdkIjYTD5EjgBGPdaWF+44hWuB8zSiX+a9Ky2A8dz10/9xytJ9gLbqebZeqTEl\n",
"FGYpptTo35W/5eNGv7Ku4jNhx5LW3OLTHz9xHMPuOIUewGkFJPXmwOcc89crnLL0VKCrJXVjSk+Z\n",
"T+wAjGlwC9W3HiSfTGoddihpKVh8uv+xjL7tVM7FTeY1Je9daQp8TqVtT/HI9DOBLPW8X0szXGPK\n",
"u/RI7IurjuOTRvNZUekFG9eeYCLNFMYOOJpv+51GW9y0u3kmahEa46beHcink2oCqwFbiNyYUpYW\n",
"SVAVZXTD21hfoTV/VZos73zZIuyY0oJII4XPXjiSP245g32AdhrROXnvSiNcUn+R8f4E4Gagu3qe\n",
"zdhoTClLi8QOwPxqw7n5sJ6Ma7Abmdkz5YZZI6VSdvOwwyqzRPZQGDvocFb0OpvquKXsFue9Kw1x\n",
"Sf1VxvsDcXOx91HPm5fX/saY5EqbxK6K6oYKQ/WFfZozpuH5nLDsaB768Tdpsv5zEc4VwSYMi5fI\n",
"btkwZughZFzbkbXAyRrR5XnvSn3c3DBvqPIIbobHn4A3Si9gY0ysMj/cMd/2fT+TTRkR4Cb+02Ix\n",
"7zWuCfIKMEiV2ck6bpknUn2bMPadA2l06QVMV9l12t0du1IXGA+8q0pEfL89MAw4RD3vr9IM25jy\n",
"Ip7cmbaJfftxfP9wYAirKy7lxsNmMa/6+bgrY18ERqqypeAWyhGRKlsy+PjjfTnwgovwt2XQNfe0\n",
"uzt2ZQ9cT/1j4B7G+zWBqcBN6nkflWLUxpQrlthzjuX7lYEIcDVrKvaj0/EA1wAtgcG4XvyfpRFL\n",
"yhLJ3FiBDz7fm6PPuZQRWytwvUZ0W967sjsuqY8DbldFxfdfBrLV83qWZtjGlDeW2HMf0/ePBoYA\n",
"04BedPDqAT2BK4AfcL34D8tdL16kwppKvPNtY07qeCkvrq/ErXlNu+t2pTYwFpgE9AuSeifckniH\n",
"quetKcXIjSl3ysWVp0Whnvc1cBgwF/iR8f7+qvQFmuKm/r0ZmCvCw8El8elPJOPvqvx3agNOu+BC\n",
"niwkqdcCRgNfsSOp1wNeALpZUjcmNZSrHvtOx/f944FXgf/h6sIrXFzsj+vFdwUm43rxH6RlL15E\n",
"FtTk1YU1ueTcS7hrwZP6ZP67UhOX1KcCNwRJXYARwCz1vNtKKWpjyjXrsRdAPe8L4FBgBfCj+P5Z\n",
"AKr8oko/XC9+MG6WynkiPCLCPqEFnAS/7sGLf1fl0ss706eQpF4D+ASYDtyoun0hlSuAVsB9yY/W\n",
"GBOvcttjjyW+3wF4BTd0r4963qqdtgv7saMXP5UdvfjNpR1rony3pzxfczM9u51Hz/+9pIPz20+E\n",
"6riRL78DPVXJBhDfbwZ8D5yinpfnvDHGmMSzHnuc1PPGA4cAW4Bp4vun7LRd+VWVf+J68a8AvXC1\n",
"+EdFaFnqAZfQuH3k2Tob6NnnNK4oJKlXwy2QMpudk3oG7tNMf0vqxqQeS+wB9bw16nnX4oZBviK+\n",
"/7z4fo2d9lE2qTJMlQ5Ae6AC8KUI40S4SIRKIYReJG8eLAP3+4vrH2jPJZ/8V4flt58IVYD3gUXA\n",
"1TlJPXAjUBV4IrnRGmOKwxJ7Lup5nwL/AKrgeu/t89zP9eJvxfXiXwKuw9XiHxOhVakFXATPtZFn\n",
"28+h13NtOH/w+/pOfvuJcDLuxPES4EpVto9nF9/fH7gXm2PdmJRlNfYCiO+fDfwHeAe4Uz1vfYH7\n",
"C/sCPYBuuBONLwLvq7Ip2bEW5v728vx133HNmwdz3s1fa57rz4rQDHgSOBK4BTemf/sviPh+JvAl\n",
"8LJ63gulEbcxZmdWYy+h4NL4Q4C6wBTx/eMK3F/5TZXbgGa4fwjX4Grxj4fZi7/5TBnU61t6TmhO\n",
"57ySugiVRbgL10ufARyoygexST1wF/A37rkZY1KU9djjJL5/PvAsbkra+9Tz8pwYa5fHuYTeA7gS\n",
"lzRfBN4rjV68REW6TOW1AaO5dEY9Ljxhjr6XR3xnAM/gZmTso8ofebbl+0fiRsccrp63IKmBG2Py\n",
"ZVMKJFhwleXzwIG4Ky2/jfux7sTqOcC1uE8BrwEvqvJbUmKNSsaZv/HW0Hc5d3ENLj1w2c41dRH2\n",
"Bp4CDgZuVuWTfNvy/aq4KRei6nlvJiNeY0x8rBSTYOp5y4ALgfuBj8T3HxTfj2skjCqbVRmuysnA\n",
"ccBWYKII40W4VITKiYpTopLZbg4fvPYe56zPpGtsUhehqggR4FvgG+DggpJ64GFgiiV1Y8oG67EX\n",
"k/h+I1yteS9c773I47mDXnwnXC++NTt68cVe/FmiUuWwRYwc8xrtKmZzbe2NOiQ4lgAdcZN1/QD0\n",
"VWVuoe35/olBXIeo5+W52IYxpvRYKSbJgrlSugL/AgYCj6jnFWtOmWC6gh7AVcAvuFr8u6rEVcsH\n",
"kKjU2O8vRk0YzGG1NvHPqlv0haDtlrg6egvgJlXGxtWe7++GmwnzWvW80UV7RsaYZLDEXkrE95sA\n",
"g3CjZ7qp580odluuF98R14s/DHey9kVVfinwcVHZvelKxn49iFb11pFVMVufCqYDuBM3xv4xYEBR\n",
"pkEQ338V2KCed30xn44xJsGsxl5K1PPmA2fgSjO++P7t4vsVitWWq8WPUOVU4GhgIzBehAkidAmu\n",
"CN2JRKVB/bVM/Opl9mqwlicyVZ8W4XzcSJcWwKGqPFHEpN4VOB64tTjPwxgTHuuxJ5j4fnPcfDJV\n",
"gSvV84pdL9/epluIO6cXfwQ7evE/S1Sa1d7AZ1Ofp1rT1bxehY2DN1P5GWBP3EyMfjGewzW4GRtP\n",
"K8mnD2NM4lmPPQTqebOBk4HXgf8T3+8TTJpV/DaVLaq8q8ppQBtgPfC5NJzybfU11b+f/IJUqLe6\n",
"yscV2CabqfwFbordw4qZ1G8F7gDaW1I3pmxKSo9dRLJwJwKXBXfdqaqjc+2Tlj32WOL7OWuqKnCV\n",
"et7vCWv7tnqHV85YM27sc40y5645LrMrr/2dTYWcNUgXFyNWAR4AzsdNxTs/UbEaYxIntJOnIhIB\n",
"1qhq/5IElw6CWvvNuMvxI8AL6nnZBT+qkDajckzmNj74ZSBzWqxkdhPm3bWAJjVUKdYUusEniqdx\n",
"NfXTgvH6xpgUFHZiX6ta0Ko85SOx5whmRXwVWAd0V8+bU6x2onJihWze/HUgM/ZZwTqgM6rFXvBD\n",
"fL8i8DLuJOvZuRcZMcaklrBr7DeJyFQReVlEaifxOGWCet4vuB7xWOA78f0eQfkjbhKVTqK8+ctA\n",
"vt5nBQpcWMKkXhl4G2iA66lbUjcmDRS7xy4iY4GGeWy6G7dAdM7H+QeARqp6da7Hl6seeyzx/YOB\n",
"IcBSoGc89WyJymWi9J/5DJ/vs4LmwKmori1BDNWB94BVQBf1vDK7zJ8x5UlKXKAkIs2Bkar6j1z3\n",
"KxCNuctXVT+pwRSHiJCEFymY2/xO3GpE/wSGqufleRyJyrXAvTMHMLLlCo4BOqC6sgTHro2bqfFX\n",
"4BpbMMOY1CUiHuDF3BUJq8beSFUXBT/3Adqo6mW59kn5HvvmCrI/8FalbC5H9cdkHEN8/zBc7/1P\n",
"3KX7O41okajcBlz3+9MMb7GSjkB7VIt9clN8vz7wKTAB6FvSE7nGmNIV5snT13CTWik5CUt1SVGD\n",
"C1vle+WArlP5tv+nZKyswhNNVxNFNeGJMJgh8j6gJ9AbeIsJHQAeBM7742le33slVwMnoFrsudDF\n",
"95viavxvAVn5fUIwxqSulCjF5HvgMpDYASQqDY+azwNPjebK2htZMnYfOvf+n36TlGP5fhtgCKoz\n",
"+KbLSjYuOnz2U7y61ypuxSX12SVouxUuqT+jnpfvMFRjTGqzxJ5AzfpKo4jPm+f8Qrtnj2bc4225\n",
"av2Dxe8950eGR6uTvfVr6hy5b8+Pxzz/4oDnLsTV1Is/la/v/wMYDUTU8wYlLFhjTKmzxJ4Eo1vK\n",
"iYcs4a2vm7Bbn9MZMqc2WRpJTIKXqFQG3gCq3fXnZZPePOXM+9dVrTp6SZ06XYs7F7r4/tHAh8DN\n",
"6nlvJSJOY0x4LLEni0iV5VV4KkO5/PLO6Mf78RrwqEa02JfhS1SqA+8Cq5c/yku7b+T1n5o1O/+g\n",
"IUMuwF3mf6163sdFatP3O+Dq6VcV9bHGmNRkiT3ZRDpsE14b24JlF1xM83WVGIZL8POK1ExUagMf\n",
"ATNXP8wrNTczArgA1YkA4vvtcXPO+ECfeC4kEt/vhJsj/kL1vAlFe2LGmFQV9pWn6U91fAXlH6f/\n",
"zvSVj7D8ghnUBKZIVP4tUWkaTxMSlXrAeOD7DQ/wbM3NvANckZPUAYLEfAiwCfhRfP/UAtv0/ctw\n",
"KzCdZUndmPLHeuyJInIh8OxfVXm1WV8yNmTSHVcGeSS/HrxEpQlupMrwzffzZmY2nwG9UH0v38P4\n",
"/im4nvgo4Fb1vDW5tl8H3IPNpW5MWrIee2lSHQ4cVncDrdc/xAkj3uRMYDUwVaLyvESlWezuEpWW\n",
"wCRgkGYxJDObMcCtBSV1APW8sbjeeyYwVXzf296m798O3AacYEndmPLLeuyJJiJAL9x0CffVvJO3\n",
"11amH3ANbsKtR4BauOGHUc1iNDAReATVF4t0KN8/C7cc37u4xTc64eZST/gwTGNMarCTp2ES2Q+3\n",
"hN3fQHfJYgvQF7e8nQI3ahY+Lqk/j+pTxTqM7+8OPAO0BDqq5/2VgOiNMSnKEnvYRDJxs11eB9yA\n",
"6giJSl2gvmaxGDfKZTiqD4QYpTGmDLHEnipEjsb13r/CraakwDhcYr89GbNHGmPSkyX2VCJSHfgX\n",
"cAawGPgB14u3pG6MiZsl9lQkciZwDJCVjJkijTHpzRK7McakGRvHbowx5ZAldmOMSTOW2I0xJs1Y\n",
"YjfGmDRjid0YY9KMJXZjjEkzltiNMSbNWGI3xpg0Y4ndGGPSjCV2Y4xJM5bYjTEmzVhiN8aYNGOJ\n",
"3Rhj0owldmOMSTPFTuwicqGIzBCRbSJyeK5td4rITBH5RUROLXmYxhhj4lWSHvuPwHm4xZi3E5ED\n",
"gYuBA4HTgedEpMx+MhARL+wY4mFxJpbFmVhlIc6yEGO8ip1wVfUXVf0tj03nAMNUdYuqzgZmAUcV\n",
"9zgpwAs7gDh5YQcQJy/sAOLkhR1AnLywA4iTF3YAcfDCDiBRktGT3hOYH3N7PtA4CccxxhiTh4oF\n",
"bRSRsUDDPDbdpaoji3AcW7DZGGNKSYnXPBWR8UA/Vf0huH0HgKo+GtweDURU9etcj7Nkb4wxxVDY\n",
"mqcF9tiLIPYgHwJviEh/XAmmFfBNUQMzxhhTPCUZ7nieiMwDjgE+FpFRAKr6E/A28BMwCuilJf1Y\n",
"YIwxJm4lLsUYY4xJLaGOLxeRB0RkqohMEZHPRKRpmPHkR0SeEJGfg1jfFZHdwo4pLwVdNJYKROT0\n",
"4KK1mSJye9jx5EVEXhGRJSLyY9ix5EdEmorI+OC9ni4iN4cdU15EpIqIfB38ff8kIo+EHVNBRKSC\n",
"iEwWkaIMDClVIjJbRKYFce5S4s4R9oVDj6vqoaraGngfiIQcT37GAAep6qHAb8CdIceTnzwvGksF\n",
"IlIBeBZ30dqBwKUickC4UeVpMC7GVLYF6KOqB+FKoTek4mupqhuBDsHf9yFABxE5PuSwCtIbV0JO\n",
"5TKGAp6qHqaq+V4fFGpiV9U1MTdrAH+FFUtBVHWsqmYHN78GmoQZT34KuGgsFRwFzFLV2aq6BXgT\n",
"dzFbSlHVScCKsOMoiKouVtUpwc9rgZ9x14+kHFVdH/xYCagALA8xnHyJSBPgTGAQOw8GSUWFxhd2\n",
"jx0ReUhE5gLdgEfDjicO3YFPwg6iDGoMzIu5bReuJYCINAcOw3U4Uo6IZIjIFGAJMD4YXJGKngJu\n",
"BbIL2zFkCowTke9EpGd+OyVquGO+CrvISVXvBu4Oxr8/BVyV7JjyEs/FWCJyN7BZVd8o1eBiJPCi\n",
"sdKWyh9vyyQRqQG8A/QOeu4pJ/ik2zo4L/WpiHiq6occ1k5E5GxgqapOLgPzxbRV1UUiUg8YKyK/\n",
"BJ8yd5L0xK6qp8S56xuE2BMuLE4RuRL3Ue2kUgkoH0V4PVPNAiD25HhTdp56whSBiGQCI4DXVfX9\n",
"sOMpjKquEpGPgSMBP+RwcjsO6CQiZwJVgFoi8pqqdg05rl2o6qLg+zIReQ9X4twlsYc9KqZVzM1z\n",
"gMlhxVIQETkd9zHtnOCEUFmQanXC74BWItJcRCrhZgD9MOSYyiQREeBl4CdVfTrsePIjInVFpHbw\n",
"c1XgFFLwb1xV71LVpqq6N3AJ8HkqJnURqSYiNYOfqwOn4gZM7CLsGvsjIvJjUIPzgH4hx5OfgbiT\n",
"u2ODYUbPhR1QXvK7aCwVqOpW4EbgU9zIg7dU9edwo9qViAwDvgT2FZF5IhJKabAQbYHLcaNMJgdf\n",
"qTiSpxHwefD3/TUwUlU/CzmmeKRq2bABMCnm9fxIVcfktaNdoGSMMWkm7B67McaYBLPEbowxacYS\n",
"uzHGpBlL7MYYk2YssRtjTJqxxG6MMWnGErsxxqQZS+zGGJNm/h9LRNvNs+E2pQAAAABJRU5ErkJg\n",
"gg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x1058dd2d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"## Read test data here\n",
"a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
"xt= a[:,0]\n",
"yt= a[:,1]\n",
"X = construct_poly(x,8)\n",
"w = wRidge(X,y,1.0)\n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" X = construct_poly(x,n) \n",
" Xt = construct_poly(xt,n) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(x,X.dot(w))\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE train: %f test %f\" % (MSE(X.dot(w),y),MSE(Xt.dot(w),yt)))\n",
" plt.plot(xt,yt)\n",
" plt.plot(x,y)\n",
"# was ist mit fehler kurven gemeint?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1.75123654094\n",
"0.781528050024\n",
"11.0428184273\n"
]
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXYAAAEKCAYAAAAGvn7fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XmclXP7wPHPNUt7qbQoIbLTIyHRk06lKImESJKtpEg8\n",
"1sfPmWPfd1lL9ihC2QrdlaJoI4R4ilJpT/sy1++P7z3NaUw1M2e5z5y53q/XeTVnu+/rzPO45nuu\n",
"+/p+v6KqGGOMSR8ZQQdgjDEmviyxG2NMmrHEbowxacYSuzHGpBlL7MYYk2YssRtjTJqxxG6MMWnG\n",
"ErtJKBGZJyKbRGTPAo/PEJFcEdnXv99ARN4WkaUiskpEvhORi/znGvqv/bvA7ZydnPNcEZksIutE\n",
"ZFwhzz8nInNEZFveOXYRf3kRGSIiq0VkkYgM3MnrevoxXhr12EUi8o3/3j9E5D4RyYx6voGIjBKR\n",
"5f6xn8h7XkSyRWSEiPzPP26rAufL8l+/yH//+yJSf1efxZQdlthNoinwG3B+3gMi0hio6D+X5xVg\n",
"PrAvUBO4EFhS4Fh7qGrVqNvwnZxzOfAwcO9Onp8JXAlMLxBDYXKARn5crYEbROSU6BeISA3gFmB2\n",
"geNVBAYAewLHA22B/0Q9/ziwDKgHNAFa+XHlmQD0ABYXEueVQEvgX0B9YCXwxG4+iykjLLGbZHgV\n",
"6Bl1/yLgZUCiHjsWGKqqG1Q1V1VnqurHJTmZqn6mqiOARTt5fpCqfg5sLMLhegJ3qOpqVZ0DPAf0\n",
"KvCae4DHcH9Qos/zjKpOUtWtqvon8BrQIuolRwBvqupmVV0CfOw/hqpuUdXHVXUSsK2QuI4APlHV\n",
"paq6CXgr773GWGI3yfAVUE1EDvVLDd1wyb7gawaJSLe88kwhZCePJ4Q/Eq8HzIp6+FuiEqiINAOa\n",
"As8U4ZCtcKP6PJ8A3UWkoojsDXQAPipieGOADiJST0QqARcAHxbxvSbNWWI3yfIKbvTbDvgBWFjg\n",
"+XOAicD/Ab/5NfhjC7xmmYisjLodkuCYq/j/ro56bA1QFcD/I/UU0F93s+iSiFyC+wPwYNTDOcCR\n",
"/jH/AL5W1feKEpiqvg3MwP0eVwOHAHcU5b0m/VliN8mguMR+AYWXYVDVVap6s6oeCdTF1cHfLXCc\n",
"PVW1RtTtpwTHvdb/t1rUY3sAf/s/Xwl8q6pTo57/x7cKETkTuBvooKor/McEN2IfDlQCagE1ReS+\n",
"ogQmIg/i/sDUBCoDIyn6aN+kOUvsJilU9XfcRdQOwDu7ee1y4CGgvl8OKfFpY3gvqroSV6dvEvXw\n",
"UeSXU9oAXfzOlEXAicBDIvJ43otF5FRcXb6Tqn4fdZxawDHAk349fQUwFOhYxPBOBV70/yBuBp4E\n",
"molIzeJ+TpN+LLGbZLoUaKOqGwo+4bcCHuG38VUF+gK/+Ml1+8uKchIRyRCRCkA2kOG3LGZHPZ/t\n",
"P58BlBORCv4IujAvA7eKSHUROQy4DJeAwV1EPRSX7JsA3+DKK//1z9MGd8H0LFX9psBxl+H+aPQV\n",
"kUwRqY77NrO9nu/HXcG/G/0zuFr/RSJSzf9sVwIL874RmLLNErtJGlX9TVWnRz8U9XNFXDlhJfAr\n",
"sA/QucAhVhXoY79mJ6fqCawHBuFaAjcAz0Y9P9Z/vjluNL3efx0icoGIRF/gDPvxzAfGAfep6hj/\n",
"86xW1b/82xJgM7BGVfNKNbfiyiUfRcX8gf9eBc4CTscl+V+ATUB0n/xPfmz1cWWbdVEXlgcCuX5s\n",
"f+FG8F128vswZYzEstGGiOyDG9HUwf1H+pyqPu5/HXwT2A+YB5yrqqtiD9cYY8zuxJrY9wL2UtWZ\n",
"IlIFmAacCVwMLFPV+0XkRqCGqt4Ul4iNMcbsUkylGFVdrKoz/Z/XAj8Ce+O+Qr/kv+wlXLI3xhiT\n",
"BHGrsYtIQ+BoYApQ1685gpsWXjde5zHGGLNrcUnsfhnmbWBA1IUjYPtFItsx2xhjkiQr1gP4rVZv\n",
"A6+oat6EkiUispeqLhaRerir9gXfZ8neGGNKQFV32fobU2L3e38HAz+o6qNRT72P68m9z/+34AzC\n",
"IgWXykQkR1Vzgo6jpCz+YFn8wSnNsUPRBsWxjthb4JYV/VZEZviP3YxbLvUtf23qecC5MZ7HGGNM\n",
"EcWU2FX1C3Zepz85lmMbY4wpGZt5WnJe0AHEyAs6gBh5QQcQIy/oAGLkBR1ADLygA0i0mCYoxXRi\n",
"ES3NNXZjjAlCUXKnjdiNMSbNWGI3xpg0Y4ndGGPSjCV2Y4xJM5bYjTEmzVhiN8aYNGOJ3Rhj0owl\n",
"dmOMSTOW2I0xJs1YYjfGmDRjid0YY9KMJXZjjEkzltiNMSbNWGI3xpg0Y4ndGJNSRCgfdAylnSV2\n",
"Y0xKEOFEEUYAa0X4VoR7RThJhOygYysO8byaQcdgid0YExgRskQ4W4QvgZdxuxvtCfQGNgMPA3+J\n",
"8JYIvUTYK7hod008r5Z43lBgjHheoJsI2Q5KxpikE6EqcAlwDbAAl8DfV2VbIa/dCzgV6Ai0A34F\n",
"PvRvXxf2nmQSz8sAegH3AK8Dt2ko9HfCzleE3GmJ3RiTNCLsA1wNXAx8CjysytRivD8bOAGX5DsC\n",
"9YBPcEn+E1WWxz3oXcXjeUcATwMVgT4aCk1P+DktsRtjUoEIxwDXAacAQ4HHVZkfh+PuA3TAJfnW\n",
"wPe4JP8RMEOV3FjPUeh5Pa8ScCtwOZADPKOhUFK+OVhiN8YERoQMoBMuoTcEHgMGq7I6QecrD7Qk\n",
"fzRfHZfgPwTGqrIqLufxvFOBp4CvgWs1FPozHsct8vktsRtjkk2ESsBFwEBgFfAQ8LYqW5McRyPy\n",
"R/Mtgenk1+Znq1Ks5CeeVx94BDgW6Keh0MfxjbiIcVhiN8Ykiwj1gH64jpZJuIQ+qVgJVOQ44L/A\n",
"WOBlVONyEVKEikAIl+RPA7LJT/KfqbJ2p+/1vEygLxAGngXu0lBoQzziKglL7MaYhBPhX7jR+Rm4\n",
"rpBHVZlbzINkATcDVwF3Ay2Atv7xnkL1xzjGK8DB5JdsmgNfkZ/of877YySe1xSXzNcDfTUU+iFe\n",
"cZSUJXZjTEL4yfEU4FrgCOBJ4FlVVpTgYI2AV3DJsxeqC/zHGwB9cBcoZ/vnGI1qXEs6futlG/IT\n",
"/Waqbx7Lg7PqcsC6ExFuAoZqKBRMsizAErsxJq5EqABcgEvoW3HllmGqbC7BwQS4FNf/fRfwOKr/\n",
"7GIRKQ90BfoDe+PaCwejurRkn2IXIfX6n3DkmgEctiaH6TU28NDBVVhdbgL+aF6V/8X7nMWOMRmJ\n",
"XUSG4GpWf6lqY/+xHOAyIO8Xf7OqflzgfZbYjSklRKiNqzNfCUzDTSj6vLgXIKMOWBt4Htct0wPV\n",
"2UV83zG4On4X4H3gSVS/LlEMBQ/teQ2BJ4BGuLLLeBGq4yZFdcRdiF1JfslmYon+oMUaZ5ISe0tg\n",
"LfByVGIPA3+r6sOxBGeMCZYIh+Lq5+cAI3D189jqzCIdgRdw5ZfbUN1UgmPsiRvt9wX+wpVphqO6\n",
"sdiH8rxs3Ge8Adf18oCGQv9I2H775tHkl2wOBz7H75tXZUGxP0cJJK0UIyINgVEFEvtaVX0oluCM\n",
"Mcnn189DuP7z44BBwNOq/BXjgSsDD+JGvhehOv4fL4lINtAEWAcsAVZquJDyTP4xM3FJtr//vsHA\n",
"M6j+XqSQPO9E3MXRhbgWxl+L/nGohbvO0NH/dyEuyX+gyhdFPU5xBZ3YLwZWA98A16nqqgLvscRu\n",
"TAoRoRzQDVc/L48rt7ymSuytfa6N8VVgCnAVqtsnKUlEquOSfWdcglwAlAPqApWBZbgkn3f7q7Cf\n",
"Fz5Ijfpr6Q1cCIzHjeLHUUiS81dgvBdXRr4WeCuWi6MiZALNcEl+X1UuKumxdn+u4BJ7HfLr63cA\n",
"9VT10uIGZ4xJPBFq4HrPrwLm4C6IfhKX6fj5bYz9gf6oDgeQiOyPS+Sdcd8KPFzN/AMN66Ltb49I\n",
"OaAOLsnn/Vu3kPt1cKtCrq62kaX9pyK9p7FX9ja2jTqEL+5uyZjfq/M7krmEps+2oPIBNyDyNvBf\n",
"DYUSMhM2UQJL7EV5TkQUiEQ95KmqF3Mwxpgi8WdmDgB6AKNwC3LNiuMJGuFG6X+vKs8lNW5mb/KT\n",
"eR3/nO8Dn2pY18d8uohk4pJ7HaBuRi51+03lxAu/5eTDlrL/M632Wxru85/q6ytWKsdPD8LfP25k\n",
"F98ACvy8RsPBtBCKSAhXGssTDmrEXk/V/dUVkYHAcaravcB7bMRuTJL59fMTcfXzlrjOlCdVid96\n",
"J34bo8I9nxzIyE7dkW0ZdAJW4BL5+8CUXdbO40g8r0K95cvvXl258hXhl17aes2IEVMzt2194l99\n",
"mfBDHWqz828A0d8OsinaH4AlwAoNa8IWBEtWV8wbQCugFu5DhXF/XZoACvwP6KOqS4obnDEmPkTI\n",
"As7C1ZNr4bo/hqqyLp7nuaOVHNFtNq9mwP5dupE1uy5f4xL5KA1r8WajxoF4Xltc3/t3wABt3Xop\n",
"cDauNFQfd2F4MKrLdnmciFSi8JJQYeWhvzSshyfkA2ETlIwp80SohmsLHAD8jrsgOipem1NIRAQ4\n",
"DOh83ndc9PAnHDK2ET8NPIV7V1RilIa1+DNR4xGX59XFXSv4N9BfQ6HR/3yRHIvriT8TeA/XE/9N\n",
"zOeOSGapH7GXlCV2YxJHhH3J39BiDPBIcTa02OWxI5KFS5idgc5VNlFu+HBWnzSP2huz6F5zg34e\n",
"j/OUKDa3m9FlwJ24dd8jGgrt+luJSC3cbk5XAovJ74kvfn99ElhiN6aMEeE4XLmlPfAi8ERcNrSI\n",
"yB64VsTOuNbE34D3nx7Nb32+ISxuEa0d2hiTTTzvX8Az/t0rNBT6tngHkExc+2N/4CjcJKpnUP0j\n",
"nnHGyhK7MWWA30N9Oi6h70v+hhZrYjpuRPbzj9sZtwLiRFy9fLTmsAS3vO6VRLUxBkE8rzLu2l4v\n",
"3K5GL2goFNuFWZFDcJ/tQmAcbhTvFdYTn2yW2I1JYyJUxiWza3AdJw8B75R0QwuJSAbQlPyWxL2B\n",
"D3D157Ea1rX+iQ/CLQewBrgY1YUxfZAYiOd1wiXdicB/NBRaspu3FPMEUhXXDtrff+Qp4JV4rRNf\n",
"spAssRuTdkSoj0s0l+MS2kPA5JIsyCURqYBbsrYzbnT+N/ktiV/ucBHQtTFehluN8XbcxcaktCwW\n",
"JJ7XAHgcOBK3YNdniT2h5C2z0B+3t+qruHXif0roeQsPZbe5MytZwRhjYiPCUbhyy+nAa0BzVYq8\n",
"tsn240SkNq6W3Bm3mcUsXCJvo+GdJCo3m/x5XKnnJFQD2XBCPC8Ll1xvxY3Uu2soVOyFv4rNjYDH\n",
"AeMQ2Qe3TvwERGb5cXyAJq4TprhsxG5MCvNXFDwFN6HoMNyyss8Vd0MLicih5JdYGuO2nnsf+FDD\n",
"u+7hRqQTLqkPBcKoJn2pWgDxvGa4i6MrcaP0n4OIIz8gKY9b9bI/sBf568Tv+vcZ82mtFGNMqeRv\n",
"aNEDN0LfjCu3vFnU9b/9lsQTyE/mlckvsXgaLsLytm41xodxHTYXoTqh+J8kduJ5e+A24ugKXA+8\n",
"liq7GW3nFjnrh9secBhwZaIutFopxphSRoQ6uDXG++JWRu0PjCtK/VwiUhWXhDvjSi2/4xL5+cCM\n",
"Yq11InI8ro48GWgSRBujeJ4A5+L+uIwGDtdQaGWy4ygSt9lHL78n/vigu2dsxG5MChDhMPI3tHgL\n",
"t6HFbjdwlog0IL8lsQUuEbuWxHDR1iQvEEgWrn7dF+iH6ohiHyMOxPMOwHWgNAD6aCg0OYg4UpGN\n",
"2I1JYf6CXG1w5ZZjcOuWHKzKTvfy9KfwNyG/xNIQ15I4GOimYS1577prY3wVWAUcjWr8Fgbb1Wk9\n",
"rzxunZV6uFr1sbg/LPcDj2gotCUZcaQTS+zGJJm/ocV5uISejSs1dFWl0Lq3RKQ8rtUuL5lvwvWW\n",
"XwtM0rCWqG89KiDBtU7ehWtjfCrWNkZ/av+euEQdfatXyGNVcAsILgYWAfOBYzUUmhdLDGWZJXZj\n",
"kkSEmrg2uf7AD8BNuA0t/rnDT0T2xO3G0xm3mfL3uBJLe2BO3NYGF6mLmzq/N9Bqd22M/izPoiTr\n",
"Orie+EW4hJ13+xOYXuDxlTHPFDU7sBq7MQkmwoG42aHdccn5YVX+sY6JROQg8kflTXAbJeftKhTb\n",
"fqOFB3Y68NzG7OyhzZ5++unvGjXKG2EXlqjzHs/in8l6cSGP/aWhUEouolXaWbujMQHx6+ctcP3n\n",
"/waew21oEb3tWyZuDZa8ZL4H+bsKfa5hLfFeo35HSTUKSdTlNm9ucPj8+a02ZWfXmbv33uu2ZGdX\n",
"BZaz60Sdd1uTcq2GZYxdPDUmyfwNLbriEnoN3IYWPfI2tJCIVMaVVs7AtSQuwiXynsC03e0qJJ5X\n",
"jsJH04WNtLdQIFE3/fnn7Es+/LBd+S1bZj3Zpcv5W7Kz5wLLNBSKrU5vUoqN2I2JAxH2IH9Di/m4\n",
"CUWjVdkmEakPdMKNyk8CppC/q9A8/0JjTYpWu66K24ptZyPqvMeX7LAOuUg2bjXGvrjJM28n6ndh\n",
"EstKMcYkmAj74Ta06AV8AjxCjnwDNCarWlfK1TiT7Gr7U7nRTKof/Qs1jllOVuUa7Jis6+IuNO4q\n",
"UefdVhT7QmN+G+NK4JJktTGaxLDEbkyciedlArV5dd+2LKjYmyw9hiPWTKf5kt+ouLghuZsORLLq\n",
"kl09k4xyuejWpWSUm49k5CXowkbaSxJyodG1MfbG7SYUwbUxWn28lLPEbkwJiOcdhesb37EkouyF\n",
"Uou1WbmsKAfZm+dRZe5SNvxUg3W/NWTb+oWQ4VG+9vvsfeYEsqoEd6FxxzbGC1Dd7SxWUzrYxVNj\n",
"ikk87xjgY+BNXM/1Lywrt4pnD2jGrOrdyPj1O9rcNJODP2iI6DHAHNwWdKM1rIsDDD2fSGfgWVxc\n",
"XYNajdEEx0bsxvjE8w4EJgD9NBQaKUJ9ZOtVNPiqL02HLOaIt7LJXlcF2d6S+KmGdX2wUUcRqYLr\n",
"wmkL9ET1i4AjMglgI3Zjikg8ry5upH4751f7VRo8/TlnTjmRQ0duI2vTQrI2vYNL5lN315IYCJET\n",
"cNvVTcStxhjTfqemdLPEbso88byqwIfAa1y9cCpdr5xG5qYFlF8TocKatzSsxd6lKGlcG+P/4ZYq\n",
"6IvqOwFHZFKAJXZTpvkTft4GpnHDr89z4lO/sLaux/5e+7itx5IoIgfj2hiX40bpi3bzDlNGWI3d\n",
"lFn+xKCXgaq8MuNCGPE/tpVbxP5e45RO6q6NsQ+ujTEMDLI2xrLDauzG7Nr9wP6sWNGe3Pe+Q2Qz\n",
"fx3ZVIeOS90kKbIXro2xHtDS2hhNYTKCDsCYIIjnXQd0ZOv603nvqc/IXlebnzofqR8+kbqtgSJn\n",
"ADOBWcAJltTNztiI3ZQ54nndccvotuCVh56l8pKmfHldY51x8fKgYyuUa2N8FLfbUldUJwUckUlx\n",
"MY/YRWSIiCwRke+iHqspImNF5GcRGSMi1WM9jzHxIJ7XDtfr3YGXb7uG6vO68MXN7XXGxT8FHVuh\n",
"XBvjTNx/q00sqZuiiEcp5kXg1AKP3QSMVdWDgc/8+8YEyp9V+jpwNq9f35aacwfw+V2X6rcXeAGH\n",
"9k8i2YjcDowEbkD1EutNN0UVl64YEWkIjFLVxv79OUArVV0i7mKPp6qHFniPdcWYpBHPa4SbvNOP\n",
"t/tkUmn5MD4YdLfOPve2oGP7B5FDcJONluNWY7Q2RrNdUXJnoi6e1lXVJf7PS3DLkhoTCH9W6SfA\n",
"7Xx09jIqL3mdD58YnnJJXUQQ6Qt8AQwFOlpSNyWR8IunqqoiUujXAhHJibrrqaqX6HhM2eLPKv0A\n",
"eI3P231BRqWv+fCJb5h9fo+gY9uB+2Y7GDcIaonqnIAjMilCREK41UaL/p4ElmJCqrpYROoB46wU\n",
"Y5LNn1U6CvidCe3vYkPFWYy9bznTrmict1VdShA5E3gG158eQXVLwBGZFBZkKeZ94CL/54uAdxN0\n",
"HmMK5c8qHQJsZOqFYTZU/JIJt25j2hX/TpmkLlIVkcG4bfS6onqrJXUTD/Fod3wDmAwcIiJ/iMjF\n",
"wL1AOxH5Gdd7e2+s5zGmmO4D9ufnR/qwavU4pvavxuTrQ6qkxrZwIifi2hjB2hhNnNlaMSbtiOdd\n",
"C1zG8q/aMv3OkXzXvTGjB3XV3KyPg47NX40xDFyGW41xZMARmVLG1ooxZY543vnAQDYtPYmZOS/w\n",
"26mH88FT16dIUj8UtxrjX8DR1vFiEsXWijFpw59V+ii5Wzsw+fwcFjVtztuvDdZt2YOCDUwEkX64\n",
"NsYhwGmW1E0iWSnGpAXxvKa4XvWueK1PZ/nBF/H8lClsqn6mKtuCC0z2wiXz2kAPVFNz6QJTagTZ\n",
"FWNM0vizSkcDfRjfujnr6vbkxQkL2VT9vICTehfcBdJpwImW1E2y2IjdlGrieXWAScBDjG+9hU1V\n",
"7mHQ7M2s3q9ZYB0wIlWBx4BWwIWoTg4kDpOWbMRu0pp4XhXcXqVvML71EraWu5fnp2axer+OASb1\n",
"Frj10nNxbYyW1E3SWVeMKZWi9iqdwfi248jNHMGLE3JZdlh3Vb5NfkA7tDFegapNyjOBscRuSh1/\n",
"VulgYBOTznge1Q94ffQ6Fh5/jyrJb2vcsY2xCaqLkx6DMVGsFGNKo3uBRkzv+39sWfMe7w1ezNwO\n",
"w1V5OqlR7NjGOBjXxmhJ3QTORuymVBHPGwh04pfHzmXNnNF4kV+ZefEy4IbkBiL1cG2MtYAW1vFi\n",
"UomN2E2p4c8qvZYlY8/jz3ff5NsL5zD+tgrABUltaxQ5C5gBfI21MZoUZO2OplQQzzsZeJ21v3Zi\n",
"2mWP83uLvxjyxVFAc1WSM4tTpBqujbElro3xy6Sc15go1u5o0oJ43tHA62xZcx7TLruDlQ2XMmRC\n",
"c+D0JCb1f+MmG23FXSC1pG5SltXYTUoTzzsAGE3uliuZfEZvNu4hPPHzcZDRMyltjSLlcG2MlwJ9\n",
"UH0v4ec0JkaW2E3K8meVfoLm3snE9m3Zlr03D/5Zl9zsSFLaGkUOw7UxLsbaGE0pYqUYk5L8WaUf\n",
"AMOY0LYeKsfz4CJla6VRCW9rdG2M/YGJwPNAJ0vqpjSxEbtJOf6s0hHALMa3WYrSn8fmTmfDnhVI\n",
"dFujyB64UXpdXMfLzwk9nzEJYCN2k1KiZpVuYWKH8aDXM3jye6w64EAS3dYocjDwFfA7rjfdkrop\n",
"lSyxm1RzD9CIr84bQu7GBxj50uMsOOFcXAdM4jahFjkFN4P0EVT72abSpjSzUoxJGeJ51wCdmXXt\n",
"tWxa8jKTr/svs3reCZycsLZGEQGuBf4DdEV1YkLOY0wSWWI3KUE87zzgOuY+cQmrZrzKryffwJgH\n",
"7wES19YoUgF4DmgMNEd1fkLOY0ySWSnGBE48ry3wGIs+7M3Cd15k1b638MrYm4DEtTWK1AfGAxWA\n",
"f1tSN+nEErsJlD+r9A1Wf38FPz/wBFsq3Muj83sAoxPW1ihyPDAVeB/ohmriavfGBMDWijGB8WeV\n",
"TmTjkpuYct51qLxDJPcAYA/g7IR0wIj0BB4CLkX1/bgf35gEK0rutBq7CcT2WaVb19/HlPMuByZw\n",
"+5Zc4EigVdyTukgWcB9wBhBC9fu4Ht+YFGKJ3SSdP6t0NLrtTSaddjLwB7dv+grNvAe3WmN8SyMi\n",
"NYBhuNJjM1RXxPX4xqQYq7GbpBLPywZGoPotE9o1ALK4/69nyS33KIlYrdGt9zIF+BHoYEndlAU2\n",
"YjdJI54nwGBUtzDxlL9Bm/HCpD6sr/0piWhrFDkNeBG4EdUX43psY1JYQhO7iMwD1gDbgC2q2iyR\n",
"5zMp717gICZ3+Qjdcg7ebZ1ZcOLHxLut0U06uhG4CjjD1k43ZU2iR+wKhNS+/pZ522eVfn3xs2xd\n",
"fTV/HdEaLzKUeLc1ilQCXgAOBo5HdUHcjm1MKZGMGru1NJZx22eVfnfzo6yfdwPbstszaHYEWEU8\n",
"V2sU2Qe31K4CLS2pm7Iq0YldgU9F5BsRuTzB5zIpyJ9V+ji/Doqw4qs7gI7csfk8XFtjj7i1NYqc\n",
"iLtI+ibQA9UNcTmuMaVQoksxLVR1kYjUBsaKyByNWmRJRHKiXuupqpfgeEwSbZ9VuuiDW1kw/G7g\n",
"bHL0MOBy4tnWKHIpblXIXqh+GJdjGpMiRCQEhIr1nmTNPBWRMLBWVR/y79vM0zQmnrc/8AWrZt3H\n",
"rGtuAXqTo8uBkUBbVb6L/SSSBTwMnIK7SDon5mMak+KKkjsTVooRkUoiUtX/uTLQHuLwH7NJeeJ5\n",
"tYFP2LDwKWZdcx1wIzn6A/A2rq0xHkl9T+AT4CDcRVJL6sb4ElljrwtMFJGZuNrnaFUdk8DzmRQg\n",
"ntcAGM+WNe8ytceFwGPk6Cjc/qXxaWsUORK3iNd03H6kq2I+pjFpxBYBM3EjnncI8Alb173ApE6d\n",
"gc/I0TBuZD1dletiP4mcgWtnvBbVV2I+njGljC0CZpJGPO9YYBSbV93Ol126AzPxwrfgZn7G3tbo\n",
"Jh3dCvQGTkN1aowhG5O2LLGbmInntQGGseHPgUy9YADwDdAfL+e/xGO1RneNZiiwD24Rr8Rsk2dM\n",
"mrBFwExMxPO6AsNYM+dypl5wPTAB6EeOdsO1Nca2CbXIfsAkYB1uuV1L6sbshiV2U2LieZcDT7B8\n",
"cg9m9L0XtyPR9eRoCHgM6BTTao0iJwFfAS8BF6O6MfaojUl/Vooxxeav0ngTcDkL3z2fuY8NBl7Q\n",
"sN4rwmXAXUC3mNoaRa4AIsCFWDeVMcViXTGmWMTzMoAHgXb8OqgPC4a/DjxMjg7CbTl3Kq788nPJ\n",
"TiDZuNF+CDfp6Je4BG5MmrCuGBNX/iYZg4FGzL7lcpZ/OQKIkKMjgA+BXOB4VUrWV+6WnhiBW+q5\n",
"Oapr4hO5MWWL1dhNkYjnVQLeAWrxzaXXsfzLkcBN5OhE3AS02biaekmT+lG4SUeTgDMtqRtTcjZi\n",
"N7slnlcdGAXMZ3KXQWxZ9R5wJTm6DtcFc5MqQ0p+AukKPANcheqweMRsTFlmid3sknhePeBjYBwT\n",
"ThmJbn5lEVqEAAAVSElEQVSX3IyLuX3bIcB/gC6qTCrZwSUDCAO9gFNQnR6nsI0p0yyxm50Sz2sE\n",
"jAGGML7tVMgdwfqaF3L/8m7A0bild38v2cGlKq6NsQ5u0tGSeMVtTFlnNXZTKPG8o3BllgcY3/pb\n",
"yH2N30+4hPuX/x9QFWgRQ1I/AJgMLAfaWFI3Jr4ssZt/EM9rCYwFrmF86xXAC0zpN5Ahk5/0Hz+3\n",
"xLNJRdrgkvozQG9UN8cpbGOMz0oxZgfieafjWhq7M751feBeRj1zH9P6PApcqcrwkh1YBOiHW8ir\n",
"O6qfxytmY8yOLLGb7cTzLgLuAzoxvvXRKLfyyph3+K3dAKC9KjNKdmApBzwFNAdORPW3uAVtjPkH\n",
"S+wGAPG8a4EBQGvGtz4FlWt4dtpsFh/dFGimSsnq4CJ1cTsnLcUl9b/jFrQxplCW2Ms4f92Xu4Ez\n",
"gX8zvnVPtmVfylM/rGXFgYuBM1XZVLKDS1PcHqdDgQiquXEK2xizC5bYyzDxvEzcRcyj2LahJV90\n",
"HMjmyufzxJxK/N3gSeARVUq2mJDIecDjQF9U345f1MaY3bHEXkaJ51UAXgOqseXvtkzufDvrap3F\n",
"oNmVWFe3pyofFe+AUgc4DmgGnIDbZLodqrPiHbsxZtdsdccySDyvGvAusIw1P/Rk2lWPsaphZ56f\n",
"so4NtTqpMmfXB5AqQFPyE3kzoAbwNW69l6mAh+rqRH4OY8qiouROS+xljHheHeAjYCpznxjAvA9f\n",
"Ytmhp/DSZzPZWLOrKit3fINkA0eQn8CbAY2A78hP4l8Dv1gN3ZjEs8RudiCetx9uiYA3mdrjTlas\n",
"f48ljVsybORLbKw5QJFtwAG45J03Gm8C/E5+Ep8KfGsTi4wJhq3HbrYTzzsCt5jXA4xv/Rxr6k+o\n",
"M+/oJle+d+zg8LY9l+FWb2wGrCc/gf8fMM2W0DWmdLERexkgntdcVN/rNHnys69Fbt3o7V3ppsZ/\n",
"aZW9129Zl83Wr4guqaj+GXC4xphdsBF7WeXq4kcCxw3u0KFLjSuuaPfSPfdsPWXKlI5DDq9+2FeV\n",
"DtTfFjQ5eQAveFYXNyb92Ii9tHNrsOTVxfNuRwHzH+jWbWm4V6+mPceMGfjp+Myvfj3+oy9Zfugy\n",
"Pr3nKF1+kM0ANaYUsoun6cj1i+cl8LwLnNF18anANBk37kLgZqADVy7bk9bhMazafxoze7bQ78+1\n",
"UboxpZQl9tIuv188ejS+Bzv2i3+N6qLtb3FLBISBC4D2XLK1Naf1e4619d6l4fhzNBzQ/+DGmLgI\n",
"NLGLyKnAo0Am8IKq3lfc4MqU/Lp4dBI/gB37xacCc3dWFxfPy8BN42/BxoxTuTz7erp2H8Cmas/o\n",
"07OuSsbHMMYkVmCJXUQygZ+Ak4GFuBHm+ar6Y3GCS1uF18WbAPPYMYl/V9R+cfG8crit5urx4V7n\n",
"8eq6wXQ//WRyM+/Uh/+4IwGfwhgTgCC7YpoBc1V1nh/IMOAM4McdXiXyNDAMmJjW3Rlu6dro6ffH\n",
"AevIn7V5KzH0i4vnVcYtjbuRfkdfxsq5n3Fh+4ZkbRqod60dFJfPYIwpNRKV2PcG/oi6vwA4vpDX\n",
"zQceA2oj8hbwBq5mXHrrwK4ufgw7JvLouvggCtTFYzqd5+0JfAD8wOkthlBt5mR6talAuXV9NCf3\n",
"1XicwxhTuiQqsRcpMQtUAN5tBLVugBq94RUgGzfCH4YrRaRukt95XfxbXBJ/Hzca32ldPKbTe14D\n",
"4BPgA9q0+p69v3yfXm2ErE2XaFhHxPt8xpjkE5EQECrWexJUY28O5Kjqqf79m4Hc6AuohdaJXO25\n",
"CXCef1uHS/DDUP057oEWh4utETuWU2Kqi8cUjucdAnzCNgZxcqg2B4w9nx4dKpKxraeG9YNEn98Y\n",
"E4wgL55m4S6etgX+xCW84l08dYm0OS7Bn+sfZxjwFqrz4x70P8+/q7p43m16EOuoiOcdA4xmZfYd\n",
"nNWiE4cPr8c53fZB9DwN66fJjscYkzxBtzt2IL/dcbCq3lPc4KJenAm0wiX5s3B/NIYBw1FdHIdg\n",
"8+ri0SWVauyiXzwo4nltgGHM2iPMNUdfTbMnfqXD1c0QztKwfhF0fMaYxErPCUpux/uTcUn+dGAa\n",
"Lsm/g+qKIrw/G2jMjqPx6Lp43u3XVOvUEc87C3iG4Q3uZdCBN9L+unc58eEzgU4a1q+Djs8Yk3jp\n",
"mdh3PEhFoCMuybcHJuKS/Huo/l1IXTxvHZX/seNoPCl18ViI512OEuHuQ1/m0716cf7pQzhk9MXA\n",
"qRq27eeMKSvSP7HveMCqQGdckj8JmA0cBqylwDoqqJaaBbD8JQJuROlN/6On8MMejbnyiDeo88MV\n",
"QHsN64+7O4YxJn2UrcS+48Fr4mrms1OhLl5S/hIBD7BVOnBhs9UsrriUG2tMouKqvkBbDeuvQcdo\n",
"jEmuspvY04B4XjbwAusz/8X5zWuyJvs1bstcR0ZuL1xS/z3gEI0xAShK7sxIVjCm6MTzKgLv8Ff5\n",
"I+h6YgP+zriZHMklI7c7cJIldWPMrlhiTzHiedVRxvBj1QZccPxebM7tSDjrOOA0IKTh0ltaMsYk\n",
"hyX2FCKetxe5TGB87dr0a7qF7JXHc1u5S4ETgDYa1qVBx2iMSX2W2FOEeF4jtspXvLXPntx++DQO\n",
"fq8NN9a6G9fZ007DujLoGI0xpYNtZp0CxPOOYouM5ZlG2bzT4D6uPvBhav76KlAd6KBhXR90jMaY\n",
"0sMSe8DE81qyKWM0Dx4Cn9btQY6MBfJWZuysYd0YZHzGmNLHEnuAZMz4M9iW8QZ3HL6aL2u1I0d+\n",
"wy31uwq4QMO6JeAQjTGlkNXYAyIjJ/VmU+ab3Nr4R76s1Zgc+R34GFgEdLekbowpKUvsAZBXp9xF\n",
"rjzJrUe+y/QaJ5Aj24CxwPfAxRrWrQGHaIwpxawUk0TiecLcKq9QnvO4/fCwzqp+l0SkNjAGGAdc\n",
"p+EU3jHKGFMqWGJPEvG8TH6s6pGpzXnwkHN0VvWREpH6wKe4jahvs6RujIkHS+xJIDfOqcK+1Wax\n",
"Tery4n7H6Kzq30pE9gM+A4ZoWO8OOkZjTPqwGnuCSZeF+7H/ut/ZlFGRkXs31Kk1v5WIHAiMB560\n",
"pG6MiTdL7Akk7Za0pNOfP7E2az7f7bGfjquzTCJyGOABd2tYHw04RGNMGrLEniDScuklXDRvHKuz\n",
"P+bYlU31xf23SESOAj4HbtawPhd0jMaY9GQ19jgTQThh2UNc+8vVLKnwuF7b5FoAiUgzYBTQX8M6\n",
"PNgojTHpzBJ7HIlQkROXjeTGOW1ZUW6g9mv6BIBEpCWu8+USDevoYKM0xqQ7S+xxIkJ9Tlr6OdfP\n",
"aciWjHO1V7ORABKRk4E3gPM1rJ8GG6UxpiywxB4HIhxH+8Ufcc3PFSiX21Y7tZwEIBHpBAwBztKw\n",
"Tgw2SmNMWWEXT2MkQne6LvicgT9DxdwTtG0oL6mfAwwGOllSN8Ykk43YS0iEDETvpPdvfem6YAXl\n",
"NKSh0P8AJCIXAvcD7TWss4KN1BhT1lhiLwERqpKhr3LTj8fSeukCsvRkDYWWAEhEegO34bay+zHY\n",
"SI0xZZEl9mISYX+yc0fxwKwKNF79KxmcrqHQagCJyDXANbhNp+cGG6kxpqyyGnsxiNCKilu/YsjX\n",
"Gfxr9fdkcEpUUr8F6Ae0sqRujAlSQhK7iOSIyAIRmeHfTk3EeZJJhD5U2zKcYV8tp8GGrxC6aii0\n",
"QSIiEpE7gR64pD4/6FiNMWVbokoxCjysqg8n6PhJI0I28Ah1N57Ky1NWU05HAzdqKKQSEQEeAtrg\n",
"kvrSQIM1xhgSW2OXBB47KUSoCQznoL+zeGZaNhk8qaHQAwASkQzgKaAp0FrDujLIWI0xJk8ia+xX\n",
"icgsERksItUTeJ6EEOFwYCotlv7Js9MOJoNIVFLPAl4EjgDaWVI3xqQS0RJu2iMiY4G9Cnnqv8BX\n",
"QF5Z4g6gnqpeWuD9qqopOaoXoSMwlO7zh3D5/y4Bemso9C6ARCQbeBWoAZypYV0fYKjGmDKmKLmz\n",
"xKUYVW1XxCBewK1qWNhzOVF3PVX1ShpPPIggwH+AgeTMfoRWywYC52oo5AFIRCoAb/kv76xh3RhM\n",
"pMaYskJEQkCoWO8p6Yh9N4HUU9VF/s8DgeNUtXuB1yjjxp0FvKuhUOB7fYpQAXgOaMxLU95g3w3X\n",
"AJ00FJoOIBGpBLwLrAIu0LBuCS5aY0xZVZQRe6Jq7PeJyLciMgtoBQws9FVLyr/M8nJfytjxByQo\n",
"jiIRoR4wDtGKfDjhHfbd0BdoFZXUqwIfAYuA7pbUjTGpLCEj9iKdWESptKUv3f7I4awFtfh+j9F8\n",
"t0cvfXW/VcmNg2OAkWToEMaOr0oGpwCnaCi0EEAiUgP4GJgBXKlhzU1mfMYYEy3IEXuR6LqsZ3hx\n",
"/3o8dMhFlN/WnI6LlslZC14XoX4yzi9CN+BjKm/9D5+Nb0gGzYGTopJ6bdxWdpOBvpbUjTGlQaAj\n",
"9oJ/deTBWb05cO0DfLdHRV7YfzS/V75blW/if24ygAjQk/3WncvQr28FMoFzNBRaByARqQd8CowE\n",
"/k/DAf2ijDEmSlFG7CmV2AHE8yqyLjOHTO3Ha/ttZtg+P7I141FgpCpbYz8vVYCXgbp0WdCLq+cO\n",
"Bv4AemkotAVAIrIv8Bnwoob17ljPaYwx8VIqE/v25z3vIHJ5inVZh3DXYSuZsmcN3EzP51Up0YQg\n",
"ERoC7wHTeHpaDof+/T4wAbhGQ6FcAIlII1xSf1TD+mhJzmOMMYlSqhM7gHieAGcDD7O03EyuPnoD\n",
"iyu2w+0h+rgqc4p+PlrietDvZ8z498jWscBQ4M68dkuJyGHAGOBODeuzJfpgxhiTQKU+sW9/redV\n",
"BcJATxaXf4gLj6/M1ow+wHTgUWCMKjv9ICJcBtwNXMg4bxGudfFODYWe3v6aiByF6365QcP6Sok/\n",
"mDHGJFDaJPbt7/G8xsDTQHmWlhvAuScegtvYIht4HHhFlXX55yALt/piB+B0xnm1gbeBqzQUemv7\n",
"6yLSDDc7tr+GdXiMH80YYxIm7RI7gHheBtATuBd4h/WZ/+W0lkfhEnwL3AbSTwFrgTf9t3VjnNcC\n",
"GAJcoKHQ2O3Hi0hLXLK/VMNa6NIHxhiTKlK+j70kNBTK1VBoKHA4IFTa9gPjvH0Z53UBmgPlgVnA\n",
"j8D3QEfGeacDL+CWCIhO6icD7+CWCLCkboxJC8HOPM0hI9b+cPG843DlmXVAPw2FZotQDThUlani\n",
"eQNxSxqcoqHQ9s2lJSKn4Zbe7aphnRhLDMYYkywpX4ohh9+AEcBwYFpJk7x4XibQBzfpaKj/7zrg\n",
"LuAsoL2GQr9vf31EzsaVa07XsE6N5XMYY0wylYbE3hTXzngO7gLoCP82tSRJXjyvLnA/0Br4BtgH\n",
"6Kih0PYt6yQiPYAHgA4a1pkxfxBjjEmilE/secH5e4c2xiX4c4CK5Cf5KcVdo0U8rxXuD8YtGgr9\n",
"vf3xiPQGbgPaa1h/iMsHMcaYJCo1iX2Hx12SPwKX4M8GquG6VoYDX5Z0IS6JyABcrf1kDevcEgdu\n",
"jDEBKpWJ/R+vi8jh5JdrauKS/AhgkoZ1W5HOFZGbgUuBthrW+SWP2hhjgpUWiX2H90TkUPKTfB3y\n",
"k/zEwpK8P/q/HeiKG6n/GXPgxhgToLRL7Du8PyIH45L82UB93PK6w4EJGtatflJ/CGgDtNOwLt3p\n",
"wYwxppRI68S+w7EiciBuVH4OsC8uyVcADgVO1bCWaDVIY4xJNWUmse9w3IjsjxvFN8It6LUm3ucw\n",
"xpiglMnEbowx6Swt14oxxhiza5bYjTEmzVhiN8aYNGOJ3Rhj0owldmOMSTOW2I0xJs1YYjfGmDRj\n",
"id0YY9JMiRO7iJwjIt+LyDYRaVrguZtF5BcRmSMi7WMP0xhjTFHFMmL/DugCTIh+UEQOB7rhNps+\n",
"FRgkImn3zUBEQkHHEAuLP1gWf3BKc+xFVeKEq6pzVPXnQp46A3hDVbeo6jxgLtCspOdJYaGgA4hR\n",
"KOgAYhQKOoAYhYIOIEahoAOIQSjoABItESPp+sCCqPsLgL0TcB5jjDGFyNrVkyIyFtirkKduUdVR\n",
"xThPMCuNGWNMGRTz6o4iMg64TlWn+/dvAlDVe/37HwNhVZ1S4H2W7I0xpgR2t7rjLkfsxRB9kveB\n",
"10XkYVwJ5iBganEDM8YYUzKxtDt2EZE/gObAByLyEYCq/gC8BfwAfARcqUEt+m6MMWVQYBttGGOM\n",
"SYxA+8tF5AER+VFEZonIOyKyR5DxFNeuJmmlMhE51Z889ouI3Bh0PMUhIkNEZImIfBd0LMUlIvuI\n",
"yDj//zOzReTqoGMqDhGpICJTRGSmiPwgIvcEHVNJiEimiMwQkeI0gKQEEZknIt/68f+jxJ0n6IlD\n",
"Y4AjVPUo4Gfg5oDjKa5CJ2mlMhHJBJ7ETR47HDhfRA4LNqpieREXe2m0BRioqkfgSpj9StPvXlU3\n",
"Aq1VtQnwL6C1iPw74LBKYgCuVFwayxUKhFT1aFXd6fygQBO7qo5V1Vz/7hSgQZDxFNcuJmmlsmbA\n",
"XFWdp6pbgGG4SWWlgqpOBFYGHUdJqOpiVZ3p/7wW+BE376PUUNX1/o/lgExgRYDhFJuINAA6Ai+w\n",
"Y9NHabLbuIMesUe7BPgw6CDKgL2BP6Lu2wSyAIhIQ+Bo3ICm1BCRDBGZCSwBxvnNEqXJI8D1QO7u\n",
"XpiiFPhURL4Rkct39qJ4tTvuVFEmOYnIf4HNqvp6ouMprjhO0koVpfHrZ1oRkSrACGCAP3IvNfxv\n",
"2E3862GfiEhIVb2AwyoSEekE/KWqM0rxejEtVHWRiNQGxorIHP9b7A4SnthVtd2unheRXrivRm0T\n",
"HUtJ7C7+UmghsE/U/X3YcQkIk0Aikg28Dbyqqu8GHU9JqepqEfkAOBbwAg6nqE4EOotIR6ACUE1E\n",
"XlbVngHHVWSqusj/d6mIjMSVVv+R2IPuijkV97XoDP/CTGlWWup13wAHiUhDESmHW4nz/YBjKhNE\n",
"RIDBwA+q+mjQ8RSXiNQSker+zxWBdsCMYKMqOlW9RVX3UdX9gfOAz0tTUheRSiJS1f+5MtAe18Dx\n",
"D0HX2J8AquC+UswQkUEBx1MsO5uklcpUdSvQH/gE1xnwpqr+GGxURScibwCTgYNF5A8RuTjomIqh\n",
"BdAD100yw7+Vpg6fesDnfo19CjBKVT8LOKZYlLayZF1gYtTvf7SqjinshTZByRhj0kzQI3ZjjDFx\n",
"ZondGGPSjCV2Y4xJM5bYjTEmzVhiN8aYNGOJ3Rhj0owldmOMSTOW2I0xJs38P+KRKkqlz/nMAAAA\n",
"AElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x105aad0d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"\n",
"def plot(n):\n",
" X = construct_poly(x,6) \n",
" Xt = construct_poly(xt,6) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" print MSE(X.dot(w),y)\n",
" \n",
" w = wRidge(X,y,0.1)\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" print MSE(X.dot(w),y)\n",
" \n",
" w = wRidge(X,y,10.0)\n",
" plt.plot(xt,Xt.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" print MSE(X.dot(w),y)\n",
" \n",
" plt.plot(xt,yt)\n",
" #plt.plot(x,y)\n",
"plot(1)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Hand in printed copy of completed notebook."
]
}
],
"metadata": {
"annotations": {
"author": "",
"categories": [
"intelligent-systems-1-2015"
],
"date": "2015-04-30",
"location": "Beginning of next lecture",
"parent": "IS_SS2015",
"submission_date": "2015-05-07",
"subtitle": "Exercise Sheet 3, Linear Regression",
"tags": [
"IntelligenSystems",
"Course"
],
"title": "Intelligent Systems 1 - Summer Semester 2015"
},
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
MaxXximus92