{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"preamble": true
},
"source": [
"(Defining latex commands: not to be shown...)\n",
"$$\n",
"\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
"\\DeclareMathOperator{\\minimize}{minimize}\n",
"\\newcommand{\\real}{\\mathbb{R}}\n",
"\\newcommand{\\normal}{\\mathcal{N}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gaussian Algebra (25 Points)\n",
"\n",
"Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
"\n",
"\\begin{equation}\n",
" \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
"\\end{equation}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
"\n",
"Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
"\n",
"\n",
"Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Linear regression (50 Points)\n",
"\n",
"In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
"\n",
"1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
"2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
]
},
{
"cell_type": "code",
"execution_count": 294,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Loading the required packages\n",
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from IPython.html.widgets import interact\n",
"import csv"
]
},
{
"cell_type": "code",
"execution_count": 295,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [],
"source": [
"def wRidge(X,y,lamb):\n",
" # Change the following line and implement the ridge regression estimator wRidge\n",
" #w = np.zeros(X.shape[-1])\n",
" #print y\n",
" #print np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)\n",
" return np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
]
},
{
"cell_type": "code",
"execution_count": 296,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Read ex1_train.csv and assign the first column and \n",
"# second column to variables x and y respectively.\n",
"\n",
"\n",
"a = np.genfromtxt('ex1_train.csv', delimiter=' ')\n",
"x= a[:,0]\n",
"y= a[:,1]\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4) Plot the training data with appropriate labels on each axes!"
]
},
{
"cell_type": "code",
"execution_count": 297,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"text/plain": [
"<matplotlib.text.Text at 0x107627890>"
]
},
"execution_count": 297,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAYYAAAEPCAYAAABGP2P1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAG1tJREFUeJzt3X285nOdx/HX2wxRhFQMxtJETDGjZZasHCsaYmdnVXIT\n",
"IjfRNCuVu12mKCpJW9RaN7EMKy2LSiaczCqDjDGMCWW2QSYKyza2YT77x/dH5zrnzJzrOue6ru/v\n",
"d13v5+NxPeZc51w378PM73N97xURmJmZvWqV3AHMzKxcXBjMzKyGC4OZmdVwYTAzsxouDGZmVsOF\n",
"wczMamQtDJIulrRE0vw+35sh6XFJc4vb5JwZzcy6Te4WwyVA/wt/AOdExLbF7aYMuczMulbWwhAR\n",
"s4FnB/mR2p3FzMyS3C2GFZkmaZ6kiyStkzuMmVk3KWNh+DawGTAR+C3wtbxxzMy6y+jcAfqLiN+9\n",
"+rWkC4Eb+j9Gkjd4MjMbhogYsqu+dIVB0piI+G1xdyowf7DH1fPLlZWkGRExI3eO4XL+vJw/nypn\n",
"h/o/VGctDJKuBHYB3ixpMXAa0CNpIml20mPAURkjmpl1nayFISL2H+TbF7c9iJmZvaaMg8/doDd3\n",
"gBHqzR1ghHpzBxih3twBRqg3d4AR6M0doB1UxYN6JEWVxxjMzHKo99rpFoOZmdVwYTAzsxqlm65q\n",
"ZmbNI7Ee8I7iVhcXBjOzipNYDRjHnwtA39uqwC+LW32v58FnM7PykxCwPoNf/McCi/lzAeh7WxJB\n",
"pNeo79rpwmBmViISawCbM/DivwXwMoNf/H8VwZ+Gfm0XBjPrIMUn5rWBMcCGxZ/9b2/iz5+cFzLI\n",
"p+YyKH6Xjai98G9Z/LkB8GsGKQAR/H5k7+vCYGYVUFwk12Pwi33/7y0j7brc//Zk8eezwF+w8n72\n",
"vrdHIljawt9tTdIn/cE+/b/YJ0ffIrYogpdbk8eFwcxKRGIU8AngndRe7NcH/peBF/kBtwheHOZ7\n",
"952Z0/f2tuK1BysaT9TTyih+r01W8PpvAh4d7PUjeH44v8tIuDCYWWlIrAVcCawJXEPtxf+pCF7K\n",
"lGs06fyXwS7qawIP069YkFokW/Z53Djg9wxeXH4TwfL2/UYr58JgZqUgsQnpXJU7gU9GsCxzpLpI\n",
"rMPAbqCNgEUM7I4aVkum3VwYzCw7ie2B60gnMX69TAPA3ajea6cXuJlZS0h8EDgf+HgE1+fOY/Vz\n",
"YTCzpipmGZ1IGmh+fwRzM0eyBrkwmFnTFFszXAC8C9ghgiczR7Jh8O6qZtYUxZTQWaRFaLu4KFSX\n",
"C4OZjZjEO0izju4E9o3gfzNHshFwYTCzEZHYFbgdOCuCE8o0b9+Gx4XBzIZN4nDgKuAjEVyUO481\n",
"hwefzaxhEqsAZwFTgZ0jeDhzJGsiFwYza4jEG4DLSfsA7TDSHT+tfNyVZGZ1k9iINJ7wHLC7i0Jn\n",
"cmEws7pIvJs06+h7wGH1HAxj1eSuJDMbksQU4ELg6Ai+nzuPtZYLg5mtULG9xaeB44C9Irg7cyRr\n",
"AxcGMxuUxKrAecAkYMcIFmeOZG3iwmBmA0isSxpLWEqajvpC5kjWRh58NrMaEuOAnwPzgb9zUeg+\n",
"Lgxm9hqJnYE7gHMjOC6CV3JnsvZzV5KZASBxMHA2cFAEN+fOY/m4MJh1uWJ7iy8ABwA9ESzIHMky\n",
"c2Ew62ISawCXkg65/6sIns4cyUrAYwxmXUpiA6AXWAbs5qJgr3JhMOtCEtuQtrf4AWlM4aXMkaxE\n",
"shYGSRdLWiJpfp/vvUnSLEkPS7pZ0jo5M5p1Gom9gJ8AJ0bwhQgidyYrl9wthkuAyf2+dyIwKyK2\n",
"AG4p7ptZE0hMI+15NCWCq3LnsXJSRN4PC5I2BW6IiK2L+wuBXSJiiaQNgN6I2LLfcyIi1PawZhUl\n",
"MRo4F+gB9o5gUdZAlkW9184yzkpaPyKWFF8vAdbPGcas6iTWJh2/KWCnCJ7PHMlKroyF4TUREZIG\n",
"bdJImtHnbm9E9LYllFmFSGwK3Aj8FJgewct5E1k7SeohtRIbe15Ju5J6IuIpSWOA29yVZNY4iR2B\n",
"75POZv6mB5mt3mtn7sHnwVwPHFJ8fQhwXcYsZpUk8RHgP4EjIvhnFwVrRNYWg6QrgV2AN5PGE04l\n",
"/WW+GtgEWAR8OCKe6/c8txjMBlEcrPNPwOHAPhHcnzmSlUi9187sXUnD4cJgNpDE6qSpqJuTpqM+\n",
"lTmSlUyVu5LMrEESbyGt+1mNtBGei4INmwuDWcVJjAfmALcBH4lgaeZIVnGlnq5qZisnsTtwBfCZ\n",
"CC7Lncc6g1sMZhUlcTTwb8C+LgrWTG4xmFWMxCjSSWt7An8dwaOZI1mHcWEwqxCJtYCZwOuBHSN4\n",
"NnMk60DuSjKriOIIzhtIa34muyhYq7jFYFYdhwKrA0dGsDxzFutgXuBmVgES6wEPAntGMDd3Hqsm\n",
"r3w26yASFwIvRvAPubNYdVX5PAYz60NiJ9JJh+NzZ7Hu4MFnsxKTWBX4DvDpCP4ndx7rDi4MZuU2\n",
"HXgS+F7uINY9PMZgVlISY4G5wA5exGbN4N1VzarvG6ST11wUrK08+GxWQhIfALYGDsidxbqPC4NZ\n",
"yUi8HvgWaSHbS7nzWPdxV5JZ+ZwCzIlgVu4g1p3cYjArEYmtgCOBCbmzWPdyi8GsJCQEnA+cHsGT\n",
"ufNY93JhMCuPA4G1ScXBLBuvYzArAYl1gQXAlAjuyp3HOpM30TOrEInzAUXwidxZrHN5Ez2zipCY\n",
"BEzFm+RZSXiMwSwjidGkTfI+6xPZrCxcGMzyOgZ4DrgidxCzV3mMwSwTiQ2BecDOESzMncc6nzfR\n",
"Myu/c4ALXBSsbDz4bJaBxO7AJOCw3FnM+nOLwazNJFYnLWKbFsEfc+cx68+Fwaz9TgDmR/CD3EHM\n",
"BuPBZ7M2kng7cCewbQSLc+ex7uLBZ7OSKTbJOw84y0XBysyFwax9PgRsSDqy06y0PCvJrA0k3kia\n",
"nrpfBMty5zFbmdKOMUhaBPwP8AqwLCIm9fmZxxisUiTOBdaK4PDcWax7dcImegH0RMQfcgcxGwmJ\n",
"bYH9gXfmzmJWj7KPMbhVYJUmsQrwbeDkCJ7JncesHmUuDAH8RNI9ko7IHcZsmI4gdYdekjuIWb3K\n",
"3JW0U0T8VtJbgFmSFkbE7NyhzOolsT5wBrBbBMtz5zGrV2kLQ0T8tvjzaUnXkvaVea0wSJrR5+G9\n",
"EdHb1oBmQ/sqcGkE9+cOYt1JUg/Q0/DzyjgrSdLrgVER8YKkNwA3A5+PiJuLnwfEThH8LGtQsxWQ\n",
"6AEuA8ZH8GLmOGZA9Vc+rw/MlnQfMAe48dWi0Mdn2h/LbGgSq5E2yZvuomBVVMoWw1CKFsMzwHsi\n",
"eCR3HrO+JE4CdgL2iaB6/8CsY9XbYqhyYTgDWC+CY3LnMXuVxGbA3cD2ETyWO49ZX91QGDYAFgJb\n",
"RPB07kxmxSZ51wM/j+BLufOY9Vf1MYYhRbAEuAb4RO4sZoUpwNuBs3MHMRuJyrYYIkISWwG9wKYR\n",
"LM0cy7qYxJrAg8ChEdyWO4/ZYDq+xQAQwUPAXcBHc2exrncqcLuLgnWCSrcY0tfsAlwAbOXVpZaD\n",
"xLuAW4Gtiy5Os1LqihZD4XbS9tx75w5i3afPJnmnuShYp6h8YSjmiZ+NF7xZHocAryO1Ws06QuW7\n",
"ktJ9RgOPkk7HmpMvmXUTifVIA857RXBv7jxmQ+mmriQieBn4OnB87izWVc4CrnZRsE7TES2G9D3W\n",
"Ah4DJkXw6zzJrFtI7ARcTdok7/nceczq0VUtBoAIXgD+FfiH3Fmss0msShpwPt5FwTpRx7QY0vfZ\n",
"EHgAeHsEPivaWkLieGAysIc3ybMq6fi9klb0y0lcAjzivWqsFSTGAnNJO/s+nDuPWSO6uTC8C5gF\n",
"bBbBS+1NZp1O4vvAAxGcljuLWaO6bozhVRE8QPpEd2DuLNZZJPYCJgBn5s5i1kpDFgZJn5K0bjvC\n",
"NNHZwPHFqlSzEZN4PfAt4Fi3RK3T1XPhXB+4W9LVkiZLGrIZUgK3AS8Be+YOYh3jZOCeCH6cO4hZ\n",
"q9U1xiBpFWAP4FBgO9L87Ysi4lctTbfiPEP2k0kcABwRwa5timUdSmJLYDYwMYIncucxG66mjjFE\n",
"xHLgKWAJ8AqwLnCNpK+OKGVrfQ8YJ7Fd7iBWXcWpbOcDZ7goWLcYssUgaTpwMPB74ELg2ohYVrQi\n",
"HomIca2POSBTfVOuxKdJZ+/u34ZY1oEkDiRt0Lh9sfWKWWU1bbqqpM8DF0fEfw/ys/ERsWD4MYen\n",
"gcLwRtI2GX8ZwaKWB7OOIrEOsACY6s0ZrRN07TqGgY/lK8CqERzX4ljWYSTOA0ZFcHTuLGbN4MLw\n",
"2mPZGLgfGBfBs61NZp1CYnvgetImef57Yx2haxe49RfB48CNwJG5s1g1SIwCvgOc4KJg3ajjWwzp\n",
"8UwAfkjaJuNPrUtmnUBiGrAvsKs3ybNO4hZDHxHMI5205dlJtlISY4BTgWNcFKxbdUVhKJwNfKaY\n",
"l262IucAF0bQ9tl2ZmXRTYVhFhCkFdxmA0jsDuwInJ47i1lOXVMYim6Bs0mLlcxqSKwOnAdMi+CP\n",
"ufOY5dQ1haFwFbCVxMTcQax0PgcsiOCG3EHMcuuKWUm1z+VzwNYRfLTJsayiJN4O3Am8O4Lf5M5j\n",
"1ipe4LbC57IO8GtgQgSLm5vMqqaYjPAj4NYIvpI7j1krebrqCkTwHPBdYHrmKFYOHwQ2Br6eO4hZ\n",
"WXRdiyE9n01Ix3++LYLnm5fMqqTYZHEBsH8Es3PnMWs1txhWouhHvgk4IncWy+rzwCwXBbNapSwM\n",
"xRGiCyU9IumEFr3N14DpEqu16PWtxIqZaQeSZiOZWR+lKwySRpEOXZ8MjAf2l7RVs98ngnuBh4EP\n",
"N/u1rdwkVgG+DZwSwdO585iVTekKAzAJeDQiFkXEMtLagyktei9vk9GdPk5aBX9R7iBmZVTGwrAR\n",
"1Ewjfbz4XivcBKwK7Nai17eSkXgrcAZwdATLc+cxK6MyFoa2TZPyNhld6Rzg3yK4P3cQs7IanTvA\n",
"IJ4Axva5P5bUaqghaUafu70R0TvM95sJfFFiG18sOpvE3qRN8rbJncWsHST1AD0NP69s6xgkjQZ+\n",
"SereeRK4C9g/Ih7q85gRrWMY+J6cBLwjgkOb9ZpWLhJrAw8Ah0Rwa+48ZjlUeksMSXsC5wKjgIsi\n",
"4sx+P292YXgT8ChpD6UnmvW6Vh4SFwARwVG5s5jlUunCMJRmF4b0mnwDWBrBic18XctP4n3AxaTC\n",
"75Xu1rVcGBp+TTYD7iadC/1CM1/b8pFYE5gPHBvBD3PnMcvJW2I0KILHgFuAw3Nnsab6EnC7i4JZ\n",
"/dxiqHldtgeuAcZF8HKzX9/aS2Jn4N+Bd0Xwh9x5zHJzi2EYIrgbWETaitkqTGIN0srmY10UzBrj\n",
"wjDQ2cCpEhvnDmIjMgO4L4JrcwcxqxoXhoFuBK4AfiG55VBFRZfgocC0zFHMKsljDCt8DyaRCsRs\n",
"YLpnKlWDxOuAXwBfimBm7jxmZeIxhhGK4C5gW2A5MFdih8yRrD4nk870vjJ3ELOqcouhrvfj70n7\n",
"959H+iTqGUslJDEB+Akw0SvYzQZyi6GJIvgP4N3AzsBPJd6WOZL1IzGatLr5BBcFs5FxYahTcbF5\n",
"P2mdwxyJg33AT6l8Bvg9cEnuIGZV566kYb0/E0gD0w+SDnx5NlcWA4mtSJMEtotgUeY4ZqXlrqQW\n",
"imAesD2wBJgnNb7fuTWHlHbgBU5zUTBrDheGYYpgaQSfAo4ErpA4S2K13Lm60DTgZdLkADNrAncl\n",
"NYHEW0ifWjcCDoxgYeZIXUFiHDAH2DGCR3LnMSs7dyW1UQRPA1OAC4DZEkd7YLq1JFYBLgTOdFEw\n",
"ay63GJpMYkvSwPQTwMcj+F3mSB1J4ijgMOA9EbySO49ZFbjFkEnRjbQjacbSfRJ7Zo7UcSQ2Ac4A\n",
"DnNRMGs+txhaqJitdClwPfC5CJbmTVR9RRfdD4E7Ijgjdx6zKnGLoQQi6AUmAm8B7inWP9jIHAxs\n",
"AHw5dxCzTuUWQxsUn3IPAs4BzgTOjWB53lTVIzEGmAe8P4K5ufOYVU29104XhjaS2Ay4HPgjcKj3\n",
"9KlfUVyvBR6I4B9z5zGrIncllVAEjwG7ALcD9xa7tlp9PgxsAZyeO4hZp3OLIZPifIfLgZ+SDgJ6\n",
"MXOk0ioWEN4P/F0Ec3LnMasqtxhKLoI7SQcBQToIaFLOPCX3DWCmi4JZe4zOHaCbFceFHi6xL3CD\n",
"xDdJK3k9N78gMQWYBGyTO4tZt3BXUklIbAx8F1gd+GgxHtHVJNYFHgAOiOCnufOYVZ27kiomgseB\n",
"PUgzb+6SOMj7LfE14DoXBbP2couhhCQmAjNJc/Y/EcFzmSO1ncT7gX8Bti663MxshNxiqLAI7gP+\n",
"EniGdBDQLpkjtZXEWqSicKSLgln7ucVQchJ7kbaXvpR0StmfMkdqOYnzgDUiOCx3FrNO4pXPHUTi\n",
"raSDgMaQDgL6ZeZILSPxPtIg/NY+S9usudyV1EGKMx3+llQc/kviyE4cmJZ4L2ls5WAXBbN83GKo\n",
"GImtSAcB/QY4ojg9rvKKLcqvBvaP4JbMccw6klsMHSqCh4AdgF+SDgKanDnSiEn8Dako7OeiYJaf\n",
"WwwVJrEraVD6WuCECF7KHKlhxZjCTOBDXq9g1lqVbTFImiHpcUlzi1vlPxG3SgS3ARNIg9J3S2yd\n",
"OVJDJPYgFYV9XRTMyqN0hQEI4JyI2La43ZQ7UJkVg7T7AWcDt0ocJ5Xy/2uNogvscmBqBLNz5zGz\n",
"PyvrBaTru4kaEUFEcCnwV8CHgJskNswca4UkPgBcBkyJ4I7cecysVlkLwzRJ8yRdJGmd3GGqIoJf\n",
"A+8F7iAdBDQ1c6QBJPYBLgH2ieDnufOY2UBZBp8lzSId6N7fKcCd8NoUzNOBMRFxeL/nB/D5Pt/q\n",
"jYjeFkStLIkdSV01twLHleEgoGIL7QuAvSO4O3ces04nqQfo6fOt0yq/8lnSpsANEbF1v+97VlId\n",
"ij2H/hnYibRiOtvFuDjG9HxgrwjuzZXDrJtVeVbSmD53pwLzc2WpugheiOBjpJbYjRKnSIxqdw6J\n",
"D5KKwp4uCmblV7oWg6TLgImk2UmPAUdFxJJ+j3GLoUHFQUCXkU7t+2gE/92m990POBeYHMG8dryn\n",
"mQ3Om+jZAMU01uOBz5LGHa5o8fsdQDpsZ48It/zMcnNhsBWS2Ja039Jc4NhWHAQkcRDwFWD3CB5s\n",
"9uubWeMqO8ZgrRfBXGA74DnSfks7N/P1JQ4Bvgy8z0XBrHrcYuhyEnsD/wpcDMyIYNkIX+8w4Auk\n",
"orCwCRHNrEncYrC6RHAjabB/InCHxBbDfS2Jj5PWl/yNi4JZdbkwGBEsAfYmnZx2h8QRjR4EJHEU\n",
"cCqwawQPNz+lmbWLu5KshsR40sD0ItJBQM/U8ZxjgM8Bu0Xwq9YmNLPhcleSDUsEC0gHAT1KGpje\n",
"Y2WPl/gkafrrri4KZp3BLQZboeJktUuBa4CT+h8EJDEdmE4aU1jU/oRm1gi3GGzEIriVdBDQxsBd\n",
"fQ8Ckvg08Cmgx0XBrLO4xWBDKgaiDwG+CpwBvA44gtRSWJwzm5nVzyufrekkxpG28l6PNKbwROZI\n",
"ZtYAFwZriWJ31tUiWJo7i5k1pt5r5+h2hLHOEcEr4KJg1sk8+GxmZjVcGMzMrIYLg5mZ1XBhMDOz\n",
"Gi4MZmZWw4XBzMxquDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxq\n",
"uDCYmVkNFwYzM6vhwmBmZjVcGMzMrIYLg5mZ1XBhMDOzGi4MZmZWw4XBzMxqZCkMkj4k6UFJr0h6\n",
"d7+fnSTpEUkLJe2RI5+ZWTfL1WKYD0wFbu/7TUnjgf2A8cBk4HxJHdeqkdSTO8NIOH9ezp9PlbM3\n",
"IstFNyIWRsTDg/xoCnBlRCyLiEXAo8CktoZrj57cAUaoJ3eAEerJHWCEenIHGKGe3AFGoCd3gHYo\n",
"26fxDYHH+9x/HNgoUxYzs640ulUvLGkWsMEgPzo5Im5o4KWiSZHMzKwOish33ZV0G3B8RNxb3D8R\n",
"ICLOKu7fBJwWEXP6Pc/FwsxsGCJCQz2mZS2GBvQNeT0wU9I5pC6kzYG7+j+hnl/MzMyGJ9d01amS\n",
"FgM7AD+Q9COAiFgAXA0sAH4EHBM5mzRmZl0oa1eSmZmVT9lmJdVN0umS5km6T9ItksbmztQISV+V\n",
"9FDxO/yHpLVzZ2rEyhYplpWkycXCyUcknZA7T6MkXSxpiaT5ubM0StJYSbcVf2cekPSp3JkaIWl1\n",
"SXOK680CSWfmzjQckkZJmitppROAKlsYgK9ExISImAhcB5yWO1CDbgbeGRETgIeBkzLnadSgixTL\n",
"StIo4FukhZPjgf0lbZU3VcMuIeWvomXAcRHxTlIX8rFV+u8fES8BuxbXm22AXSX9deZYwzGd1FW/\n",
"0q6iyhaGiHihz901gWdyZRmOiJgVEcuLu3OAjXPmadRKFimW1STg0YhYFBHLgKtICyorIyJmA8/m\n",
"zjEcEfFURNxXfP0i8BBp3VJlRMQfiy9XA0YBf8gYp2GSNgb2Ai6kdtLPAJUtDACSvijpN8AhwFm5\n",
"84zAYcAPc4focBsBi/vc9+LJTCRtCmxL+kBUGZJWkXQfsAS4rZgsUyVfBz4LLB/qgaUuDJJmSZo/\n",
"yG0fgIg4JSI2Ab5L+qVLZaj8xWNOAf4UETMzRh1UPfkrxLMsSkDSmsA1wPSi5VAZEbG86EraGHhv\n",
"lfZNkrQ38LuImMsQrQUoxzqGFYqI3et86ExK+Il7qPySDiU17XZrS6AGNfDfvwqeAPpOUBhL7fYr\n",
"1mKSVgW+D1weEdflzjNcEfG8pB8A2wG9mePU6z3A30raC1gdeKOkyyLi4MEeXOoWw8pI2rzP3SnA\n",
"3FxZhkPSZFKzbkoxsFVlVVhweA+wuaRNJa1G2sX3+syZuoYkARcBCyLi3Nx5GiXpzZLWKb5eA9id\n",
"Cl1zIuLkiBgbEZsBHwFuXVFRgAoXBuDMolvjPtKOh8dnztOob5IGzWcV08fOzx2oEStapFhWEfEy\n",
"8Engx6RZGf8eEQ/lTdUYSVcCPwO2kLRY0sdyZ2rATsBBpNk8c4tblWZYjQFuLa43c4AbIuKWzJlG\n",
"YqVdq17gZmZmNarcYjAzsxZwYTAzsxouDGZmVsOFwczMargwmJlZDRcGMzOr4cJgZmY1XBjMzKyG\n",
"C4NZE0javjh06XWS3lAcRjM+dy6z4fDKZ7MmkXQ6aYOyNYDFEfHlzJHMhsWFwaxJit1D7wGWAjuG\n",
"/3FZRbkryax53gy8gbQ54hqZs5gNm1sMZk0i6XrS2SBvA8ZExLTMkcyGpdQH9ZhVhaSDgf+LiKsk\n",
"rQL8TFJPRPRmjmbWMLcYzMyshscYzMyshguDmZnVcGEwM7MaLgxmZlbDhcHMzGq4MJiZWQ0XBjMz\n",
"q+HCYGZmNf4fBBSJ6m7pJCYAAAAASUVORK5CYII=\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x107742310>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot the input data here\n",
"plt.plot(x,y)\n",
"plt.xlabel('x')\n",
"plt.ylabel('y') "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
"Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
]
},
{
"cell_type": "code",
"execution_count": 305,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def construct_poly(x,d):\n",
" ## Implement a method which given an array of size N, \n",
" ## returns an array of dimension (N,d)\n",
" poly=np.empty((len(x),1), float)\n",
" xr = np.reshape(x,(len(x),1))\n",
"\n",
" for i in range(d-1):\n",
" poly = np.append(poly, (xr**i), axis=1)\n",
" #print poly \n",
" #print np.shape(poly)\n",
" return poly"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
]
},
{
"cell_type": "code",
"execution_count": 306,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def MSE(y_predict,y_true):\n",
" ## Implement mean squared error for a given input y and its predictions.\n",
" return np.linalg.norm(y_predict-y_true)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
]
},
{
"cell_type": "code",
"execution_count": 314,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecXFX9//HXm9B7/SFVkN4ElI5AQOliBASkCJGm1Aio\n",
"NOXmCiJIERQQlSZdBEQQBCISBKTol9CLhCK9KQgIQoD3749zQzabTbKzOzNnyuf5eMxjd6fc+5mU\n",
"95459xTZJoQQQmeZJncBIYQQ6i/CPYQQOlCEewghdKAI9xBC6EAR7iGE0IEi3EMIoQNFuIcQQgeK\n",
"cA8NJelpSe9JmqfX/WMkfSRp0ernhSVdIelVSW9IekDSbtVji1XPfavXbbvJnHN7SX+V9F9JN0+h\n",
"tl2r4+4xhefMLunCqq5Xq+9n6/H4EEnHSHpe0puS7pE0R/XYcEkf9qp5/R6vXVjSNZL+JelFST+T\n",
"NKR6bC1Jo6rHXpF0maRP9HjtSEnjehz3TUmLTflvI3STCPfQaAaeBHYcf4eklYCZqsfGuwD4J7Ao\n",
"MDfwNeDlXseaw/ZsPW6/ncw5/wWcDBw3uaIkzQUcATzYq47eRgLzAosDSwDzV/eNVwJrAWvZnh3Y\n",
"Bfhfj8dv71XzX3o89lPgNWABYBVgA2Df6rE5gTOBT1a3t4Bze7zWwCU9jju77aen8D5Cl4lwD81w\n",
"IbBrj593A84H1OO+1YDzbL9r+yPb99q+fiAns32T7cuBF6fwtB8Bp5J+EUzJCsBVtt+2/SZwVXXf\n",
"+F8QI4C9bD9bnfth2+/1eL16H7DXsX9j+33bLwPXjz+27ettX1Gd913gdGDdXsed0rFDl4twD81w\n",
"JzC7pGWrbocdSIHf+zlnSNphfFdNH+oSZpLWAD5DahlPzQ3AtpLmrMJ8W+C66rGVgA+A7apulcck\n",
"7dvjtQZWrbpzHpP0vfHdLj2OvZOkmSQtBGwO/HEydaxP+pTR89hbVd02D0r6Zj/eS+giEe6hWS4g\n",
"td43Bh4Gnu/1+HbArcD3gSerPvnVej3nNUmv97gtU2sRVbieDuzv/i2sdHr19V+kLpRxwM+r+xYG\n",
"5gCWAhYDvgKMlPSF6vG/ACvYno/0S2FH4Ds9jj0SWBF4E3gW+Jvt3/dR86dJfy49X3sZsCypy2gv\n",
"4ChJX+3H+wldIsI9NINJ4b4zfXfJYPsN24fbXpHUr30vqQukp3lsz9Xj9tgAatkXuN/23T3um9In\n",
"gouAx4BZgdlJ1w/Gf+p4t/r6A9vv2X4AuBTYonpPT9n+Z/X9g8APSL8AkCRSy/23wMykkJ5b0vE9\n",
"Ty5pSdInhQNt3z7+ftuP2H7JyR2kLqav1PQnETpahHtoCtvPkIJxc+DKqTz3X8BJwIJVV8iAT9vH\n",
"fRsBW1fdKC8C6wAnSfrpZI6xGfCL6lrAf4FfUIU3cH8N5x1v/C+SeYHPAqfZHmf738B5PY6NpE8C\n",
"o0i/PC6awjFDmESEe2imPYCNqguEE5F0vKQVJE1bDTXcB3jc9us9n9afk0iaRtKMwHTANJJmkDRd\n",
"9fBwUnfGyqQRKn8ndY8cOZnD3Q/sJWlGSTMBewP3Adh+gtSVdKSk6SUtR7qe8Ieqjs0lzV99vyzw\n",
"PSZ8GnmNdMF3n2o45ZykTzX3Vc9fCPgzKfx/2cd7HCZpLiVrAAcCk3TphO4V4R6axvaTtu/peVeP\n",
"72cCfge8DjwBLAJ8qdch3ug1ZvxbkznVrsA7wBnAeqTuk19UNfzH9ivV7WXgfeBN228BSNpZUs8L\n",
"l8OBpUnXCJ4j9a3v1uPxHUlDFf9FCvXv2R4/tn4j4D5JbwPXAlcAx1Z1GNgG2IoU9I8D7wEHVa/d\n",
"kzT8cmTPsew9zrtD9Zo3gV8DP7J9wWT+PEIX0mA365B0DrAl8Irtlar7RpL+cb5aPe3wgQ5rCyGE\n",
"ULt6tNzPJfVL9mTgZNurVrcI9hBCaKJBh7vtW0kfpXuLCRYhhJBJI/vcD5B0n6Szq4tFIYQQmqRR\n",
"4f5z0sWgVUgjAk5q0HlCCCH0YdpGHNT2K+O/l3QWcE3v50ga3JXcEELoUran2u3dkHCXtIDt8Ys2\n",
"bQ080Nfz+lNgq5I00vbI3HUMVNSfV9SfT7vVLjEnaR7DAcCNoJ3687pBd8tIugT4K7CMpGcl7Q4c\n",
"L+l+SfeRljE9aIoHCSGEMBGJeSSOBsYCnwLWtdm5v68fdMvd9o593H3OYI8bQgjdSGI+4GDSbOgr\n",
"gTVsnqz1ODFDdeBG5y5gkEbnLmCQRucuYJBG5y5gkEbnLmAQRucuoC8Sn5A4kbRQ3RzAZ2z2Gkiw\n",
"Qx1mqA6UJLdzn3sIIdSDxELAd0m7j10AnGDz3OSf37/sjJZ7CCFkILGoxBmkAScfACvYjJhSsNci\n",
"wj2EEJpIYnGJXwJjSAu/LWtziD3FbSFr1pChkCGEECYmsRRpU/YvkVYsXdqe6h6+6bWl5gGWqW79\n",
"EuEeQggNJLEcab+ATYGfAUvYvDHJ80pNDyzBhBDveZuOdKG137uPxQXVEEJoAImVSBu0bAicApzG\n",
"SL1F2kayrwBfhLSX7mN93F52kcK6v9kZ4R5CCHUksSrTv3kUc49dj+Wv+CNr/vQpZnh7cVKAL026\n",
"eNpXgD/hwu9P/fgR7iGEFqNSIo3hXgBYsPra+zY3E1qwj9JH67UVVO9lIca3vP/9qfV4d57PM+tL\n",
"czPbC0YfjUXuWf9jwGMu3K9+9smeN8I9hNAsVdDNQ9+B3fu+caTVYnvfXqi+vk7aunBK/c49b4+7\n",
"mHRf3jq+t1lJLe7e9SwNvM27c73IU0Pn4YXVZmPIuEtZ/orTmP+Bx1z4g4bUE+EeQqg3lRpC2rx8\n",
"BSYO7PmB/zJpUE9yc+G3B3juniNGet4+VR27r+B/vj+t/ep9LTqZ489NWt9l4mP/8u55eGH1Q4Al\n",
"gR8B59m8N5D3VosI9xBCXanUbMAlwKzA5Uwc4C+58P8y1TUtaf+IvoJ5VuAf9Ap80ieDZXs8bwnS\n",
"Jud9/YJ4xoU/ApAQaePz75MugP4QuMBmXBPeKqmGCPcQQp2o1KKkfRnuBPZ34aaF2WCo1JxM2qWy\n",
"EPA0k3btTPYTRRXqmwBHAfMCxwCX2DSk62VKItxDCHWhUqsDV5F2VPtJK13UbLQq1LckhfospFC/\n",
"zObDfDX1LztjElMIYbJU6iuk2ZR7uvDVuetpFolpgGGk7pchwNHAlTYfZS2sBhHuIYRJVKNfDiNd\n",
"PN3UhcdkLqkpJIYA25ImH40DSuCadgr18SLcQwgTqabB/xJYEVjLhV/IXFLDVaG+AynU3wQOB66z\n",
"adsuqAj3EMLHquGGVwL/BjZw4f9mLqmhJKYDdiKt/fIK8C1gVDuH+nix5G8IAQCVWoY0GuZOYNtO\n",
"DnaJ6SX2JI2UGQ58A1jP5sZOCHaIlnsIAVCpDYFLgSNc+Ozc9TRKNfplb9LSu48Bu9ncmreqxohw\n",
"D6HLqdQewLHAV1345tz1NNjRwBbAV23uyF1MI0W4h9ClVGoa4Dhga2A9F/5H5pIaSmIfYHtgXZtX\n",
"c9fTaBHuIXQhlZoFuJC0bspag12psNVJfJk0Zn29bgh2iAuqIXQdlVoI+AvwBrBxFwT7OsCvgC/Z\n",
"PJG7nmaJcA+hi6jUZ0ijYX4L7N6fzSHamcSypKGdX7P5e+56mim6ZULoEio1DDgL+KYLX5G7nkaT\n",
"WAD4I3CYzfW562m2CPcQOly1lMDBwEHAFi78t8wlNZzE7MB1wFk252UuJ4tYFTKEDqZS0wGnA2sA\n",
"W7nws5lLajiJ6YFrgSeAfTplUtJ4sSpkCF1OpeYi9a2/Sxrq+FbmkhqumqR0NvAOsH+nBXst4oJq\n",
"CB1IpZYA7gAeAL7cDcFeOZa07d2OOTbSaCXRcg+hw6jUeqQW+0gXPjN3Pc0isT+wDWmS0ju568kt\n",
"wj2EDqJSuwInAru48I2562kWiW1Iy/R+zua13PW0ggj3EDpAtZTAD0jL1w514Yczl9Q0Ep8DzgQ2\n",
"s3kqdz2tIsI9hDanUjMBvyZt/LymC3fF9HoAieWBK4BdbO7JXU8riQuqIbQxlfoEMJq0JdznuyzY\n",
"FySNZf+2Tdd0QfVXhHsIbUqlPk1aSuBaUh/7/zKX1DQSc5Bmn55pc0HuelrRoCcxSToH2BJ4xfZK\n",
"1X1zA78BPgk8DWxv+41er4tJTCEMkEptAZwHHOjCl2Yup6mqSUp/BB4BDui2sez9zc56tNzPBTbr\n",
"dd9hwCjbSwM3VT+HEOpApQ4grREzrAuDfRpS5vwHGNFtwV6Luiw/IGkx4JoeLfdHgQ1svyylPkHb\n",
"y/Z6TbTcQ6iBSk0LnAIMBb7owk9nLSgDiR8D6wAb27ybu54cci8/ML/tl6vvXwbmb9B5QugKKjUH\n",
"aY9TAeu68H8yl9R0EiOArUiTlLoy2GvR8KGQti2pz48Hkkb2+HG07dGNrieEdqNSiwF/AG4BRrhw\n",
"102rl9gO+A4p2P+du55mkjSU9Gmtttc1sFtmqO2XJC0A3BzdMiHUTqXWJo3jPg74mYtMy7hmJLEB\n",
"aTmFjW3uy11Pbrm7Za4GdgOOr75e1aDzhNCxVOqrwE+Br7vwtbnryUFiBeAy0kJgXR/stajHUMhL\n",
"gA2AeUn960cBvyf9hSxKDIUMoSbV5hrfB/YgrcF+f+aSPlZtW3cm8D4wFni8+joWeNLmvTqea2Hg\n",
"duAIm4vqddx219/sjM06QmghKjUjaZjjUqShji9lLuljElsB5wAjgadIS+suSap1SVJj7kUmDf3H\n",
"ScHf70lWEnMCtwLn25xQv3fR/iLcQ2gzKjUfqQvzeWA3F26JESHV2PIjgG8C29ncMZnnTUeauLgk\n",
"kwb/J4FXmDT0xwJP9Bz9IjEDcD1wP/CtGMs+sQj3ENqISi1PGhFzMXCUC3+UuSQAJGZlwqJk29i8\n",
"MMDjTEtq2fcV/IsDrzIh9BcF3gS+avPhYN9Dp4lwD6FNqNTGwEXAt134/Nz1jCexBOmTxN+AfWvp\n",
"VqnxPEOARZgQ+nMCpzTqfO0uwj2ENqBS3yT1YW/nwrdmLudjEhsDFwJHA6dH10jryD0UMoQwBSo1\n",
"hLRj0ubA51x4bOaSgI83mD4Y+Dawg83ovBWFgYpwD6HJVGo2Ut/6zMDaLvx65pIAkJgJ+BWwPLCW\n",
"zT8zlxQGIdZzD6GJqu3wriHNCdmshYJ9UeA2UiZ8LoK9/UW4h9Bcw4EZgb1deFzmWgCQWB+4i7Qw\n",
"2c4272QuKdRBXFANoUlUah7gIWBzFx6TvZ7Uv74PUAC72tyQuaTQD3FBNYTWczxwaYsE+wzAacDa\n",
"pJUWW+KCbqifCPcQmkCl1iXtWLZ89lrEAqSVJl8C1rZ5K3NJoQGizz2EBlOp6UiLbR3swm9mrUWs\n",
"SZqUdD3wlQj2zhUt9xAabwTwAmlN8mwkvg78GNjT5vc5awmNFxdUQ2gglVoEGAOslWuiUrWg10mk\n",
"bqEv2zyco45QH3FBNYTWcCppB6VcwT4faW+Fd4E1bN6YyktCh4g+9xAaRKW2BFYijZJp/vnFqqT+\n",
"9TuBrSLYu0u03ENoAJWamTTUcG8XbvrqhhI7krbo28/msmafP+QX4R5CYxwJ3OXCo5p50mr53GOB\n",
"7YEvxL6j3SvCPYQ6U6nlgL2BlZt6XjEXcAkwHbC6zWvNPH9oLdHnHkIdVZtbnwEc7cID2rVoQOcV\n",
"KwB3A48Cm0awhwj3EOprZ2AOUsA3hcSXgdHAMTbfsvmgWecOrSu6ZUKoE5WaCzgBGObCDQ/YauPq\n",
"o4A9gC1t7m70OUP7iHAPoX5+CFzlwg0PWYnZgfOBeUn96y81+pyhvUS4h1AHKrUGsDVNWBhMYing\n",
"98CtwPY27zf6nKH9RJ97CIOkUtOSFgb7TqN3VpLYDLgdONXmGxHsYXKi5R7C4O0LvAFc1KgTVBtr\n",
"fJe0CNk2Nrc16lyhM0S4hzAIKrUg8H1gPReNWYVPYhbgbGAJYE2bZxtxntBZolsmhME5GfilCz/a\n",
"iINLLEbqhnkfWD+CPfRXhHsIA6RSGwNrkEbJ1P/4YkPSol/nAbvZvNuI84TOFN0yIQyASs1Imqh0\n",
"gAu/U9djp/71A4AjgJ1tbqrn8UN3iHAPYWAOBR5w4WvreVCJGYGfA58h7W/6VD2PH7pHhHsINVKp\n",
"JUkt61XrelyxEHAl8E9gHZv/1vP4obtEn3sINagWBjsdOM6F63ZxU2Id0sJfVwE7RLCHwYqWewi1\n",
"2Q5YkLR9Xl1I7EW6KPt1m7p284TuFeEeQj+p1OykoY87uPC4QR9PTA+cAmwIrGfz2GCPGcJ4DQ13\n",
"SU8DbwIfAuNsr9HI84XQYD8AbnDh2wd7IIn5gd+SZrauafPmYI8ZQk+NbrkbGGr73w0+TwgNpVKr\n",
"AjsCKwz6WGIN4HLS+PWRNh8N9pgh9NaMbhk14RwhNIxKTUManniECw94hyOJVYHvAesC+9pcWacS\n",
"Q5hEo0fLGPiTpL9L2qvB5wqhUfYidS2eO5AXS6wucTXwB9IyvZ+KYA+N1uiW+7q2X5Q0HzBK0qO2\n",
"b23wOUOoG5WaHzgG+LwL19R9IrE2aaekFYHjSGuv/6/+VYYwqYaGu+0Xq6+vSvodaR2Oj8Nd0sge\n",
"Tx9te3Qj6wlhAE4Afu3C9/f3BRLrk0J9SeBY4Ms27zWovtDhJA0Fhtb8OjdmlVIkzQwMsf2WpFmA\n",
"G4HS9o3V42Yk67rwXxtSQAiDpFJDSVvZLe/Cb0/xuWk9mA1Job4wKdQvsBn0kMkQepJk21O9ltnI\n",
"lvv8wO8kjT/PReODvYdvA9s0sIYQBkSlpictDDZiSsFehfompFCfl9SFc4lNwzfIDmFKGhbutp8C\n",
"VpnK09ZTqaVc+PFG1RHCAB0CPElaDmASVahvSQr1WYCjgd/afNi0CkOYgtwzVM8EDiJtUxZCS1Cp\n",
"xUnhvnrv3ZUkpgGGkXZfGkIK9StjrHpoNQ3rc5/qiVOf+yeAR4GlXfjVLIWE0EO1MNjVwB0ufOzH\n",
"94shwLakcerjSLNVr4lQD83W3z73rKtCuvDLpJl6++SsI4QehpFGuZwIKdQldgIeAA4GDgNWs/l9\n",
"BHtoZVlb7ralUssBo4HFXDi2EQvZqNSswEPAcEb6VmBn0m5Ir5Ba6n+yyfMfJoRKW7TcAVz4EdI6\n",
"1l/LXUvoekfx0ZDbGOklgMeA3YBvkDamHhXBHtpJ9nCvnAgcUq3hEULT6eBFV2XcTN/k5GfWB7YH\n",
"httsZDM6Qj20o1YJ07+Qlgb+Yu5CQneRmElDxh3IWwveyW2H/pO3F9zOZhObWCYjtLWWCPdquNmJ\n",
"pElNITScxCwSBwNPsM5Jw5nvocfZcOQqNnfmri2EemiJcK9cASyqUmvmLiR0LonZJL4LPAGszdJX\n",
"f5UvHL4gM7y9qwvHBKTQMVom3F34A+AnpMkjIdSVxBwSR5JCfVXgCzbbsdOwrwGXufA9eSsMob5a\n",
"Jtwr5wAbqdSnchcSOoPEXBIjSaG+DGnky442D6rUusAWpNmmIXSUlgp3F34L+BXwrdy1hPYmMa/E\n",
"D4HHgUWAtWx2tXkUQKWmI+2udIgL/ydjqSE0REuFe+VnwC4qNXfuQsLASEwvsVC1Dkuzzz2/xI9J\n",
"49TnIc0m3cNmbK+nHgi8DPym2TWG0AzZZ6j2+Vipc4HHe67tEdqDxIzADcCngRlI3SGPA2Or2/jv\n",
"n6/n9H2JBYDvAMOBi4HjbZ7t87mlFgHGAOu48D/qVUMIzdDfGaqtGu4rAqOAxV04tiVrE1VL/VLS\n",
"3rk7kpbCXYK0VstS1dfx389JWlK3d+iPBZ7r79K5EgsDh5KWCvg1cILNC1N8TakrgAdduKjxLYaQ\n",
"XSts1jFgLvygSo0h/Yc9O3c9Yeqq9c1PBuYDNqta5W8B91a33s8fH/zjQ3810i+EJYF5JJ6i7+B/\n",
"xuZDiU+SFvHaATgLWM7m5anWWWoLYGXSv60QOlZLttwBVGoj4DRgxVo3Jg7NJ3EIqUtkPZs3Bnms\n",
"mZnQ4u/d6v9/wDOk/vRfAD+x6ddy0So1M/AgsI8L3zCYGkPIpa1b7pWbgf8BmwPXZq4lTIHEjsAI\n",
"YJ3BBjuAzTukJXYf6ONcMwGfAl6web3GQx8B/D2CPXSDVhwtA8SSBO1CYiPgFGALm+cafT6bd20e\n",
"qjXYVWpZ0gqPBzWmshBaS8uGe+W3wBIqtVruQsKkJFYmXUDd3ubB3PVMTrW70hnAMS78fO56QmiG\n",
"lg53Fx5HahXGkgQtprqg+Qdgf5tbctczFTsBcwGn5y4khGZp6XCvnAVsolKLZa4jVCTmBv4InGhz\n",
"We56pkSl5gROAL5ZrV8UQldo+XB34TdJwyFH5K4lfHxB82rgWptTc9fTDz8Ernbhu3IXEkIztXy4\n",
"V34K7KZSc+UupJtJDAEuBJ4lTRxqaSq1OrANcHjuWkJotrYIdxd+jtS/u3fuWrpVNUnpFFLf9fB6\n",
"Lh3QCCo1BDgTONSFax0yGULba4twr5wEHKhS0+cupEt9B9gA2NrmvdzF9MO+pBmyF+QuJIQc2ibc\n",
"Xfg+4CHSFPXQRBK7APsBm9u0/PK4KrUAcBSwbzVfIoSu0zbhXjkR+HY1bjk0gcTGpE9Nm9u0yxjx\n",
"k4GzXPjh3IWEkEu7hfso0oqDm+QupBtIrApcBHzFpi2CUqU2BtYGjs5dSwg5tVW4x5IEzSOxGHAN\n",
"sI/NrZnL6ReVmpE0UekAF34ndz0h5NRW4V65FFhOpVbJXUinkpgHuB44zuaK3PXU4LvAwy58Te5C\n",
"Qsit7cLdhd8njXuPJQkaoJqkdA3we5vTctfTXyq1JGnrvANz1xJCK2i7cK/8Etiy2i4t1Ek1Seli\n",
"0g5JbTPxp7rAfhrwYxd+Jnc9IbSCtgx3F34DOI9YkqBuqklKPwVmA3Zv9UlKvXwFWBj4Se5CQmgV\n",
"bRnulVOAr6vUHLkL6RCHAesC29i8n7uY/lKp2Umhvk+1imgIgTYO9+rj9/XAXrlraXcSu5GWdtjC\n",
"5s3c9dSoBEa5cFuM6AmhWRoW7pI2k/SopMclNWqRqZOAEbEkwcBJbAocT5qk9ELuempRjZjamTRK\n",
"JoTQQ0PCXdIQ0gWuzYDlgR0lLVfv87jwPcA/gO3rfexuIPFZ0tor29o8mrueWqjUNMDPgSNduF8b\n",
"ZIfQTRrVcl8DGGv7advjSGPThzXoXLEkwQBIfIq0LvveNrfnrmcA9iTNVj47dyEhtKJGhftCpDW/\n",
"x3uuuq8RrgemAz7foON3FAlJLEvaSemHNlflrqlWKvX/gGNIuyu106ieEJqmUeHetJX4YkmCqZOY\n",
"TWKYxM+BJ4A/AWfZnJG5tIE6GbjAhe/PXUgIrWraBh33eaDnBKNFSK33iUga2ePH0bZHD/B8FwM/\n",
"VKlPx3/4j8esr0y65rEZ8FngTtKnnNOBh+zm/QKuJ5X6ImlhsE/nriWEZpA0FBha8+vcgOWuJU0L\n",
"PEbqKnkBuBvY0fYjPZ5j23XrJ1epw4FlXHh4vY7ZTiTmBTYmhfmmwH+AG0iBfovNfzOWVxfVnIYH\n",
"gd1c+M+56wkhh/5mZ0Na7rY/kLQ/KVyGAGf3DPYG+QUwVqUWcuF2WXd8wCSmBdZkQpgvA9xM+jMv\n",
"bZ7MWF6jnABcF8EewtQ1pOXerxPXueUOoFKnAu+68GH1PG6rkFiEFOSbARsB/yS1zG8A/tpOM0tr\n",
"pVJfAM4BVnLhlt8NKoRG6W92dlq4Lw78DVjchd+q57FzkJgRWI8JfefzAzeSAv1Gm5cyltc0KjUr\n",
"8ACwnwtfl7ueEHLqb3a27fIDfXHhp4CbgD1y1zIQ1TDFZSQOlLgOeAUYCbwBDAfmt9nJ5vxuCfbK\n",
"scBfIthD6L+OarkDqNTqwOXAEi78Qb2PX28Ss5O6WMZ3t0zLhAuhN9m8nrG87FRqPeA3wIou/O/c\n",
"9YSQW9YLqjm58N9U6mnSMrCXZi5nEhLTMPEwxc8Ad5AC/YvAw+06TLHeVGom0gzU/SLYQ6hNx4V7\n",
"5UTgeJW6zYUnGV/fbBLzMWGY4iakYYrXkxbs6ohhig0yErjXhX+Xu5AQ2k2nhvsfSJNc/k+l9nPh\n",
"y5t58mqY4lpM6GpZmjRM8XqgsHmqmfW0o6p7bTgxWSmEAem4PveJzlFqDeAi4FZgRCNH0EgsSgrz\n",
"TUmTt54mhfn1wB2dPEyx3lRqBuD/gGNd+OLc9YTQSrpytExvLnw3sCrwETBGpdaq17ElZpTYROJk\n",
"iYdJYbQhaaXF5WxWtTnc5pYI9podQdrH9ZLchYTQrjq65T7R+UptQ1r/+3RSi7CmkTTVei3LMKGr\n",
"ZV3gfiaMbLnH5sO6Ft2FVGpl0sJmq3TDTOMQatWVk5imes5SC5E21p4Z+JoLT3GKfjVM8fNMCPQh\n",
"TJgR2vXDFOtNpaYF7gJOd+FzctcTQiuKbpk+VC3BTUnj4O9SqV17bvIhMY3EZySOkLiFtLrlPqTd\n",
"nrYEFrXZy+byCPaG+DbwL+Dc3IWE0O66quU+0fnTx/+LeH/msfzq7mt5dYX1ScMU32DChdBbbN7J\n",
"VWM3UanlSBe+V3PhpzOXE0LL6tpJTFMjMR2wFnhTpnv7PTb57uYM33ATbjv0TO44ZK0Ypth8KjWE\n",
"NFmpiGAPoT66oltG4pMSe0tcAbwKnApMw7hZD+baM2Zjlle3YdNv78BIfUOlps9cbjc6APiAdME7\n",
"hFAHHdktIzGEdCF0c1If+3xMWE1xVF+LbqnUfKTW40LAzi78aCNqCxNTqSVIF1HXduHHc9cTQqvr\n",
"ytEy1czQHYDvAe8Av2PCMMWpbqRcXVzdm7T58veBX1R7tIYGUKlpSKt4/sGFT8pdTwjtoKvCvepH\n",
"3xk4EngR+AFpqOKA3pxKLUua2fo8sKcLv1KPOsPEVOobwO7AOi4ccwRC6IeuGAopMb3EXqT9Wr8G\n",
"7Gmzvs2fBrOyYtUlszbwEHCvSm1en4rDeCq1KOkT0u4R7CHUX1u23KsdinYHDgUeBY62ua2e9X18\n",
"rlJDgV+TlhX4rgu/24jzdJOq++s64HYXPiZ3PSG0k45suUvMJDECGAtsAWxvs2mjgh3AhUcDq5Au\n",
"yv69Gh8fBmdX4BOkJY9DCA3QFi13iVmAb5JmMN4JHGPzf42sb5IaUmtzF+Bk4EfAKS481Yu0YWIq\n",
"tQBwH7CMiNxdAAAJ10lEQVSpC4/JXU8I7aYjLqhKzAbsBxwE3EIK9fubUd9ka0qbcF9IGo0zPBa3\n",
"6r/qF+TvgAdd+Hu56wmhHbV1t4zEnBLfB54AVgI2tNk+d7DDx5twbwD8BbinWm0y9M/2pI1Ljs5d\n",
"SAidrqVa7hJzAyNIrfVrgWNtHstRX39U68NfSPpUMcKF385cUsuqJondD3zZhe/KXU8I7aqtWu4S\n",
"80ocCzxOmiG6ps1urRzsAC58J2kzEEibgayRs54WdypwcQR7CM2RteUO/gTpIukewG+A422ezlLQ\n",
"IKnUtsAZwM+AH8XY7QlUahhwEvBpF45VNkMYhLa4oAp+ndSt8WOb57IUUkcqtTBpM5AZSZuBdP0K\n",
"kyo1F/AgsJML35K7nhDaXbuE+4I2L2YpoEGq9VIOAg6rvl7UzevTqNQ5wLsuvF/uWkLoBG0R7jk3\n",
"62g0lVoFuJg0pnsfF34jc0lNp1KbAr8AVnLht3LXE0InaKsLqp3Ihe8FPgu8BtynUhtkLqmpVGo2\n",
"UrDvHcEeQvNFy70JVGoL4CzSGjWFC7+fuaSGU6nTgZlcePfctYTQSaLl3kJc+DrS+jQrAn9VqWUy\n",
"l9RQKvUFYBhwSO5aQuhWEe5NUq0J/yXSbk+3qdTe1XT8jqJS65OuNezqwq/nrieEbhXdMhmo1HKk\n",
"zUCeAfZy4Vczl1QX1fLIlwE7uvBNmcsJoSNFt0wLc+FHgLVIm4zcq1KbZS5p0FRqI1Kw7xDBHkJ+\n",
"0XLPTKU2JF1o/R1wqAv/L3NJNav62C8GtouJSiE0VtaWu6SRkp6TNKa6tX3LtFFc+GZgZWAB4G8q\n",
"tVLmkmqiUpuQgn3bCPYQWkdDWu6SCuAt2ydP4TnRcu+huri6K3AicCxwaqtvBlJ1J50PbO3Ct+eu\n",
"J4Ru0Ap97hHcNXBhu/CvgTWB7YDrVWrBzGVNlkptSQr2YRHsIbSeRob7AZLuk3S2pDkbeJ6O4sJP\n",
"AusDt5M2A9k6c0mTUKmtgHOBrVz4jtz1hBAmNeBuGUmjSJsc93YkaZ/T8cP7jgYWsL1Hr9cbKHvc\n",
"Ndr26AEV06FUam3Sqpl/Bg5qhc1AquV7fwl80YX/lrueEDqdpKHA0B53FS2xcJikxYBrbK/U6/7o\n",
"c++Hao2WnwLrAjvnDNRqS8EzgC1c+J5cdYTQzbKuCilpAdsvVt8fBKxue6eBFBgSldoOOI0U9Mc1\n",
"ezMQlfpKdf7NXXhMM88dQpggd7ifT1pLxcBTwDdsvzyQAsME1WYg5wPTkjYD+WeTzrsDcAqwmQvf\n",
"14xzhhD6Fuu5d6hqM5BDgO+Q+uEvavD5diJtkbeJCz/QyHOFEKYuwr3DqdSqpPVpxgD7NWIzEJXa\n",
"BfgxsLELP1Tv44cQatcK49xDA1X93qsBb5DWp1mvnsdXqd2A44EvRLCH0H6i5d4BVOqLwK+Ac4CR\n",
"LjxukMfbHfgBKdgfrUOJIYQ6iZZ7F3HhP5AuYK8C3K5SSw/0WCq1J2n+wUYR7CG0r2i5d5BqfZp9\n",
"SOF8BHCWi/7/BavUN0iT0DZy4bGNqTKEMBhxQbWLqdTypIutT5M2A3mtH6/ZF/gu8HkXfqKxFYYQ\n",
"Biq6ZbqYCz9M2gxkLOli6yZTer5K7U8aWrlhBHsInSFa7h2u2iHp18DlwOG9NwNRqRHACFJXzNPN\n",
"rzCEUItouQcAXPjPpM1AFgbu7rkZiEodDBwIDI1gD6GzRMu9S1QXW3cDTgCOAWYA9iK12J/NWVsI\n",
"of/igmrok0otQVpGeB5SH/vzmUsKIdQgwj1MlkoNAaZ34Xdz1xJCqE2EewghdKC4oBpCCF0swj2E\n",
"EDpQhHsIIXSgCPcQQuhAEe4hhNCBItxDCKEDRbiHEEIHinAPIYQOFOEeQggdKMI9hBA6UIR7CCF0\n",
"oAj3EELoQBHuIYTQgSLcQwihA0W4hxBCB4pwDyGEDhThHkIIHSjCPYQQOlCEewghdKAI9xBC6EAR\n",
"7iGE0IEi3EMIoQMNONwlbSfpIUkfSvpMr8cOl/S4pEclbTL4MkMIIdRiMC33B4Ctgb/0vFPS8sAO\n",
"wPLAZsAZkjruE4KkoblrGIyoP6+oP592rr0WAw5d24/a/kcfDw0DLrE9zvbTwFhgjYGep4UNzV3A\n",
"IA3NXcAgDc1dwCANzV3AIA3NXcAgDM1dQDM0okW9IPBcj5+fAxZqwHlCCCFMxrRTelDSKOATfTx0\n",
"hO1rajiPa6oqhBDCoMgeXO5Kuhk4xPY91c+HAdg+rvr5eqCwfVev10XghxDCANjW1J4zxZZ7DXqe\n",
"6GrgYkknk7pjlgLuHkhxIYQQBmYwQyG3lvQssBZwraQ/Ath+GLgMeBj4I7CvB/vxIIQQQk0G3S0T\n",
"Qgih9WQdfy7paEn3SbpX0k2SFslZT60knSDpkeo9XClpjtw11WJKE9FalaTNqslxj0s6NHc9tZJ0\n",
"jqSXJT2Qu5ZaSVpE0s3Vv5kHJR2Yu6ZaSJpR0l1V3jws6Ue5axoISUMkjZE0xUEtuScX/dj2yrZX\n",
"Aa4Cisz11OpGYAXbKwP/AA7PXE+t+pyI1qokDQFOI02OWx7YUdJyeauq2bmk+tvROOAg2yuQumP3\n",
"a6c/f9v/Azas8ubTwIaSPpe5rIEYQer2nmK3S9Zwt/1Wjx9nBV7LVctA2B5l+6Pqx7uAhXPWU6sp\n",
"TERrVWsAY20/bXsccClp0lzbsH0r8HruOgbC9ku2762+fxt4hDSvpW3Yfqf6dnpgCPDvjOXUTNLC\n",
"wBbAWUw8kGUSuVvuSPqhpGeA3YDjctczCLsD1+UuosMtBDzb4+eYIJeJpMWAVUmNmrYhaRpJ9wIv\n",
"AzdXA0DayU+A7wAfTe2JDQ93SaMkPdDHbSsA20faXhQ4j1R4S5la/dVzjgTet31xxlL71J/620hc\n",
"/W8BkmYFLgdGVC34tmH7o6pbZmFg/XZaZ0bSF4FXbI9hKq12qN8498myvXE/n3oxLdjynVr9koaT\n",
"PiZ9vikF1aiGP/928DzQ86L7Iky81EVoMEnTAVcAF9q+Knc9A2X7P5KuBVYDRmcup7/WAb4kaQtg\n",
"RmB2Sefb3rWvJ+ceLbNUjx+HAWNy1TIQkjYjfUQaVl2saWftMKns78BSkhaTND1p9dGrM9fUNSQJ\n",
"OBt42PYpueuplaR5Jc1ZfT8TsDFtlDm2j7C9iO3Fga8Cf55csEP+PvcfVV0E95JWajskcz21+hnp\n",
"QvCoamjSGbkLqsXkJqK1KtsfAPsDN5BGC/zG9iN5q6qNpEuAvwJLS3pW0tdz11SDdYFdSKNMxlS3\n",
"dhr5swDw5ypv7gKusX1T5poGY4rdlDGJKYQQOlDulnsIIYQGiHAPIYQOFOEeQggdKMI9hBA6UIR7\n",
"CCF0oAj3EELoQBHuIYTQgSLcQwihA/1//iAy03wTGHoAAAAASUVORK5CYII=\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x10891a750>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"##This function provides an interactive mode to change polynomial degree. \n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" X = construct_poly(x,n) #n=d\n",
" w = wRidge(X,y,1.0)\n",
" plt.plot(x,X.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" plt.plot(x,y)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_test.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
]
},
{
"cell_type": "code",
"execution_count": 308,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"array([[ 1.01010000e+04, 1.01010000e+03, 1.01010000e+02],\n",
" [ 1.01010000e+03, 1.01010000e+02, 1.01010000e+01],\n",
" [ 1.01010000e+02, 1.01010000e+01, 1.01010000e+00]])"
]
},
"execution_count": 308,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"A.T.dot(A)"
]
},
{
"cell_type": "code",
"execution_count": 309,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import scipy\n",
"from scipy import linalg\n",
"import numpy as np"
]
},
{
"cell_type": "code",
"execution_count": 310,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"1\n",
"[[ 9.80101960e-03 9.80101960e-04 9.80101960e-05]\n",
" [ 9.80101960e-04 9.80101960e-05 9.80101960e-06]\n",
" [ 9.80101960e-05 9.80101960e-06 9.80101960e-07]]\n",
"True\n"
]
}
],
"source": [
"A = np.array([[10,1,0.1],[10,1,0.2],[10,1.1,0.7]])\n",
"a = np.array([[10,1,0.1]])\n",
"A = a.T.dot(a)\n",
"\n",
"B,rank = linalg.pinv(A,return_rank=True)\n",
"print rank\n",
"print B\n",
"print np.allclose(A,A.dot(B.dot(A)))"
]
},
{
"cell_type": "code",
"execution_count": 327,
"metadata": {
"collapsed": false
},
"outputs": [
{
"ename": "ValueError",
"evalue": "shapes (8,10) and (6,) not aligned: 10 (dim 1) != 6 (dim 0)",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-327-96e4905b3c2b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0myt\u001b[0m\u001b[0;34m=\u001b[0m \u001b[0ma\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconstruct_poly\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m8\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0mw\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mwRidge\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0myt\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1.0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 7\u001b[0m \u001b[0;34m@\u001b[0m\u001b[0minteract\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mn\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m16\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 8\u001b[0m \u001b[0;32mdef\u001b[0m \u001b[0mplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mn\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m<ipython-input-295-aa77e456f942>\u001b[0m in \u001b[0;36mwRidge\u001b[0;34m(X, y, lamb)\u001b[0m\n\u001b[1;32m 4\u001b[0m \u001b[0;31m#print y\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 5\u001b[0m \u001b[0;31m#print np.dot(np.dot(np.linalg.inv((np.dot(X.T,X)+lamb*np.identity(X.shape[-1]))),X.T),y)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m \u001b[0;32mreturn\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlinalg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0minv\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mT\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m+\u001b[0m\u001b[0mlamb\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0midentity\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mT\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0my\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mValueError\u001b[0m: shapes (8,10) and (6,) not aligned: 10 (dim 1) != 6 (dim 0)"
]
}
],
"source": [
"## Read test data here\n",
"a = np.genfromtxt('ex1_test.csv', delimiter=' ')\n",
"xt= a[:,0]\n",
"yt= a[:,1]\n",
"X = construct_poly(x,8)\n",
"w = wRidge(X,y,1.0)\n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" #X = construct_poly(x,n) #n=d\n",
" #w = wRidge(X,yt,1.0)\n",
" plt.plot(xt,X.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),yt))\n",
" plt.plot(xt,yt)\n",
"# was ist mit fehler kurven gemeint?"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Hand in printed copy of completed notebook."
]
}
],
"metadata": {
"annotations": {
"author": "",
"categories": [
"intelligent-systems-1-2015"
],
"date": "2015-04-30",
"location": "Beginning of next lecture",
"parent": "IS_SS2015",
"submission_date": "2015-05-07",
"subtitle": "Exercise Sheet 3, Linear Regression",
"tags": [
"IntelligenSystems",
"Course"
],
"title": "Intelligent Systems 1 - Summer Semester 2015"
},
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
MaxXximus92