{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"preamble": true
},
"source": [
"(Defining latex commands: not to be shown...)\n",
"$$\n",
"\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
"\\DeclareMathOperator{\\minimize}{minimize}\n",
"\\newcommand{\\real}{\\mathbb{R}}\n",
"\\newcommand{\\normal}{\\mathcal{N}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gaussian Algebra (25 Points)\n",
"\n",
"Prove that the product of two univariate (scalar) Gaussian distributions is a Gaussian again, i.e. show, by explicitly performing the required arithmetic transformations, that\n",
"\n",
"\\begin{equation}\n",
" \\normal(x;\\mu,\\sigma^2)\\normal(x;m,s^2) = \\normal[x; (\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}(\\frac \\mu{\\sigma^2}+\\frac m{s^2}),(\\frac 1{\\sigma^2}+\\frac 1{s^2})^{-1}]\\normal[m,\\mu,\\sigma^2+s^2].\n",
"\\end{equation}"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Maximum Likelihood Estimator of Simple Linear Regression (25 Points)\n",
"\n",
"Derive the formula $\\mathbf{w}_{MLE} = (X^TX)^{-1}X^T\\mathbf{y}$ from the lecture, by calculating the derivative of $p(\\mathbf{y}\\,|X,\\mathbf{w}) = \\normal(\\mathbf{y}\\,|X\\mathbf{w}, \\sigma^2I)$ with respect to $\\mathbf{w}$, setting it to zero and solving it for $\\mathbf{w}$.\n",
"\n",
"\n",
"Note: _To refresh your linear algebra you might find it useful to have a look in [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/pdfs/Murray_cribsheet.pdf)._"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Linear regression (50 Points)\n",
"\n",
"In this exercise you will perform a regression analysis on a toy dataset. You will implement ridge regression and learn how to find a good model through a comparative performance analysis.\n",
"\n",
"1) Download the [training set](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv)! <br>\n",
"2) Implement $\\mathbf{w}_{RIDGE}$ as a function of a given $X, \\mathbf{y}$ array and a regularization parameter $\\lambda$!"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
":0: FutureWarning: IPython widgets are experimental and may change in the future.\n"
]
}
],
"source": [
"# Loading the required packages\n",
"%matplotlib inline\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from IPython.html.widgets import interact"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[1 2 3]\n",
"[1, 1, 1]\n",
"3\n"
]
}
],
"source": [
"testx = np.array([1, 2, 3]);\n",
"testy = [1, 1, 1];\n",
"\n",
"print(testx);\n",
"print(testy);\n",
"\n",
"foo = testx.T * testy;\n",
"\n",
"print(len(foo));"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [],
"source": [
"def wRidge(X,y,lamb):\n",
" # Change the following line and implement the ridge regression estimator wRidge\n",
" # w = np.zeros(X.shape[-1])\n",
" a = np.linalg.inv(lamb + np.dot(X.T, X));\n",
" # print('a');\n",
" # print(a);\n",
" b = X.T;\n",
" # print('b');\n",
" # print(b);\n",
" c = y;\n",
" # print('y');\n",
" # print(y);\n",
" ab = np.dot(a, b);\n",
" abc = np.dot(ab, c);\n",
" # print('abc');\n",
" # print(abc);\n",
" w = abc;\n",
" return w;"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"3) Load \"ex1_train.csv\" into a numpy array! The first column in the csv file is $X$ and the second column is $\\mathbf{y}$, assign them to each variable!"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[-3. -2.5 -1. -0.5 0. 1. 2. 2.5 3. 4. ]\n",
"[ 4.87103994 -1.03777715 -6.73176788 -4.08540431 0. 8.73176788\n",
" 13.27437941 13.53777715 13.12896006 13.94558004]\n",
"(-3.0, 4.0, -6.731767878463172, 13.945580037536574)\n"
]
}
],
"source": [
"# Read ex1_train.csv and assign the first column and \n",
"# second column to variables x and y respectively.\n",
"from numpy import genfromtxt\n",
"data = genfromtxt('ex1_train.csv', delimiter=' ');\n",
"X = data[:, 0];\n",
"y = data[:, 1];\n",
"\n",
"print(X);\n",
"print(y);\n",
"\n",
"print(min(X), max(X), min(y), max(y));"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"4) Plot the training data with appropriate labels on each axes!"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXAAAAD7CAYAAABzGc+QAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAADu5JREFUeJzt3X9sXeddx/HPZw2Ts8VQTZuystlKlXQC5rFuQoFQJO4E\n",
"21ILLfQPGJUQ5Ucs/mD2hCbYWCQaNKHxQxRiIyQUtmkI9gMBKwtLf4SuV1RCFIrSLiEJXcKC3dJl\n",
"QxvDFrbo1i9/nBPl2jm2r++v5zz3vl9S1Huee+85nzryJ8fPec61I0IAgPy8LHUAAEBnKHAAyBQF\n",
"DgCZosABIFMUOABkigIHgEzt6vcBbLNOEQA6EBHe7gV9/VMcor/H6CDT8dQZcshU11xkItMo5Gqn\n",
"O5lCAYBMUeAAkKlRLfBm6gAVmqkDbKKZOkCFZuoAFZqpA1Ropg5QoZk6wCaaqQN0wuVcS/8OYEds\n",
"NxEPAFinne7s+yoUAKgre2pampiTxsek5TVpaT7i/OnUudpFgQMYSUV5HzohnTxwY3Rmvz2lXEp8\n",
"VOfAAYy8ibn15S0V25OzafLsHAUOYESNj1WP79k92BydYwoFwKZynyPe2vJa9fjK6mBzdI4CB1Bp\n",
"GOaIt7Y0L83sX///d/SKtLiQLtPOsIwQQCX77oelh9558zPTD0ecvnvwiXqv+EdqcraYNllZlRYX\n",
"6vKPE8sIgRHUu2mP/OeIt1N+XWpR2J2gwIEh0ttpj97NEQ/3XHo6FDgwVDZbGjc9qx2fafZmjnj4\n",
"59LTocCBodK7aY+I86ftKRXl380ccS//UUErChwYKr1dGtebOeLhn0tPhRt5gKGyNC/NXF4/lnpp\n",
"XP7rreuKM3BgiPRu2qOX8l9vXVesAwfQd3Veb11X7XQnBQ4ANdROdzIHDgCZosABIFMUOABkigIH\n",
"gExR4ACQKQocADJFgQNApihwAMgUBQ4AmaLAASBTFDgAZIoCB4BMUeAAkCkKHAAyRYEDQKYocADI\n",
"FAUOAJmiwAEgUxQ4AGSKAgeATFHgAJApChwAMtVWgdv+qO1rts+1jL3K9hnbz9p+1Pat/YsJANio\n",
"3TPwj0k6vGHsA5LORMQbJD1WbgMABqStAo+IJyR9fcPwuyR9vHz8cUk/3sNcAIBtdDMHvjcirpWP\n",
"r0na24M8AIA27erFTiIibMdmz9s+3rLZjIhmL44LAMPCdkNSY0fvidi0dzfufJ+kUxHxpnL7kqRG\n",
"RHzZ9m2SHo+I76p4X0SEdxIKAEZdO93ZzRTKZyXdVz6+T9KDXewLALBDbZ2B2/6kpB+W9GoV892/\n",
"LulvJP2FpElJVyX9ZET8d8V7OQMHgB1qpzvbnkLpZwgAwHr9nkIBACREgQNApihwAMgUBQ4AmaLA\n",
"ASBTPbkTExhV9tS0NDEnjY9Jy2vS0nzE+dOpc2E0UOBAh4ryPnRCOnngxujMfntKlDgGgSkUoGMT\n",
"c+vLWyq2J2fT5MGoocCBjo2PVY/v2T3YHBhVFDjQseW16vGV1cHmwKiiwIGOLc1LM5fXjx29Ii0u\n",
"pMmDUcNnoQBdKC5kTs4W0yYrq9LiAhcw0Qt8mBUAZIoPswKAIUaBA0CmKHAAyBQFDgCZosABIFMU\n",
"OABkigIHgExR4ACQKQocADJFgQNApihwAMgUBQ4AmaLAASBTFDgAZIoCB4BMUeAAkCkKHAAyRYED\n",
"QKYocADIFAUOAJmiwAEgUxQ4AGSKAgeATFHgAJApChwAMkWBA0CmKHAAyNSubndg+6qk/5H0LUkv\n",
"RsTBbvcJANhe1wUuKSQ1IuJrPdgXAKBNvZpCcY/2AwBokyOiux3Y/y7pGyqmUP44Ik5ueD4iohYF\n",
"b09NSxNz0viYtLwmLc1HnD+dOhcAbNROd/ZiCuWuiHjB9msknbF9KSKe2BDkeMtmMyKaPTjujhTl\n",
"feiEdPLAjdGZ/faUKHEAqdluSGrs6D3dnoFvCHC/pJWI+L2WsVqcgdt3Pyw99M6bn5l+OOL03YNP\n",
"BACba6c7u5oDt/0K2+Pl41dKeoekc93ss3/Gx6rH9+webA4A6I1up1D2SvqM7ev7+vOIeLTrVH2x\n",
"vFY9vrI62BwA0BtdFXhEfEnSnT3K0mdL89LM/vVz4EevSIsL6TIBQOd6OgdeeYCazIFL1y9kTs4W\n",
"0yYrq9LiAhcwAdRRO905UgUOALno+0VMAEA6FDgAZIoCB4BMUeAAkCkKHAAyRYEDQKYocADIFAUO\n",
"AJmiwAEgUxQ4AGSKAgeATFHgAJApChwAMkWBA0CmKHAAyBQFDgCZosABIFMUOABkigIHgExR4ACQ\n",
"KQocADJFgQNApihwAMgUBQ4AmaLAASBTFDgAZIoCB4BMUeAAkCkKHAAyRYEDQKYocADIFAUOAJmi\n",
"wAEgUxQ4AGSKAgeATFHgAJApChwAMkWBA0Cmui5w24dtX7L9Rdvv70UoAMD2HBGdv9m+RdK/SfpR\n",
"Sc9L+mdJ90bExZbXRES426AAMEra6c5dXR7joKTLEXG1POCnJB2RdHGrNwGdsKempYk5aXxMWl6T\n",
"luYjzp9OnQtIpdsCf52kpZbt5yR9f5f7BG5SlPehE9LJAzdGZ/bbU6LEMaq6nQPvfP4F2JGJufXl\n",
"LRXbk7Np8gDpdXsG/rykiZbtCRVn4evYPt6y2YyIZpfHxcgZH6se37N7sDmA/rDdkNTYyXu6LfCn\n",
"JN1he5+k/5T0bkn3bnxRRBzv8jgYectr1eMrq4PNAfRHeWLbvL5t+/7t3tPVFEpEfFPSeyQ9IumC\n",
"pE+3rkABemdpXpq5vH7s6BVpcSFNHiC9rpYRtnUAlhGiR4oLmZOzxbTJyqq0uMAFTAyrdrqTAgeA\n",
"GmqnO7mVHgAyRYEDQKa6XYUysrgrEEBqFHgHuCsQQB0whdIR7goEkB4F3hHuCgSQHgXeEe4KBJAe\n",
"Bd4R7goEkB438nSIuwIB9BN3YgJAprgTEwCGGAUOAJmiwAEgUxQ4AGSKAgeATFHgAJApChwAMkWB\n",
"A0CmKHAAyBQFDgCZosABIFMUOABkigIHgExR4ACQKQocADJFgQNApihwAMgUBQ4AmaLAASBTFDgA\n",
"ZIoCB4BMUeAAkKldqQNguNlT09LEnDQ+Ji2vSUvzEedPp84FDAMKHH1TlPehE9LJAzdGZ/bbU6LE\n",
"ge4xhYI+mphbX95SsT05myYPMFwocPTR+Fj1+J7dg80BDCcKHH20vFY9vrI62BzAcKLA0UdL89LM\n",
"5fVjR69Iiwtp8gDDxRHR3wPYERHu60FQW8WFzMnZYtpkZVVaXOACJrC9drqTAgeAGmqnOzueQrF9\n",
"3PZzts+Wfw53ui8AwM51sw48JD0QEQ/0KgwAoH3dXsRkagQAEum2wGdtP2P7I7Zv7UkiAEBbtryI\n",
"afuMpNdWPHVM0j9K+mq5/SFJt0XEL1TsIyT9RstQMyKanQYGgGFkuyGp0TJ0/0BWodjeJ+lURLyp\n",
"4jlWoQDADvV7FcptLZv3SDrX6b4AADvXzSqU37Z9p4rVKF+S9Iu9iQQAaAc38gBADfV1CgUAkBYF\n",
"DgCZosABIFMUOABkigIHgExR4ACQKQocADJFgQNApihwAMhUN7fSY4gVv8tyYk4aHyt+u/zSPL/L\n",
"EqgXChw3Kcr70Anp5IEbozP77SlR4kB9MIWCChNz68tbKrYnZ9PkAVCFAkeF8bHq8T27B5sDwFYo\n",
"cFRYXqseX1kdbA4AW6HAUWFpXpq5vH7s6BVpcSFNHgBV+DxwVCouZE7OFtMmK6vS4gIXMIHBaac7\n",
"KXAAqCF+oQMADLGRLHDbjdQZNqpjJqmeucjUHjK1r665tjOSBS6pkTpAhUbqAJtopA5QoZE6QIVG\n",
"6gAVGqkDVGikDrCJRuoAnRjVAgeA7FHgAJCpgaxC6esBAGBIJV9GCADoD6ZQACBTFDgAZGpgBW77\n",
"fbZfsv2qQR1zK7Y/ZPsZ20/bfsz2RA0y/a7ti2Wuv7b9HTXI9BO2/9X2t2y/NXGWw7Yv2f6i7fen\n",
"zHKd7Y/avmb7XOos19mesP14+fd23vZcDTKN2X6y/H67YPvDqTNdZ/sW22dtn0qdRZJsX7X9hTLT\n",
"P2312oEUeFmOb5f0H4M4Xpt+JyLeHBF3SnpQ0v2pA0l6VNIbI+LNkp6V9GuJ80jSOUn3SPr7lCFs\n",
"3yLpDyUdlvQ9ku61/d0pM5U+piJTnbwo6Zcj4o2SfkDSL6X+WkXEmqS3ld9v3yvpbbZ/KGWmFu+V\n",
"dEFSXS4IhqRGRLwlIg5u9cJBnYE/IOlXB3SstkTEcsvmHkn/lSrLdRFxJiJeKjeflPT6lHkkKSIu\n",
"RcSzqXNIOijpckRcjYgXJX1K0pHEmRQRT0j6euocrSLiyxHxdPl4RdJFSd+ZNpUUEf9bPny5pFsk\n",
"fS1hHEmS7ddLmpb0J5Lq9JlNbWXpe4HbPiLpuYj4Qr+PtVO2f9P2oqT7JP1W6jwb/LwkPv3vhtdJ\n",
"WmrZfq4cwxZs75P0FhUnBEnZfpntpyVdk/R4RFxInUnS70v6FUkvbffCAQpJf2f7KdszW72wJ78T\n",
"0/YZSa+teOqYimmAd7S+vBfHbMcWuT4YEaci4pikY7Y/oOIv8udSZypfc0zS/0XEJ/qdp91MNVCX\n",
"H2+zYXuPpL+U9N7yTDyp8qfLO8trO4/YbkREM1Ue2z8m6SsRcbZmn4VyV0S8YPs1ks7YvlT+pHeT\n",
"nhR4RLy9atz2lKTbJT1jWyqmBP7F9sGI+Eovjt1Jrgqf0IDOdrfLZPtnVfxI9yODyCPt6OuU0vOS\n",
"Wi80T6g4C0cF298m6a8k/VlEPJg6T6uI+Ibtz0n6PknNhFF+UNK7bE9LGpP07bb/NCJ+JmEmRcQL\n",
"5X+/avszKqYPKwu8r1MoEXE+IvZGxO0RcbuKb7i3DqK8t2P7jpbNI5LOpspyne3DKn6cO1Je9Kmb\n",
"lHOET0m6w/Y+2y+X9G5Jn02Yp7ZcnC19RNKFiPiD1Hkkyfarbd9aPt6tYlFD0u+5iPhgREyU3fRT\n",
"kj6furxtv8L2ePn4lSpmLzZd4TTodeB1+jH4w7bPlXNyDUnvS5xHkhZUXFA9Uy4h+qPUgWzfY3tJ\n",
"xWqGz9l+KEWOiPimpPdIekTFioFPR8TFFFla2f6kpH+Q9AbbS7b7Pg3Xhrsk/bSKlR5nyz+pV8rc\n",
"Junz5ffbk5JORcRjiTNtVId+2ivpiZav099GxKObvZhb6QEgU9yJCQCZosABIFMUOABkigIHgExR\n",
"4ACQKQocADJFgQNApihwAMjU/wMrPuHX/p0aFAAAAABJRU5ErkJggg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x1050d4790>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot the input data here\n",
"def plotData(X, y):\n",
" plt.plot(X, y, 'bo')\n",
" plt.axis([min(X) - 1, max(X) + 1, min(y) - 1, max(y) + 1])\n",
" \n",
" \n",
"plotData(X, y) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"5) Implement a function which constructs features upto a input polynomial degree $d$!<br>\n",
"Note: _Constructing higher polynomial features is similar to what you implemented in Exercise 3 (SVM) of the previous exercise sheet._"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def construct_poly(x,d):\n",
" ## Implement a method which given an array of size N, \n",
" ## returns an array of dimension (N,d)\n",
" res = np.zeros((len(x), d));\n",
" for i in range(len(x)) :\n",
" xi = x[i];\n",
" for j in range(d) :\n",
" res[i][j] = np.power(xi, j);\n",
" \n",
" return res;\n",
"\n",
"# testX = [1, 2, 3, 4, 5, 6];\n",
"# testY = [1, 1, 1, 1, 1, 1];\n",
"# foo = construct_poly(testX, testY);\n",
"# print(foo[1,:]);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"6) Implement the Mean Squared Error Loss (MSE) as a function of the predicted and true values of the target variable! <br>"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def MSE(y_predict,y_true):\n",
" ## Implement mean squared error for a given input y and its predictions.\n",
" error = sum(np.power((y_predict - y_true), 2)) / float(len(y_predict));\n",
" return error;\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"7) By comparing the MSE find the degree $d$ for the polynomial that fits the training data best! You might find it useful to use the code below to interactively change the variable $d$, set $\\lambda = 1$ and keep it fixed. Plot the error as a function of different values of $d$!<br>"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"('Best d: ', 6)\n",
"('Best theta: ', array([ -1.48019799e+00, 6.47756826e+00, 1.20573199e+00,\n",
" -6.46681792e-01, 1.20936028e-03, 1.08880545e-02]))\n",
"('mse errors for each d: ', [88.938436293253972, 58.242200994141662, 24.151575218954584, 17.772988325257703, 2.6042621684559881, 1.9740426997964349, 1.6839081464600774, 1.762196096936659, 1.8951684291047843, 1.8996652873425053])\n"
]
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXIAAAEACAYAAACuzv3DAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAEilJREFUeJzt3X+QXedd3/H3JxbBNjZRRNq1SqTaOFWdVqGNB1KnQLPJ\n",
"OMTSBKFpZ0SYEnYysf5oi22YASwzU9A/lCQzDLHd4Q9MnG5LSCsMCLsjCy1GW2DaCYTIxbKtEfZU\n",
"lRKqNYE4scCeyY9v/7jH4/V6tbt3tfcePbvv18yOzvPcc+753pH0mec+5+x5UlVIktr1ur4LkCRd\n",
"GoNckhpnkEtS4wxySWqcQS5JjTPIJalxywZ5kruSPJHkZJK7ur4tSWaSnE5yLMnm0ZcqSVrMkkGe\n",
"ZCdwO/DdwD8B3p/kRuAAMFNVO4DHurYkqQfLjchvAj5TVS9V1deB/wH8K2APMN3tMw3sHV2JkqSl\n",
"LBfkJ4Hv66ZSrgZ2A28GJqpqrttnDpgYYY2SpCVsWurFqjqV5KPAMeBvgMeBry/Yp5L4e/6S1JMl\n",
"gxygqh4EHgRI8vPA54G5JNdV1fkkW4HnFjvWgJek1amqrHTfZYM8yd+tqueSbAf+JXALcAMwBXy0\n",
"+/PwWhRzKZJdR+HR9w1aB7sfgN1Hq47sGkcNfUpysKoO9l3HOPmZN4YN+pmHGgQvG+TAQ0m+Dfgq\n",
"8G+r6stJPgIcSvJh4Aywb+hK19y1Vy7ef81V461DksZrJVMr/2KRvr8Gbh1JRav2wkuL9194cbx1\n",
"SNJ4raPf7Dx3H+x/ZrA92fXd/iycvb+visZstu8CejDbdwE9mO27gB7M9l3A5S6jXFgiSY1rjnxw\n",
"vp27Yfsdg+mUCy/C2furTh4Z1/klaS0Mm53rKsglaT0YNjvX0dSKJG1MBrkkNc4gl6TGGeSS1DiD\n",
"XJIaZ5BLUuMMcklqnEEuSY0zyCWpcQa5JDXOIJekxhnkktQ4g1ySGrdskCe5J8mTSZ5I8utJvjnJ\n",
"liQzSU4nOZZk8ziKlSS91pJBnuR6YD9wc1W9DbgC+ABwAJipqh3AY11bktSD5UbkX2GwVufVSTYB\n",
"VwN/AewBprt9poG9I6tQkrSkJYO8W5vzF4GzDAL8+aqaASaqaq7bbQ6YGGmVkqSLWnLx5SQ3Aj8O\n",
"XA98GfiNJD8yf5+qqiQXXWYoycF5zdmqml1tsZK0HiWZ5JXFhoc/fqml3pL8EPDeqrq9a38QuAV4\n",
"D/DuqjqfZCtwvKpuWuR4l3qTpCGt9VJvp4BbklyVJMCtwFPAI8BUt88UcHg1xUqSLt2yiy8n+WkG\n",
"Yf0N4HPA7cC1wCFgO3AG2FdVzy9yrCNySRrSsNm5bJCPsxhJ0tpPrUiSLnMGuSQ1ziCXpMYZ5JLU\n",
"OINckhpnkEtS4wxySWqcQS5JjTPIJalxBrkkNc4gl6TGGeSS1DiDXJIaZ5BLUuMMcklqnEEuSY1b\n",
"NsiT/MMkJ+b9fDnJnUm2JJlJcjrJsSSbx1GwJOnVhlohKMnrgC8A7wDuAL5YVR9Lcjfwxqo6sGB/\n",
"VwiSpCGNeoWgW4FnquocsAeY7vqngb1DvpckaQ0MG+QfAD7dbU9U1Vy3PQdMrFlVkqQV27TSHZO8\n",
"HvgB4O6Fr1VVJVl0jibJwXnN2aqaHbJGSVrXkkwCk6s+fqVz5El+EPg3VXVb1z4FTFbV+SRbgeNV\n",
"ddOCY5wjl6QhjXKO/Id5ZVoF4GFgqtueAg4P8V6SpDWyohF5km8B/i9wQ1W90PVtAQ4B24EzwL6q\n",
"en7BcY7IJWlIw2bnULcfjroYSdLobz+UJF1mDHJJapxBLkmNM8glqXEGuSQ1ziCXpMYZ5JLUOINc\n",
"khpnkEtS4wxySWqcQS5JjTPIJalxBrkkNc4gl6TGGeSS1DiDXJIat6IgT7I5yUNJnk7yVJJ/lmRL\n",
"kpkkp5McS7J51MVKkl5rpSPye4EjVfVW4DuBU8ABYKaqdgCPdW1J0pgtu9RbkjcAJ6rqOxb0nwLe\n",
"VVVzSa4DZqvqpgX7uNSbJA1pFEu93QD8ZZJPJvlckge6xZgnqmqu22cOmFhFvZKkS7RphfvcDPxY\n",
"Vf1Jko+zYBqlqirJokP7JAfnNWeranaVtUrSupRkEphc9fErmFq5DvhfVXVD1/5e4B7gO4B3V9X5\n",
"JFuB406tSNKlW/Oplao6D5xLsqPruhV4EngEmOr6poDDQ9a6riQ7dye7jib7Zgd/7tzdd02SNoaV\n",
"TK0A3AF8KsnrgWeBDwFXAIeSfBg4A+wbSYUNGIT2O++FB97ySu/+G5OdVJ080l9lkjaCZadWLunN\n",
"N8jUSrLrKDz6vte+svto1ZFd469IUstGcdeKlnXtlYv3X3PVeOuQtBEZ5GvihZcW77/w4njrkLQR\n",
"GeRr4tx9sP+ZV/fd/iycvb+feiRtJM6Rr5HBBc/tdwymUy68CGfv90KnpNUYNjsNckm6zHixU5I2\n",
"GINckhpnkEtS4wxySWqcQS5JjTPIJalxBrkkNc4gl6TGGeSS1DiDXJIaZ5BLUuNWtEJQkjPAV4Cv\n",
"A1+tqnck2QL8N+Dv060QVFXPj6hOSdJFrHREXsBkVb29qt7R9R0AZqpqB/BY15YkjdkwUysLn8S1\n",
"B5jutqeBvWtSkSRpKMOMyH8vyWeT7O/6JqpqrtueAybWvDpJ0rJWNEcOfE9V/b8kfweYSXJq/otV\n",
"VUkWfbB5koPzmrNVNbuqSiVpnUoyCUyu+vhhF5ZI8nPABWA/g3nz80m2Aser6qYF+7qwhCQNac0X\n",
"lkhydZJru+1vAb4feAJ4GJjqdpsCDg9friTpUi07Ik9yA/DbXXMT8Kmq+oXu9sNDwHYucvuhI3JJ\n",
"Gp5rdkpS41yzU5I2GINckhpnkEtS4wxySWqcQS5JjTPIJalxBrkkNc4gl6TGGeSS1DiDXJIaZ5BL\n",
"UuMMcklqnEEuSY0zyCWpcQa5JDXOIJekxq0oyJNckeREkke69pYkM0lOJzmWZPNoy5QkXcxKR+R3\n",
"AU8BLy8ndACYqaodwGNdW5LUg5UsvvxmYDfwq8DLSw/tAaa77Wlg70iqkyQtayUj8l8Cfgr4xry+\n",
"iaqa67bngIm1LkyStDKblnoxyfuB56rqRJLJxfapqkpy0RWckxyc15ytqtlV1ClJ61aXr5OrPr7q\n",
"ohlMkv8AfBD4GnAl8K3AbwHfDUxW1fkkW4HjVXXTIscPtRK0JGn47FxyaqWqfqaqtlXVDcAHgN+v\n",
"qg8CDwNT3W5TwOHVFixJujTD3kf+8vD9I8B7k5wG3tO1JUk9WHJq5ZLf3KkVSRramk6tSJIufwa5\n",
"JDXOIJekxhnkktQ4g1ySGmeQS1LjDHJJapxBLkmNM8glqXEGuSQ1ziCXpMYZ5JLUOINckhpnkEtS\n",
"4wxySWqcQS5JjVsyyJNcmeQzSR5P8lSSX+j6tySZSXI6ybEkm8dTriRpoWVXCEpydVX9bZJNwB8B\n",
"PwnsAb5YVR9Lcjfwxqo6sMixrhAkSUNa8xWCqupvu83XA1cAX2IQ5NNd/zSwd8g6JUlrZNkgT/K6\n",
"JI8Dc8DxqnoSmKiquW6XOWBihDVKkpawabkdquobwD9N8gbgd5O8e8HrleSi8zNJDs5rzlbV7Cpr\n",
"laR1KckkMLnq45ebI19wsn8PvAjcDkxW1fkkWxmM1G9aZH/nyCVpSGs6R57kTS/fkZLkKuC9wAng\n",
"YWCq220KOLy6ciVJl2q5qZWtwHSS1zEI/f9SVY8lOQEcSvJh4Aywb7RlSpIuZqiplaHf3KkVSRra\n",
"mt9+KEm6vBnkktQ4g1ySGmeQS1LjDHJJapxBLkmNM8glqXHLPmtFl7dk527YdidceyW88BKcu6/q\n",
"5JG+65I0PgZ5wwYh/s574YG3vNK7/8ZkJ4a5tHE4tdK0bXe+OsRh0N5+Rz/1SOqDQd60a69cvP+a\n",
"q8Zbh6Q+GeRNe+GlxfsvvDjeOiT1ySBv2rn7YP8zr+67/Vk4e38/9Ujqg08/bNzgguf2OwbTKRde\n",
"hLP3e6FTatuw2WmQS9JlxsfYStIGs2yQJ9mW5HiSJ5OcTHJn178lyUyS00mOvbwknCRpvJadWkly\n",
"HXBdVT2e5BrgT4G9wIeAL1bVx5LcDbyxqg4sONapFUka0ppPrVTV+ap6vNu+ADwNfDuwB5judptm\n",
"EO6SpDEbao48yfXA24HPABNVNde9NAdMrGllkqQVWfGzVrppld8E7qqqF5JXRv1VVUkWnaNJcnBe\n",
"c7aqZldXqiStT0kmgclVH7+S2w+TfBPw34FHq+rjXd8pYLKqzifZChyvqpsWHOccuSQNac3nyDMY\n",
"en8CeOrlEO88DEx121PA4WEKlSStjZXctfK9wB8Afwa8vPM9wB8Dh4DtwBlgX1U9v+BYR+TrmM9C\n",
"l0Zj2Oxcdo68qv6Ii4/cb13pibS++Cx06fLhb3ZqlXwWunS5MMi1Sj4LXbpcGORaJZ+FLl0uDHKt\n",
"ks9Cly4XPsZWq+az0KXR8HnkktQ4n0cuSRuMQS5JjTPIJalxBrkkNc4gl6TGGeSS1DiDXJIaZ5BL\n",
"UuMMcklqnEEuSY1byVJvDyaZS/LEvL4tSWaSnE5yLMnm0ZYpSbqYlYzIPwnctqDvADBTVTuAx7q2\n",
"JKkHywZ5Vf0h8KUF3XuA6W57Gti7xnVJklZotXPkE1U1123PARNrVI8kaUiXfLGzBs/BHd2zcCVJ\n",
"S9q0yuPmklxXVeeTbAWeu9iOSQ7Oa85W1ewqzylJ61KSSWBy1cevZGGJJNcDj1TV27r2x4C/qqqP\n",
"JjkAbK6q11zwdGEJSRremq8QlOTTwLuANzGYD/9Z4HeAQ8B24Aywr6qev9RiJEku9SZJzXOpN0na\n",
"YAxySWqcQS5JjTPIJalxBrkkNc4gl6TGGeSS1DiDXJIaZ5BLUuMMcklqnEEuSY1b7WNspd4kO3fD\n",
"tjvh2ivhhZfg3H1VJ4/0XZfUF4NcTRmE+DvvhQfe8krv/huTnRjm2qh8+qGakuw6Co++77Wv7D5a\n",
"dWTXaM/d3zeBvs7tZ+7nG9+w2emIXI259srF+6+5apRn7fObQF/n9jOP77yXrKpG9kO3pOe4f4DJ\n",
"Ps7b589G+cxw21GoGvwcr1e2dz06vvPW2M7rZ944n/nVdVDD7L9e71qZ7LuAHkz2XcB4nLsP9j8z\n",
"2J7t+m5/Fs7eP9rz9vNN4LXnnh3juf3M4zvvpXFqRU2pOnkk2QnsvgP+/K3wx0/D2ftH/7X3hZcW\n",
"77/w4mjP2+e5/czjO++lWa8jcq1jVSePDC5sPvOfqo7sGs/c5fxvAi8bxzeBPs/tZx7feS/NyO9a\n",
"GdmbS9I6VpfL4suSpNFzakWSGmeQS1LjDHJJaty6C/IktyU5leTPk9zddz2jlmRbkuNJnkxyMsmd\n",
"fdc0DkmuSHIiySN91zIOSTYneSjJ00meSnJL3zWNWpJ7un/XTyT59STf3HdNay3Jg0nmkjwxr29L\n",
"kpkkp5McS7J5ufdZV0Ge5ArgPwK3Af8I+OEkb+23qpH7KvATVfWPgVuAf7cBPjPAXcBTwEa5Wn8v\n",
"cKSq3gp8J/B0z/WMVJLrgf3AzVX1NuAK4AN91jQin2SQV/MdAGaqagfwWNde0roKcuAdwDNVdaaq\n",
"vgr8V+AHe65ppKrqfFU93m1fYPAf/O/1W9VoJXkzsBv4VWDdP5QtyRuA76uqBwGq6mtV9eWeyxq1\n",
"rzAYpFydZBNwNfCFfktae1X1h8CXFnTvAaa77Wlg73Lvs96C/NuBc/Pan+/6NoRuFPN24DP9VjJy\n",
"vwT8FPCNvgsZkxuAv0zyySSfS/JAkqv7LmqUquqvgV8EzgJ/ATxfVb/Xb1VjM1FVc932HDCx3AHr\n",
"Lcg3ytfs10hyDfAQcFc3Ml+XkrwfeK6qTrABRuOdTcDNwC9X1c3A37CCr9stS3Ij8OPA9Qy+YV6T\n",
"5F/3WlQPavCLPsvm2noL8i8A2+a1tzEYla9rSb4J+E3g16rqcN/1jNg/B/Yk+T/Ap4H3JPnPPdc0\n",
"ap8HPl9Vf9K1H2IQ7OvZdwH/s6r+qqq+BvwWg7/7jWAuyXUASbYCzy13wHoL8s8C/yDJ9UleD/wQ\n",
"8HDPNY1UkgCfAJ6qqo/3Xc+oVdXPVNW2qrqBwcWv36+qH+27rlGqqvPAuSQ7uq5bgSd7LGkcTgG3\n",
"JLmq+zd+K4OL2xvBw8BUtz0FLDs4W1dPP6yqryX5MeB3GVzl/kRVreur+8D3AD8C/FmSE13fPVV1\n",
"tMeaxmmjTKfdAXyqG6A8C3yo53pGqqr+d/dN67MMroV8DviVfqtae0k+DbwLeFOSc8DPAh8BDiX5\n",
"MHAG2Lfs+/isFUlq23qbWpGkDccgl6TGGeSS1DiDXJIaZ5BLUuMMcklqnEEuSY37/1VfWTzms98x\n",
"AAAAAElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x10520d050>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"bestTheta = 1;\n",
"bestMSE = 9999999999;\n",
"bestD = 0;\n",
"alpha = 1.0;\n",
"\n",
"dMax = 10;\n",
"mseErrors = [];\n",
"\n",
"for d in range(dMax) :\n",
" newPoly = construct_poly(X, d);\n",
" newTheta = wRidge(newPoly, y, alpha);\n",
" newMSE = MSE(newPoly.dot(newTheta), y);\n",
" mseErrors.append(newMSE);\n",
" if newMSE < bestMSE :\n",
" bestMSE = newMSE;\n",
" bestTheta = newTheta;\n",
" bestD = d;\n",
" \n",
"print('Best d: ', bestD);\n",
"print('Best theta: ', bestTheta);\n",
"print('mse errors for each d: ', mseErrors);\n",
"plotData(range(dMax), mseErrors);"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXcAAAEKCAYAAADpfBXhAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XecHVX9xvHPk4SQQAgldEITBESqSm9RaSLSpEtvChqQ\n",
"3pTJIE1RVAQUBQHp7SeCgoBCAFEBJdSETui9Q4Ak5Pv740xks9lkb925e/d5v177yu69M2e+G8iz\n",
"Z8+cOUcRgZmZtZd+ZRdgZmaN53A3M2tDDnczszbkcDcza0MOdzOzNuRwNzNrQw53M7M25HC3HiVp\n",
"vKSPJQ3r9PoYSVMkLVZ8PVzS1ZJek/S2pAcl7V68t0Rx7HudPrabwTW3l/RPSR9IurWb+o7p1OYE\n",
"SZ9ImqfDMRtKulfS+5Kem3pdSctI+pOkVyW9IemvkpbpcN4Kkm4svqcpXVz7c5JuKb7fxyVt1en9\n",
"2SSd1eHv5Lbu/8atr3K4W08L4Clgp6kvSFoRGFy8N9WFwDPAYsA8wK7AK53amjMi5ujwceUMrvkG\n",
"cBpwSrfFRZzUsU3gx8CtEfFmUevywMXA0cBQYCXgv1PrAa4BlgEWAO4G/tSh+YnAZcDena8raUBx\n",
"7LXA3MB+wEWSPtvhsN8CcwHLFcd8v7vvx/ou+QlV60mSngbOAbaMiNWL134KvAmcACwREc9Keg9Y\n",
"JyIe6KKNJUg/IAZExHQ94Jlcex/gWxHx5QqPF/AkkEXEhcVrlwCPR0RWwfnzAK8DwyLirQ6vLw08\n",
"FhH9Ory2AvCv4gfK1NduBO6KiOMkLQfcBSwSEe9XUr/1be65Wxn+DQyVtJyk/sAOwEVdHHOWpB2m\n",
"DtV0Qc0sElgPmA+4usNra5By/wFJL0q6UNLcMzh/feCljsFepX7A54vPVyf9JnN8MSzzgKRtamzX\n",
"+gCHu5XlQmA3YCNgLPBCp/e3A+4Afgg8VYzJf6nTMa9LeqvDx7INrnF34MqImNDhtUWBXYBtgM+S\n",
"hpN+1flEScOBM4BDKrzWo8Crkg6XNIukjUk/HGYr3h8OrAC8DSwEfA+4oOjRm03H4W5lCFK4f4sU\n",
"oH+gUy88It6OiKMjYgXS+PV9pPHsjoZFxNwdPh5tVIGSZgO2BS7o9NYE4LyIeCIiPgBOAjbrdO58\n",
"wE3AmRFxeSXXi4hJwFbA14GXgIOBK4Dni0M+BCYBJ0TE5Ii4HbgV2LiGb8/6AIe7lSIiniWNm38N\n",
"+L9ujn0D+Bmw8EyGQCq6bBXHbg28ERGdZ6RMdw+go6K+m4BrIuLkqoqLeDAiRkTEvBHxNWAp0k3Z\n",
"jtftPBTlm2bWJYe7lWlv4CsR8WHnNyT9WNLnJQ2QNAewP+lGZsfx64rG3CX1kzQImAXoJ2lWSbN0\n",
"c9rU3yg6Ow/YU9KSRe/+KOC64jpDgRuBf0TEMTOoZRAwsPh8VkmzdnhvRUmDiimPh5F+Yzm/ePs2\n",
"4Fng6OLvZB1gRHE9s+k43K00EfFURNzb8aUOnw8G/gi8RZqxsiiwRacm3u40J31GUwN3Iw2nnEW6\n",
"SfohcPbUN4tz1+nw9SKk4Jwu3CPivOL1u4DxRVsHFm9vDXyJFP5Ta3q3GH+fOstnAvBQ8b1+CIzr\n",
"0PyuwIukKZ9fBjYqhmuIiMnAlqQhoLeL+neNiMdm8D1bH1f3VEhJvyeNE74aESsWr40C9gFeKw47\n",
"OiL+WteFzMysYo3ouZ8HbNrptQBOi4hViw8Hu5lZD6o73CPiDtKvzp01ew6ymZnNQDPH3EdKul/S\n",
"uZLmauJ1zMysk2aF+6+BJYFVSHN2f9ak65iZWRcGNKPRiHh16ueSzqGYKtaRJM/PNTOrQUR0O+zd\n",
"lHCXtFBEvFR8uTXwYFfHVVJgq5I0KiJGlV1HrVx/uVx/eXpz7VB5x7jucJd0KbABMK+k54AMGCFp\n",
"FdKsmaeBb9d7HTMzq1zd4R4RO3Xx8u/rbdfMzGrnJ1RrN7rsAuo0uuwC6jS67ALqNLrsAuo0uuwC\n",
"6jC67AJ6QmmbdUiK3jzmbmZWhkqz0z13M7M25HA3M2tDTZkKaWZmjaNcw4Bli4+KONzNzHqIhIBZ\n",
"gTmBodN8zPr23Cw5+jPM/+CSzP3UcIY+tyBDXxzGHC/MSb/Z+/PWZz7m9WU+mXZL35lcyzdUzcxm\n",
"rgjlwXQO5K5CeqbvTRnKkFemMN/DHzL/w5OY/6Fg3kf7M/eTg5j9tcF8OPd7vL/g67y/4Eu8O/xZ\n",
"3lzqaZ5f6zGeWe9ZYsC7wLugxyvJToe7mbUFiX6kXa4GknrHs87g80HAHFQe1FNfnwS8C7xT/NnV\n",
"R3pv9pc/ZPmr5mCJ2+ZhvnHzM+TFhZn1/cXpN+kziMmkDdE7fzwZWUzs/vusLDsd7mbWY5RLpLBc\n",
"CFi4+PPTj4+HLMVHcy/FewtP5M2lJ/Da8h/xygof88rKn/DOooJ+nYO6Y3gPACYCH3fxZ+fPpw/k\n",
"GQV1+ngvgmmCt/heFuHTsfBlgeWKPxck7RE8XYhHFm/U9XfocDeznlIE3TC6CuzpX5tEWi32048P\n",
"5n2Hu0auzQtrrMGs717I4rd9yHzjFmDocwsy2xsLM+u7i6IYwORBTzNp8FNMGvw4E4c+xgfzjWPs\n",
"to9wz3ffASZFNH7DcOUaAizDtCG+bPHa+3wa3I90+Hx8ZDG50bWAw93MmkC5+pM2K/880wb2AsAH\n",
"fBrYL9I5wIuPyOL9/7UnZgEOAI4FrgSyCF6fwbU7zhjp+PGZou2uhjpeiKz7kCu+r8Vm0P48wBNd\n",
"tR9ZvNNd243mcDezhlKuOYBLgSHAVUwb4C9HFh9V3Fa6Qfl14KfAM8AhETxcY10DSPtHdBXMQ4DH\n",
"6BT4wOJ8OoSyLLAU8AZd/4B4NrKYUkttzeBwN7OGUa7FSPsy/Bv4XmQxqea2xArAacCiwKHADc0Y\n",
"TgFQrrmYfkhlEWA80wb44x1/o2hlDnczawjlWg24hrSj2s8rGebosh0xH3A88E3gR8BvIqj5h0Rf\n",
"VWl2+iEmM5sh5doWOAvYJ7K4tqY2xKzASOAo4CJguQjebFyV1hWHu5lNp5j9chTp5ukmkcWYqttI\n",
"4+pbAacC44B1Ini0oYXaDDnczWwayjUQ+C2wArBmZPFi1W2IVUnj6vMC+0dwc2OrtO54VUgz+59i\n",
"uuHNpAeNNqg22CUWlDgXuAG4HFjVwV4Oh7uZAaBcy5Jmw/wb+GZk8UHF54pBEkcDD5GmFC4bwW8i\n",
"aMqDPNY9D8uYGcr1ZeAy4JjI4tyKz0vj6tsBPwHuBdaM4InmVGnVcLib9XHKtTdwErBjZHFrxeeJ\n",
"LwG/AGYH9ojoG3uT9hYOd7M+Srn6AacAWwPrRRaPVXSeWIT0w2Aj4IfA+RF80rRCrSYOd7M+SLlm\n",
"J805n4c0I6ailQolPgv8AziHNK7+XvOqtHr4hqpZH6NciwC3A28DG1UR7EOBPwHHRXCsg721OdzN\n",
"+hDl+gJpNsyVwF6VbA4B/9sI4yLgtgjObmKJ1iAeljHrI5RrS9Jwyncii8o24vxUDswFbNvwwqwp\n",
"HO5mba5YSuAQ4GBgs8jinqrOF9sBuwGrdd6NyFqXw92sjSnXLMCZwOrAWpHFc1WdL1YmLRy2SQSv\n",
"NqFEaxKHu1mbUq65SWPrH5KmOlZ1A1RiXtJSvyMjuLcJJVoT+YaqWRtSrqWAfwEPAlvVEOyzkH4w\n",
"XB7BZU0o0ZrMPXezNqNc65GCeVRk8ZsamzmN1OM/tmGFWY9yuJu1EeXajbQv6S6RxU01tSH2ATYG\n",
"1vCTp72Xw92sDRRLCRwP7AyMiCzG1tSOWJu0tMD6EbzdwBKthznczXo55RoMXEDa+HmNyOK1mtoR\n",
"w0nDOXtG8EgDS7QS+IaqWS+mXAsCo4FJwFfrCPbBwB+BX0Xwl8ZVaGVxuJv1Usq1Emkpgb+Qxtg/\n",
"qqmdtCb7b4EngR83rkIrU93DMpJ+D3wdeDUiVixem4e0xdbiwHhg+4jw+J1ZgyjXZsD5wIGRRb1T\n",
"FQ8GPg+sG0HUW5u1hkb03M8DNu302lHAzRGxDPD34mszawDlGklaI2bLeoNdYmPgcGCrCCY0oj5r\n",
"DYqo/we1pCWA6zr03B8BNoiIV6Q0JhgRy3U6JyJCdV/crI9QrgGknY9GAJtHFuPrak8sDdwJbBfB\n",
"7XUXaD2i0uxs1myZBSLileLzV4AFmnQdsz5BueYk7XEqYJ3I4p262ktrs18LZA729tT0qZAREZK6\n",
"/PVA0qgOX46OiNHNrsest1GuJYA/A7cBB0UWk+tqL63NfiFwewS1PsFqPUTSCNJva9Wd18RhmRER\n",
"8bKkhYBbPSxjVj3lWgu4mrTX6a8iq/8frMTxwJeBr3oJ396n7GGZa4HdSdOqdietLGdmVVCuHYHT\n",
"gT0ji4bMPZfYlvRvcnUHe3uru+cu6VJgA2Be0vj6caR9Fq8AFmMGUyHdczfrWrG5xg+BvYFvRBYP\n",
"NKRdsRJp9tqmEfy3EW1az6s0OxsyLFMLh7vZ9JRrEGma42dJUx1fbki7aW32e4BjI7ikEW1aOSrN\n",
"Tj+hatYilGs+Us96IGnxr0YF+yyk36SvcLD3HQ53sxagXMsDdwG3AjtGFh82sPmfAR8DxzSwTWtx\n",
"XhXSrGTKtRFwMXBYZPGHhrYt9gI2wWuz9zkOd7MSKdd3gFHANyOLOxratliLNIXSa7P3QQ53sxIo\n",
"V3/SjklfA9aNLJ5oaPtiEeAqYC+vzd43OdzNephyzQFcAswGrBVZvNXQ9sUg0trsZ0Tw50a2bb2H\n",
"p0Ka9aBiO7xbgCeA/SOLSQ1tP63NfgFpxs1OXsK3/ZT9hKqZdW0PYBCwX2QxpQntfx9YEa/N3ue5\n",
"527WQ5RrGPAw8LXIYkzD2xcbAX8A1ozgmUa3b63BPXez1vNj4LImBfvSwEXA9g52A4e7WY9QrnVI\n",
"O5Yt3/C2xRyk9ZxGRXBbo9u33slPqJo1mXLNAvwGOCSyeLehbX+6NvudxTXMAPfczXrCQcCLwJVN\n",
"aDsjrci6vW+gWkcOd7MmUq5FSRvEr9mIjTamaVt8E9gTWM1rs1tnDnez5volaQelRj+BuhJpGOZr\n",
"EbzS3fHW9zjczZpEub5OmnO+c0PbFcNIu5t9P4L/NLJtax+e527WBMo1G2lO+36Rxc0Na1cMAG4E\n",
"/hvBEY1q13oPb9ZhVq5jgbsaGeyFnwITgaMb3K61GQ/LmDWYcn0O2A9YuaHtij2BzfDa7FYBh7tZ\n",
"AxWbW58F/CiyeLFh7Yo1SU+4bhBBQ1eRtPbkYRmzxvoWMCcp4BtCYmHS2ux7RzCuUe1ae/MNVbMG\n",
"Ua65gbHAlpHF3Q1pM63NfhvwpwhOakSb1rv5hqpZzzsRuKaBwS7SXPZngJMb0ab1HR5zN2sA5Vod\n",
"2JrGLgx2ELAKsI6XFrBqOdzN6qRcA0g97MMbtWWexIZMXbYg+KARbVrf4nA3q98BwNvAxY1oTGKp\n",
"oq0dIhjfiDat7/ENVbM6KNfCwP3AepHFI3W3l9Zm/xfw6wjOrLc9az++oWrWM04DftugYO9H2ibv\n",
"XzRwKqX1TR6WMauRcm0ErA7s1aAmjwPmB3b0DVSrl8PdrAbKNYjUux4ZWUyouz2xDemHxOoRfFxv\n",
"e2YOd7PaHAk8GFn8pd6GJFYEziatzf5y3ZWZ4XA3q5pyLQ2MBFatu61P12Y/2GuzWyN5toxZFYqF\n",
"wf4K3BxZ/LSuttLa7H8FxkRweCPqs/bn2TJmzbEdsDBp+7x6nQp8QnpYyayhPCxjViHlGkqa+rhD\n",
"ZDGprrbEHsDmpBuoXpvdGq6p4S5pPPAuqXcyKSJWb+b1zJrseODGyOLOehqRWAP4CTDCa7NbszS7\n",
"5x7AiIh4s8nXMWsq5VoV2An4fF3tiHmBK4F9IhjbiNrMutITY+6+aWq9mnL1A34NHBNZvF5zO+kJ\n",
"1POByyO4tkHlmXWp2eEewN8k/UfSvk2+llmz7EsaWjyvznYOBoYBx9RdkVk3mjoVUtJCEfGSpPmA\n",
"m4GREXFH8V5AzBLB5KYVYFYn5VoAeAj4amTxQM3tpHH260g3UMc3qDzrgyqdCtnUMfeIeKn48zVJ\n",
"fyStw3HHp0dscLF0+9Q9IUdHxOhm1mNWg1OBC+oM9rmBy4BvO9itWpJGACOqPq9ZPXdJswH9I+I9\n",
"SbMDNwF5RNxUvB8sfcO98fimX2xKAWZ1Uq4RpFUal48s3q+pjbRV3lXACxEc2MDyrI9qhYeYFgDu\n",
"kHQfcBfw56nB/j+rnbm8VN/sA7NmUK6BpIXBDqo12AvfBZYAP4FqPatpwzIR8TRp/8cZW/LWKQz/\n",
"59Gw9i7NqsOsRocCT5HWfamJxBeADFjLKz1aTyt3+YGJQ37HqudvKzG01DrMOlCuJUnhPjKy2sYt\n",
"i/+nLwdGRvBEI+szq0S54T7klZNZ8ZJ+zP/g/qXWYVYoFgY7HTgtsni6pjbSOPvZwC0RXNbI+swq\n",
"VWq4Rxav8N5CN7HSRYcW/yDMyrYlsDRQz4qPewMrAN9vSEVmNSh/Vci5njmcL5w7D3M9vXHZpVjf\n",
"plxDSKs9HhBZTKypDbECcDKwfQQfNrI+s2qUHu5x/MRxTBg2lpX/cHLZtVifdxxwe2Rxay0nS8wO\n",
"XAEcFsG47o43a6bSwx2AfpOPYKWLVtbgNxctuxTrm5RrBWAP4LA6mjkDuCeCCxpSlFkdWiPc53nq\n",
"Rvp98jpfPKeunW3MatFhYbAssnilpjbEbsCapHntZqVriXCPLIJ3h/+Y5a7ZSmJg2fVYn7M7MCvw\n",
"21pOllgO+BmwQwT1PPBk1jAtEe4ALH7H6cz19BTW/PkRZZdifYdyDSPdAP1OZFH1jkgSg0nz2Y+N\n",
"oOb1Z8warWXCPbKYzEtfvISlbzyo7FqsTzkFuCKyuLfG808DxgG/a1xJZvVrmXAHYPCbh7LI3fNo\n",
"7Z99vexSrP0p1zrAZsAPazpfbA9sBOwXQfPWzjarQVPXc5/phWewspl23uI2pvQbFpdds0IZdVnf\n",
"oFyzAP8FToosqn6KVGIp4F/A1yL4b6PrM5uRVlgVsjZvLjWSJW5bXhsdtWTZpVhbOxB4hTReXhWJ\n",
"WYvzTnCwW6tquZ47gHb52pNMmv2RuPwqD89YwynXosAYYO3I4rGqzxe/ABYHtvFwjPW03ttzB3hm\n",
"/YzFb99E3/nC4LJLsbb0C+DMGoN9K2ArYC8Hu7Wy1gz3fxx9Ma8tP4F3hp9YdinWXpRrM2Bl0vTH\n",
"6s4Vi5NWe9wxgrcaXZtZI7VkuEcQPLHJb1nw/n2LpwfN6qZcs5GWCPhuZPFRVeeKWYBLgZ9G8O9m\n",
"1GfWSK0bnGP2PI6P5hrMM+vuUXYp1jaOAf4TWdxYw7k/At4mPYlq1vJaNtzj/QUnMG6bm5j9tazs\n",
"Wqz3U67lgG8DB1d9rtgU+BawewRTGl2bWTO0bLgDcN8ehzLw/eE6dPjaZZdivVexu9JZwAmRxQtV\n",
"nSsWAc4DvhXBa82oz6wZWjrc4+3Fx3H/ro/z0ZynlV2L9Wo7A3MDZ1ZzksQA4BLgzAhub0ZhZs3S\n",
"0uEOwP27HsfQF76oXEuUXYr1Pso1F3AqaWGwyVWe/kNgEjXMrDErW+uH++vLX839u03gzaVOKrsU\n",
"65VOBK6NLO6q5iSJrwD7ArtEUPVqkWZla/lwj+AT7t/lDIa8/E3lmrvseqz3UK7VgG2Ao6s6TywA\n",
"XAjsFsHLzajNrNlaPtwBeHH1n/PIFuKD+bybvFVEufoDvwGOjCwqfuBIoh8p2M+L4G/Nqs+s2XpF\n",
"uEfwOmP2uoEBH31fubxTk1XiAOA9UlBX40hgEDCq0QWZ9aReEe4APL3hCbz0hYF8MmDnskux1qZc\n",
"CwHHAQdEVvnKeBLrAgcBO0dQ7c1Xs5bSa8I9gnu4a+QzTJp9VDFv2WxGTgPOiSzGVnqCxLykaY97\n",
"R/B80yoz6yG9JtwBeGSrk5kwbBiwcdmlWGtSro2AtUjLBVR2jhBwPnB5BH9pUmlmPap3hXv0v5w7\n",
"jg4+HlLTtmjW3pRrEOlBpZGRxYQqTj0YmBc4timFmZWgV4V7BB/x0I6/YcqAlZVrlbLrsZZzBDA2\n",
"sriu0hMkVgeOIi3jO7FplZn1sF4V7gBMGnIW/zysH5MHHlF2KdY6lGtp0tZ5B1Z8jpgLuAz4dgTj\n",
"m1SaWSl6XbhHMJ5797kNtGWxXZr1ccUN9jOAn0QWz1Z0ThpnPwf4cwR/bGZ9ZmXodeEOwAcL/JL7\n",
"d/2A4KCyS7GWsC0wHPh5FeccACwJHN6UisxK1pIbZHd/Lv2Y+8kn+d5y89B/8mKRxTuNrs96B+Ua\n",
"CowFdoos7qjoHLEqcBOwVgRPNLM+s0br3RtkdyOCKby11Ok8s8FbpMWdrO/KgZurCPY5gCuAkQ52\n",
"a2dNC3dJm0p6RNLjko5swiXO528nz8OU/gd7SYK+qZgx9S3SLJnuj0/j7GcDt0ZwWTNrMytbU8Jd\n",
"Un/SDa5NgeWBnSR9rpHXiOAtXlztMt76zEfA9o1s21pfsXH6r4FjI4tKd0jaC1gR8AJ01vaa1XNf\n",
"HXgiIsZHxCTSdLMtm3CdM7np1KEEh3lJgj5nHyCAcys5WGIF4BRg+wiqecDJrFdqVrgvAjzX4evn\n",
"i9caKoL7efQbj/LxnPMAX210+9aalGt+4ATS7krdblgtMTtwOXBYBOOaXZ9ZK2hWuPfgFJx+Z3Lb\n",
"Dz4ADuu5a1rJTgMujCweqPD4XwH/ieCCJtZk1lIGNKndF4CODxgtCtOvtCdpVIcvR0fE6BqudTV3\n",
"f+80Njx6mHKtVMU/eOuFlGtz0sJgK1V0vNgVWBv4UjPrMmsWSSOAEVWf14x57pIGAI+ShkpeBO4G\n",
"doqIcR2OqXme+/TX40d8Y58N+eK5j0YWezSiTWs9yjUn8BCwe2RxS7fHi2WBfwBfjcA/9K0tlDrP\n",
"PSImA98DbiQ9YHJ5x2BvgrP52ynLEdpCuRo+tm8t41Tg+gqDfTBpPvsPHOzWFzVrWIaIuAG4oVnt\n",
"T3stnpfm/TvPrDuMJe4YSVrlz9qIcm1Imlq7YoWn/Ax4BPht04oya2G98gnVGTiT6343nGAf5Zqj\n",
"7GKscZRrCPA70uyYbpeakNiOtKHLvhE9eXPfrHW0U7iP5o1lJ/Luog8Ce5ddjDXUScDtkcX13R0o\n",
"8RnShh07RPBu0ysza1FtE+5FD+0srv3tFOBg5WrakJP1HOVaj7Tq48HdHitmJc1nPyGC/za7NrNW\n",
"1jbhXriQJzddlUmDXiQFgvViyjWY9ATqdyOLNys45RTSNNxfNbUws16grcK9+DX8Em49fjxwnHIN\n",
"L7kkq88o4L7IotvNNCS2ALYG9vI4u1mbhXvhLP556AgmD7wM+K9yuQffCynXasAewMhujxWLkW64\n",
"7hRBJT18s7bXKzfr6L5tbgF+xyg9CVwM3AEcFFm814zrWWMp16zAf4GTIotLZnqsmAUYDVwTwak9\n",
"UJ5Zqdp6s44KnAkcyah4FVgVmAKMUa41yy3LKnQM8BRwaQXHHg+8Q5rXbmaFdu259wd+SPqV/kLg\n",
"JEZpXdL632eSeoSTm3Ftq49yrQz8DVglsnhhpseKTUibXH8hgkrXdDfr1fp0zz2CTyIYBXwOEDCO\n",
"UbES47ZaH1gPuE25PlNmjTa9Yvrq74EjKwj2hYHzgV0c7GbTa8twnyqCVyM4CPgisCSX//F2jp94\n",
"PZMGXwPcpVy7eZOPlnIY8AZw3swOKn4zuwQ4K4LbeqIws96mLYdlZnxNViRt8rAKK59/DlvtuQPi\n",
"YdJj7W/1ZC02LeX6HOnG95cii/EzPC7tg3oaacnfjSP4pGcqNGsNlWZnnwr3T6/N2sApzPLBvOy1\n",
"7lMseN9KiN0iq2k9eauTcvUnBfvFkcWZMzwu9dh/Qwr2zSJ4o4dKNGsZfXrMvTsR/BPYgEmzH8bZ\n",
"Yxblyss/YPKsVyrXKco1sOz6+qCRwGTSDe8uFUsLXAEsQVqf3cFuNhN9suc+bR30A3ZkyMsnst32\n",
"s7PQva8z8INtIotHyq6tL1CupYC7gLUii8e7PEbMAfwReIt0A/XjHizRrKV4WKZKEgPRJ/uw2pkn\n",
"8ZUfDuKdxU5kgYdOiKykv6A+QLn6AX8H/hxZdDlPXWJe4HpgDHCAx9itr/OwTJUimBhT+p/F3Qcu\n",
"wpWXn0X0z3jhS09r/RMr3RzCqrcvMBvwi67elFiUNBb/N+A7DnazyjncO4ngg3hi00O44ZfDeWfx\n",
"l1n9jPu1xulXSMxddm3tRLkWI81c2iuymC60i/1P7wDOieAYLwZmVh0Py3RDe47Yjvkf+j1jtx3A\n",
"3088gQ+H/TyCCWXX1ZsVzxZcD9wZWZww3fvii8CfgWMiZj7n3ayv8bBMg8R5o69ktjcWY/krb+Hb\n",
"XziChf7ztMR3igWrrDa7AQsCP+78hsQI0t67+zvYzWrnnnuFit7mLnwy4HT+efir3PKjfkT/HwJX\n",
"RDCl7Pp6C+VaCLgf2CSyGDPNe2Ir0obWO0Rwaxn1mbU6z5ZpEuVaEriI9+cfxDn/6sfbnwngaOAm\n",
"jwvPXPED8o/AQ5HFD6Z5T+wBnAxs7i3yzGbMwzJNElk8DWzAkFev4aClFmbz79wM/BK4RcJLCs/c\n",
"9sAywI86vihxCJADIxzsZo3hnnsdivXhL2JKv9v5xdP38O5iR5M2mfhBBA+XXF5LUa75gAeArSKL\n",
"u+B/68ScSNoeb+MIniuxRLNewT33HhBZ/BtYlX5TgkMWP4RDFtmZNH3vFonzJBYvucRW8kvgkg7B\n",
"PnWdmI2A9RzsZo3lcK9TZPFeZLE3cBRDX7yaUZqN+R5eDngeuFfi5xLzlVxmqZRrS2B10gYqU9eJ\n",
"uRRYGvhKBK+XWJ5ZW/KwTAMp13DSBhKDgF0ZFROAHwA7A6cDp0XQp/ZxVa65gYeAnSOL2ySGAP8H\n",
"vA/sHMFHpRZo1st4WKYEkcXzwMakGSF3M0obMUoHAquReqmPSxxU9Fz7ip8B1xTBPoy0lMCzwPYO\n",
"drPmcc+9SZRrFdJuQfcD+0cWb0usRLqBuCKQARe183opyrUJcDawIqNiKHAT8BfgSE8bNauNe+4l\n",
"iyzuI23v9zpwv3JtEMEDEXwD2IW0aNYDElsWs0bainLNQQr2/RgVCwL/AC6I4AgHu1nzuefeA5Rr\n",
"M+Ac4ALdFmOOAAAJ/UlEQVQgiywmFoG+GenBnQ+Ao9ppP1DlOhMYzKj4Fam3flwE55Rcllmv5557\n",
"C4ksrgdWAVYA/qlcy0YQEfyleP0M4DyJGyRWLbPWRlCuDYEtOffOq4AbgZEOdrOe5XDvIZHFq8AW\n",
"wLnAP5RrP+VSBFMiuBhYjrQS4vUSl0osXWa9tVKu9YFL+McRv+a5tS8gzYi5uuy6zPoaD8uUQLk+\n",
"B1xMmjWyb2Tx2v/eS1MFDwIOBq4Ejo/gpVIKrZJyjSB0BTf88jruHvl1YIsI7i67LrN24mGZFhZZ\n",
"jAPWBB4F7lOuTf/3XvB+BCcCy5Lmgj8kcZLEXOVUWxkdvuCmTJztOi786wDuHjkHsK6D3aw87rmX\n",
"TLm+TLrR+kfgyMhimrnfxVZzxwFbAacCZ7TSZiESg1j7Jz9l3Z/sz3Vn38m4b34vggfKrsusXZXa\n",
"c5c0StLzksYUH5t2f1bfFFncCqwMLATco1zT7NkawXMR7AusR3qE/zGJ/creLERioMR3WObPz7L+\n",
"SfvyyFb7xdhvru9gN2sNTem5S8qA9yLitJkc4557B8Va57sBPwVOAn4ZWUy3CYjEaqTpk4uRlja4\n",
"qic3Cyl+qOwO/IAVL36NrfZYiv6TvxFZ3NlTNZj1Za0w5u7grkJkEZHFBcAawHbAX5Vr4emOC+6J\n",
"YEPgAOBw4B6JjZv9IJTEAIndgUeAHdlmlzP45i6L03/y1x3sZq2nmeE+UtL9ks6V1NI3A1tJZPEU\n",
"sD5wJ3Cvcm3d5XHB30jDNCcDvwL+LrFGo+uR6C/xLWAssBewF6P0S1a6+AjgG5HFvxp9TTOrX83D\n",
"MpJuJm1y3NmxwL+BqdP7fgQsFBF7dzo/SLvvTDU6IkbXVEybUq61gIuAW4CDI4v3uzxODAD2IK1X\n",
"cw9ps5CxdV1b9AO2BUYBb5OW672FUdqCtM/p5pHFPfVcw8y6J2kEMKLDS1lL7KEqaQnguohYsdPr\n",
"HnOvQLFGy+nAOsC3ZhaoEoOB7wJHkB6IGhXBs1VdLw3vbE36wTuBNFPnpghCubYBzgI2iyzureX7\n",
"MbP6lLpBtqSFIuKl4vODgdUiYudaCrREubYjLVNwOnBKZDHD1SSLOfGHAfuTplme1N2GGEWob04K\n",
"9SCF+vVTF/lSrm2L638tshhT/3dkZrUoO9z/QFozJYCngW9HxCu1FGifKjYD+QMwANg1snhmpseL\n",
"BUkzanYk/VD4eefNQopQ3wQ4HpiVNLTzp44rNyrXDsAvgE0ji/sb9x2ZWbVKDfdKONxro1z9gENJ\n",
"M2UOjiwu7vYcsRSpR74haZrl2cBE4KukUJ+TNLZ+dedplcq1M2nDjY0jiwcb952YWS0c7m1OuVYl\n",
"rU8zBvhuZPF2t+eIlUmbhawAPAfMTwr1K7raNES5dgF+AmwUWTzcuOrNrFYO9z5AuWYjLUnwddIw\n",
"zR0VnSfWBRYh9dQnz6Dt3Um9/I0ii7pm3phZ4zjc+xDl2hz4HfB7YFRkManO9vYiDddsGFk80oAS\n",
"zaxBWuEJVeshkcWfSTewVwHuVK5lam1LufYhjc9/xcFu1nu5595GivVp9ieF8zHAOZFV/h9Yub5N\n",
"egjtK5HFE82p0szq4WGZPky5lifdbB1P2gxkpnPci3MOID389NXI4snmVmhmtfKwTB9W3ABdE3iC\n",
"tBnIxjM7Xrm+R5pa+WUHu1l7cM+9zSnXV0hPqV4FHD3dZiC5DiJt6/eVyGJ8z1doZtVwz90AiCxu\n",
"IW0GMhy4u+NmIMp1CHAgMMLBbtZe3HPvI4qbrbuT5sWfQFpqYF9Sj/25Mmszs8r5hqp1SbmWIi0j\n",
"PIw0xv5CySWZWRUc7jZDytUfGBhZfFh2LWZWHYe7mVkb8g1VM7M+zOFuZtaGHO5mZm3I4W5m1oYc\n",
"7mZmbcjhbmbWhhzuZmZtyOFuZtaGHO5mZm3I4W5m1oYc7mZmbcjhbmbWhhzuZmZtyOFuZtaGHO5m\n",
"Zm3I4W5m1oYc7mZmbcjhbmbWhhzuZmZtyOFuZtaGHO5mZm3I4W5m1oZqDndJ20l6WNInkr7Q6b2j\n",
"JT0u6RFJG9dfppmZVaOenvuDwNbA7R1flLQ8sAOwPLApcJaktvsNQdKIsmuoh+svl+svT2+uvRo1\n",
"h25EPBIRj3Xx1pbApRExKSLGA08Aq9d6nRY2ouwC6jSi7ALqNKLsAuo0ouwC6jSi7ALqMKLsAnpC\n",
"M3rUCwPPd/j6eWCRJlzHzMxmYMDM3pR0M7BgF28dExHXVXGdqKoqMzOriyLqy11JtwKHRsS9xddH\n",
"AUTEKcXXfwWyiLir03kOfDOzGkSEujtmpj33KnS80LXAJZJOIw3HfBa4u5bizMysNvVMhdxa0nPA\n",
"msBfJN0AEBFjgSuAscANwAFR768HZmZWlbqHZczMrPWUOv9c0o8k3S/pPkl/l7RomfVUS9KpksYV\n",
"38P/SZqz7JqqMbMH0VqVpE2Lh+Mel3Rk2fVUS9LvJb0i6cGya6mWpEUl3Vr8P/OQpAPLrqkakgZJ\n",
"uqvIm7GSTi67plpI6i9pjKSZTmop++Gin0TEyhGxCnANkJVcT7VuAj4fESsDjwFHl1xPtbp8EK1V\n",
"SeoPnEF6OG55YCdJnyu3qqqdR6q/N5oEHBwRnycNx363N/39R8RHwJeLvFkJ+LKkdUsuqxYHkYa9\n",
"ZzrsUmq4R8R7Hb4cArxeVi21iIibI2JK8eVdwPAy66nWTB5Ea1WrA09ExPiImARcRnporteIiDuA\n",
"t8quoxYR8XJE3Fd8/j4wjvRcS68REROKTwcC/YE3SyynapKGA5sB5zDtRJbplN1zR9KJkp4FdgdO\n",
"KbueOuwFXF92EW1uEeC5Dl/7AbmSSFoCWJXUqek1JPWTdB/wCnBrMQGkN/k5cDgwpbsDmx7ukm6W\n",
"9GAXH98AiIhjI2Ix4HxS4S2lu/qLY44FJkbEJSWW2qVK6u9FfPe/BUgaAlwFHFT04HuNiJhSDMsM\n",
"B9bvTevMSNoceDUixtBNrx0aN899hiJiowoPvYQW7Pl2V7+kPUi/Jn21RwqqUhV//73BC0DHm+6L\n",
"Mu1SF9ZkkmYBrgYuiohryq6nVhHxjqS/AF8CRpdcTqXWBraQtBkwCBgq6Q8RsVtXB5c9W+azHb7c\n",
"EhhTVi21kLQp6VekLYubNb1Zb3io7D/AZyUtIWkgafXRa0uuqc+QJOBcYGxE/KLseqolaV5JcxWf\n",
"DwY2ohdlTkQcExGLRsSSwI7ALTMKdih/zP3kYojgPtJKbYeWXE+1fkW6EXxzMTXprLILqsaMHkRr\n",
"VRExGfgecCNptsDlETGu3KqqI+lS4J/AMpKek7Rn2TVVYR1gF9IskzHFR2+a+bMQcEuRN3cB10XE\n",
"30uuqR4zHab0Q0xmZm2o7J67mZk1gcPdzKwNOdzNzNqQw93MrA053M3M2pDD3cysDTnczczakMPd\n",
"zKwN/T9xkbdVhKP+GwAAAABJRU5ErkJggg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x1053ff350>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"x = X;\n",
"\n",
"##This function provides an interactive mode to change polynomial degree. \n",
"@interact(n=[1,16])\n",
"def plot(n):\n",
" X = construct_poly(x,n)\n",
" w = wRidge(X,y,alpha)\n",
" plt.plot(x,X.dot(w))\n",
" plt.title(\"MSE %f\" % MSE(X.dot(w),y))\n",
" plt.plot(x,y)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"8) Apply models with different values of $d$ after being trained on the training dataset, to the test data available [here](http://webdav.tuebingen.mpg.de/lectures/ei-SS2015/data/ex1_train.csv). Compare the errors on the test data to the ones from the training by plotting the error curves as functions of the polynomial degree in a single plot! What do you conclude? <br>"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"## Read test data here"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"9) With a fixed optimal $d$, change the value of $\\lambda$ to one of the following values $[0.1, 1.0, 10.0]$ and find the minimum MSE!<br>"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Hand in printed copy of completed notebook."
]
}
],
"metadata": {
"annotations": {
"author": "",
"categories": [
"intelligent-systems-1-2015"
],
"date": "2015-04-30",
"location": "Beginning of next lecture",
"parent": "IS_SS2015",
"submission_date": "2015-05-07",
"subtitle": "Exercise Sheet 3, Linear Regression",
"tags": [
"IntelligenSystems",
"Course"
],
"title": "Intelligent Systems 1 - Summer Semester 2015"
},
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
MaxXximus92