diff --git a/.gitignore b/.gitignore index 88570c4..2d2d2c8 100644 --- a/.gitignore +++ b/.gitignore @@ -2,4 +2,4 @@ *.aux *.out *.synctex.gz - +.ipynb* diff --git a/is/UB2/ExerciseSheet2.ipynb b/is/UB2/ExerciseSheet2.ipynb new file mode 100644 index 0000000..c3f7487 --- /dev/null +++ b/is/UB2/ExerciseSheet2.ipynb @@ -0,0 +1,801 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "preamble": true + }, + "source": [ + "(Defining latex commands: not to be shown...)\n", + "$$\n", + "\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n", + "\\DeclareMathOperator{\\minimize}{minimize}\n", + "\\newcommand{\\real}{\\mathbb{R}}\n", + "$$" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Monster vs. Mouse (50 Points)\n", + "\n", + "While trying to sleep, you hear some noise under your bed. Being a big Science Fiction fan you obviously immediately think it's a monster. A BIG monster... or maybe two?!? Frightened to death, you do not dare to move, but your brain is racing. And it recalls that the other day, you saw a mouse disappearing on the other side of the bedroom... So maybe it is (just?!?) a mouse. Or, just something else... Hmmm. Remembering your Emperical Inference classes, you decide to evaluate the probability that, given some noise under your bed, there is a monster.\n", + "\n", + "You define the random variables:\n", + "\n", + " + n = some noise under your bed (Values 0 or 1)\n", + " + M = 0, 1 or 2 monsters under your bed (Values 0, 1 or 2)\n", + " + m = a mouse under your bed (Values 0 or 1)\n", + " + e = something else (e.g. only air) under your bed (Values 0 or 1),\n", + "\n", + "and express your beliefs about monsters, mice and noise, by assigning numbers to P(M), P(m), P(e), P(n|M), P(n|m), P(n|e). Given that you heard some noise under your bed, you then calculate the Maximum a Posteriori (MAP) of M (i.e., the maximum of P(M|n=1)).\n", + "\n", + "Please share your beliefs and your results with us." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Programming a Hand-Featured SVM (50 points)\n", + "\n", + "The goal of this exercise is to implement a very simple SVM using an off-the-shelf optimizer, in the case of a 2-dimensional input space.\n", + "Most of it has already been implented in an iPython Notebook that you can find on the homepage of the course, and that we reproduce hereafter.\n", + "\n", + "You are asked to hand out a print out of the lines marked \"### CHANGE THIS LINE ###\", as well as your plots." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## The SVM equation in the linear separable case\n", + "\n", + "Let $(x_1,t_1), \\ldots, (x_N,t_N) \\in \\real^p \\times \\{-1,1\\}$ be N data points with their binary labels. For this exercise, $p=2$. \n", + "In the lecture, we saw that trying to maximize the margin between two linearly separable classes of data points led to the problem:\n", + "\\begin{equation}\n", + " \\minimize_{\\substack{ \\{ w \\ \\in \\real^2,\\ b \\ \\in \\ \\real \\} }} \\frac{1}{2} \\norm{w}^2 \\quad \\mathrm{subject \\ to} \\quad t_n(w^T x_n + b) \\geq 1 \\quad n = 1 \\ldots N ,\n", + "\\end{equation}\n", + "whereby the points are subsequently classified according to the rule:\n", + "\\begin{equation}\n", + " t = \\left \\{ \n", + " \\begin{array}{cc}\n", + " \\ \\ 1 \\quad \\mathrm{if} \\quad w^T x + b \\geq 0 \\\\\n", + " -1 \\quad \\mathrm{if} \\quad w^T x + b < 0\n", + " \\end{array}\n", + " \\right . .\n", + "\\end{equation}\n", + "The function $\\mathop{l}(w,b) := \\frac{1}{2} \\norm{w}^2$ is called the _objective_ function. The equations $t_n(w^T x_n + b) \\geq 1$ are called the (_inequality_) _constraints_.\n", + "\n", + "In practice, one often preferres to transform this problem into its so-called _dual_ formulation. However, in this exercise, we will stick to the above equations: the _primal_ formulation." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Loading the required packages." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "%matplotlib inline\n", + "import numpy as np\n", + "from numpy import *\n", + "from scipy import optimize\n", + "from matplotlib import pyplot as plt" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Definition of the data point \n", + "\n", + "_X = location of the data points_ $\\in \\real^{N \\times 2}$ \n", + "_t = label class of each point_ $\\in \\real^N$" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "X_ = np.array([[1.,3.],[2.,6.],[2.,3.],[-1.,0],\n", + " [1.,0.],[2.,2.],[3.,1.],[0.,-1.]])\n", + "t_ = np.array([1,1,1,1,-1,-1,-1,-1])" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Definition of the Objective Function and its Gradient\n", + "\n", + "_wb = $(w_1, w_2, b) = (w , b)$_ $\\in \\real^2 \\times \\real$\n", + "\n", + "(Note that the gradient of the objective function (here called 'jac') does not appear in the equations of an SVM. But our off-the-shelf optimizer uses it for its computations.)" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "def objective(wb):\n", + " ### changed ###\n", + " return 1/2*(wb[0]*wb[0]+wb[1]*wb[1])\n", + "\n", + "def jac(wb):\n", + " return hstack((wb[0:2],array([0])))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Definition of the Inequality Constraints and of their Gradient\n", + "\n", + "(Again, the gradient does not appear in the SVM equations, but it is needed for our optimizer.)" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": { + "collapsed": true + }, + "outputs": [], + "source": [ + "def ineq(wb):\n", + " #writing \"return a-b\" would encode the constraint a-b >= 0\n", + " ### changed ###\n", + " return t_*(list(map(lambda x:np.dot(wb[0:2].T,x),X_))+wb[2])-1\n", + "\n", + "def grad(wb):\n", + " return -1*hstack(((-t*X.T).T, -t[:,newaxis]))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Optimisation of the SVM" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "collapsed": false, + "scrolled": true + }, + "outputs": [], + "source": [ + "def optimizeSVM(X_,t_):\n", + " global X,t\n", + " X = X_\n", + " t = t_\n", + " wb0 = np.random.randn(3) #initialization\n", + " #wb0 = array([-1.,-2.,0.2])\n", + "\n", + " cons = {'type':'ineq', #constraints\n", + " 'fun': ineq,\n", + " 'jac': grad}\n", + "\n", + " opt = {'disp':False}\n", + "\n", + " opt_result = optimize.minimize(objective, wb0, jac=jac,constraints=cons, #the optimizer\n", + " method='SLSQP', options=opt)\n", + " wb_opt = opt_result[\"x\"] #result of the optimization\n", + " \n", + " return wb_opt\n", + "\n", + "wb_opt = optimizeSVM(X_,t_) #the optimized parameters" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Plotting the Result" + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuYXFWd7vHvSwK5kKBwVEYkiANqAsEQjgQiAo1XaB0d\n", + "RlG8zIghmTkDpAMiwjgjncILCAqSgBwNgiJ4GW6O3JQg9hkQDiA3RcBLPJjg4SaOmGiiQX7zx6rQ\n", + "3aE76e61q2rvqvfzPDxQ1VVr/4wPv16svfa7FBGYmVn1bNHqAszMbGzcwM3MKsoN3MysotzAzcwq\n", + "yg3czKyi3MDNzCoqq4FLeqWkuwf89ZSknqKKMzOz4amofeCStgB+DcyJiFWFDGpmZsMqcgnlDcAK\n", + "N28zs+YosoEfDnytwPHMzGwTCllCkbQVaflkt4h4IntAMzPbrPEFjXMIcOdQzVuSw1bMzMYgIrSp\n", + "nxfVwN8DfH2sRZSBpMURsbjVdWxOFeqsQo3gOovmOos1kslv9hq4pK1JNzCvyB3LzMxGLnsGHhF/\n", + "AF5QQC1mZjYKfhKzX1+rCxihvlYXMAJ9rS5ghPpaXcAI9bW6gBHqa3UBI9TX6gKKUtiDPMNeQIoq\n", + "rIGbmZXJSHqnZ+BmZhXlBm5mVlFu4GZmFeUGbmZWUW7gZmYV5QZuZlZRbuBmZhXlBm5mVlFu4GZm\n", + "FeUGbmZWUW7gZmYV5QZuZlZRbuBmZhXlBm5mVlFFnMjzfEmXSXpA0v2S9i2iMDMz27QizsQ8G7g2\n", + "It4paTywdQFjmjXMTKl7GvRMhYmrYd0qWHJfxLWtrststLIauKTnAftHxAcAIuJp4KkiCjNrhJlS\n", + "91w4exnsuuG9BbDLTAk3caua3CWUlwFPSLpQ0l2SlkmaXERhZo0wDXoGNm+AZbDrTrCwVTVZE0nP\n", + "R5rU6jKKkruEMh7YCzgmIu6Q9DngJODkgR+StHjAy76I6Mu8rtmYTIWJQ70/BdrmX2obgrQFcATw\n", + "KWA+cHVL6xmCpC6gazTfyW3gDwMPR8Qd9deXkRr4IBGxOPM6ZoVYDeuGen8NrG12LdYk0t7AOcAz\n", + "wFuIuLPFFQ2pPrHt2/BaUu/mvpO1hBIRjwKrJL2i/tYbgJ/kjGnWSKtgyQL4xcD35sOKlbC0VTVZ\n", + "g0gvRFoGfBs4D9ivrM17rLJPpZc0Czgf2ApYAXwwIp4a8HOfSm+lMlPq3gkWToFJa2DtSljqG5ht\n", + "JO2G+19AL3AxsJgBPakqRtI7sxt4EUWYmRVC2p+0XPIk0EPEfS2uaMxG0juL2AduZtZa0g7AGcD+\n", + "wIeBS2n07LQE/Ci9mVWXtBXSCcCPgIeAGUT8eyc0b/AM3MyqSnoTsIR0721fIn6xmW+0HTdwM6sW\n", + "aWfgTGAWsIiI0u3pbhYvoZhZNUiTkE4GfgjcBezeyc0bPAM3s7KTBLwNOAu4E9iLiJWtLaoc3MDN\n", + "rLzSQ4JnAy8FFhDxvRZXVCpeQjGz8pGmIJ0G3AIsB2blNm+J7STOkXhdITWWgBu4mZWHJKTDgQeA\n", + "FwN7EHEmEevHPiTjJBbUxxwH3FtMsa3nJRQzKwdpD1ImzfOAw4n4Qf6Q7EN6MnM9cEgEd+WOWSae\n", + "gZtZa6WM7rOB7wHfBF6d27wlXiTxJeBK0i+F17Zb8wY3cDNrFWkLpHnAg8AEYDciziPiL2MfkvES\n", + "PaRU1N8BMyK4KIJniim6XLyEYmbN14CMbokDSbPtJ4CuiPaPtnYDN7PmkV5IOhXnraTDX75KRNbs\n", + "WGJHUpDVfsCHgMsj6IgsFC+hmFnjSeORjiYtbawGphPxlZzmLTFB4kTSrpIVpOWSyzqleYNn4GbW\n", + "aIMzug8iIntpQ+Jg0gM+PwP2iaDjgqyggAYu6SHg98BfgPURMSd3TDNrAymj+3TgAArK6JZ4GemR\n", + "+pnAsRHlO5y4mYpYQgmgKyJmu3mb2UYZ3b+igIxuickSNVKQ1e3AzE5v3lDcEoqPTDOzwjO6JQT8\n", + "LWnWfTswOwIHWdUV0cADuEHSX4AvRMSyAsY0sypJGd2fBfakoIxuiemkXwYvAeZFcGP2mDXtCfz/\n", + "6I3Hc8cqgyIa+H4R8YjS9qDlkh6MiJsGfkDS4gEv+yKir4DrmlmrSZOAE4Ae4HPA+4hYlzckU4GP\n", + "AfOATwLnRDDmLBQA1bQd8HHgncC7gNI1cEldQNeovlPk0XGSeoE1EfHZAe/5VHqzdvPcjO7jczO6\n", + "68sl7wU+DdwAnBTBo1lj1jQOOJLUvC8FTo7e+G3OmM3S8FPpJU0GxkXEaklbA28CajljmlnJNSCj\n", + "W2IW6SnKKcC7Irgle8ya9iVtX1wHvDl6457cMcsmdwlle+DK9MuY8cAlEXF9dlVmVj7SFODfgPmk\n", + "pymX5sS8piHZljQ7Pgw4GTg/gjFnoQCopu2B00gTyhOBS6K3PU+pz2rgEfH/SDctzKxdpRnau0mP\n", + "q99Iyuh+JG9ItiCtcX+ClBi4WwRPZo1Z05bAUaRfMl8GZkRv/D5nzLLzk5hmNrzGZHTPIS1tPA10\n", + "FxHzqpoOItX5CHBA9MYDuWNWgRu4mT2X9HzS/az3AL3AF3NiXtOQvAg4FTiEFGR1cW7Mq2raEfgM\n", + "sC8pyOrKdl0uGYobuJn1k7YAjiCtcX+LlNH9m7whGU9a2vgYcBEpdOqprDFrmkBq2McD5wLzojf+\n", + "mDNmFbmBm1lSkYxu1dRN2gVzPzAneuOXuWNWlRu4WaerSEa3avpr0sNC04Ge6I3rcsZrB84DN+tU\n", + "FcnoVk2TVdMppCyUW4A93LwTz8DNOlEFMrpVk4BDgTOBW4E9ozcezq2znbiBm3WSimR0q6YZpCCr\n", + "vwKOiF7nJw3FSyhmnaDxGd13UEBGt2raRjWdAfwncBUw2817eJ6Bm7W7CmR015dL3kcKsvouMDN6\n", + "47GcMTuBG7hZu0oZ3WcCsyh/RvdSYBLwjuiN/5s7ZqfwEopZu5EmIZ1MWtq4C9g9t3lLTJU4HbgZ\n", + "uA7YM7d5q6btVNO5pBn3RcA+bt6j4xm4Wbt4bkb3Xg3I6J5ZUEb3hiCrS0mhU5XI6C4bN3CzduCM\n", + "7o7kJRSzKpOmIJ1GesBlOTArt3lLbCtxDnA9cAmwd27zVk3bq6YLgMtJT1Pu7+adzzNwsyqqTkb3\n", + "eOBoOiiju5kKaeCSxpFumDwcEX9TxJhmNozqZHR3kep8lA7K6G6mombgi0jJYFMLGs/MNuaMbttI\n", + "9hq4pB2BbuB8wKfPmxVN2gJpHvAgMIGU0X1eTvOWGC/RQwqy+h0pdOqinOatmiaoppOAe4CfArtF\n", + "b1zh5t04RczAzwJOALYpYCwzG0h6NenAgrJndB9C2gXzAB2e0d1MWQ1c0luBxyPibkldm/jc4gEv\n", + "+yKcbWC2SYMzuv8FuKjEGd1nAbsBi6I3rs0Zr5PVe2jXqL6TE0Qm6VPA35NufEwkzcIvj4h/GPCZ\n", + "iAgvrZiNhDQe+CfSGvfFQI2IvOPHxATgWOAjwHnAqRH8IWvMmiaT1syPIq13nxW98aecMW2wkfTO\n", + "rAa+0cUOBD688S4UN3CzERqc0b2wARndxzUgo/sEZ3Q3xkh6Z9H7wH2zwmy0nNFtY1TYk5gR8X8i\n", + "4m1FjWfW9hqf0X07zuhua34S06wVnNFtBXADN2smZ3RbgRxmZdYMzui2BvAM3KyRnNFtDeQGbtYo\n", + "zui2BvMSilnRnNFtTeIZuFlRBmd0fx9ndFuDuYGbFcEZ3dYCbuBmOZzRbS3kBm42FtIWwBGkxMD/\n", + "IGV0/yZvSMaTwqE+RtrCNyOCvCCrmiYAx5Ee0T8XmBe98cecMa083MDNRkvam7S04Yxuayk3cLOR\n", + "ql5G9wxSRvd1OeNZeXkbodnmSOORjiYdP7YGmE7El3Oat8QEiROBe0l5KDMiuCyneaumyarpFFIW\n", + "yq3AHm7e7c0zcLNNGZzR/Toi7ssfclBG9z4NyOje0xndncEN3Gwo/RndBwLH44xuK6GsJRRJEyXd\n", + "JukeSfdLOrWowsxaYnBG90qc0W0lljUDj4h1kg6KiD8qneV3s6TXRsTNBdVn1jz9Gd2/BOYS8fO8\n", + "4ZzRbY2VvYQS8eye0q2AcYATzDrUTKl7GvRMhYmrYd0qWHJflO+U8o3rnAHfODMlBs4iHf57dQHL\n", + "Jc7otobLbuBKDzTcBewCnBcR92dXZZUzU+qeC2cvg103vLcAdpkpUaYmPlSdf4Q33gWX7JUyutfl\n", + "jC8xlfQgzjzgk8A5EazPGrOm7YCPA+8k5ZdcEL15T3tae8jeRhgRz0TEnsCOwAGSurKrssqZBj0D\n", + "myLAMth1J1jYqpqGMlSdk2GLf4MX5jRvCUm8j/TQzItI69xn5TRv1TRONS2ojxmk0Kllbt62QWG7\n", + "UCLiKUnXAK8G+gb+TNLiAS/7Inyzpd1MhYlDvT8l/ed+aewB2w71fk6dzui2ItQnv12j+U5WA5f0\n", + "AuDpiPidpEnAG0nBPoNExOKc61j5rU6N5jnWwNpm1zIkaQrwb70we6gfj6VOiW1JSxuHAScD50eQ\n", + "F2RV0/akIKs3AycClzh0qjPUJ7Z9G15L6t3cd3KXUF4M3CjpHuA24Koo4NQRq55VsGQBgx9ImQ8r\n", + "VqaZaetIQjqctAyxw7Hwwdw6JbaQmF8fcxwpo/sLOc1bNY1XTYuA+0gPDc2I3rjYzds2RZk32zd/\n", + "ASkiQg29iJXCTKl7J1g4BSatgbUrYWlLb2AOzug+ZkNGd06dAzK61wMLG5DR3eOMboOR9U43cGs/\n", + "jc/oPpGU0Z0bOuWMbhvWSHqnH6W39tGcjO7pEWQdP1bP6P4Q6RF9Z3TbmLmBW3tobEb348CBEWQ/\n", + "46CauklBVvfjjG7L5AZu1dbYjO7XkGbJRWR070J6pH46aZ3bMa+WzXngVk2Nzei+h7RTZbcCM7pv\n", + "A27BGd1WIM/ArXoan9G9b0EZ3X9Hyui+BWd0WwO4gVt1pIzuM4ADaExG96IIrskuc3BG9wcc82qN\n", + "4iUUK7/BGd2/opiM7kkSi4E76M/ozmre9Yzuz5Ayuq8G9nLztkbyDNzKrTkZ3auyxnRGt7WIG7iV\n", + "k7Qzaf24Chnd55DCvJzRbU3lJRQrF2kSKcTnTlLO/O5EXJW5XDJV4nTgJuBaYM/c5q2atlNN55Jm\n", + "3BcB+7h5W7N5Bm7lIIl0Ks7nSM17LyJ+lTckAt5LWtq4AdgjgkezxqxpHHAkKYXwMlLolE+hspZw\n", + "A7fWk15J2sL3UmABETfkD/lsRvfWwGER3Jo9pjO6rWS8hGKtI01BOg34AbAcmJXbvCW2lTgHuB64\n", + "BJiT27xV0/aq6ULgCtJ/Iezv5m1l4Bm4NV9aLjmctKf7RmAPIh7JG5ItSOdQfoLUaHeL4MmsMWva\n", + "EjiadA7ll4Hp0RtZQVZmRXIDt+aSXkVa2tgGePeGjO68IQdldHcXlNF9EKnOR0gzbmd0W+nkHqk2\n", + "jXQH/kWkQ1e/GBFLiijM2kzK6D6FNPMuc0b3NAZndF/hjG4rq9wZ+HrguIi4R+nMwTslLY/wbMXq\n", + "qpXRfTz9Gd0fdEa3lV1WA4+IR0nHQBERayQ9AOxAOivQOl1JM7qlmd0wrQemToTV6+i+7Rbm8Pek\n", + "jO69ndFtVVHYGrjSk3OzSbGZ1slKnNGdmvfcs2HZrmy7Ag4+Dl60/nXcvu0n4prfnpJTo1mzFbKN\n", + "sL58chmwKCLWFDGmVVDK6D6GUmd0T+thy7N35aCPwYI5sGounPObLbl237ljrdGsVbJn4JK2BC4H\n", + "Lo6Ibw3zmcUDXvZFOKGt7fRndP+WMmd07/U/p3HADHh4Lvzve+D30+o/nTIpt16zHJK6gK5RfScn\n", + "H0hpP+9XgCcj4rhhPuNT6dtZdTK6dwOW8F+T9uE/rpnCQwdt9Inu70Rce0judcyKMpLembuEsh/w\n", + "fuAgSXfX/zo4c0yrgmpmdF/FF176Xh762kYz+fkrYOXSnOuYtULuLpSb8eP4nadxGd1nUmxG9/tJ\n", + "QVbfAXaP3niMXpBm/gW6F6ZlkzVrYeXSiPuuzbmeWStkLaGM6AJeQmkfjcnofiXpl8GOwMKCMrpn\n", + "k7YaTgSOccyrVVEzllCsEzQ2o/tm4DqKy+j+PGnG7Yxua3vOQrHhNS6j+z3A6RSb0T2f9Ki+M7qt\n", + "Y7iB29Aak9H9KtJWwyIzuufWx1yLM7qtw3gJxQZrXEb30vp4RWd0X07acuiMbus4noFb4oxus8px\n", + "AzdndJtVlBt4J2tcRvengG6Ky+jekZTRPRdndJs9yw28EzUuo/ufgZMpNqP7Q8CHSRnd85zRbdbP\n", + "DbzT9Gd0B8VldB9QH3PMGd3PGbOmbtIuGGd0mw3DDbxTNCaj+yWkm577kZHRPWjMmnYh7SqZASyK\n", + "3vAj7mbD8DbCdtef0X0/8AdS6FRuRvdW9Yzue4EVZGd0g2qarJpOIR0Icisw083bbNM8A29ngzO6\n", + "Dyooo/vNpOyS4jK64e9IGSu3ArOjN7KCrMw6hRt4Oxqc0f1hICvmNQ3Jy0hNdg+Ky+ieQfpl8GLg\n", + "iOiN7+eOadZJvITSTobO6P5mgRndd1B8RvfVpFm3m7fZKHkG3i4GZ3S/hoif5Q3XsIzu95Eyur9L\n", + "Wud+LGdMs05WxJmYFwBvAR6PiD3yS7JR6c/o3hNYRPEZ3UcWlNG9J2k9fiLwDse8muUrYgnlQsDH\n", + "qDXb4Izuu0kP45Q1o/tc0ozbGd2jIM3slg75jvSuvvT3md2trsnKJXsGHhE3Kc0CrRmqldF9JPBx\n", + "nNE9aqlZzz0blu3a/+6CXaSZ+Pg328Br4FUyOKP7H4lYnj/ksxndU4B3RXBL9pg17Vsfcx3O6B6j\n", + "aT2Dmzek190LATdwA5rUwCUtHvCyLyL6mnHdtiFNIcWnzgdOBZYS8ee8IdmWFGT1LlJ+yfkR5AVZ\n", + "1bQ9cBrwZuAjwCUOnRqrqROHfn/KpObWYc0iqQvoGs13mtLAI2JxM67TdgZndH+f4jO6r6S4jO6j\n", + "SL9kvoIzuguwet3Q769Z29w6rFnqE9u+Da+V7nFtkpdQyqo/o/t5wOFE3Jw/5LMZ3U/TmIzuA5zR\n", + "XZRVS2DBLoOXUeavgJVLW1eTlY0yd5wh6evAgcD/IKXRnRwRFw74eUSEsi7SSQZndC8GvlBwRvdJ\n", + "pIzuvCArZ3Q3XLqRudPCtGyyZi2sXOobmJ1jJL0zu4EXUYQxVEb3vzYgo/uUCJ7KGvO5Gd2nOaPb\n", + "rHgj6Z1eQimDwRndbyXih/lDPpvR/QTQFcFPssd0RrdZqbiBt1JjM7pfS5opO6PbrE05zKoVBmd0\n", + "r6H4jO5fAjOc0W3W3jwDb7b+jO4ncUa3mWVwA2+WwRndxwOXFpzRfWwEV2eX6Yxus8rwEkqjDZ3R\n", + "nXXAwjAZ3VnN2xndZtXjGXgjDc7onkvEz/OGG5TRfQewVwQrs8Z0RrdZZbmBN0J/Rvcs4Fic0W1m\n", + "DeAllCKljO6TSTGvdwG7F5jR/QPSDNkZ3WYGeAZejP6M7rNIjbsRGd0zndFtZgO5geeSXkF6OnFn\n", + "Ukb3DflDOqPbzDbPSyhjJU1BOg24hTRDnpXbvCW2lVgKLAcuAfbObd6qaXvVdCFwOekUn/3dvM3a\n", + "g2fgo5WWS95NtTK6v0xaLnFGt1kbcQMfDWkPBmd0/yB/SGd0m9nYuIGPRMrorpFuKvYCXyx5Rve+\n", + "pCCrK53Rbda+3MA35bkZ3bs1IKN7RoEZ3ceTMrrnOaPbrP1lN3BJB5Nujo0Dzo+IT2dXVQb9Gd3P\n", + "AG8h4s78IRue0T3HGd1mnSPrRB5J44CfAm8Afk16vPs9Ef1rrmU/kWem1D0NeqbCxNWw7oXw5Yvg\n", + "9aSM7pOAr5Y8o3s6KaP7upzxipCOAJvWk05UX70OVi3xEWBmY9OME3nmAL+IiIfqF/wG8HagEjfN\n", + "Zkrdc+HsZfDswbFr4A0Pwrenw3Qi8pY2xATSo/QfAc4DFkTwh6wxa5pM+sVyFGm9+7DojT/ljFmE\n", + "1Lznnj34EN4Fu0gzcRM3a4zcBv4SYGBO9MPAPpljNs006BnYvAGmwLgPwaRr85v3waSljZ8B+xSc\n", + "0X0LsGf0xsM5YxZrWs/g5g3pdfdCwA3crAFyG/iIlgEkLR7wsi8i+jKvW4hdYJuh3p8Ck8Y6Zj2j\n", + "+yxgJo3J6P5A9Jbjz2+wqROHfn/KmP8szTqJpC6gazTfyW3gvwamDXg9jTQLHyQiFmdep1jSVsCi\n", + "T8Orh/rxGlg7+iGZDJwIHAN8Fjg8gnVZZda0DWm3ygdI+SWfj954OmfMxlk9zP/WNaP+szTrRPWJ\n", + "bd+G15J6N/ed3Efpfwi8XNLOSk3x3cC3M8dsrJTR/SOg6+PwzwsYvLQxH1asTA/BjHA4JHEoaRfI\n", + "DGB2BJ/Kad6qSarp/aR7CduRMrqXlLd5A6xaAgs2WiaavwJWjvjP0sxGJ2sXCoCkQ+jfRviliDh1\n", + "o5+XYxfK4IzuRURcDelG5k6wcApMWgNrV8LS+2JkB/dKTCctbbwEWNiAjO5jqhTzmm5k7rQwLZus\n", + "WQsrl/oGptnYjKR3ZjfwIopocAGTgBOAHtIvms8Qkbe0IaYCHyPll3wSOCeC9Vlj1rQdaZnknaT8\n", + "kguiN+9pTzOrrmZsIyyvwRndd5IyuvOOH0sZ3e8lHT/WiIzuS3FGt5mNUHs28P6M7pcCC4j4Xv6Q\n", + "zCKtjTuj28xKob3ywAdndC8nZXRnNe96Rvc5wPU4o9vMSqQ9ZuCDM7pvxBndZtYBqt/AndFtZh2q\n", + "ug28cRndpwKH4IxuMyu56jXwwRnd36K4jO6jSFsDndFtZpVQrQbemIzuA0lLG87oNrNKqUYDl15I\n", + "mnEXmdG9I+mm536kWfJlBWd095Qho9vM2le5txFK45GOBn4CrCZldH8lp3lLTJA4EbgXWEFaLrk0\n", + "p3mrpsmq6RTgNtIWxj3cvM2s0co7A5f2Jy2XPAkcRET+0kZjMroPJWWs3ErpMrrNrJ2Vr4FLOwCn\n", + "AwcAHwYuJTOwpcEZ3X8FHFHOjG4za2flWUKRtkI6gRT1+itgBhH/ntO8JSZL1Eixt7eTskuymrdq\n", + "2kY1nQH8J3AVMNvN28xaoRwz8JTRvQT4JTCXiJ/nDYeAvyXNum8nZXTnBVml5ZL3kYKsvkvK6H4s\n", + "Z0wzsxytbeCDM7qPBa4uYLlkYEb3vAIzupeSjlp7R5Uyus2sfbVmCUWahHQyKeb1LmB3Iq7KXC6Z\n", + "KnE6cDNwHbBnbvNWTduppnNJM+6LgH3cvM2sLMY8A5d0GLCYtOd574jYfF7I4Izuu0gZ3b8aaw1p\n", + "yIZldG8IsnJGt5mVUs4Syo9JW+i+MKJP92d07wz8IxE3ZFy7PuSzGd1b0+YZ3em4smk96fT31etg\n", + "1RIfV2bW2cbcwCPiQdgwqd6MlNE9nxQUtZSIP4/1umk4tiWdYHMY6dT28yPIC7Kqaft6fW8mnS5/\n", + "SVlCp1Lznns2LNu1/90Fu0gzcRM361zNuom5A8VndF9BMRnd44GjKXVG97Sewc0b0uvuhYAbuFmH\n", + "2mQDl7Sc9KDKxj4aEVeN9CJK2wP/iTRb74sY/b7pARnd6ykuo7uLtATzKKXO6J46cej3p0xqbh1m\n", + "1iiSuoCu0Xxnkw08It6YUc/AcRaP9bsbZXSfSMrozg2dqlhG9+p1Q7+/Zm1z6zCzRqlPbPs2vJbU\n", + "u7nvFLWNcAQL4aMcUIyX6CEFWf0OmB7BVzNDpyaoppOAe4CfArtFb1xR7uYNsGoJLNgot2X+Cli5\n", + "tDX1mFkZaKxbryUdSnpg5gXAU8DdEXHIEJ+LiBhVgx+Q0f040BPB/WMqcuCYNR1C2gXzAHBc1TK6\n", + "043MnRamZZM1a2HlUt/ANGtfI+mdY27gRRbR/9lnM7pfQ8rovryA5ZK/Ju07nwEscsyrmVXBSHpn\n", + "KcKsBmR03wP8grS7JOuAhQEZ3beTol6d0W1mbaXlYVYbZXTv64xuM7ORaVkD3yije1EE12SP6Yxu\n", + "M+sgTV9CkZgksRi4g/6M7qzmXc/o/gzO6DazDtK0GfiAjO4z6c/oXpU1pjO6zayDNaWBS7yStLSx\n", + "I3BkgRnd5wATcUa3mXWgpmwjhPgN8Eng3AjWZ41X03akIKt3kvJLLojeyAqyMjMrm5FsI2zWEsoe\n", + "BWV0H0lq3s7oNrOO15QGXkDzLmVGt5lZK7V8H/im1DO6TwPeRMkyus3MWq2UDVw1bQkcRakzus3M\n", + "Wqt0DVw1HUQKsnqEUmd0m5m1VmkaePUyus3MWqvlDVw1TSA17OOBc4F50Rt/bG1VZmbl19IGrpq6\n", + "SUFW9wNzqpbRbWbWSmNu4JLOAN4K/BlYAXwwIp4a0Xdr2oUUZDUd6HHMq5nZ6OWEWV0P7B4Rs0hR\n", + "sP+yuS8MyOi+DbiFEmV01w8ULb0q1FmFGsF1Fs11Nt+YG3hELI+IZ+ovbyPlnAxJNUk1vYN0nNnL\n", + "SRndp0Vv/Gms12+ArlYXMEJdrS5gBLpaXcAIdbW6gBHqanUBI9TV6gJGqKvVBRSlqDXwecDXN/Hz\n", + "60kZ3R9wzKuZWTE22cAlLSc13o19NCKuqn/mX4E/R8TXNjHU1cDnozeygqzMzKxfVhqhpCOABcDr\n", + "I2LdMJ/xXm4zszFoWBqhpIOBE4ADh2veIynAzMzGZswzcEk/B7YCNkS63hoRRxVVmJmZbVrDD3Qw\n", + "M7PGaMqhxpLOkPSApHslXSHpec247mhIOkzSTyT9RdJera5nY5IOlvSgpJ9LOrHV9QxF0gWSHpP0\n", + "41bXsimSpkn6fv3/7/sk9bS6pqFImijpNkn3SLpf0qmtrmk4ksZJulvSVa2uZVMkPSTpR/Vab291\n", + "PUOR9HxJl9V75v2S9h3us806lX7UD/20wI+BQ0kn25eKpHGkAy0OBnYD3iNpRmurGtKFpBrLbj1w\n", + "XETsTgpPO7qMf571e0sHRcSewKuAgyS9tsVlDWcRKRKj7P9JH0BXRMyOiDmtLmYYZwPXRsQM0v/v\n", + "wyayNqWBj+ahn1aJiAcj4metrmMYc4BfRMRDEbEe+Abw9hbX9BwRcRPwX62uY3Mi4tGIdKpTRKwh\n", + "/QuyQ2urGlrEs8FuWwHj6L/nVBqSdgS6gfOBKmxaKG2N9dWJ/SPiAoCIeHpTESXNmoEPNA+4tgXX\n", + "rbKXAKuEvdvRAAACK0lEQVQGvH64/p5lkrQzMJs0sSgdSVtIugd4DPh+RNzf6pqGcBZpR9ozm/tg\n", + "CQRwg6QfSlrQ6mKG8DLgCUkXSrpL0jJJk4f7cGENXNJyST8e4q+/GfCZkTz00zAjqbGkyv6fpZUk\n", + "aQpwGbCoPhMvnYh4pr6EsiNwQNlyPCS9FXg8Iu6mxDPbAfaLiNnAIaSls/1bXdBGxgN7AZ+PiL2A\n", + "PwAnberDhYiIN27q5/WHfrqB1xd1zdHaXI0l9mtg2oDX00izcBsjSVsClwMXR8S3Wl3P5kTEU5Ku\n", + "AV4N9LW4nIFeA7xNUjcwEdhG0kUR8Q8trmtIEfFI/e9PSLqStDx5U2urGuRh4OGIuKP++jI20cCb\n", + "tQtlw0M/b9/UQz8lUraZxA+Bl0vaWdJWwLuBb7e4psqSJOBLwP0R8blW1zMcSS+Q9Pz6P08C3gjc\n", + "3dqqBouIj0bEtIh4GXA4cGNZm7ekyZKm1v95a9Jh6aXaMRURjwKrJL2i/tYbgJ8M9/lmrYEvBaYA\n", + "y+vbdz7fpOuOmKRDJa0i7Uq4RlIpYm4h3cgAjgG+S7rT/82I8p0VKunrpJjgV0haJemDra5pGPsB\n", + "7yft6ri7/lcZd8+8GLixvgZ+G3BVRHyvxTVtTpmX+7YHbhrw53l1RFzf4pqGshC4RNK9pF0onxru\n", + "g36Qx8ysolqxC8XMzArgBm5mVlFu4GZmFeUGbmZWUW7gZmYV5QZuZlZRbuBmZhXlBm5mVlH/DYXM\n", + "z2nC5qRuAAAAAElFTkSuQmCC\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "def plotSVM(X,t,wb):\n", + " plt.plot(X[t>0,0],X[t>0,1], 'ro')\n", + " plt.plot(X[t<0,0],X[t<0,1], 'bo')\n", + " plt.axis([-2, 6, -2, 7])\n", + " x = np.arange(-2,6,1)\n", + " y0 = -1*wb[0]/wb[1]*x-wb[2]/wb[1]\n", + " y1 = -1*wb[0]/wb[1]*x-(wb[2]+1)/wb[1]\n", + " y_1 = -1*wb[0]/wb[1]*x-(wb[2]-1)/wb[1]\n", + " plt.plot(x,y0)\n", + " plt.plot(x,y1)\n", + " plt.plot(x,y_1)\n", + " \n", + "plotSVM(X_,t_,wb_opt) ### PROVIDE THIS PLOT ###" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## New Data Set\n", + "\n", + "You are now given the following new data set." + ] + }, + { + "cell_type": "code", + "execution_count": 18, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "# Generating the data\n", + "N_half = 20\n", + "u1 = random.rand(N_half)\n", + "x1 = cos(2*pi*u1) + .3*random.rand(N_half)\n", + "y1 = sin(2*pi*u1) + .3*random.rand(N_half)\n", + "u2 = random.rand(N_half)\n", + "x2 = 2.*cos(2*pi*u2) + .3*random.rand(N_half)\n", + "y2 = 2.*sin(2*pi*u2) + .3*random.rand(N_half)\n", + "\n", + "# New Data\n", + "X_c = transpose(vstack((hstack((x1,x2)),hstack((y1,y2)))))\n", + "t_c = hstack((ones(N_half),-ones(N_half)))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Remark that the data is distributed on two circles, centered on zero, with different radii." + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[-3, 3, -3, 3]" + ] + }, + "execution_count": 20, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAErlJREFUeJzt3V+MXOV9xvHnsR3HRHaTJly4gFdWDW2aLoJESZSoLayU\n", + "ENiJKsJFHCG1aoK8F41qc5E0DbjA9g8iLapKlt5ElFASRHJBEAKx3tRCTBMkcBOCExybBFtF3tAU\n", + "RGmRLXBTwq8XM1tm7ZnZmXPOzDnvme9HivCcOTPzTjDPvOd33j+OCAEA0rOu7AYAALIhwAEgUQQ4\n", + "ACSKAAeARBHgAJAoAhwAEpUrwG1vsn3A9kHbh23fUlTDAAD9Oe84cNtvi4hXbW+Q9Jikz0fEY4W0\n", + "DgDQU+4SSkS82v7jRknrJb2c9z0BAGvLHeC219k+KOkFSY9GxOH8zQIArKWIHvgbEXGxpPMkXWJ7\n", + "JnerAABr2lDUG0XEK7YflvR+Sc2V47ZZbAUAMogI93s+V4DbPlvS6xHx37bPknSZpL8YthEpsz0f\n", + "EfNlt2NU6vz96vzdJL5f6gbp/Obtgf+apLttr1OrHPP1iHgk53sCAAaQK8Aj4mlJ7yuoLQCAITAT\n", + "M79m2Q0YsWbZDRihZtkNGLFm2Q0YsWbZDShb7ok8a36AHXWugQPAKAySnfTAASBRBDgAJIoAB4BE\n", + "EeAAkCgCHAASRYADQKIIcABIFAEOAIkiwAEgUQQ4ACSKAAeARBHgAJAoAhwAEkWAA0CiCHAASBQB\n", + "DgCJIsABIFEEOAAkigAHgEQR4ACQKAIcABK1oewGAGuxpxvStj3Slk3SiVPS8kLEocWy2wWUjQBH\n", + "pbXC+8Nflu44/82jczvsaRHimHSUUFBx2/asDm+p9XhqdzntAaqDAEfFbdnU/fjms8bbDqB6CHBU\n", + "3IlT3Y+ffG287QCqJ1eA295m+1HbP7Z9yPaeohoGtCwvSHNHVx/bdUw6fns57enNnm7Ys0v2zmbr\n", + "n9ONstuEenNEZH+xvVXS1og4aHuzpCclfSIijnScExHh/E3FpGoF4dTuVtnk5GvS8durdgOzx83W\n", + "o9Lj11atrUjDINmZK8C7fOADkm6PiEeGaQSQOnt2Sdp3+ZnPNJYiFmfH3yKkbpDsLGwYoe3tkt4r\n", + "6UBR7wmkYzQ3WxkDj34KCfB2+eQ+SddGxMki3hNIS/E3WxkDj7XkDnDbb5H0LUn3RMQDPc6Z73jY\n", + "jIhm3s8FqmV5QZrbsTps895s7TUGvrFbEgFeM7ZnJM0M85pcAW7bku6UdDgibut1XkTM5/kcoOoi\n", + "Di3a02qFa1E3WxkDP0naHdvmymPbN631mrw98N+R9AeSfmT7qfax6yJiKef7Aslph3WBPWPGwKO/\n", + "XAEeEY+JyUDAiIyiLIM6KXQYYdcPYBghkFkKY+AxGmMfB561EQCA1QbJTsofAJAoAhwAEkWAA0Ci\n", + "CHAASBQBDgCJIsABIFEEOAAkigAHgEQVth44qoe1pIF6I8BrirWkgfqjhFJbvdaSntpdTnsAFI0e\n", + "eAKylUJYSxqoOwK84rKXQlhLGqg7SigVZU83Wjudb78nWylkeUGaO7r6GGtJA3VCD7yCVve653uc\n", + "1b8UMpotvgBUCQFeSZ03IF/vcc7apZDit/gCUCWUUCqp8wbkxyTtPe15SiEA6IFXVOcNyEva/7xB\n", + "0uGXpdf+lVIIAIkt1Sqp+8iTXcekJ/YQ3PlN241t0p4t0qYT0qllaeFQBP+/olIGyU564BXEDcjR\n", + "mbYbH5a+fIf0/z+Oc9KOaVuEOFJDDxwTZdZe2iddfvrxhrS0GDFbRpuAbtjUGDjNFqnrDNXNEjNU\n", + "kRxKKJgoJ6SuM1RPSl2HZVIvR5UR4KitbuEraWFO2tFZA98lHTsunTEsk3o5qo4aOGqpR/gefVy6\n", + "VpKmpN2bpbNOSq8dl27vFsh56uVZeu709tGJUSiYWNukPZ3hLUl3SOc3pN3t8F0zGLPWy7P03Ont\n", + "IwtuYqKWirhZOWy9fEWvH48pqecCZFleM203Zu2lnXZz1l6athv92oX6yd0Dt/1VSR+X9GJEXJi/\n", + "SUB+WcO30/IQ9fJOa/14dCuVvGfIHxx67JCKKaHcpdZf6K8V8F7AGbLUhrOGb6dDEYvTthoD1Ms7\n", + "9fvx6BW8P5Fe6fWabsf7lYjEAmYTI3eAR8R3bW/P3xTgTFl7mlnDt9v7aMhA7Pfj0St4PyY9OScd\n", + "HfQHZ9LHs7Nhdws3MVFpeXqaWcK3CP1+PHbaX+j2mndIJx+Xbhz0B6eIElGq2LD7TWMJcNvzHQ+b\n", + "EdEcx+cifan2NHv9ePQL3mF+cIooEaWr14bdjaTLR7ZnJM0M85qxBHhEzI/jc1A/dZs5WVTwFlUi\n", + "SlM9N+xud2ybK49t37TWayihoNKGCbwURmYUGbxllYjKx4bdK3LPxLT9DUmXSnqXpBcl3RgRd3U8\n", + "z0xM5DJtN0Y9cxLpmJT18scyEzMirs77HkA/g/Y0U62XYzisl/8mSiiojUkemTFp2LC7han0KFWR\n", + "08Hb9fKjnccmZ2QGJhE9cJSm6JuOkz0yA5OI5WSRSRHD9bjpCPTGcrIYiaJ6ztx0BPKhBo6hZVn6\n", + "tBtuOgL5EOAYWlE9Z246AvlQQsHQiuo5c9MRyIebmBhatxr4LunYE9IewhcoxiDZSYAjk0GntwPI\n", + "hgAHgEQNkp3cxASARHETswBs7wSgDAR4TmzvBKAs1MBzsmeXpH1nTAeXGksRi0wHB0qS+pUxU+nH\n", + "op7bOwEpm5QrY25i5sb2TkD19Nr4eGqo5R6qjgDPbXlBmju6+tiuY9JxpoMDpZmMK2NKKDmxvRNQ\n", + "RZNxZcxNTAC1U4eNj5mJCWBitUJ8KtkrYwJ8jFIfsgSgWhhGOCZ1HrLUb+u0IrZVA5AdAV6IXkOW\n", + "GrslJRto/bZOk6QiNyQGMDyGERainkOW+m2dVtS2agCyowdeiHoOWcqydRobEqelWxlMav14Uxqr\n", + "PgK8EMsL0tyOM4cspT2Zp9/WaSF1vbnChsTpmLYbvyHd8R7pnA2SXpf0VukD66RT90vnrJxHaay6\n", + "CPAC1HUyT3vT4R2nb522sulwv+dQfe+U/vK3pHP+uuPYXumdL5x23h3S+Y1WaSzpv891lDvAbV8h\n", + "6TZJ6yX9Y0T8Te5WJagd1rX6C77WpsNsSJy2X5F+/ebTjt0s6eou51Iaq6ZcAW57vaR/kPRRSc9L\n", + "+p7tByPiSBGNQ/nagbxq2OBO+wvt8srCYgRL5ibqrT2Ob+xyjNJYNeXtgX9Q0tGIeE6SbH9T0pWS\n", + "CPCa6TekkF53mk5I/ybpV08//p/S/6gj3ymNVVfeYYTnSlruePyz9jHUSLvnffe50vl/Luk77eMM\n", + "G0zbv0s3XCP9vPPYZ6SfPyfd0pCWdkr/0pCWnpD28CNdTXl74APNw7c93/GwGRHNnJ+LMenoeZ+9\n", + "cmxv+5+XiNpoytr3OHZxH6MabM9ImhnmNXkD/HlJ2zoeb1OrF75KRMzn/ByUpNuEnZsl3aBWgFMb\n", + "TVvnPQ6Uq92xba48tn3TWq/JW0L5vqQLbG+3vVHSpyQ9mPM9USG9JvOsF7VRoGy5euAR8brtP5H0\n", + "bbX+m76TESj10msyzwHppWVqo0CpWE4WfXUbfbJLOsaNLZRpEpZvZjlZ5LbWZB5g3Oq8fPOw6IED\n", + "SIo9uyTtu/zMZxpLEYu1mVhGD7wiJuFyDxifei7fnAUBPmJc7gFFq+fyzVmwocPI9dqtZ4oZjEAm\n", + "ywvS3NHVx9JfvjkLeuAjx+UeUKS6Lt+cBQE+clzuAUWr4/LNWVBCGTku9zA+03Zj1l7aaTdn7aVp\n", + "u1F2mzA69MBHjMs9jAtL/k4exoEDNTFrL+2Tzhgf3ZCW2HgjPYNkJyUUoCZ6LTzGkr/1RQkFlbGy\n", + "ZdsWadMJ6dSytMCl/+B6LTzGkr/1RYCjEqjf5rcsLcxJO05feIwlf+uLGjgqgfptMabtxhQLj9UC\n", + "a6EgGdRvi8EOO5OFm5ioBOq3wPAIcFRCu367asIT9VugP2rgqAzqt8CbBslOAhwAKoiJPABQY4xC\n", + "KQm79ADIiwAvAbv0ACgCNfASTMqmrMAoTMrVKxN5KotdeoAsuHpdjZuYpWCXHiAb9pjtRICXgl16\n", + "gGy4eu1ECaUEa+3SMyk1PmB4XL12IsBL0mtTVmp8QD/LC9LcjtX/fUzu1WvmUSi2PylpXtK7JX0g\n", + "In7Q4zxGoQyBESpAf61OzlTt95gd9SiUpyVdJekrOd4DZ6DGB/TT6+p1EmUO8Ih4RpJsOtfFosYH\n", + "YDCMQqkcRqgAGEzfHrjt/ZK2dnnq+oh4aNAPsT3f8bAZEc1BXztp1hqhAqCebM9ImhnqNXmn0tt+\n", + "VNLnuIkJAMUZ53KyBDQAjFnmALd9le1lSR+S9LDtfcU1CwCwFlYjrBlmcQL1wGqEE2aYWZwEPZA+\n", + "ArxWeq3U1titjokPTNcH6oFx4LUy6CxOluQE6oAAr5VBZ3EyXR+oAwK8Vgadxcl0faAOqIHXyOCz\n", + "OFmSE6gDhhFOqElZkhNI1SDZSYADQAWNcyo9AGDMCHAASBQBDgCJIsABIFEEOAAkigAHgEQR4ACQ\n", + "KAIcABLFVHpkwnriQPkIcAyN9cSBamAqPYZmzy5J+y4/85nGUsTi7PhbtBpXB6gDtlTDiFR3PXGu\n", + "DjBJCHBkUOX1xAfbVm4cuBLAqBHgyKDK64lX4+qAKwGMAwGOoQ2+cUQZqnJ1UJ0rAdQXAY5M2mFd\n", + "wSCqytVBNa4EUG8EOGqlOlcHVbkSQJ0xjBAYge418F3HpCf2VKPUhKpjSzWgROw7ijwIcABI1Ej3\n", + "xLR9q+0jtn9o+37bb8/6XgCA4eVZjfCfJf12RFwk6aeSriumSQCAQWQO8IjYHxFvtB8ekHReMU0C\n", + "AAyiqPXAr1ElxwQDQH31HQdue7+krV2euj4iHmqfs1fSLyLi3hG0DwDQQ98Aj4jL+j1v+9OSGpI+\n", + "ssZ58x0PmxHRHKx5ADAZbM9ImhnqNVmHEdq+QtLfSbo0Il7qcx7DCAFgSCMdB277WUkbJb3cPvR4\n", + "RHw2SyMAAKsxkQcAEjXSiTwAgHIR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BE\n", + "EeAAkCgCHAASRYADQKIIcABIFAEOAIkiwAEgUQQ4ACSKAAeARBHgAJAoAhwAEkWAA0CiCHAASBQB\n", + "DgCJIsABIFEEOAAkigAHgEQR4ACQKAIcABJFgANAojIHuO2/sv1D2wdtP2J7W5ENAwD0l6cH/rcR\n", + "cVFEXCzpAUk3FdSmpNieKbsNo1Tn71fn7ybx/SZB5gCPiBMdDzdLeil/c5I0U3YDRmym7AaM0EzZ\n", + "DRixmbIbMGIzZTegbBvyvNj2zZL+UNKrkj5USIsAAAPp2wO3vd/2013+9/uSFBF7I2JK0j9J+vsx\n", + "tBcA0OaIyP8m9pSkxYiY7vJc/g8AgAkUEe73fOYSiu0LIuLZ9sMrJT2VpQEAgGwy98Bt3yfpNyX9\n", + "UtIxSX8cES8W2DYAQB+FlFAAAOM3lpmYdZ70Y/tW20fa3+9+228vu01Fsv1J2z+2/Uvb7yu7PUWx\n", + "fYXtZ2w/a/vPym5PkWx/1fYLtp8uuy2jYHub7Ufbfy8P2d5TdpuKYnuT7QPtrDxs+5a+54+jB257\n", + "y8q4cdu7JV0UEbtG/sFjYPsySY9ExBu2vyRJEfHFkptVGNvvlvSGpK9I+lxE/KDkJuVme72kn0j6\n", + "qKTnJX1P0tURcaTUhhXE9u9JOinpaxFxYdntKZrtrZK2RsRB25slPSnpEzX69/e2iHjV9gZJj0n6\n", + "fEQ81u3csfTA6zzpJyL2R8Qb7YcHJJ1XZnuKFhHPRMRPy25HwT4o6WhEPBcR/yvpm2rdiK+FiPiu\n", + "pP8qux2jEhH/EREH238+KemIpHPKbVVxIuLV9h83Slov6eVe545tMSvbN9s+LumPJH1pXJ87ZtdI\n", + "Wiy7EVjTuZKWOx7/rH0MibG9XdJ71eo81YLtdbYPSnpB0qMRcbjXublmYp72ofslbe3y1PUR8VBE\n", + "7JW01/YX1Zr085miPnvU1vpu7XP2SvpFRNw71sYVYJDvVzPcua+BdvnkPknXtnvitdC+or+4fT/t\n", + "27ZnIqLZ7dzCAjwiLhvw1HuVWC91re9m+9OSGpI+MpYGFWyIf3d18bykzhvp29TqhSMRtt8i6VuS\n", + "7omIB8puzyhExCu2H5b0fknNbueMaxTKBR0Pe076SZHtKyT9qaQrI+JU2e0ZsbpMyvq+pAtsb7e9\n", + "UdKnJD1YcpswINuWdKekwxFxW9ntKZLts22/o/3nsyRdpj55Oa5RKLWd9GP7WbVuNqzcaHg8Ij5b\n", + "YpMKZfsqSQuSzpb0iqSnImK23FblZ3tW0m1q3SS6MyL6DtdKie1vSLpU0rskvSjpxoi4q9xWFcf2\n", + "70r6jqQf6c1y2HURsVReq4ph+0JJd6vVuV4n6esRcWvP85nIAwBpYks1AEgUAQ4AiSLAASBRBDgA\n", + "JIoAB4BEEeAAkCgCHAASRYADQKL+D9ax1tF4g59sAAAAAElFTkSuQmCC\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "# Plot new data\n", + "plt.plot(X_c[t_c>0,0],X_c[t_c>0,1], 'ro')\n", + "plt.plot(X_c[t_c<0,0],X_c[t_c<0,1], 'bo')\n", + "plt.axis([-3, 3, -3, 3])" + ] + }, + { + "cell_type": "code", + "execution_count": 35, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "[-1, 6, -1, 6]" + ] + }, + "execution_count": 35, + "metadata": {}, + "output_type": "execute_result" + }, + { + "data": { + "image/png": [ + "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n", + "AAALEgAACxIB0t1+/AAAE+9JREFUeJzt3X+M5HV9x/HXizvxQA5oa2tF9oIeJdEOBoilUPyxpILs\n", + "RGtMWloTa2Jz+1d7e6GtqT9aOZuobZpGWIxNegrRlmibM7Yie6tYnRZSQCmH3nlYexcJi/IrpbV3\n", + "wiUg7/4xczq7N7M7M9/vfL/fz/f7fCTkdnZmvvP2sr72e5/P+/P5OCIEAEjPKWUXAACYDAEOAIki\n", + "wAEgUQQ4ACSKAAeARBHgAJCozAFu+2zbe20/aPuQ7cvyKAwAsL7NOVzjRklLEfGbtjdLelEO1wQA\n", + "bMBZFvLYPkvS/oh4RX4lAQBGkXUI5eWSnrR9i+37be+xfXoehQEA1pc1wDdLukTSxyPiEkk/kvSe\n", + "zFUBADaUdQz8EUmPRMQ3eo/3ak2A22azFQCYQER4veczBXhEPGZ7xfYFEfFdSW+U9O1xi6gy27sj\n", + "YnfZdUwq5fpTrl2i/rLVoP4Nb37z6ELZKelW26dKOiLpXTlcEwCwgcwBHhHflPQrOdQCABgDKzE3\n", + "1im7gIw6ZReQQafsAjLqlF1ARp2yC8ioU3YB05apD3ykD7Aj5TFwACjDKNnJHTgAJIoAB4BEEeAA\n", + "kCgCHAASRYADQKIIcABIFAEOAIkiwAEgUXnshYIKsFttaWZB2rpFOnpcWlmMOLhUdl0ApocAr4Fu\n", + "eF9+o7Tn/J9+d3673RIhDtQXQyi1MLOwOryl7uNtO8upB0ARCPBa2Lpl8PfPOK3YOgAUiQCvhaPH\n", + "B3//2DPF1gGgSAR4LawsSvOHV39vxxHp4ZvKqQdAEdhOtia6E5nbdnaHTY49Iz18ExOYQLpGyU4C\n", + "HAAqiP3AAaDGCHAASBQBDgCJYiVmDbGsHmgGArxmWFYPNEfmLhTbD0n6P0k/lvRsRFy65nm6UApk\n", + "zy1L+9508jPt5YilueIrAjCJUbIzjzvwkDQbEU/lcC1kxrJ6oCnymsTkDrsyWFYPNEUeAR6SvmL7\n", + "PtvzOVwPmaS3rN5ute25ZfvaTvfPVrvsmoAU5DGEckVEPGr75yXdYfs7EXFn/wts7+572ImITg6f\n", + "iwEiDi7ZLUntJJbVM+kKdNmelTQ71nvyXEpv+3pJxyLir/u+xyQmhmLSFRhs6kvpbZ9ue2vv6xdJ\n", + "ulrSgSzXRNMw6QpMKusQykskfd72iWvdGhFfzlwVGoRJV2BSmQI8Ir4n6aKcakEjrSxK89tXj4FX\n", + "e9IVqAq2k0Xp2MscOBn7gQNAotgPHABqjAAHgEQR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBR\n", + "BDgAJIoAB4BEEeAAkCgCHAASlceRakhMd/e/mYXuYQpHj0sri+z+B6SHAG8YzqAE6oMhlMaZWVgd\n", + "3lL38bad5dQDYFIEeONwBiVQFwyhTFn1xps5gxKoCwJ8iqo53swZlEBdcKTaFNlzy9K+N538THs5\n", + "Ymmu+Iq6OIMSqL5RspM78Kmq5nhzL6yTCuzqDUUB5SPAp4rx5jxUcygKKF8uXSi2N9neb/u2PK5X\n", + "HyuL0vzh1d9jvHl8tD4Cg+R1B75L0iFJW3O6Xi1EHFyyW5LajDdnUs2hKKBsmQPc9rmS2pI+JOkP\n", + "M1dUMymON1cPQ1HAIHkMoXxU0rslPZ/DtYABGIoCBsl0B277zZKeiIj9tmfXed3uvoediOhk+Vw0\n", + "C0NRaIJehs6O9Z4sfeC2PyzpdyU9J2mLpDMlfS4i3tn3msb2gQPApEbJztwW8th+g6Q/joi3jFsE\n", + "0BT0s2NUZSzkme6yTiBh9LMjbyylBwpS1a0VUE2jZCfbyQKFoZ8d+SLAgcLQz458EeBAYehnR74Y\n", + "AwcKxFa+GFWhbYRZigAwfbQwpoX9wAFIooWxrhgDBxqBLXnriAAHGoEWxjpiCAVohPVbGBkfTxMB\n", + "DjTCyqI0v331MEq3hZHx8XTRhQI0xLAWRpb4VxNdKAB+YvjpUIyPp4pJTKDxWOKfKgIcaDyW+KeK\n", + "MXAALPGvIJbSA0Ci2A8cAGqMAAeARBHgAJAoAhwAEsVCniFadntGWtgqbTkqHV+RFg9GMCsPoDII\n", + "8AFadvty6cY90k/2hpiXtrdsEeIAqoIhlAFmpIX+8JakPdL52yT2TgZQGZnuwG1vkfSvkl4o6VRJ\n", + "/xwR782jsDJtlQbuDXGGxN4QYOtVVEamAI+I47avjIinbW+WdJft10bEXTnVV4qj0sC9IY5J7A3R\n", + "cGy9iirJPIQSEU/3vjxV0iZJT2W9ZtlWpMV5adXeEDukIw9L7A3ReNM9msxute25ZfvaTvfPVjuP\n", + "66KeMk9i2j5F0v2Stkv6m4g4lLmqkh2MWGrZaks7z5BOOyY987B0ExOYmObWq9zdY1yZAzwinpd0\n", + "ke2zJH3J9mxEdPpfY3t338PO2uerqBfW/J8Ga0xz69Vhd/ftneJnsfZsz0qaHec9ubURRsQPbd8u\n", + "6TWSOmue253X56SO/vLUDT+aLPu1OVihyXo3tp0Tj21fv9F7snahvFjScxHxv7ZPk3SVpA9muWad\n", + "0V+evu4RZC1174rz3nqVgxUwnkzbydq+UNKn1J0MPUXS30XEX615TaO2k13vDnvOXt4nnXT2YFta\n", + "Xorg7MGGGzwGvuOIdM8CY+DNM/UzMSPigKRLslyjTja6wy6zv5yhm+qb7t096oil9DkatoKz3V3B\n", + "uVRWf3kZQzf8wpjM8IOHgZMR4Dna6A6711++vT9Ii+gv3+gXS96fx1g/UAwCPEcb3WGX1V9e9NBN\n", + "0b8wgKYiwHM0yh12Gf3lRQ/dsJcMUAwCPEdVXcFZ9NANe8kAxeBU+oZo2e1tE/5iGXdCctAY+A7p\n", + "yD3SwjR+mTFhijoaJTsJcKxryITk4bulXRuF+KS/MIqoD6g6ArxBpnUXWvXFR1WvD5jU1BfyoBqm\n", + "2bZX9QnJqtcHTBNHqtXANI+Aq/qEZNXrA6aJAK+Bad6FVv1wi6rXh/I04XAMhlBqYJp3oVVtjTyh\n", + "6vWhHE05HINJzBooum0PqDp7blnad9LkttRejlhKYnKbScyG4C4UWKsZh2MQ4DXBEXBAv2YcjsEk\n", + "JgBJdZv0W1mU5g+v/l5eR99VB3fgBWG5N6qsbpN+TTkcg0nMArDcG1VXh0m/uhklOxlCKcA0F9oA\n", + "+WjGpF/dEOAFYLk3qq8Zk351Q4AXgOXeqL5mTPrVDZOYBSjrLExgVE2Z9KsbJjELUtT+2ADqYer7\n", + "gduekfRpSb8gKST9bUQsjlsEAOSp2xY5s9CdnD16XFpZTO1fE0UspX9W0nUR8YDtMyT9h+07IuLB\n", + "jNcFgInUrad9PZkmMSPisYh4oPf1MUkPSjonj8IAYDIzC6vDW+o+3la7tt3culBsnyfpYkn35nVN\n", + "ABhfc3rac+lC6Q2f7JW0q3cnvvb53X0POxHRyeNzMR0s+0fa0uxptz0raXas92TtQrH9AklflLQv\n", + "Im4Y8DyTmAlh2T8mUaVJw8Fj4DuOSPcspDQGXkQXiiV9StJ/R8R1kxaB6uCUd4xryKThYenuXeWG\n", + "+Lake9qL6EK5QtI7JH3L9v7e994bEcsZr4uSsOwf4xs2adjeqZL2qO+FdVKBPYlMAR4Rd4nl+LXC\n", + "sn+MrzmThlVD+GKVqp/y3rLbc/bytXZnzl5u2QkfOlAXaU4a1gF7oWCVKp+vOWSCdXvLVhXqa66V\n", + "RWl++8mThmyENW3shYJkMMFaXXWYNKwaTqVHrTDBWl1NmTSsGsbAkQwmWIHVCHAko+oTrEDRGANH\n", + "UthXHU0x9ZWYeRUBAFiNU+kBoMYIcABIFAEOAIkiwAEgUQQ4ACSKAAeARBHgAJAo9kJpKM69BNJH\n", + "gDcQ27IC9cAQSgPNSAv94S1Je6Tzt0k7y6oJwPgI8AZiW1agHhhCaSC2ZUXKuodHzCx0z+I8elxa\n", + "WWzq4REEeAP1tmXd3j+MwrasSEE3vC+/cfXxbfPb7ZaaGOLsRthQbMuKFNlzy9K+k47Vk9rLEUu1\n", + "OlaPI9UwVC+sCWwkZuvA+ZvuWZzNk3kS0/bNth+3fSCPggBguKMD52+6Byk3Tx5dKLdIuiaH6wBT\n", + "1bLbc/bytXZnzl5u2e2ya8K4Vhal+cOrv7fjiPTwVOdv7Fbbnlu2r+10/2xV4mcn8xBKRNxp+7zs\n", + "pQDTw+Kleog4uGS3JLV3dodNjj0jPXzTNCcwqzxxmsskZi/Ab4uICwc8xyQmSjdnL++TTpr8akvL\n", + "SxG1mvxCvsqaOK3MJKbt3X0POxHRKeJzgRNYvITJFTNxantW0uw47ykkwCNidxGfAwzD4iVMrpiJ\n", + "096NbefEY9vXb/QeltKjEXqLl1ZNfrF4CaMpZ+J0FJnHwG1/RtIbJP2cpCckfSAibul7njFwVAKL\n", + "lzCp7kTmtsImTrufuXF2shITACpolOxkCAUAEkWAA0Ci2AsFQGHYCjZfBDiAQlR5RWOqGEIBUJCZ\n", + "hdXhLXUfb+MovwkR4AAKwlaweWMIBUBBsq9oZAx9NQIcQEFWFqX57auHUUZf0cgY+slYyIMktOz2\n", + "jLSwVdpyVDq+Ii2Ouooyy3uRrywrGpt0nJpUod0IgSyy7OXNPuDV0gvrCf/exxtDb8JwCwGOypuR\n", + "FvoDWJL2SOe3pZ3aIAyyvBdVM/oYelOGW+hCQeVl2cubfcDrZJxdAZvRssgdOCovy17e7ANeH+Md\n", + "p9aMlkUCHJXX28t7e/9QyKh7eWd5L6pn9DH0ZpxeTxcKkpBlL+8q7gNOZ8x0DR4D33FEumchlTFw\n", + "9gMHKmhIZ8zhu6VdhHh+yjiEIU8EOFCwlt3+WenPz5Re8UJJR6Xv/UD6s/5gnrOX90kn9TO3peWl\n", + "iNr1M2My9IEDBWrZ7QukPa+UzvnQT7/9M78nfaJl7zgR4nTGIC+0EQI5mZEWXrU6vCVJN0sv3dbt\n", + "O5dEZwzyQ4ADOdkqbRn2T9r+u+teZ8yqfmY6YzAJhlBQO2V1eByVjj835Ln+u+uDEUstW+2KdcYg\n", + "PQQ4aqXMvU9WpMXTpAvfv2YY5V3So2vvrnu1ENjIhC4U1ErZHR69LpQPrulC+QB31xhXIV0otq+R\n", + "dIOkTZI+ERF/mfWawDj6h0xOl1496DVFdXhwZ40iZQpw25skfUzSGyV9X9I3bH8hIh7MozhgI2uH\n", + "TP50yOvo8EAdZe1CuVTS4Yh4KCKelfRZSW/NXhYwmrXbxV4t6f1rXkOHB+oq6xDKyySt9D1+RNKv\n", + "ZrwmMLK1i2Je3/vzzdJTp0sH6PBAnWUN8JFmQG3v7nvYiYhOxs8FJA1eFPN6SX8hff0fWZaOhNie\n", + "lTQ7znuyBvj3Jc30PZ5R9y58lYjYnfFzgIHYLhZ10bux7Zx4bPv6jd6TqY3Q9mZJ/ynp1yX9QNLX\n", + "Jb29fxKTNkJMWxW3iwWyKmQ3Qttz+mkb4Scj4iPjFgEAWI3tZAEgUaNkJ5tZAUCiCHAASBQBDgCJ\n", + "IsABIFEEOAAkigAHgEQR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAAkCgC\n", + "HAASRYADQKIIcABIFAEOAIkiwAEgUQQ4ACSKAAeARE0c4LZ/y/a3bf/Y9iV5FgUA2FiWO/ADkt4m\n", + "6d9yqqWSbM+WXUMWKdefcu0S9Zct9fpHMXGAR8R3IuK7eRZTUbNlF5DRbNkFZDBbdgEZzZZdQEaz\n", + "ZReQ0WzZBUwbY+AAkKjN6z1p+w5JvzjgqfdFxG3TKQkAMApHRLYL2F+T9EcRcf+Q57N9AAA0VER4\n", + "vefXvQMfw9AP2agAAMBksrQRvs32iqTLJN1ue19+ZQEANpJ5CAUAUI5CulBSXPRj+xrb37H9X7b/\n", + "pOx6xmH7ZtuP2z5Qdi2TsD1j+2u9n5mDthfKrmkctrfYvtf2A7YP2f5I2TWNy/Ym2/ttJ9msYPsh\n", + "29/q/W/4etn1jMP22bb32n6w9/Nz2bDXFtVGmNSiH9ubJH1M0jWSXiXp7bZfWW5VY7lF3dpT9ayk\n", + "6yLil9Udovv9lP7+I+K4pCsj4iJJr5Z0pe3XllzWuHZJOiQp1X+ih6TZiLg4Ii4tu5gx3ShpKSJe\n", + "qe7Pz4PDXlhIgCe46OdSSYcj4qGIeFbSZyW9teSaRhYRd0r6n7LrmFREPBYRD/S+PqbuD/A55VY1\n", + "noh4uvflqZI2SXqqxHLGYvtcSW1Jn9A6DQoJSK5222dJel1E3CxJEfFcRPxw2OtZyDPYyySt9D1+\n", + "pPc9FMz2eZIulnRvuZWMx/Ypth+Q9Likr0XEobJrGsNHJb1b0vNlF5JBSPqK7ftsz5ddzBheLulJ\n", + "27fYvt/2HtunD3txbgFu+w7bBwb895a8PqNAqf6zsVZsnyFpr6RdvTvxZETE870hlHMlvT6VfTls\n", + "v1nSExGxXwnewfa5IiIuljSn7hDc68ouaESbJV0i6eMRcYmkH0l6z3ovzkVEXJXXtSrg+5Jm+h7P\n", + "qHsXjoLYfoGkz0n6+4j4p7LrmVRE/ND27ZJeI6lTcjmj+DVJv2G7LWmLpDNtfzoi3llyXWOJiEd7\n", + "fz5p+/PqDoveWW5VI3lE0iMR8Y3e471aJ8DLGEJJ4bf6fZJ+yfZ5tk+V9NuSvlByTY1h25I+KelQ\n", + "RNxQdj3jsv1i22f3vj5N0lWS9pdb1Wgi4n0RMRMRL5f0O5K+mlp42z7d9tbe1y+SdLW6jRSVFxGP\n", + "SVqxfUHvW2+U9O1hry+qjTCpRT8R8ZykP5D0JXVn4v8hIobOBFeN7c9I+ndJF9hesf2usmsa0xWS\n", + "3qFu98b+3n8pddW8VNJXe2Pg90q6LSL+peSaJpXicOJLJN3Z9/f/xYj4csk1jWOnpFttf1PdLpQP\n", + "D3shC3kAIFF0oQBAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAAkCgCHAAS9f/kGFG7S57jjwAA\n", + "AABJRU5ErkJggg==\n" + ], + "text/plain": [ + "" + ] + }, + "metadata": {}, + "output_type": "display_data" + } + ], + "source": [ + "X_transformed = 2.5*transpose(vstack((hstack((np.abs(x1),np.abs(x2))),hstack((np.abs(y1),np.abs(y2)))))) ### CHANGE THIS LINE ####\n", + "### prbly other transormation\n", + "plt.plot(X_transformed[t_c>0,0],X_transformed[t_c>0,1], 'ro')\n", + "plt.plot(X_transformed[t_c<0,0],X_transformed[t_c<0,1], 'bo')\n", + "plt.axis([-1, 6, -1, 6])" + ] + }, + { + "cell_type": "code", + "execution_count": 36, + "metadata": { + "collapsed": false + }, + "outputs": [ + { + "data": { + "text/plain": [ + "(40, 2)" + ] + }, + "execution_count": 36, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "shape(X_transformed)\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "##Plotting the result in the transformed space. \n", + "(Lines are commented, because code does not compile, until X has been correctly transformed.)" + ] + }, + { + "cell_type": "code", + "execution_count": 37, + "metadata": { + "collapsed": false, + "scrolled": true + }, + "outputs": [ + { + "ename": "ValueError", + "evalue": "all the input array dimensions except for the concatenation axis must match exactly", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mValueError\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mwb_opt_c\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0moptimizeSVM\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mX_transformed\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mt_c\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 2\u001b[0m \u001b[1;31m#plotSVM(X_transformed,t_c,wb_opt_c) ### PROVIDE THIS PLOT ###s\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32m\u001b[0m in \u001b[0;36moptimizeSVM\u001b[1;34m(X_, t_)\u001b[0m\n\u001b[0;32m 13\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 14\u001b[0m opt_result = optimize.minimize(objective, wb0, jac=jac,constraints=cons, #the optimizer\n\u001b[1;32m---> 15\u001b[1;33m method='SLSQP', options=opt)\n\u001b[0m\u001b[0;32m 16\u001b[0m \u001b[0mwb_opt\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mopt_result\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m\"x\"\u001b[0m\u001b[1;33m]\u001b[0m \u001b[1;31m#result of the optimization\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 17\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;32m/opt/anaconda3/lib/python3.4/site-packages/scipy/optimize/_minimize.py\u001b[0m in \u001b[0;36mminimize\u001b[1;34m(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)\u001b[0m\n\u001b[0;32m 433\u001b[0m \u001b[1;32melif\u001b[0m \u001b[0mmeth\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'slsqp'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 434\u001b[0m return _minimize_slsqp(fun, x0, args, jac, bounds,\n\u001b[1;32m--> 435\u001b[1;33m constraints, callback=callback, **options)\n\u001b[0m\u001b[0;32m 436\u001b[0m \u001b[1;32melif\u001b[0m \u001b[0mmeth\u001b[0m \u001b[1;33m==\u001b[0m \u001b[1;34m'dogleg'\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 437\u001b[0m return _minimize_dogleg(fun, x0, args, jac, hess,\n", + "\u001b[1;32m/opt/anaconda3/lib/python3.4/site-packages/scipy/optimize/slsqp.py\u001b[0m in \u001b[0;36m_minimize_slsqp\u001b[1;34m(func, x0, args, jac, bounds, constraints, maxiter, ftol, iprint, disp, eps, callback, **unknown_options)\u001b[0m\n\u001b[0;32m 402\u001b[0m \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 403\u001b[0m \u001b[0ma\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mvstack\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma_eq\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0ma_ieq\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 404\u001b[1;33m \u001b[0ma\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mconcatenate\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0ma\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mzeros\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m[\u001b[0m\u001b[0mla\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m 405\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 406\u001b[0m \u001b[1;31m# Call SLSQP\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n", + "\u001b[1;31mValueError\u001b[0m: all the input array dimensions except for the concatenation axis must match exactly" + ] + } + ], + "source": [ + "#wb_opt_c = optimizeSVM(X_transformed,t_c)\n", + "#plotSVM(X_transformed,t_c,wb_opt_c) ### PROVIDE THIS PLOT ###s" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Feel free (not mandatory!) to rewrite the function plotSVM to plot the data and the separation curve, not in the transformed space (as above), but in the original space." + ] + } + ], + "metadata": { + "annotations": { + "author": "", + "categories": [ + "intelligent-systems-1-2015" + ], + "date": "2015-04-23", + "location": "Beginning of next lecture", + "parent": "IS_SS2015", + "submission_date": "2015-04-30", + "subtitle": "Exercise Sheet 2, Support Vector Machines", + "tags": [ + "IntelligenSystems", + "Course" + ], + "title": "Intelligent Systems 1 - Summer Semester 2015" + }, + "celltoolbar": "Edit Metadata", + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.4.3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +} diff --git a/is/UB2/ISUB2.pdf b/is/UB2/ISUB2.pdf new file mode 100644 index 0000000..5488770 --- /dev/null +++ b/is/UB2/ISUB2.pdf Binary files differ diff --git a/is/UB2/ISUB2.tex b/is/UB2/ISUB2.tex new file mode 100644 index 0000000..9c2eaed --- /dev/null +++ b/is/UB2/ISUB2.tex @@ -0,0 +1,109 @@ +\documentclass[a4paper,12pt]{scrartcl} +\usepackage[ngerman]{babel} +\usepackage{graphicx} %BIlder einbinden +\usepackage{amsmath} %erweiterte Mathe-Zeichen +\usepackage{amsfonts} %weitere fonts +\usepackage[utf8]{inputenc} %Umlaute & Co +\usepackage{hyperref} %Links +\usepackage{ifthen} %ifthenelse +\usepackage{enumerate} +\usepackage{listings} +\lstset{language=Python} + +\usepackage{algpseudocode} %Pseudocode +\usepackage{dsfont} % schöne Zahlenräumezeichen +\usepackage{amssymb, amsthm} %noch stärker erweiterte Mathe-Zeichen +\usepackage{tikz} %TikZ ist kein Zeichenprogramm +\usetikzlibrary{trees,automata,arrows,shapes} + +\pagestyle{empty} + + +\topmargin-50pt + +\newcounter{aufgabe} +\def\tand{&} + +\newcommand{\makeTableLine}[2][0]{% + \setcounter{aufgabe}{1}% + \whiledo{\value{aufgabe} < #1}% + {% + #2\tand\stepcounter{aufgabe}% + } +} + +\newcommand{\aufgTable}[1]{ + \def\spalten{\numexpr #1 + 1 \relax} + \begin{tabular}{|*{\spalten}{p{1cm}|}} + \makeTableLine[\spalten]{A\theaufgabe}$\Sigma$~~\\ \hline + \rule{0pt}{15pt}\makeTableLine[\spalten]{}\\ + \end{tabular} +} + +\def\header#1#2#3#4#5#6#7{\pagestyle{empty} +\begin{minipage}[t]{0.47\textwidth} +\begin{flushleft} +{\bf #4}\\ +#5 +\end{flushleft} +\end{minipage} +\begin{minipage}[t]{0.5\textwidth} +\begin{flushright} +#6 \vspace{0.5cm}\\ +% Number of Columns Definition of Columns second empty line +% \begin{tabular}{|*{5}{C{1cm}|}}\hline A1&A2&A3&A4&$\Sigma$\\\hline&&&&\\\hline\end{tabular}\\\vspace*{0.1cm} +\aufgTable{#7} +\end{flushright} +\end{minipage} +\vspace{1cm} +\begin{center} +{\Large\bf Sheet #1} + +{(Hand in #3)} +\end{center} +} + + + +%counts the exercisenumber +\newcounter{n} + +%Kommando für Aufgaben +%\Aufgabe{AufgTitel}{Punktezahl} +\newcommand{\Aufgabe}[2]{\stepcounter{n} + \textbf{Exercise \arabic{n}: #1} (#2 Points)\\} + + + + +\begin{document} + %\header{BlattNr}{Tutor}{Abgabedatum}{Vorlesungsname}{Namen}{Semester}{Anzahl Aufgaben} + \header{1}{}{2015-04-22}{Intelligent Systems I}{\textit{Maximus Mutschler}\\ \textit{Jan-Peter Hohloch} + }{SS 15}{4} + \vspace{1cm} + \Aufgabe{Monster vs. mouse}{50} + Assume: + $P(M=0)=0.7,P(M=1)=0.2,P(M=2)=0.1;P(m)=0.2, P(e)=0.5$, mice are also afraid of monsters, so if there is a monster under the bed there is no mouse.\\ + $P(n|M=1)=0.7,P(n|M=2)=0.8,P(n|M=0)=0.1, P(n|m)=0.2, P(n|e)=0.1$\\ + $\Rightarrow P(n)=0.36$\\ + $\hat{M}^{MAP}=\arg\max\limits_M P(M|n=1)$\\ + $P(M=0|n=1)=\frac{P(n|M=0)\cdot P(M=0)}{P(n)}=\frac{0.1\cdot 0.7}{0.36}\approx 0.194\\ + P(M=1|n=1)=\frac{P(n|M=1)\cdot P(M=1)}{P(n)}=\frac{0.7\cdot 0.2}{0.36}\approx 0.389\\ + P(M=2|n=1)=\frac{P(n|M=2)\cdot P(M=2)}{P(n)}=\frac{0.8\cdot 0.1}{0.36}\approx 0.222$\\ + $\hat{M}^{MAP}=1$, so I estimate there is one monster under my bed.\\\\ + \Aufgabe{SVM}{50} + Changed lines:\\ + \begin{lstlisting} + def objective(wb): + return 1/2*(wb[0]*wb[0]+wb[1]*wb[1]) + \end{lstlisting} + \begin{lstlisting} + def ineq(wb): + return t_* + (list(map(lambda x:np.dot(wb[0:2].T,x),X_)) + +wb[2])-1 + \end{lstlisting} + \includegraphics[width=.4\textwidth]{plot1.png} + %TODO: wtf? +\end{document} + diff --git a/is/UB2/plot1.png b/is/UB2/plot1.png new file mode 100644 index 0000000..78120b5 --- /dev/null +++ b/is/UB2/plot1.png Binary files differ diff --git a/mb/Uebung1.pdf b/mb/Uebung1.pdf deleted file mode 100644 index 9b89efb..0000000 --- a/mb/Uebung1.pdf +++ /dev/null Binary files differ diff --git a/mb/Uebung1.tex b/mb/Uebung1.tex index f58006d..5dfc6ea 100644 --- a/mb/Uebung1.tex +++ b/mb/Uebung1.tex @@ -149,9 +149,9 @@ 0&0&0&1 \end{pmatrix}\\ &=\begin{pmatrix} - 0&-1&0&-1\\ - 0&0&1&-1\\ - -1&0&0&2\\ + 0&0&-1&2\\ + -1&0&0&-1\\ + 0&1&0&1\\ 0&0&0&1 \end{pmatrix} \end{align*}