{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"preamble": true
},
"source": [
"(Defining latex commands: not to be shown...)\n",
"$$\n",
"\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
"\\DeclareMathOperator{\\minimize}{minimize}\n",
"\\newcommand{\\real}{\\mathbb{R}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Monster vs. Mouse (50 Points)\n",
"\n",
"While trying to sleep, you hear some noise under your bed. Being a big Science Fiction fan you obviously immediately think it's a monster. A BIG monster... or maybe two?!? Frightened to death, you do not dare to move, but your brain is racing. And it recalls that the other day, you saw a mouse disappearing on the other side of the bedroom... So maybe it is (just?!?) a mouse. Or, just something else... Hmmm. Remembering your Emperical Inference classes, you decide to evaluate the probability that, given some noise under your bed, there is a monster.\n",
"\n",
"You define the random variables:\n",
"\n",
" + n = some noise under your bed (Values 0 or 1)\n",
" + M = 0, 1 or 2 monsters under your bed (Values 0, 1 or 2)\n",
" + m = a mouse under your bed (Values 0 or 1)\n",
" + e = something else (e.g. only air) under your bed (Values 0 or 1),\n",
"\n",
"and express your beliefs about monsters, mice and noise, by assigning numbers to P(M), P(m), P(e), P(n|M), P(n|m), P(n|e). Given that you heard some noise under your bed, you then calculate the Maximum a Posteriori (MAP) of M (i.e., the maximum of P(M|n=1)).\n",
"\n",
"Please share your beliefs and your results with us."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Programming a Hand-Featured SVM (50 points)\n",
"\n",
"The goal of this exercise is to implement a very simple SVM using an off-the-shelf optimizer, in the case of a 2-dimensional input space.\n",
"Most of it has already been implented in an iPython Notebook that you can find on the homepage of the course, and that we reproduce hereafter.\n",
"\n",
"You are asked to hand out a print out of the lines marked \"### CHANGE THIS LINE ###\", as well as your plots."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The SVM equation in the linear separable case\n",
"\n",
"Let $(x_1,t_1), \\ldots, (x_N,t_N) \\in \\real^p \\times \\{-1,1\\}$ be N data points with their binary labels. For this exercise, $p=2$. \n",
"In the lecture, we saw that trying to maximize the margin between two linearly separable classes of data points led to the problem:\n",
"\\begin{equation}\n",
" \\minimize_{\\substack{ \\{ w \\ \\in \\real^2,\\ b \\ \\in \\ \\real \\} }} \\frac{1}{2} \\norm{w}^2 \\quad \\mathrm{subject \\ to} \\quad t_n(w^T x_n + b) \\geq 1 \\quad n = 1 \\ldots N ,\n",
"\\end{equation}\n",
"whereby the points are subsequently classified according to the rule:\n",
"\\begin{equation}\n",
" t = \\left \\{ \n",
" \\begin{array}{cc}\n",
" \\ \\ 1 \\quad \\mathrm{if} \\quad w^T x + b \\geq 0 \\\\\n",
" -1 \\quad \\mathrm{if} \\quad w^T x + b < 0\n",
" \\end{array}\n",
" \\right . .\n",
"\\end{equation}\n",
"The function $\\mathop{l}(w,b) := \\frac{1}{2} \\norm{w}^2$ is called the _objective_ function. The equations $t_n(w^T x_n + b) \\geq 1$ are called the (_inequality_) _constraints_.\n",
"\n",
"In practice, one often preferres to transform this problem into its so-called _dual_ formulation. However, in this exercise, we will stick to the above equations: the _primal_ formulation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##Loading the required packages."
]
},
{
"cell_type": "code",
"execution_count": 53,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"import math as m\n",
"from numpy import *\n",
"from scipy import optimize\n",
"from matplotlib import pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definition of the data point \n",
"\n",
"_X = location of the data points_ $\\in \\real^{N \\times 2}$ \n",
"_t = label class of each point_ $\\in \\real^N$"
]
},
{
"cell_type": "code",
"execution_count": 54,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"X_ = np.array([[1.,3.],[2.,6.],[2.,3.],[-1.,0],\n",
" [1.,0.],[2.,2.],[3.,1.],[0.,-1.]])\n",
"t_ = np.array([1,1,1,1,-1,-1,-1,-1])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definition of the Objective Function and its Gradient\n",
"\n",
"_wb = $(w_1, w_2, b) = (w , b)$_ $\\in \\real^2 \\times \\real$\n",
"\n",
"(Note that the gradient of the objective function (here called 'jac') does not appear in the equations of an SVM. But our off-the-shelf optimizer uses it for its computations.)"
]
},
{
"cell_type": "code",
"execution_count": 55,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def objective(wb):\n",
" ### changed ###\n",
" return 1/2*(wb[0]*wb[0]+wb[1]*wb[1])\n",
"\n",
"def jac(wb):\n",
" return hstack((wb[0:2],array([0])))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definition of the Inequality Constraints and of their Gradient\n",
"\n",
"(Again, the gradient does not appear in the SVM equations, but it is needed for our optimizer.)"
]
},
{
"cell_type": "code",
"execution_count": 94,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def ineq(wb):\n",
" #writing \"return a-b\" would encode the constraint a-b >= 0\n",
" ### changed ###\n",
" return t_*(dot(wb[0:2],X_.T)+wb[2])-1\n",
"\n",
"def grad(wb):\n",
" return -1*hstack(((-t*X.T).T, -t[:,newaxis]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Optimisation of the SVM"
]
},
{
"cell_type": "code",
"execution_count": 95,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [],
"source": [
"def optimizeSVM(X_,t_):\n",
" global X,t\n",
" X = X_\n",
" t = t_\n",
" wb0 = np.random.randn(3) #initialization\n",
" #wb0 = array([-1.,-2.,0.2])\n",
"\n",
" cons = {'type':'ineq', #constraints\n",
" 'fun': ineq,\n",
" 'jac': grad}\n",
"\n",
" opt = {'disp':False}\n",
"\n",
" opt_result = optimize.minimize(objective, wb0, jac=jac,constraints=cons, #the optimizer\n",
" method='SLSQP', options=opt)\n",
" wb_opt = opt_result[\"x\"] #result of the optimization\n",
" \n",
" return wb_opt\n",
"\n",
"wb_opt = optimizeSVM(X_,t_) #the optimized parameters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plotting the Result"
]
},
{
"cell_type": "code",
"execution_count": 96,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8lVX1x/HPl0lQcDZHcADNAaeSeRDFiesETuGEmmCW\n",
"glrO08NjhZoz5IhjzmapmZZpSqmZWmqZ2q8gDTSn1AzMufX7Y13wYhd49j3n3jPc9X69eOU596yz\n",
"t6TLzXr22ltmRgghhNrTodITCCGE0DKRwEMIoUZFAg8hhBoVCTyEEGpUJPAQQqhRkcBDCKFGlZTA\n",
"JX1R0tNNfr0raVK5JhdCCGHRVK594JI6AK8A/c1sTlm+NIQQwiKVs4SyHTArkncIIbSNcibwscBN\n",
"Zfy+EEIIi1GWEoqkLnj5ZGMze7PkLwwhhLBEncr0PaOA3zeXvCXFYSshhNACZqbF/bxcCXxf4OaW\n",
"TqIaSJpsZpMrPY8lqYV51sIcIeZZblU1T68KXACMBHbH7P8++1EVzXMxiix+S07gkpbBH2BOKPW7\n",
"QgihZNKqwA+BfwEDMHu3wjNqNSU/xDSz98xsZTObW44JhRBCi0lbAU8CDwGj6zl5Q/lKKPVgRqUn\n",
"UNCMSk+ggBmVnkBBMyo9gYJmVHoCBc2o6OjSOOA84DDM7ljMJ2e0zYRaX9kaeRY5gGS1UAMPIdQo\n",
"qRNwDrALvup+rsIzKosiuTNW4CGE2iWtDNwKfAT0x+ydCs+oTcVhViGE2iRtgde7nwR2aW/JG2IF\n",
"HkKoRdJYYBpwJGa3Vno6lRIJPIRQO6SOwBRgH2A7zP6QFs4gYJYZb7TG9NpalFBCCLVBWgG4B9gK\n",
"6NeC5P014E5g/VaYXUVEAg8hVD9pE+AJ4HlgR8z+WTyULhKXA0cBQ814tJVm2eaihBJCqG7SHsDl\n",
"wDcxuz4tlNWB24E3gAFm1FXDYazAQwjVSeqAdAZ+psmoFiTv/viq/T5gz3pL3hAr8BBCNZKWA24A\n",
"lsP3d7+eFs4hwNnAeDN+0gozrAqxAg8hVBdpQ+Bx4O/4TpPCyVuis8RU4ERgeD0nb4gVeAihmki7\n",
"AlcDJ2F2ZVooq+CnEM4D+ptR1wdZQazAQwjVwOvdpwGXAru1IHl/Ce/IfATYrT0kb4gVeAih0qQe\n",
"wLXAGni9+x9p4ewPXAh83Yzbyz/B6hUJPIRQOVIfvLnmMWA/zD4sHkon4CxgDLCtGc+2ziSrV8kl\n",
"FEnLS7pd0guSnpc0sBwTCyHUOWlH4FHg+/gZ3inJeyXgZ8BmQL/2mLyhPCvwi4B7zWwv+bm8y5Th\n",
"O0NoNX2lhp4wqQd0nQsfzIGpfzK7t9LzajckAccBRwN7YfZwWjib4av224GTzfikcGyuLsDHlrXy\n",
"RQhtpKQELt+rOczMDgIws0+gfTw8CLWpr9QwCC6aDn3mvzcBeveViCTeBvwO3auA3vh9lXPSwtkb\n",
"uASYZLboi9Sbjc21BvBjvOxyZ0pstSq1hLIu8KakayQ9JWm6pKXLMbEQWkNPmNQ0eQNMhz69YGKl\n",
"5tRuSOvgJZMPgeEpyVuio8QU/OadHVqQvAfhu1TuBu5Kia1mpZZQOgFfAo40syclXYhvoD+96Yck\n",
"TW7ycoaZzShx3BBapAd0be797tCtrefSrkjbAjcBZwJTSbjLUWL5xthueL37zaShc43Hj6A9xDK7\n",
"JyW2LUkaAYxIiSk1gb8MvGxmTza+vh1P4Asxs8kljhNCWcyFD5p7fx6839ZzaRe83n0Unhf2w+zB\n",
"tHA2xssd9wLHmfFx4Vivd18IbAMMs8z+L2Xstta4sJ0x/7WkbEkxJZVQzOw1YI6kDRrf2g6oiwtF\n",
"Q32aA1MnwMym742HWbP9dpdQTlI3fH/3wcDAFiTv0cCvgClmHJ2YvFcFfgmsCQyo9uTdUiXfSi9p\n",
"c+BKoAswCzjEzN5t8vO4lT5Ulb5SQy+Y2B26zYP3Z8O0eIBZZlJP/IHhTOBQzP5TPJQOeBn2UPwU\n",
"wSeShs7VD/gR3pJ/hmX235T4alEkd5acwMsxiRBCHZGG4TfFXwCcm1jvXha4HlgJ2MuM15KGzjUO\n",
"OBc4zDKr6Z0mRXJndGKGEMrD691fBzJgHGb3pYWzAb5DZAawtxkfFY7N1RnfobIzsI1l1i5KuZHA\n",
"Qwilk5YCLgYGAkMwm7mEiM+FszNwDXCKGdOTYnOtDNyGb0/sb5m9kxJfy+I0whBCaaQ18FXzCsCg\n",
"lOQtIYmTgSuA0S1I3lvg+7sfB3ZpT8kbYgUeQiiFNAjfPnwpMAUr/sBQoju+6u6J7+9OO4Uw11h8\n",
"99ARltltKbH1IhJ4CKFltKBB5lDM7k4LpTe+v/sJYGszih9klasj3hC0F7CdZfaHlLHrSSTwEEIa\n",
"qQu+w2QkMAxL22MtsT1+3+UZwCVmFN+lkmtF4GagI9DPMnsrZex6EzXwEEJx0qrAA3jZY0BK8m6s\n",
"dx8L/ADYx4yLE5P3pviK/Vlgp/aevCFW4CGEoqSt8Oaca4A8sd69NDAd2BAYYMbspKFz7QlcBhxj\n",
"md2QElvPIoGHEJZMGgech1+8cEdaKGsDd+DHbAw1K37ujHJ1wEstB+Kr7t+njF3vIoGHEBbNL2k5\n",
"B9gFGIGlNchIjMBr1t8DLkwsmSwH3Aj0wOvdb6SM3R5EDTyE0DxpZeA+vOzRPyV5N9a7JwG3AAea\n",
"cUFi8t4Qr3e/iO80ieTdjFiBhxD+l7QFXva4FTgFs0+Lh9IVr1dvCQwy48WkoXPtit/ac6JldnVK\n",
"bHsTCTyEsDAtaJA5ErNb00JZC3/Q+SIw2Iz3Csd6vfsU4DBgV8vs8ZSx26NI4CEEJ3XEG3P2AbbD\n",
"0hpkJIbiZ5JcBHwvsWTSA7gOWA0/z+TVlLHbq0jgIQTQggaZTkA/zP6ZFs7XgG8DB5nxs6TYXH3w\n",
"UwgfBfa1zAp3ZbZ3kcBDaO+kvnhb+13ACZh9UjyUpYCpwFBgiBl/TRo61074yvt0y+zylNhQhgQu\n",
"6SXg38CnwMdm1r/U7wwhtBFpD+By4BgsrUFGYnX8IKs3gIFmzC0cm0vA8cAkYE/L7JGUsYMrxwrc\n",
"gBFm9nYZviuE0BakDkAOjAN2wtIaZCQG4Mn7CuC7ZhTvysy1DH7d2br4fZUvp4wdPlOuEkpcmRZC\n",
"rZCWww+TWhavdyftsZY4BDgbGG/GT5Jic62Ll2ueBoZbZh+kxIeFlaORx4AHJP1O0oQyfF8IobVI\n",
"G+KXH/wd32lSOHlLdJaYBpwIDG9B8h4JPIZfgn5IJO/SlWMFPsTMXpW0CnC/pD+b2cNNPyBpcpOX\n",
"M8xsRhnGDSGk0IIGmZMwuyotlC8APwTmAv3NeLdwrNe7j8Zr3mMti3//myNpBDAiKaact9JLyoB5\n",
"ZnZek/fiVvoQKkkLGmS+BuyF2W/Twvky3pxzPXB6Yr27G14n3wQYY5n9PWXs9qxI7iyphCJpaUk9\n",
"Gv96GWAH/KzeEEI18H8/fwiMwuvdqcn7AODnwLfMODUxefcCHsEvXxgaybv8Si2hrArcIWn+d91o\n",
"Zr8oeVYhhNJJffAHhr8B9sOKN8hIdMIfVI4GtjVLW5gp13D8HJVzgfMtK+Mf9cMCZS2hNDtAlFBC\n",
"aHvSjvjNN6djaQ0yEivhyfdTYF8zCm8Rbqx3fwM4HTjQsljQtVSR3BmdmCHUE//j8HHAUcCeWFqD\n",
"jMTm+CmEPwRONqP4KYS5lgIuAfoBgy2zWSljh3SRwEOoF/4c6iqgN35fZVKDjMQ+wMXARDNuSYrN\n",
"tQb+oHMOnrznpcSHlokLHUKoB9K6eK37A/ym+MLJW6KjxJl4zXv7FiTvQfjlCz8B9onk3XZiBR5C\n",
"rZNG4lePTQGmkfBgS2IF4CZgKaCfGWmnEOYaD3wXb8y5NyU2lC4SeAi1yuvdR+Gdkfti9lBaOJvg\n",
"u1TuAY4z4+PCsbm6ABfijSfDLLO/pIwdyiMSeAi1SOqGnyK4KTAQs5fSwhkNTAeONeO6pNhcq+IH\n",
"Wb0NDLTM/p0SH8onauAh1BqpJ/AwvgAbkpK8JTpI5PgZ3g0tSN79gCeBX+KdlZG8KyhW4CHUEmk4\n",
"ftP7+cB5ifXuZfF2+BXxevfrSUPnOghvzJlgmd2ZEhtaRyTwEGqB17u/DmTAgSR2PEtsgN+48xCw\n",
"txkfFY7N1RlP3KOArS2z51PGDq0nEngI1U5aCt+fPQAYjKU1yEjsDFyDN+ZcmRSbaxW8K/MD/LLh\n",
"f6XEh9YVNfAQqpm0BjADWB4YlJK8JSRxCv6wc/cWJO8t8Xr3b4FdI3lXn1iBh1CtpEF4S/ulwJTE\n",
"end34FpgLfz87n8kDZ1rX/xB5xGW2W0psaHtRAIPoRpJ4/HGnEMwuyctlN74/u4ngP3NKH4KYa6O\n",
"wJnAXsBIy+yPKWOHthUJPIRqogUNMtvgLfH/lxbODvhOkxy41Iziq/ZcKwI346XVfpbZWyljh7YX\n",
"NfAQqoW0Kr6/ek38MKrCybux3n0sXjbZ24xLEpP3pviK/VlgVCTv2hAr8BCqgdQP+BFwNXAGZsVv\n",
"vhFL4xcFbwAMMGNO0tC59sLr7EdbZjemxIbKKksCl9QR+B3wspntWo7vDKHdkMbh+6wPw9IaZCTW\n",
"xuvdzwLDzHi/cKzXu88ADgB2tMyeShk7VF65VuBHAc8DPcr0fSHUP6kzcA6wM7ANZs+lhbMNfpLg\n",
"WcDUxJLJ8sANQHe83v1GytihOpRcA5e0FtCA/xEurk4LoQhpZeA+4ItA/5Tk3VjvnoQ/cDzAjIsS\n",
"k/dGwOPA34DtI3nXrnKswC/Ar3BatgzfFUL9k7bAry27BTgVs+LXlomuwGXAFsAgM15MGjrXbvhi\n",
"6wTL7JqU2FB9SkrgknYB3jCzpyWNWMznJjd5OcPMZpQybgg1SxoLTAOOwNIaZCTWwq8t+xswxIz3\n",
"Csfm6gCcCkzAuyofTxk7tL7GHDoiKaaUW+klTQEOBD4BuuKr8B+Z2bgmn4lb6UPQQg0yYzD7Q1o4\n",
"Q4Hb8D3i5ySWTHrgN9R/AdjTMnstZexQGUVyZ0kJ/HODbQ0c+/ldKJHAQ7unBQ0yHYGvYGl7rCUO\n",
"xxtzDjLj50mxudbHd6k8Aky0zAqfQhgqq0juLPc+8PL81yCEeiFtite77wROxOyT4qEshZdbhuAl\n",
"k5lJQ+caBVwHnGaZXZ4SG2pD2VbgixwgVuChvZL2xB84HoPZDWmhrI439ryGr7znFo7NJeAEYBJ+\n",
"S/wjKWOH6lCJFXgIQeqAN8gcCOyE2e/TwhmI3zl5GTDFjOJdmbmWwc/+Xhs/v/vllLFDbYkEHkI5\n",
"ScsBN+JNbf2wtD3WEl/FG3MONePupNhc6+Glmt/jN+d8kBIfak8cZhVCuUgb4gdCvQhsl5K8JTpL\n",
"fB84HhjeguS9HfAb4Argq5G824dYgYdQDtKuwFX4g8qr00L5An5xw7/xw6jeLRzr9e5j8Ga6sZZF\n",
"j0V7Egk8hFJ4vfsU4DBgVyytQUbiy3hzznXA5MR6dzdgOrAxMNAy+3vK2KH2RQIPoaW0oEFmVfw8\n",
"k1fTwjkAOB843IwfJ8Xm6oVvT/wzMNQy+09KfKgPUQMPoSWk9fHLft/ETxIsnLwlOkmcD0wGtm1B\n",
"8t4aP4zqJuCASN7tV6zAQ0ilzxpksLQGGYmVgVuBj4F+ZrxTONbr3UcAp+GJ+/6UsUP9iQQeQlFa\n",
"qEFmDyytQUZic7zscRtwihnFTyHM1RW4BNgKGGSZ/S1l7FCfIoGHUIS0DH7d2Xp4vTupQUbiK8D3\n",
"gYlm3JIUm2tN/EHnbGCwZTYvJT7Ur6iBh7Ak0rr4Huv38ZviCydviY4SZ+HNOdu3IHkPxveW34m3\n",
"xUfyDgvECjyExZFG4p2VU4BpJBweJLECfgphF7ze/c+koXMdBnwHONgyuzclNrQPkcBDaI7Xu4/G\n",
"a977YvZQWjib4KvmnwLHmVH8FMJcXYCpwHB8i+BfUsYO7Uck8BA+T+qGt6T3BQZi9lJaOGMa479l\n",
"xg+SYnOthh9k9U+8OeffKfGhfYkaeAhNSb3wyw86AkNSkrdEB4kcuAgY1YLk3R94Ergf2COSd1iS\n",
"Uu/E7Ar8ClgKr/PdZWYnlWNiIbQ5v1XqFuA84LzEevdywPXACni9+/WkoXMdDJwDjLfM7kqJDe1X\n",
"SQnczD6QtI2Z/UdSJ+ARSUMtcX9sCBWlhRpkDsTsF2nhfBG4C/glsJcZha8tU67OwLnAKPwI2OdT\n",
"xg7tW8k1cLMFbbxd8D92vl3qd4ba1Fdq6AmTekDXufDBHJj6J6u+3RNN5/kRfHQB2LqwBjAYs1kp\n",
"3yWxC74//GQzrkyKzbUK3tTzPn75wr9S4kMoOYHLT2N7CugNXGoWK4j2qK/UMAgumg595r83AXr3\n",
"laimJN7cPP8I8xrgoHsTkrdEB+Bk4HBgdzMeS5mHcm2Jd2XeCJxumRXuygxhvpIfYprZf81sC2At\n",
"YLikESXPKtScnjCpaVIEmA59esHESs2pOc3NczPoDkwo+h0SPfDzu3cB+rcgee8H/AI4zjI7JZJ3\n",
"aKmybSM0s3cl3YOf1TCj6c8kTW7ycoZZHDpfb3pA1+be7w7d2noui1PqPCV64/Xux4H9zPiw6NjK\n",
"1QnvyBwDjLTM/lg0NtS/xsXviJSYUnehrAx8Ymb/ku+d3R7IP/85M5tcyjih+s2FZq/wmuf13eog\n",
"dZnoZ5n8jyLzlNgRP/87By41o/gulVwrwYI2+v6W2VtFY0P70LiwnTH/taRsSTGlllBWBx6U9Ay+\n",
"IrnbzH5Z4neGGjQHpk6AmU3fGw+zZsO0Ss1pIdJqwIN94JVJsNBJfkuap4QkjgOuBfY245LE5L0Z\n",
"fp7JH4BRkbxDuShhq2vLBpDMzNSqg4Sq0Fdq6AUTu0O3efD+bJhWFQ8wpX74aX5XAWf0hZ2KzlNi\n",
"6ca49YExZsxJGjrXXsClwNGW2Y2l/Y2E9qRI7owEHuqbdBC+z3oCZnemhbIOvlPkWeBrZsXLQcrV\n",
"ETgD2B/vqnwqZewQiuTOOAsl1CctaJBpAEZg9lxaONviV5adCUxNLJksj28PXBroZ5m9mTJ2CEXF\n",
"WSih/kirAPcBG+CXLxRO3o317qPx5L2/GRclJu+N8Hr3TGCHSN6hNcUKPNQXLWiQuRk4FSu+x1qi\n",
"G3AZsDkwyIwXk4bOtTswHTjeMrs2JTaElogEHuqHtC9+jvYRmN2WFkpP/EHnLGCIGe8Vjs3VAT9H\n",
"ZTywi2X2RMrYIbRUJPBQ+/wgtTOBPYGRWFqDjMQw/Kb4C4FzEksmy+J7w1fB692vpYwdQikigYfa\n",
"Jq2IN8h0APphxfdYSwg/y2QycJAZP08aOtcG+K07v8bvqyx8CmEI5RAJPNQuaVM8gd4BnIhZ8WvL\n",
"xFL4LfGD8JLJzCWELByfqwFv7DnNMrs8JTaEcokEHmqTFjTIHIPZDWmhrAH8CHgVf1g5t3BsLgEn\n",
"AkcCYyyzR1PGDqGcIoGH2qIFDTIHADth9vu0cAbid05eBkwx47+FY3N1B64BeuHnmbySMnYI5RYJ\n",
"PNQOaXngBqAHXu9+Iy2c8cAU4FAz7k6KzbUeXq75PX5zTrOHd4XQlqKRJ9QGaSP8wLQXge1SkrdE\n",
"F4mLgeOA4S1I3tsDj+E3zX81kneoFrECD9VP2g24EjgBs2vSQlkVv3zh3/jlC+8WjvV69zeBY4Gv\n",
"WBbn2IfqEgk8VC+/ru9U/LacXTF7PC2crfDmnOuALLHevTTeVbkRMMAym50ydghtIRJ4qE5SD7xB\n",
"5gt4vTupQUZiHHAeforgj5Nic62Nb018Hhhq2YKLu0OoKlEDD9VHWh/4LfAGsE1K8pboJHEBcDqw\n",
"TQuS94jGsW8ADozkHapZqVeq9eSzVZIBV5jZ1HJMLLRT0ii85HEaltYgI7Ey3hL/MdDPjHcKx3q9\n",
"+wi8ZHOAZfZAytghVEJJFzrIr6lazcyekdQd32I12sxeaPKZuNAhLJkk4ARgErAPZo+khbMFXva4\n",
"FTjFjOKnEObqClyCX8g92jL72xJCQmh1rX6hg/kfbV9r/Ot5kl4A1gBeWGxgCE1JywBXA+vi53e/\n",
"nBbOWPxOy4lmCy4OXszn+zZAz0nQoysrvmaM77g6S3/6R2CQZVb4FMIQKq1sDzElrQNsie/VDaEY\n",
"aV28QeZpYDhWfI+1REe8MWcfYHsznllyTN8GGHQRTO9Dz0dh733gic3e4lcfXmv/fS6Sd6gpZXmI\n",
"2Vg+uR04yszmleM7QzsgjcQbZK4EDklM3isA9wD98Hr3EpO36zkJpvfhy1fA2DFw93SY8dRK2NoT\n",
"0/8GQqisklfg8rsHfwTcYIu4NFbS5CYvZ5hFQ0S75vXuo4HjgbEk/vMgsQlwF3A3cJwZhU8hpOPS\n",
"3Rh1OKz9a7jqUXh7/cYfdO+WMocQyk3SCGBEUkyJDzGF7xh4y8yOWcRn4iFm+IzUDW9J3wQYg9nf\n",
"08LZA7gc+KYZ1yfF5lqd17s/zzsjl+eOH8CHyzb5acPPze4dlfJ9IbSmIrmz1BLKEPxUuG0kPd34\n",
"a6cSvzPUK6kX8Aj+z93QlOQt0UHiDOACYFQLknd/4Ak+/PQebl155sLJe/wsmD0t5ftCqAYlrcAL\n",
"DRAr8AAgDce3+J0LnE/CP3gSy+GNNcsBe5vxetLQuQ4BzgYmWGZ3+YPMXhO9bDLvfZg9zexP96Z8\n",
"ZwitrUjujAQeWpcWNMicBhyA2f1p4XwRr3c/gJdNCl9bplyd8Xb6HfH93bG9NdSMVt8HHsJiSV2B\n",
"i/GdIoOwtAYZiV3w/eEnm3FlUmyuVfBTCN/DD6P6V0p8CLUgzkIJrUNaE5gBLAsMTknejfXuU/Fb\n",
"c3ZvQfL+EvAkXm/fLZJ3qFexAg/lJw3GV7/fB85KrHf3wC8LXgM/v/sfSUPn2h+4EPi6ZXZ7SmwI\n",
"tSYSeCgvaQLwXeBgzJIeDEr0xuvdvwX2M+PDwrG5OgFnAWOAbS2zZ1PGDqEWRQIP5SF1AS4Ctsa3\n",
"CP4lLZwd8ZMtJwOXmVF81Z5rJeAW/ETMfpbZ2yljh1CrogYeSuenUj4IrA4MTEneEpI4Hr/tfS8z\n",
"Lk1M3psBTwDPAA2RvEN7EivwUBqpH36UwlXAtzErfm2ZWKYxrjcwwIw5SUPn2hs/BvYoy+ymlNgQ\n",
"6kEk8NBy0kHAOcAEzO5KC2Ud/BTCP+A3xb9fODZXR+DbwH7AjpbZUyljh1AvIoGHdH6A2bnAKGAE\n",
"Zs+nhbMtcBNwJjA1sWSyfGNsN7ze/WbK2CHUk6iBhzTSKsB9wPr45QuFk3djvftoPAHvZ8ZFicl7\n",
"Y7ze/Vdgh0jeob2LFXgoTtoSv7bsRuB0zIpfWya64Y05mwMDzXgpaehco4HpwHGW2bUpsSHUq0jg\n",
"oRhpP3yb4Dcw+2FaKD2BHwMzgcFmFL7pXbk64DfMHwrsbJk9kTJ2CPUsEnhYPKkTXqveAxiJ2R/T\n",
"whmGn0J4AXBuYslkWeB6YCW83v1aytgh1LtI4GHRpBXxBhnh9e63ioci4HC8MWecGfclDZ1rA3yX\n",
"yq+AvS2zwqcQhtBeRAIPzZM2xRPoHcCJmBW+tkxiKfwclIF4yWRW0tC5GvDzUE61zK5IiQ2hPSnH\n",
"nZhXAzsDb5jZpqVPKVSctBdwKXA0ZjemhbIGfsH1q8AgMwpfcq1cAk4EjsTP7/5NytghtDclX+gg\n",
"aRgwD/hBcwk8LnSoIVJH4Az8mrwxWFqDjMRAPHlfBnw3sd7dHW+n7wXsYZm9kjJ2PfKbg3pOgh5d\n",
"Ye4HMGdq3BzUfrTJhQ5m9rCkdUr9nlBh0vL4tWXdgX6YvZEWzqH4w86vmvHTpNhc6+Hlmt8BW1tm\n",
"H6TE1yNP3oMugul9Pnt3Qm+pL5HEw3zRyBNA2ghvkPkbsH1K8pboInExcBwwrAXJe3vgMfym+UMj\n",
"ec/Xc9LCyRv8da+JlZlPqEZt8hBT0uQmL2eY2Yy2GDcUIO0OXAkcj9k1aaGsil/c8C/8MKp3C8d6\n",
"vftbjb/2scx+lTJ2/evRtfn3u3dr23mEtiJpBDAiJaZNEriZTW6LcUICqQN+0fB4YBfMHk8LZyu8\n",
"OecaIDej+CmEuZbGuyo3wu+rnJ0ydvswdxF/EplX+NCvUFsaF7Yz5r+WlC0pJkoo7ZG0LJ58d8Dr\n",
"3anJexzwM+AoM7LE5L02flelAUMjeS/KnKkwYebC742fBbOnVWY+oRqVYxfKzfgtLCsBbwCnW5M/\n",
"isculCqjBQ0yDwMTseINMhKd8ONjdwFGm/Fc0tC5RgA3N37HBZaV+A9fnfMHmb0metlk3vswe1o8\n",
"wGw/iuTOkhN4OSYR2ogWNMichtnlaaGsjLfEf4SfJPhO4Vivdx8JnArsb5k9kDJ2CO1Rm2wjDDVA\n",
"EnACMBHf3/1oWjhb4B2ZtwKnmFH8FMJcXfGmoC8Dgyyzv6WMHUJYtEjg9U5aBn/QuDZ+nklSg4zE\n",
"WGAacKQZtybF5loLr7W/iCfv91LiQwiLFw8x65m0Hr7H+j1g65TkLdFR4my8OWe7FiTvIcDj+H2Z\n",
"YyN5h1B+sQKvV9J2eGfld4CLSXjYIbEC/rCxM9DPjH8mDZ3ra/idlQdZZj9LiQ0hFBcJvN54vfub\n",
"wLHAWBKbpiT64rtUfgIcb0bxUwhzdcHLLcOAIZbZX1PGDiGkiQReT7RQg8xAzP6eFs4eeEv7N824\n",
"Pik21+r4QVZv4M05c1PiQwjpogZeL7SgQQZgaErylugg8W3gQmBUC5L3APwslfuAPSN5h9A2YgVe\n",
"D7SgQeZc4PzEevdyeK18Obze/XrS0LkOAb6HH0T1k5TYEEJpIoHXMq93H4GfaXIAZvenhbMhXu9+\n",
"ADjGjI8Lx+bqDJyPt+MPt8xeSBk7hFC6SOC1SuoKXAJsBQzC0hpkJHYFrgJOMuOqpNhcX8BPIZyH\n",
"17v/lRIfQiiPqIHXImlN/LLfHsDglOTdWO8+De+O3K0FyfvLwJP4WSq7RfIOoXJiBV5rpCHAbfil\n",
"wWcl1rt74GehrI7Xu19NGjrX/viDzq9bZrenxIYQyi8SeC2RDsMbcw7GLOlUOok+eL37Mfwwqg8L\n",
"x+bqBJwNjAa2tcyeTRk7hNA6IoHXAi1okBmObxH8S1o4OwHXARlweeJlwysBt+Dnd/ezzN5OGTuE\n",
"0HqiBl58SrLdAAAPrklEQVTtpNWBh4DVgAEpyVtCEicAVwN7mXFZYvLeDK93Pw00RPIOobrECrya\n",
"Sf3xw6CuAs7ArPjNN2IZPHGvh99XOSdp6Fx747tcJllmN6fEhhDaRskJXNJO+IOtjsCVZnZ2ybMK\n",
"oAUNMhMwuzMtlHXxevczwHAzCt+jqFwd8YOo9gN2sMyeThk7hNB2SkrgkjriuyG2A14BnpT0E7Pa\n",
"aeroKzX0hEk9oOtc+GAOTP1T4gPCspI6A+cBO+FHwD6fFs5I4Eb8GNipiSWT5YGbgG54vfvNtLH7\n",
"NkDPSX6j+twPYM7UuAIshNZT6gq8PzDTzF4CkHQLsDtQEwm8r9QwCC6aDn3mvzcBeveVqEgSl1bB\n",
"G2T+g1++UHiPtYSAo4AT8V0mDyYNnWtjfNV+L3CcZVa4K9PH79sAgy6C6X0+e3dCb6kvkcRDaB2l\n",
"PsRcExaqrb7c+F5N6AmTmiZvgOnQp5dfPda2pC/hDwwfBXZNTN7d8F0mBwMDW5C8R+ONQVMss6NT\n",
"k7frOWnh5A3+ulfb/16G0E6UugIv9MdzSZObvJxhiWdUt5Ye0LW597t7CaHtaEGDzDcw+2FaKD3x\n",
"+yr/Cgw24z+FY3N1wLcWfhXY2TJ7ImXshfVo9vfSb1QPISyJ/FC6ESkxpSbwV4CeTV73xFfhCzGz\n",
"ySWO0yrmwgfNvT+P4g/9SiJ1As4CxgAjMftjWjjD8T3aFwDnJta7lwWuB1bC692vpYz9v+Y2+3sJ\n",
"89rm9zKEGte4sJ0x/7WkbEkxpZZQfgesL2kdebPJV/CbXGrCHJg6AWY2fW88zJrtTTOtS1oJ+Bmw\n",
"OV7vLpy8G/d3H4HXyw8x45zE5L0Bfl/lP/DOyhKTN8CcqTBh5sLvjZ8Fs1v/9zKEdkoJR2k0/wXS\n",
"KD7bRniVmZ35uZ+bmamkQVpRX6mhF0zsDt3mwfuzYVqrP8CUNsPLHncAJ2JW/NoysRS+P3sAMNqM\n",
"mUsIWTg+1874LfWnWmZXpMQueW59G7zm3b2br7xnT4sHmCG0TJHcWXICL8ck2hVpL/wkwKMxuzEt\n",
"lDWAH+Olq4PNKHzzjXIJOAk/P3xvy+w3KWOHENpWkdwZnZhtxffMnwHsD+yI2VNp4QzC75y8FJhi\n",
"RvGuzFzd8VV3T7ze/Y+UsUMI1SkSeFuQlseba5YG+mGpDTKMB6YAXzXjp0mxuXrj+7ufALa2zAqf\n",
"QhhCqG5xmFVrkzbCk+dMYIeU5C3RReIS4FhgWAuS9/bAb4DLgPGRvEOoL7ECb03S7sB04HjMrk0L\n",
"ZVW8ZPIOfhjVu4Vjvd79rcZf+1hmv0oZO4RQGyKBtwapA37R8HhgFyytQUaiH34K4TVAnljvXhr/\n",
"j8aG+H2Vs1PGDiHUjkjg5SYtC/wAWAWvdyftsZYYhx9mdZgZdyTF5lob35r4HDDUMosmmhDqWNTA\n",
"y0naAPgt8BqwTUrylugscSG+ch/RguQ9onHs64FxkbxDqH+xAi8XqQG/MPg0zC5PC2Vl/KLiD4H+\n",
"ZrxTONbr3ROBk4EDLLMHUsYOIdSuSOClkoQf4XokMAazR9PC2QIve9wKnGLGp4Vjc3XFd5hsCQyy\n",
"zF5MGTuEUNsigZdCCxpkeuHnmbySFs5Y/NyVI824NSk211p4V+aLwGDL7L2U+BBC7YsaeEtJ6+F7\n",
"rOfiN+cUTt4SHSXOxptztmtB8h6K7y3/ETA2kncI7VOswFtC2h64AW+Nv4SEA2UkVgRuxg//6mfG\n",
"W0lD5zq8cdxxltnPU2JDCPUlEngKr3d/E++M3AdLa5CR6Iu3td8FnGBG8VMIcy0FTAWGAkMss7+m\n",
"jB1CqD+RwIvSggaZjYABWFqDjMSe+APHY8y4ISk21+p4V+YbwEDLrPAphCGE+hU18CKktYFH8Cvk\n",
"hqYkb4kOEt8Bzgd2akHyHoDXu38O7BnJO4QwX6zAl8TvqbsZOAe4ILHevRx+CmEPvN79RtLQuQ4B\n",
"zsYPoqqZm45CCG2jxStwSXtLek7Sp/Ib1euLJKQj8TsnD8Ts/MTkvSG+cn4J32lSOHkrV2flmobv\n",
"Lx8eyTuE0JxSVuDP4pfxJnUd1gSpK35t2VbAYMz+lhbOrsBVwElmXJUUm+sL+F2Xc4H+ltm7/p19\n",
"G6DnJL/9fe4HMGdqXFcWQvvW4gRuZn+G+Rsz6oi0Jt4g83dgEFZ8j7VEB+AU4GvAbmb8NmnoXF9u\n",
"HPt64HTL7L/+vX0bYNBFML3PZ5+e0FvqSyTxENqvqIE3JQ3BzySZCnwvsWTSA7gOWA2vd7+aNHSu\n",
"/fHLob9umd2+8E97Tlo4eYO/bpgIRAIPoZ1abAKXdD+ekD7vZDO7u+ggkiY3eTnDzGYUjW0z0mHA\n",
"t4GDMftZWih98L3djwL7mlH45hvl6oQ/qBwNbGuZPfu/n+rRtfno7t1S5hlCqF7yDRMjUmIWm8DN\n",
"bPsS5tP0eyaX43tahdQFX3EPx7cIJjXISOyEr7xPN0t7HqBcK+GHWP0Xv2z47eY/OfeD5t+fF0fG\n",
"hlAnGhe2M+a/lpQtKaZc+8BrsxAurQY8CKwKDExJ3hKSOAG4GtizBcl7c+BJ4CmgYdHJG2DOVJgw\n",
"c+H3xs+C2dNSxgwh1BcllHkXDpTG4CvXlYF3gafNbFQznzMzq74EL/XHD4OaDnwHs+LXloll8MS9\n",
"HjDGjJeThs61D3AxMNEyu6XYmH0boNdEL5vMex9mT4sHmCHUryK5s8UJvJyTaHP6rEEGS9tjLbEu\n",
"fp7J08DhZiyivNFMbK6OwHeAscAYy+yZlLFDCO1HkdzZvnahSJ3x+yZ3xI+AfSEtnJF4Z+UUYJoZ\n",
"xXep5FoBuAlYCq93/zNl7BBC+Lz2cxaKtApwP9AbP4yqcPJurHcfjSfvfc2Ympi8N8G7Mv8P2DGS\n",
"dwihHNrHCtxb/X+Mn+GdYVb82jLRDe823RQYaMZLSUPnGo3X2Y+1zK5LiQ0hhMWp/wSuzxpksM83\n",
"yCwplJ74fZV/AYaY8Z/Csbk6ABlwCL7L5MmUsUMIYUnqN4FLnYCz8PNatsWaa5BZXDjD8YOszgPO\n",
"TyyZLIu3w6+I17tfTxk7hBCKqM8ELq2EJ18D+mGL22P9+VAEfB1fPR9oxi+Shs61Ad6V+RCwt2X2\n",
"UUp8CCEUVX8JXNoM3+Z3O3AyZsWvLRNL4fuzBwCDzZiVNHSunfFb6k+xzKanxIYQQqr6SuDS3vgx\n",
"sJMwuzktlDXwxp5XgEFmzCscm0vAycA3gNGW2W9Sxg4hhJaojwQudcQPotoP2AGzp9PCGYSfwX0J\n",
"cGZivbs7cC2wFl7v/kfK2CGE0FK1n8Cl5fEGmW54vfvNtHDG4405h5hxT1Jsrt54ueYJYH/LrPAp\n",
"hCGEUKrabuSRNsaT51/wlXfh5C3RReIS4FvAsBYk7x2A3wCX4ndWRvIOIbSp2l2Ba0GDzHGYXZsW\n",
"yqr4Q863gQFm/LtwrNe7vwV8E99l8uuUsUMIoVxqL4FLHYDTgUOBnTF7Ii2cfvjDyquBM8wofgph\n",
"rqWBK4ENgAGW2ZyUsUMIoZxqK4FrQYPMSni9+7W0cMYB5wKHmXFnUmyutfF695+AYZZZXKYQQqio\n",
"2qmBSxsAjwP/wDsrCydvic4SFwKnAdu0IHlvA/wWv3lnXCTvEEI1aPEKXNI5wC7AR8As4BAze7dc\n",
"E/vcYPMbZE7F7Iq0UFbGLyr+EOhvxjuFY73ePRHf472/ZfbLlLFDCKE1lbIC/wWwiZltju8COak8\n",
"U2pCEtLJwBXA6BYk7y3wa8seB3ZZXPJuvFD0s9e5uuL/0TgUGFQtyfvz86xGtTBHiHmWW8yz7bU4\n",
"gZvZ/fbZNWSP440s5SN1x1fOuwP9sbTuRomx+PnfJ5hxkhlLOkJ2xILYXGsBv8b3lg+2zF5MGbuV\n",
"jaj0BAoYUekJFDSi0hMoaESlJ1DQiEpPoKARlZ5AuZSrBv5VoHz3M0q9gceAufjNOa8UD6WjxNl4\n",
"c852ZtyWNHSuofje8h8BYy2z91LiQwihrSy2Bi7pfmC1Zn50spnd3fiZU4CPzOymssxI2h6/eOEM\n",
"4BISLu2UWBG4GegI9DPjraShcx3eOO44y+znKbEhhNDWSrrUWNLBwARgpJk1e7mvpNa9NTmEEOpU\n",
"q11qLGkn4Dhg60Ul7yITCCGE0DItXoFL+ivQBW9HB3jMzL5RromFEEJYvJJKKCGEECqnTToxJZ0j\n",
"6QVJf5D0Y0nLtcW4KSTtLek5SZ/Kb7GvKpJ2kvRnSX+VdEKl59McSVdLel1S0v2jbU1ST0kPNf7/\n",
"/SdJkyo9p+ZI6irpcUnPSHpe0pmVntOiSOoo6WlJd1d6Losj6SVJf2yca9I5Sm1F0vKSbm/Mmc9L\n",
"Grioz7ZVK33rN/2U7ln8AuSqO11QfmHF94GdgI2BfSVtVNlZNesafI7V7mPgGDPbBBgIHFGNv5+N\n",
"z5a2MbMtgM2AbSQNrfC0FuUo4HkofhlKhRgwwsy2NLP+lZ7MIlwE3GtmG+H/v7+wqA+2SQJv9aaf\n",
"MjCzP5vZXyo9j0XoD8w0s5fM7GP8wubdKzyn/2FmD0PxowoqxcxeM7NnGv96Hv4vyBqVnVXzzOw/\n",
"jX/ZBd8eW/iC7rYiaS2gAT+psxY2LVTtHBurE8PM7GoAM/tkcUeUVOIwq/I2/bQPawJNj659ufG9\n",
"UCJJ6wBb4guLqiOpg6RngNeBh8zs+UrPqRkX4DvSCh/NXEEGPCDpd5ImVHoyzVgXeFPSNZKekjRd\n",
"0tKL+nDZErik+yU928yvXZt8prxNP60wxypV7X8srUny4xpuB45qXIlXHTP7b2MJZS1geLWd4yFp\n",
"F+AN83toq3Zl28QQM9sSGIWXzoZVekKf0wn4EnCJmX0JeA84cXEfLgsz235xP29s+mkARpZrzFRL\n",
"mmMVewXo2eR1T3wVHlpIUmf8uIQbzCzpeOFKMLN3Jd0DbAXMqPB0mhoM7CapAegKLCvpB2Y2rsLz\n",
"apaZvdr4v29KugMvTz5c2Vkt5GXgZTN7svH17SwmgbfVLpT5TT+7L67pp4pU20rid8D6ktaR1AX4\n",
"CvCTCs+pZkkScBXwvJldWOn5LIqkleWXdiOpG7A98HRlZ7UwMzvZzHqa2brAWODBak3ekpaW1KPx\n",
"r5cBdsA3L1QN83sO5sjvPwDYDnhuUZ9vqxr4NKA7cH/j9p1L2mjcwiSNkTQH35Vwj6SfVXpO85nZ\n",
"J8CRwH34k/5bzWyRT6YrRdLN+EXPG0iaI+mQSs9pEYYAB+C7Op5u/FWNu2dWBx5srIE/DtxtVh3H\n",
"Gi9GNZf7VgUebvL7+VMz+0WF59ScicCNkv6A70KZsqgPRiNPCCHUqNq5Ui2EEMJCIoGHEEKNigQe\n",
"Qgg1KhJ4CCHUqEjgIYRQoyKBhxBCjYoEHkIINSoSeAgh1Kj/B3rn1tXOP7uAAAAAAElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x106d19790>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def plotSVM(X,t,wb):\n",
" plt.plot(X[t>0,0],X[t>0,1], 'ro')\n",
" plt.plot(X[t<0,0],X[t<0,1], 'bo')\n",
" plt.axis([-2, 6, -2, 7])\n",
" x = np.arange(-2,6,1)\n",
" y0 = -1*wb[0]/wb[1]*x-wb[2]/wb[1]\n",
" y1 = -1*wb[0]/wb[1]*x-(wb[2]+1)/wb[1]\n",
" y_1 = -1*wb[0]/wb[1]*x-(wb[2]-1)/wb[1]\n",
" plt.plot(x,y0)\n",
" plt.plot(x,y1)\n",
" plt.plot(x,y_1)\n",
" \n",
"plotSVM(X_,t_,wb_opt) ### PROVIDE THIS PLOT ###"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## New Data Set\n",
"\n",
"You are now given the following new data set."
]
},
{
"cell_type": "code",
"execution_count": 59,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Generating the data\n",
"N_half = 20\n",
"u1 = random.rand(N_half)\n",
"x1 = cos(2*pi*u1) + .3*random.rand(N_half)\n",
"y1 = sin(2*pi*u1) + .3*random.rand(N_half)\n",
"u2 = random.rand(N_half)\n",
"x2 = 2.*cos(2*pi*u2) + .3*random.rand(N_half)\n",
"y2 = 2.*sin(2*pi*u2) + .3*random.rand(N_half)\n",
"\n",
"# New Data\n",
"X_c = transpose(vstack((hstack((x1,x2)),hstack((y1,y2)))))\n",
"t_c = hstack((ones(N_half),-ones(N_half)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remark that the data is distributed on two circles, centered on zero, with different radii."
]
},
{
"cell_type": "code",
"execution_count": 60,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[-3, 3, -3, 3]"
]
},
"execution_count": 60,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAEn1JREFUeJzt3X+MZWV9x/HPh0WykN3YpKRFZCYkC7XVQZAYWxKLkyjC\n",
"3LZB0qjxj6ZCdv6o6Q4x2ihsLWsMkUiaytB/LAGLEjUNEgJxGLshTJEUUJRV1l0qu4nZkQgEqWQn\n",
"7Ka6fPvHvSN3du+d++Occ895znm/ErJ779y595nd5XOf+32+z3McEQIApOe0sgcAABgPAQ4AiSLA\n",
"ASBRBDgAJIoAB4BEEeAAkKhMAW57q+0nbe+zfcD2F/MaGABgc87aB277rIh4zfbpkh6T9OmIeCyX\n",
"0QEA+spcQomI1zq/PUPSFkmvZH1OAMBgmQPc9mm290l6UdIjEXEg+7AAAIPkMQN/PSIukXSepMtt\n",
"z2YeFQBgoNPzeqKIeNX2dyS9W9LK+v22OWwFAMYQEd7s65kC3PbZkn4bEb+2faakKyR9ftRBpMz2\n",
"nojYU/Y4ilLnn6/OP5vEz5e6YSa/WWfgb5F0t+3T1C7HfD0iHs74nACAIWQK8Ih4RtKlOY0FADAC\n",
"dmJmt1L2AAq2UvYACrRS9gAKtlL2AAq2UvYAypZ5I8/AF7CjzjVwACjCMNnJDBwAEkWAA0CiCHAA\n",
"SBQBDgCJIsABIFEEOAAkigAHgEQR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BE\n",
"EeAAkCgCHAASRYADQKKyXpUeaBx7piVNLUjbt0pHj0urixH7l8oeF5qHAAdG0A7vy26T7rjgjXvn\n",
"d9gzIsQxaZRQgJFMLWwMb6l9e3pXOeNBkxHgwEi2b+19/7YzJzsOgBIKMKKjx3vfv3ZskqOgDg+J\n",
"AAdGtLooze/YWEbZeVg6cvukRkAdHuscEcW+gB0R4UJfBJigdoBO72qXTdaOSUdun2Rw2nPL0kNX\n",
"nvqV1nLE0tykxoFiDZOdzMCBEXXCusSZLnV4tGUKcNtTkr4m6Q8khaR/i4jFPAYGoJ/N6/DUx5sj\n",
"6wz8N5I+GRH7bG+T9EPbeyPiYA5jQ00VGTDNCK/+dXjq482SKcAj4gVJL3R+v2b7oKRzJRHg6KnI\n",
"gGlKeEXsX7JnJLVOqcO36+O9+tRbu1Rq2QdFyK0Gbvt8Se+S9GRez4k66rcRJo+AKfK5q6V/HZ76\n",
"eJPkspGnUz65V9L1EbGWx3OirooMGMKrKn3qmIzMM3Dbb5L0bUn3RMT9fR6zp+vmSkSsZH1dpKrI\n",
"gCG8qtCnjvHYnpU0O9L3ZOkDt21Jd0v6VUR8ss9j6APH7/SuU+88LD2xUEwNPJ/nTknZferIxzDZ\n",
"mTXA3yvpUUk/UbuNUJJuiIjlUQaBZikyYAgv1EXhAZ7XIAAAGw2TnZxGCACJIsABIFEEOAAkigAH\n",
"gEQR4ACQKAIcABLFeeAAMmnGCZDVRIADGFtTToCsKkooADLodwLk9K5yxtMszMABZDD4BEhKLMUh\n",
"wAFkMMzl3SixFIUSCoAMVhel+UMb7+s+vpYSS5GYgQMY22aXd2s/gotsFIkAB5BJ/8u7SVxko1iU\n",
"UAAUaFCJBVlwHjiAQnGRjfFwQQcASBQXdACAGiPAASBRdKEAqCx2cW6OAAdQSeziHIwSCoCKYhfn\n",
"IMzAgS4zdmtKWtgubT0qHV+VFvdHMNsrBbs4ByHAgY4Zu3WZdNsd0u9mffPSjhlbhHgZ2MU5CCUU\n",
"oGNKWugOb0m6Q7pgWuIjeynYxTkIM3A0ymYlku1Sz4/s2yQ+spdg8EFZIMDRGINKJEelnh/Z1yQ+\n",
"spdk84OyQAkFlTBjt+bs5Y/YK3P28ozdyvs1BpVIVqXFeWnDR/ad0uEjEh/ZUUnMwFG6SS0eDiqR\n",
"7I9YmrHVknZtk85ck44dkW5nARNVlTnAbd8l6S8kvRQRF2UfEpqm38y41Z4Z5xaew5RIOmFNYCMJ\n",
"eZRQvirpqhyeBw01qcVDSiSom8wz8Ij4nu3zsw8FTTWpxUNKJKgbauAoXWdmvKO7jFLUzJgSCepk\n",
"IgFue0/XzZWIWJnE6yINzIwByfaspNmRviePK/J0SigP9lrE5Io8ADA6rsgDADWWOcBtf1PSf0v6\n",
"I9urtq/NPiwAwCBc1BiVwDGuwEbDZCddKCgdx7gC42EGjtz1mk1L7R2XvWbYc/byQ9KVJz9PS1pe\n",
"ipib9PiBKmAGjonrNZu+Tnrnr6W4Tzp3/b7uGTbHuALjoQsFuVg/TfBt0j1/KF3waNfX7pLe8o6u\n",
"8JY2ngLIMa7AeJiBI7Nes+7dnV8v7/y6pcf3rc+wJ7kTE6gTAhyZ9TpN8GZJn9MbAX6ix/etz7DZ\n",
"iQmMhwBHZv1q2Ouz7mulX74qhbrKKCfPsDmjBBgdAY7M+tWwn5JeaUnfXw9qZthAvmgjzIE905Km\n",
"FqTtW6Wjx6XVxSZdeLVXDXyndPgJaYGQBsZDG+EEtMP7stukO7pqwPM77Bk1JcSpYVcXO1zrjRl4\n",
"RvbcsvTQKZtQpNZyxBKbUFCaPjtcDz0uXU+IVx+nEU7E9p4LeNI2NqGgVP2uNbref4/0UULJ7GjP\n",
"BTxprXGbUPi4Xi3scK0/Ajyz1UVpfsfGGvjOw9KRRm1C4UCq6mGHa/0R4BlF7F+yZyS1drXLJmvH\n",
"pCO3N2UBc12/j+ut9sf1Rv1ZVAU7XOuPAM9BJ6wbHVJ8XK8euoPqjwBHLvi4Xi3r6xFv76xHHJC+\n",
"RHDXDwGOXPBxvTpYj2gO+sCRmxm7Nc3H9dJxgYx6YCcmJooDqaqB9YjmYCMPUDOsRzQHAQ7UTGc9\n",
"4lD3faxH1BM1cKCGWI9I3zDZSYAD4BiECmIRc0Kafh440kbbYboI8Iw4Dxyp4xiEdLGImdnUwsbw\n",
"ltq3pzmyE0mg7TBdBHhmnAeOtNF2mK7MAW77KtvP2n7O9mfyGFRaOA8caaPtMF2ZauC2t0j6V0kf\n",
"kPS8pB/YfiAiDuYxuDRwHjiqa5juEk4tTFfWRcz3SDoUET+XJNvfknS1pMYEOOeBo6pG6S7hGIQ0\n",
"ZQ3wt0pa7br9C0l/mvE5k8N54KgiukvqL2uAD7ULyPaerpsrEbGS8XUBDEB3SVpsz0qaHeV7sgb4\n",
"85Kmum5PqT0L3yAi9mR8HQAjorskLZ2J7cr6bds3DfqerF0oT0m60Pb5ts+Q9FFJD2R8TgA5oLuk\n",
"/jLNwCPit7b/XtJ3JW2RdGezOlCAalrvPvmVdGxOevmE9MvTpefpLqkXDrMCaqZP98mhx6XrCe90\n",
"cBohSsHJduXikmr1wGmEmDhOtitH95vmWdI7ez1mmO4T3nzTQoAjV/QeT97Jb5r/2Odxg7pPePNN\n",
"D4dZIVf0Hk/eyW+aH5S0+6THDNN90u/Nd7r95osKYgaOXNF7PHknv2le3vn1L6VXzpKeGfZsE958\n",
"00OAI1ed3uMd3TM5eo+L1etN83JJt0jf/48RFi15800PAY5ccbLd5OX1psmbb3poIwRqIK+r0HM1\n",
"++qgDxwAEjVMdtKFAgCJogYOVBgba7AZAhyoKDbWYBBKKEBFsbEGgxDgQEWxsQaDEOBARbGxBoMQ\n",
"4EBFcUUdDEIfOFBhbKxpLjbyoDFot0PdcEEH1N6M3TpX+sKM9CcXSGd+UO2DnMZtt+ONACkhwJGs\n",
"Xn3S6+dgj3MRCfquq8OeaUlTC9L2rdLR49LqYsR+/g5OQoAjWb36pG+W9Dm1Z+GjtttxNaFqaIf3\n",
"ZbdJd3T9XczvsGdEiG9EFwqS1a9Pekvn11Hb7ei7roqphY3hLbVvT7OB6SQEOJLVr0/6hMZrt6Pv\n",
"uiq293wjlbbxRnoSAhzJ6tUnfZ107L+kp56QFkatW9N3XRVHe76RSmu8kZ6ENsISsECTn7z7pOm7\n",
"Ll/vGvjOw9ITC036/4Q+8Arqs0BzSHr8+ib94wQ20/7/ZHpXu2yydkw6cnvT/v8gwCvInluWHrry\n",
"1K+0liOWhr4ALYB644o8lcQCDYB8jB3gtj9s+6e2T9i+NM9B1RsLNMA6e6Zlzy3bH1lp/zrTKntM\n",
"KcmykecZSddI+kpOY2mI1UVpfsepCzRH6HRAo7BhJ7uxAzwinpUkm/L2KCL2L9kzklqNXqAB+m/Y\n",
"abHzdUhspS9BJ6z5B4qGYz0oq00D3PZeSef0+NKNEfHgsC9ie0/XzZWIWBn2ewHUFetB3WzPSpod\n",
"6XuythHafkTSpyLiR32+ThshgFOwYWdzkzwPnICuKHZ9oqpYD8pu7Bm47WskLUo6W9Krkp6OiFM2\n",
"ojADLw+7PoF0sROz4dj1CaSLnZiNxyo/UGcEeK2xyg/UGQFea6uL7Zp3N3Z9AnVBDbzmOJYTSBOL\n",
"mACQKBYxAaDGCHAASBQBDgCJ4jTCGhln2zxb7YF0EeA1Mc7h+ByoD6SNEkpt9Dscf3pXvt8DoCoI\n",
"8NoYZ9s8W+2BlBHgtTHOtnm22gMpI8BrY5xt82y1B1eGTxk7MWtknG3zbLVvNs6Mry620gPYFGfG\n",
"V9ckL6kGoIIG9/mzkJ0yAhyoqeH6/FnIThmLmEBtDdPnz0J2ypiBA7U1uDzCleHTRoCjdJzHUpTh\n",
"yiOdP2v+vBNEgKNUnMdSpNVFaX7Hxj9byiN1QhshSkUbW7Ho808XbYRIAG1sRaI8Um90oaBktLEB\n",
"4yLAUTLa2IBxUQNH6cqu09IFgyriLBRgAA5zQlUNk51jl1Bs32r7oO0f277P9pvHfS6gPFyVCOnK\n",
"UgP/T0nviIiLJf1M0g35DAmYJLpgkK6xAzwi9kbE652bT0o6L58hAZNEFwzSlVcXynWi1xRJogsG\n",
"6dp0EdP2Xknn9PjSjRHxYOcxuyVdGhF/3ec5WMREpZXdBQP0UngXiu2PS5qX9P6I6PlR1HZI+nzX\n",
"XSsRsTL2iwJADdmelTTbdddNhQW47ask/bOk90XEy5s8jhk4AIyo0Bm47ecknSHplc5dj0fEJ8YZ\n",
"BABgIzbyAECiCt3IAwAoFwEOAIkiwAEgUQQ4ACSKAAeARBHgAJAoAhwAEkWAA0CiCHAASBQBDgCJ\n",
"IsABIFEEOAAkigAHgEQR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAAkCgC\n",
"HAASRYADQKIIcABIFAEOAIkiwAEgUQQ4ACRq7AC3/QXbP7a9z/bDtqfyHBgAYHNZZuBfioiLI+IS\n",
"SfdLuimnMSXF9mzZYyhSnX++Ov9sEj9fE4wd4BFxtOvmNkkvZx9OkmbLHkDBZsseQIFmyx5AwWbL\n",
"HkDBZsseQNlOz/LNtm+W9DeSXpP0Z7mMCAAwlE1n4Lb32n6mx39/JUkRsTsipiX9u6R/mcB4AQAd\n",
"jojsT2JPS1qKiJkeX8v+AgDQQBHhzb4+dgnF9oUR8Vzn5tWSnh5nAACA8Yw9A7d9r6S3SToh6bCk\n",
"v4uIl3IcGwBgE7mUUAAAkzeRnZh13vRj+1bbBzs/332231z2mPJk+8O2f2r7hO1Lyx5PXmxfZftZ\n",
"28/Z/kzZ48mT7btsv2j7mbLHUgTbU7Yf6fy73G97oewx5cX2VttPdrLygO0vbvr4SczAbW9f7xu3\n",
"vUvSxRGxs/AXngDbV0h6OCJet32LJEXEZ0seVm5s/7Gk1yV9RdKnIuJHJQ8pM9tbJP2PpA9Iel7S\n",
"DyR9LCIOljqwnNj+c0lrkr4WEReVPZ682T5H0jkRsc/2Nkk/lPShGv39nRURr9k+XdJjkj4dEY/1\n",
"euxEZuB13vQTEXsj4vXOzSclnVfmePIWEc9GxM/KHkfO3iPpUET8PCJ+I+lbai/E10JEfE/S/5Y9\n",
"jqJExAsRsa/z+zVJByWdW+6o8hMRr3V+e4akLZJe6ffYiR1mZftm20ck/a2kWyb1uhN2naSlsgeB\n",
"gd4qabXr9i869yExts+X9C61J0+1YPs02/skvSjpkYg40O+xmXZinvSieyWd0+NLN0bEgxGxW9Ju\n",
"259Ve9PPtXm9dtEG/Wydx+yW9H8R8Y2JDi4Hw/x8NcPKfQ10yif3Srq+MxOvhc4n+ks662nftT0b\n",
"ESu9HptbgEfEFUM+9BtKbJY66Gez/XFJLUnvn8iAcjbC311dPC+peyF9Su1ZOBJh+02Svi3pnoi4\n",
"v+zxFCEiXrX9HUnvlrTS6zGT6kK5sOtm300/KbJ9laR/kHR1RBwvezwFq8umrKckXWj7fNtnSPqo\n",
"pAdKHhOGZNuS7pR0ICK+XPZ48mT7bNu/1/n9mZKu0CZ5OakulNpu+rH9nNqLDesLDY9HxCdKHFKu\n",
"bF8jaVHS2ZJelfR0RMyVO6rsbM9J+rLai0R3RsSm7Vopsf1NSe+T9PuSXpL0TxHx1XJHlR/b75X0\n",
"qKSf6I1y2A0RsVzeqPJh+yJJd6s9uT5N0tcj4ta+j2cjDwCkiUuqAUCiCHAASBQBDgCJIsABIFEE\n",
"OAAkigAHgEQR4ACQKAIcABL1/z552MkEaOwoAAAAAElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x106eb0950>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot new data\n",
"plt.plot(X_c[t_c>0,0],X_c[t_c>0,1], 'ro')\n",
"plt.plot(X_c[t_c<0,0],X_c[t_c<0,1], 'bo')\n",
"plt.axis([-3, 3, -3, 3])"
]
},
{
"cell_type": "code",
"execution_count": 80,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"[[ 0.79320872 0.87751141]\n",
" [ 0.34859625 1.10144607]\n",
" [ 0.84730164 0.86600705]\n",
" [ 0.87178605 0.67973604]\n",
" [ 1.02320953 0.05223257]\n",
" [ 1.12153457 0.49210989]\n",
" [ 0.85257302 0.46268987]\n",
" [ 1.02866962 0.7313684 ]\n",
" [ 0.44710416 0.8148271 ]\n",
" [ 1.00715163 0.60103491]\n",
" [ 0.97326449 0.48955367]\n",
" [ 0.8782296 0.68866347]\n",
" [ 0.17089173 1.06423413]\n",
" [ 1.01157981 0.39264252]\n",
" [ 0.3417288 0.84054634]\n",
" [ 1.11146543 0.40394795]\n",
" [ 0.88481468 0.84173672]\n",
" [ 0.70515482 0.89868454]\n",
" [ 0.70562623 1.03892147]\n",
" [ 0.49678118 0.83054123]\n",
" [ 0.40706008 1.42381673]\n",
" [ 1.39657279 0.39009172]\n",
" [ 1.04451919 1.19192739]\n",
" [ 0.53297958 1.50011112]\n",
" [ 0.99420448 1.42983513]\n",
" [ 1.39050571 0.74889203]\n",
" [ 1.28308941 1.24967158]\n",
" [ 1.31448187 0.92596038]\n",
" [ 1.15165617 1.19119312]\n",
" [ 1.11444156 1.05553909]\n",
" [ 1.32773815 0.98155402]\n",
" [ 1.10494624 1.24527458]\n",
" [ 1.22212946 1.28531447]\n",
" [ 0.92884761 1.4680145 ]\n",
" [ 0.76518426 1.42363411]\n",
" [ 1.0229485 1.27350279]\n",
" [ 1.37170015 0.8541038 ]\n",
" [ 0.26667152 1.37509233]\n",
" [ 1.3817956 1.13790164]\n",
" [ 0.70795683 1.32292239]]\n"
]
},
{
"data": {
"text/plain": [
"[0, 6, 0, 6]"
]
},
"execution_count": 80,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAWgAAAEACAYAAACeQuziAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAE8lJREFUeJzt3W2QnWV9x/HvnwQaQlKpYoVIGGyi0zYLI6gIVcO2lYds\n",
"qfVFoTpDnYLJq5owtWpFQePY1qnTjiU6dqaRMJgy9gUWR9sQUMtWmQIGJMUAKsnUYYEgKBUTkszw\n",
"8O+Lcw57sjkP9+6es+fKnu9n5syeh/tc9395+OXKdV/XdUdmIkkqzzGDLkCS1JoBLUmFMqAlqVAG\n",
"tCQVyoCWpEIZ0JJUqK4BHREnRsTNEfFwRDwUEefORWGSNOwWVjjmOmBbZv5xRCwETuhzTZIkIDot\n",
"VImIVwD3Z+ZvzF1JkiToPsTxOuDpiLghIr4fEZsjYvFcFCZJw65bQC8Ezga+mJlnA88BH+17VZKk\n",
"rmPQjwGPZeaO+uubmRLQEeFmHpI0A5kZnT7vGNCZ+WRETETEGzLzx8A7gQene5KjWURszMyNg66j\n",
"X/z9jm7z+febz78bVOvcVpnFsR64KSKOA/YAV8y2MElSd10DOjP/B3jLHNQiSWriSsLuxgddQJ+N\n",
"D7qAPhsfdAF9Nj7oAvpofNAFDFrHedCVGojI+TwGLUn9UCU77UFLUqEMaEkqlAEtSYUyoCWpUAa0\n",
"JBXKgJakQhnQklQoA1qSCmVAS1KhDGhJKpQBLUmFMqAlqVAGtCQVyoCWpEIZ0JJUKANakgplQEtS\n",
"oQxoSSqUAS1JhTKgJalQBrQkFcqAlqRCGdCSVCgDWpIKZUBLUqEMaEkq1MIqB0XET4BfAi8Cz2fm\n",
"Of0sSpJUMaCBBEYz85l+FiNJmjSdIY7oWxWSpCNUDegEvhUR90bEun4WJEmqqTrE8bbM3BsRrwa+\n",
"GRE/zMzv9rMwSRp2lQI6M/fWfz4dEbcA5wAvB3REbGw6fDwzx3tYoyQd9SJiFBid1ncys1uji4EF\n",
"mbkvIk4Abgc+lZm31z/PzHR8WpKmoUp2VulBvwa4JSIax9/UCGdJUv907UF3bcAetCRNW5XsdCWh\n",
"JBXKgJakQhnQklQoA1qSCmVAS1KhDGhJKpQBLUmFMqAlqVAGtCQVyoCWpEIZ0JJUKANakgplQEtS\n",
"oQxoSSqUAS1JhTKgJalQBrQkFcqAlqRCGdCSVCgDWpIKZUBLUqEMaEkqlAEtSYUyoCWpUAa0JBXK\n",
"gJakQhnQklQoA1qSCmVAS1KhKgV0RCyIiPsj4hv9LkiSVFO1B30V8BCQfaxFktSka0BHxKnAGPAl\n",
"IPpekSQJqNaD/hzwYeClPtciSWqysNOHEXEJ8FRm3h8Rox2O29j0cjwzx3tSnSTNE/UMHZ3WdzLb\n",
"DytHxN8Cfwq8ACwCfhX4ama+r+mYzEyHPiRpGqpkZ8eAntLY+cCHMvMPp3sSSdLhqmTndOdBO4tD\n",
"kuZI5R502wbsQUvStPWjBy1JmiMGtCQVyoCWpEIZ0JJUKANakgplQEtSoQxoSSqUAS1JhTKgJalQ\n",
"BrQkFcqAlqRCGdCSVCgDWpIKZUBLUqEMaEkqlAEtSYUyoCWpUAa0JBXKgJakQhnQklQoA1qSCmVA\n",
"S1KhDGhJKpQBLUmFMqAlqVAGtCQVyoCWpEIZ0JJUqK4BHRGLIuKeiNgZEQ9FxGfmojBJGnYLux2Q\n",
"mYci4ncz80BELATujIi3Z+adc1CfJA2tSkMcmXmg/vQ4YAHwTN8qkiQBFQM6Io6JiJ3AT4E7MvOh\n",
"/pYlSeo6xAGQmS8Bb4yIVwC3RcRoZo43Po+IjU2Hjzd/JkmCiBgFRqf1ncyc7kmuBQ5m5t/XX2dm\n",
"xrQakaQhVyU7q8ziOCkiTqw/Px64ALi/NyVKktqpMsRxCnBjRBxDLdC3Zua3+1uWJGnaQxxHNOAQ\n",
"hyRNW0+GOCRJg2FAS1KhDGhJKpQBLUmFMqAlqVAGtCQVyoCWpEIZ0JJUKANakgplQEtSoQxoSSqU\n",
"AS1JhTKgJalQBrQkFcqAlqRCGdCSVCgDWpIKZUBLUqEMaEkqlAEtSYUyoCWpUAa0JBXKgJakQhnQ\n",
"klQoA1qSCmVAS1KhDGhJKpQBLUmF6hrQEbE8Iu6IiAcjYldEbJiLwiRp2EVmdj4g4mTg5MzcGRFL\n",
"gPuAd2fmw/XPMzOj/6VK0vxRJTu79qAz88nM3Fl/vh94GFjWmxIlSe1Maww6Ik4HzgLu6UcxkqRJ\n",
"C6seWB/euBm4qt6Tbv5sY9PL8cwc70l1kjRPRMQoMDqt73Qbg643fCzw78CtmfmPUz5zDFqSpqlK\n",
"dla5SBjAjcDPM/MvZnISSdLhehXQbwe+AzwANA6+OjO3Vz2JJOlwPQnoXpxEknS4nkyzkyQNhgEt\n",
"SYUyoCWpUAa0JBXKgJakQhnQklQoA1qSCmVAS1KhDGhJKpQBLUmFMqAlqVCV94MuRcTIGCzfAEsX\n",
"wb5DMLEpc9e2QdclSb12VAV0LZzPuw42r5x8d92KiBFmG9IGv6TSHFUBXQvQ5nCG2uux9cCMw7Sf\n",
"wS9JM1VkQLfvzS5d1PobS46fWXsN7YJ/zY0Rlz1oj1rSIBQX0J16s7D8UOtv7T84k/YmA7dd8L/1\n",
"JNh4fuvvSFJ/DWQWR8TIWMSa7RGXjdd+joxNftquN3vaepjYBOt2H/7Z2j3w6Ofbn61Tew372gT/\n",
"ix2+I0n9Nec96O492vbDGJm7ttV60mPra8Ma+w/Co5/v3KutMiwysQnWrTi8po8BF3f4jiT11wCG\n",
"OLpd6GvXm60NY9TDuOsww+S48+Iz4RrgQmD1Ee012jw8+PetgqtPOvz4w78jSf02gIDu1qNt1Zvt\n",
"NoxxuNa99I/Xf65u2V5z8Ne+v/U6WL2ydr/c24Hdh+CFX48YGWvXY3eqnqRemvObxkas2Q63XnTk\n",
"J2PbM7etqR0zMlYb7606jFH1HJc8Ay99r0p7tRpe+Sl4/Sq4vmlo48q9sPsJOHl/cwi3/kPhygPw\n",
"2B5Y8IRhLalZpezMzFk9ak1M5/hVY7D2EcicfLx/N6wam20tk+e4dPzw9huPS8en187F21u3c03T\n",
"87WP1H6nbsfWjuvV7+jDh4+j+1ElO+d8iGNmF/qmq/M4dnXthmMWND3fvBIu2Qq/0uZPwgVNx81u\n",
"QY2k4TKQedBVL/TN3OzHsWuqTL8DePMr4YU2bTQf6ywQSdUVt1ClF3rXS5/YBFe+A7Ysnnyv1fS7\n",
"F6nNEvk48De0P3b/QS8kSqpqzi8SHm0iLnwA3npGbajiaeAQcH3TEY0QXk1txsdnn4HjHoeXVsAH\n",
"F09O1Vu7B+7eCuddPmUO+G646ypDWhouVbKzmIAeiRhbDhuWwqJ9cGgCNu3KHHhoHTkj5DvAN4EH\n",
"n4dVx8IFHD5fujYbpdVMlFrPufMMFknDoUp29n2Io0rwjkSMnQfXbYaXe5brYMVIBIMP6anj2auB\n",
"L++BH2+FV11emyvdMDnO3WqcPeKyj7Q+h2PTko7UNaAjYgvwB8BTmXnGdBqvGrzLYUPzMQCbYeUY\n",
"zHjWQ6965G3Gs++G5efBzw/Cmp/Bi3th4ePdx7l7NbtE0lCoMFfvHcBZwA+mO5fvYtjeYmJwroFb\n",
"m4+7FMZbHXcpjM9kfuEqGFsLjzS3tRYeWQWznofceh53tTnOczEH3IcPH0fHo1N2Nh5de9CZ+d2I\n",
"OH0m4b8UWs4jXgKH/ZV+X+3K2xH2w4x6lv3okTe1Pu2bBkzO3PjtRfCjZ+HC++DE/f2ZAy5pvujr\n",
"GHTV4J2ATetgRXOoroU9j8I05y3XVP2DYWamd9OANrv37Ya7PmEwS+qkrwFdNXh3ZW4biWAM1i+B\n",
"4/fDwUfh8zO9QNjrHvmU1qc5jrzs0/24TZek+a8nAR0RG5tejmfmOEwveOvv9SSwJmDTlXDmFjil\n",
"8d4VsHemPfIprVdepVjrPY/8Vut2nLkhDZOIGAVGp/OdngR0Zm5s91kvg3c6fgF5LbXlJS8Cz0IC\n",
"b1kTMauZHdNbpbh8A6xsE8TO3JCGSb3jOt54HRGf7PadrgtVIuIrwPnAq4CngE9k5g1Nn2cWtpJw\n",
"TcT2W+GIBSHvhgNfg8VQW27yD3DgOXjq2FrS7j0IT/RygUzEZePwgfPhNmpLwF/eW/oleGYnPHGt\n",
"49DScOrJQpXMfG/vSmqv1bxlqM3IWAqLnoQliyBOhH3der/tLhK+sSmcbwP+EhbfBqfXd884CTij\n",
"twtk9h1qWuoNHAv8E9TuBXk2rLvOG9FKaqeIzZJaLWi5Es78BeS/wbJGoDZvQ9QpSNtdJGzsK3d7\n",
"va1rprQJvZyOB4ePV98O/PXUs3mxUFJbA7mr91St5i1vgVNWwTKYDNRmm2HlabUgPUJ99shhd/++\n",
"Ag5cUH++cMrPqXozHa+x3Puuq2Bse23+c8uzebFQUktF9KDbDUk0trqfbpC2mT1y90K4fDWsbOzc\n",
"3G4H595Mx6tp7MlR23TpyHFx+MWSXp1L0vxSREB3G5KYSZC2mj0yErFjDNa/AK+9Elb8GSyeuoPz\n",
"1Hnavdtlb2ITXHkmbDll8r2PAUtO6XQjWknDq4jtRluNQV8Be5/tMAa9FvbcDRtmejFvJGLsNFj/\n",
"S/jNpbDsBHj+OTg4AV/YlfmpdnWtg913wVUzOW/E6nvh/DdNTv5rbFXqdqPSsCliu9Eq2i1oAV5+\n",
"70k44cLaLI79s11p2HTOt5wDq7fAcdQeJ6yDy0ciduzK3Nb7PT1O3g+fbvG+49CSjlREQEPHBS19\n",
"+av/SMTYCvjIlvrUu4bmAO79nh5uNyqpumICeq4thw1nTQnnhkYAPwlLrqH2D+kFancdXM1sLiL2\n",
"6ma2kobB0Ab0UljU6eLjSMTYObCseebyx4HPzWJPj97dzFbSMCjiIuEgrInYfjVcNPXi4xVwYAdc\n",
"uhw2tFoufiHcd3vmm+euUknzUZXsLGKhyiBMwKatsPsi4FpgI7W9OnbAZ3dlth1/PhH2z2WdkobX\n",
"0A5xNGaOPN5mK9ROe0qXegdySfPL0AY0tJ85MhIxtgxe/R44uBKOb1wcrC9iubvcO5BLmk+Gdgy6\n",
"XS+41eKU98PBR+DBZ+CT7camx2D7tkwXm0iq5KhZqDLX2qwQXDES0XLjpuvh+DH42Xcyt10W8ZFW\n",
"bfZqgyVJahjKgO60QnBJ08XBxvb6C4Fj4K31Xveh5vcb86Orzo2evMP30kW1hSsTm5xmJ6mVoQzo\n",
"TisEGxcHW+z/8Wvr4Lr/hnu2wvOba7vvA7AOnn8U7u523jZ3+F7hpv2SWhnKaXadZmg09pLusAf1\n",
"Rc3hXH//2NPg3O5nXr6h9R2+T2u5r7Wk4TaUAd1qQ//GNqO7MrfdBVc9DM+0+u6SKeHc9H6FMeil\n",
"LXvubpYkqZWhHOJot3teY5rcrsxtayJ20GK2xn54vlWb1cag3SxJUnVDGdDQcfc84OVe9ormi4lr\n",
"Yc8EbF0Hl099v9r+HG6WJKm6oZ0HXUVjU/+pvex271dps3ah8DQ3S5KGXJXsNKAlaQDcLEmSjmIG\n",
"tCQVyoCWpEIZ0JJUKANakgrVNaAj4uKI+GFEPBIRfzUXRZVkJGJsTcT2yyLG10RsH4kYG3RNkoZD\n",
"x4UqEbEA+ALwTuBxYEdEfD0zH56L4gZtJGLsVPjnW+G1jffm2+b8ETGameODrqNf/P2OXvP5d6uq\n",
"Ww/6HGB3Zv4kM58H/hX4o/6XVYblsOHcpnCGlzdMmk+bG40OuoA+Gx10AX02OugC+mh00AUMWreA\n",
"fi0w0fT6MaYE1nzWaVvSua5F0vDpFtCzW2Z4lOu0Lelc1yJp+HRc6h0R5wIbM/Pi+uurgZcy8++a\n",
"jhnqEJekmZrVXhwRsRD4EfD7wBPA94D3DstFQkkapI6zODLzhYj4ALW7Py0ArjecJWluzHo3O0lS\n",
"f8xqJeF8XsQSEVsi4qcR8YNB19IPEbE8Iu6IiAcjYldEbBh0Tb0UEYsi4p6I2BkRD0XEZwZdU69F\n",
"xIKIuD8ivjHoWnotIn4SEQ/Uf7/vDbqeXouIEyPi5oh4uP7fZ8t7ms64B11fxPIjmhaxMI/GpyPi\n",
"HcB+4MuZecag6+m1iDgZODkzd0bEEuA+4N3z5d8fQEQszswD9WspdwIfysw7B11Xr0TEB4E3AUsz\n",
"812DrqeXIuJ/gTdlZst7gx7tIuJG4L8yc0v9v88TMvPZqcfNpgc9rxexZOZ3gf8bdB39kplPZubO\n",
"+vP9wMPAssFW1VuZeaD+9Dhq11Dmzf/sEXEqMAZ8CZivN8yYl79XRLwCeEdmboHatb5W4QyzC+ih\n",
"XsQyn0TE6cBZwD2DraS3IuKYiNgJ/BS4IzMfGnRNPfQ54MPAS4MupE8S+FZE3BsR6wZdTI+9Dng6\n",
"Im6IiO9HxOaIWNzqwNkEtFcX54H68MbNwFX1nvS8kZkvZeYbgVOB1RExOuCSeiIiLgGeysz7mae9\n",
"TOBtmXkWsAb48/qQ43yxEDgb+GJmng08B3y01YGzCejHgeVNr5dT60XrKBERxwJfBf4lM7826Hr6\n",
"pf7Xx/8A3jzoWnrkd4B31cdpvwL8XkR8ecA19VRm7q3/fBq4hdqQ6nzxGPBYZu6ov76ZWmAfYTYB\n",
"fS/w+og4PSKOA/4E+Pos2tMciogArgceysx/HHQ9vRYRJ0XEifXnxwMXAPcPtqreyMyPZebyzHwd\n",
"8B7gPzPzfYOuq1ciYnFELK0/PwG4EJg3s6ky80lgIiLeUH/rncCDrY7tuFCly0nm9SKWiPgKcD7w\n",
"qoiYAD6RmTcMuKxeehtwOfBARDSC6+rM3D7AmnrpFODGiDiGWkdka2Z+e8A19ct8G258DXBLrQ/B\n",
"QuCmzLx9sCX13Hrgpnrndg9wRauDXKgiSYXylleSVCgDWpIKZUBLUqEMaEkqlAEtSYUyoCWpUAa0\n",
"JBXKgJakQv0/lfRzUKDZYtIAAAAASUVORK5CYII=\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x106a6b390>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#X_transformed = 2.5*transpose(vstack((hstack((np.abs(x1),np.abs(x2))),hstack((np.abs(y1),np.abs(y2)))))) #radius ### CHANGE THIS LINE ####\n",
"#X_transformed = transpose(vstack((hstack((sqrt(x1**2+y1**2),sqrt(x2**2+y2**2))),map(lambda x: m.atan2(x[0],x[1]),X_c)))) # radius and angle\n",
"#X_transformed = transpose(vstack((hstack((x1**2,x2**2)),hstack((y1**2,y2**2))))) #radius ### CHANGE THIS LINE ####\n",
"X_transformed = transpose(vstack((hstack((abs(x1)**(0.5),abs(x2)**(0.5))),hstack((abs(y1)**(0.5),abs(y2)**(0.5))))))\n",
"print X_transformed\n",
"### prbly other transormation\n",
"plt.plot(X_transformed[t_c>0,0],X_transformed[t_c>0,1], 'ro')\n",
"plt.plot(X_transformed[t_c<0,0],X_transformed[t_c<0,1], 'bo')\n",
"plt.axis([0, 6, 0, 6])"
]
},
{
"cell_type": "code",
"execution_count": 76,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"(40, 2)"
]
},
"execution_count": 76,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"shape(X_transformed)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##Plotting the result in the transformed space. \n",
"(Lines are commented, because code does not compile, until X has been correctly transformed.)"
]
},
{
"cell_type": "code",
"execution_count": 81,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"ename": "ValueError",
"evalue": "all the input array dimensions except for the concatenation axis must match exactly",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-81-6c4875b82fa6>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mwb_opt_c\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moptimizeSVM\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_transformed\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mt_c\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0;31m#plotSVM(X_transformed,t_c,wb_opt_c) ### PROVIDE THIS PLOT ###s\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#plotSVM(X_transformed,t_c,[-2,2,-1])\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m<ipython-input-57-49e7f2f9b2de>\u001b[0m in \u001b[0;36moptimizeSVM\u001b[0;34m(X_, t_)\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 14\u001b[0m opt_result = optimize.minimize(objective, wb0, jac=jac,constraints=cons, #the optimizer\n\u001b[0;32m---> 15\u001b[0;31m method='SLSQP', options=opt)\n\u001b[0m\u001b[1;32m 16\u001b[0m \u001b[0mwb_opt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mopt_result\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"x\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;31m#result of the optimization\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 17\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/Users/Maximus/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.pyc\u001b[0m in \u001b[0;36mminimize\u001b[0;34m(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)\u001b[0m\n\u001b[1;32m 433\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mmeth\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'slsqp'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 434\u001b[0m return _minimize_slsqp(fun, x0, args, jac, bounds,\n\u001b[0;32m--> 435\u001b[0;31m constraints, callback=callback, **options)\n\u001b[0m\u001b[1;32m 436\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mmeth\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'dogleg'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 437\u001b[0m return _minimize_dogleg(fun, x0, args, jac, hess,\n",
"\u001b[0;32m/Users/Maximus/anaconda/lib/python2.7/site-packages/scipy/optimize/slsqp.pyc\u001b[0m in \u001b[0;36m_minimize_slsqp\u001b[0;34m(func, x0, args, jac, bounds, constraints, maxiter, ftol, iprint, disp, eps, callback, **unknown_options)\u001b[0m\n\u001b[1;32m 402\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 403\u001b[0m \u001b[0ma\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvstack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma_eq\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ma_ieq\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 404\u001b[0;31m \u001b[0ma\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconcatenate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mla\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 405\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 406\u001b[0m \u001b[0;31m# Call SLSQP\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mValueError\u001b[0m: all the input array dimensions except for the concatenation axis must match exactly"
]
}
],
"source": [
"wb_opt_c = optimizeSVM(X_transformed,t_c)\n",
"#plotSVM(X_transformed,t_c,wb_opt_c) ### PROVIDE THIS PLOT ###s\n",
"#plotSVM(X_transformed,t_c,[-2,2,-1]) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Feel free (not mandatory!) to rewrite the function plotSVM to plot the data and the separation curve, not in the transformed space (as above), but in the original space."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"annotations": {
"author": "",
"categories": [
"intelligent-systems-1-2015"
],
"date": "2015-04-23",
"location": "Beginning of next lecture",
"parent": "IS_SS2015",
"submission_date": "2015-04-30",
"subtitle": "Exercise Sheet 2, Support Vector Machines",
"tags": [
"IntelligenSystems",
"Course"
],
"title": "Intelligent Systems 1 - Summer Semester 2015"
},
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
MaxXximus92