{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"preamble": true
},
"source": [
"(Defining latex commands: not to be shown...)\n",
"$$\n",
"\\newcommand{\\norm}[1]{\\left \\| #1 \\right \\|}\n",
"\\DeclareMathOperator{\\minimize}{minimize}\n",
"\\newcommand{\\real}{\\mathbb{R}}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Monster vs. Mouse (50 Points)\n",
"\n",
"While trying to sleep, you hear some noise under your bed. Being a big Science Fiction fan you obviously immediately think it's a monster. A BIG monster... or maybe two?!? Frightened to death, you do not dare to move, but your brain is racing. And it recalls that the other day, you saw a mouse disappearing on the other side of the bedroom... So maybe it is (just?!?) a mouse. Or, just something else... Hmmm. Remembering your Emperical Inference classes, you decide to evaluate the probability that, given some noise under your bed, there is a monster.\n",
"\n",
"You define the random variables:\n",
"\n",
" + n = some noise under your bed (Values 0 or 1)\n",
" + M = 0, 1 or 2 monsters under your bed (Values 0, 1 or 2)\n",
" + m = a mouse under your bed (Values 0 or 1)\n",
" + e = something else (e.g. only air) under your bed (Values 0 or 1),\n",
"\n",
"and express your beliefs about monsters, mice and noise, by assigning numbers to P(M), P(m), P(e), P(n|M), P(n|m), P(n|e). Given that you heard some noise under your bed, you then calculate the Maximum a Posteriori (MAP) of M (i.e., the maximum of P(M|n=1)).\n",
"\n",
"Please share your beliefs and your results with us."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Programming a Hand-Featured SVM (50 points)\n",
"\n",
"The goal of this exercise is to implement a very simple SVM using an off-the-shelf optimizer, in the case of a 2-dimensional input space.\n",
"Most of it has already been implented in an iPython Notebook that you can find on the homepage of the course, and that we reproduce hereafter.\n",
"\n",
"You are asked to hand out a print out of the lines marked \"### CHANGE THIS LINE ###\", as well as your plots."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The SVM equation in the linear separable case\n",
"\n",
"Let $(x_1,t_1), \\ldots, (x_N,t_N) \\in \\real^p \\times \\{-1,1\\}$ be N data points with their binary labels. For this exercise, $p=2$. \n",
"In the lecture, we saw that trying to maximize the margin between two linearly separable classes of data points led to the problem:\n",
"\\begin{equation}\n",
" \\minimize_{\\substack{ \\{ w \\ \\in \\real^2,\\ b \\ \\in \\ \\real \\} }} \\frac{1}{2} \\norm{w}^2 \\quad \\mathrm{subject \\ to} \\quad t_n(w^T x_n + b) \\geq 1 \\quad n = 1 \\ldots N ,\n",
"\\end{equation}\n",
"whereby the points are subsequently classified according to the rule:\n",
"\\begin{equation}\n",
" t = \\left \\{ \n",
" \\begin{array}{cc}\n",
" \\ \\ 1 \\quad \\mathrm{if} \\quad w^T x + b \\geq 0 \\\\\n",
" -1 \\quad \\mathrm{if} \\quad w^T x + b < 0\n",
" \\end{array}\n",
" \\right . .\n",
"\\end{equation}\n",
"The function $\\mathop{l}(w,b) := \\frac{1}{2} \\norm{w}^2$ is called the _objective_ function. The equations $t_n(w^T x_n + b) \\geq 1$ are called the (_inequality_) _constraints_.\n",
"\n",
"In practice, one often preferres to transform this problem into its so-called _dual_ formulation. However, in this exercise, we will stick to the above equations: the _primal_ formulation."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##Loading the required packages."
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"%matplotlib inline\n",
"import numpy as np\n",
"from numpy import *\n",
"from scipy import optimize\n",
"from matplotlib import pyplot as plt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definition of the data point \n",
"\n",
"_X = location of the data points_ $\\in \\real^{N \\times 2}$ \n",
"_t = label class of each point_ $\\in \\real^N$"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"X_ = np.array([[1.,3.],[2.,6.],[2.,3.],[-1.,0],\n",
" [1.,0.],[2.,2.],[3.,1.],[0.,-1.]])\n",
"t_ = np.array([1,1,1,1,-1,-1,-1,-1])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definition of the Objective Function and its Gradient\n",
"\n",
"_wb = $(w_1, w_2, b) = (w , b)$_ $\\in \\real^2 \\times \\real$\n",
"\n",
"(Note that the gradient of the objective function (here called 'jac') does not appear in the equations of an SVM. But our off-the-shelf optimizer uses it for its computations.)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"def objective(wb):\n",
" ### changed ###\n",
" return 1/2*(wb[0]*wb[0]+wb[1]*wb[1])\n",
"\n",
"def jac(wb):\n",
" return hstack((wb[0:2],array([0])))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Definition of the Inequality Constraints and of their Gradient\n",
"\n",
"(Again, the gradient does not appear in the SVM equations, but it is needed for our optimizer.)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"def ineq(wb):\n",
" #writing \"return a-b\" would encode the constraint a-b >= 0\n",
" ### changed ###\n",
" return t_*(dot(wb[0:2],X_.T)+wb[2])-1\n",
"\n",
"def grad(wb):\n",
" return -1*hstack(((-t*X.T).T, -t[:,newaxis]))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Optimisation of the SVM"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [],
"source": [
"def optimizeSVM(X_,t_):\n",
" global X,t\n",
" X = X_\n",
" t = t_\n",
" wb0 = np.random.randn(3) #initialization\n",
" #wb0 = array([-1.,-2.,0.2])\n",
"\n",
" cons = {'type':'ineq', #constraints\n",
" 'fun': ineq,\n",
" 'jac': grad}\n",
"\n",
" opt = {'disp':False}\n",
"\n",
" opt_result = optimize.minimize(objective, wb0, jac=jac,constraints=cons, #the optimizer\n",
" method='SLSQP', options=opt)\n",
" wb_opt = opt_result[\"x\"] #result of the optimization\n",
" \n",
" return wb_opt\n",
"\n",
"wb_opt = optimizeSVM(X_,t_) #the optimized parameters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Plotting the Result"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAIABJREFUeJzt3XeYXVX1xvHvSyCkAiIoIqFIQMRY0JBCKIOAkBjpvbdI\n",
"TYSfhCZycpAiKgqJSgmIIAgKiKKgAkoEFEEUC2IDSwJKURQTBQRcvz/WDJmBSXLXvXfmllmf58mj\n",
"986se3YmDysn++z9bpkZKaWUWs9yjR5ASiml6mQDTymlFpUNPKWUWlQ28JRSalHZwFNKqUVlA08p\n",
"pRZVUwOX9GZJD3T79YykGfUaXEoppSVTvdaBS1oOeAwYZ2YL6vKhKaWUlqieUyjbAo9k804ppf5R\n",
"zwa+N/ClOn5eSimlpajLFIqkwfj0ycZm9lTNH5hSSmmZlq/T50wGftJb85aUYSsppVQFM9OyvqHm\n",
"X8C1wEFL+JrV4xp9/QuY1egxtMs4W2GMOc4BNE7oMPiLwUkGatpxvvrnacv6nprvwCUNxx9gTqv1\n",
"s1JKqW4kAccBJwEHYHZb5/uNHFVd1dzAzezfwGp1GEtKKdWH31heCmwITMDsT40dUN/InZiLzWv0\n",
"ACo0r9EDqMC8Rg+gQvMaPYAKzWv0ACo0r9EDAEAaDfwIeB7YvJfmPa+/h9RX6raRZ4kXkMyWNRGf\n",
"Ukr1IL0PuBwogIvo6wbXhyrpnfVahZJSSo3jO8E/gj+L2xmzHzZ4RP0iG3hKqbVJqwBfBFYBxmL2\n",
"eINH1G9yDjyl1LqktwE/Bv4AvGcgNW/IBp5SalXS3sD3gBKzD2L2QqOH1N9yCiWl1Fqk5YFzgV2A\n",
"7TD7WYNH1DDZwFNKrUN6HfBl4Dl8vvvpWDnjgT+Y0RaZTTmFklJqDdJ44H7gbmBqpHlLSOJI4BvA\n",
"m/tohP0u78BTSs1PmgacBUzD7OuxUoYAnwPGAZPM+H0fjLAhsoGnlJqXNASYA2wGbIHZb2PlrAPc\n",
"ADwMTDBjUf0H2Tg5hZJSak7SKOBOfH33hCqa97bAvfhBM/u0W/OGbOAppWYkbQ3cB1wH7InZwspL\n",
"kcSJ+Oaefcz4lKfKtp+cQkkpNQ+PgP1Q56/9MfturJyReBbKKGCcGW19Rm828JRSc5BGAJcBbwLG\n",
"YzY/Vs6bgRvxVSr7mfF8/QfZXHIKJaXUeNKGeATsIvxhZbR57wzcBXzKjA8MhOYNdWjgklaRdL2k\n",
"X0t6SNKEegwspTRASDvid82zgcMxe67yUgZJnNVZO9WMS/tolE2pHlMoFwC3mNnu8i2uw+vwmSn1\n",
"mTHSlFEwYyQMWQjPLYDZD5rd0uhxDTjSIDy3+xBgR8x+FCvntcDVwGBgrBlP1n+Qza2mBi5pZWAL\n",
"MzsIwMxeBJ6px8BS6gtjpCkT4YK5MLrrvWmw/hiJbOL9SHoN3nyH4Vvin4iVswm+vvsG4BQzXqz/\n",
"IJtfrVMo6wFPSbpc0k8lzZU0rB4DS6kvjIIZ3Zs3wFwYvTZMb9SYBhzpHfiW+N/gYVTR5n0AcCve\n",
"uGdGmrdKraCyfU41rnUKZXngXcCxZvZjSecDJwOnd/8mSbO6vZxnZvNqvG5KVRkJQ3p7fwQM7e+x\n",
"DEjSvvi06wzMromVMhg4D9gB2NqMB0P1pdYErgc+DnwtUtsfJHUAHZGaWhv4o8CjZvbjztfX4w28\n",
"BzObVeN1UqqLhZ5i9yqL4Nn+HsuAIq0AfAKYCmyD2S9i5bwB39TzNLCpGf8M1ZfaArgW+CxwU6S2\n",
"v3Te2M7rei2pWFZNTQ3czB6XtEDShmb2O2Bb4Fe1fGZKfWkBzJ4G63efRjkcHpnveRupL0hrAF8B\n",
"FgKbYvaPWDmT8AjZi4GzzPhfxbU+XTId+DBwoBX2nci1m13Np9LL57MuxZ8EPwIcYmbPdPt6nkqf\n",
"msoYacraMH0EDF0Ez86HOfkAs49IE/HmfRlwBmaVN18h4Gh8SvYQM0J/Rio1DLgEeCuwqxX2x0h9\n",
"o1XSO2tu4PUYREqpzfiW+COAM4BDMftmrJyhwEXAO4FdzXgkVF/qTcBXgV8AR1ph/4nUN4NKemfu\n",
"xEwp1Zc0FL/jPgbYrIrmvR7wA3yKd7Mqmvdk4B58ZuCgVmzelcoslJRS/Uhd+du/xyNg/x0r573A\n",
"lcA5wOxIiqBKLQecChwF7GaF3R25divKBp5Sqg9pW+Aq/MDh8wnMz3bOd58CHAvsacadoUuXWhlv\n",
"/KsBm1phf4nUt6ps4Cml2vh894nAB4G9Ce7zkFgJuAJYA18i+FiovtRb8RTCW4E9rLD/RupbWTbw\n",
"lFL1pO752+MxC+VvS7wFb753AHtHUwRVag/8vMsPWWFXRmrbQTbwlFJ1pI3wlR53AfthFmu+Yjfg\n",
"QuAkMy4P1ZZaHp8n3x14rxX2QKS+XWQDTynFSTvja6xPxSwU4SoxCD9hfm9gshk/CdWXWh3fVfki\n",
"MNYK+3ukvp1kA08pVc4jYM8A9gemYnZfrJzVgGsA4RGwfwvVl9oUj+y4CjjdCnspUt9usoGnlCoj\n",
"vRY/4X0FfEt8KH9b4t34EsNrgdOiEbAqdRg+bXKEFXZjpLZdZQNPKS2btAk+3309cAqe/R8o52A8\n",
"zOpIM24I1ZZaET9xZ0tgSyvsN5H6dpYNPKW0dNKBeIzrMZh9JVbKYOB84D3AVmY8FKovtRZ+1/4o\n",
"MM4KWxipb3fZwFNKvZMGA58C3gt0YBZKGpV4Ix4B+yQwzox/hepLdeBTNhcAH7eij4ObWlA28JTS\n",
"q0lr4s33b/h8d+ioRIkt8YeVnwU+VkUE7HHAScABVthtkWsPJNnAU0o9SZvj+dsXAmdXEQG7OH/b\n",
"COVvq9RwPIRqQ2CCFfanSP1Akw08peR8S/wxwEeAgzH7VqycYcBcYGNgghmh/G2VGo3vyrwf2NwK\n",
"y1OSliHjZFNK4IeRXwFMwyNgo837TXiE60vApCqa9/vwCNnPAodm865MzXfgkv4E/Av/g3vBzMbV\n",
"+pkppX4krYcvEXwImIjF8rMlJgNfAD4KfLaKCNiPAIcDO1th90SuPdDVYwrFgA4ze7oOn5VS6k/S\n",
"9ngM61nAnGAE7HL4XPeR+Kk5PwhdutQq+I7KlfAI2Mcj9al+c+B5ZFpKrURaDjgZn/PeHbO7YuWs\n",
"DHwRWBXfEv/XUH2pt+F3/TcDM62wFyL1ydXrDvx2SS8BF5vZ3Dp8Zkqpr0hd+duvx5cIhg4/kOjK\n",
"3/4OsLsZofxtldoH31l5nBV2daQ29VSPBj7JzP4qaXXgNkm/sVf8bS5pVreX8ywY+J5SqhNpY7z5\n",
"3g7shcUOP5DYE3/Q+CEzQvnbKrUCflrPTsC2VtjPI/XtTlIH0BGqqeep9JIKYJGZndftvTyVPqVm\n",
"IO2Or+2eidkXYqV0z9/e1YxQ/rZKvR5fW/4ssJ8V+cxsWfr8VHpJw+QnciBpOL7l9pe1fGZKqc6k\n",
"5ZHOBT4J7FBF814dP67s7fh8d7R5j8fXdt8JTM3mXT+1TqG8HrjR1/+zPHC1md1a86hSSvUhrYbH\n",
"txowFrNY/rbomb9tVJy/3bklfhpwJnC4FXZT5Npp2eo6hdLrBXIKJaXGkMbizfca4DQsdviBxOL8\n",
"bSOUv61SQ4DPABOBXayw30XqU2W9M7fSp9SOpEOBjwFHYvbVWCk987eNUP62Sq2NR8D+ERhvhS2K\n",
"1KfKZQNPqZ1IK+Lxqx3AVpj9OlZOV/72AjwCNpS/rVLvAa7G59s/lRGwfSsbeErtQnojPmXyV2Ac\n",
"ZrH8bdFB9/zt2JZ4AR/q/LWfFfa9yLVTdbKBp9QOpK3wue45wMeCW+IFHA+cCOxvxu2hS5caAXwe\n",
"WA+fMpkfqU/VywaeUivzJWAfxLfFH0hwFZjEcOAyYANgvBl/DtWX2hDfGHQPsIUV9lykPtUmG3hK\n",
"rcr3XlwCvAWYgMUOP5Domb9thCJcVWonPP/7w1ZkhEYjZANPqRVJ6+PN9wFgEhbLz5aYik97nA5c\n",
"HJzvHgTMAg4C3m+F3Ru5dqqfbOAptRppCnA5UAIXVhEBezpwGLCTGaH8bZVaFV9lMhSPgH0iUp/q\n",
"Kxt4Sq3CI2BPAz4A7ILZD2PlvAbfUTkS2NSMUP62Sr0Dj4D9GnCSFfZipD7VXzbwlFqBtAqev70K\n",
"HgEby98Wb8eb7zeBmWaE8rdVan/g08B0K+zaSG3qO9nAU2p20hh8vvsW4AQsdviBxOL8bSOUv90Z\n",
"AftJYArwHissw+qaSDbwlJqZtBeeKXI8ZlfFSumZv22E8rdVag3gK8Az+Hz3PyP1qe9lA0+pGUnL\n",
"4813F2A7zH4WK6crf/s/eATsP0L1pSYC1+HLFM+0wv4XqU/9Ixt4Ss1Geh3efJ/HI2BD+dkSE/A7\n",
"58uB0oyKm2/nlvgj8WWCh1phN0eunfpXTQc6pJTqTC8ffnA38L5I85aQxBHATcAxZhTB5j0Ub/pH\n",
"AZOyeTe/vANPqVlI04CzgGmYfT1WyhD8rMrxwCQzfh+qL7Uuvkrlt8BEK+zfkfrUGHW5A5c0SNID\n",
"kr5Rj89LaUCRhiDNBY4Dtqiiea8N3AWMACZU0by3A34EXAnsm827ddTrDvyDwEP4BoGUUqWkUXj+\n",
"9p/xPJNY/rbYBt8Z+QngU1VEwJ4EzAD2ssK+H7l2arya78AlrYWvEb0UyKPTUqqUtDVwH77aY89I\n",
"8+6c756J76zc14zzgs17JTw7fGdgXDbv1lSPO/BPAzOBlerwWSm1P4+A/T/8v5v9MPturJyReBDV\n",
"OvipOQtC9aU2wjcGfR+fMnk+Up+aR00NXNJU4Ekze0BSx1K+b1a3l/PMbF4t102pZUkj8Pzt9YHx\n",
"mMXyt8Wb8YeNPwQOMCOUv61SuwAXA6dYYZdFalPf6uyhHaGaWk6ll3Q2cADwIjAEvwu/wcwO7PY9\n",
"eSp9SgDShnjzvQ84GosdfiCxOH/bCOVvd0bAfhTYD9jNCrs/Up/6XyW9s6YG/oqLbQWcYGbvjw4i\n",
"pbYn7Yg/J/oIcEkwArZ7/vYeZoTyt1Xqtfhxa4OAva2wpyL1qTEq6Z31XgeeJ1Cn1J00CCiAQ4Ad\n",
"MftRrJyu/O0h+Jb4J0P1pd6Fr3K5Djg1I2DbS93uwJd4gbwDTwOV9Bq8+Q7HV5mEDj+QeCfefD1/\n",
"2wg1X5U6CE8SPNoKuy5SmxqvEXfgKSUAvXz4wU3AiVVEwHblbx9rxpdDtaUGd9ZuC3RYYb+K1KfW\n",
"kQ08pXqT9gUuAGZgdk2slBWA84DJwNZmPBiqL7Umvr77KXx99zOR+tRasoGnVC/SCviOyKnANpj9\n",
"IlbOG/AUwX/iR56F8rdVagvgWuBzwDkZAdv+soGnVA96+fCDhfiRZ7H8bbFZZ73nb8cjYI/Fz8s8\n",
"yAr7duTaqXVlnGxKtZImAj8Gvge8P9K8O7fEH43vjDzCjDOCzXsYHkJ1GJ4imM17AMk78JSq5Vvi\n",
"jwDOAA7F7JuxcoYCFwLvwiNgHw7Vl3oT/qD0l8BmVth/IvWp9eUdeErVkIbiW+KPASZV0bzXBX4A\n",
"DAYmVtG8dwDu6RzDgdm8B6a8A08pSloHv/P9PTARs0WxcrYDvgh8DLggmCK4HHAKcDS+Jf7uyLVT\n",
"e8kGnlKEtC0e4fpx4NPBLfE987eNUISrSq0MXAGsjp8S/5dIfWo/2cBTqoTPd5+In5qzD2Z3xMpZ\n",
"CfgCsCYeAftoqL7UxviDztuBPa2w/0bqU3vKBp7Sskgj8cN+RwHjMIvlb4vu+dv7mBHK31apPfC1\n",
"3SdYYVdEalN7ywae0tJIG+Hz3Xfjhy/Emq/YFc/fPsmMz4dqSy0PnA3sCWxvhf00Up/aXzbwlJZE\n",
"2hnfWHMqZpfGShkEnAnsC0w2I5S/rVKr47sqXwLebYX9PVKfBoZs4Cm9kkfAnoEfVjIVs/ti5awG\n",
"fAnP3x5rRih/W6XG4imEVwMfscJeitSngSMbeErdSa/Fm+8KwFjMYvnbomf+djwC9lDgXOAIK+yr\n",
"kdo08NR6JuYQ/MHMiviGhK+b2Sn1GFhK/U7aBG++NwCnYLHDDyQW528bofxtlVoRTzDcCtjSCvt1\n",
"pD4NTDU1cDN7TtLWZvYfScsDd0va3Cw3F6QWIx2Ix7gei1ksf1v0zN82QvnbKrUWHgH7F2C8Ffav\n",
"SH0auGqeQjF7eQvvYHzO7+laPzO1pjHSlFEwYyQMWQjPLYDZD5rd0uhxvVL3cT4Hz58H/90ANgK2\n",
"xiyWvy268refxNd3h/K3VWor/LzK2cC5VvTxEVmprdTcwCUtB/wUWB+40MweqnlUqeWMkaZMhAvm\n",
"wuiu96bB+mMkmqmJ9zbOX8G/d4NDbog375752/EI2OPwnZkHWmG3Rq6dEtQhzMrM/mdm7wTWAraU\n",
"1FHzqFLLGQUzujdFgLkwem2Y3qgx9aa3cb4Vhj8Lh1b6GZ0RsNPxO+/DzDgr2LyH4ytMDgAmZPNO\n",
"1arbKhQze0bSzcBYYF73r0ma1e3lPDPr8fXU+kb6qemvMgKG9vdYlqbWcUoMw9eGj8FTBP8Qub5K\n",
"jcY3Bv0UmGSFPRupT+2r8+a3I1JT6yqU1YAXzeyf8njN7YDyld9nZrNquU5qfgvhud7eXwTN06Ck\n",
"YSfABr19qZJxSvTM3zZCEa4q9T58S/4s4MKc707ddd7Yzut6LalYVk2td+BvAK7onAdfDviimX23\n",
"xs9MLWgBzJ4G63efnjgcHpkPcxo5rpdJ6wFfXQN+dyw8+xl/ZgNUNk6JyXgY1ZnAZ6qIgP0IMA3Y\n",
"2Qr7YTW/hZReSYE0zOouIJmZqU8vkprCGGnK2jB9BAxdBM/OhzlN8QBT2h4/duxsYPYYmFzpOCWW\n",
"A04FjsIjYENLZFVqFTz7exVgDyvs8Zp+L2nAqKR3ZgNP7cv/ZXgyfuDv3pjdGStnZbzxrwbsYUYo\n",
"f1ul3oZPudyCJwm+EKlPA1slvTO30qf2JK2EH36wBn5K/GOxct6KR8DehjfvUP62Su2NT8scb4Vd\n",
"FalNqVLZwFP7kTbG73zvwO+8oxGwi/O3jVD+dmcE7LnALsB2VtjPIvUpRWQDT+1F2g24CDgRs8tj\n",
"pXTlb+8BbG9GKH9bpV4HfBl4HhhrheWu5NSnsoGn9uBZPGcCewM7YPaTWDnd87fHmhHK31ap8XgC\n",
"4RXArIyATf0hG3hqfb4f4RpAeATs32LlbIrvqvT8bSPUfFVqGnAWMM0K+3qkNqVaZANPrU16Nx7/\n",
"ei1wWhURsIcBHwOOMCOUv61SQ/AHlZsBW1hhv43Up1SrbOCpdUkHA58AjsLs+lgpPfO3jVD+tkqN\n",
"wv/i+DOeZ7IwUp9SPWQDT61HGgycD7wH2IpgAqbEWnjzfQwYb0Yof1ultsZP7fkU8MncEp8aJRt4\n",
"ai3SG/H56ieAcVjs8AOJDrz5ev52bEu8gA8BJwD7WZGxEamxsoGn1iFtiT+s/CzwMcwqj3AVXfnb\n",
"JwMHmBGKcFWpEcBleIbKeCvsz5H6lPpCNvDU/CQBM/BMkgMx+06snOHApcCb8SmTP4XqS22Ibwy6\n",
"D9jcCus1eTGl/lbzgQ4p9SlpGHAVcDAwoYrmPRr4Eb65ZlIVzXtH4G58tclh2bxTM8k78NS8pPXx\n",
"O9+fA5NYfP5qheX0zN+OzXcPAgrgEGBHK+xHkWun1B+ygafmJHXlb38U+CyB2MzOCNjF+dtGKH9b\n",
"pVbF7/qH41vin4jUp9RfsoGn5uIRsB8GjgR2xewHsXK652+PNSOUv61S78Dv+m8CTswI2NTMaj1S\n",
"bRSel/w6wIBLzGx2PQaWBiBpZbz5ropvif9rrJye+dtGqPmq1H74+vIZVtg1kdqUGqHWO/AXgOPN\n",
"7GeSRgA/kXSbmYV2taWENAZvvt8Bdscslr8tFudvG6H8bZVaAd/RORXYxgr7RaQ+pUapqYGb2ePg\n",
"/0Q1s0WSfg2sCbFtyWmAk/bE13Z/CLMrY6X0zN82lpm/LY2ZAqNmwMghrPLE/5g2aHWGvzQf2NQK\n",
"+0c1v4WUGqFuc+CS1gU2Ae6t12emNucRsOcAuwPvxeyBWDk987eNZeZve/OeeAHMHc1a98Aee8ID\n",
"b32aO1442V58KJt3ail1WQfeOX1yPfBBM1tUj89MbU5aHbgVeDs+3x1t3uOB+/E12u+rpHm7UTPg\n",
"ktGMvRD22QluvhBu//mqvLTusbHfQEqNV/MduKQV8GCgq8zsa0v4nlndXs4zs3m1Xje1MKkrf/sq\n",
"4HQsdviBxOL8bSOWv73C0GFMORTWvB8u+wE8vUHnF0YMDX1OSnUmqQPoCNXUciq9fIvzFcDfzez4\n",
"JXxPnkqfFpMOw6dNjsDsxlgp3fO3dzUjlL+tUuvwt2G/4PH3r8RNl8J/R3T76pRvm90yOfJ5KfWl\n",
"SnpnrVMok4D9ga0lPdD5a4caPzO1I2lFpIvxJL8tq2jeo4A78fXdE6po3tsC9/IvfYXrRzzcs3kf\n",
"/gjMnxP5vJSaQU134BVdIO/Ak9SVv70AOASLHX4g0TN/Ox4BeyKeRLivFXaHP8hce7pPmyx6FubP\n",
"MXvwlsiYUuprlfTObOCpb/m83pfw028+HtwS35W//SFgfzNC+dsqNRLPQhkF7G6FLYjUp9RIlfTO\n",
"3Eqf+oY/Hzkev/vdH7PbY+WMAD4PrIdHwM4P1ZfaCN8YdDd++MLzkfqUWkE28FR/0nD88IMNgPFY\n",
"7PADiQ2BG/EY2C3MCEW4qtTOwCXAqVbYpZHalFpJNvBUX9JovPneD2yO2bOxcnbED184DZhbRQTs\n",
"GcABwFQr7L7ItVNqNdnAU/1IU/FpjwK4KDjf3TN/2wjlb6vUa/G59hXwCNgnI/UptaJs4Kl2HgF7\n",
"OnAYsBNm98TK6Zm/bYTyt1VqE3y++3rgFCvsxUh9Sq0qG3iqjfQavPmOBDbFA84C5XTlb38dOKmK\n",
"CNgDgfOAY6ywr0RqU2p12cBT9aS34833m8BMLHb4gcTi/G0jlL+tUoPxdeHvBba2wh6M1KfUDrKB\n",
"p+pI+wCzgeMwuzpWSs/8bSOUv61SawLXAX8Hxllh/4zUp9QusoGnGA8vOxfYCdgWs5/HylkD+Aqw\n",
"ENjUjFCEq0ptjkfIXgScZYX9L1KfUjvJBp4qJ70eb57/wSNgY81XTMSb92XAGWZU3Hw7t8Qfgx9W\n",
"fIgVllvf04BXlzzwNABIE4Af44FS7480bwlJHIU/qDzKjFnB5j0MT72cBmyWzTsll3fgael8S/wH\n",
"gI8Ch2H2jVg5Q4HPAWOBzcx4OFRfaj38QelDePP+d6Q+pXaWd+BpyaQh+K7I6fiuymjzXgfPIhmC\n",
"R8BGm/f2+Hb6LwD7Z/NOqae8A0+9k9bGI2D/AEwgeFSexLb4+vBzgfODW+KXA04GjgX2sMLujFw7\n",
"pYEiG3h6NWkbvPl+EvhUFRGwM/H87b3NmBe6dKmV8PnuNfBT4h+L1Kc0kNQ8hSLp85KekPTLegwo\n",
"NZAkpJl4894Ps/OCzXskvj57N2BcFc17Y+A+4K9ARzbvlJau5gMdJG0BLAKuNLO39fL1PNChFUgj\n",
"8SCqdYDdMYvlb4uu/O278J2VofxtldoNX9t9ohV2eaS2XfnJQaNmwMghsPA5WDA7Tw4aOPrlQAcz\n",
"u0vSurV+Tmog6c14870HP68ylr8tdgEuBk41I5S/rVLLA2cCewM7WGE/idS3K2/eEy+AuaMXvztt\n",
"fWkM2cRTl1yFMtBJO+F3zedjdnikeUsMkjgLzzN5XxXNezXgW/gSw7HZvLsbNaNn8wZ/vfb0xown\n",
"NaN+eYgpaVa3l/PMbF5/XDcthTQImAUchG/MuTdWTvf87U3NCOVvq9S78VUu1wKnZQTsK40c0vv7\n",
"I4b27zhSf5GfH9sRqemXBm5ms/rjOqlC0qrA1cBQPAI2lr8teuZvG6Hmq1IH42FWR1phN0RqB46F\n",
"S/iX0KLQCUepdXTe2M7rei2pWFZNTqEMNNI78S3xD+FhVNHmfSBwK57dPTPSvFVqsEp9Dl/jvVU2\n",
"76VZMBumvWLj0+GPwPw5jRlPakb1WIVyDbAV8FrgSeB0s8WrCHIVShOR9gc+DUzH7NpYKd3zt3cx\n",
"41eh+lJvxO/YnwAOtML+FakfiPxB5trTfdpk0bMwf04+wBw4KumdNTfwegwi9TGPgD0PmAzsillo\n",
"zb5EV/7234ADzXgmVF9qS+Aa4LPAxzICNqVl65dlhKnJSW/AI1z/ic93hw4/kOjK374QOLuKCNjp\n",
"wKnAQVbYdyLXTiktXTbwdiZthjfvS4AzscrvfDu3xHflbx9sxrdCl/YI2LnAxsBEK+yPkfqU0rJl\n",
"A29HHgF7FL5M8BDMbo6VMwzfFfkOPAL2kVB9qfXxVSo/ByZZYf+J1KeUKpOrUNqNNBS4HDgS2KyK\n",
"5r0e8ANgEDCxiuY9Gfghftd/UDbvlPpO3oG3E480+CrwW2AiFsvPltgeuBI4C5hTRQTsh/G/OHaz\n",
"wu6OXDulFJcNvF1I2wFfBD4GXBBMEVwOOAWf897DjFD+tkqt3HntVfEt8X+N1KeUqpMNvNX5fPdJ\n",
"wAxgL8y+HytnZTx/+/X4lvhQhKtKjcHv+r8D7G6F/TdSn1KqXjbwViathB83tiYwDrNHY+VsDNwI\n",
"3A7saUao+arUnvja7g9ZYVdGalNKtcsG3qqkjfDm+31gH8xi+dtid3xt90wzvhCq9QjYc4Ddgfda\n",
"YQ9E6lNK9ZENvBVJu+CrPE7G7LJYKcvjDyn3AnYwIxThqlKr4xt7XsDnu/8eqU8p1U828FbiEbBn\n",
"AvsCUzD7cayc1fEt7QaMNeNvofpSm+J5JlcBp1thL0XqU0r1lQ28VUir4fnbg4CxmD0VK2cs3nyv\n",
"AU4zI9R8VeowfNrkCCvsxkhtSqlvZANvBdK78JUe1wGnYLHDDyQOBc4FjjQjFOGqUisCs4EtgS2t\n",
"sN9E6lNKfScbeLOTDgI+CRyN2XWxUlYELsBP+djSjF+H6kuNwu/aFwDjrLCFkfqUUt/KBt6spMF4\n",
"dvd2QAdmsfxt0ZW//TgwzoxQ/rZKdeDTLecDH7eij3OHU0ph2cCbkbQm3nyfwiNgY/nbYiu8+c4B\n",
"zq0iAvZ44ERgfyvs9si1U0r9p+YwK0k7SPqNpN9LOqkegxrQpC3wI89uBnaJNG8JSRyHL/M72Ixz\n",
"gs17ON7Dkgc4AAAPEklEQVT49wPGZ/NOqbnVdAcuX9b2GWBb4DHgx5JuMrPQXGsjjZGmjIIZI2HI\n",
"QnhuAcx+0Kz/j63yLfHHAqcBB2H27Vg5w/H87Y2ACWb8KVRfagP8Qen9wOZWWPjwXD8CbNQMP1F9\n",
"4XOwYHYeAZZS36l1CmUc8LCZ/QlA0rXAThB7WNYoY6QpE+GCuTC6671psP4YiX5t4tIw4GLgbXiK\n",
"4B9i5ayP78p8AJhkRqj5qtRU4PPA6cDF1cx3e/OeeAHMHb343WnrS2PIJp5S36h1CuWN+AqFLo92\n",
"vtcSRsGM7s0bYC6MXtuPAesf0pvw/Gzh+d3R5j2ls/4ifNqk4uatUsup1Cx8S/1OVthF1T+sHDWj\n",
"Z/MGf712//0sUxpgar0Dr+g/dkmzur2cZ2bzarxuXYyEIb29PwKG9ssApB3wJMAzgc9UEQH7EWAa\n",
"sKsZPwhdutRr8B2VI4FNrbDHI/WvNrLXn6WfqJ5SWhZJHfiS34rV2sAfA0Z1ez0Kvwvvwcxm1Xid\n",
"PrEQnuvt/UXEpiDCpOXwg36PAnbH7K5YOavg+duvwSNgQ/nbKvV2fL77m8BMK+yFSH3vFvb6s4RF\n",
"ffuzTKlNdN7Yzut6LalYVk2tUyj3AxtIWle+bnkv4KYaP7PfLIDZ0+Dh7u8dDo/M9+V3fUNaGW+e\n",
"U/AlgtHm/TZ8lcofgfdU0bz3Bb4LFFbYcfVp3gALZsO0h3u+d/gjML/vfpYpDXAK/Ku99w+QJuOb\n",
"PQYBl5nZOa/4upmZarpIHxojTVkbpo+AoYvg2fkwp88eYErd87ePx2KHH0jsha/6+T8zvhiqLbUC\n",
"8HFgR2BXK+znkfrKxjdmis95jxjqd97z5+QDzJSqU0nvrLmB12MQA4K0B/A5YCZmX4iVsjyeZbIL\n",
"sJsZofxtlXo98BXg38B+Vtg/IvUppf5XSe/MnZh9TVoeOBvYE9ges5/GynkdvjHneTwC9ulQfakJ\n",
"eAjW5UCZEbAptY9s4H1JWh24FvgfHgEby98W4/HmeyVQRCJgO7fEfwD4KHCYFfaNyLVTSs2v5q30\n",
"aQmksfhD3nuBHapo3tOAbwDTzWL53So1BLgUX8++eTbvlNpT3oH3Bakrf/sIzL4aK2UIvgpmErCF\n",
"Gb8N1ZdaG7gB+AMwwQpbFKlPKbWOvAOvJ2lFpIuAmcCWVTTvUcCdwCrA+Cqa9zbAffi0zd7ZvFNq\n",
"b3kHXi/SWngE7F+A8ZjF8rfF1viRaZ8CPmlW2S5XeHm++wTg/4B9rbDvRa6dUmpN2cDrQerK354N\n",
"nBvcEi+88c4E9jPju6FLlxqJB1Gtg0fAzo/Up5RaVzbwWngE7HHAScCBmN0aK2cEcBmwPj5l8udQ\n",
"fak347s678HPq1zCdvaUUjvKBl4tqUf+Np2RupWXswG+K/M+YHOz3nNZllhfamfgEuDDVtjcSG1K\n",
"qT1kA6+GNBq/830AmITFDj+QeD9+5/0R4JLgfPcgoAQOBN5vhd0buXZKqX1kA4+S3ofvapwFXBic\n",
"7x4EFMAhwI5m/Ch06VKr4g86h+ARsE9E6lNK7SUbeKU8ArYrf3tnzH4YK+c1wNXAcHxLfKj5qtQ7\n",
"8bv+G4GTrLAXI/UppfaTDbwSUlf+9ir4lvjQ4QcS78Cb703AiWaEIlxVan/g08B0K+zaSG1KqX1l\n",
"A18W6W14870FOAGL5WdL7AtcAMww45pQrUfAngdMBt5jhf0yUp9Sam/ZwJdG2hvf1n48ZlfFSlkB\n",
"+AQwFdjGjF+E6ku9AY+A/Sc+3/3PSH1Kqf1lA++NR8B25W9vh9nPYuWsgTffhfiRZ6H8bZXarLP+\n",
"EuBMK+x/kfqU0sBQdQOXH1AwC18HvakFc66bltQjfxuzWP62mIg338uAM8youPl2bok/Cv+5HmKF\n",
"3Ry5dkppYKnlDvyX+B3qxXUaS+NJXfnbVwCzsMoPP+jcEn8EcAZwmBmhCFeVGgpcCLwL2MwK63G+\n",
"pB9XNmqGn/6+8DlYMDuPK0tpYKu6gZvZb6BrN3kbkKYBZwHTMPt6rJShwGeBccAkM34fqi+1Lv6g\n",
"9LfARCvs3z0/f8wUmHgBzB29+N1p60tjyCae0sCVcbLSEKS5eKbJFlU073WAu4BhwIQqmvd2wI/w\n",
"U3f2fWXzdqNm9Gze4K/Xnh65VkqpvSz1DlzSbcAavXzpVLPKT3mRNKvby3lmNq/S2j4ljcIPP/gz\n",
"nmeyMFbOtsBV+Gnvn64iAvYkYAawlxX2/SV/98ghvb8/Ymjlo00pNTNJHUBHpGapDdzMtqthPN0/\n",
"Z1Y9PqeupB7521VEwM4Ejgf2MeOO0KVLrQR8AVgTGGeFPbr0ioVLCLpaFMpgSSk1r84b23ldryUV\n",
"y6qp1xRK60yES0I6AW/e+2P2iWDzHok/6NwNGFdF894IPyfzSWCrZTdvgAWzYdrDPd87/BGYPydy\n",
"7ZRSe1Ggd/UslHbBDzBYDXgGeMDMJvfyfWZmzdHgpa787TcBu2Gxww8k3oxnkdyNHzb8fKi+1C74\n",
"2u6TrbDLYtceM8XnvEcM9Tvv+XPyAWZK7auS3ll1A6/nIPqFtCG+0uNe4BgsdviBRFf+9qlmXBqq\n",
"9QjYM4F9gd2ssPsj9SmlgaeS3jkwdmJKOwKXAqcBc6uIgD0DOACYasZ9oUuXWg2frhkEjLXCnorU\n",
"p5TSkrR3A5d65G9jFsvfFq/FI2AH4xGwT4bqS70LX+VyHXBqRsCmlOqpfRu4tCq+xG84viU+lr8t\n",
"NsGb7w3AKWaEmq9KHQR8EjjaCrsuUptSSpVozwYudeVvfx04qYoI2APw5YXHmvHlUG2pwXh297ZA\n",
"hxX2q0h9SilVqv0auLQfcD4wA7NY/rYYjOdvbw9sbcaDofpSawLXA0/h67ufidSnlFJE+zRwqUf+\n",
"Nmax/G3xBnyu+ml8fXcof1ultgCuBT4HnJMRsCmlvtYeDVzqkb+NWSx/W0zCI2QvBs6qIgL2WHyF\n",
"y0FW2Lcj104ppWq1fgOXeuRvY5Xf+XZuiT8aOB04xIzQxhiVGoY3/bfhKYJ/iNSnlFItWreBe47t\n",
"kUAJHIrZN2PlDAUuAt4JbGbGI6H6Um/CH5Q+iOd3/ydSn1JKtWrNOFlpKPB5/O55syqa93rAD/G/\n",
"wKpp3jsA9+B3/Qdk804pNULr3YFL6+B3vr/DI2B7yc9eWjnvxbO3zwFmByNglwNOxY89280Kuzty\n",
"7ZRSqqfWauDSdsAX8QOHz68iAvYU/IHjnmbcGbp0qZXxo9ZWx0+J/0ukPqWU6q01GrjPd5+In5qz\n",
"N8EDISRWwpvvGvgp8Y+F6kttjKcQ3g7saYX9N1KfUkp9ofkbuDQSuBwYhS8RrCA/u3s5b8Gb7x3A\n",
"3lVEwO6Br+0+wQq7IlKbUkp9qbkbuLQR3nzvBPbDLNZ8xW74Se8nmXF5qLbU8sDZwB7A9lbYTyP1\n",
"KaXU16pu4JK6dj3+F3gEOMSsjlvH9fLhB6dgFsvf9gjYs4C9gclm/CRUX2p1fFflS3gE7N8j9Sml\n",
"1B9qWUZ4K/BWM3sHviLklLqMSBqEdDaeZ/K+Kpr3asC3gbF4BGxFzbvzQFFUaixwP37ww+Rma95d\n",
"42xmrTBGyHHWW46z/1XdwM3sNlu86/FeYK2aRyO9FvgWMAGf744dniDejTffnwA7mPG3QHmHSh3a\n",
"ef3jrbBTrbCXItfvJx2NHkAFOho9gAp1NHoAFepo9AAq1NHoAVSoo9EDqJd6zYEfCoSS/15FLx9+\n",
"cD0+bRLL3xYH42FWR5lxfai21Iq8g6l4dviWVtivI/UppdQIS23gkm7Dl9690qlm9o3O7/kw8F8z\n",
"+1LVo5AOxGNcj8HsK7FSBuPTLdsAW5nxUKi+1FrA9SzPcGC8FfavSH1KKTVKTYcaSzoYmAZsY0s4\n",
"JFhS356anFJKbarPDjWWtAMwE9hqSc27kgGklFKqTtV34JJ+jx/2+3TnW/eY2dH1GlhKKaWlq2kK\n",
"JaWUUuP0S5yspE9I+rWkn0v6qqSV++O6EZL2kPQrSS/JV8Q0FUk7SPqNpN9LOqnR4+mNpM9LekLS\n",
"Lxs9lqWRNErSHZ1/3g9KmtHoMfVG0hBJ90r6maSHJJ3T6DEtiaRBkh6Q9I1Gj2VpJP1J0i86xxpa\n",
"ptxfJK0i6frOnvmQpAlL+t7+ygPvm00/9fVLYBeIpRT2B0mDgM8AOwAbA/tIektjR9Wry/ExNrsX\n",
"gOPN7K34noNjmvHn2flsaWszeyfwdmBrSZs3eFhL8kHgIag8nrlBDOgws03MbFyjB7MEFwC3mNlb\n",
"8D/3JS5r7pcG3iebfurMzH5jZr9r9DiWYBzwsJn9ycxewLf579TgMb2Kmd0FhM4jbQQze9zMftb5\n",
"/xfh/4Gs2dhR9c7s5cNCBgODWPzMqWlIWguYAlwKtMKihaYdY+fsxBZm9nkAM3txaREljTiR51CI\n",
"nT2ZeCOwoNvrRzvfSzWStC6wCX5j0XQkLSfpZ8ATwB1mFtrn0E8+ja9Iq/g82gYy4HZJ90ua1ujB\n",
"9GI94ClJl0v6qaS5koYt6Zvr1sAl3Sbpl738en+376l9008fj7FJNfs/S1uSpBH4zt8Pdt6JNx0z\n",
"+1/nFMpawJbNluMhaSrwpJk9QBPf2XYzycw2ASbjU2dbNHpAr7A88C7gc2b2LuDfwMlL++a6MLPt\n",
"lvb1zk0/U/Adkw2xrDE2scfwPPQuo/C78FQlSSvg0Q1XmdnXGj2eZTGzZyTdjIe0zWvwcLrbDNhR\n",
"0hRgCLCSpCvN7MAGj6tXZvbXzv99StKN+PTkXY0dVQ+PAo+a2Y87X1/PUhp4f61C6dr0s9PSNv00\n",
"kWa7k7gf2EDSupIGA3sBNzV4TC1LfsLTZcBDZnZ+o8ezJJJWk7RK5/8fCmwHPNDYUfVkZqea2Sgz\n",
"Ww+Pb/5eszZvScPkB8QgaTjwXnzxQtMws8eBBZI27HxrW+BXS/r+/poDnwOMAG7rXL7zuX66bsUk\n",
"7SJpAb4q4WZJ32r0mLqYB3sdC3wHf9L/ZbPmC9ySdA3wQ2BDSQskHdLoMS3BJGB/fFXHA52/mnH1\n",
"zBuA73XOgd8LfMPMvtvgMS1LM0/3vR64q9vP85tmdmuDx9Sb6cDVkn6Or0I5e0nfmBt5UkqpRTVi\n",
"FUpKKaU6yAaeUkotKht4Sim1qGzgKaXUorKBp5RSi8oGnlJKLSobeEoptahs4Cml1KL+H2VQxj6V\n",
"FLQ4AAAAAElFTkSuQmCC\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x106476a90>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"def plotSVM(X,t,wb):\n",
" plt.plot(X[t>0,0],X[t>0,1], 'ro')\n",
" plt.plot(X[t<0,0],X[t<0,1], 'bo')\n",
" plt.axis([-2, 6, -2, 7])\n",
" x = np.arange(-2,6,1)\n",
" y0 = -1*wb[0]/wb[1]*x-wb[2]/wb[1]\n",
" y1 = -1*wb[0]/wb[1]*x-(wb[2]+1)/wb[1]\n",
" y_1 = -1*wb[0]/wb[1]*x-(wb[2]-1)/wb[1]\n",
" plt.plot(x,y0)\n",
" plt.plot(x,y1)\n",
" plt.plot(x,y_1)\n",
" \n",
"plotSVM(X_,t_,wb_opt) ### PROVIDE THIS PLOT ###"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## New Data Set\n",
"\n",
"You are now given the following new data set."
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Generating the data\n",
"N_half = 20\n",
"u1 = random.rand(N_half)\n",
"x1 = cos(2*pi*u1) + .3*random.rand(N_half)\n",
"y1 = sin(2*pi*u1) + .3*random.rand(N_half)\n",
"u2 = random.rand(N_half)\n",
"x2 = 2.*cos(2*pi*u2) + .3*random.rand(N_half)\n",
"y2 = 2.*sin(2*pi*u2) + .3*random.rand(N_half)\n",
"\n",
"# New Data\n",
"X_c = transpose(vstack((hstack((x1,x2)),hstack((y1,y2)))))\n",
"t_c = hstack((ones(N_half),-ones(N_half)))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Remark that the data is distributed on two circles, centered on zero, with different radii."
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[-3, 3, -3, 3]"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAAEwtJREFUeJzt3W2MXOV5xvHrwgTZyFZSwgcHWAvJQY3oUkhKQ5FSWClx\n",
"7Z22IkQKEVKrpsj+0EheFDVtAlbBbYSIiqqC6ZcIAaVEJKpIhIKy3tZCrCgSpiTBCcamsS1FdiCA\n",
"Uhpky1gJcPfDzIZZe972vMw5z5n/T0Ls2ZmdeYZlrnn2ft4cEQIApOesqhsAAMiGAAeARBHgAJAo\n",
"AhwAEkWAA0CiCHAASFSuALe92vaztvfZPmD7zqIaBgAYzHnngds+NyJO2j5b0tOSvhQRTxfSOgBA\n",
"X7lLKBFxsvPlOZJWSXoj72MCAIbLHeC2z7K9T9Jrkp6MiAP5mwUAGKaIHvi7EXGFpIskXWN7Jner\n",
"AABDnV3UA0XEm7a/J+lKSYtL37fNZisAkEFEeNDtuQLc9vmS3o6IX9peI2mTpL9faSNSZntnROys\n",
"uh1lafLra/Jrk3h9qRul85u3B/4hSQ/ZPkvtcszDEfFEzscEAIwgV4BHxAuSPlZQWwAAK8BKzPwW\n",
"q25AyRarbkCJFqtuQMkWq25AyRarbkDVci/kGfoEdjS5Bg4AZRglO+mBA0CiCHAASBQBDgCJIsAB\n",
"IFEEOAAkigAHgEQR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAAkCgCHAAS\n",
"RYADQKIIcABIFAEOAInKdSo9kDp7uiVNzUnrVkvHT0nHdkXsn6+6XcAoCHBMrHZ4X32PdN+H3/vu\n",
"to32tAhxpIBT6TGx7NkFaffmM29pLUTMz5b//PT+0d8o2UkPHBNs3ere31+7puxnpvePIjCIiQl2\n",
"/FTv7594q/znnppbHt5S+3rD9vKfG01BgGOCHdslbTv83vVTkj59Ulp9oT270O4ll6Vf73/NVeU+\n",
"L5qEEgomVsT+eXtaUmu79PaF0kUbpcfOlXRZ+58ySxr9ev+X/pZ03j2UUjCKXD1w21O2n7T9ou39\n",
"tueKahgwDhH759sDlqtekR44d/mtZZY0Tu/9S9KtkjaV/Lxokrw98F9L+mJE7LO9VtIPbO+JiIMF\n",
"tA0YqNhZHOMd0Hyv9/8nD0tXnie9I2mLpGtKfV40S64Aj4hXJb3a+fqE7YOSLpBEgKNUxc/iGP+A\n",
"ZjvEZ5+TdvaYyjiOgVSkrrBBTNsXS/qopGeLekygv6JncfQqaWw9Ih29N9vj1f150QSFDGJ2yieP\n",
"Sro5Ik4U8ZjAYMWWPJYPaK5d0+4BH7237IHEqp4XzZA7wG2/T9K3JX0jIh7rc5+dXZeLEbGY93kx\n",
"6YoveXRCc+zBWdXzol5sz0iaWdHP5FlKb9uSHpL0vxHxxT73YSk9Cte7Br71iLR3jt4rmmCU7Mwb\n",
"4J9Qe/XDjyUtPdAtEbGwkkYAWbRDfAOlBzRS6QFeVCMAAMuNkp0spQeARLGUHkP1WjDTvoWtUIEq\n",
"EeAYqPdg4Wcukz5g6YEPvfc9tkIFxo0SCobotWDm0guWh7fE/h3A+BHgGKLXgpl+f7ixfwcwTgQ4\n",
"hui1YObtPvdl/w5gnAhwDNFrr44Xfy7d9PPl32P/DmDcmAeOoXotmGnfwiKaJuKw5XpgIQ+AFemz\n",
"Te9h6ZmbCfHxYiEPgBXisOWUEOAAuoz3ZCLkQ4AD6DL+k4mQHQEOoAsnBKWEQUwAy7BNbz0wCwUA\n",
"EsUsFABoMAIcABJFgANAoghwAEgUAQ4AiSLAASBRHKkGZDBtt6akuXXS6uPSqWPSrv0RzJXGWBHg\n",
"wApN262rpXvuk36z6dM2aeO0rdNDnKBHmQhwYIWmpLnu8Jak+6QPt6Ttkn4TzisJeiALauDACq2T\n",
"eu7Yt1ZatmNfv6Df0A56IDd64DXEiSj1dlzquWPfCemt7pLJKul3n5J0zWn3Oz3ogawI8JrpcyLK\n",
"RntahHg9HJN2bZM2dveut0pHjkp7Ty+Z7Oj8uzvET0hszYpCsJlVzdizC9LuzWfe0lqImJ8df4vQ\n",
"y7Td2iBtXyutOSG9dVS6d0qa2y2d8bv7O0lf7Xy9VTqyV5qjBo5hRslOeuC1w4koKegE8LIQvsH+\n",
"2173PSC9cYP0wlLQE94oSu4At/2ApD+W9HpEXJa/SZOOE1HGoYzpff1q429J/z0fwV9PKFwRs1Ae\n",
"lLSlgMeBJE5EKd/S9L7d0uZ/l67dLW2+Wrpn2m7ledxObXzZ765TG+d3h1IUUgO3fbGkx3v1wKmB\n",
"rxwnopRr1l7oVatuSQt5e8q9auOUTJAFNfBEdcKaN31JRp3HnUWv2vhKsHITKzGWALe9s+tyMSIW\n",
"x/G8QC+D5nGPuy3dWLk52WzPSJpZ0c9QQsGk6RWUdZjeV2ZpB+mhhAL0sD9iftpWq2a16jJLO6li\n",
"VfJgRUwj/KakayV90PYxSbdFxIO5WwaUKG+tugx1Le1UhVXJw+UO8Ii4sYiGAJNuwBL9ZdMQBw10\n",
"NmsQdGpueXhL7evWsl0fJxklFKAmRintDBrolKRmDYKyKnkYAhyokWGlnUF7kYfkUfYpTwerkodh\n",
"P3AgIYMGOps3CMqq5GHogQMJGTTQGVLPKWepDoJG7J+3p9WuebMquRcCHEjIsIHOUQZBU8Kq5MHY\n",
"DxxIzKD9VtiLpTlGyU4CHABqaJTsZBATABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAA\n",
"kCiW0gMN1qz9wXE6AhxoKA5Jbj5KKEBD9ds7fEN7f3A0AAEONFTz9gfH6Sih5MCJ2aizST4keVLe\n",
"mwR4RpyYjbob9ZDkppmk9ybbyWZkzy5IuzefeUtrIWJ+dvwtAs40ifuDN+W9OUp20gPPjBOzUX/D\n",
"Dklupsl5bzKImRknZgP1NDnvTQI8M07MBuppct6b1MBzaA+WbODEbKBmlr83f7m2PSFn/YmUZqRw\n",
"JiZKxTJt1F2fGSmHpWdurnuIM4iJ0rBMG2mYmlse3lL7urVdDRjczV0Dt73F9ku2D9n+chGNQv2x\n",
"TBtpaPaMlFw9cNurJP2LpE9JelnSc7a/GxEHi2gcqtevTMIybaSh2TNS8pZQPi7pcET8VJJsf0vS\n",
"dZII8AYYVCaZmuBl2kjJsV3Sto3LyyjNmZGSN8AvlHSs6/pnkq7K+ZioiX5lkpa0/eiELtNGWiL2\n",
"z9vTate8mzdbLG+AjzSFxfbOrsvFiFjM+bwYg0Flkv0R89O2WhO2TBvp6YR17f+/tD0jaWYlP5M3\n",
"wF+WNNV1PaV2L3yZiNiZ83lqqek7ng3bzW4yl2kjdXV933Y6totL17ZvH/YzeQP8+5IusX2xpFck\n",
"fU7SjTkfMwmTsOPZpO5mh+Zq2vs290Ie27OS7pa0StL9EXHnabc3ciFPU3Y8GybLbnYs8EFdpfS+\n",
"HctCnojYLWl33sdJT7Pnly5ZaZmEBT6ot2a9b9nMKrNmzy/NigU+qLdmvW8J8MwmZ8ezlWCBD+qt\n",
"We9b9kLJqOnzS7Oa5HMYUX9Ne9+yGyEK1asGvlU6sleaowYOjI7tZFGJSTyHESgaAQ4AiWI/cFSK\n",
"+eBAuQhwlIL54NXhg3NyEOAoxaCdDMX+KaWYtlvnSf9wlXTp/V3TNvngbC7mgaMUzAcfr6W/eK6R\n",
"fu/+0/4bs5CquQhwlIL54OO19BdPvz+p+eBsJgIcpejsZLhsxRs7GZZn6S+et/vczgdnM1EDRyk4\n",
"8GG8lv7i+SNJOyTd0XUbH5zNxTxwoAG6Z/08JWmPpEPSqTekF1+RbmvKB2ddD2MoA/PAgQkxCX/x\n",
"NO0whiLQAweQhJQOYygCPXDUFotNsHLNOoyhCAQ4xo5VmsimWYcxFIFphBWyp1v27IJ9w2L739Ot\n",
"qts0Dpzag2yadRhDEeiBV2SSB2RYpYksmnYYQxEI8MpMzS0Pb6l93Wr8XiFFrNKkhj6ZOmHN77mD\n",
"AK/M5A7IdFZpbjz91J5RF5ukXkPnwwdFIcArM7kDMnnnLKe802HqHz6oFwK8Msd2Sds2Li+jTM6A\n",
"TCesMgVWyjX0lD98UD8EeEUYkMkuaw29DqWLlD98UD8EeIUYkMkmSw29LqULttlFkQhwJCdLDb0u\n",
"pYu8A7hANwIcSVppDb0upYtJ2HQK45M5wG1/VtJOSR+R9PsR8cOiGgVkMajGXafSRZ4BXKBbnh74\n",
"C5Kul/T1gtoCZDasxj2gdLF31l5gTjZSlDnAI+IlSbLZKRbVG1bj7lO62Hu19GdVD2wCWVEDRyOM\n",
"UuM+vXQxay/UYWATyGpggNveI2l9j5tujYjHR30S2zu7LhcjYnHUnwVGkaXGXZeBTUCSbM9ImlnJ\n",
"zwwM8IjYlKM93Y+zs4jHAfrJMj2vTgObQKdju7h0bfv2YT9TVAmFQjgqlWV6HnOykbrMZ2Lavl7S\n",
"LknnS3pT0vMRcca5dJyJiTqbtlsbmJONGholOznUGABqaJTs5Eg1AEgUAQ4AiSLAASBRBDgAJIoA\n",
"B4BEEeAAkCgCHAASRYADQKIIcABIFAEOAIliP3AAA9nTLWlqTlq3Wjp+Sjq2K2I/+8XUAAEOoK92\n",
"eF99j3Rf18EX2zba0yLEq0cJBcAAU3PLw1tqX2/YXk170I0ABzDAup6nFklrObWoBghwAAMc73lq\n",
"kXSCU4tqgABHrdnTLXt2wb5hsf3v6VbVbZosx3ZJ2w4v/97WI9JRTi2qAQYxUVsMoI2mzFkiEfvn\n",
"7WlJre3tssmJt6Sj9/Lfvx44kQe1Zc8uSLs3n3lLayFi/ozj+yZRnw+5w9IzNxOyaeNEHiSOAbTh\n",
"mCUyyQhw1BgDaMPxITfJCHDUGANow/EhN8kYxERtMYA2imO7pG0bl5dR+JCbFAxiAolrD2Ru4EOu\n",
"YUbJTgIcAGqIWSgA0GAEOAAkigAHgEQR4ACQqMwBbvsu2wdt/8j2d2y/v8iGAQAGy9MD/09JvxMR\n",
"l0v6iaRbimkSAGAUmQM8IvZExLudy2clXVRMkwAAoyiqBn6TJBYOAMAYDVxKb3uPpPU9bro1Ih7v\n",
"3GeHpF9FxCMltA8A0MfAAI+ITYNut/15SS1Jnxxyv51dl4sRsTha8wBgMtiekTSzop/JupTe9hZJ\n",
"/yTp2oj4xYD7sZQeAFao1L1QbB+SdI6kNzrfeiYivpClEQCA5djMCgASxWZWANBgBDgAJIoAB4BE\n",
"EeAAkCgCHAASRYADQKIIcABIFAEOAIkiwAEgUQQ4ACSKAAeARBHgAJAoAhwAEkWAA0CiCHAASBQB\n",
"DgCJIsABIFEEOAAkigAHgEQR4ACQKAIcABJFgANAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAA\n",
"kKjMAW77q7Z/ZHuf7SdsTxXZMADAYHl64P8YEZdHxBWSHpN0e0FtSortmarbUKYmv74mvzaJ1zcJ\n",
"Mgd4RBzvulwr6Rf5m5OkmaobULKZqhtQopmqG1CymaobULKZqhtQtbPz/LDtOyT9uaSTkv6gkBYB\n",
"AEYysAdue4/tF3r886eSFBE7ImKDpH+V9M9jaC8AoMMRkf9B7A2S5iNiusdt+Z8AACZQRHjQ7ZlL\n",
"KLYviYhDncvrJD2fpQEAgGwy98BtPyrptyW9I+mIpL+KiNcLbBsAYIBCSigAgPEby0rMJi/6sX2X\n",
"7YOd1/cd2++vuk1Fsv1Z2y/afsf2x6puT1Fsb7H9ku1Dtr9cdXuKZPsB26/ZfqHqtpTB9pTtJzv/\n",
"X+63PVd1m4pie7XtZztZecD2nQPvP44euO11S/PGbW+XdHlEbC39icfA9iZJT0TEu7a/JkkR8ZWK\n",
"m1UY2x+R9K6kr0v664j4YcVNys32Kkn/I+lTkl6W9JykGyPiYKUNK4jtP5R0QtK/RcRlVbenaLbX\n",
"S1ofEftsr5X0A0mfbtDv79yIOGn7bElPS/pSRDzd675j6YE3edFPROyJiHc7l89KuqjK9hQtIl6K\n",
"iJ9U3Y6CfVzS4Yj4aUT8WtK31B6Ib4SI+C9J/1d1O8oSEa9GxL7O1yckHZR0QbWtKk5EnOx8eY6k\n",
"VZLe6HffsW1mZfsO20cl/YWkr43recfsJknzVTcCQ10o6VjX9c8630NibF8s6aNqd54awfZZtvdJ\n",
"ek3SkxFxoN99c63EPO1J90ha3+OmWyPi8YjYIWmH7a+ovejnL4t67rINe22d++yQ9KuIeGSsjSvA\n",
"KK+vYRi5b4BO+eRRSTd3euKN0PmL/orOeNp/2J6JiMVe9y0swCNi04h3fUSJ9VKHvTbbn5fUkvTJ\n",
"sTSoYCv43TXFy5K6B9Kn1O6FIxG23yfp25K+ERGPVd2eMkTEm7a/J+lKSYu97jOuWSiXdF32XfST\n",
"IttbJP2NpOsi4lTV7SlZUxZlfV/SJbYvtn2OpM9J+m7FbcKIbFvS/ZIORMTdVbenSLbPt/2Bztdr\n",
"JG3SgLwc1yyUxi76sX1I7cGGpYGGZyLiCxU2qVC2r5e0S9L5kt6U9HxEzFbbqvxsz0q6W+1Bovsj\n",
"YuB0rZTY/qakayV9UNLrkm6LiAerbVVxbH9C0lOSfqz3ymG3RMRCda0qhu3LJD2kduf6LEkPR8Rd\n",
"fe/PQh4ASBNHqgFAoghwAEgUAQ4AiSLAASBRBDgAJIoAB4BEEeAAkCgCHAAS9f963qCLZIi3PQAA\n",
"AABJRU5ErkJggg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x105f235d0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Plot new data\n",
"plt.plot(X_c[t_c>0,0],X_c[t_c>0,1], 'ro')\n",
"plt.plot(X_c[t_c<0,0],X_c[t_c<0,1], 'bo')\n",
"plt.axis([-3, 3, -3, 3])"
]
},
{
"cell_type": "code",
"execution_count": 45,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"[-1, 6, -1, 6]"
]
},
"execution_count": 45,
"metadata": {},
"output_type": "execute_result"
},
{
"data": {
"image/png": [
"iVBORw0KGgoAAAANSUhEUgAAAXAAAAEACAYAAACqOy3+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\n",
"AAALEgAACxIB0t1+/AAADx5JREFUeJzt3GuspVV9x/HvjzlO5sIgSa3cHIJNJFFsI7QaW7UeU6Ew\n",
"0VoTW2tibW2G+qIFalqVjlbmlb5p44XWN16ItcY2wdiIAl45ijGhIjMKDAiaIAMFtDME5orDzL8v\n",
"zj5xczyXvc/zzOxZM99P8mQ/z17rWes/Mzu/ec6atSdVhSSpPadMugBJ0soY4JLUKANckhplgEtS\n",
"owxwSWqUAS5Jjeoc4ElOT3J9knuS7Ejy8j4KkyQtbaqHMT4C3FhVb0oyBazvYUxJ0jLS5Ys8SZ4N\n",
"bKuq3+ivJEnSKLouoTwf+HmS65LckeTjSdb1UZgkaWldA3wKuAj4WFVdBOwDru5clSRpWV3XwB8C\n",
"Hqqq7w2ur2degCfxP1uRpBWoqizV3inAq+rRJDuTnF9V9wGvBe4et4jjWZKtVbV10nWsVMv1t1w7\n",
"WP+knQD1L/vw28culCuAzyZZDfwEeHsPY0qSltE5wKvqB8BLe6hFkjQGv4m5vJlJF9DRzKQL6GBm\n",
"0gV0NDPpAjqamXQBHc1MuoCjrdM+8JEmSKrlNXBJmoRRstMncElqlAEuSY0ywCWpUQa4JDXKAJek\n",
"RhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWqU\n",
"AS5JjTLAJalRBrgkNcoAl6RGTXUdIMkDwJPAYeBQVb2s65iSpOV1DnCggOmq2t3DWJKkEfW1hJKe\n",
"xpEkjaiPAC/g60luT3J5D+NJkkbQxxLKK6rqkSS/Dnwtyb1VdetwhyRbhy5nqmqmh3kl6YSRZBqY\n",
"HuuequqzgGuAvVX1L0PvVVW5xCJJYxglOzstoSRZl2TD4Hw9cAlwZ5cxJUmj6bqEcgbwhSRzY322\n",
"qr7auSpJ0rJ6XUJZcAKXUCRpbEd9CUWSNDkGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWqUAS5J\n",
"jTLAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQo\n",
"A1ySGmWAS1KjDHBJalQvAZ5kVZJtSW7oYzxJ0vL6egK/CtgBVE/jSZKW0TnAkzwP2AR8AkjniiRJ\n",
"I+njCfxDwLuAIz2MJUka0VSXm5O8DvhZVW1LMr1Ev61DlzNVNdNlXkk60QwydHqse6pWvmyd5APA\n",
"nwNPA2uA04DPV9XbhvpUVbm0IkljGCU7OwX4vMleDfxDVb1+3CIkSc80Snb2vQ/cXSiSdIz09gS+\n",
"6AQ+gUvS2CbxBC5JOkYMcElqlAEuSY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQB\n",
"LkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS\n",
"1KhOAZ5kTZLbkmxPsiPJB/sqTJK0tKkuN1fVwSSvqar9SaaA7yR5ZVV9p6f6JEmL6LyEUlX7B6er\n",
"gVXA7q5jSpKW1znAk5ySZDvwGHBLVe3oXpYkaTmdllAAquoI8JIkzwa+kmS6qmaG+yTZOnQ5M79d\n",
"kk52SaaB6bHuqao+C/gn4EBV/fPQe1VV6W0SSToJjJKdXXehPCfJ6YPztcDFwLYuY0qSRtN1CeUs\n",
"4NNJTmH2L4PPVNU3upclSVpOr0soC07gEookje2oL6FIkibHAJekRhngktQoA1ySGmWAS1KjDHBJ\n",
"apQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RG\n",
"GeCS1CgDXJIaZYBLUqMMcElqlAEuSY3qFOBJNia5JcndSe5KcmVfhUmSlpaqWvnNyZnAmVW1Pcmp\n",
"wPeBP66qe4b6VFWle6mSdPIYJTs7PYFX1aNVtX1wvhe4Bzi7y5iSpNH0tgae5DzgQuC2vsaUJC1u\n",
"qo9BBssn1wNXDZ7E57dvHbqcqaqZPuaVpBNFkmlgeqx7uqyBDyZ9FvAl4Kaq+vAC7a6BS9KYRsnO\n",
"rv+IGeDTwK6qeudKi5AkPdOxCPBXAt8GfgjMDfSPVXXzOEVIkp7pqAd4X0VIkp7pqG8jlCRNjgEu\n",
"SY0ywCWpUQa4JDXKAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLU\n",
"KANckhplgEtSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1CgDXJIa1TnAk3wqyWNJ7uyjIEnS\n",
"aPp4Ar8OuLSHcSRJY+gc4FV1K/B4D7VIksbgGrgkNWrqWEySZOvQ5UxVzRyLeSWpFUmmgemx7qmq\n",
"PiY+D7ihqn5zgbaqqnSeRJJOIqNkp0soktSoPrYRfg74LnB+kp1J3t69LEnScnpZQllyApdQJGls\n",
"LqFI0gnMAJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhpl\n",
"gEtSowxwSWqUAS5JjTLAJalRBrgkNcoAl6RGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY3qHOBJLk1y\n",
"b5L7k7ynj6IkSctLVa385mQV8CPgtcDDwPeAt1TVPUN9qqrStdCT2YuTTRvhyrVw9m44dw2sXgOr\n",
"D0KOwJFnwYE9cN9ueD/w0jPh3afB2v3AQTiyAQ4/DdkHT6+DQ+vgp3vh0C4447nw3PWQvbB/J3z/\n",
"HPjtDbDuINST8NPH4aq7qm4cp97kxZtg45WwYQ3sOQg7P1p1142/2r72bDhwFhx+BFb9Lzz4f3Du\n",
"H0KthYNrIKtgHbAaOFDw5GE47RCsPwhP7IKngF87c7bv/sBaZvv/Yj/suQ92v394XqklI2VnVa34\n",
"AH4XuHno+mrg6nl9qsscJ/txAWzaDPcX1LegtkDV0LFl6P0/gCf/Ag4v1F5Q7xg6/zeozfPG2jzU\n",
"PnfvG2DXBbBp9M/EBZtg8/3PHHrz/XDBpsXbtxS8u2BzwbcK/mpwbKl54wzah+/71uD46/rVMd/4\n",
"8Ny8Hh6tHaNkZ9cJ3gR8fOj6rcC14xbhsfhxKdw8l0rvnRe4c8f7Bq9/ukz7uH3nri+Dm0b/TFx6\n",
"88JDX3bT0u2vG7y+d+hYqN/7Frhequ9lI9fu4XE8HaNk51THp/yR1l+SbB26nKmqmY7znjQ2wJq5\n",
"88X+sFYNXtcu0z5u37nrUxfvvoANaxZ+/9S1S7evH7wu95FcqMKl+p46Ru3S5CSZBqbHuadrgD8M\n",
"bBy63gg8NL9TVW3tOM9Jaw8cnDt/epE+hwevB5ZpH7fv3PXexbsvYM/Bhd/fe2Dp9n2D18V+lcMV\n",
"zb9e7Dni8NC80vFt8GA7M3ed5Jrl7um6C+V24AVJzkuyGngz8MWOY2rITvjo5fBjgEuA985r3wJc\n",
"PHjdBXv+Eo4s1A7wjqHzVwOXzxtr81D73L13wu4H4doxKv4oXP7jeSP/BB68dvH2LcCLBhVdAjwy\n",
"OOb/aheq8OLBPe+Y13cLcNcjv5xXOvF02oUCkOQy4MPM/rz6yar64Lz2KnehdPLiZNO5cMUaOGeJ\n",
"XSg/2g3XAC89a7ALZR+/3IVyeHanyaHBLpQH98IvdsGZZ8Bz18227dsJd5wDF22A9U9BPQEPPA5/\n",
"t7JdKOdeMbt8sfcAPHjtr+5COfcKWHPOL3ehTD0MP9014i6UA/DkbjhYs7tQnloL+wa7UNYDT+2H\n",
"PT+C3de4C0WtGiU7Owd4H0VIkp5plOz0m5iS1CgDXJIaZYBLUqMMcElqlAEuSY0ywCWpUQa4JDXK\n",
"AJekRhngktQoA1ySGmWAS1KjDHBJapQBLkmNMsAlqVEGuCQ1ygCXpEYZ4JLUKANckhplgEtSowxw\n",
"SWqUAS5JjTLAJalRKw7wJH+S5O4kh5Nc1GdRkqTldXkCvxN4I/Dtnmo5LiWZnnQNXbRcf8u1g/VP\n",
"Wuv1j2LFAV5V91bVfX0Wc5yannQBHU1PuoAOpiddQEfTky6go+lJF9DR9KQLONpcA5ekRk0t1Zjk\n",
"a8CZCzRtqaobjk5JkqRRpKq6DZDcAvx9Vd2xSHu3CSTpJFVVWap9ySfwMSw6yXIFSJJWpss2wjcm\n",
"2Qm8HPhykpv6K0uStJzOSyiSpMk4JrtQWvzST5JLk9yb5P4k75l0PeNI8qkkjyW5c9K1rESSjUlu\n",
"GXxm7kpy5aRrGkeSNUluS7I9yY4kH5x0TeNKsirJtiRNblZI8kCSHw5+Df8z6XrGkeT0JNcnuWfw\n",
"+Xn5Yn2P1TbCpr70k2QV8K/ApcCLgLckeeFkqxrLdczW3qpDwDur6gJml+j+pqXf/6o6CLymql4C\n",
"/BbwmiSvnHBZ47oK2AG0+iN6AdNVdWFVvWzSxYzpI8CNVfVCZj8/9yzW8ZgEeINf+nkZ8OOqeqCq\n",
"DgH/CbxhwjWNrKpuBR6fdB0rVVWPVtX2wfleZj/AZ0+2qvFU1f7B6WpgFbB7guWMJcnzgE3AJ1hi\n",
"g0IDmqs9ybOBV1XVpwCq6umqemKx/n6RZ2HnADuHrh8avKdjLMl5wIXAbZOtZDxJTkmyHXgMuKWq\n",
"dky6pjF8CHgXcGTShXRQwNeT3J7k8kkXM4bnAz9Pcl2SO5J8PMm6xTr3FuBJvpbkzgWO1/c1xzHU\n",
"6o+NJ5QkpwLXA1cNnsSbUVVHBksozwN+v5X/lyPJ64CfVdU2GnyCHfKKqroQuIzZJbhXTbqgEU0B\n",
"FwEfq6qLgH3A1Ut17kVVXdzXWMeBh4GNQ9cbmX0K1zGS5FnA54H/qKr/nnQ9K1VVTyT5MvA7wMyE\n",
"yxnF7wF/lGQTsAY4Lcm/V9XbJlzXWKrqkcHrz5N8gdll0VsnW9VIHgIeqqrvDa6vZ4kAn8QSSgt/\n",
"q98OvCDJeUlWA28Gvjjhmk4aSQJ8EthRVR+edD3jSvKcJKcPztcCFwPbJlvVaKpqS1VtrKrnA38G\n",
"fLO18E6yLsmGwfl64BJmN1Ic96rqUWBnkvMHb70WuHux/sdqG2FTX/qpqqeBvwW+wuy/xP9XVS36\n",
"L8HHmySfA74LnJ9kZ5K3T7qmMb0CeCuzuze2DY6WdtWcBXxzsAZ+G3BDVX1jwjWtVIvLiWcAtw79\n",
"/n+pqr464ZrGcQXw2SQ/YHYXygcW6+gXeSSpUe5CkaRGGeCS1CgDXJIaZYBLUqMMcElqlAEuSY0y\n",
"wCWpUQa4JDXq/wHT38XUvMRKQQAAAABJRU5ErkJggg==\n"
],
"text/plain": [
"<matplotlib.figure.Figure at 0x106c8ddd0>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#X_transformed = 2.5*transpose(vstack((hstack((np.abs(x1),np.abs(x2))),hstack((np.abs(y1),np.abs(y2)))))) ### CHANGE THIS LINE ####\n",
"X_transformed = transpose(vstack((hstack((sqrt(x1**2+y1**2),sqrt(x2**2+y2**2))),np.zeros(2*N_half))))\n",
"### prbly other transormation\n",
"plt.plot(X_transformed[t_c>0,0],X_transformed[t_c>0,1], 'ro')\n",
"plt.plot(X_transformed[t_c<0,0],X_transformed[t_c<0,1], 'bo')\n",
"plt.axis([-1, 6, -1, 6])"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"(40, 2)"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"shape(X_transformed)\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"##Plotting the result in the transformed space. \n",
"(Lines are commented, because code does not compile, until X has been correctly transformed.)"
]
},
{
"cell_type": "code",
"execution_count": 46,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"ename": "ValueError",
"evalue": "all the input array dimensions except for the concatenation axis must match exactly",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mValueError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m<ipython-input-46-6c4875b82fa6>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mwb_opt_c\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0moptimizeSVM\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX_transformed\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mt_c\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 2\u001b[0m \u001b[0;31m#plotSVM(X_transformed,t_c,wb_opt_c) ### PROVIDE THIS PLOT ###s\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 3\u001b[0m \u001b[0;31m#plotSVM(X_transformed,t_c,[-2,2,-1])\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m<ipython-input-22-49e7f2f9b2de>\u001b[0m in \u001b[0;36moptimizeSVM\u001b[0;34m(X_, t_)\u001b[0m\n\u001b[1;32m 13\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 14\u001b[0m opt_result = optimize.minimize(objective, wb0, jac=jac,constraints=cons, #the optimizer\n\u001b[0;32m---> 15\u001b[0;31m method='SLSQP', options=opt)\n\u001b[0m\u001b[1;32m 16\u001b[0m \u001b[0mwb_opt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mopt_result\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m\"x\"\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;31m#result of the optimization\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 17\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;32m/Users/Maximus/anaconda/lib/python2.7/site-packages/scipy/optimize/_minimize.pyc\u001b[0m in \u001b[0;36mminimize\u001b[0;34m(fun, x0, args, method, jac, hess, hessp, bounds, constraints, tol, callback, options)\u001b[0m\n\u001b[1;32m 433\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mmeth\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'slsqp'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 434\u001b[0m return _minimize_slsqp(fun, x0, args, jac, bounds,\n\u001b[0;32m--> 435\u001b[0;31m constraints, callback=callback, **options)\n\u001b[0m\u001b[1;32m 436\u001b[0m \u001b[0;32melif\u001b[0m \u001b[0mmeth\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;34m'dogleg'\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 437\u001b[0m return _minimize_dogleg(fun, x0, args, jac, hess,\n",
"\u001b[0;32m/Users/Maximus/anaconda/lib/python2.7/site-packages/scipy/optimize/slsqp.pyc\u001b[0m in \u001b[0;36m_minimize_slsqp\u001b[0;34m(func, x0, args, jac, bounds, constraints, maxiter, ftol, iprint, disp, eps, callback, **unknown_options)\u001b[0m\n\u001b[1;32m 402\u001b[0m \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 403\u001b[0m \u001b[0ma\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mvstack\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma_eq\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0ma_ieq\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 404\u001b[0;31m \u001b[0ma\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mconcatenate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ma\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mla\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m 405\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m 406\u001b[0m \u001b[0;31m# Call SLSQP\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
"\u001b[0;31mValueError\u001b[0m: all the input array dimensions except for the concatenation axis must match exactly"
]
}
],
"source": [
"wb_opt_c = optimizeSVM(X_transformed,t_c)\n",
"#plotSVM(X_transformed,t_c,wb_opt_c) ### PROVIDE THIS PLOT ###s\n",
"#plotSVM(X_transformed,t_c,[-2,2,-1]) "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Feel free (not mandatory!) to rewrite the function plotSVM to plot the data and the separation curve, not in the transformed space (as above), but in the original space."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": []
}
],
"metadata": {
"annotations": {
"author": "",
"categories": [
"intelligent-systems-1-2015"
],
"date": "2015-04-23",
"location": "Beginning of next lecture",
"parent": "IS_SS2015",
"submission_date": "2015-04-30",
"subtitle": "Exercise Sheet 2, Support Vector Machines",
"tags": [
"IntelligenSystems",
"Course"
],
"title": "Intelligent Systems 1 - Summer Semester 2015"
},
"celltoolbar": "Edit Metadata",
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
MaxXximus92