Support Vector Machine 구현하기

29 minute read

개요

SVM은 널리 사용되는 머신러닝 기법 중 하나로 우리가 사랑하는 Scikit learn 라이브러리에서 사용 가능하다.
기본적인 분석은 Scikit learn에서 충분히 가능하지만 더 자세한 분석을 위해서 SVM의 구조를 코딩할 줄 알아야한다.
SVM의 기본적인 구조를 Python을 이용해서 코딩하였다.

SVM이란?

SVM은 두 개의 범주를 분류하는 기법 중 하나이다. 기본적으로 선형으로 분류하지만, kernel trick을 활용하여 비선형 분류도 어느정도 가능하다.
SVM은 산업공학적인 기법이 들어간 머신러닝 기법이다. 데이터 집합을 분리하는 초평면을 정하고, 해당 초평면과 가장 가까운 데이터 2개를 선정(Support Vector)하여 다음과 같이 나타낼 수 있다. w는 초평면의 법선 벡터이다. $\mathbf{w}\cdot\mathbf{x} - b=+1\,$ $\mathbf{w}\cdot\mathbf{x} - b=-1\,$
+1과 -1은 우리가 임의로 target variable을 분류하기 위해 라벨을 붙인 것이다. 위 두 식은 각각 초평면을 의미한다.
위의 식을 연립하여 두 초평면 사이의 거리를 구하면 $\tfrac{2}{|\mathbf{w}|}$ 이 나온다. 해당 거리를 마진이라고 부르며, 이 마진을 최대화하는 것이 우리의 목적이다.
해당 식은 $y_i(\mathbf{w}\cdot\mathbf{x}_i - b) \ge 1,\text{ for all } 1 \le i \le n.$ 의 Constraints를 지니고 있으며 결국 $\arg\min_{(\mathbf{w},b)}|\mathbf{w}|$
단, $y_i(\mathbf{w}\cdot\mathbf{x}_i - b) \ge 1,\text{ for all } 1 \le i \le n.$ 와 같은 최적화 문제로 표현할 수 있다.
위 문제의 w의 norm은 제곱근을 포함하고 있기 때문에 계산상의 편의를 위해 $\tfrac{1}{2}|\mathbf{w}|^2$ 로 대체하면 Quadratic Programming 최적화 문제가 된다.
라그랑주 승수법을 이용하면 위의 문제는 Saddle Point를 찾는 문제가 되는데 식은 다음과 같이 된다. $\arg\min_{\mathbf{w},b } \max_{\boldsymbol{\alpha}\geq 0 } \left{ \frac{1}{2}|\mathbf{w}|^2 - \sum_{i=1}^{n}{\alpha_i[y_i(\mathbf{w}\cdot \mathbf{x_i} - b)-1]} \right}$
여기서 알파는 Lagrangian multiplier로 Original Problem을 Lagrangian Primal로 변환시켜주는 변수이다. 모든 알파는 0보다 크거나 같다.
KKT 조건에 따라 $y_i(\mathbf{w}\cdot\mathbf{x_i} - b) - 1 > 0$ 를 만족하는 모든 점들에 대응하는 알파값들은 0이 되기 때문에 결국 다음을 만족하는 support vector들만이 영향을 끼치게 된다. $y_i(\mathbf{w}\cdot\mathbf{x_i} - b) = 1$
이 식에서 우린 $\mathbf{w}\cdot\mathbf{x_i} - b = \frac{1}{y_i} = y_i \iff b = \mathbf{w}\cdot\mathbf{x_i} - y_i$ 를 알 수 있고, 결국 b는 y와 x에 의존하는 변수이다. support vector 중에서 b가 가장 작은 x,y를 고르는 것이다.

코드 구현

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

data_ = pd.read_csv('D:/정석-한양대/4학년 2학기/응용데이터애널리틱스/creditcard_dataset.csv', encoding = 'cp949', index_col = 0)
columns_ = data_.columns

data_.head()

	Time	V1	V2	V3	V4	V5	V6	V7	V8	V9	...	V21	V22	V23	V24	V25	V26	V27	V28	Amount	Class
190368	128803	-2.272473	2.935226	-4.871394	2.419012	-1.513022	-0.480625	-2.126136	1.883507	-1.297262	...	0.718504	0.893850	-0.031632	0.322913	-0.058406	-0.411649	0.573803	0.176067	175.90	1
9179	13126	-2.880042	5.225442	-11.063330	6.689951	-5.759924	-2.244031	-11.199975	4.014722	-3.429304	...	2.002883	0.351102	0.795255	-0.778379	-1.646815	0.487539	1.427713	0.583172	1.00	1
42700	41204	-8.440284	6.147653	-11.683706	6.702780	-8.155839	-3.716264	-12.407313	5.626571	-6.232161	...	2.192855	-0.282597	0.008068	0.403858	-0.018788	0.522722	0.792691	0.067790	30.26	1
239501	150139	-6.682832	-2.714268	-5.774530	1.449792	-0.661836	-1.148650	0.849686	0.433427	-1.315646	...	0.220526	1.187013	0.335821	0.215683	0.803110	0.044033	-0.054988	0.082337	237.26	1
154286	101051	-1.465316	-1.093377	-0.059768	1.064785	11.095089	-5.430971	-9.378025	-0.446456	1.992110	...	1.160623	-1.259697	-15.981649	-0.883670	-3.536716	-0.592965	0.675525	0.424849	0.92	1

5 rows × 31 columns

Kernel Function

SVM은 어떤 커널을 사용하느냐에 따라 다양한 분석이 가능해진다.
SVM은 기본적으로 선형 분류만 가능하지만, 데이터의 차원을 높여서 비선형 분류가 가능하게끔 하는 것이 Kernel trick이다. $|\mathbf{w}|^2 = \mathbf{w}^T\cdot \mathbf{w}$ 에서 $\mathbf{w} = \sum_{i=1}^n{\alpha_i y_i\mathbf{x_i}}$ 로 치환하면 쌍대형식의 최적화 문제로 바뀐다. $\tilde{L}(\mathbf{\alpha})=\sum_{i=1}^n \alpha_i - \frac{1}{2}\sum_{i, j} \alpha_i \alpha_j y_i y_j \mathbf{x}_i^T \mathbf{x}_j=\sum_{i=1}^n \alpha_i - \frac{1}{2}\sum_{i, j} \alpha_i \alpha_j y_i y_j k(\mathbf{x}_i, \mathbf{x}_j)$ 를 최대화하는 문제이다.
단, $\alpha_i \geq 0, ~~~\sum_{i=1}^n \alpha_i y_i = 0, ~~~\text{ for all } 1 \le i \le n.$ 의 Constraint를 갖느다. 이때 제약조건은 b를 최소화하는 과정에서 생기는 조건이다.
위 식에서는 kernel을 $k(\mathbf{x}_i,\mathbf{x}_j)=\mathbf{x}_i\cdot\mathbf{x}_j$ 로 정의하여 $\mathbf{w} = \sum_i \alpha_i y_i \mathbf{x}_i.$ 의 식이 얻어진다.
이를 통해 우리는 kernel이 어떤 의미를 가지는데 대충 알 수 있다.
이번 구현에서는 linear kernel과 rbf kernel을 구현해볼 것이다.

def Kernel_(x, y, params = 0, type_ = 'linear'):
    if type_ == 'rbf' :
        Kernel = np.exp(- (np.sum(x **2, axis = 1).reshape(-1,1) + np.sum(y **2, axis = 1).reshape(1,-1) - 2 * x @ y.T)* params)
        return Kernel
        
    elif type_ == 'linear':
        Kernel = x @ y.T
        return Kernel 

Cvxopt solver를 이용해서 SVM의 쌍대문제를 풀고 알파와 w, b를 구한다.

P, q, G, h, A, b는 모두 minimize문제에 맞춰진 계수들이다 즉 , soft-margin의 듀얼식은 max문제인데 이에 목적식에 -1을 곱하여 minimize문제를 만들어준다. 그리하여 원래 soft margin식과는 목적식의 부호가 다른 것을 알 수 있다.
P : max 듀얼 문제의 alpha에 대한 목적식 중 2차항에 해당되는 계수에 -1을 곱한 것을 계수들을 matrix시킨 것을 의미한다.=> 여기서는 minimize(primal)문제로 해석하여 식을 푼다
q : max 듀얼 문제의 alpha에 대한 목적식 중 1차항에 해당되는 계수에 -1을 곱한 계수들의 vector를 의미한다. => 여기서는 minimize(primal)문제로 해석하여 식을 푼다
G : 듀얼문제에 constraint에서 alpha에 대한 부등식에서 alpha에 대한 계수를 가진 Matix를 의미한다. 여기서는 0보다 크거나 같다에 대응되는 식에는 -를 곱해주어 부등호 방향을 맞추어준다. C보다 작거나 같다는 그대로 계수가 1이 된다.
h : 듀얼문제에 constraint에서 alpha에 대한 부등식에서 우변에 존재하는 상수항들의 vector를 의미한다. 0과 C가 있다.
A : 듀얼문제에 constraint에서 alpha에 대한 등식에서 alpha에 대한 계수(여기서는 y에 해당)vector를 의미한다.
b :듀얼문제에 constraint에서 alpha에 대한 등식에서 우변에 존재하는 상수항들의 vector를 의미한다.(여기서는 다 0)
우린 Hard Margin model이 아니라 Soft margin model을 만들 것이다.
C는 soft margin에서 어느정도만큼 틀리는 것을 봐줄 것인가를 결정하는 하이퍼 파라미터이다. c가 커질 수록 잘못 판별한 데이터의 영향력이 커지므로 적당한 c는 과적합을 방지할 수 있다.

$\arg\min_{\mathbf{w},\mathbf{\xi}, b } \left{\frac{1}{2} |\mathbf{w}|^2 + C \sum_{i=1}^n \xi_i \right}$

위 식과 같이 C가 커질 수록 크사이(slack variable)의 영향력이 커진다. 많이 봐주고 싶으면 c를 줄이는 것이 좋다.

Linear Kernel

from cvxopt import matrix as cvxopt_matrix
from cvxopt import solvers as cvxopt_solvers

def Convolution(pred, real) :
    pred = np.array(pred)
    y = np.array(real)
    TP = np.sum((pred == 1) & (y == 1))
    FP = np.sum((pred == 1) & (y != 1))
    FN = np.sum((pred != 1) & (y == 1))
    TN = np.sum((pred != 1) & (y != 1))
    return TP, FP, FN, TN
    
def precision_recall(X) :
  TP,FP,FN,TN = X
  return TP / (TP + FP), TP / (TP + FN)

def Minmax_(X) :
    X = np.array(X)
    return (X - X.min(axis = 0)) / (X.max(axis= 0) - X.min(axis = 0)), X.max(axis =0) , X.min(axis =0)

def Standar_(X) :
    X = np.array(X)
    return (X - X.mean(axis =0)) / X.std(axis = 0), X.mean(axis =0), X.std(axis = 0)

C = 10  # 여기 C는 hyperparameter이므로 변형 가능함 (이번에는 10으로 고정하고 진행)
X = Standar_(np.array(data_[columns_[:-1]]))[0]
y = np.array(data_[columns_[-1]])* 1.

y = y.reshape(-1,1) 
m,n = X.shape

H = Kernel_(X, X) * 1.
H *= y@y.T

P = cvxopt_matrix(H)
q = cvxopt_matrix(-np.ones((m, 1)))
G = cvxopt_matrix(np.vstack((-np.eye(m),np.eye(m))))
h = cvxopt_matrix(np.hstack((np.zeros(m), np.ones(m) * C)))
A = cvxopt_matrix(y.reshape(1, -1))
b = cvxopt_matrix(np.zeros(1))

#Run solver
sol = cvxopt_solvers.qp(P, q, G, h, A, b)
alphas = np.array(sol['x'])

     pcost       dcost       gap    pres   dres
-5.8129e+02 -1.0298e+05  4e+05  1e+00  2e-12
-4.2201e+02 -5.1005e+04  1e+05  3e-01  1e-12
-2.6726e+02 -2.0455e+04  4e+04  9e-02  6e-13
-1.8589e+02 -1.0262e+04  2e+04  4e-02  4e-13
-9.6012e+01 -5.7119e+03  8e+03  1e-02  3e-13
-8.7623e+01 -2.4313e+03  3e+03  4e-03  3e-13
-1.3650e+02 -1.2150e+03  1e+03  1e-03  3e-13
-1.7550e+02 -6.5484e+02  5e+02  3e-04  3e-13
-1.9086e+02 -5.3416e+02  3e+02  1e-04  3e-13
-2.0543e+02 -4.3445e+02  2e+02  5e-05  3e-13
-2.1462e+02 -4.0309e+02  2e+02  3e-05  3e-13
-2.2073e+02 -3.7532e+02  2e+02  2e-05  3e-13
-2.3983e+02 -3.3847e+02  1e+02  5e-06  3e-13
-2.4876e+02 -3.1464e+02  7e+01  3e-06  3e-13
-2.4776e+02 -3.1241e+02  6e+01  2e-06  3e-13
-2.5088e+02 -3.0203e+02  5e+01  2e-06  4e-13
-2.5455e+02 -2.9130e+02  4e+01  9e-07  3e-13
-2.5610e+02 -2.8591e+02  3e+01  6e-07  3e-13
-2.6030e+02 -2.7701e+02  2e+01  1e-07  4e-13
-2.6374e+02 -2.7136e+02  8e+00  5e-08  4e-13
-2.6524e+02 -2.6894e+02  4e+00  1e-08  4e-13
-2.6632e+02 -2.6743e+02  1e+00  4e-09  4e-13
-2.6662e+02 -2.6697e+02  4e-01  6e-14  5e-13
-2.6679e+02 -2.6680e+02  8e-03  2e-13  5e-13
-2.6679e+02 -2.6679e+02  8e-05  8e-14  5e-13
Optimal solution found.

w = ((y * alphas).T @ X).reshape(-1,1)
S = ((alphas > 1e-4) & (alphas < C-1e-4)).flatten()
b = y[S] - np.sum(Kernel_(X, X[S], type_ = 'linear')* y * alphas , axis = 0).reshape(-1,1)

print('Alphas = ',alphas[(alphas > 1e-4) & (alphas < C-1e-4)])
print('')
print('w = ', w.flatten())
print('')
print('b = ', np.mean(b))
print('')
print("support vector : ", np.array(range(m))[S])

Alphas =  [0.93391225 0.7316745  9.0780053  9.43345925 0.20215315 4.75997642
 0.04898054 0.9437629  1.86790277 0.47439455 3.11719321 4.15402803
 0.69513099 6.68553404 2.71619509 0.50311192 3.00745528 0.39546727
 7.15310875 9.09066704 7.95663185 3.62974155 5.19419471 1.73525479
 7.67969285 1.70451727 0.97884377 3.32793679 1.40845885 1.15099822]

w =  [-0.16485571  1.51871314 -0.04860628  0.70754103  1.1347836   0.91858636
 -0.4016664  -1.39914304 -0.99450461 -1.44138493 -2.46660065  0.33934072
 -1.34516894 -0.15421702 -1.75442775 -0.11663225 -0.85224685 -1.15180703
 -0.59394569  0.02060395 -1.26201162  0.11997614  0.12094323  0.17223573
  0.1805074  -0.24004217 -0.3830853  -1.02168134  0.42591415  1.06549357]

b =  -1.1870552539320571

support vector :  [  4  23  48  65  96 120 143 146 283 295 310 313 318 339 352 445 462 470
 516 538 565 610 659 671 743 748 757 826 836 870]

Scikit learn의 모델과 비교해보기

from sklearn.svm import SVC

clf = SVC(C=10, kernel = 'linear', tol = 1e-5)
clf.fit(X, y.ravel())

print('Coefficients of the support vector in the decision function = ')
print(np.abs(clf.dual_coef_))
print('')
print('w = ',clf.coef_)
print('')
print('b = ',clf.intercept_)
print('')
print('support vector = ', clf.support_)

Coefficients of the support vector in the decision function = 
[[ 4.76005618  0.04895032  0.94363719  1.86780052  0.47456087  3.11693822
   4.1543475   0.69503632 10.          6.68541497  2.71612563 10.
   0.50264113  3.00726877  0.39548227  7.15298708  9.09077709  7.95706495
  10.          3.63026574  5.19409721  1.73514001 10.          7.67990269
   1.70454615  0.97827602  3.32858113  1.40848044 10.          1.15113005
  10.          0.93389797 10.         10.          0.73152197 10.
  10.         10.         10.         10.         10.         10.
   9.0780111  10.         10.          9.43391843  0.20215899 10.        ]]

w =  [[-0.16486255  1.51878818 -0.04858365  0.70753321  1.13475239  0.918642
  -0.40165619 -1.39919131 -0.99447832 -1.44142707 -2.46657961  0.33933682
  -1.34522546 -0.15421286 -1.75447871 -0.11663618 -0.85227882 -1.15186411
  -0.59393017  0.02060847 -1.26206137  0.11998512  0.12095547  0.17224891
   0.18051361 -0.24005502 -0.38307938 -1.02168697  0.42592808  1.06555549]]

b =  [-1.18702913]

support vector =  [120 143 146 283 295 310 313 318 325 339 352 406 445 462 470 516 538 565
 585 610 659 671 721 743 748 757 826 836 856 870 894   4  12  18  23  29
  31  32  33  39  42  45  48  52  64  65  96  98]

pred_sol = np.sign(np.sum(Kernel_(X, X  ,type_ = 'linear')* y * alphas , axis = 0).reshape(-1,1) + b[0])

np.sum(clf.predict(X) - pred_sol.flatten())

0.0

차이가 전혀 없다!

해당 모델의 Performance 확인

precision_recall(Convolution(pred_sol,y))

(0.9891304347826086, 0.91)

RBF Kernel

from cvxopt import matrix as cvxopt_matrix
from cvxopt import solvers as cvxopt_solvers

def Convolution(pred, real) :
    pred = np.array(pred)
    y = np.array(real)
    TP = np.sum((pred == 1) & (y == 1))
    FP = np.sum((pred == 1) & (y != 1))
    FN = np.sum((pred != 1) & (y == 1))
    TN = np.sum((pred != 1) & (y != 1))
    return TP, FP, FN, TN
    
def precision_recall(X) :
  TP,FP,FN,TN = X
  return TP / (TP + FP), TP / (TP + FN)

def Minmax_(X) :
    X = np.array(X)
    return (X - X.min(axis = 0)) / (X.max(axis= 0) - X.min(axis = 0)), X.max(axis =0) , X.min(axis =0)

def Standar_(X) :
    X = np.array(X)
    return (X - X.mean(axis =0)) / X.std(axis = 0), X.mean(axis =0), X.std(axis = 0)

C = 10  # 여기 C는 hyperparameter이므로 변형 가능함 (이번에는 10으로 고정하고 진행)
X = Standar_(np.array(data_[columns_[:-1]]))[0]
y = np.array(data_[columns_[-1]])* 1.

y = y.reshape(-1,1) 
m,n = X.shape

def Kernel_(x, y, params = 2, type_ = 'rbf') :

    if type_ == 'rbf' :
      Kernel = np.exp(- (np.sum(x **2, axis = 1).reshape(-1,1) + np.sum(y **2, axis = 1).reshape(1,-1) - 2 * x @ y.T)* params)
      return Kernel
    
    elif type_ == 'linear':
      Kernel = x @ y.T
      return Kernel

H = Kernel_(X, X) * 1.
H *= y@y.T

P = cvxopt_matrix(H) 
q = cvxopt_matrix(-np.ones((m, 1)))
G = cvxopt_matrix(np.vstack((-np.eye(m),np.eye(m))))
h = cvxopt_matrix(np.hstack((np.zeros(m), np.ones(m) * C)))
A = cvxopt_matrix(y.reshape(1, -1))
b = cvxopt_matrix(np.zeros(1))

sol = cvxopt_solvers.qp(P, q, G, h, A, b)
alphas = np.array(sol['x'])

#Results

w = ((y * alphas).T @ X).reshape(-1,1)
S = ((alphas > 1e-4) & (alphas < C-1e-4)).flatten()
b = y[S] - np.sum(Kernel_(X, X[S], type_ = 'rbf')* y * alphas , axis = 0).reshape(-1,1)

print('Alphas = ',alphas[(alphas > 1e-4) & (alphas < C-1e-4)])
print('')
print('w = ', w.flatten())
print('')
print('b = ', np.mean(b))
print('')
print("support vector : ", np.array(range(m))[S])

     pcost       dcost       gap    pres   dres
 1.1233e+03 -2.1902e+04  2e+04  2e-12  8e-15
 4.0674e+02 -3.1858e+03  4e+03  6e-13  6e-15
-1.3134e+02 -4.7529e+02  3e+02  3e-13  2e-15
-1.7599e+02 -2.0145e+02  3e+01  2e-13  9e-16
-1.7673e+02 -1.7721e+02  5e-01  4e-13  6e-16
-1.7674e+02 -1.7678e+02  4e-02  3e-13  5e-16
-1.7674e+02 -1.7674e+02  2e-03  3e-13  4e-16
-1.7674e+02 -1.7674e+02  4e-05  2e-14  4e-16
Optimal solution found.
Alphas =  [1.79257312e+00 1.79257312e+00 1.79257311e+00 1.79257312e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79256632e+00 1.79257312e+00 1.79257312e+00
79353230e+00 1.17711994e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79257312e+00
74407303e+00 1.79256950e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79257312e+00 1.74407303e+00 1.79257276e+00
79257312e+00 1.79258873e+00 1.48827655e+00 1.79304876e+00
79301497e+00 1.79257314e+00 1.79256925e+00 1.79257312e+00
79257312e+00 1.48496977e+00 1.79245816e+00 1.79257313e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79154051e+00
79257312e+00 1.79755839e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79257309e+00
79257312e+00 1.79245816e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79257311e+00 1.79257302e+00 1.79257312e+00
79257303e+00 1.79154368e+00 1.79257312e+00 1.79257312e+00
79256806e+00 1.79257312e+00 1.46044104e+00 1.79257312e+00
79257312e+00 1.79237607e+00 1.79256784e+00 1.79257310e+00
79257312e+00 1.79257312e+00 1.79257309e+00 1.79257311e+00
79257312e+00 1.79257312e+00 1.79257311e+00 1.79257276e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79257312e+00 1.79256922e+00 1.49059469e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79257312e+00
79257312e+00 1.79257312e+00 1.79257312e+00 1.79257312e+00
22891792e+00 1.79237607e+00 1.79257312e+00 1.79257312e+00
94260877e-01 1.57338779e-01 2.06854357e-01 2.02158145e-01
07426976e-01 2.06750579e-01 1.00636850e-01 2.06813301e-01
06825450e-01 2.07193025e-01 2.07423984e-01 1.47241141e-01
07426973e-01 2.05310829e-01 2.07426925e-01 2.06304917e-01
07420753e-01 2.07426975e-01 2.00513631e-01 9.61397433e-02
07430835e-01 2.05903293e-01 2.07426951e-01 2.07426976e-01
75762238e-01 2.07426301e-01 2.06808625e-01 2.07322172e-01
07412053e-01 2.07401828e-01 2.07361534e-01 2.04009426e-01
87757113e-01 1.79010026e-01 2.05641578e-01 2.07278304e-01
07426673e-01 2.06176920e-01 2.03705885e-01 2.04075885e-01
06550658e-01 2.07416178e-01 2.07399432e-01 2.07426651e-01
06564684e-01 2.06891969e-01 2.07426596e-01 2.07426976e-01
04883925e-01 2.07426245e-01 2.06483472e-01 2.07426771e-01
98840536e-01 1.67294654e-01 2.07410808e-01 2.07342889e-01
07426961e-01 2.07421842e-01 1.80954702e-01 1.66359204e-01
01072031e-01 1.85260237e-01 2.05812439e-01 2.06541034e-01
07405354e-01 2.07394271e-01 2.07376247e-01 2.06090220e-01
54499380e-01 1.31036320e-01 1.64979954e-01 2.07319402e-01
07426952e-01 2.05409413e-01 2.07290917e-01 2.05759674e-01
07167184e-01 2.07421056e-01 1.17811075e-01 2.07148857e-01
06679906e-01 1.91692248e-01 2.06024521e-01 1.78310592e-01
07424177e-01 6.97419864e-02 1.95292589e-01 1.67886442e-01
06687140e-01 2.07402285e-01 2.07423859e-01 1.89127396e-01
06428527e-01 2.07379160e-01 2.02825342e-01 2.02187077e-01
06479696e-01 2.07422358e-01 2.07214701e-01 2.07425881e-01
07426139e-01 2.07367611e-01 2.07426976e-01 2.07426967e-01
07383572e-01 2.07411132e-01 2.07426579e-01 1.91048700e-01
07068517e-01 2.07426976e-01 1.58402781e-01 2.07426945e-01
07426976e-01 2.07424230e-01 2.06351699e-01 1.38690740e-01
07426960e-01 2.07426972e-01 2.07275532e-01 2.07391939e-01
67365621e-01 1.76557425e-01 2.07423274e-01 2.04790932e-01
71881317e-01 2.07421983e-01 2.06877025e-01 2.07426976e-01
88209562e-01 2.07085218e-01 2.07405577e-01 2.02448049e-01
07343035e-01 2.07426963e-01 1.17175904e-01 1.17799636e-01
07426971e-01 2.05363202e-01 2.07409677e-01 1.97768159e-01
04158060e-01 2.06797653e-01 1.72199510e-01 2.07095358e-01
73378355e-01 2.07111553e-01 7.55342879e-02 1.44119506e-01
07426975e-01 2.07274648e-01 2.01860743e-01 2.07417933e-01
07338341e-01 2.05956902e-01 2.06853811e-01 1.70122597e-01
04236129e-01 2.05324137e-01 2.07272697e-01 2.07178579e-01
07193482e-01 1.13746500e-01 2.89384930e-04 2.07426566e-01
07421898e-01 2.06641767e-01 2.05142637e-01 2.07295124e-01
07424286e-01 2.07249452e-01 1.38897822e-01 2.04622922e-01
56932132e-02 2.07287201e-01 2.07426976e-01 2.07422351e-01
07426976e-01 2.07426974e-01 2.07426793e-01 2.05532223e-01
07415267e-01 2.07426267e-01 2.06898377e-01 2.07426976e-01
07426942e-01 2.07426976e-01 2.03373077e-01 8.22750212e-03
07407554e-01 2.07426973e-01 1.67921317e-01 2.07412409e-01
07338105e-01 2.07415129e-01 2.07274232e-01 2.07426976e-01
07426975e-01 1.32099899e-01 2.07347175e-01 2.07385941e-01
07144471e-01 2.07409841e-01 2.03851741e-01 2.07426976e-01
07373337e-01 2.07426976e-01 2.06893871e-01 1.83157192e-01
07402876e-01 2.05658944e-01 2.07426976e-01 2.06998083e-01
05574685e-01 2.07426976e-01 2.02980570e-01 2.06968871e-01
07426972e-01 2.07121022e-01 2.07426976e-01 2.04952184e-01
07426939e-01 1.94790358e-01 2.06249522e-01 1.96378909e-01
73989560e-01 1.66427579e-01 1.28494180e-01 2.07423940e-01
06138951e-01 1.51399676e-01 2.02249473e-01 8.08907536e-02
07426965e-01 2.05912822e-01 2.07426976e-01 2.07426871e-01
07352194e-01 2.03796750e-01 2.07273446e-01 2.07426976e-01
04002550e-01 2.07426023e-01 9.34392060e-02 2.07426396e-01
13367069e-01 2.07414089e-01 2.07426770e-01 2.07199496e-01
14243322e-01 2.04534926e-01 2.07426959e-01 2.04492868e-01
07426976e-01 2.06259150e-01 2.07352496e-01 2.07420266e-01
07319221e-01 2.07422595e-01 2.07399836e-01 2.07420017e-01
58227528e-02 2.07205610e-01 2.07426784e-01 2.06913903e-01
64284810e-01 2.07422782e-01 1.80632889e-01 1.43505728e-01
04510924e-01 2.06672245e-01 2.07426976e-01 2.07403342e-01
07426976e-01 1.43060797e-01 2.00372824e-01 2.07424398e-01
05414243e-01 2.10631361e-01 2.05658396e-01 1.98574336e-01
06393457e-01 2.07242606e-01 2.07423185e-01 1.83243190e-01
07426968e-01 2.06183968e-01 2.07426829e-01 2.07426967e-01
16757380e-02 2.06564059e-01 2.07425261e-01 1.99076636e-01
07324677e-01 2.07395419e-01 2.09386621e-01 1.99588797e-01
88364973e-01 2.06485970e-01 2.05090706e-01 2.00218195e-01
07161651e-01 2.07420266e-01 2.00402114e-01 2.07411706e-01
06671964e-01 2.07425247e-01 2.11075920e-01 2.07383361e-01
85611811e-01 2.07426283e-01 2.07410336e-01 2.07209473e-01
04070187e-01 1.32874174e-01 2.05147444e-01 2.02247178e-01
07340564e-01 2.07422380e-01 2.07426823e-01 2.01174744e-01
07426973e-01 1.44817628e-01 2.06718871e-01 6.65832503e-02
07379420e-01 2.07401492e-01 1.98331616e-01 2.07179622e-01
97485206e-01 2.07120895e-01 2.07426961e-01 2.07409036e-01
03891782e-01 2.07347208e-01 2.07426871e-01 2.07426068e-01
90122315e-01 2.07424571e-01 1.99470537e-01 2.07330149e-01
07426976e-01 2.01992650e-01 2.06403276e-01 2.07426975e-01
04751356e-01 2.07426976e-01 1.95920720e-01 2.07426074e-01
98777017e-01 1.46921008e-01 2.05433865e-01 2.07171995e-01
50372980e-01 1.08211048e-01 2.07425624e-01 2.07426934e-01
18793286e-01 2.07225386e-01 2.07047765e-01 2.07426976e-01
07423754e-01 2.07426975e-01 2.07195325e-01 2.07422030e-01
05497505e-01 1.97770201e-01 1.95915495e-01 2.07421839e-01
06991617e-01 2.03098481e-01 2.07426976e-01 1.91793893e-01
29778846e-01 2.07426855e-01 2.07426976e-01 2.07426642e-01
78510597e-01 2.07099097e-01 2.04508932e-01 2.06773392e-01
94646489e-01 2.02525897e-01 2.05397253e-01 1.86726244e-01
15887790e-01 2.04167583e-01 2.06513514e-01 2.06747815e-01
07251664e-01 2.07413721e-01 2.06345006e-01 2.07426936e-01
06861248e-01 2.07426975e-01 1.91875817e-01 2.06861031e-01
07192539e-01 2.05215066e-01 2.07090382e-01 2.07425985e-01
07291464e-01 1.84566713e-01 2.07420165e-01 2.02040477e-01
07426892e-01 1.76899035e-01 2.07420822e-01 2.07426976e-01
07426976e-01 2.07426976e-01 2.06155694e-01 2.07425749e-01
07241653e-01 2.07426976e-01 2.07426975e-01 2.07342391e-01
07426976e-01 1.64652128e-01 1.92939610e-01 1.98112840e-01
06549113e-01 2.07426307e-01 2.07531176e-01 2.06164471e-01
05032501e-01 2.07411944e-01 2.07246945e-01 1.86014375e-01
07281247e-01 1.61528804e-01 2.07035833e-01 1.78545951e-01
07426604e-01 2.02865384e-01 2.07426976e-01 2.07426976e-01
07316799e-01 2.07425652e-01 2.07426976e-01 2.06882398e-01
07289195e-01 2.07381440e-01 2.07180074e-01 2.07426960e-01
09665144e-02 2.07426976e-01 2.07381368e-01 2.05896748e-01
06958959e-01 2.06391072e-01 2.07423215e-01 2.07424395e-01
07385565e-01 1.79609158e-01 2.05181507e-01 2.05925808e-01
07426976e-01 2.07418929e-01 1.79544023e-01 2.07426289e-01
17487541e-01 2.07323409e-01 2.07426922e-01 2.05177622e-01
07426976e-01 2.07305694e-01 2.07426976e-01 1.41273675e-01
07338565e-01 1.90124446e-01 2.07374668e-01 2.07426972e-01
07426438e-01 2.07321981e-01 2.07426976e-01 2.07426921e-01
07426976e-01 2.07426916e-01 2.07426975e-01 2.07424579e-01
04610313e-01 1.76738797e-01 2.05544717e-01 2.07324158e-01
07426697e-01 2.07426971e-01 1.97150543e-01 2.06057093e-01
07426976e-01 1.89976562e-01 2.07304216e-01 1.58065495e-01
07426728e-01 1.52270548e-01 2.07426976e-01 2.01546651e-01
07426957e-01 2.06800032e-01 2.03907718e-01 2.07426976e-01
07426971e-01 2.02898851e-01 2.07336894e-01 2.07426976e-01
07079309e-01 2.03965508e-01 2.07426976e-01 1.30615949e-01
06726057e-01 2.05682159e-01 2.07367957e-01 2.07416774e-01
06974544e-01 2.07218583e-01 2.07335501e-01 2.05525557e-01
89423810e-01 2.07426894e-01 2.07425356e-01 2.07424456e-01
07426976e-01 2.07350961e-01 1.94726744e-01 1.97904447e-01
07421611e-01 2.07425653e-01 2.05670256e-01 2.07425437e-01
04225605e-01 2.07330764e-01 2.07301860e-01 2.07422400e-01
65238081e-01 2.07426976e-01 1.94517019e-01 2.07382142e-01
07014623e-01 2.04645657e-01 2.07426968e-01 1.94234120e-01
06041247e-01 1.86759308e-01 1.90882644e-01 2.07423674e-01
07426976e-01 2.07389432e-01 2.07421482e-01 1.04060095e-01
85990442e-01 2.06647787e-01 2.07426902e-01 1.79969675e-01
57658368e-01 2.07426975e-01 1.99805431e-01 2.06910718e-01
04673561e-01 2.07426937e-01 2.07426976e-01 2.07426976e-01
07426846e-01 2.00689508e-01 1.74918222e-01 2.02332885e-01
07313455e-01 2.07426955e-01 1.97604872e-01 1.99180879e-01
52231200e-01 2.06463459e-01 2.07426976e-01 2.07423384e-01
07426911e-01 2.05680776e-01 2.06816825e-01 2.07412049e-01
99384993e-01 2.00001814e-01 2.04248654e-01 2.07395748e-01
65022539e-01 2.07426515e-01 1.88975444e-01 2.07730172e-01
05054074e-01 2.07426976e-01 2.07426686e-01 1.78314444e-01
04500666e-01 2.06686793e-01 2.07426166e-01 2.06766514e-01
07425307e-01 2.03856822e-01 2.07426814e-01 2.07426976e-01
07426976e-01 2.07392575e-01 1.43249462e-01 2.07426976e-01
99429989e-01 2.06053179e-01 2.07426960e-01 2.07405808e-01
66999729e-02 2.07426394e-01 2.07423741e-01 2.07425165e-01
43731526e-01 2.07426976e-01 2.07426975e-01 2.07426964e-01
06680985e-01 2.07426964e-01 2.07426972e-01 2.07426627e-01
57629185e-01 2.03904184e-01 2.07426554e-01 2.07426969e-01
07426975e-01 2.07426975e-01 2.07426926e-01 2.04401277e-01
07426976e-01 2.07135880e-01 2.07423480e-01 9.05327888e-02
86774801e-01 2.05428956e-01 2.06289214e-01 1.76878129e-01
27507634e-01 2.07278960e-01 1.94572221e-01 2.07264928e-01
05221399e-01 2.07426858e-01 2.06828964e-01 7.55612786e-02
07387326e-01 2.07426706e-01 2.07424188e-01 2.07425784e-01
31232283e-01 2.07426691e-01 2.07421009e-01 2.07426976e-01
07426976e-01 2.07388453e-01 2.07332911e-01 2.07426959e-01
07075110e-01 2.07426609e-01 2.07022963e-01 2.07426954e-01
07395938e-01 1.79608339e-01 1.05157940e-01 2.06369388e-01
04292148e-01 5.96596752e-02 2.07426976e-01 2.07208115e-01
07425301e-01 2.07426975e-01 2.04804983e-01 1.86231906e-01
27873225e-01 1.98539484e-01 2.07426976e-01 2.07426854e-01
07244499e-01 2.07246381e-01 2.06455876e-01 2.07425915e-01
07425331e-01 1.31580058e-01 2.07227244e-01 2.07426727e-01
07426976e-01 2.07426976e-01 2.02776444e-01 2.05019776e-01
07251229e-01 2.07426973e-01 1.62392478e-01 2.07257720e-01
07424877e-01 2.06839276e-01 1.63231325e-01 2.07426976e-01
06845638e-01 2.07426101e-01 2.06486433e-01 2.07226305e-01
07423817e-01 2.06119217e-01 2.07405690e-01 1.97130337e-01
07426967e-01 2.07009947e-01 2.06245339e-01 2.07398529e-01
05592373e-01 1.35203254e-01 2.07248576e-01 2.07426976e-01
07198259e-01 2.07426933e-01 1.23434674e-01 2.07307120e-01
07426976e-01 2.07426571e-01 2.06805037e-01 2.07378176e-01
50461770e-01 2.06538619e-01 2.07384436e-01 2.07426912e-01
96360141e-01 1.35940488e-01 2.07191142e-01 1.33901032e-01
07426069e-01 2.07426976e-01 2.07424783e-01 2.07268740e-01
07418482e-01 2.07416791e-01 2.07426373e-01 2.06521681e-01
70338370e-01 2.07422149e-01 1.97527802e-01 2.07375339e-01
07272150e-01 2.06881007e-01 1.45437617e-01 2.07426289e-01
07424631e-01 2.02845953e-01 2.06802663e-01 2.04311115e-01
07424811e-01 2.04528322e-01 2.06086690e-01 2.07424166e-01
07187956e-01 2.07420817e-01 2.05916446e-01 2.07426936e-01
07426976e-01 2.07426028e-01 2.05613321e-01 2.07426976e-01
01624067e-01 1.97234815e-01 2.07411269e-01 2.02384804e-01
13654922e-01 2.01147636e-01 2.05741563e-01 2.02952289e-01
07396398e-01 2.03333398e-01 2.07426454e-01 1.99681386e-01
05563257e-01 2.07426976e-01 2.07387986e-01 2.07426455e-01
06584662e-01 2.02105180e-01 1.08971488e-01 1.70696418e-01
07426976e-01 2.07426976e-01 2.07426244e-01 1.95064636e-01
07425872e-01 2.06483589e-01 2.07426974e-01 2.07423620e-01
04101104e-01 1.56789578e-01 2.07426936e-01 2.07423683e-01
83688805e-01 2.06536731e-01 2.04655760e-01 2.07426976e-01
23434232e-01 2.06279791e-01 2.06097816e-01 2.06243974e-01
07415973e-01 2.07426976e-01 2.07426967e-01 2.05422579e-01
04443364e-01 2.07424342e-01 1.91161053e-01 2.07426976e-01
79543447e-01 2.07426976e-01 2.06745198e-01 2.00308895e-01
07283581e-01 2.07421359e-01 1.11193729e-01 2.07426976e-01
07426974e-01 1.85985926e-01 2.05586699e-01 2.07426976e-01
06241403e-01 2.06712474e-01 2.07422248e-01 2.07395210e-01
07382443e-01 2.02354504e-01 2.07426924e-01 2.07426903e-01
07426122e-01 2.07426912e-01 2.06289192e-01 2.07426976e-01
07426964e-01 2.07426975e-01 2.07426867e-01 1.98949288e-01
07426965e-01 2.07153242e-01 2.07284094e-01 2.07423329e-01
03270619e-01 2.07400644e-01 1.37689092e-01 2.00237317e-01
90661702e-01 2.07425619e-01 1.99525528e-01 2.05400678e-01
03707510e-01 2.04254439e-01 2.02040863e-01 2.07205170e-01
22986226e-01 1.98864773e-01 2.07277307e-01 2.07236469e-01
07425935e-01 2.02973219e-01 2.07411284e-01 1.94104046e-01
47001217e-01 2.07029417e-01 2.07426976e-01 2.06928812e-01
03334184e-01 2.02123502e-01 1.63716212e-01 2.07357485e-01
07426976e-01 1.73121465e-01 2.07264847e-01 2.05602816e-01
85254950e-01 2.07426155e-01 2.07422981e-01 2.07376564e-01
02390877e-01 2.07360821e-01 2.07262227e-01 2.07426749e-01
95740606e-01 2.07421142e-01 2.07426976e-01 2.07271951e-01
83067576e-01 2.07398469e-01 2.07426973e-01 1.71683642e-01
03470961e-01 2.07271594e-01 2.00719980e-01 2.07426976e-01
97869721e-01 1.95438272e-01 1.65039104e-01 2.07426976e-01
92255656e-01 2.07313566e-01 2.06769605e-01 2.00275089e-01
93904700e-01 1.72779877e-01 2.07217566e-01]

w =  [ -20.18676701 -240.14832418  230.32058373 -353.60683227  375.82698525
 -174.77641435 -147.64948471 -293.07395374   85.1922569  -294.62720709
 -379.18157477  385.54472168 -427.92765205  -44.36418703 -467.46292388
   -4.47906698 -369.90094812 -376.57314699 -277.98990279   99.90986887
63424093  145.44219878  -25.68515181  -19.22964807  -33.18816907
   -5.80714284    7.81903048   45.38315421   41.02164717   22.70585013]

b =  -0.7925630472941848

support vector :  [  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17
19  20  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35
37  38  39  40  41  42  43  44  45  46  47  48  49  50  51  52  53
55  56  57  58  59  60  61  62  63  64  65  66  67  68  69  70  71
73  74  75  76  77  78  79  80  81  82  83  84  85  86  87  88  89
91  92  93  94  95  96  97  98  99 100 101 102 103 104 105 106 107
109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125
127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143
145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161
163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179
181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197
199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215
217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233
235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251
253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269
271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287
289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305
307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323
325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341
343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359
361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395
397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413
415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431
433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449
451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467
469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485
487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503
505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521
523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539
541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557
559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575
577 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594
596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612
614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630
632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648
650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666
668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684
686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702
704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720
722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738
740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756
758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774
776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792
794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810
812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828
830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846
848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864
866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882
884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900
902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918
920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936
938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954
956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972
974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990
992 993 994 995 996 997 998 999]

from sklearn.svm import SVC
clf = SVC(C = 10, kernel = 'rbf', gamma=2, tol = 1e-4)
clf.fit(X, y.ravel()) 

print('Coefficients of the support vector in the decision function = ')
print(np.abs(clf.dual_coef_))
print('')
print('b = ',clf.intercept_)
print('')
print('support vector = ', clf.support_)

Coefficients of the support vector in the decision function = 
[[0.19427572 0.15734218 0.20686856 0.20215737 0.20743168 0.20675751
10065097 0.20681694 0.20683949 0.20720004 0.20738255 0.14721231
20743128 0.20531684 0.20738893 0.20631745 0.20742473 0.20743935
20051885 0.09617714 0.20742985 0.20591031 0.20743316 0.20742716
17575769 0.20742808 0.20681241 0.20733182 0.20741603 0.2074462
20736915 0.20401339 0.1877162  0.17904257 0.20564435 0.20726029
20743951 0.20619093 0.20368087 0.20409158 0.20655612 0.20737539
20735413 0.20738973 0.20657057 0.20689556 0.20742759 0.20744131
20489204 0.20742798 0.20648072 0.20743166 0.19880382 0.16729958
20740992 0.20731707 0.20743065 0.20743448 0.18095954 0.16636637
20107325 0.18527193 0.20578218 0.20651167 0.20741051 0.20739117
20737985 0.20610321 0.15450253 0.13099134 0.16497422 0.20732516
20746328 0.20540958 0.20725401 0.20576531 0.20716769 0.207434
11787092 0.20715367 0.20668463 0.19166303 0.20603827 0.17831758
20742984 0.07021285 0.19529932 0.16788023 0.20670123 0.20741733
20742354 0.18913514 0.20639017 0.2073795  0.20278572 0.20219554
2064453  0.20743647 0.20722013 0.20742965 0.20743191 0.20736892
20744086 0.20740304 0.20739789 0.20741756 0.20743031 0.19100303
2070681  0.20743362 0.15846509 0.20743296 0.20743128 0.2074299
20635692 0.13867073 0.20742609 0.20743245 0.20728078 0.2073989
16731887 0.17651064 0.20747065 0.20479833 0.171918   0.20742776
20688049 0.20743071 0.18821977 0.207099   0.20740526 0.20246321
20730447 0.20738673 0.11715814 0.11772264 0.20739005 0.20541251
20736748 0.19777709 0.20416006 0.20680344 0.17223805 0.20708198
17338171 0.20711497 0.07560496 0.14411287 0.20743084 0.20727414
2018745  0.20742359 0.20734265 0.20596256 0.2068677  0.17011455
2041958  0.2053767  0.20728463 0.2071914  0.20719433 0.11373797
20742715 0.2073839  0.20664907 0.20514708 0.20730996 0.20742557
20725468 0.13890746 0.20462188 0.07568524 0.20730124 0.20742587
20743404 0.20738979 0.20742664 0.20743192 0.20554422 0.20740409
20743297 0.20690608 0.2074326  0.2074398  0.2074308  0.20337859
0044639  0.20740921 0.20743443 0.16790408 0.20741573 0.20734473
20745188 0.20728125 0.20743991 0.20743111 0.13210691 0.20735124
20738721 0.2071483  0.20742333 0.20380641 0.20742494 0.20737218
20743094 0.206899   0.18312405 0.20740693 0.20566415 0.20742656
20695921 0.2055765  0.20743231 0.20298563 0.20696961 0.2074323
20713093 0.20739549 0.2049583  0.20743441 0.19479364 0.20621083
19639495 0.17396268 0.16644264 0.12848771 0.20742377 0.20609649
15142609 0.20226219 0.08091001 0.20744034 0.20591371 0.2074303
20743971 0.20736066 0.20380269 0.2072659  0.20742365 0.2039982
20742734 0.09352387 0.20743102 0.11343357 0.20737409 0.20743161
20720363 0.11418252 0.20451269 0.20738872 0.20450265 0.20738788
20626481 0.20735639 0.20741913 0.20732589 0.20742722 0.20740735
20742579 0.07713075 0.20721418 0.20743103 0.20691528 0.16429494
20742358 0.18063951 0.14369457 0.20448191 0.2066779  0.20742591
20741067 0.20743087 0.14306064 0.20036855 0.20742323 0.20541109
21058691 0.20565875 0.19857925 0.20639525 0.20721217 0.20743723
18322227 0.20742505 0.20623215 0.20743193 0.20738605 0.06172465
20653682 0.20742948 0.19908798 0.20733008 0.20735825 0.20938513
19959663 0.18836901 0.20649894 0.20509767 0.20022286 0.20717139
20745659 0.20041115 0.20738376 0.20668579 0.20742044 0.21107746
20738731 0.18561726 0.20744068 0.20741704 0.20721481 0.20408297
13292813 0.20515203 0.20225136 0.20734624 0.20743575 0.20743189
2011778  0.2073929  0.14483335 0.20672578 0.066635   0.20739316
20740524 0.1983437  0.20718377 0.19748254 0.20717252 0.2074318
2074104  0.20390533 0.20735097 0.20743273 0.20738345 0.19013035
20743168 0.19946299 0.20733136 0.20743428 0.20199385 0.20639465
20742875 0.2047468  0.20742578 0.19592635 0.20743809 0.19877975
14696428 0.20542625 0.20717929 0.15039311 0.10817906 0.20739236
20743969 0.21880752 0.20724261 0.20704943 0.20743146 0.20742295
20747934 0.20720108 0.20742783 0.20550413 0.19777662 0.19592361
20742356 0.20699579 0.20305251 0.2074012  0.19176702 0.12986659
20743228 0.20743578 0.20743194 0.17849539 0.20711052 0.20450901
20678423 0.19465468 0.20253068 0.20540232 0.1867358  0.11584783
20413777 0.20651981 0.2067526  0.20722515 0.2074134  0.20635119
20742667 0.20686854 0.20743643 0.19188422 0.20686594 0.20715304
20522216 0.20709564 0.20743141 0.20729005 0.18457371 0.20742188
20205018 0.20742606 0.17692852 0.2074219  0.20742625 0.20743105
207441   0.20611652 0.20738766 0.20724007 0.20738225 0.20743089
20734902 0.20738634 0.16463179 0.19290227 0.19808257 0.20651303
20743376 0.20753163 0.20616665 0.20503742 0.20741085 0.20725261
18602034 0.20728859 0.16153067 0.20704904 0.17850811 0.20739093
20286862 0.20743374 0.20738841 0.20732125 0.2073921  0.20743049
2068884  0.20729541 0.20739279 0.20718542 0.2074312  0.00938444
20738369 0.20738244 0.2058668  0.20695975 0.20639272 0.20743603
20743773 0.20738932 0.17957891 0.20518664 0.20593327 0.20743261
20743182 0.17955183 0.20738945 0.11745496 0.2073305  0.20739367
2051899  0.20742574 0.20730908 0.20745815 0.14120766 0.2073398
19013425 0.20738256 0.20742868 0.20743378 0.20729152 0.20743935
20743978 0.20743505 0.20742596 0.20743146 0.2074263  0.20460177
17673433 0.20555266 0.20733671 0.20743219 0.20743246 0.19715575
20606281 0.20743215 0.18998244 0.2073171  0.1581     0.20742849
15228653 0.20739178 0.20156172 0.20743979 0.20679895 0.20390719
20742827 0.20743457 0.20290248 0.20733705 0.20742626 0.20708668
20397047 0.20743105 0.1305964  0.20668213 0.20569002 0.20737598
20742243 0.20697415 0.20721851 0.20734083 0.20549438 0.1894333
20742728 0.20742442 0.20743709 0.20740154 0.20730697 0.1946947
1979096  0.20744431 0.20737909 0.20563251 0.20743094 0.20423111
2073355  0.20730058 0.20742956 0.16523764 0.20738976 0.19450185
20738573 0.2069736  0.20461204 0.20738685 0.19420101 0.20604688
18676618 0.1908941  0.20737826 0.20738976 0.20739437 0.20743113
10403112 0.18599997 0.20665139 0.2074341  0.17997993 0.15760293
20743992 0.19981395 0.20692169 0.20468685 0.20743056 0.20740154
20743401 0.20743445 0.20070312 0.17492    0.2023381  0.20732057
20742662 0.19761954 0.19918905 0.15220292 0.20646885 0.20743094
20743676 0.20743068 0.20567921 0.2068306  0.20741611 0.199399
19996713 0.20426253 0.20740152 0.16505059 0.20738379 0.18897945
207736   0.20502022 0.2074198  0.2074244  0.17833201 0.20454092
20668779 0.20743004 0.20681928 0.20743868 0.20382928 0.20743418
20743122 0.20743373 0.20735378 0.14579096 0.20743401 0.19944259
20605928 0.20742633 0.20741168 0.02667927 0.20738576 0.20743118
20742904 0.14374012 0.2074198  0.20742873 0.20738771 0.20668656
20743407 0.20742637 0.2074307  0.15764139 0.2039157  0.20738466
20743108 0.2074261  0.20746353 0.20738303 0.20440281 0.2074318
20713657 0.20742955 0.09051151 0.18673723 0.20543347 0.20628922
17686475 0.12757543 0.20731182 0.19453688 0.20727871 0.20518115
20742863 0.20685553 0.07558359 0.20738725 0.20743223 0.20739625
20742465 0.13125239 0.20743203 0.20742511 0.20744036 0.20739318
20739618 0.20729283 0.20746032 0.20707682 0.20743966 0.20703597
20742947 0.20740888 0.17958215 0.10521115 0.20636944 0.2043012
05961702 0.20743458 0.20723091 0.2073793  0.20743269 0.20481917
18624325 0.12788496 0.1985404  0.20739517 0.20743278 0.20724631
2072596  0.20646071 0.20742478 0.20740227 0.13157376 0.20723249
20741559 0.20744018 0.207441   0.20277467 0.20497575 0.20725062
20744004 0.16235855 0.20727089 0.20743239 0.20684377 0.1632368
20743128 0.20685115 0.20743144 0.20649063 0.20718354 0.20746618
20607353 0.20741132 0.19714504 0.20738441 0.20700355 0.20625273
20740242 0.20559863 0.13518079 0.20725434 0.20745815 0.20720526
2074339  0.12339703 0.20726083 0.20744087 0.20743175 0.20684291
20737979 0.15048379 0.20653729 0.20738962 0.20744173 0.1963617
13585555 0.20714556 0.13387435 0.20743343 0.20743265 0.20742874
20727446 0.20742378 0.20742085 0.20743839 0.20650149 0.1703536
20742934 0.1975379  0.20738264 0.20727947 0.2068848  0.14543311
20743176 0.20743127 0.20286197 0.20685544 0.20431495 0.20742877
20452271 0.20610068 0.20742521 0.20719461 0.20743382 0.20592146
20742661 0.20742605 0.20746265 0.20561114 0.20743962 0.20157781
19723793 0.2073729  0.20239831 0.11451379 0.20110238 0.20574678
20291973 0.20739305 0.20336836 0.20742764 0.19968498 0.20556737
20740651 0.20739176 0.20742637 0.20658595 0.20206837 0.10898484
17067358 0.20738606 0.20738859 0.20737968 0.19502779 0.20738662
20648734 0.20748014 0.2074351  0.20410247 0.15682219 0.20742812
20747265 0.18370009 0.20650265 0.20470634 0.20738313 0.12349673
20629496 0.20610094 0.20620919 0.20741734 0.20743128 0.20742815
20542838 0.20445639 0.2074234  0.19117662 0.20743215 0.17955309
20742768 0.20675674 0.20035512 0.20728866 0.20742606 0.11141255
2073902  0.20743309 0.1859914  0.20558829 0.20743247 0.20624704
20671951 0.20742359 0.20740844 0.20739655 0.20237572 0.2074308
20742817 0.20743932 0.20743225 0.20629543 0.20743141 0.20743923
20742788 0.20744061 0.19896133 0.20743215 0.20715275 0.20729686
20742826 0.20326596 0.20739983 0.13766786 0.20027378 0.1906687
20743843 0.19952483 0.20541435 0.20368544 0.20426176 0.20207326
20717199 0.12291511 0.19886663 0.2072868  0.20724223 0.20738532
20297501 0.20741647 0.19411178 0.14701423 0.20703545 0.20738874
2069307  0.20334021 0.20212269 0.16375042 0.20737188 0.20743386
17316461 0.20726567 0.20561637 0.18526912 0.20743126 0.20742185
20737579 0.20234691 0.20737455 0.20731499 0.2074258  0.19569566
20746104 0.20743937 0.20727757 0.18307296 0.20741074 0.20742665
17169245 0.20347463 0.20727091 0.20068993 0.2074334  0.19787976
19544577 0.16503822 0.20743121 0.19213377 0.20731273 0.20678328
20028059 0.1938646  0.17276456 0.20722987 1.79256939 1.79256329
79256754 1.79257409 1.79256861 1.79256782 1.79258091 1.79257222
79258376 1.79256144 1.79260906 1.79257063 1.79352141 1.17710246
7925661  1.79257435 1.79258904 1.79255868 1.79256897 1.79257335
74411096 1.79255676 1.7925746  1.79257375 1.79256739 1.79256738
74407628 1.79257148 1.79256566 1.79258327 1.48830748 1.79304915
79301098 1.79257053 1.79256526 1.79256808 1.79255868 1.48500043
79245079 1.79256966 1.79255798 1.79255995 1.79256914 1.79153654
79257137 1.79758755 1.79253966 1.79255976 1.79256836 1.79257131
79261522 1.79255981 1.79256907 1.79240485 1.79256941 1.79256623
79257216 1.79256783 1.79260134 1.7925643  1.79257734 1.79154265
79256731 1.79255073 1.79256182 1.79256657 1.46045705 1.79257203
79255909 1.79237054 1.792601   1.7925635  1.79261818 1.79261687
79256022 1.79255891 1.79257248 1.79257352 1.79256765 1.79255878
79256574 1.79256534 1.79256809 1.79255913 1.79256009 1.79257124
79255563 1.49058704 1.79256168 1.79260144 1.79257203 1.79257437
79256053 1.79257214 1.79261818 1.79256538 1.22894188 1.79241597
79261491 1.79256009]]

b =  [-0.79257296]

support vector =  [100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117
119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135
137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153
155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171
173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189
191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207
209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225
227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243
245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261
264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280
282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298
300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316
318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334
336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352
354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370
372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388
390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406
408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424
426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442
444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460
462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478
480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496
498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514
516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532
534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550
552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568
570 571 572 573 574 575 576 577 579 580 581 582 583 584 585 586 587
589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605
607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623
625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641
643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659
661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677
679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695
697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713
715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731
733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749
751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767
769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785
787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803
805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821
823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839
841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857
859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875
877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893
895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911
913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929
931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947
949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965
967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983
985 986 987 988 989 990 991 992 993 994 995 996 997 998 999   0   1
 3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19
21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37
39  40  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55
57  58  59  60  61  62  63  64  65  66  67  68  69  70  71  72  73
75  76  77  78  79  80  81  82  83  84  85  86  87  88  89  90  91
93  94  95  96  97  98  99]

pred_sol = np.sign(np.sum(Kernel_(X, X, params=2, type_ = 'rbf')* y * alphas , axis = 0).reshape(-1,1) + b[0])

np.sum(clf.predict(X) - pred_sol.flatten())

0.0

차이가 존재하지 않는다!

해당 모델의 Performance 확인

precision_recall(Convolution(pred_sol,y))

(1.0, 1.0)

RBF의 모델이 Linear 모델보다 더 좋은 성능을 보여주었다. 하지만 RBF 모델은 train dataset에 overfitting이 될 가능성이 매우 높기 대문에 주의해야한다. gamma의 값이 작을 수록 margin의 크기가 커지므로 overfitting으로부터 벗어날 수 있다.

Twitter Facebook LinkedIn

Jeongseok Oh

Support Vector Machine 구현하기

개요

SVM이란?

코드 구현

Kernel Function

Linear Kernel

해당 모델의 Performance 확인

RBF Kernel

You May Also Enjoy

Attention 구현하기

ResNet 구현하기

손실함수 모음집

t-SNE 구현하기