Mathematical representation of SVM

Mathematical representation of SVM

In the previous blog we have seen basics of support vector machine.

In this blog we have to see maths intuition behind the support vector machine.

The most important in case of SVM is creat hyperplane and calculate marginal distance between positive hyperplane and negative hyperplane, so that it is easy to classify data.

First step is to calculate or marginal distance between  +ve hyperplane and – ve hyperplane.

Consider following example.

Ex.  Suppose their are two points   (-5,0) & (5,5).

Figure1: SVM Classifier

A straight line which divides these  two points is having slop -1and passes through origin

Solution   Equation of of hyper plane

 y is either +1 or -1 depending on class 1 or class 2

Hear  slope = -1  Therefore  m = -1and   &  b = 0, As line passes through origin

Putting it in equation 2 we get

For point (-5,0)   calculate y value. In this m = -1 and b=0    

Any point left to the line is always -ve

For point ( 5, 5) value of y is calculated as

Any point  right side of line is always -ve

Thus straight line will divide the data. Points above the line is in one group and points below the line is in another group.

For SVM  Only straight line is not sufficient we need to calculate  Hyperplane

Fig 2 shows hyperplane (middle blue line) and margin middle (two dotted lines)

 

 In figure 2 Blue line is dividing green circle and red circle i.e class1 and class2 which is hyperplane

Disance between  dotted  blue lines is a  marginal distance

One red square and two green dots which are on dotted lines are Support vectors

If b  represents intersection of line on x axis then equation of blue line is 

Classification for number of attributesTo understand the classification let us consider two input attributes A₁ and A₂

Training tuples are 2D, (X =(x_1, x₂) where x₁  and x₂  are attributes of A1and A2)

Considering w₀ as the additional weight . and rewriting Equation (5) as

w₀₊ w₁x₁₊ w₂x₂ = 0 -----------------------6                                                                                         

Any point that lies above the hyperplane satisfies the following equation

w₀₊ w₁x₁₊ w₂x₂  > 0  ----------------------7                                                                                        

Any point that lies below the hyperplane satisfies the following equation

w₀₊ w₁x₁₊ w₂x₂  > 0  ----------------------8                                                                                               

Adjusting the weight and hyperplanes define the sides of the margin can be written as

H₁ : w₀₊ w₁x₁₊ w₂x₂  ≥   1         for y_{i =}  +1  ---------------9                                                             

H₂ : w₀₊ w₁x₁₊ w₂x₂  ≤   -1        for y_{i =}  -1   -------------------10                                                             

From the above two equation (9) and (10) we can say that,any tuple that falls on or above H₁  belongs to class +1 and any tuple that falls on or bellow H₂ belongs to class -1  

Combine two equations (5) and (6) get

Y_i  (w₀₊ w₁x₁₊ w₂x₂)  ≥   1 ,       -------------------11                                                                           

Any Training tuples that fall on hyperplanes H₁ or  H₂  (i.e.,the “sides” defining the margin)satisfy Eq.(11) and are called support vectors .That is. They are equally close to  the (separating) Hyperplane

The distance from the separating hyperplane to ant point on H₁  is 1/  where   is the Euclidean norm of  W.By definition , this is equal to the distance from any point on  H₂ to the separating hyperplane Therefore, the maxmal margin is 2/

Multiple choice questions for SVM

Q1.      SVM  is ----------type of learning(Supervised)

Q2       SVM is used to solve -----------Type of problems.(Clssification and Regression)

Q3       In SVM decision boundaries which classify the data is called ---------(hyperplane)

Q4       SVM is used for --------type of data(linearly seperable and non linearly seperable)

Q5       Support vector machine is used for---- (single diamensinal and multidimensional data)

Q6       In SVM equation of line ------to separate the data

Q7      In SVM equation of plane  ------to separate the data

Q8       ---- are used to make non-separable data into separable data in non linear classifier.(Kernel)

Q9       ----- map data from low dimensional space to high dimensional space forSVM

            classification.r(kernel)

Q10     In SVM Hyperplane must have ----margin(high)

Q11     Equation of hyperplane is ---

Q12     Equation of hyperplane margin is---

Q13     Support vectors are the points which lies on---(boundary of the hyperplane)

Q14)    SVM is used for ---------type of data(Structured and unstructured)

Q15     ----------------- is the real strength of svm(Hyperplane)

 

In above figure

1.         What is equation of width of the  hyperplane.
2.        Which are support vector points.
3.     What is the equation of hyperplane
4.        What is the equation of positive hyperplane
5.        What is the equation of negative hyperplane