Support Vector Machine(SVM): Basics

 

ü Support Vector Machine(SVM):

ü It is a supervised Machine Learning Algorithm. It is used to Classification and Regression model. 

What is SVM

SVM is a Machine Learning algorithm. It creates the best line or decision boundary(Hyper plane) that can separate out n-dimensional space into classes. The decision boundary is used to classify new data.

1.   SVM is a type of supervised Learning Algorithm.
2.    It require labeled data (Input and output both) for training.
3.    It is used to solve classification and regression problems.        

When should we use a Support Vector Machine?

Support vector machine is used two classify two-dimensional and multidimensional data and also regression model.

Decision Boundary:Decision boundarie (Line or Plane) plays important role in the classification   

                        Figure1:Hyperplane and line Classify the data                                                              

 To classify blue and red dots Support vector creates a decision boundary between these two data sets.

If  it is multidimensional data, plane is used and if it is  two-dimensional data line is used. This line or a plane is called Hyperplane as shown in figure1

Following are the important concepts in SVM

Define following concept.

1. Linearly and non linear SVM.
2. Hyper Plane.
3.Distance Margin
4.Support Vectors

Types of SVM

SVM can be of two types:

• Linear SVM: Linear SVM is used for linearly separable data, which means if a dataset can be classified into two classes by using a single straight line, then such data is termed as linearly separable data, and classifier is used called as Linear SVM classifier.Figure 2 is an example of Linear SVM.

Figure2 Linear Classification

• Non-linear SVM: Non-Linear SVM is used to classify non-linearly separated data, which means if a datasetc not be classified  by using a straight line, then such data is termed as non-linear data 

Figure3: Non Linear SVM

Figure 3 is an example of Non Linear SVM. In figure 3.It is not possible to draw a line to separate blue and yellow spot. Plane is used to classify data. To ceat plane, need to convert into a multidimensional plot using kernal. Plane is created using kernel to classify data

Figure4:Plane to seperate non linear data

Differce between Linear SVM and Non Linear SVM

 

Linear SVM	Non-Linear SVM
Linear line will separate the data and classify	Linear line will not  separate the data
Data is classified with the help of hyperplane.	Kernels  are used to make non-separable data into separable data.
Data can be easily classified by drawing a straight line.	Need  to map data from low diamensional space to  high dimensional space for classification.

Summary of Linear SVM and Non-Linear SVM.

• In Linear SVM Data set can be classified into two classes by using a single straight line, 
• In Non Linear SVM Data set can not  be classified into two classes by using a single straight line
• In Non Linear classifier plane is used to classify the data.
• Hyperplane(Line/Plane) is used to classify data in SVM
• Kernels are used to classify non linear Data.
• Kernels are used to map data from low dimensional space to high dimensional space for classification.
• Kernels  are used to make non-separable data into separable data in non linear classifier

What is Hyperplane? 

What is Margin?

What is Support Vector?

A most important step to build a  SVM Model  is to create Hyperplane which has a maximum margin

Definations

Hyperplane: 

Hyperplane is a decision plane or space that divides set of objects having different classes between the data points. There can be multiple lines/decision boundaries to seperate the classes in n-dimensional space, but best decision boundary need to find out that helps to classify the data points. This best boundary is known as the hyperplane of SVM.In figure 4 large margin is the best boundary.

                    Figure 5:Hyperplane

Margin Maximum margin hyperplane is a decision plane or space that divides set of objects having different classes between the data points. We always create a hyperplane that has a maximum margin, which means the maximum distance. It can be easily seen that line from figure 4 with a large margin is the one that best separates the two classes. 

Support Vector The data points on hyperplane boundary is called Support Vector.Figure 6 shows Hyper plane,Margin, Support Vector.

Figure6:Hyperplane.Margin,Support Vector

To build SVM Model the most important part is to choose the hyperplane which is as far as possible from the data points of each category.

How Doeds SVM Works?

Step1: Collect data set of green and blue dot. It is supervised learning with labels green for green dot and blue for  blue dot.

Step 2: Apply SVM .It creates hyperplane. It is two diamensional data Hyperplane is line which separate green and blue dots.

Step 3:Distance between positive hyperplane and Negative hyperplane should be maximum hen it is maximum margin hyperplane.Check Hyperplane is a maximum margin hyperplane. 

Step 4: Find Support Vectors.

Step5: Support vectors are used to classify the new points.

What are the advantages of support Vector Machine over other classification problem?

SVMs are extremely popular because of their ability to handle multiple continuous and categorical variables.

Disadvantages of Neural Network over SVM

1.Having many local minima

2 To find number of neron and number of layers in the architecture.

 3.Trai and error method  to decide structure and hyperparameters  to reach optimality of that NN 

There are many linear classifiers (hyper planes) that separate the data. However the goals of SVM are separating the data with hyper plane and extend this to non-linear boundaries using kernel trick. 

Advantages of SVM are

  It is  maximum margin classifier or hyper plane.  It gives  maximum separation between two class

• Text and Hypertext category
• Classification of Images
• Classification of satellite data.
• Recognition of hand written character
• Biological and other sciences to classify proteins with up to 90% of the compounds classified correctly.