SVMs Classification Understanding by CHIRAG


What is a Support Vector Machine?

 1.) It is a supervised machine learning problem where we try to find a hyperplane that best separates the two classes. 

2.)Support Vector Machines (SVMs in short) are machine learning algorithms that are used for classification and regression purposes. SVMs are one of the powerful machine learning algorithms for classification, regression and outlier detection purposes. 

3.) An SVM classifier builds a model that assigns new data points to one of the given categories. 

4.)Thus, it can be viewed as a non-probabilistic binary linear classifier.


 Don’t get confused between SVM and logistic regression. Both the algorithms try to find the best hyperplane, but the main difference is logistic regression is a probabilistic approach whereas support vector machine is based on statistical approaches.

Support Vector Machines intuition 

Now, we should be familiar with some SVM terminology.

Hyperplane

A hyperplane is a decision boundary which separates between given set of data points having different class labels. The SVM classifier separates data points using a hyperplane with the maximum amount of margin. This hyperplane is known as the maximum margin hyperplane and the linear classifier it defines is known as the maximum margin classifier.

or we can understand by this further

Optimal Hyperplane

Suppose we have a dataset that has two tags (green and blue), and the dataset has two features x1 and x2. We want a classifier that can classify the pair(x1, x2) of coordinates in either green or blue. The SVM algorithm helps to find the best line or decision boundary. SVM algorithm finds the closest point of the lines from both the classes. These points are called support vectors. The distance between the vectors and the hyperplane is called as margin. And the goal of SVM is to maximize this margin. The hyperplane with maximum margin is called the optimal hyperplane.

Support Vectors

Support vectors are the sample data points, which are closest to the hyperplane. These data points will define the separating line or hyperplane better by calculating margins.

Margin

A margin is a separation gap between the two lines on the closest data points. It is calculated as the perpendicular distance from the line to support vectors or closest data points. In SVMs, we try to maximize this separation gap so that we get maximum margin.

Types of SVMs


There are two different types of SVMs, each used for different things:

  • Simple SVM: Typically used for linear regression and classification problems.


  • Kernel SVM: Has more flexibility for non-linear data because you can add more features to fit a hyperplane instead of a two-dimensional space.

Kernel functions

A kernel is a function used in SVM for helping to solve problems. With the help of kernel we can go to higher dimensions and perform smooth calculations. We can go up to an infinite number of dimensions using kernels. Kernel plays a vital role in classification and is used to analyze some patterns in the given dataset. They are very helpful in solving a non-linear problem by using a linear classifier.

Sometimes, we cannot have a hyperplane for certain problems. This problem arises when we go up to higher dimensions and try to form a hyperplane. We have various svm kernel functions to convert the non-linear data to linear. 

Linear

These are commonly recommended for text classification because most of these types of classification problems are linearly separable.

The linear kernel works really well when there are a lot of features, and text classification problems have a lot of features. Linear kernel functions are faster than most of the others and you have fewer parameters to optimize.

Here's the function that defines the linear kernel:

f(X) = w^T * X + b

In this equation, w is the weight vector that you want to minimize, X is the data that you're trying to classify, and b is the linear coefficient estimated from the training data. This equation defines the decision boundary that the SVM returns.

Polynomial

The polynomial kernel isn't used in practice very often because it isn't as computationally efficient as other kernels and its predictions aren't as accurate.

Here's the function for a polynomial kernel:

f(X1, X2) = (a + X1^T * X2) ^ b

This is one of the more simple polynomial kernel equations you can use. f(X1, X2) represents the polynomial decision boundary that will separate your data. X1 and X2 represent your data.

Gaussian Radial Basis Function (RBF)

One of the most powerful and commonly used kernels in SVMs. Usually the choice for non-linear data.

Here's the equation for an RBF kernel:

f(X1, X2) = exp(-gamma * ||X1 - X2||^2)
In this equation, gamma specifies how much a single training
point has on the other data points around it. ||X1 - X2|| is the dot product
between your features.

Sigmoid

More useful in neural networks than in support vector machines, but there are occasional specific use cases.

Here's the function for a sigmoid kernel:

f(X, y) = tanh(alpha * X^T * y + C)

In this function, alpha is a weight vector and C is an offset value to account for some mis-classification of data that can happen.

=================================================

IMPORTANT QUESTIONS & ANSWERS

Q1. What is a SVM algorithm?

A. SVM algorithm is used for both classification and regression tasks. It finds an optimal hyperplane to separate data points of different classes in a high-dimensional space.

Q2. Why is SVM the best algorithm?

A. SVM is considered one of the best algorithms because it can handle high-dimensional data, is effective in cases with limited training samples, and can handle non-linear classification using kernel functions.

Q3. What are the steps of SVM algorithm?

A. The steps of the SVM algorithm involve: (1) selecting the appropriate kernel function, (2) defining the parameters and constraints, (3) solving the optimization problem to find the optimal hyperplane, and (4) making predictions based on the learned model.

Q4. What does SVM do in machine learning?

A. In machine learning, SVM is used to classify data by finding the optimal decision boundary that maximally separates different classes. It aims to find the best hyperplane that maximizes the margin between support vectors, enabling effective classification even in complex, non-linear scenarios.

==============================================================

HOPE YOU GUYS LIKE THIS ARTICLE , STAY CONNECTED FOR MORE SUCH ARTICLES

THANK YOU 

CHIRAG GUPTA






    Comments

    Post a Comment

    Popular posts from this blog

    Overfitting , Underfitting Bias & Variance Understanding by CHIRAG

    Image
      What is Overfitting? Overfitting means model has High accuracy score on training data but low accuracy score on test data. (That means our model best fit line satisfied all the training data points . ie. Low Training Error & High Testing Error & High Training Accuracy & Low Test Accuracy ) Reasons for Overfitting:   High variance(High Test Error) .  Low Bias(Low Training Error). The model is too complex. The size of the training data. What is Bias? Bias is the difference between the average prediction of our model and the correct value which we are trying to predict. A model with high bias pays very little attention to the training data and oversimplifies the model. Let’s assume we have trained the model and are trying to predict values with input ‘x_train’. The predicted values are y_predicted.  Bias is the error rate of y_predicted and y_train. In simple terms.   think of bias as the error rate of the training data. When the error rate is high,...
    Read more

    Linear Regression Understanding by CHIRAG

    Image
      Linear Regression 1 .)Linear regression is a supervised learning algorithm used when target / dependent variable continues real number. 2.)Linear regression is a type of statistical analysis used to predict the relationship between dependent and independent Variables using BEST FIT LINE. 3.) It work on the principle of ordinary least square (OLS)/ Mean square errror (MSE) Simple Linear Regression In a simple linear regression, there is one independent variable and one dependent variable. The model estimates the slope and intercept of the line of best fit, which represents the relationship between the variables. The slope represents the change in the dependent variable for each unit change in the independent variable, while the intercept represents the predicted value of the dependent variable when the independent variable is zero. Mathematically, we can represent a linear regression as: y= a0+a1x+ ε Y= Dependent Variable (Target Variable) X= Independent Variable (predictor Variab...
    Read more

    Understanding Confusion Matrix by CHIRAG

    Image
    What Is a Confusion Matrix? A confusion matrix is a performance evaluation tool in machine learning, representing the accuracy of a classification model.  It displays the number of true positives, true negatives, false positives, and false negatives. A Confusion matrix is an N x N matrix used for evaluating the performance of a classification model, where N is the total number of target classes.     For a binary classification problem, we would have a 2 x 2 matrix, as shown below, with 4 values: Let’s understanding this matrix: The target variable has two values: 0 & 1 The  columns  represent the  Predicted values  of the target variable The  rows  represent the  Actual values  of the target variable. But wait – what’s TP, FP, FN, and TN here? That’s the crucial part of a confusion matrix. Let’s understand each term below. Important Terms in a Confusion Matrix True Positive (TP)  The predicted value matches the actual value...
    Read more