Understanding Confusion Matrix by CHIRAG

What Is a Confusion Matrix?

  1. A confusion matrix is a performance evaluation tool in machine learning, representing the accuracy of a classification model. 
  2. It displays the number of true positives, true negatives, false positives, and false negatives.
  3. A Confusion matrix is an N x N matrix used for evaluating the performance of a classification model, where N is the total number of target classes.  

For a binary classification problem, we would have a 2 x 2 matrix, as shown below, with 4 values:

Let’s understanding this matrix:

    • The target variable has two values: 0 & 1
    • The columns represent the Predicted values of the target variable
    • The rows represent the Actual values of the target variable.
    But wait – what’s TP, FP, FN, and TN here? That’s the crucial part of a confusion matrix. Let’s understand each term below.

    Important Terms in a Confusion Matrix

    True Positive (TP) 

    1. The predicted value matches the actual value, or the predicted class matches the actual class.
    2. The actual value was positive, and the model predicted a positive value.

    True Negative (TN) 

    1. The predicted value matches the actual value, or the predicted class matches the actual class.
    2. The actual value was negative, and the model predicted a negative value.

    False Positive (FP) – Type I Error

    1. The predicted value was falsely predicted.
    2. The actual value was negative, but the model predicted a positive value.
    3. Also known as the type I error.

    False Negative (FN) – Type II Error

    1. The predicted value was falsely predicted.
    2. The actual value was positive, but the model predicted a negative value.
    3. Also known as the type II error.
    1.Understanding Confusion Matrix in an easier way:

    We have a total of 20 cats and dogs and our model predicts whether it is a cat or not.

    Actual values = [‘dog’, ‘cat’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’]
    Predicted values = [‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘cat’, ‘cat’, ‘cat’, ‘dog’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’, ‘dog’, ‘dog’, ‘cat’]

    True Positive (TP) = 6

    You predicted positive and it’s true. You predicted that an animal is a cat and it actually is.

    True Negative (TN) = 11

    You predicted negative and it’s true. You predicted that animal is not a cat and it actually is not (it’s a dog).

    False Positive (Type 1 Error) (FP) = 2

    You predicted positive and it’s false. You predicted that animal is a cat but it actually is not (it’s a dog).

    False Negative (Type 2 Error) (FN) = 1

    You predicted negative and it’s false. You predicted that animal is not a cat but it actually is.

    2. Classification Measure

    Basically, it is an extended version of the confusion matrix. There are measures other than the confusion matrix which can help achieve better understanding and analysis of our model and its performance.

    Why Use Confusion Matrix?

    The confusion Matrix allows us to measure Recall and Precision, which, along with Accuracy and the AUC-ROC curve, are the metrics used to measure the performance of ML models.

     Accuracy, Precision, Recall (TPR, Sensitivity), F1-Score, FPR (Type I Error), FNR (Type II Error)


    What Are Precision and Recall?

    In pattern recognition, object detection, and classification task(ML), precision and recall are performance metrics that use to retrieve data from training data.

    Precision (positive predicted) is the fraction of relevant data among the retrieved data. On the other hand, recall (sensitivity) is the fraction of relevant data that were retrieved. Both precision and recall are based on relevance.

    What Is F1-Score?

    In practice, when we try to increase the precision of our model, the recall goes down, and vice-versa. The F1-score captures both the trends in a single value:

    F1-score is a harmonic mean of Precision and Recall, and so it gives a combined idea about these two metrics. It is maximum when Precision is equal to Recall.

    But there is a catch here. The interpretability of the F1-score is poor. This means that we don’t know what our classifier is maximizing – precision or recall. So, we use it in combination with other evaluation metrics which gives us a complete picture of the result.

    The F1 score is the weighted average of precision and recall. In both precision and recall, there is false positive and false negative, so it also considers both. F1 score is more useful than accuracy, especially when you have an uneven class distribution.




    Comments

    Post a Comment

    Popular posts from this blog

    Overfitting , Underfitting Bias & Variance Understanding by CHIRAG

    Image
      What is Overfitting? Overfitting means model has High accuracy score on training data but low accuracy score on test data. (That means our model best fit line satisfied all the training data points . ie. Low Training Error & High Testing Error & High Training Accuracy & Low Test Accuracy ) Reasons for Overfitting:   High variance(High Test Error) .  Low Bias(Low Training Error). The model is too complex. The size of the training data. What is Bias? Bias is the difference between the average prediction of our model and the correct value which we are trying to predict. A model with high bias pays very little attention to the training data and oversimplifies the model. Let’s assume we have trained the model and are trying to predict values with input ‘x_train’. The predicted values are y_predicted.  Bias is the error rate of y_predicted and y_train. In simple terms.   think of bias as the error rate of the training data. When the error rate is high,...
    Read more

    Linear Regression Understanding by CHIRAG

    Image
      Linear Regression 1 .)Linear regression is a supervised learning algorithm used when target / dependent variable continues real number. 2.)Linear regression is a type of statistical analysis used to predict the relationship between dependent and independent Variables using BEST FIT LINE. 3.) It work on the principle of ordinary least square (OLS)/ Mean square errror (MSE) Simple Linear Regression In a simple linear regression, there is one independent variable and one dependent variable. The model estimates the slope and intercept of the line of best fit, which represents the relationship between the variables. The slope represents the change in the dependent variable for each unit change in the independent variable, while the intercept represents the predicted value of the dependent variable when the independent variable is zero. Mathematically, we can represent a linear regression as: y= a0+a1x+ ε Y= Dependent Variable (Target Variable) X= Independent Variable (predictor Variab...
    Read more