Tech Corner
Stay ahead of the Machine Learning curve
In this blog, we’ll see what machine learning is all about. How is it growing big globally. How are businesses stepping up their game with machine learning.
In machine learning, discriminant analysis is a technique that is used for dimensionality reduction, classification, and data visualization. It is employed to reduce the number of dimensions (or variables) in a dataset while retaining as much information as is possible.
Linear discriminant analysis (LDA) is also known as normal discriminant analysis (NDA), or discriminant function analysis. It is a generalization of Fisher's linear discriminant, which is used in statistics and other fields to identify a linear combination of features that characterizes or separates two or more classes of objects or events. The linear combination found can be used as a linear classifier as well as dimensionality reduction before classification.
LDA is most commonly used for supervised classification problems. It is related rather closely to analysis of variance (ANOVA) as well as regression analysis. Both of these also seek to express a dependent variable as a linear combination of other features or measurements.
The major difference is that while LDA has continuous independent variables and a categorical dependent variable (the class label), ANOVA makes use of categorical independent variables and a continuous dependent variable.
Since logistic regression and probit regression also explain a categorical variable with the values of continuous independent variables, they happen to be more similar to linear discriminant analysis than ANOVA.

LDA is very sensitive to outliers and the size of the smallest group needs to be bigger than the number of predictor variables. The assumptions of linear discriminant analysis are exactly the same as the assumptions of Multivariate analysis of variance (MANOVA) . They are are explained here:
Independent variables necessarily have to be normal for every level of the grouping variable.
Variances among group variables need to be the same across levels of predictors. You can test for homoscedasticity using Box's M statistic.
It is preferable to use linear discriminant analysis when covariances are equal and to use quadratic discriminant analysis when covariances are not equal.
Predictive power can fall if the correlation between predictor variables rises.
Participants are randomly sampled. LDA assumes the score of a participant on one variable to be independent of all the other participants’ scores on that variable.
Linear discriminant analysis works fairly well even if these assumptions are slightly violated and it could still be reliable when dichotomous variables are being used ( even though the assumption of multivariate normality tends to be violated in this situation).

Linear discriminant analysis helps with classification, dimension reduction, and data visualization. Even though it is rather simple, it generates robust and interpretable classification results. In fact, it is usually the first method that is used while working on real-world classification problems before increasingly complex and flexible methods are employed.
Here are some noteworthy applications of LDA:
LDA is used to predict the probability of a firm going bankrupt, based on accounting ratios and other financial variables. Edward Altman’s 1968 model is still used widely in practical applications, with an accuracy of 80% to 90%, in spite of limitations like the nonconformance of accounting ratios to the normal distribution assumptions of Linear discriminant analysis.
When a computer performs facial recognition, each face is represented by a large number of pixel values. The main purpose of linear discriminant analysis over here is to reduce the number of features to a number that is more manageable. Every one of the new dimensions is a linear combination of pixel values, which form a template.
The features learned by using Principal Component Analysis (PCA) are known as Eigenfaces and the features that are learning from linear discriminant analysis are known as Fisherfaces, named after Sir Ronald Fisher.
LDA used to be employed widely in marketing to identify the factors that differentiate between various types of consumers and/or products, base on surveys or other forms of collected data. Now, other techniques like logistic regression are used more often. Here are the steps involved in using LDA for marketing:
Linear discriminant analysis is used to separate the alteration zones. If data from various zones is available, LDA can identify the pattern within the data and classify it in an effective manner. It is also used in biomedical studies, product management, and positioning.