Is General Linear Models under the umbrella of Generalized Linear Model(GLM)?yes…then How?
Hello folks, it’s been long time since I wrote an article on medium. But now I’ll try to write more often. I will use GLM as short hand for Generalized Linear Model.
So my focus here is to give a basic idea about Why Generalized linear models(GLM) became the main tool of applied statisticians. I am sure many of us have heard about Linear Regression, Multiple Linear Regression, Logistic Regression, Poisson Regression, Binomial Regression and also general linear models. Note that I did not need to include Linear Regression, Multiple Linear Regression when I used term General Linear Model because Linear Regression and Multiple Linear Regression are nothing but the specific Linear Models. But since these are more familiar terms so I used them. Note that The term general linear model usually refers to conventional linear regression models for a continuous response variable given continuous and/or categorical predictors. It includes multiple linear regression, as well as ANOVA and ANCOVA (with fixed effects only).
One important thing to know before going into GLM is to know about the type of response. And focus on the image below:-
So I want you to focus on the box highlighted which means Yi should follow Normal distribution. So we use General Linear model mainly when:-
- Our response variable and error term follows Normal distribution.
- Relationship between response and explanatory variable is linear.
- Errors are homoscedastic and non-correlated.
- If the independent explanatory variables are not correlated.
But in the real life situation is not always Normal right? Then If situation gets non-normal ,I mean if our response and errors are non -normal then what? We can make appropriate transformations but we can not make all the assumptions satisfy at the same time. There are several other issues which we face after transforming the data in General Linear Model, but I’ll not discuss them here.
Hence “The generalized linear model (GLM) is a flexible generalization of General Linear model that allows for response variable that have error distribution models normal and non-normal distribution as well. It allows the linear predictor (e.g. b0+b1*X)to be related to the response via a function which is called link function.”
I hope you till here you must have get a basic idea about why GLM.GLM is a wider class of models under which allows to handle normal response and non-normal responses such as categorical response ,count response ,proportion response etc. All these types of responses are modelled using some known probability distributions. And they all share some common properties and comes under Exponential family. Here take a look at exponential family:-
If you can write a probability distribution in the above form f then you distribution will belong to exponential family and the examples of distributions belonging to exponential family are Binomial, Poisson, Normal ,exponential etc.
The term generalized linear model (GLM) refers to a larger class of models and was used by McCullagh and Nelder. There are three important concepts to understand the GLM framework.
- Random Component:-means the probability distribution of response yi.
- Systematic Component:- linear combination of explanatory variables .
- Link Function:- defines how the expected value of response is related to the linear predictor of explanatory variables.
These models follow the assumptions below.
Note that General Linear Models are specific GLMS when errors are independent and follows normal distribution. And the link function is identity because we model the mean directly in case of General Linear Models.
Conclusion:-
My purpose here was to give a brief overview of Why we need GLM when there are already so many statistical algorithms are there. So in this article at first we knew some of the famous statistical algorithm like Linear and Multiple Linear Regression, anova but we realized these are the specific models of a broader class called General Linear Model. Then we reviewed the assumptions of General Linear models and realized that in real life satisfying all these assumptions at the same time is quite difficult. Then we got a brief introduction about another class called GLM which can handle our normal and non-normal worries as well. Later we saw a few important concepts to understand framework of GLMs and then an overview of general assumptions followed by this larger class of Models. And in the end we realized that General Linear Models are specific kind of GLMs.
Please leave your feedback and if you like it then just give a clap and share with your folks. Your appreciation will motivate me to write further on this topic in depth. To understand the this topic I will suggest you to go through the references.
References:-
- An Introduction to Generalized Linear Models, second edition by Annette Dobson.
- Extending the Linear Models By Julian Faraway.
- https://www.ime.usp.br/~abe/lista/pdftGzimaFtH4.pdf
- https://online.stat.psu.edu/stat504/lesson/6/6.1
