Random notes on Lasso regression
01 Jun 2023LASSO regression
Notes taken from this blog.
LASSO stands for Least Absolute Shrinkage and Selection Operator. It adds a L1 regularization to the standard regression model.
- Shrinkage means “data values are shrunk towards a central point as the mean”. This in turn leads to sparsity hence more interpretability of the important features (since some coefficients are zeroed.)
- By adding a L1 regularizer (penalty term based on the absolute values of the coefficients), some coefficients are forced to be exactly zero. The resulting models are therefore sparser with less paramters.
- In contrast, a ridge regression uses a L2 regularization (penalty term based on the sum of squares of the coefficients) which does not necessarily lead to zero coefficients.
