{"id":175044,"date":"2021-03-08T11:07:11","date_gmt":"2021-03-08T05:37:11","guid":{"rendered":"https:\/\/www.jigsawacademy.com\/?p=175044"},"modified":"2022-07-06T20:02:26","modified_gmt":"2022-07-06T14:32:26","slug":"blogs-ai-ml-ridge-regression","status":"publish","type":"post","link":"https:\/\/www.jigsawacademy.com\/blogs\/ai-ml\/ridge-regression","title":{"rendered":"Ridge Regression: An Interesting Overview In 2021"},"content":{"rendered":"\r\n

Introduction<\/strong><\/h2>\r\n\r\n\r\n\r\n

When one analyses data with multicollinearity, the technique of \u2018Ridge regression\u2019 is used for model tuning using\u00a0ridge and lasso regression\u00a0or L2 regularization. It is useful since whenever multicollinearity happens in the data, the data exhibits large variances, and the least-squares will be unbiased. Thus the predicted values and actual values will have large variances.<\/p>\r\n\r\n\r\n\r\n

The ridge regression cost function denoted below Lambda is the ridge function penalty term denoted by the alpha parameter.<\/p>\r\n\r\n\r\n\r\n

Min(||Y \u2013 X(theta)||^2 \u03bb||theta||^2)<\/p>\r\n\r\n\r\n\r\n

If values of alpha get bigger, the penalty is larger, and the coefficient’s magnitude is smaller. Thus it can prevent multicollinearity by parameter-shrinking, and further model complexity is reduced due to the shrinkage of the coefficient.<\/p>\r\n\r\n\r\n\r\n

In this article let us look at:<\/p>\r\n\r\n\r\n\r\n

    \r\n
  1. Ridge Regression Models<\/a><\/strong><\/li>\r\n
  2. Standardization<\/a><\/strong><\/li>\r\n
  3. Assumptions of Ridge Regressions<\/a><\/strong><\/li>\r\n
  4. Linear Regression Model<\/a><\/strong><\/li>\r\n
  5. Regularization<\/a><\/strong><\/li>\r\n<\/ol>\r\n\r\n\r\n\r\n

    1. Ridge Regression Models<\/strong><\/h2>\r\n\r\n\r\n\r\n

    For machine learning models, the ridge regression formula\u00a0is given by<\/p>\r\n\r\n\r\n\r\n

    Y = XB e<\/p>\r\n\r\n\r\n\r\n

    Here, Y- dependent variable, X- independent variables, e- residual errors and B- regression coefficients in the ridge regression derivation. When the lambda function is also considered and identified L2 regularization data is ready, one can undertake standardization.<\/p>\r\n\r\n\r\n\r\n

    2. Standardization<\/strong><\/h2>\r\n\r\n\r\n\r\n

    Ridge regression\u00a0uses standardized variables. Hence to standardize the independent and dependent variables, their mean value needs to be subtracted and divided by the standard deviation. However, one will need to notate whether the variables have been standardized and ensure that the final values of displayed regression coefficients are in the original scale. Thus, the ridge trace is always a standardized scale.<\/p>\r\n\r\n\r\n\r\n

    Bias and variance trade-off:<\/p>\r\n\r\n\r\n\r\n

    Actual dataset ridge regression building makes a trade-off between variance and bias, which follows the trends in the \u03bb function mentioned below.<\/p>\r\n\r\n\r\n\r\n