VIF quantifies multicollinearity in OLS regressions

Assesses how much variation is increased by multicollinearity

High VIF indicated that predictor variables can be linearly predicted by each other

\(VIF_j = \frac{1}{1-R^{2}_j}\)

\(VIF_j\) is the Variance Inflation Factor for the \(j^{th}\) predictor variable.

\(R^{2}_j\) is the coefficient of determination obtained by regressing the \(j^{th}\) predictor variable against all other predictor variables.

Ranges from 1-5 (or 10)

\(RSS + \lambda \sum_{j=1}^{p} \beta_j^2\)

**Penalized Regression**: Adds a penalty to OLS to regularize coefficients, aiding in handling multicollinearity and reducing complexity.

**Coefficient Shrinkage**: Coefficients shrink towards zero, enhancing stability and accuracy.

**L2 Regularization**: Employs squared coefficient sum as a penalty, regulated by \(\lambda\).

**Bias-Variance Trade-off**: Slightly increases bias to reduce variance, preventing overfitting.

**Efficient Computation**: Features a closed-form solution, ensuring computational efficiency.

**No Feature Elimination**: Maintains all features due to non-zero coefficients, unlike Lasso.

**Effective in** \(p > n\): Remains effective when predictors outnumber observations.

**Interpretability**: Less interpretable because all predictors are included.