**Least Angle Regression (LAR)**

Similar to ridge regression, LAR is a shrinkage estimator. It is similar to forward selection, but only enters 'as much' of the β estimate as necessary. The β estimate is increased with each iteration of the algorithm, approaching the least squares estimate of β.

1) Start with predictors near 0, let r = y

_{i}- y

^{est}

2) Find X

^{i}most correlated with r.

3) Move β

_{est}--> β

_{ls }until corr (X

^{i}, r) = corr(X

^{j}, r) for some X

^{j}

4) Continue until all X's have been entered

If μ

_{k}= Xβ, then LAR finds μ

_{k}that makes the smallest angle between each of the predictors and r.

LAR, as a shrinkage estimator, minimizes a penalized residual sum of squares:

min{e'e + λβ)

where λ is a penalty or shrinkage factor.

**Least Absolute Shrinkage and Selection Operator (LASSO)**

In essence, with LASSO we have a LAR estimator that is penalized such that |β| < c where c is some constant.

If we minimize the following penalized residual sum of squares:

If we minimize the following penalized residual sum of squares:

min{e'e + λ|β|)

**References:**

## No comments:

## Post a Comment