fat, in percentx_001, …, x_100
\[\mathbf{X}^\top\mathbf{X}\,b = \mathbf{X}^\top y\]
\[H_0: \beta_1 = \cdots = \beta_d = 0\]
\[C_p = \frac{1}{n}\left(RSS + 2d\widehat\sigma^2\right), \qquad \text{BIC} = \frac{1}{n}\left[RSS + \log(n)\,d\widehat\sigma^2\right]\]
AIC() and BIC() in R: \(\sigma^2\) estimated in each model by \(RSS/n\)\[-2\log L = n\log\left(2\pi\,RSS/n\right) + n\]

\[\widehat\beta^R_\lambda = \left(\tilde{\mathbf{X}}^\top\tilde{\mathbf{X}} + \lambda\mathbf{I}_p\right)^{-1}\tilde{\mathbf{X}}^\top\left(y - \bar y\mathbf{1}\right)\]
\[\tilde{\mathbf{x}}_{\cdot j}^\top r = \frac{\lambda}{2}\,\text{sign}\left(\widehat\beta^L_{\lambda j}\right) \quad \text{if } \widehat\beta^L_{\lambda j} \ne 0, \qquad \left|\tilde{\mathbf{x}}_{\cdot j}^\top r\right| \le \frac{\lambda}{2} \quad \text{if } \widehat\beta^L_{\lambda j} = 0\]
| Method | Tuning | Nonzero | 10-fold CV | Test MSE |
|---|---|---|---|---|
| Null | 0 | 192.81 | 153.95 | |
| Ridge | \(\lambda = 1.6 \times 10^{-4}\) | 100 | 10.86 | 8.09 |
| Lasso | \(\lambda = 3.2 \times 10^{-3}\) | 20 | 10.27 | 13.26 |
| PCR | \(M = 12\) | 100 | 10.23 | 8.71 |
| PLS | \(M = 10\) | 100 | 10.29 | 9.52 |

AIC() and BIC() go to \(-\infty\) at a perfect fitDS 3030