Getting Started with HW 2

DSAN 5300: Statistical Learning

Update Log

• Original version posted 10 Feb 2025, 10:00pm

Full Text for HW-2.5: ISLR-6.8 #8(a-d)

Here, for ease of access (since the problem in this case is from the previous edition of ISLR, pdf here—thank you Prof. James for the PDF link!), is the full text of Section 6.8 Problem 8. Remember that you only need to do (a) through (d)! The full problem is here just for completeness (you can think about how you’d approach parts (e) and (f)).

ISLR Section 6.8, Exercise #8

In this exercise, we will generate simulated data, and will then use this data to perform best subset selection.

1. Use the rnorm() function to generate a predictor $X$ of length $n=100$, as well as a noise vector $ϵ$ of length $n=100$.

2. Generate a response vector $Y$ of length $n=100$ according to the model

   $$Y={\beta }_0+{\beta }_{1}X+{\beta }_2{X}^2+{\beta }_3{X}^3+ϵ,$$

   where ${\beta }_0$, ${\beta }_1$, ${\beta }_2$, and ${\beta }_3$ are constants of your choice.

3. Use the regsubsets() function to perform best subset selection in order to choose the best model containing the predictors $X,X^2,\dots ,X^{10}$. What is the best model obtained according to $C_p$, BIC, and adjusted $R^2$? Show some plots to provide evidence for your answer, and report the coefficients of the best model obtained. Note you will need to use the data.frame() function to create a single data set containing both $X$ and $Y$.

4. Repeat (c), using forward stepwise selection and also using backwards stepwise selection. How does your answer compare to the results in (c)?

5. Now fit a lasso model to the simulated data, again using $X,X^2,\dots ,X^{10}$ as predictors. Use cross-validation to select the optimal value of $\lambda$. Create plots of the cross-validation error as a function of $\lambda$. Report the resulting coefficient estimates, and discuss the results obtained.

6. Now generate a response vector $Y$ according to the model

   $$Y={\beta }_0+{\beta }_7{X}^7+ϵ,$$

   and perform best subset selection and the lasso. Discuss the results obtained.