Getting Started with HW 2

DSAN 5300: Statistical Learning

Published

Sunday, March 9, 2025

Update Log
  • Original version posted 10 Feb 2025, 10:00pm

Full Text for HW-2.5: ISLR-6.8 #8(a-d)

Here, for ease of access (since the problem in this case is from the previous edition of ISLR, pdf here—thank you Prof. James for the PDF link!), is the full text of Section 6.8 Problem 8. Remember that you only need to do (a) through (d)! The full problem is here just for completeness (you can think about how you’d approach parts (e) and (f)).

ISLR Section 6.8, Exercise #8

In this exercise, we will generate simulated data, and will then use this data to perform best subset selection.

  1. Use the rnorm() function to generate a predictor X of length n=100, as well as a noise vector ϵ of length n=100.

  2. Generate a response vector Y of length n=100 according to the model

    Y=β0+β1X+β2X2+β3X3+ϵ,

    where β0, β1, β2, and β3 are constants of your choice.

  3. Use the regsubsets() function to perform best subset selection in order to choose the best model containing the predictors X,X2,,X10. What is the best model obtained according to Cp, BIC, and adjusted R2? Show some plots to provide evidence for your answer, and report the coefficients of the best model obtained. Note you will need to use the data.frame() function to create a single data set containing both X and Y.

  4. Repeat (c), using forward stepwise selection and also using backwards stepwise selection. How does your answer compare to the results in (c)?

  5. Now fit a lasso model to the simulated data, again using X,X2,,X10 as predictors. Use cross-validation to select the optimal value of λ. Create plots of the cross-validation error as a function of λ. Report the resulting coefficient estimates, and discuss the results obtained.

  6. Now generate a response vector Y according to the model

    Y=β0+β7X7+ϵ,

    and perform best subset selection and the lasso. Discuss the results obtained.