[SOLVED] CS1009 Homework 4- Regularization

In this homework, we will focus on regularization and cross validation. We will continue to build regression models for the Capital Bikeshare program in Washington D.C. See homework 3 for more information about the Capital Bikeshare data that we’ll be using extensively.

Data Preparation

Question 1

In HW3 Questions 1-3, you preprocessed the data in preparation for your regression analysis. We ask you to repeat those steps (particularly those in Question 3) so that we can compare the analysis models in this HW with those you developed in HW3. In this HW we’ll be using models from sklearn exclusively (as opposed to statsmodels)

1.1 [From HW3] Read data/BSS_train.csv and data/BSS_test.csv into dataframes BSS_train and BSS_test , respectively. Remove the dteday column from both train and test dataset. We do not need it, and its format cannot be used for analysis. Also remove the casual and registered columns for both training and test datasets as they make count trivial.

1.2 Since we’ll be exploring Regularization and Polynomial Features, it will make sense to standardize our data. Standardize the numerical features. Store the dataframes for the processed training and test predictors into the variables X_train and X_test . Store the appropriately shaped numpy arrays for the corresponding train and test count columns into y_train and y_test .

1.3 Use the LinearRegression library from sklearn to fit a multiple linear regression model to the training set data in X_train . Store the fitted model in the variable BikeOLSModel .

1.4 What are the training and test set 𝑅² scores? Store the training and test 𝑅² scores of the

BikeOLSModel in a dictionary BikeOLS_r2scores using the string ‘training’ and ‘test’ as keys.

1.5 We’re going to use bootstrapped confidence intervals (use 500 bootstrap iterations) to determine which of the estimated coefficients for the BikeOLSModel are statistically significant at a significance level of 5% .

We’ll do so by creating 3 different functions:

1. make_bootstrap_sample(dataset_X, dataset_y) returns a bootstrap sample of dataset_X and dataset_y
2. calculate_coefficients(dataset_X, dataset_y, model) returns in the form of a dictionary regression coefficients calculated by your model on dataset_X and dataset_y . The keys for regression coefficients dictionary should be the names of the features. The values should be the coefficient values of that feature calculated on your model. An example would be {‘hum’: 12.3, ‘windspeed’: -1.2, ‘Sunday’: 0.6 … }
3. get_significant_predictors(regression_coefficients, significance_level) takes as input a list of regression coefficient dictionaries (each one the output of calculate_coefficients and returns a python list of the feature names of the significant predictors e.g. [‘Monday’, ‘hum’, ‘holiday’, … ]

In the above functions dataset_X should always be a pandas dataframe with your features, dataset_y a numpy column vector with the values of the response variable and collectively they form the dataset upon which the operations take place. model is the sklearn regression model that will be used to generate the regression coefficients. regression_coefficients is a list of dictionaries of numpy arrays with each numpy array containing the regression coefficients (not including the intercept) calculated from one bootstrap sample. significance_level represents the significance level as a floating point number. So a 5% significance level should be represented as 0.05.

Store the feature names as a list of strings in the variable BikeOLS_significant_bootstrap and print them for your answer.