Question 1: Please download the bike sharing data set from the following website: https://archive.ics.uci.edu/ml/datasets/bike+sharing+dataset. (I also include the data set on crowdmark.) This dataset contains the hourly and daily count of rental bikes between years 2011 and 2012 in Capital bikeshare system with the corresponding weather and sea- sonal information. There are two data sets and we will use the day.csv. Import the data into R and perform the following exploratory analysis.
a) The variable ”registered” records the number of registered users used the bike sharing service on a particular day. Please provide the mean value of the variable ”registered” for each day of the week.
b) Plot the conditional density plot of the variable ”registered” conditional on each month of the year.
c) Produce a two-dimensional levelplot of the variable ”registered” against the combina- tion of temperature (variable ”temp”) and humidity (variable ”hum”).
Question 2: Perform linear regression model on the bike sharing data set from Question 2.
- Provide the summary result of the regression model with ”registered” as the response variable and ”temp”, ”hum” as the predictors.
- What other predictors do you think might be important for the modelling of the variable ”registered”? Please construct another linear model including more predictors and provide the summary result of the second model.
- Perform 100 times of 5-fold cross validation and each time you randomly partition the dataset into five equal parts. You will use 80% of the data as the training data and 20% as the validating data. For models in a) and b), calculate the total sum of squared prediction error divived by the size of the validation data and by the number of cross-validations. Which model has better predictive power?
Question 3: In the following marketing set, we have 9 years with the sales in 10 million euro and the advertising expenditure in million euro.
1
Year Sales Advertisement 1 34 23
2 56 36
3 65 45
4 86 52 5 109 53 6 109 58 7 122 63 8 124 68 9 131 70
a) Formulate the response vector Y, which has nine entries.
b) Formulate the data matrix of X, the first column should be all ones corresponding to the intercept, and the second column should be the predictors. The dimension of X should be 9 ⇥ 2.
c) Write R code to compute XtX.
d) Write R code to compute ✓ = (XtX)XtY. This is the estimated linear regression
coecient of the linear model with Y as the response and X as the data matrix.
e)Run the linear regression using Y as the response and X as the predictor using lm command in R and compare the output with your own calculation.
f) Now two additional data points arrived. They are Year 10, Sales 96, and Advertisement 53; Year 11, Sales 107, and Advertisement 63. Please use the online algorithm to update the linear model. Use the two new observations together to perform the sequential learning and update the model using stochastic gradient descent algorithm using the learning rate = 0.01. Note here in the updating scheme ✓ˆnew = ✓ˆold rEn, the term En will be the sum of the squared prediction errors over the two new observations:
En = (y10 yˆ10)2 + (y11 yˆ11)2.
In this example, we implement the online algorithm when the new data come in batches, not
one at a time.
Question 4:
matrix
Consider a data set with the response vector Y = (y1, . . . , yn)t and the data
0B 1 x 1 1 x 1 2 1C B. . .C. @…A
1 xn1 xn2
We model the relationship between X and Y using the linear regression model: yi = ✓0 + ✓1xi1 + ✓2xi2 + ✏i, i = 1,…,n, where ✏ ⇠ N(0,2). Let the parameter vector be denoted as ✓ = (✓0,✓1,✓2)t. We wish to minimize the sum of squared residuals: SSE =
2
Pni=1(yi (✓0 + ✓1xi1 + ✓2xi2))2. Let the fitted value be denoted as yˆi = ✓0 + ✓1xi1 + ✓2xi2, and let the fitted value vector be denoted as Yˆ.
a) Show that SSE = Pni=1(yi yˆi)2.
b) Show that SSE = (Y Yˆ)t(Y Yˆ).
c) Show that Yˆ = X✓.
d) Simplify the derivative equation @SSE = 0. @✓
e) Find the solution of ✓ which solves the equation in part d.
Question 5: Analyze the QSAR fish toxicity data set from the site: https://archive.ics.uci.edu/ml/datasets/QSAR+fish+toxicity. Perform variable selection on the six predictors using the lasso package.
a) Based on the output of “lars”, please provide the sequence of candidate models. For example, the first model is {X5}, the second model is {X5,X3} and the third model is {X5, X3, X10}, etc.
b) Use the cross validation method, select the best value for the fraction s based on the plot of cross validation error againt the fraction s. The fraction s measures the ratio of the L1 norm of the penalized estimate over the L1 norm of the regular penalized estimate.
c) Use the optimum s you select, perform the penalized regression and output the opti- mum model and the estimated coecients.
Question 6: Simulate a data set with 100 observations yi = 30 + 5x + 2×2 + 3×3 + ✏i, where ✏i follows independent normal distribution N(0,1).
a) Perform polynomial regression on your simulated data set and using x, I(x2), I(x3) as the predictors. Compare the estimated coecients with the true model and report the R-square. b) Formulate the design matrix of this regression and write down the first two rows of the design matrix based on your data set.
c) Perform polynomial regression on your simulated data set and using x, I(x2), I(x3), I(x4) as the predictors. Compare the estimated coecients with the true model and report the R- square. Is the R-square increased or decreased compared to the model in part (a)? Explain why?
