Description

Problem 1 [25%]

It is mentioned in Chapter 7 of ISL that a cubic regression spline with one knot at ξ can be obtained using a basis of the form x, x², x³, [x − ξ]³₊, where [x − ξ]³₊ = (x − ξ)³ if x > ξ and equals 0 otherwise. We will now show that a function of the form

f(x) = β₀ + β₁x + β₂x² + β₃x³ + β₄[x − ξ]₊³

is indeed a cubic regression spline, regardless of the values of β₀,β₁,β₂, β₃,β₄.

1. Find a cubic polynomial

f₁(x) = a₁ + b₁x + c₁x² + d₁x³

such that f(x) = f₁(x) for all x ≤ ξ. Express a₁,b₁,c₁,d₁ in terms of β₀,β₁,β₂,β₃,β₄.

2. Find a cubic polynomialf₂(x) = a₂ + b₂x + c₂x² + d₂x³

such that f(x) = f₂(x) for all x > ξ. Express a₂,b₂,c₂,d₂ in terms of β₀,β₁,β₂,β₃,β₄. We have now established that f(x) is a piecewise polynomial.

3. Show that f₁(ξ) = f₂(ξ). That is, f(x) is continuous at ξ.

Problem 2 [25%]

Use linear, cubic, and natural regression splines investigated Chapter 7 of ISL to the Auto data set. Is there evidence for non-linear relationships in this data set? Create some informative plots to justify your answer.

Problem 3 [25%]

You will now derive the Bayesian connection to the lasso as discussed in Section 6.2.2. of ISL.

1. Suppose that y_i = β₀ + ^Pp_j=1 x_ijβ_j + _i where ₁,…,_n are independent and identically distributed from a normal distribution N(0,1). Write out the likelihood for the data as a function of values β.
2. Assume that the prior for β : β₁,…,β_p is that they are independent and identically distributed according to a Laplace distribution with mean zero and variance c. Write out the posterior for β in this setting using Bayes theorem.
3. Argue that the lasso estimate is the value of β with maximal probability under this posterior distribution. Compute log of the probability in order to make this point. Hint: The denominator (= the probability of data) can be ignored in computing the maximum probability.
4. Suppose that ₁,…,_n are independent and identically distributed according to the Laplace distribution.

What are the maximum likelihood/MAP estimates of β_i under this assumption? Hint: See https: //en.wikipedia.org/wiki/Least_absolute_deviations

1

Problem 4 [25%]

Based on a true story, according to: The Drunkard’s Walk: How Randomness Rules Our Lives, Leonard Mlodinow

Suppose that you applied for a life insurance and underwent a physical exam. The bad news is that your application was rejected because you tested positive for HIV. The test’s sensitivity is 99.7% and specificity is 98.5% [https://en.wikipedia.org/wiki/Diagnosis_of_HIV/AIDS#Accuracy_of_HIV_testing]. However, after studying the CDC website, you find that in your ethnic group (age, gender, race, …) only one in 10,000 people is infected. What is the probability that you actually have HIV?

2