COMP4434 Assignment #3 Solved

Q1. Regularization

We use polynomial regression for the prediction task of a dataset. The given dataset includes a train set (train.csv) and a test set (test.csv). To illustrate the effect of regularization, please first implement the following regression models using python language (third-party packages are allowed). Then, plot the data points of the train set and the regression lines of the trained models. Finally, compute the RMSE of the trained models using the test set and make a comparative discussion about underfitting and overfitting.

• Polynomial regression without regularization (polynomial to 5th power)
• L1 Regularized polynomial regression: 𝜆 = 1 and 𝜆 = 100
• L2 Regularized polynomial regression: 𝜆 = 1 and 𝜆 = 100

  The given datasets can be downloaded at: https://drive.google.com/drive/folders/1LSZNIEWf6XKnQtRw8L01tS6yAB67Aad2?usp=sharing

  Q2. Recommender System

  Build up a collaborative filtering-based recommender system to provide effective hotel recommendation. The training dataset as shown in the table below contains the ratings from 4 users to 3 hotels. The ratings range from 1 point to 5 points.

  Hotel 1 Hotel 2 Hotel 3 User 1 5 1 ? User 2 4 ? 3 User 3 ? 4 5 User 4 3 3 4

  We use the gradient descent algorithm to solve cost minimization in the collaborative filtering

model. Some settings are as follows.

• The constant learning rate 𝛼 = 0.0002
• The regularization parameter 𝜆 = 0.02
• The dimension for user/item feature vectors 𝐾 = 2

  0.19 0.62

• The initial values for parameters 𝑥 = [0.77 0.43 0.31] and 𝜃𝑇 = [0.68 0.78] 0.48 0.44 0.51 0.18 0.08

  0.36 0.92

a)  If we finally obtain 𝑥(1) = [1.268 0.994]𝑇 and 𝜃(3) = [0.271 0.694]𝑇 after the training procedure, what is the rating of user 3 on hotel 1?

b) Calculate the values of 𝑥(1) (i.e., the first element in the item feature vector of 1

hotel 1) and 𝜃(2) (i.e., the first element in the user feature vector of user 2) after the first 1

iteration.

c)  Implement the gradient descent algorithm to update the parameters 𝑥 and 𝜃 using python language. Please calculate the ratings of user 2 on hotel 2 after 50 rounds and upload the source code file.

ps. For a) and b), the detailed calculation process is required and the intermediate and final results should be rounded to 3 decimal places.

Q3. Neural Network

Consider the following neural network:

Where𝑎 =∑ 𝑤𝑖𝑧 𝑧 =𝑓(𝑎)for𝑖=1,2,3,4 𝑧 =𝑎 (an input neuron) 𝑓(𝑥)=relu(𝑥) 𝑖𝑗𝑗𝑗𝑖𝑖𝑖 00 3

and 𝑓 (𝑥) = 𝑓 (𝑥) = 𝑓 (𝑥) = sigmoid(𝑥). relu(𝑥) corresponds to a rectifier linear unit transfer 124

function defined as: relu(𝑥) = max {0,𝑥}. The cost function is defined as 𝐽(𝑤) = 1 (𝑧4 − 𝑦)2. 2

(a) Write a function 𝐹 to simulate the neural network.
(b) Assume that we are given a training data 𝑥 = 1.0, 𝑦 = 0.1 what is the value of 𝜕𝐽 ?

𝜕𝑤4 3