Question 1
The following figure shows a neural network with two inputs, one hidden layer with two hidden neurons and one output. (For simplicity, we omit the intercept terms here). We initialize the parameters as follows: w11 = 0.1 w12 = 0.4, w21 = −0.1, w22 = −0.1,v11 = 0.06, v12 = −0.4. Given one observation x1 = 1, and x2 = 0, and the observed ouput t1 = 0, update the network parameter w11, using the learning rate λ = 0.01.
Question 2
Principal compnent method can be used to summarize the data in a lower dimension. Suppose each observation in the data set Xi has two features Xi1, and Xi2. We wish to use the principal compnent method to present the data in one dimension space. We have the following data set.
−3 6
−6 6
−8 3.5 X= −7 6
−7 5 −9 6
Calculate the first principal component for the first observation.
Question 3
ID3(S,A) is an important algorithm in the construction of decision tree.The set S denote the collection of observations. The set A denote the collection of predictors. In this question, let A = {X1, X2}. Let S be the following data set:
Y X1 X2
111
1 0 1 S = 0 0 1
0 0 0 110
We would like to build a classification tree for the response variable Y .
• What is the misclassification error rate if we do a majority vote for Y without splitting X1 or X2?
• What is the misclassification error rate if we split the data set based on X1 = 1 versus X1 = 0? What is the misclassification error rate if we split the data set based on X2 = 1 versus X2 = 0?
• Should we split the tree based on the predictor X1 or X2 or not split the tree?
• Decision tree is very sensitive to the data set. If there are small changes in the data set, the resulting tree can be very different. Ensemble method can overcome this problem and improve the performance of the decision tree? Use two or three sentences to describe what ensemble method is and name three ensemble methods that can used to improve decision trees.
Question 4
One of the hierarchical cluster algorithms is agglomerative (bottom up) procedure. The procedure starts with n singleton clusters and form hierarchy by merging most similar clusters until all the data points are merged into one single cluster. Let the distance between two data points be the Euclidean distance d(x, y) = (x1 −y1)2 +…+(xd −yd)2. Let the distance between two clusters A and B be minx∈A,y∈Bd(x,y), the minumum distance between the points from the two clusters. THere are 5 observations a, b, c, d and e. Their Euclidean distances are given in the following matrix:
abcde 0 4 3 6 11 4 0 5 7 10 3 5 0 9 2 6 7 9 0 13
1110213 0
For example, based on the matrix above, the distance between a and b is 4. Please derive the four steps in the agglomerative clustering procedure to construct the hierarchical clustering for the dataset. For each step, you need to specify which two clusters are merged and why you choose these two to merge.
Question 5
Analyze the German data set from the site: https://archive.ics.uci.edu/ml/datasets/statlog+(german+credit+ data). Apply the support vector machine analysis and the random forest analysis on the dataset. Please randomly select 800 obervations as the training set and use your two models to predict the default status of the remaining 200 loans. Repeat this cross-validation one thousand times and calculate the avergae misclassification errors of the two models.
Question 6
The idea of support vector machine (SVM) is to maximize the distance of the separating plane to the closest observation which are referred as the support vectors. Let g(x) = w0 + w1x1 + w2x2 = 0 be the separating line. For a given sample x = (x1, x2), the distance of x to the straight line g(x) = 0, is
|w0 + w1x1 + w2x2| w 12 + w 2 2
• Let the separating line be x1 + 2×2 − 3 = 0, and the given observation is x = (1.5, 1.5). Calculate the distance of the observation to the separating line.
• In the linear SVM, the dot product xTi xj is an important operation which facilitates the calculation
of the Euclidean distance. Let the nonlinear mapping of the sample from the original space to the
projected space by φ. In nonlinear SVM, the dot product between the images of the mapping φ(xi)
spacex =(x ,x )andx =(x ,x ). Thenonlinearmappingsareφ(x)=(x2 ,x2 , 2x x )and i i1 i2 j j1 j2 i i1 i2 i1 i2
and φ(x ) are calculated by the kernel function K(x , x ) = φ(x )T φ(x ). Suppose in the original
j ijij
√
√
φ(xj) = (x2j1,x2j2,
function, determine the degrees of the polynomial kernel function.
Question 7
You don’t need to submit this question on Crowdmark. This question is only for your practice. In the following table we have the playlist of 10 Spotify users. There are 5 artists A, B, C, D and E. If th user chooses the artist, the corresponding entry will be 1, otherwise, it will be zero.
obs A B C D E 111011 210110 301110 401100 501101 610001 711111 801110 900111
10 1 0 1 1 1
• Suppose A is the antecendent and B is the consequent. Calculate the confidence of B and the lift of A on B. Based on the lift value, do you recommend B to the user after the user has played artist A? Why?
