In this homework, you will implement the linear discriminant analysis algorithm in Matlab, Python, or R. Here are the steps you need to follow:
- Read Section 6.8 from the textbook.
- You are given a data set, which contains 3000 training and 3000 test images in the files hw07_training_images.csv and hw07_test_images.csv together with the labels in the files hw07_training_labels.csv and hw07_test_labels.csv. These data points are clothing images of size 28 pixels × 28 pixels (i.e., 784 pixels). These images are from three distinct classes, namely, T-shirt, trouser, and dress. The figure below shows five sample images from each class.
- Implement the linear discriminant analysis algorithm. (Hint: If the within-class scatter matrix is not invertible, you can, for example, add 1e-10 to its diagonal entries to get rid of the singularity.)
- Calculate the two-dimensional projections of the training and test data points using the linear discriminant analysis by setting 𝑅 to 2. Draw the two-dimensional projections of the training and test data points. Your figures should be similar to the following figures.
4 3 2 1 0
−1 −2
Training points
Test points
Dimension 2
Dimension 2
−2 0 2 4 6 Dimension 1
−2 0 2 4 6 Dimension 1
T−shirt trouser dress
4 3 2 1 0
−1 −2
T−shirt trouser dress
5. For two-dimensional representation, learn a five-nearest neighbor classifier using the projections of training data points and calculate the confusion matrices on the projections of training and test data points, respectively. Your results should be similar to the following confusion matrices.
y_predicted y_train 1 2 3 1886 14112 2 40845 89 3 87 47880
y_predicted y_test 1 2 3 1 834 8 158 2 49851100 3133 61806
