Description

Question 1 –

Consider the following three two-dimensional data sets each containing two clusters of data points (shown with ‘circles’ and ‘crosses’). Suppose you would like to apply either Principal Components Analysis (PCA) or Linear Discriminant Analysis (LDA) to reduce the dimensionality of each of these data sets from 2-D to 1-D where the two clusters remain separated in the reduced dimensional data set. For each data set, address each of the following two questions:

1. Will PCA be effective and keeping the two clusters separated in the reduced dimensionality data? Why or why not? If yes, state what characteristics of the data set allow PCA to be effective. If no, state what characteristics of the data set cause PCA to fail.
2. Will LDA be effective and keeping the two clusters separated in the reduced dimensionality data? Why or why not? If yes, state what characteristics of the data set allow LDA to be effective. If no, state what characteristics of the data set cause LDA to fail.

Question 2

This question focuses on Principal Components Analysis and Data Whitening.

1. Suppose z is the whitened version of x where Σ is the covariance for the original data set X and µ is the mean (x is one data point in X). Write the formula for computing z from x.
2. Suppose the original data set, X, has features with widly varying range and variance. Could this data be whitened? Why would you or would you not want to perform whitening on data of this type? Generate two dimensional data with these characteristics (i.e., the two features have very different range and variances from each other). Implement python code that applies principal components analysis (without dimensionality reduction, only decorrelation) and data whitening to your generated data set. Scatter plot the original data, the data after application of the PCA transformation and the data after data whitening. What is the covariance of the original data, after PCA and after data whitening? Use these results to motivate your discussion of why you might or might not want to apply data whitening to data with features of widely varying range and variance.