CFRM405 Homework 5 Solved

Description

1. Give an example of a 2 × 2 matrix that has no real eigenvectors. Justify your solution with intuition (without solving completely for the eigenvectors and eigenvalues).
2. Consider an n×p matrix A. Show that the number of linear independent rows is the same as the number of linearly independent columns.

Hint: Write A = CR where C is a matrix of the linearly independent columns of A. Why can we write A like this? Then consider the CR product in the “row” interpretation of matrix multiplication.

3. Let A be an m×n matrix (assume m > n). The full singular value factorization A = UΣV ^T contains more information than necessary to reconstruct A.
   • What are the smallest matrices U^˜, Σ and^˜ V^˜T such that U^˜Σ^˜V^˜T = A?
   • Let . That is, think about U from the full singular value factorization as a block matrix consisting of the matrix U˜ found in part (a) and the remaining (unneeded) columns Uˆ.

Find expressions for U^˜TU^˜ and U^˜U^˜T.

• Use the reduced singular value factorization obtained in part (a) to find an expression for the matrix H = A(A^TA)⁻¹A^T. How many matrices must be inverted (diagonal and orthogonal matrices don’t count)?

4. Let x and y be vectors of m The least squares solution for a best-fit line for a plot of y versus x is

βˆ = (XTX)−1XTy

where

• Suppose you know the full singular value factorization X = UΣV ^T. Find an expression for β^ˆ in terms of U, Σ, and V . Hint: Only square matrices can be invertible.
• Repeat part (a) using the reduced singular value factorization X = U^˜Σ^˜V^˜T.

5. Let X˜ be an m × n matrix (m > n) whose columns have sample mean zero, and let X^˜ = U^˜Σ^˜V^˜T be a reduced singular value factorization of X^˜. The squared Mahalanobis distance to the point ˜x_i^T (the i^th row of X^˜) is

d2i = x˜iTSˆ−1x˜i

where        = cov(X^˜). Explain how to compute d²_i without inverting a matrix.

6. (a) Suppose A = LU where L is lower triangular and U is upper triangular. Explain how you would solve the problem Ax = b using L, U, and the concepts of forward and backward substitution.

(b) Compute the LU factorization of

• 2 −1^

−3      2 _

• 1 1

by hand using elimination matrices.