SVD and Symmetric Matrix
SVD is closely related to the diagonalization of symmetric matrices.
Given any matrix . We consider , which is an symmetric matrix (Note: ). Then we can diagonalize it as follows:
, where is an orthogonal matrix and is an diagonal matrix. We denote the diagonal entries of by and arrange them in descending order and the column vectors of are , which form an orthonormal basis because is orthogonal.
Notice that for any ,
Therefore, is orthogonal. Moreover, when , we have , which means all eigenvalues are nonnegative. Suppose . Let and for . Then we normalize for
, where .
Hence is an orthonormal set. Extend it to to form an orthonormal basis for if .
Define for and for . Hence for and we have
, where is the orthogonal matrix formed by and is the diagonal matrix with diagonal entries .
Hence we obtain i.e. the SVD of .
From the above construction, we have the following four fundamental subspaces: