Opening the book…
- Understand how to find the singular value decomposition of a matrix.
The Singular Value Decomposition
We begin this section with an important definition.
Let \(A\) be an \(m\times n\) matrix. The singular values of \(A\) are the square roots of the positive eigenvalues of \(A^TA.\)
Singular Value Decomposition (SVD) can be thought of as a generalization of orthogonal diagonalization of a symmetric matrix to an arbitrary \(m\times n\) matrix. This decomposition is the focus of this section.
The following is a useful result that will help when computing the SVD of matrices.
Let \(A\) be an \(m \times n\) matrix. Then \(A^TA\) and \(AA^T\) have the same nonzero eigenvalues.
- Proof
-
Suppose \(A\) is an \(m\times n\) matrix, and suppose that \(\lambda\) is a nonzero eigenvalue of \(A^TA\). Then there exists a nonzero vector \(X\in \mathbb{R}^n\) such that \[\label{nonzero} (A^TA)X=\lambda X.\]
Multiplying both sides of this equation by \(A\) yields: \[\begin{aligned} A(A^TA)X & = A\lambda X\\ (AA^T)(AX) & = \lambda (AX).\end{aligned}\] Since \(\lambda\neq 0\) and \(X\neq 0_n\), \(\lambda X\neq 0_n\), and thus by equation \(\eqref{nonzero}\), \((A^TA)X\neq 0_m\); thus \(A^T(AX)\neq 0_m\), implying that \(AX\neq 0_m\).
Therefore \(AX\) is an eigenvector of \(AA^T\) corresponding to eigenvalue \(\lambda\). An analogous argument can be used to show that every nonzero eigenvalue of \(AA^T\) is an eigenvalue of \(A^TA\), thus completing the proof.
Notice that \(AA^T\) and \(A^TA\) are symmetric, so they will each have an orthonormal set of eigenvectors. Given an \(m\times n\) matrix \(A\), we will see how to express \(A\) as a product \[A=U\Sigma V^T\nonumber \] where
- \(U\) is an \(m\times m\) orthogonal matrix whose columns are eigenvectors of \(AA^T\).
- \(V\) is an \(n\times n\) orthogonal matrix whose columns are eigenvectors of \(A^TA\).
- \(\Sigma\) is an \(m\times n\) matrix whose only nonzero values lie on its main diagonal, and are the singular values of \(A\).
How can we find such a decomposition? We are aiming to decompose \(A\) in the following form:
\[A=U\left[ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right] V^T\nonumber \] where \(\sigma\) is of the form \[\sigma =\left[ \begin{array}{ccc} \sigma _{1} & & 0 \\ & \ddots & \\ 0 & & \sigma _{k} \end{array} \right]\nonumber \]
Thus \(A^T=V\left[ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right] U^T\) and it follows that \[A^TA=V\left[ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right] U^TU\left[ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right] V^T=V\left[ \begin{array}{cc} \sigma ^{2} & 0 \\ 0 & 0 \end{array} \right] V^T\nonumber \] and so \(A^TAV=V\left[ \begin{array}{cc} \sigma ^{2} & 0 \\ 0 & 0 \end{array} \right] .\) Similarly, \(AA^TU=U\left[ \begin{array}{cc} \sigma ^{2} & 0 \\ 0 & 0 \end{array} \right] .\) Therefore, you would find an orthonormal basis of eigenvectors for \(AA^T\) make them the columns of a matrix such that the corresponding eigenvalues are decreasing. This gives \(U.\) You could then do the same for \(A^TA\) to get \(V\).
We formalize this discussion in the following theorem.
Let \(A\) be an \(m\times n\) matrix. Then there exist orthogonal matrices \(U\) and \(V\) of the appropriate size such that \(A= U \Sigma V^T\) where \(\Sigma\) is of the form \[\Sigma = \left[ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right]\nonumber \] and \(\sigma\) is of the form \[\sigma =\left[ \begin{array}{ccc} \sigma _{1} & & 0 \\ & \ddots & \\ 0 & & \sigma _{k} \end{array} \right]\nonumber \] for the \(\sigma _{i}\) the singular values of \(A.\)
- Proof
-
There exists an orthonormal basis, \(\left\{ \vec{v}_{i}\right\} _{i=1}^{n}\) such that \(A^TA\vec{v}_{i}=\sigma _{i}^{2}\vec{v}_{i}\) where \(\sigma _{i}^{2}>0\) for \(i=1,\cdots ,k,\left( \sigma _{i}>0\right)\) and equals zero if \(i>k.\) Thus for \(i>k,\) \(A\vec{v}_{i}=\vec{0}\) because \[A\vec{v}_{i}\cdot A\vec{v}_{i} = A^TA\vec{v}_{i} \cdot \vec{v}_{i} = \vec{0} \cdot \vec{v}_{i} =0.\nonumber \] For \(i=1,\cdots ,k,\) define \(\vec{u}_{i}\in \mathbb{R}^{m}\) by \[\vec{u}_{i}= \sigma _{i}^{-1}A\vec{v}_{i}.\nonumber \]
Thus \(A\vec{v}_{i}=\sigma _{i}\vec{u}_{i}.\) Now \[\begin{aligned} \vec{u}_{i} \cdot \vec{u}_{j} &= \sigma _{i}^{-1}A \vec{v}_{i} \cdot \sigma _{j}^{-1}A\vec{v}_{j} = \sigma_{i}^{-1}\vec{v}_{i} \cdot \sigma _{j}^{-1}A^TA\vec{v}_{j} \\ &= \sigma _{i}^{-1}\vec{v}_{i} \cdot \sigma _{j}^{-1}\sigma _{j}^{2} \vec{v}_{j} = \frac{\sigma _{j}}{\sigma _{i}}\left( \vec{v}_{i} \cdot \vec{v}_{j}\right) =\delta _{ij}.\end{aligned}\] Thus \(\left\{ \vec{u}_{i}\right\} _{i=1}^{k}\) is an orthonormal set of vectors in \(\mathbb{R}^{m}.\) Also, \[AA^T\vec{u}_{i}=AA^T\sigma _{i}^{-1}A\vec{v}_{i}=\sigma _{i}^{-1}AA^TA\vec{v}_{i}=\sigma _{i}^{-1}A\sigma _{i}^{2}\vec{v} _{i}=\sigma _{i}^{2}\vec{u}_{i}.\nonumber \] Now extend \(\left\{ \vec{u}_{i}\right\} _{i=1}^{k}\) to an orthonormal basis for all of \(\mathbb{R}^{m},\left\{ \vec{u}_{i}\right\} _{i=1}^{m}\) and let \[U= \left[ \begin{array}{ccc} \vec{u}_{1} & \cdots & \vec{u}_{m} \end{array} \right ]\nonumber \] while \(V= \left( \vec{v}_{1}\cdots \vec{v}_{n}\right) .\) Thus \(U\) is the matrix which has the \(\vec{u}_{i}\) as columns and \(V\) is defined as the matrix which has the \(\vec{v}_{i}\) as columns. Then \[U^TAV=\left[ \begin{array}{c} \vec{u}_{1}^T \\ \vdots \\ \vec{u}_{k}^T \\ \vdots \\ \vec{u}_{m}^T \end{array} \right] A\left[ \vec{v}_{1}\cdots \vec{v}_{n}\right]\nonumber \] \[=\left[ \begin{array}{c} \vec{u}_{1}^T \\ \vdots \\ \vec{u}_{k}^T \\ \vdots \\ \vec{u}_{m}^T \end{array} \right] \left[ \begin{array}{cccccc} \sigma _{1}\vec{u}_{1} & \cdots & \sigma _{k}\vec{u}_{k} & \vec{0} & \cdots & \vec{0} \end{array} \right] =\left[ \begin{array}{cc} \sigma & 0 \\ 0 & 0 \end{array} \right]\nonumber \] where \(\sigma\) is given in the statement of the theorem.
The singular value decomposition has as an immediate corollary which is given in the following interesting result.
Let \(A\) be an \(m\times n\) matrix. Then the rank of \(A\) and \(A^T\)equals the number of singular values.
Let’s compute the Singular Value Decomposition of a simple matrix.
Let \(A=\left[\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right]\). Find the Singular Value Decomposition (SVD) of \(A\).
Solution
To begin, we compute \(AA^T\) and \(A^TA\). \[AA^T = \left[\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right] \left[\begin{array}{rr} 1 & 3 \\ -1 & 1 \\ 3 & 1 \end{array}\right] = \left[\begin{array}{rr} 11 & 5 \\ 5 & 11 \end{array}\right].\nonumber \]
\[A^TA = \left[\begin{array}{rr} 1 & 3 \\ -1 & 1 \\ 3 & 1 \end{array}\right] \left[\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right] = \left[\begin{array}{rrr} 10 & 2 & 6 \\ 2 & 2 & -2\\ 6 & -2 & 10 \end{array}\right].\nonumber \]
Since \(AA^T\) is \(2\times 2\) while \(A^T A\) is \(3\times 3\), and \(AA^T\) and \(A^TA\) have the same nonzero eigenvalues (by Proposition \(\PageIndex{1}\)), we compute the characteristic polynomial \(c_{AA^T}(x)\) (because it’s easier to compute than \(c_{A^TA}(x)\)).
\[\begin{aligned} c_{AA^T}(x)& = \det(xI-AA^T)= \left|\begin{array}{cc} x-11 & -5 \\ -5 & x-11 \end{array}\right|\\ & = (x-11)^2 - 25 \\ & = x^2-22x+121-25\\ & = x^2-22x+96\\ & = (x-16)(x-6)\end{aligned}\]
Therefore, the eigenvalues of \(AA^T\) are \(\lambda_1=16\) and \(\lambda_2=6\).
The eigenvalues of \(A^TA\) are \(\lambda_1=16\), \(\lambda_2=6\), and \(\lambda_3=0\), and the singular values of \(A\) are \(\sigma_1=\sqrt{16}=4\) and \(\sigma_2=\sqrt{6}\). By convention, we list the eigenvalues (and corresponding singular values) in non increasing order (i.e., from largest to smallest).
To find the matrix \(V\):
To construct the matrix \(V\) we need to find eigenvectors for \(A^TA\). Since the eigenvalues of \(AA^T\) are distinct, the corresponding eigenvectors are orthogonal, and we need only normalize them.
\(\lambda_1=16\): solve \((16I-A^TA)\vec{y}= 0\). \[\left[\begin{array}{rrr|r} 6 & -2 & -6 & 0 \\ -2 & 14 & 2 & 0 \\ -6 & 2 & 6 & 0 \end{array}\right] \rightarrow \left[\begin{array}{rrr|r} 1 & 0 & -1 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right], \mbox{ so } \vec{y}=\left[\begin{array}{r} t \\ 0 \\ t \end{array}\right] =t\left[\begin{array}{r} 1 \\ 0 \\ 1 \end{array}\right], t\in \mathbb{R}.\nonumber \]
\(\lambda_2=6\): solve \((6I-A^TA)\vec{y}= 0\). \[\left[\begin{array}{rrr|r} -4 & -2 & -6 & 0 \\ -2 & 4 & 2 & 0 \\ -6 & 2 & -4 & 0 \end{array}\right] \rightarrow \left[\begin{array}{rrr|r} 1 & 0 & 1 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right], \mbox{ so } \vec{y}=\left[\begin{array}{r} -s \\ -s \\ s \end{array}\right] =s\left[\begin{array}{r} -1 \\ -1 \\ 1 \end{array}\right], s\in \mathbb{R}.\nonumber \]
\(\lambda_3=0\): solve \((-A^TA)\vec{y}= 0\). \[\left[\begin{array}{rrr|r} -10 & -2 & -6 & 0 \\ -2 & -2 & 2 & 0 \\ -6 & 2 & -10 & 0 \end{array}\right] \rightarrow \left[\begin{array}{rrr|r} 1 & 0 & 1 & 0 \\ 0 & 1 & -2 & 0 \\ 0 & 0 & 0 & 0 \end{array}\right], \mbox{ so } \vec{y}=\left[\begin{array}{r} -r \\ 2r \\ r \end{array}\right] =r\left[\begin{array}{r} -1 \\ 2 \\ 1 \end{array}\right], r\in \mathbb{R}.\nonumber \]
Let \[\vec{v}_1=\frac{1}{\sqrt{2}}\left[\begin{array}{r} 1\\ 0\\ 1 \end{array}\right], \vec{v}_2=\frac{1}{\sqrt{3}}\left[\begin{array}{r} -1\\ -1\\ 1 \end{array}\right], \vec{v}_3=\frac{1}{\sqrt{6}}\left[\begin{array}{r} -1\\ 2\\ 1 \end{array}\right].\nonumber \]
Then \[V=\frac{1}{\sqrt{6}}\left[\begin{array}{rrr} \sqrt 3 & -\sqrt 2 & -1 \\ 0 & -\sqrt 2 & 2 \\ \sqrt 3 & \sqrt 2 & 1 \end{array}\right].\nonumber \]
Also, \[\Sigma = \left[\begin{array}{rrr} 4 & 0 & 0 \\ 0 & \sqrt 6 & 0 \end{array}\right],\nonumber \] and we use \(A\), \(V^T\), and \(\Sigma\) to find \(U\).
Since \(V\) is orthogonal and \(A=U\Sigma V^T\), it follows that \(AV=U\Sigma\). Let \(V=\left[\begin{array}{ccc} \vec{v}_1 & \vec{v}_2 & \vec{v}_3 \end{array}\right]\), and let \(U=\left[\begin{array}{cc} \vec{u}_1 & \vec{u}_2 \end{array}\right]\), where \(\vec{u}_1\) and \(\vec{u}_2\) are the two columns of \(U\).
Then we have \[\begin{aligned} A\left[\begin{array}{ccc} \vec{v}_1 & \vec{v}_2 & \vec{v}_3 \end{array}\right] &= \left[\begin{array}{cc} \vec{u}_1 & \vec{u}_2 \end{array}\right]\Sigma\\ \left[\begin{array}{ccc} A\vec{v}_1 & A\vec{v}_2 & A\vec{v}_3 \end{array}\right] &= \left[\begin{array}{ccc} \sigma_1\vec{u}_1 + 0\vec{u}_2 & 0\vec{u}_1 + \sigma_2 \vec{u}_2 & 0 \vec{u}_1 + 0 \vec{u}_2 \end{array}\right] \\ &= \left[\begin{array}{ccc} \sigma_1\vec{u}_1 & \sigma_2 \vec{u}_2 & 0 \end{array}\right]\end{aligned}\] which implies that \(A\vec{v}_1=\sigma_1\vec{u}_1 = 4\vec{u}_1\) and \(A\vec{v}_2=\sigma_2\vec{u}_2 = \sqrt 6 \vec{u}_2\).
Thus, \[\vec{u}_1 = \frac{1}{4}A\vec{v}_1 = \frac{1}{4} \left[\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right] \frac{1}{\sqrt{2}}\left[\begin{array}{r} 1\\ 0\\ 1 \end{array}\right] = \frac{1}{4\sqrt 2}\left[\begin{array}{r} 4\\ 4 \end{array}\right] = \frac{1}{\sqrt 2}\left[\begin{array}{r} 1\\ 1 \end{array}\right],\nonumber \] and \[\vec{u}_2 = \frac{1}{\sqrt 6}A\vec{v}_2 = \frac{1}{\sqrt 6} \left[\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right] \frac{1}{\sqrt{3}}\left[\begin{array}{r} -1\\ -1\\ 1 \end{array}\right] =\frac{1}{3\sqrt 2}\left[\begin{array}{r} 3\\ -3 \end{array}\right] =\frac{1}{\sqrt 2}\left[\begin{array}{r} 1\\ -1 \end{array}\right].\nonumber \] Therefore, \[U=\frac{1}{\sqrt{2}}\left[\begin{array}{rr} 1 & 1 \\ 1 & -1 \end{array}\right],\nonumber \] and \[\begin{aligned} A & = \left[\begin{array}{rrr} 1 & -1 & 3 \\ 3 & 1 & 1 \end{array}\right]\\ & = \left(\frac{1}{\sqrt{2}}\left[\begin{array}{rr} 1 & 1 \\ 1 & -1 \end{array}\right]\right) \left[\begin{array}{rrr} 4 & 0 & 0 \\ 0 & \sqrt 6 & 0 \end{array}\right] \left(\frac{1}{\sqrt{6}}\left[\begin{array}{rrr} \sqrt 3 & 0 & \sqrt 3 \\ -\sqrt 2 & -\sqrt 2 & \sqrt2 \\ -1 & 2 & 1 \end{array}\right]\right).\end{aligned}\]
Here is another example.
Find an SVD for \(A=\left[\begin{array}{r} -1 \\ 2\\ 2 \end{array}\right]\).
Solution
Since \(A\) is \(3\times 1\), \(A^T A\) is a \(1\times 1\) matrix whose eigenvalues are easier to find than the eigenvalues of the \(3\times 3\) matrix \(AA^T\).
\[A^TA=\left[\begin{array}{ccc} -1 & 2 & 2 \end{array}\right] \left[\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right] =\left[\begin{array}{r} 9 \end{array}\right].\nonumber \]
Thus \(A^TA\) has eigenvalue \(\lambda_1=9\), and the eigenvalues of \(AA^T\) are \(\lambda_1=9\), \(\lambda_2=0\), and \(\lambda_3=0\). Furthermore, \(A\) has only one singular value, \(\sigma_1=3\).
To find the matrix \(V\): To do so we find an eigenvector for \(A^TA\) and normalize it. In this case, finding a unit eigenvector is trivial: \(\vec{v}_1=\left[\begin{array}{r} 1 \end{array}\right]\), and \[V=\left[\begin{array}{r} 1 \end{array}\right].\nonumber \]
Also, \(\Sigma =\left[\begin{array}{r} 3 \\ 0\\ 0 \end{array}\right]\), and we use \(A\), \(\vec{v}^T\), and \(\Sigma\) to find \(U\).
Now \(AV=U\Sigma\), with \(V=\left[\begin{array}{r} \vec{v}_1 \end{array}\right]\), and \(U=\left[\begin{array}{rrr} \vec{u}_1 & \vec{u}_2 & \vec{u}_3 \end{array}\right]\), where \(\vec{u}_1\), \(\vec{u}_2\), and \(\vec{u}_3\) are the columns of \(U\). Thus \[\begin{aligned} A\left[\begin{array}{r}\vec{v}_1 \end{array}\right] &= \left[\begin{array}{rrr} \vec{u}_1 & \vec{u}_2 & \vec{u}_3 \end{array}\right]\Sigma\\ \left[\begin{array}{r} A\vec{v}_1 \end{array}\right] &= \left[\begin{array}{r} \sigma_1 \vec{u}_1+0\vec{u}_2+0\vec{u}_3 \end{array}\right]\\ &= \left[\begin{array}{r} \sigma_1 \vec{u}_1 \end{array}\right]\end{aligned}\] This gives us \(A\vec{v}_1=\sigma_1 \vec{u}_1= 3\vec{u}_1\), so \[\vec{u}_1 = \frac{1}{3}AV_1 = \frac{1}{3} \left[\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right] \left[\begin{array}{r} 1 \end{array}\right] = \frac{1}{3} \left[\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right].\nonumber \]
The vectors \(\vec{u}_2\) and \(\vec{u}_3\) are eigenvectors of \(AA^T\) corresponding to the eigenvalue \(\lambda_2=\lambda_3=0\). Instead of solving the system \((0I-AA^T)\vec{x}= 0\) and then using the Gram-Schmidt process on the resulting set of two basic eigenvectors, the following approach may be used.
Find vectors \(\vec{u}_2\) and \(\vec{u}_3\) by first extending \(\{ \vec{u}_1\}\) to a basis of \(\mathbb{R}^3\), then using the Gram-Schmidt algorithm to orthogonalize the basis, and finally normalizing the vectors.
Starting with \(\{ 3\vec{u}_1 \}\) instead of \(\{ \vec{u}_1 \}\) makes the arithmetic a bit easier. It is easy to verify that \[\left\{ \left[\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right], \left[\begin{array}{r} 1 \\ 0 \\ 0 \end{array}\right], \left[\begin{array}{r} 0 \\ 1 \\ 0 \end{array}\right]\right\}\nonumber \] is a basis of \(\mathbb{R}^3\). Set \[\vec{e}_1 = \left[\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right], \vec{x}_2 = \left[\begin{array}{r} 1 \\ 0 \\ 0 \end{array}\right], \vec{x}_3 =\left[\begin{array}{r} 0 \\ 1 \\ 0 \end{array}\right],\nonumber \] and apply the Gram-Schmidt algorithm to \(\{ \vec{e}_1, \vec{x}_2, \vec{x}_3\}\).
This gives us \[\vec{e}_2 = \left[\begin{array}{r} 4 \\ 1 \\ 1 \end{array}\right] \mbox{ and } \vec{e}_3 = \left[\begin{array}{r} 0 \\ 1 \\ -1 \end{array}\right].\nonumber \]
Therefore, \[\vec{u}_2 = \frac{1}{\sqrt{18}} \left[\begin{array}{r} 4 \\ 1 \\ 1 \end{array}\right], \vec{u}_3 = \frac{1}{\sqrt 2} \left[\begin{array}{r} 0 \\ 1 \\ -1 \end{array}\right],\nonumber \] and \[U = \left[\begin{array}{rrr} -\frac{1}{3} & \frac{4}{\sqrt{18}} & 0 \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & \frac{1}{\sqrt 2} \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & -\frac{1}{\sqrt 2} \end{array}\right].\nonumber \]
Finally, \[A = \left[\begin{array}{r} -1 \\ 2 \\ 2 \end{array}\right] = \left[\begin{array}{rrr} -\frac{1}{3} & \frac{4}{\sqrt{18}} & 0 \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & \frac{1}{\sqrt 2} \\ \frac{2}{3} & \frac{1}{\sqrt{18}} & -\frac{1}{\sqrt 2} \end{array}\right] \left[\begin{array}{r} 3 \\ 0 \\ 0 \end{array}\right] \left[\begin{array}{r} 1 \end{array}\right].\nonumber \]
Consider another example.
Find a singular value decomposition for the matrix \[A= \left[ \begin{array}{ccc} \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \end{array} \right]\nonumber \]
Solution
First consider \(A^TA\) \[\left[ \begin{array}{ccc} \frac{16}{5} & \frac{32}{5} & 0 \\ \frac{32}{5} & \frac{64}{5} & 0 \\ 0 & 0 & 0 \end{array} \right]\nonumber \] What are some eigenvalues and eigenvectors? Some computing shows these are \[\left\{ \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right] ,\left[ \begin{array}{c} -\frac{2}{5}\sqrt{5} \\ \frac{1}{5}\sqrt{5} \\ 0 \end{array} \right] \right\} \leftrightarrow 0,\left\{ \left[ \begin{array}{c} \frac{1}{5}\sqrt{5} \\ \frac{2}{5}\sqrt{5} \\ 0 \end{array} \right] \right\} \leftrightarrow 16\nonumber \] Thus the matrix \(V\) is given by \[V=\left[ \begin{array}{ccc} \frac{1}{5}\sqrt{5} & -\frac{2}{5}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{5} & \frac{1}{5}\sqrt{5} & 0 \\ 0 & 0 & 1 \end{array} \right]\nonumber \] Next consider \(AA^T\) \[\left[ \begin{array}{cc} 8 & 8 \\ 8 & 8 \end{array} \right]\nonumber \] Eigenvectors and eigenvalues are \[\left\{ \left[ \begin{array}{c} -\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \end{array} \right] \right\} \leftrightarrow 0,\left\{ \left[ \begin{array}{c} \frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} \end{array} \right] \right\} \leftrightarrow 16\nonumber \] Thus you can let \(U\) be given by \[U=\left[ \begin{array}{cc} \frac{1}{2}\sqrt{2} & -\frac{1}{2}\sqrt{2} \\ \frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \end{array} \right]\nonumber \] Lets check this. \(U^TAV=\) \[\left[ \begin{array}{cc} \frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \\ -\frac{1}{2}\sqrt{2} & \frac{1}{2}\sqrt{2} \end{array} \right] \left[ \begin{array}{ccc} \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{2}\sqrt{5} & \frac{4}{5}\sqrt{2}\sqrt{5} & 0 \end{array} \right] \left[ \begin{array}{ccc} \frac{1}{5}\sqrt{5} & -\frac{2}{5}\sqrt{5} & 0 \\ \frac{2}{5}\sqrt{5} & \frac{1}{5}\sqrt{5} & 0 \\ 0 & 0 & 1 \end{array} \right]\nonumber \] \[=\left[ \begin{array}{ccc} 4 & 0 & 0 \\ 0 & 0 & 0 \end{array} \right]\nonumber \]
This page titled 6.8: Singular Value Decomposition was last modified on Wed, 27 Aug 2025 04:36:06 GMT and is shared under a CC BY 4.0 license and was authored, remixed, and/or curated by Doli Bambhania, Fatemeh Yarahmadi, and Bill Wilson.
The book could not be loaded just now.