Opening the book…
- Understand the meaning of orthogonal diagonalization
- Determine if a matrix is orthogonally diagonalization
Orthogonal Diagonalization
We begin this section by recalling some important definitions. Recall that non-zero vectors are called orthogonal if their dot product equals \(0\). A set is orthonormal if it is orthogonal and each vector is a unit vector.
An orthogonal matrix \(U\) is one in which \(UU^{T} = I\). In other words, the transpose of an orthogonal matrix is equal to its inverse. A key characteristic of orthogonal matrices, which will be essential in this section, is that the columns of an orthogonal matrix form an orthonormal set.
We now recall another important definition.
A real \(n\times n\) matrix \(A,\) is symmetric if \(A^{T}=A.\) If \( A=-A^{T},\) then \(A\) is called skew symmetric.
Before proving an essential theorem, we first examine the following lemma which will be used below.
Let \(A=\left ( a_{ij} \right )\) be a real symmetric \(n \times n\) matrix, and let \(\vec{x}, \vec{y} \in \mathbb{R}^n\). Then \[A\vec{x} \cdot \vec{y} = \vec{x} \cdot A \vec{y}\nonumber\]
- Proof
-
This result follows from the definition of the dot product together with properties of matrix multiplication, as follows: \[\begin{aligned} A\vec{x} \cdot \vec{y} &= \sum_{k,l}a_{kl}x_{l}y_{k} \\ &=\sum_{k,l} (a_{lk})^Tx_{l}y_{k} \\ &= \vec{x}\cdot A^{T}\vec{y} \\ &= \vec{x}\cdot A \vec{y}\end{aligned}\]
The last step follows from \(A^T = A\), since \(A\) is symmetric.
We can now prove that the eigenvalues of a real symmetric matrix are real numbers. Consider the following important theorem.
Let \(A\) be a real symmetric matrix. Then the eigenvalues of \(A\) are real numbers and eigenvectors corresponding to distinct eigenvalues are orthogonal.
- Proof
-
Recall that for a complex number \(a+ib,\) the complex conjugate, denoted by \(\overline{a+ib}\) is given by \(\overline{a+ib}=a-ib.\) The notation, \(\overline{\vec{x}}\) will denote the vector which has every entry replaced by its complex conjugate.
Suppose \(A\) is a real symmetric matrix and \(A\vec{x}=\lambda \vec{x}\). Then \[{\overline{\lambda \vec{x}}}^{T}\vec{x}=\left( \overline{A \vec{x}}\right) ^{T}\vec{x}=\overline{\vec{x}}^{T}A^{T}\vec{x}= \overline{\vec{x}}^{T}A\vec{x}=\lambda \overline{\vec{x}}^{T} \vec{x}\nonumber \] Dividing by \(\overline{\vec{x}}^{T}\vec{x}\) on both sides yields \(\overline{\lambda }=\lambda\) which says \(\lambda\) is real. To do this, we need to ensure that \(\overline{\vec{x}}^{T}\vec{x} \neq 0\). Notice that \(\overline{\vec{x}}^{T}\vec{x} = 0\) if and only if \(\vec{x} = \vec{0}\). Since we chose \(\vec{x}\) such that \(A\vec{x} = \lambda \vec{x}\), \(\vec{x}\) is an eigenvector and therefore must be nonzero.
Now suppose \(A\) is real symmetric and \(A\vec{x}=\lambda \vec{x}\), \(A \vec{y}=\mu \vec{y}\) where \(\mu \neq \lambda\). Then since \(A\) is symmetric, it follows from Lemma \(\PageIndex{1}\) about the dot product that \[\lambda \vec{x}\cdot \vec{y}=A\vec{x}\cdot \vec{y}=\vec{x}\cdot A\vec{y}=\vec{x}\cdot \mu \vec{y}=\mu \vec{x}\cdot \vec{y}\nonumber \] Hence \(\left( \lambda -\mu \right) \vec{x}\cdot \vec{y}=0.\) It follows that, since \(\lambda -\mu \neq 0,\) it must be that \(\vec{x}\cdot \vec{y}=0\). Therefore the eigenvectors form an orthogonal set.
The following theorem is proved in a similar manner.
The eigenvalues of a real skew symmetric matrix are either equal to \(0\) or are pure imaginary numbers.
- Proof
-
First, note that if \(A=0\) is the zero matrix, then \(A\) is skew symmetric and has eigenvalues equal to \(0\).
Suppose \(A=-A^{T}\) so \(A\) is skew symmetric and \(A\vec{x}=\lambda \vec{x}\). Then \[\overline{\lambda \vec{x}}^{T}\vec{x}=\left( \overline{A \vec{x}}\right) ^{T}\vec{x}=\overline{\vec{x}}^{T}A^{T}\vec{x}=- \overline{\vec{x}}^{T}A\vec{x}=-\lambda \overline{\vec{x}}^{T} \vec{x}\nonumber \] and so, dividing by \(\overline{\vec{x}}^{T}\vec{x}\) as before, \(\overline{\lambda }=-\lambda .\) Letting \(\lambda =a+ib,\) this means \(a-ib=-a-ib\) and so \(a=0.\) Thus \(\lambda\) is pure imaginary.
Consider the following example.
Let \(A=\left[ \begin{array}{rr} 0 & -1 \\ 1 & 0 \end{array} \right] .\) Find its eigenvalues.
Solution
First notice that \(A\) is skew symmetric. By Theorem \(\PageIndex{2}\), the eigenvalues will either equal \(0\) or be pure imaginary. The eigenvalues of \(A\) are obtained by solving the usual equation \[\det (\lambda I - A ) = \det \left[ \begin{array}{rr} \lambda & 1 \\ -1 & \lambda \end{array} \right] =\lambda ^{2}+1=0\nonumber \]
Hence the eigenvalues are \(\pm i,\) pure imaginary.
Consider the following example.
Let \(A=\left[ \begin{array}{rr} 1 & 2 \\ 2 & 3 \end{array} \right] .\) Find its eigenvalues.
Solution
First, notice that \(A\) is symmetric. By Theorem \(\PageIndex{1}\), the eigenvalues will all be real. The eigenvalues of \(A\) are obtained by solving the usual equation \[\det (\lambda I - A) = \det \left[ \begin{array}{rr} \lambda - 1 & -2 \\ -2 & \lambda - 3 \end{array} \right] = \lambda^2 -4\lambda -1=0\nonumber \] The eigenvalues are given by \(\lambda_1 =2+ \sqrt{5}\) and \(\lambda_2 =2-\sqrt{5}\) which are both real.
Recall that a diagonal matrix \(D=\left ( d_{ij} \right )\) is one in which \(d_{ij} = 0\) whenever \(i \neq j\). In other words, all numbers not on the main diagonal are equal to zero.
If a matrix \(A\) is symmetric, then we can actually find a matrix \(U\) that is an orthogonal matrix of eigenvectors. Thus \(U^{-1}\) is simply its transpose \(U^T\), and \(U^TAU\) is diagonal. When this happens we say that \(A\) is orthogonally diagonalizable.
Let \(A\) be an \(n\times n\) matrix. Then \(A\) is said to be orthogonally diagonalizable if there exists an orthogonal matrix \(U\) such that \[U^{-1}AU=D\nonumber \] where \(D\) is a diagonal matrix. Note that since \(U\) is an orthogonal matrix, \(U^{-1}=U^T\), so we have \[U^T AU=D\nonumber \]
Consider the following important theorem.
Let \(A\) be a real symmetric matrix. Then there exists an orthogonal matrix \(U\) such that \[U^{T}AU = D\nonumber \] where \(D\) is a diagonal matrix. Moreover, the diagonal entries of \(D\) are the eigenvalues of \(A\).
We can use this theorem to diagonalize a symmetric matrix, using orthogonal matrices. Consider the following corollary.
If \(A\) is a real \(n\times n\) symmetric matrix, then there exists an orthonormal set of eigenvectors, \(\left\{ \vec{u}_{1},\cdots ,\vec{u} _{n}\right\} .\)
- Proof
-
Since \(A\) is symmetric, then by Theorem \(\PageIndex{3}\), there exists an orthogonal matrix \(U\) such that \(U^{T}AU=D,\) a diagonal matrix whose diagonal entries are the eigenvalues of \(A.\) Therefore, since \(A\) is symmetric and all the matrices are real, \[\overline{D}=\overline{D^{T}}=\overline{U^{T}A^{T}U}=U^{T}A^{T}U=U^{T}AU=D\nonumber \] showing \(D\) is real because each entry of \(D\) equals its complex conjugate.
Now let \[U=\left[ \begin{array}{cccc} \vec{u}_{1} & \vec{u}_{2} & \cdots & \vec{u}_{n} \end{array} \right]\nonumber \] where the \(\vec{u}_{i}\) denote the columns of \(U\) and \[D=\left[ \begin{array}{ccc} \lambda _{1} & & 0 \\ & \ddots & \\ 0 & & \lambda _{n} \end{array} \right]\nonumber \] The equation, \(U^{T}AU=D\) implies \(AU = UD\) and \[\begin{aligned} AU &=\left[ \begin{array}{cccc} A\vec{u}_{1} & A\vec{u}_{2} & \cdots & A\vec{u}_{n} \end{array} \right] \\ &=\left[ \begin{array}{cccc} \lambda _{1}\vec{u}_{1} & \lambda _{2}\vec{u}_{2} & \cdots & \lambda _{n}\vec{u}_{n} \end{array} \right] \\ &= UD\end{aligned}\] where the entries denote the columns of \(AU\) and \(UD\) respectively. Therefore, \(A\vec{u}_{i}=\lambda _{i}\vec{u}_{i}\). Since the matrix \(U\) is orthogonal, the \(ij^{th}\) entry of \(U^{T}U\) equals \(\delta _{ij}\) and so \[\delta _{ij}=\vec{u}_{i}^{T}\vec{u}_{j}=\vec{u}_{i}\cdot \vec{u} _{j}\nonumber \] This proves the corollary because it shows the vectors \(\left\{ \vec{u} _{i}\right\}\) form an orthonormal set.
Let \(A\) be an \(n \times n\) matrix. Then the principal axes of \(A\) is a set of orthonormal eigenvectors of \(A\).
In the next example, we examine how to find such a set of orthonormal eigenvectors.
Find an orthonormal set of eigenvectors for the symmetric matrix \[A = \left[ \begin{array}{rrr} 17 & -2 & -2 \\ -2 & 6 & 4 \\ -2 & 4 & 6 \end{array} \right]\nonumber \]
Solution
Recall Procedure 6.1.1 for finding the eigenvalues and eigenvectors of a matrix. You can verify that the eigenvalues are \(18,9,2.\) First find the eigenvector for \(18\) by solving the equation \(\left( 18 I-A\right)\vec{x} = \vec{0}\). The appropriate augmented matrix is given by \[\left[ \begin{array}{ccc|c} 18-17 & 2 & 2 & 0 \\ 2 & 18-6 & -4 & 0 \\ 2 & -4 & 18-6 & 0 \end{array} \right]\nonumber \] The reduced row-echelon form is \[\left[ \begin{array}{rrr|r} 1 & 0 & 4 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] Therefore an eigenvector is \[\left[ \begin{array}{r} -4 \\ 1 \\ 1 \end{array} \right]\nonumber \] Next find the eigenvector for \(\lambda =9.\) The augmented matrix and resulting reduced row-echelon form are \[\left[ \begin{array}{ccc|c} 9-17 & 2 & 2 & 0 \\ 2 & 9-6 & -4 & 0 \\ 2 & -4 & 9-6 & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 1 & 0 & - \frac{1}{2} & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] Thus an eigenvector for \(\lambda =9\) is \[\left[ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right]\nonumber \] Finally find an eigenvector for \(\lambda =2.\) The appropriate augmented matrix and reduced row-echelon form are \[\left[ \begin{array}{ccc|c} 2-17 & 2 & 2 & 0 \\ 2 & 2-6 & -4 & 0 \\ 2 & -4 & 2-6 & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 1 & 0 & 0 & 0 \\ 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] Thus an eigenvector for \(\lambda =2\) is \[\left[ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right]\nonumber \]
The set of eigenvectors for \(A\) is given by \[\left\{ \left[ \begin{array}{r} -4 \\ 1 \\ 1 \end{array} \right], \left[ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right], \left[ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right] \right\}\nonumber \] You can verify that these eigenvectors form an orthogonal set. By dividing each eigenvector by its magnitude, we obtain an orthonormal set: \[\left\{ \frac{1}{\sqrt{18}}\left[ \begin{array}{r} -4 \\ 1 \\ 1 \end{array} \right] ,\frac{1}{3}\left[ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right] ,\frac{1}{\sqrt{2}}\left[ \begin{array}{r} 0 \\ -1 \\ 1 \end{array} \right] \right\}\nonumber \]
Consider the following example.
Find an orthonormal set of three eigenvectors for the matrix \[A = \left[ \begin{array}{rrr} 10 & 2 & 2 \\ 2 & 13 & 4 \\ 2 & 4 & 13 \end{array} \right]\nonumber \]
Solution
You can verify that the eigenvalues of \(A\) are \(9\) (with multiplicity two) and \(18\) (with multiplicity one). Consider the eigenvectors corresponding to \(\lambda =9\). The appropriate augmented matrix and reduced row-echelon form are given by \[\left[ \begin{array}{ccc|c} 9-10 & -2 & -2 & 0 \\ -2 & 9-13 & -4 & 0 \\ -2 & -4 & 9-13 & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 1 & 2 & 2 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] and so eigenvectors are of the form \[\left[ \begin{array}{c} -2y-2z \\ y \\ z \end{array} \right]\nonumber \] We need to find two of these which are orthogonal. Let one be given by setting \(z=0\) and \(y=1\), giving \(\left[ \begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right]\).
In order to find an eigenvector orthogonal to this one, we need to satisfy \[\left[ \begin{array}{r} -2 \\ 1 \\ 0 \end{array} \right] \cdot \left[ \begin{array}{c} -2y-2z \\ y \\ z \end{array} \right] =5y+4z=0\nonumber \] The values \(y=-4\) and \(z=5\) satisfy this equation, giving another eigenvector corresponding to \(\lambda=9\) as \[\left[ \begin{array}{c} -2\left( -4\right) -2\left( 5\right) \\ \left( -4\right) \\ 5 \end{array} \right] =\left[ \begin{array}{r} -2 \\ -4 \\ 5 \end{array} \right]\nonumber \] Next find the eigenvector for \(\lambda =18.\) The augmented matrix and the resulting reduced row-echelon form are given by \[\left[ \begin{array}{ccc|c} 18-10 & -2 & -2 & 0 \\ -2 & 18-13 & -4 & 0 \\ -2 & -4 & 18-13 & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 1 & 0 & - \frac{1}{2} & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] and so an eigenvector is \[\left[ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right]\nonumber \]
Dividing each eigenvector by its length, the orthonormal set is \[\left\{ \frac{1}{\sqrt{5}} \left[ \begin{array}{r} -2\\ 1 \\ 0 \end{array} \right] , \frac{\sqrt{5}}{15} \left[ \begin{array}{r} -2 \\ -4 \\ 5 \end{array} \right] , \frac{1}{3}\left[ \begin{array}{r} 1 \\ 2 \\ 2 \end{array} \right] \right\}\nonumber \]
In the above solution, the repeated eigenvalue implies that there would have been many other orthonormal bases which could have been obtained. While we chose to take \(z=0, y=1\), we could just as easily have taken \(y=0\) or even \(y=z=1.\) Any such change would have resulted in a different orthonormal set.
Recall that an \(n\times n\) matrix \(A\) is said to be non defective or diagonalizable if there exists an invertible matrix \(P\) such that \(P^{-1}AP=D\) where \(D\) is a diagonal matrix.
As indicated in Theorem \(\PageIndex{3}\) if \(A\) is a real symmetric matrix, there exists an orthogonal matrix \(U\) such that \(U^{T}AU=D\) where \(D\) is a diagonal matrix. Therefore, every symmetric matrix is diagonalizable because if \(U\) is an orthogonal matrix, it is invertible and its inverse is \(U^{T}\). Therefore every symmetric matrix is in fact orthogonally diagonalizable. The next theorem provides another way to determine if a matrix is orthogonally diagonalizable.
Let \(A\) be an \(n \times n\) matrix. Then \(A\) is orthogonally diagonalizable if and only if \(A\) has an orthonormal set of eigenvectors.
Recall from Corollary \(\PageIndex{1}\) that every symmetric matrix has an orthonormal set of eigenvectors. In fact the following theorem shows these three conditions are equivalent.
The following conditions are equivalent for an \(n \times n\) matrix \(A\):
- \(A\) is symmetric.
- \(A\) has an orthonormal set of eigenvectors.
- \(A\) is orthogonally diagonalizable.
- Proof
-
The complete proof is beyond this course, but to give an idea assume that \(A\) has an orthonormal set of eigenvectors, and let \(P\) consist of these eigenvectors as columns. Then \(P^{-1}=P^T\), and \(P^TAP=D\) a diagonal matrix. But then \(A=PDP^T\), and \[A^T=(PDP^T)^T = (P^T)^TD^TP^T=PDP^T=A\nonumber\] so \(A\) is symmetric.
Now given a symmetric matrix \(A\), one shows that eigenvectors corresponding to different eigenvalues are always orthogonal. So it suffices to apply the Gram-Schmidt process on the set of basic eigenvectors of each eigenvalue to obtain an orthonormal set of eigenvectors.
We demonstrate this in the following example.
In the following example, the orthogonal matrix \(U\) will be found to orthogonally diagonalize a matrix.
Let \(A=\left[ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right] .\) Find an orthogonal matrix \(U\) such that \(U^{T}AU\) is a diagonal matrix.
Solution
In this case, the eigenvalues are \(2\) (with multiplicity one) and \(1\) (with multiplicity two). First we will find an eigenvector for the eigenvalue \(2\). The appropriate augmented matrix and resulting reduced row-echelon form are given by \[\left[ \begin{array}{ccc|c} 2-1 & 0 & 0 & 0 \\ 0 & 2- \frac{3}{2} & - \frac{1}{2} & 0 \\ 0 & - \frac{1}{2} & 2- \frac{3}{2} & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 1 & 0 & 0 & 0 \\ 0 & 1 & -1 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] and so an eigenvector is \[\left[ \begin{array}{r} 0 \\ 1 \\ 1 \end{array} \right]\nonumber \] However, it is desired that the eigenvectors be unit vectors and so dividing this vector by its length gives \[\left[ \begin{array}{c} 0 \\ \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array} \right]\nonumber \] Next find the eigenvectors corresponding to the eigenvalue equal to \(1\). The appropriate augmented matrix and resulting reduced row-echelon form are given by: \[\left[ \begin{array}{ccc|c} 1-1 & 0 & 0 & 0 \\ 0 & 1- \frac{3}{2} & - \frac{1}{2} & 0 \\ 0 & - \frac{1}{2} & 1- \frac{3}{2} & 0 \end{array} \right] \rightarrow \cdots \rightarrow \left[ \begin{array}{rrr|r} 0 & 1 & 1 & 0 \\ 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 \end{array} \right]\nonumber \] Therefore, the eigenvectors are of the form \[\left[ \begin{array}{r} s \\ -t \\ t \end{array} \right]\nonumber \] Two of these which are orthonormal are \(\left[ \begin{array}{c} 1 \\ 0 \\ 0 \end{array} \right]\), choosing \(s=1\) and \(t=0\), and \(\left[ \begin{array}{c} 0 \\ - \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} \end{array} \right]\), letting \(s=0\), \(t= 1\) and normalizing the resulting vector.
To obtain the desired orthogonal matrix, we let the orthonormal eigenvectors computed above be the columns. \[\left[ \begin{array}{rrr} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right]\nonumber \]
To verify, compute \(U^{T}AU\) as follows: \[U^{T}AU = \left[ \begin{array}{rrr} 0 & - \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \\ 1 & 0 & 0 \\ 0 & \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} \end{array} \right] \left[ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & \frac{3}{2} & \frac{1}{2} \\ 0 & \frac{1}{2} & \frac{3}{2} \end{array} \right] \left[ \begin{array}{rrr} 0 & 1 & 0 \\ -\frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \\ \frac{1}{\sqrt{2}} & 0 & \frac{1}{\sqrt{2}} \end{array} \right]\nonumber \] \[=\left[ \begin{array}{rrr} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{array} \right] = D\nonumber \] the desired diagonal matrix. Notice that the eigenvectors, which construct the columns of \(U\), are in the same order as the eigenvalues in \(D\).
We conclude this section with a Theorem that generalizes earlier results.
Let \(A\) be an \(n \times n\) matrix. If \(A\) has \(n\) real eigenvalues, then an orthogonal matrix \(U\) can be found to result in the upper triangular matrix \(U^T A U\).
This Theorem provides a useful Corollary.
Let \(A\) be an \(n \times n\) matrix with eigenvalues \(\lambda_1, \cdots, \lambda_n\). Then it follows that \(\det(A)\) is equal to the product of the \(\lambda_i\), while \(trace(A)\) is equal to the sum of the \(\lambda_i\).
- Proof
-
By Theorem \(\PageIndex{5}\), there exists an orthogonal matrix \(U\) such that \(U^TAU=P\), where \(P\) is an upper triangular matrix. Since \(P\) is similar to \(A\), the eigenvalues of \(P\) are \(\lambda_1, \lambda_2, \ldots, \lambda_n\). Furthermore, since \(P\) is (upper) triangular, the entries on the main diagonal of \(P\) are its eigenvalues, so \(\det(P)=\lambda_1 \lambda_2 \cdots \lambda_n\) and \(trace(P)=\lambda_1 + \lambda_2 + \cdots + \lambda_n\). Since \(P\) and \(A\) are similar, \(\det(A)=\det(P)\) and \(trace(A)=trace(P)\), and therefore the results follow.
This page titled 6.7: Orthogonal Diagonalization was last modified on Thu, 10 Apr 2025 05:35:40 GMT and is shared under a CC BY 4.0 license and was authored, remixed, and/or curated by Doli Bambhania, Fatemeh Yarahmadi, and Bill Wilson.
The book could not be loaded just now.