What is the significance of eigenvalues and eigenvectors?

Often, when applying matrix algebra in mathematics, there is mention of eigenvalues and eigenvectors of matrices. In many cases, only a definition of eigenvalues and eigenvectors of a matrix is referenced. This definition is as follows:

Given a square matrix $A$ , its eigenvalues are scalars $\lambda$ such that there are non-zero vectors $\vec{\zeta}$ satisfying $A \vec{\zeta} = \lambda\vec{\zeta}$ . The non-zero vectors $\vec{\zeta}$ satisfying the above are called the eigenvectors of matrix $A$ corresponding to the eigenvalue $\lambda$ .

Citing just a reference to the above definition, the technique to find the eigenvalues and eigenvectors is then employed.

A reader of such material may be left with the question as to why such values and vectors have any significance. Why should one care about values and vectors satisfying the above definition?

To answer this, it is important to first note that matrices represent linear maps from one vector space to another. In fact, the main goal of Linear algebra subject is just to study linear maps.

Eigenvalues and eigenvectors are defined for linear maps just as they are defined for matrices. In fact, the above definition of eigenvalues and eigenvectors for matrices exists only because of a similar definition for linear maps:

Given a linear map $t:V\to V$ , eigenvalues are scalars $\lambda$ such that there are non-zero vectors $\vec{\zeta}\in V$ satisfying $t(\vec{\zeta}) = \lambda\vec{\zeta}$ . And, the non-zero vectors $\vec{\zeta}$ satisfying the above are called the eigenvectors of linear map $t$ corresponding to the eigenvalue $\lambda$ .

Let us understand the definition for linear maps first before going to the definition for matrices.

Firstly, note that we are analyzing the linear maps where the domain and codomain are the same vector space $V$ . Let us say the vector space $V$ is of dimension $n$ .

For a linear map $t$ , if $\vec{\zeta}$ gets mapped to $\lambda\vec{\zeta}$ , then that means that under the map $t$ the vector does changes merely magnitude, not direction.

Thus, eigenvectors of a linear map $t$ are significant for two reasons:

these are vectors which just change in magnitude, but not in direction under the map $t$ . Such vectors are important in many physical problems, where “direction” needs to be preserved.
Say, we find $n$ linearly independent eigenvectors. Since $V$ is $n$ -dimensional, the set of $n$ linearly independent eigenvectors form a basis for the vector space $V$ . Then, we have found a basis such that each element vector of the basis changes only magnitude (albeit by a different factor (eigenvalue) for each vector of the basis). This then allows us to describe the map in simple terms with respect to that basis. For example, let us consider a map $t:\mathbb{R}^2\to\mathbb{R}^2$ , with $t(2,1) = (4, 2)$ and $t(1,2) = (3,6)$ . Thus, $\vec{\zeta_1}=(2,1)$ and $\vec{\zeta_2}=(1,2)$ change magnitude by factors of $2$ and $3$ respectively and do not change direction, and hence are eigenvectors of $t$ with eigenvalues $\lambda_1=2$ and $\lambda_2=3$ respectively. Now, given any vector $\vec{v} = a\vec{\zeta_1}+b\vec{\zeta_2}\in\mathbb{R}^2$ , we can immediately say $t(\vec{v}) = a t(\vec{\zeta_1}) + b t(\vec{\zeta_2}) = a\lambda_1\vec{\zeta_1}+b\lambda_2\vec{\zeta_2}$ .

We now go to the topic of eigenvalues and eigenvectors for matrices. As we mentioned, matrices represent linear maps. This goes both ways. That is, given a basis and a square matrix, a linear map can be found that is represented by the matrix with respect to the basis. Conversely, given a linear map and any choice of basis, the map is represented by a square matrix with respect to the basis. Note that, when discussing eigenvalues and eigenvectors, we consider linear maps where domain and codomain are the same vector space, and hence the matrices representing them are square.

Note that for a given linear map, as the choice of basis changes, the matrix used to represent the linear map with respect to the basis also changes. Thus, multiple matrices can represent the same linear map (with respect to different bases). Square matrices that represent the same map (with respect to possibly different bases) are called similar matrices. Naturally, we like to find a matrix as simple as possible (with as many zeros as possible) to represent the linear map. More precisely, we like to find a diagonal matrix – that is, we want to find a basis with respect to which the matrix representing a given linear map is diagonal. This is where eigenvalues and eigenvectors come into picture.

Say, for a linear map $t:V\to V$ where $V$ is an $n$ -dimensional vector space, there are $n$ linearly independent eigenvectors $\vec{\zeta_1},\vec{\zeta_2},\ldots,\vec{\zeta_n}$ with corresponding eigenvalues $\lambda_1, \lambda_2,\ldots,\lambda_n$ (Note that, all eigenvalues are not necessarily distinct. That is, there could be more than one linearly independent eigenvector corresponding to one eigenvalue), then the following diagonal matrix represents $t$ with respect to the basis $Z = \langle \vec{\zeta_1},\vec{\zeta_2},\ldots,\vec{\zeta_n}\rangle$ :

Now, by representing a linear map by a matrix with respect to a given basis, we know that multiplying the matrix and the “tall vector” of coefficients (with respect to the basis) of an “input” vector gives the “tall vector” of coefficients (with respect to the basis) of the map’s “output” vector. Thus, if $A$ represents a linear map $t: V\to V$ with respect to a basis $B$ , $\langle c_1,c_2,\ldots, c_n\rangle$ are the coefficients of an “input” vector $\vec{v}\in V$ (with respect to the same basis $B$ ), then

$A\begin{bmatrix}c_1\\c_2\\c_3\\\vdots\\c_n\end{bmatrix}$

gives the “tall vector” of coefficients of the map’s “output” vector (with respect to the same basis $B$ ).

Now, by finding eigenvalues of a linear map and representing the linear map by a diagonal matrix, we have an advantage, since multiplication by a diagonal matrix is straightforward. For example,

$\begin{bmatrix} \lambda_1 & 0 & 0 & \cdots & 0 \\ 0 & \lambda_2 & 0 & \cdots & 0 \\ 0 & 0 & \lambda_3 & \cdots & 0 \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & 0 & \cdots & \lambda_n \end{bmatrix}\begin{bmatrix}c_1\\c_2\\c_3\\\vdots\\c_n\end{bmatrix} = \begin{bmatrix}\lambda_1 c_1\\\lambda_2 c_2\\\lambda_3 c_3\\\vdots\\\lambda_n c_n\end{bmatrix}$

Thus, for a linear map, if there are $n$ linearly independent eigenvectors $\vec{\zeta_1},\vec{\zeta_2},\ldots,\vec{\zeta_n}$ with corresponding eigenvalues $\lambda_1,\lambda_2,\ldots,\lambda_n$ (not necessarily all distinct) and $\langle c_1,c_2,\ldots, c_n\rangle$ represents the coefficients of an “input” vector with respect to the basis $Z = \langle \vec{\zeta_1},\vec{\zeta_2},\ldots,\vec{\zeta_n}\rangle$ , then $\langle \lambda_1 c_1,\lambda_2 c_2,\ldots,\lambda_n c_n\rangle$ represents the coefficients of the map’s “output” vector with respect to the same basis $Z$ .

Note that, while we talked above about representing maps by matrices, all we have done so far is reiterate the point 2 above for maps. We still have not talked about eigenvalues and eigenvectors of matrices themselves. Well, we are almost there.

Going back to our definition of eigenvalues and eigenvectors for matrices, say, we found an eigenvalue $\lambda$ of a matrix $A$ – that is, we have a non-zero vector $\vec{\zeta}$ such that $A\vec{\zeta} = \lambda\vec{\zeta}$ . Then, $\lambda$ is an eigenvalue also for any linear map that $A$ represents with respect to any basis. That is, say, matrix $A$ represents some map $t$ with respect to some basis $B$ , then $\lambda$ is an eigenvalue for the map $t$ . This is because if $A\vec{\zeta} = \lambda\vec{\zeta}$ , this implies the coefficients with respect to the chosen basis get multiplied by $\lambda$ under the map. If the coefficients of a vector get multiplied by $\lambda$ , the vector gets multiplied by $\lambda$ as well. (Recall, coefficients of a multiple of a vector are the multiples of the coefficients of the vector). Further, the coefficients being non-zero, the vector with these coefficients is also non-zero.

The preceding paragraph implies that similar matrices (the matrices that represent the same linear map) have same eigenvalues. Eigenvectors could be different for similar matrices because although we have the same underlying linear map, similar matrices use (possibly) different bases. By finding an eigenvector for a matrix, we just identified the coefficients of an eigenvector for the linear map represented by the matrix with respect to a specific basis. Since coefficients change from basis to basis, eigenvectors could be different for similar matrices.

In summary, matrices represent linear maps. In trying to find eigenvalues and eigenvectors of a matrix, we are actually trying to find the eigenvalues and eigenvectors of an underlying linear map. An eigenvalue of a matrix is also an eigenvalue of the linear map that it represents. On the other hand, an eigenvector of a matrix represents the coefficients of an eigenvector of the linear map with respect to the chosen basis.

Leave a Reply Cancel reply