Eigenvalues and eigenvectors, a.k.a. characteristic values and characteristic vectors, reveal the true nature of a linear transformation. They also show up in surprising places in applications.
"Eigenvalues" and "eigenvectors" are fundamental concepts in linear algebra that have a wide range of direct applications.
They are only defined for square matrices.
Note that we allow complex eigenvalues even for real matrices.
Some authors also include $\mathbf{0}$ as an eigenvector. We do not follow that convention.
Eigenvalues are scalars; Eigenvectors are vectors, and eigenvectors are always associated with eigenvalues.
A $2 \times 2$ matrix represents a linear transformation of the plane (to itself).
For a square matrix $A$, a pair of eigenvalue and eigenvector $\lambda$ and $\mathbf{v}$ is defined by the equation \[ A \mathbf{v} = \lambda \mathbf{v}. \]
Notice that it is possible for a matrix to have more than one eigenvalue. Similarly, associated with a given eigenvalue, there may be more than one eigenvectors.
The equation \[ A \mathbf{v} = \lambda \mathbf{v} \] can be interpreted geometrically:
If $\lambda > 0$, under the map $\mathbf{v} \mapsto A \mathbf{v}$, the (very special) eigenvector is only scaled by a factor of $\lambda$, and its direction remain unchanged.
That is, $\mathbf{v}$ is stretched or squeezed, but not rotated.
The equation $A \mathbf{v} = \lambda \mathbf{v}$ and the "eigen-pairs" it defines can be generalized to vector spaces of functions.
The set of all smooth functions on an interval $(a,b)$ also form a vector space (although this is an infinite dimensional vector space).
Let's call this vector space of differentiable functions $V$. Then \[ \frac{d}{dx} \] is a linear function from $V$ to $V$.
We have a special class functions \[ \frac{d}{dx} e^{\lambda x} = \lambda \, e^{\lambda x} \]
This shows that the smooth function $f(x) = e^{\lambda x}$ is an eigenvector of the linear function \[ \frac{d}{dx} : V \to V \] associated with the eigenvalue $\lambda$.
The previous defines eigenvalues and eigenvectors in relation to one another. More precisely, we are defining "eigen-pairs".
Eigenvalues can also be defined directly by itself through this alternative but equivalent definition.
This definition is equivalent to the previous one, and we use them interchangeably.
In particular, if $A$ is singular, then 0 must be an eigenvalue of $A$.
Conversely, if $0$ is an eigenvalue of $A$, $A$ must be nonsingular.
From the results above, we can see that
For the special case of a $2 \times 2$ matrix \[ A = \begin{bmatrix} a & b \\ c & d \end{bmatrix} \] we can find the eigenvalues easily. (Note that this is just a special trick that only work for $2 \times 2$ cases and cannot be generalized)
This matrix has two eigenvalues $\lambda_1,\lambda_2$ (which may or may not be distinct) and they satisfy the equations \begin{align*} \lambda_1 \lambda_2 &= ad - bc \\ \lambda_1 + \lambda_2 &= a + d \end{align*}
You may recognize the values on the r.h.s.: They are the determinant and trace $A$, respectively.
That is, \begin{align*} \lambda_1 \lambda_2 &= \det A \\ \lambda_1 + \lambda_2 &= \operatorname{tr} A \end{align*}
For now, it is not yet clear why this is true. We will get to the reason soon.
The two eigenvalues $\lambda_1,\lambda_2$ of a $2 \times 2$ matrix $A$ satisfies \begin{align*} \lambda_1 \lambda_2 &= \det A \\ \lambda_1 + \lambda_2 &= \operatorname{tr} A \end{align*}
That is, they are the roots of the quadratic polynomial \[ \lambda^2 - (\operatorname{tr}) A \lambda + \det A. \] This polynomial is known as the characteristic polynomial of $A$.
It is possible for $\lambda_1 = \lambda_2$, i.e., the characteristic polynomial has a "double root". In this case, we consider the eigenvalue to be an eigenvalue of "multiplicity 2" (not to be confused with geometric multiplicity).
We do not require the eigenvalue to be real. Indeed, it is natural to consider non-real eigenvalues.
Consider the matrix \[ A = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}, \] which represents a $90^\circ$ counterclockwise rotation of the entire plane.
As such, the direction of a vector $\mathbf{v}$ cannot be preserved under the map $\mathbf{v} \mapsto A \mathbf{v}$.
The two eigenvalues $\lambda_1,\lambda_2$ satisfy the system of equations \begin{align*} \lambda_1 \lambda_2 &= 1 \\ \lambda_1 + \lambda_2 &= 0 \end{align*}
Therefore, the eigenvalues are \begin{align*} \lambda_1 &= +i \\ \lambda_1 &= -i \end{align*} where $i$ is the imaginary unit, i.e., $i^2 = -1$.
One observation from the problem above is that eigenvectors associated with an given eigenvalue of a matrix is never unique.
Indeed, if $\mathbf{v}$ is an eigenvector associated with an eigenvalue $\lambda$ of a matrix $A$, then so is $r \mathbf{v}$ for any nonzero scalar $r$ since \[ A (r \mathbf{v}) = r A \mathbf{v} = r \lambda \mathbf{v} = \lambda (r \mathbf{v}). \]
Similarly, if $\mathbf{v}_1,\mathbf{v}_2$ are eigenvectors associated with $\lambda$, then $\mathbf{v}_1 + \mathbf{v}_2$, if nonzero, is also an eigenvector associated with $\lambda$ since \[ A (\mathbf{v}_1 + \mathbf{v}_2) = A \mathbf{v}_1 + A \mathbf{v}_2 = \lambda \mathbf{v}_1 + \lambda \mathbf{v}_2 = \lambda (\mathbf{v}_1 + \mathbf{v}_2). \]
Therefore, together with $\mathbf{0}$, the set of all eigenvectors associated with an eigenvalue form a subspace.
It would be incorrect to say that $E_\lambda$ is the "set of eigenvectors" associated with $\lambda$ since we do not consider $\mathbf{0} \in E_\lambda$ as an eigenvector.
In standard textbook of linear algebra (not numerical linear algebra), the computation of eigenvalues generally involve determinants.
Indeed, $\lambda$ is an eigenvalue of $A$ if and only if \[ \det(\lambda I - A) = 0. \]
In other words, the eigenvalues of $A$ are exactly the roots of the polynomial \[ p_A (\lambda) = \det(\lambda I - A), \] which is called the characteristic polynomial of $A$.
The multiplicity of an eigenvalue $\lambda$ of $A$ as a root of $p_A$ is called the algebraic multiplicity of $\lambda$ (not to be confused with the geometric multiplicity).
However, this method is almost never used in practical applications for its inefficiency and numerical instability.