Matrices can be classified into a few general types based on a variety of properties. These names allow us to talk about matrices in a more complex way.
We can classify matrices into families sharing certain properties, and we will name each family.
These names allow us to talk about matrices and their applications more efficiently.
Depending on the property we focus on, we can classify matrices differently. We will discuss a few ways.
The most crude classification can be done just by looking at the sizes (dimensions) of the matrices.
We may want to further distinguish matrices with more rows than columns or more columns than rows, i.e. \begin{align*} & \begin{bmatrix} * & * \\ * & * \\ * & * \\ * & * \\ \end{bmatrix} &&\text{vs.} \begin{bmatrix} * & * & * \\ * & * & * \\ \end{bmatrix} \end{align*} but there are no good names for them.
We also say a matrix is sparse if it contains "many" zero entries.
The threshold (how many is "many") is inherently vague and potentially subjective, but it is nonetheless a very useful concept in computation. E.g., \[ \begin{bmatrix} 3 & 0 & 0 & 0 \\ 0 & 0 & 2 & 0 \\ 0 & 0 & 0 & 0 \\ \end{bmatrix} \] is a fairly sparse matrix.
Similarly, a matrix with "many" nonzero entries are dense.
The threshold is equally vague or subjective, yet it is just as useful. E.g., \[ \begin{bmatrix} 3 & 2 & 1 & 0 \\ 5 & 3 & 2 & 9 \\ 2 & 1 & 0 & 8 \\ \end{bmatrix} \] is quite dense.
The rank of a matrix is an important numerical quantity that convey some very useful information.
The following matrices \[ \begin{bmatrix} 1 & 2 & 2 \\ 2 & 4 & 4 \\ 2 & 4 & 4 \\ \end{bmatrix} , \quad \begin{bmatrix} 1 & 2 & 0 \\ 2 & 4 & 5 \\ 2 & 4 & 1 \\ \end{bmatrix} , \quad \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 0 \\ \end{bmatrix} \] are of rank 1, 2, and 3.
A zero matrix is a matrix consists of only zero entries. E.g. \[ \begin{bmatrix} 0 & 0 & 0 \\ 0 & 0 & 0 \end{bmatrix}. \]
In general, a zero matrix is (dimension usually clear from context) \[ \mathbf{0} = \begin{bmatrix} 0 & \cdots & 0 \\ \vdots & \ddots & \vdots \\ 0 & \cdots & 0 \end{bmatrix}. \]
Clearly, \[ \mathbf{0} + A = A + \mathbf{0} = A \] So it really behaves like the number "0" in the world of matrices.
Similarly, as long as dimensions are matched correctly, \[ \mathbf{0} A = \mathbf{0} \] (Note that the two zero matrices may not have the same dimension)
In the family of square matrices have the most detailed and interesting classifications. This is partly due to square matrices represents functions from $\mathbb{R}^n$ to $\mathbb{R}^n$ itself.
All the classifications in the following slides focus on square matrices.
As we have seen previously, the $n \times n$ identity matrix, denoted $I_n$, is the matrix \[ I_n = \begin{bmatrix} 1 & & \\ & \ddots & \\ & & 1 \end{bmatrix}. \] (missing entries are $0$'s)
That is, it has $1$'s on the main diagonal and $0$'s elsewhere.
Whenever the dimension is clear from the context, we simply use $I = I_n$, an it is always assumed to be square.
It has the very special property that \[ IA = A \] for any matrix $A$ as long as their dimensions match correctly.
A square matrix $A$ is said to be diagonal if all its nonzero entries are on the main diagonal (upper left corner to lower right corner). E.g., \[ \begin{bmatrix} 3 & 0 & 0 \\ 0 & 4 & 0 \\ 0 & 0 & 8 \end{bmatrix} \] is a $3 \times 3$ diagonal matrix.
It is okay for some diagonal entries to also be zero, e.g., \[ \begin{bmatrix} 3 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 5 \end{bmatrix} \] as we only require nonzero entries to be on the main diagonal.
In general, we can write an $n \times n$ diagonal matrix as \[ \begin{bmatrix} d_1 & & \\ & \ddots & \\ & & d_n \end{bmatrix} \]
A square matrix $A$ is said to be lower triangular if all its nonzero entries are on or below the main diagonal E.g., \[ \begin{bmatrix} 3 & 0 & 0 \\ 2 & 4 & 0 \\ 1 & 6 & 8 \\ \end{bmatrix} \] is a $3 \times 3$ lower triangular matrix.
Similarly, \[ \begin{bmatrix} 3 & 2 & 1 \\ 0 & 8 & 7 \\ 0 & 0 & 6 \\ \end{bmatrix} \] is an upper triangular matrix.
This is an important classification. See our previous discussion for detail.
That is, a symmetric matrix remain unchanged if we take the transpose.
Examples: \begin{align*} A &= \begin{bmatrix} 3 & 1 \\ 1 & 5 \end{bmatrix} & B &= \begin{bmatrix} 3 & 1 & 9 \\ 1 & 5 & 0 \\ 9 & 0 & 6 \end{bmatrix} & C &= \begin{bmatrix} 0 & 0 & 9 \\ 0 & 0 & 0 \\ 9 & 0 & 0 \end{bmatrix} \end{align*} are all symmetric matrices.
Of course, square zero matrices, identity matrices, or diagonal matrices in general are all naturally symmetric.
Starting from any matrix $A$ (not necessary square), we can create square symmetric matrices \begin{align*} & A^\top A &&\text{and} & &AA^\top \end{align*}
The transformation $A \mapsto A^\top A$ (or $A A^\top$) is a useful transformation used in many place. For example, we will see this in the construction of covariance matrix in statistics/data analysis or the normal equation in optimization.
There are also quite a few variations on the property of being "symmetric".
Basically, the columns (or rows) of an orthogonal matrix are unit vectors that are orthogonal to one another.
From the above equation, we can see that \[ Q^{-1} = Q^\top. \] That is, an orthogonal matrix is always invertible, and its transpose is also its inverse.
We can also describe this the point of view of basis: the column (or rows) of an orthogonal $n \times n$ matrix form an orthonormal basis for $\mathbb{R}^n$. This is usually how orthogonal matrices are created.
The more fitting name orthonormal matrix is almost never used (and I don't know why).