The "rank" of a matrix measures the maximum number of linear independent rows or columns in the matrix, and it is an important numerical quantity associated with a matrix. Square matrices that have the maximum rank have something called "inverse".
Recall that matrices are simply rectangular arrays of numbers. E.g., \[ \begin{bmatrix} 1 & 2 \\ 3 & 5 \end{bmatrix} \]
Of course, things only become interesting when we define sum and product between matrices and interpret matrices as representations of linear functions between (finite dimensional) vector spaces.
For example, the matrix above represents the linear function \[ \begin{bmatrix} x \\ y \end{bmatrix} \quad\mapsto\quad \begin{bmatrix} 1 & 2 \\ 3 & 5 \end{bmatrix} \; \begin{bmatrix} x \\ y \end{bmatrix} = \begin{bmatrix} 1x + 2y \\ 3x + 5y \end{bmatrix}, \] which sends vectors in the plane to vectors in the plane.
Note that there may be more than one way of choosing a linearly independent subset of columns. However, regardless of our choice, the maximum number of columns in such a linearly independent subset of columns is always the same, and this number is the column rank.
For example, we can see that the columns in \[ \begin{bmatrix} 1 & 2 \\ 3 & 5 \end{bmatrix} \] i.e., \[ \begin{bmatrix} 1 \\ 3 \end{bmatrix} \;,\; \begin{bmatrix} 2 \\ 5 \end{bmatrix} \] form a linearly independent set.
That is, we can find a set of two linearly independent columns. Therefore, the column rank of this matrix is 2.
Using the same example again, \[ \begin{bmatrix} 1 & 2 \\ 3 & 5 \end{bmatrix} \] is of full column rank.
But, \[ \begin{bmatrix} 1 & 2 \\ 3 & 6 \end{bmatrix} \quad\text{and}\quad \begin{bmatrix} 1 & 2 & 0 \\ 3 & 5 & 1 \end{bmatrix} \] are not of full column rank.
It is particularly important to understand a $m \times n$ matrix with $m < n$ can never be of full column rank. E.g., a $3 \times 5$ matrix can never be of full column rank. Can you see why?
Similarly, we can also derive a "rank" from the number of linearly independent rows.
It turns out, despite the rather different definitions, the column rank and row rank of a matrix are always the same.
One consequence is that $A$ and $A^\top$ have the same column/row ranks.
Therefore, we simply call this number the rank of the matrix $A$ and use the notation $\operatorname{rank} A$.
This turns out to be an important quantity of a matrix. It tells us a great deal of information about the corresponding linear function.
The rank of a matrix $A$ measures how "big" is the range the associated linear function $f(\mathbf{x}) = A \mathbf{x}$.
That is, the rank of an $m \times n$ matrix $A$ is exactly the dimension of the set \[ \{ A \mathbf{x} \mid \mathbf{x} \in \mathbb{R}^n \} \]
For example, the function \[ \begin{bmatrix} x \\ y \end{bmatrix} \;\mapsto\; \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} \] "projects" the entire plane onto the $x$-axis (horizontal axis). The rank of the matrix, i.e. the dimension of the range, is therefore 1.
A square $n \times n$ matrix $A$ is said to be of full rank if its rank is $n$ (the maximum possible rank).
Full rank square matrices come in pairs with a very special relationship.
We will have to define a few new concepts firsts.
It is important to note that some matrix may not have an inverse at all. An non-example is \[ A = \begin{bmatrix} 1 & 0 \\ 0 & 0 \end{bmatrix} \] which has no inverse.
If an inverse of a matrix does exist, then it must be unique. This is a nontrivial fact, and we will explore the reasoning in other context. Therefore we say it is the inverse of that matrix.
For a square matrix $A$, we use the notation $A^{-1}$ for the inverse of $A$, if exists.
In this case, we say $A$ is invertible (or nonsingular).
The following descriptions are equivalent
Note that for square matrices, we do not have to distinguish left and right inverses.
It is straightforward to verify the following: \[ \begin{aligned} (A^{-1})^{-1} &= A & (A^\top)^{-1} &= (A^{-1})^\top \\ (AB)^{-1} &= B^{-1} A^{-1} & (A^n)^{-1} &= (A^{-1})^n. \end{aligned} \] You should spend some time to explain why each one is true.
It turns out there is a simple description of invertible matrices: they are exactly square matrices of full rank.
Can you see why this is true?
It may be quite confusing that, in linear algebra, there are multiple different ways of saying the same thing. This is actually a feature, not a bug. Indeed, we are barely scratching the surface here. We will see at least 6 different ways of characterizing invertible matrices.
For an $n \times n$ matrix $A$, we can consider the linear function $f : \mathbb{R}^n \to \mathbb{R}^n$ given by \[ f(\mathbf{x}) = A \mathbf{x}. \]
If $A$ has an inverse $A^{-1}$, then it is easy to verify that $f$ also has an inverse function, and this inverse function is given by \[ f^{-1}(\mathbf{x}) = A^{-1} \mathbf{x}. \]
Determine if the following matrices are invertible. If yes, find the inverse matrices. \[ \begin{aligned} A &= \begin{bmatrix} 2 & 0 \\ 0 & 3 \end{bmatrix} & B &= \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix} & C &= \begin{bmatrix} 1 & 1 \\ 2 & 2 \end{bmatrix} \end{aligned} \] \[ \begin{aligned} D &= \begin{bmatrix} 2 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 4 \end{bmatrix} & E &= \begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{bmatrix} & F &= \begin{bmatrix} 1 & 1 & 1 \\ 2 & 2 & 2 \\ 3 & 3 & 3 \end{bmatrix} \end{aligned} \]
Matrix determinant is a particularly useful tool for determine if a matrix is invertible. Indeed, it can also help us to compute the inverse matrix.
However, in this course, we will not pay much attention to determinants. This is intentional.