Systems of simultaneous linear equations in several variables
Many real-world problems can be expressed as systems of simultaneous linear equations.
A few terms are used frequently in our discussions:
Make sure you can distinguish equations and functions.
A linear equation only involve constants or products between one constant and one unknown. E.g., the term $3x$ is okay, but $x^2$ or $\sin(x)$ are not allowed in a linear equation.
Systems of linear equations (or linear systems) are among the most basic computational problems, and they appear naturally in applications.
A linear system is a system of simultaneous equations involving one or more unknowns (variables/indeterminates) where each equation only involve addition and products between constants and unknowns: \[ \left\{ \begin{aligned} 3x + 5y &= 6 \\ 6x - y &= 1 \end{aligned} \right. \] is a system of two linear equations in the two unknowns $x$ and $y$.
Make sure you can identify the different parts of a linear system:
The values for the unknowns that will balance all equations in a linear system is a solution, and the set of all solutions is the solution set.
To solve a linear system means to find the set of all solutions.
Note that the same system may be arranged differently. E.g., \[ \left\{ \begin{aligned} y + 2x &= 5 + x \\ x - y - 1 &= 0, \end{aligned} \right. \] which is equivalent to the above. we should always rearrange and combine terms before we solve them.
There are three commonly used classes of methods for solving small linear systems by hand:
It is important to recognize that not every linear system has a unique solution. There are actually three possibilities.
Don't say we cannot solve this system. Actually, we can.
Think about what is the best way to write down the solution set.
In general there are three kinds of solution sets to linear systems:
It is a bit strange that there is nothing between "one solution" and "infinitely many solutions". E.g., there cannot be just two solutions. We will see why that is the case soon.
For the linear system \[ \left\{ \begin{aligned} a x + b y &= \alpha \\ c x + d y &= \beta \end{aligned} \right. \] in the unknowns $x$ and $y$, its determinant is \[ ad - bc \] and it tells us if the system has a unique solution:
The value of the determinant does not directly distinguish between the cases of infinite and empty solution sets.
Determinant is an important tool. We use the notation \[ \left| \begin{smallmatrix} a & b \\ c & d \end{smallmatrix} \right| \; = \; ad - bc. \] But we will very rarely use it in this course.
The "diet problem" is of great historic importance in the development of applied mathematics. Consider the problem of choosing preserved foods to meet certain nutritional requirements of expedition where supply may be difficult to come by.
| Food source | Vitamin A | Vitamin C | k-Calories |
| Beef | 0.02 | 0.00 | 3.13 |
| Fish | 0.55 | 3.70 | 3.66 |
| Pasta | 0.05 | 0.00 | 0.750 |
Let $M_A, M_C, M_{\text{cal}}$ be the minimum amount of vitamin A, vitamin C, and calories a person requires each day (measured in kIU, mg, and kcal respectively). What is the combination of the three food sources that will meet these requirements?
We can now view a linear system as a single equation involving a matrix-vector product: \[ \left\{ \begin{aligned} a x + b y &= \alpha \\ c x + d y &= \beta \\ \end{aligned} \right. \quad\Leftrightarrow\quad \begin{bmatrix} a & b \\ c & d \end{bmatrix} \, \begin{bmatrix} x \\ y \end{bmatrix} \;=\; \begin{bmatrix} \alpha \\ \beta \end{bmatrix}. \]
In general, a system of $m$ equations involving $n$ unknowns can be written as a compact equation \[ A \mathbf{x} = \mathbf{b} \] where $A$ is an $m \times n$ matrix that collects all the coefficients, $\mathbf{x}$ is a column vector containing $n$ entries $x_1,\dots,x_n$ each representing an unknown, and $\mathbf{b}$ is a column vector containing the $m$ constant terms on the right hand side of the equations.
A linear system can be expressed as \[ A \mathbf{x} = \mathbf{b}. \]
The special case of $\mathbf{b} = \mathbf{0}$ is known as the homogeneous case.
It's called homogeneous because every nonzero term in such a system has degree 1.
In this context the solution set is also called the null space of $A$.
The null space is always nonempty since it contains at least $\mathbf{0}$.
Indeed, the null space of an $m \times n$ matrix is always a subspace of $\mathbb{R}^n$.
Indeed, the solution is unique.
This idea works for larger systems as long as the inverse of the coefficient matrix exists.
A "square" linear system (in which the number of unknowns matches the number of equations) in $n$ unknown can be written as \[ A \mathbf{x} = \mathbf{b} \] where $A$ is an $n \times n$ square matrix. This system has a unique solution if and only if $A$ is invertible (i.e. nonsingular). And the solution is \[ \mathbf{x} = A^{-1} \mathbf{b} \] We can get this by multiplying both sides of the equation by $A^{-1}$.
Square linear systems in real-world applications tend to be large. When we care about efficiency, we generally avoid computing matrix inverse.
They can be solved much more efficiently with "matrix decompositions".
A very useful decomposition is called the QR decomposition.
Every (real) square matrix $A$ can be "decomposed" into a product \[ A = QR \] where $Q$ is an orthogonal matrix and $R$ is an upper triangular matrix. If $A$ is nonsingular, so is $R$.
We will learn more about matrix decompositions soon.