Critical points
A critical point of a real-valued function of several variables refers to a point in the domain where either all (first-order) partial derivatives are zero or some of the partial derivatives are undefined. They are crucial in the study of functions. In particular, all local extrema are must be critical points (although the converse is not true).
Critical points of real valued functions in two variables
How to find crtical points
For a function \(f(x,y)\) that is defined on an open domain, the problem of finding critical points of \(f\) is exactly the problem of finding points \((x,y)\) such that
- either both \(\frac{\partial f}{\partial x} = \frac{\partial f}{\partial y} = 0\)
- or one of \(\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}\) is undefined.
Classifictaion of critical points
For a differentiable function, isolated critical points can be categorized as degenerate or nondegenerate.
Nondegenerate critical points
Nondegenerate critical points are the simpler kind, they comprise local maxima, local minima, or saddle points.
- At a local maximum, the function possesses a peak value within a neighborhood of the point.
- At a local minimum, the function has a low point (value-wise) within a neighborhood.
- Saddle points, on the other hand, may appear to be a local maxima in one direction but a local minima in another. They are neither local maxima nor local minima.
As long as the function has continuous second order partial derivatives, we can classify a nondegenerate critical point through a simple test.
In the language of linear algebra, we can state the same test in terms of determinant, trace, or eigenvalues.
Degenerate critical points
Degenerate critical points are much more complex to classify since the second derivative test does not lead to a definitive conclusion. Essentially, the function’s graph is so “flat” near a degenerate critical point that the first and second order partial derivatives don’t provide enough information to understand the situation. To conclusively classify such points, one may need further analysis or higher derivative information.
Fitting a line through two points
Suppose we have two points \((x_1,y_1) = (1, 1)\) and \((x_2,y_2) = (3, 2)\). Consider a straight line that is the graph of the function
\[ y = f(x) = mx + b \]
for some real numbers \(m\) and \(b\). Let us define the function
\[ E(m,b) = [ y_1 - f(x_1) ]^2 + [ y_2 - f(x_2) ]^2 \]
This function is the sum of squared \(y\)-difference between the line and the two given points. The problem of finding the line that passes through both point is therefore equivalent to the problem of finding the right choices of \(m\) and \(b\) so that \(E(m,b)\) is minimized. Even through we can find \(m\) and \(b\) directly, the goal of this exercise is to recover them using the tool of calculus.
Find and classify the critical points for this function.
Best fitting line among three points
Suppose we now have three points \[ \begin{aligned} (x_1,y_1) &= (1, 1) & (x_2,y_2) &= (3, 2) & (x_3,y_3) &= (4, 2) \end{aligned} \] Consider a straight line that is the graph of the function
\[ y = f(x) = mx + b \]
for some real numbers \(m\) and \(b\). Let us define the function
\[ E(m,b) = [ y_1 - f(x_1) ]^2 + [ y_2 - f(x_2) ]^2 + [ y_3 - f(x_3) ]^2 \]
This function is the sum of squared \(y\)-difference between the line and the given points. The problem of finding the “best fitting” line is therefore equivalent to the problem of finding the right choices of \(m\) and \(b\) so that \(E(m,b)\) is minimized. This problem is known as the linear least square problem, and it is a form of linear regression.
Find and classify the critical points for this function.