Taylor's theorem

Calculus

In the comprehensive article on integrals, we saw how integration can be used for approximating the area between the function and the \(X\)-axis. What if you wished to approximate the function itself? That's where Taylor's Theorem can help you.

Prerequisites

To understand Taylor's Theorem, we recommend familiarity with the concepts in

Follow the above links to first get acquainted with the corresponding concepts.

Function approximation

Here's what we already know, written in a slightly different way.

$$ f(x) = f(a) + \int_a^x f'(t)dt $$

This can be refined by further decomposing the integral.

$$ f(x) = f(a) + \int_a^x \left(f'(a) + \int_a^t f''(p)dp \right) dt $$

This can be re-written as

$$ f(x) = f(a) + f'(a) \int_a^x dt + \int_a^x \int_a^t f''(p)dp dt $$

Note that, \( \int_a^x dt = (x - a) \). So, we have

$$ f(x) = f(a) + (x-a) f'(a) + \int_a^x \int_a^t f''(p)dp dt $$

Continue substituting \( f^n(p) \) with elements with higher order derivatives, and you end up with the the following relationship for a \(k\)-times differentiable function.

$$ f(x) = f(a) + (x-a) f'(a) + (x-a)^2 \frac{f''(a)}{2!} + (x-a)^3 \frac{f'''(a)}{3!} + \ldots + (x-a)^k\frac{f^k(a)}{k!} $$

This relationship is a famous result in calculus known as Taylor's Theorem.

Taylor's theorem is a handy way to approximate a function at a point \( x \), if we can readily estimate its value and those of its derivatives at some other point \( a \) in its domain. It appears in quite a few derivations in optimization and machine learning.

Taylor's theorem approximation demo.

Let's try to approximate the function \( f(x) = \exp(x) \) using Taylor's theorem. The true function is shown in blue color and the approximated line is shown in red color.

You can change the approximation anchor point \( a \) using the relevant slider. You can also change the number of terms in the Taylor series expansion by changing its slider.

Function approximation using Taylor's Theorem. (Blue indicates true function, red the approximation)

Two things to note here:

  • The approximated function (red line) is quite accurate in the region surrounding the anchor \( a \).
  • The accuracy of the approximation improves as you include more terms in the expansion. At order 1, we are merely using \( \exp(a) \) for approximating the entire function, leading to poor approximation.

Another demo

Let's try to approximate a more wavy function \( f(x) = \sin(x) \) using Taylor's theorem. The true function is shown in blue color and the approximated line is shown in red color.

You can change the approximation anchor point \( a \) using the relevant slider. You can also change the number of terms in the Taylor series expansion by changing its slider.

Function approximation using Taylor's Theorem. (Blue indicates true function, red the approximation)

Again, we can make the observations:

  • Approximation better in vicinity of the anchor
  • More terms in the expansion improves approximation.

But there is one more thing. With more derivatives included in the expansion, the approximation is able to introduce more curves.

  • With no derivatives, the approximation is a flat line, parallel to the \(X\)-axis
  • With 2 derivatives, we introduce one curve but it is useless beyond the immediate vicinity.
  • With 4 derivatives, we introduce one more curve into the approximation.

Amazing, isn't it?

Taylor's Theorem for multivariate functions

Taylor's Theorem is also applicable to multivariate functions.

For example, in the case of a bivariate function \( f: \real^2 \to \real \), the Taylor's expansion with 2 terms is

$$ f(x+a, y+b) = f(a,b) + x\dox{f}\bigg\rvert_{a,b} + y\doy{f}\bigg\rvert_{a,b} + \frac{1}{2}\left( x^2 \doxx{f}\bigg\rvert_{a,b} + xy\doxy{f}\bigg\rvert_{a,b} + xy \doyx{f}\bigg\rvert_{a,b} + y^2 \doyy{f}\bigg\rvert_{a,b} \right) $$

Here, \( g\bigg\rvert_{a,b} \) means that the derivative is evaluated at the point \( (a,b) \).

To understand multivariate derivatives, head on to our comprehensive article.

Where to next?

Check out our other interactive tutorials in calculus.

Already a calculus expert? Check out comprehensive courses on multivariate calculus, optimization, machine learning or deep learning

Please support us

Help us create more engaging and effective content and keep it free of paywalls and advertisements!

Please donate

Let's connect

Please share your comments, questions, encouragement, and feedback.