# Taylor's theorem

##### Calculus

In the comprehensive article on integrals, we saw how integration can be used for approximating the area between the function and the $X$-axis. What if you wished to approximate the function itself? That's where Taylor's Theorem can help you.

## Prerequisites

To understand Taylor's Theorem, we recommend familiarity with the concepts in

Follow the above links to first get acquainted with the corresponding concepts.

## Function approximation

Here's what we already know, written in a slightly different way.

$$f(x) = f(a) + \int_a^x f'(t)dt$$

This can be refined by further decomposing the integral.

$$f(x) = f(a) + \int_a^x \left(f'(a) + \int_a^t f''(p)dp \right) dt$$

This can be re-written as

$$f(x) = f(a) + f'(a) \int_a^x dt + \int_a^x \int_a^t f''(p)dp dt$$

Note that, $\int_a^x dt = (x - a)$. So, we have

$$f(x) = f(a) + (x-a) f'(a) + \int_a^x \int_a^t f''(p)dp dt$$

Continue substituting $f^n(p)$ with elements with higher order derivatives, and you end up with the the following relationship for a $k$-times differentiable function.

$$f(x) = f(a) + (x-a) f'(a) + (x-a)^2 \frac{f''(a)}{2!} + (x-a)^3 \frac{f'''(a)}{3!} + \ldots + (x-a)^k\frac{f^k(a)}{k!}$$

This relationship is a famous result in calculus known as Taylor's Theorem.

Taylor's theorem is a handy way to approximate a function at a point $x$, if we can readily estimate its value and those of its derivatives at some other point $a$ in its domain. It appears in quite a few derivations in optimization and machine learning.

## Taylor's theorem approximation demo.

Let's try to approximate the function $f(x) = \exp(x)$ using Taylor's theorem. The true function is shown in blue color and the approximated line is shown in red color.

You can change the approximation anchor point $a$ using the relevant slider. You can also change the number of terms in the Taylor series expansion by changing its slider.

Two things to note here:

• The approximated function (red line) is quite accurate in the region surrounding the anchor $a$.
• The accuracy of the approximation improves as you include more terms in the expansion. At order 1, we are merely using $\exp(a)$ for approximating the entire function, leading to poor approximation.

## Another demo

Let's try to approximate a more wavy function $f(x) = \sin(x)$ using Taylor's theorem. The true function is shown in blue color and the approximated line is shown in red color.

You can change the approximation anchor point $a$ using the relevant slider. You can also change the number of terms in the Taylor series expansion by changing its slider.

Again, we can make the observations:

• Approximation better in vicinity of the anchor
• More terms in the expansion improves approximation.

But there is one more thing. With more derivatives included in the expansion, the approximation is able to introduce more curves.

• With no derivatives, the approximation is a flat line, parallel to the $X$-axis
• With 2 derivatives, we introduce one curve but it is useless beyond the immediate vicinity.
• With 4 derivatives, we introduce one more curve into the approximation.

Amazing, isn't it?

## Taylor's Theorem for multivariate functions

Taylor's Theorem is also applicable to multivariate functions.

For example, in the case of a bivariate function $f: \real^2 \to \real$, the Taylor's expansion with 2 terms is

$$f(x+a, y+b) = f(a,b) + x\dox{f}\bigg\rvert_{a,b} + y\doy{f}\bigg\rvert_{a,b} + \frac{1}{2}\left( x^2 \doxx{f}\bigg\rvert_{a,b} + xy\doxy{f}\bigg\rvert_{a,b} + xy \doyx{f}\bigg\rvert_{a,b} + y^2 \doyy{f}\bigg\rvert_{a,b} \right)$$

Here, $g\bigg\rvert_{a,b}$ means that the derivative is evaluated at the point $(a,b)$.

## Where to next?

Check out our other interactive tutorials in calculus.

Already a calculus expert? Check out comprehensive courses on multivariate calculus, optimization, machine learning or deep learning