Chapter VII · Additional theorems in the differential and integral calculus

Taylor's series

§152 35 min

When the Taylor polynomials of a function converge to it, why this holds for sine and cosine everywhere, and why infinitely many derivatives are not enough.

Why this matters

If a function has derivatives of every order, Taylor’s theorem holds for every nn. It is then tempting to let n→∞n \to \infty and write the function as an infinite sum, a power series. §152 shows when that is legitimate: exactly when the remainder tends to 00. This gives the series for sin⁡x\sin x and cos⁡x\cos x at once, and gives the binomial series in part. It also contains Hardy’s warning that the existence of all the derivatives is not enough, a warning with a famous counter-example.

The idea

Suppose ff has derivatives of all orders near aa. For each nn, Taylor’s theorem splits f(a+h)f(a + h) into the Taylor polynomial Pn−1(a+h)P_{n-1}(a + h) and the remainder RnR_n. As nn grows, the polynomials are the partial sums of one infinite series,

f(a)+hf′(a)+h22!f′′(a)+h33!f′′′(a)+⋯ .f(a) + h f'(a) + \frac{h^2}{2!} f''(a) + \frac{h^3}{3!} f'''(a) + \cdots.

Whether this series adds up to f(a+h)f(a + h) is a question about the remainders only: since f(a+h)=Pn−1(a+h)+Rnf(a + h) = P_{n-1}(a + h) + R_n for every nn, the partial sums tend to f(a+h)f(a + h) if and only if Rn→0R_n \to 0.

Explore Taylor polynomials of sin⁡x\sin x

Try this. Press “Raise n” and watch the polynomials hug the curve over a wider and wider stretch. Then drag the probe far out, to x=6x = 6, and raise nn until the error is tiny.

f(x)=sin⁡xf(x) = \sin x, centre a=0a = 0, degree n=3n = 3
xy
  • f(x)
  • Taylor polynomial Pn
  • remainder f − Pn
  • centre a (drag)
  • probe x (drag)
P3(x)=x−0.1667x3P_{3}(x) = x - 0.1667x^{3}
At x=4.2x = 4.2: f(x)=−0.87158f(x) = −0.87158, P3(x)=−8.148P_{3}(x) = −8.148, error ∣f(x)−P3(x)∣=7.28|f(x) - P_{3}(x)| = 7.28.
What to notice

Every fixed xx is eventually captured, however far out: at x=6x = 6 the error falls below 0.0010.001 once nn reaches 1919. Far from the centre the polynomials first go badly wrong and only later settle down.

Explore The same experiment for 1/(1+x2)1/(1+x^2)

Try this. Set the probe at x=1.5x = 1.5 and raise nn from 11 to 2020. Then move it to x=0.8x = 0.8 and repeat.

f(x)=11+x2f(x) = \frac{1}{1+x^2}, centre a=0a = 0, degree n=4n = 4
xy
  • f(x)
  • Taylor polynomial Pn
  • remainder f − Pn
  • interval of convergence |x − a| < R
  • centre a (drag)
  • probe x (drag)
P4(x)=1−x2+x4P_{4}(x) = 1 - x^{2} + x^{4}
At x=1.8x = 1.8: f(x)=0.23585f(x) = 0.23585, P4(x)=8.2576P_{4}(x) = 8.2576, error ∣f(x)−P4(x)∣=8.02|f(x) - P_{4}(x)| = 8.02 — this xx is outside the interval of convergence (R=1R = 1), so raising n will not help.
Smooth on the whole line, yet its Taylor series at 0 converges only for |x| < 1.
What to notice

At x=0.8x = 0.8 the error shrinks steadily; at x=1.5x = 1.5 it explodes. The green interval marks where the series converges. Here the reason is invisible on the real line: it lies in the complex plane, at x=±ix = \pm i, as Chapter X will explain.

Precise statements

Definition Taylor's series and Maclaurin's series Hardy: §152

Let ff have derivatives of all orders in an interval (a−η,a+η)(a - \eta, a + \eta). Its Taylor series about aa is

f(a)+hf′(a)+h22!f′′(a)+⋯+hnn!f(n)(a)+⋯ .f(a) + h f'(a) + \frac{h^2}{2!} f''(a) + \cdots + \frac{h^n}{n!} f^{(n)}(a) + \cdots.

When a=0a = 0 (and hh is written xx) it is called Maclaurin’s series: f(0)+xf′(0)+x22!f′′(0)+⋯f(0) + x f'(0) + \frac{x^2}{2!} f''(0) + \cdots. The remainder Rn=hnn!f(n)(a+θnh)R_n = \frac{h^n}{n!} f^{(n)}(a + \theta_n h) is Lagrange’s form of the remainder.

Theorem When the series represents the function Hardy: §152

Let ff have derivatives of all orders in (a−η,a+η)(a - \eta, a + \eta), and let ∣h∣<η|h| < \eta. The Taylor series converges to f(a+h)f(a + h) if and only if Rn→0R_n \to 0 as n→∞n \to \infty.

Theorem The sine and cosine series Hardy: §152 (1)

For all real xx,

sin⁡x=x−x33!+x55!−⋯ ,cos⁡x=1−x22!+x44!−⋯ .\sin x = x - \frac{x^3}{3!} + \frac{x^5}{5!} - \cdots, \qquad \cos x = 1 - \frac{x^2}{2!} + \frac{x^4}{4!} - \cdots.

More generally sin⁡(x+h)\sin(x + h) and cos⁡(x+h)\cos(x + h) equal their Taylor series in powers of hh, for all xx and hh.

Theorem The binomial series, first part Hardy: §152 (2)

Let mm be rational and 0≤x<10 \le x < 1. Then

(1+x)m=1+(m1)x+(m2)x2+⋯ ,(mn)=m(m−1)⋯(m−n+1)n!.(1 + x)^m = 1 + \binom{m}{1} x + \binom{m}{2} x^2 + \cdots, \qquad \binom{m}{n} = \frac{m(m-1)\cdots(m-n+1)}{n!}.

If mm is a positive integer the series stops and this is the ordinary binomial theorem. The expansion is also true for −1<x<0-1 < x < 0, but Lagrange’s form of the remainder cannot prove it there in general; Hardy completes the proof with Cauchy’s form in §168.

Proofs

The first theorem is immediate: the partial sums are Pn−1(a+h)=f(a+h)−RnP_{n-1}(a + h) = f(a + h) - R_n, and they tend to f(a+h)f(a + h) exactly when Rn→0R_n \to 0. Everything therefore rests on estimating RnR_n.

Proof The sine series
Strategy

Every derivative of sin⁡\sin is ±sin⁡\pm\sin or ±cos⁡\pm\cos, so it is bounded by 11 everywhere, wherever the unknown point a+θnha + \theta_n h is. That gives ∣Rn∣≤∣h∣n/n!|R_n| \le |h|^n/n!, which tends to 00 for every hh.

All derivatives exist and are bounded by 1

dndxnsin⁡x=sin⁡(x+12nπ)\dfrac{d^n}{dx^n}\sin x = \sin(x + \tfrac12 n\pi), so ∣f(n)(t)∣≤1|f^{(n)}(t)| \le 1 for every tt and every nn.

Bound the remainder

Lagrange’s form gives ∣Rn∣=∣h∣nn! ∣f(n)(x+θnh)∣≤∣h∣nn!|R_n| = \dfrac{|h|^n}{n!}\,|f^{(n)}(x + \theta_n h)| \le \dfrac{|h|^n}{n!}, whatever θn\theta_n is.

∣h∣n/n!→0|h|^n/n! \to 0 for every fixed hh

Once n>2∣h∣n > 2|h|, each further factor ∣h∣/(n+1)|h|/(n + 1) is less than 12\tfrac12, so the terms are eventually at most a constant times 2−n2^{-n}. (This is Examples XXVII, 12, which you can watch in the next figure.)

Conclude

Rn→0R_n \to 0, so sin⁡(x+h)\sin(x + h) equals its Taylor series for all xx and hh. Taking x=0x = 0, the derivatives at 00 are 0,1,0,−1,0,1,…0, 1, 0, -1, 0, 1, \dots, which gives sin⁡h=h−h3/3!+⋯\sin h = h - h^3/3! + \cdots. The cosine is the same.

Explore Why hn/n!h^n/n! always wins

Try this. Here xn/n!x^n/n! with x=5x = 5. Find an n0n_0 for δ=0.01\delta = 0.01. Then raise xx to 1010 or more with its slider and do it again.

ϕ(n)=xnn!\phi(n) = \frac{x^n}{n!} , candidate limit l=0l = 0
nφ(n)
  • band l ± δ
  • l
  • n₀ (drag)
  • n ≥ n₀, good
  • n ≥ n₀, bad
15 terms with n ≥ n₀ fall outside the band: n = 1, 2, 3, 4, 5, ….
What to notice

The terms first grow (while n<xn < x) and then collapse, because from then on each step multiplies by x/(n+1)x/(n+1), which is eventually less than 12\tfrac12. A larger xx only delays the collapse; it cannot prevent it. That is why the sine series converges at every point.

For the binomial series the derivatives are no longer bounded by a constant, and the position of θn\theta_n starts to matter.

Proof The binomial series for 0≤x<10 \le x < 1
Strategy

Write down Lagrange’s remainder and bound the factor (1+θnx)m−n(1 + \theta_n x)^{m-n} using only 0<θn<10 < \theta_n < 1. For x≥0x \ge 0 this factor is at most 11 once n>mn > m, and what is left tends to 00.

The derivatives

For f(x)=(1+x)mf(x) = (1 + x)^m, f(n)(x)=m(m−1)⋯(m−n+1)(1+x)m−nf^{(n)}(x) = m(m-1)\cdots(m-n+1)(1 + x)^{m-n}, so f(n)(0)/n!=(mn)f^{(n)}(0)/n! = \binom{m}{n}, and Lagrange’s remainder about 00 is

Rn=(mn)xn(1+θnx)m−n.R_n = \binom{m}{n} x^n (1 + \theta_n x)^{m-n}.

The awkward factor is at most 1 when x≥0x \ge 0

If x≥0x \ge 0 then 1+θnx≥11 + \theta_n x \ge 1, and once n>mn > m the exponent m−nm - n is negative, so (1+θnx)m−n≤1(1 + \theta_n x)^{m-n} \le 1. Hence ∣Rn∣≤∣(mn)∣xn|R_n| \le \bigl|\binom{m}{n}\bigr| x^n.

(mn)xn→0\binom{m}{n} x^n \to 0 when ∣x∣<1|x| < 1

The ratio of consecutive terms is ∣(mn+1)xn+1∣/∣(mn)xn∣=∣m−n∣n+1 ∣x∣\Bigl|\binom{m}{n+1}x^{n+1}\Bigr| \Big/ \Bigl|\binom{m}{n}x^n\Bigr| = \dfrac{|m - n|}{n + 1}\,|x|, which tends to ∣x∣<1|x| < 1. So from some point on the terms shrink by a fixed factor less than 11, and tend to 00 (Examples XXVII, 13).

Where the argument breaks for x<0x < 0

If −1<x<0-1 < x < 0, then 1+θnx<11 + \theta_n x < 1 and (1+θnx)m−n>1(1 + \theta_n x)^{m-n} > 1. Knowing only 0<θn<10 < \theta_n < 1, the factor could be as large as (1+x)m−n(1 + x)^{m - n}, which grows like (1+x)−n(1 + x)^{-n}. The bound becomes ∣(mn)∣(1+x)m(∣x∣/(1+x))n\bigl|\binom{m}{n}\bigr|(1+x)^m \bigl(|x|/(1 + x)\bigr)^n, which tends to 00 if ∣x∣<12|x| < \tfrac12 but not if ∣x∣>12|x| > \tfrac12.

Explore Lagrange's remainder is too pessimistic for xx near −1-1

Try this. With m=12m = \tfrac12, compare the two rows of dots at x=0.8x = 0.8, at x=−0.4x = -0.4 and at x=−0.8x = -0.8. Then try other values of mm.

(1+x)m(1 + x)^{m} with m=0.5m = 0.5, x=−0.8x = -0.8: size of the remainder after nn terms (logarithmic scale)
nsize
  • true remainder |Rₙ|
  • Lagrange's bound, knowing only 0 < θ < 1
After 40 terms: ∣R40∣=6.59×10−7|R_{40}| = 6.59 \times 10^{-7}, while Lagrange's bound is 6.09×10206.09 \times 10^{20}. For x<0x < 0 the bound behaves like (∣x∣/(1+x))n=4n\bigl(|x|/(1+x)\bigr)^n = 4^n, times a power of nn. The bound does not tend to 0, so it proves nothing, although the true remainder still tends to 0.
What to notice

The blue dots (the true remainder) always fall towards 00 when ∣x∣<1|x| < 1. The hollow orange dots (the best bound Lagrange's form allows when θn\theta_n is unknown) fall too for x>−12x > -\tfrac12, but rise for x<−12x < -\tfrac12. The theorem is still true there; only this particular proof fails, which is why Hardy needs a different form of the remainder.

The last warning of §152 is the most important. A function may have derivatives of every order, its Taylor series may even converge everywhere, and yet the sum may be a different function.

Explore A Taylor series that converges to the wrong function

Try this. Raise nn as far as it will go and watch the Taylor polynomial of e−1/x2e^{-1/x^2} about 00.

f(x)=e−1/x2f(x) = e^{-1/x^2}, centre a=0a = 0, degree n=4n = 4
xy
  • f(x)
  • Taylor polynomial Pn
  • remainder f − Pn
  • centre a
  • probe x (drag)
P4(x)=0P_{4}(x) = 0
At x=1.8x = 1.8: f(x)=0.73444f(x) = 0.73444, P4(x)=0P_{4}(x) = 0, error ∣f(x)−P4(x)∣=0.734|f(x) - P_{4}(x)| = 0.734.
Every derivative vanishes at 0, so its Taylor series there is 0: it converges, but not to the function.
What to notice

Every Taylor polynomial is the zero polynomial, because every derivative of e−1/x2e^{-1/x^2} at 00 is 00 (exercise below). The series converges, to 00, for every xx; but the function is positive for every x≠0x \ne 0. So Rn=f(x)R_n = f(x) for all nn: it never tends to 00.

Worked examples and exercises

Exercise The cosine series ●○○ Hardy: §152 (1)

Prove that cos⁡x=1−x22!+x44!−⋯\cos x = 1 - \dfrac{x^2}{2!} + \dfrac{x^4}{4!} - \cdots for all real xx, and that the error after the term in x2n−2x^{2n-2} is at most ∣x∣2n/(2n)!|x|^{2n}/(2n)!.

Hint
Copy the proof for sin⁡\sin: cos⁡(n)x=cos⁡(x+12nπ)\cos^{(n)} x = \cos(x + \tfrac12 n\pi).

Solution

All derivatives of cos⁡\cos are bounded by 11, so ∣Rn∣≤∣x∣n/n!→0|R_n| \le |x|^n/n! \to 0 and the series converges to cos⁡x\cos x. The derivatives at 00 are 1,0,−1,0,1,…1, 0, -1, 0, 1, \dots, which gives the series. Since the coefficient of x2n−1x^{2n-1} is 00, the polynomial through the term in x2n−2x^{2n-2} is also P2n−1P_{2n-1}, and its remainder is R2n=x2n(2n)!cos⁡(2n)(θx)R_{2n} = \frac{x^{2n}}{(2n)!}\cos^{(2n)}(\theta x), of size at most ∣x∣2n/(2n)!|x|^{2n}/(2n)!.

Exercise How many terms? ●○○ Hardy: §152 (1)

How many non-zero terms of the sine series are needed to compute sin⁡1\sin 1 with an error guaranteed to be less than 10−610^{-6}?

Hint
With the terms through x2n−1x^{2n-1} (that is, nn non-zero terms) the coefficient of x2nx^{2n} is zero, so the error is at most 1/(2n)!1/(2n)!.

Solution

We need (2n)!>106(2n)! > 10^6. Now 8!=40 3208! = 40\,320 is too small and 10!=3 628 80010! = 3\,628\,800 is enough, so 2n=102n = 10 and n=5n = 5: sin⁡1≈1−16+1120−15040+1362 880=0.8414710…\sin 1 \approx 1 - \frac16 + \frac1{120} - \frac1{5040} + \frac1{362\,880} = 0.8414710\ldots, with error at most 1/10!≈2.8×10−71/10! \approx 2.8 \times 10^{-7}. (In fact sin⁡1=0.8414709…\sin 1 = 0.8414709\ldots.)

Exercise The geometric series as a binomial series ●●○ Hardy: §152 (2); cf. Examples LV, 2

For m=−1m = -1 the binomial series is 1−x+x2−⋯1 - x + x^2 - \cdots. Find the remainder RnR_n exactly, show that it tends to 00 for −1<x<1-1 < x < 1, and compare with what Lagrange’s form gives for x=−0.8x = -0.8.

Hint
The first nn terms form a geometric progression with ratio −x-x.

Solution

∑k=0n−1(−x)k=1−(−x)n1+x\sum_{k=0}^{n-1}(-x)^k = \dfrac{1 - (-x)^n}{1 + x}, so Rn=11+x−1−(−x)n1+x=(−x)n1+xR_n = \dfrac{1}{1+x} - \dfrac{1 - (-x)^n}{1 + x} = \dfrac{(-x)^n}{1 + x}, which tends to 00 when ∣x∣<1|x| < 1.

Lagrange’s form writes the same remainder as (−1)nxn(1+θnx)−n−1(-1)^n x^n (1 + \theta_n x)^{-n-1}. Knowing only 0<θn<10 < \theta_n < 1, at x=−0.8x = -0.8 the factor (1−0.8θn)−n−1(1 - 0.8\theta_n)^{-n-1} could be as large as 0.2−n−1=5n+10.2^{-n-1} = 5^{n+1}, so the bound is 0.8n⋅5n+1=5⋅4n0.8^n \cdot 5^{n+1} = 5 \cdot 4^n, which is useless. The true θn\theta_n is the one solving (1+θnx)n+1=1+x(1 + \theta_n x)^{n+1} = 1 + x (as in Examples LV, 2); it makes the factor exactly 1/(1+x)1/(1 + x), but Lagrange’s theorem does not tell us that.

Exercise A smooth function with zero Taylor series ●●● Hardy: §152, the warning

Let f(x)=e−1/x2f(x) = e^{-1/x^2} for x≠0x \ne 0 and f(0)=0f(0) = 0. Assuming the elementary properties of the exponential (Ch. IX), show that f(n)(0)=0f^{(n)}(0) = 0 for every nn.

Hint
Show by induction that for x≠0x \ne 0, f(n)(x)=pn(1/x) e−1/x2f^{(n)}(x) = p_n(1/x)\, e^{-1/x^2} for some polynomial pnp_n.
Hint
Then use the fact that tke−t2→0t^k e^{-t^2} \to 0 as ∣t∣→∞|t| \to \infty, for every kk.

Solution

For x≠0x \ne 0, f′(x)=2x−3e−1/x2f'(x) = 2x^{-3}e^{-1/x^2}, and differentiating pn(1/x)e−1/x2p_n(1/x)e^{-1/x^2} gives {−x−2pn′(1/x)+2x−3pn(1/x)}e−1/x2\bigl\{-x^{-2}p_n'(1/x) + 2x^{-3}p_n(1/x)\bigr\}e^{-1/x^2}, again of the same form. So f(n)(x)=pn(1/x)e−1/x2f^{(n)}(x) = p_n(1/x)e^{-1/x^2} with pnp_n a polynomial.

Now argue by induction that f(n)(0)=0f^{(n)}(0) = 0. It holds for n=0n = 0. If f(n)(0)=0f^{(n)}(0) = 0, then

f(n+1)(0)=lim⁡x→0f(n)(x)−0x=lim⁡x→01x pn(1/x) e−1/x2=lim⁡∣t∣→∞t pn(t) e−t2=0,f^{(n+1)}(0) = \lim_{x \to 0} \frac{f^{(n)}(x) - 0}{x} = \lim_{x\to 0} \frac1x\,p_n(1/x)\,e^{-1/x^2} = \lim_{|t| \to \infty} t\,p_n(t)\,e^{-t^2} = 0,

because each term tke−t2→0t^k e^{-t^2} \to 0. So every Taylor polynomial at 00 is identically 00, and Rn=f(x)≠0R_n = f(x) \ne 0 for every x≠0x \ne 0 and every nn.

Check yourself

Check yourself True or false?

If ff has derivatives of every order at every point near aa, then its Taylor series about aa converges to f(x)f(x) for xx near aa.

Why

False. This is exactly Hardy’s warning. e−1/x2e^{-1/x^2} (with value 00 at 00) has derivatives of all orders everywhere, its Maclaurin series is 0+0+0+⋯0 + 0 + 0 + \cdots, and that converges to 00, not to the function. One must show Rn→0R_n \to 0 directly.

Check yourself Choose one

To prove that f(x)=∑f(k)(0)xk/k!f(x) = \sum f^{(k)}(0)x^k/k! for a given xx, what exactly must be shown?

Why

The partial sums are f(x)−Rnf(x) - R_n, so they tend to f(x)f(x) exactly when Rn→0R_n \to 0. Convergence of the series is not enough (it may converge to something else, as for e−1/x2e^{-1/x^2}); terms tending to 00 is not even enough for convergence; and f(n)(0)f^{(n)}(0) need not tend to 00 (for exe^x it is always 11).

Check yourself Enter a number

Using the bound 1/(2n)!1/(2n)! for the error after nn non-zero terms, how many non-zero terms of the sine series guarantee ∣sin⁡1−(partial sum)∣<10−6|\sin 1 - \text{(partial sum)}| < 10^{-6}?

Why

(2n)!>106(2n)! > 10^6 first happens at 2n=102n = 10, since 8!=40 3208! = 40\,320 and 10!=3 628 80010! = 3\,628\,800. So 55 terms, through x9/9!x^9/9!.

Check yourself True or false?

Lagrange’s form of the remainder, with only 0<θn<10 < \theta_n < 1 known, is enough to prove the binomial series for (1+x)1/2(1 + x)^{1/2} at x=−0.4x = -0.4.

Why

True. The worst case bound behaves like (∣x∣/(1+x))n=(0.4/0.6)n=(2/3)n\bigl(|x|/(1+x)\bigr)^n = (0.4/0.6)^n = (2/3)^n, times a power of nn, which tends to 00. The argument fails only for −1<x<−12-1 < x < -\tfrac12, where ∣x∣/(1+x)>1|x|/(1 + x) > 1. Check it in the widget.

Hardy’s language

The reader should be careful to guard himself against supposing that the existence of all the derivatives of f(x) is a sufficient condition for the validity of Taylor’s series.

— G. H. Hardy, §152

Summary

Summary
  • The Taylor series of ff about aa converges to f(a+h)f(a + h) exactly when the remainder Rn→0R_n \to 0; this must be checked directly.
  • For sin⁡\sin and cos⁡\cos every derivative is bounded by 11, so ∣Rn∣≤∣h∣n/n!→0|R_n| \le |h|^n/n! \to 0: the series hold for all xx.
  • For (1+x)m(1 + x)^m Lagrange’s form proves the series for 0≤x<10 \le x < 1 (indeed for −12<x<1-\tfrac12 < x < 1); near −1-1 it is too crude, and Cauchy’s form (§168) is needed.
  • Derivatives of all orders are not enough: e−1/x2e^{-1/x^2} has Maclaurin series 00.