When the Taylor polynomials of a function converge to it, why this holds for sine and cosine everywhere, and why infinitely many derivatives are not enough.
If a function has derivatives of every order, Taylor’s theorem holds for everyn. It is then tempting to let n→∞ and write the function as an infinite sum, a power series. §152 shows when that is legitimate: exactly when the remainder tends to 0. This gives the series for sinx and cosx at once, and gives the binomial series in part. It also contains Hardy’s warning that the existence of all the derivatives is not enough, a warning with a famous counter-example.
The idea
Suppose f has derivatives of all orders near a. For each n, Taylor’s theorem splits f(a+h) into the Taylor polynomial Pn−1(a+h) and the remainder Rn. As n grows, the polynomials are the partial sums of one infinite series,
f(a)+hf′(a)+2!h2f′′(a)+3!h3f′′′(a)+⋯.
Whether this series adds up to f(a+h) is a question about the remainders only: since f(a+h)=Pn−1(a+h)+Rn for every n, the partial sums tend to f(a+h) if and only if Rn→0.
ExploreTaylor polynomials of sinx
Try this.Press “Raise n” and watch the polynomials hug the curve over a wider and wider stretch. Then drag the probe far out, to x=6, and raise n until the error is tiny.
f(x)=sinx, centre a=0, degree n=3
xy
f(x)
Taylor polynomial Pn
remainder f − Pn
centre a (drag)
probe x (drag)
P3(x)=x−0.1667x3 At x=4.2: f(x)=−0.87158, P3(x)=−8.148, error ∣f(x)−P3(x)∣=7.28.
What to notice
Every fixed x is eventually captured, however far out: at x=6 the error falls below 0.001 once n reaches 19. Far from the centre the polynomials first go badly wrong and only later settle down.
ExploreThe same experiment for 1/(1+x2)
Try this.Set the probe at x=1.5 and raise n from 1 to 20. Then move it to x=0.8 and repeat.
f(x)=1+x21, centre a=0, degree n=4
xy
f(x)
Taylor polynomial Pn
remainder f − Pn
interval of convergence |x − a| < R
centre a (drag)
probe x (drag)
P4(x)=1−x2+x4 At x=1.8: f(x)=0.23585, P4(x)=8.2576, error ∣f(x)−P4(x)∣=8.02 — this x is outside the interval of convergence (R=1), so raising n will not help. Smooth on the whole line, yet its Taylor series at 0 converges only for |x| < 1.
What to notice
At x=0.8 the error shrinks steadily; at x=1.5 it explodes. The green interval marks where the series converges. Here the reason is invisible on the real line: it lies in the complex plane, at x=±i, as Chapter X will explain.
Precise statements
DefinitionTaylor's series and Maclaurin's seriesHardy: §152
Let f have derivatives of all orders in an interval (a−η,a+η). Its Taylor series about a is
f(a)+hf′(a)+2!h2f′′(a)+⋯+n!hnf(n)(a)+⋯.
When a=0 (and h is written x) it is called Maclaurin’s series: f(0)+xf′(0)+2!x2f′′(0)+⋯. The remainder Rn=n!hnf(n)(a+θnh) is Lagrange’s form of the remainder.
TheoremWhen the series represents the functionHardy: §152
Let f have derivatives of all orders in (a−η,a+η), and let ∣h∣<η. The Taylor series converges to f(a+h) if and only if Rn→0 as n→∞.
TheoremThe sine and cosine seriesHardy: §152 (1)
For all real x,
sinx=x−3!x3+5!x5−⋯,cosx=1−2!x2+4!x4−⋯.
More generally sin(x+h) and cos(x+h) equal their Taylor series in powers of h, for all x and h.
TheoremThe binomial series, first partHardy: §152 (2)
If m is a positive integer the series stops and this is the ordinary binomial theorem. The expansion is also true for −1<x<0, but Lagrange’s form of the remainder cannot prove it there in general; Hardy completes the proof with Cauchy’s form in §168.
Proofs
The first theorem is immediate: the partial sums are Pn−1(a+h)=f(a+h)−Rn, and they tend to f(a+h) exactly when Rn→0. Everything therefore rests on estimating Rn.
ProofThe sine series
Strategy
Every derivative of sin is ±sin or ±cos, so it is bounded by 1 everywhere, wherever the unknown point a+θnh is. That gives ∣Rn∣≤∣h∣n/n!, which tends to 0 for every h.
All derivatives exist and are bounded by 1
dxndnsinx=sin(x+21nπ), so ∣f(n)(t)∣≤1 for every t and every n.
Bound the remainder
Lagrange’s form gives ∣Rn∣=n!∣h∣n∣f(n)(x+θnh)∣≤n!∣h∣n, whatever θn is.
∣h∣n/n!→0 for every fixed h
Once n>2∣h∣, each further factor ∣h∣/(n+1) is less than 21, so the terms are eventually at most a constant times 2−n. (This is Examples XXVII, 12, which you can watch in the next figure.)
Conclude
Rn→0, so sin(x+h) equals its Taylor series for all x and h. Taking x=0, the derivatives at 0 are 0,1,0,−1,0,1,…, which gives sinh=h−h3/3!+⋯. The cosine is the same.
∎
ExploreWhy hn/n! always wins
Try this.Here xn/n! with x=5. Find an n0 for δ=0.01. Then raise x to 10 or more with its slider and do it again.
ϕ(n)=n!xn , candidate limit l=0
nφ(n)
band l ± δ
l
n₀ (drag)
n ≥ n₀, good
n ≥ n₀, bad
15 terms with n ≥ n₀ fall outside the band: n = 1, 2, 3, 4, 5, ….
What to notice
The terms first grow (while n<x) and then collapse, because from then on each step multiplies by x/(n+1), which is eventually less than 21. A larger x only delays the collapse; it cannot prevent it. That is why the sine series converges at every point.
For the binomial series the derivatives are no longer bounded by a constant, and the position of θn starts to matter.
ProofThe binomial series for 0≤x<1
Strategy
Write down Lagrange’s remainder and bound the factor (1+θnx)m−n using only 0<θn<1. For x≥0 this factor is at most 1 once n>m, and what is left tends to 0.
The derivatives
For f(x)=(1+x)m, f(n)(x)=m(m−1)⋯(m−n+1)(1+x)m−n, so f(n)(0)/n!=(nm), and Lagrange’s remainder about 0 is
Rn=(nm)xn(1+θnx)m−n.
The awkward factor is at most 1 when x≥0
If x≥0 then 1+θnx≥1, and once n>m the exponent m−n is negative, so (1+θnx)m−n≤1. Hence ∣Rn∣≤(nm)xn.
(nm)xn→0 when ∣x∣<1
The ratio of consecutive terms is (n+1m)xn+1/(nm)xn=n+1∣m−n∣∣x∣, which tends to ∣x∣<1. So from some point on the terms shrink by a fixed factor less than 1, and tend to 0 (Examples XXVII, 13).
Where the argument breaks for x<0
If −1<x<0, then 1+θnx<1 and (1+θnx)m−n>1. Knowing only 0<θn<1, the factor could be as large as (1+x)m−n, which grows like (1+x)−n. The bound becomes (nm)(1+x)m(∣x∣/(1+x))n, which tends to 0 if ∣x∣<21 but not if ∣x∣>21.
∎
ExploreLagrange's remainder is too pessimistic for x near −1
Try this.With m=21, compare the two rows of dots at x=0.8, at x=−0.4 and at x=−0.8. Then try other values of m.
(1+x)m with m=0.5, x=−0.8: size of the remainder after n terms (logarithmic scale)
nsize
true remainder |Rₙ|
Lagrange's bound, knowing only 0 < θ < 1
After 40 terms: ∣R40∣=6.59×10−7, while Lagrange's bound is 6.09×1020. For x<0 the bound behaves like (∣x∣/(1+x))n=4n, times a power of n. The bound does not tend to 0, so it proves nothing, although the true remainder still tends to 0.
What to notice
The blue dots (the true remainder) always fall towards 0 when ∣x∣<1. The hollow orange dots (the best bound Lagrange's form allows when θn is unknown) fall too for x>−21, but rise for x<−21. The theorem is still true there; only this particular proof fails, which is why Hardy needs a different form of the remainder.
The last warning of §152 is the most important. A function may have derivatives of every order, its Taylor series may even converge everywhere, and yet the sum may be a different function.
ExploreA Taylor series that converges to the wrong function
Try this.Raise n as far as it will go and watch the Taylor polynomial of e−1/x2 about 0.
f(x)=e−1/x2, centre a=0, degree n=4
xy
f(x)
Taylor polynomial Pn
remainder f − Pn
centre a
probe x (drag)
P4(x)=0 At x=1.8: f(x)=0.73444, P4(x)=0, error ∣f(x)−P4(x)∣=0.734. Every derivative vanishes at 0, so its Taylor series there is 0: it converges, but not to the function.
What to notice
Every Taylor polynomial is the zero polynomial, because every derivative of e−1/x2 at 0 is 0 (exercise below). The series converges, to 0, for every x; but the function is positive for every x=0. So Rn=f(x) for all n: it never tends to 0.
Worked examples and exercises
ExerciseThe cosine series●○○Hardy: §152 (1)
Prove that cosx=1−2!x2+4!x4−⋯ for all real x, and that the error after the term in x2n−2 is at most ∣x∣2n/(2n)!.
Hint
Copy the proof for sin: cos(n)x=cos(x+21nπ).
Solution
All derivatives of cos are bounded by 1, so ∣Rn∣≤∣x∣n/n!→0 and the series converges to cosx. The derivatives at 0 are 1,0,−1,0,1,…, which gives the series. Since the coefficient of x2n−1 is 0, the polynomial through the term in x2n−2 is also P2n−1, and its remainder is R2n=(2n)!x2ncos(2n)(θx), of size at most ∣x∣2n/(2n)!.
ExerciseHow many terms?●○○Hardy: §152 (1)
How many non-zero terms of the sine series are needed to compute sin1 with an error guaranteed to be less than 10−6?
Hint
With the terms through x2n−1 (that is, n non-zero terms) the coefficient of x2n is zero, so the error is at most 1/(2n)!.
Solution
We need (2n)!>106. Now 8!=40320 is too small and 10!=3628800 is enough, so 2n=10 and n=5: sin1≈1−61+1201−50401+3628801=0.8414710…, with error at most 1/10!≈2.8×10−7. (In fact sin1=0.8414709….)
ExerciseThe geometric series as a binomial series●●○Hardy: §152 (2); cf. Examples LV, 2
For m=−1 the binomial series is 1−x+x2−⋯. Find the remainder Rn exactly, show that it tends to 0 for −1<x<1, and compare with what Lagrange’s form gives for x=−0.8.
Hint
The first n terms form a geometric progression with ratio −x.
Solution
∑k=0n−1(−x)k=1+x1−(−x)n, so Rn=1+x1−1+x1−(−x)n=1+x(−x)n, which tends to 0 when ∣x∣<1.
Lagrange’s form writes the same remainder as (−1)nxn(1+θnx)−n−1. Knowing only 0<θn<1, at x=−0.8 the factor (1−0.8θn)−n−1 could be as large as 0.2−n−1=5n+1, so the bound is 0.8n⋅5n+1=5⋅4n, which is useless. The true θn is the one solving (1+θnx)n+1=1+x (as in Examples LV, 2); it makes the factor exactly 1/(1+x), but Lagrange’s theorem does not tell us that.
ExerciseA smooth function with zero Taylor series●●●Hardy: §152, the warning
Let f(x)=e−1/x2 for x=0 and f(0)=0. Assuming the elementary properties of the exponential (Ch. IX), show that f(n)(0)=0 for every n.
Hint
Show by induction that for x=0, f(n)(x)=pn(1/x)e−1/x2 for some polynomial pn.
Hint
Then use the fact that tke−t2→0 as ∣t∣→∞, for every k.
Solution
For x=0, f′(x)=2x−3e−1/x2, and differentiating pn(1/x)e−1/x2 gives {−x−2pn′(1/x)+2x−3pn(1/x)}e−1/x2, again of the same form. So f(n)(x)=pn(1/x)e−1/x2 with pn a polynomial.
Now argue by induction that f(n)(0)=0. It holds for n=0. If f(n)(0)=0, then
because each term tke−t2→0. So every Taylor polynomial at 0 is identically 0, and Rn=f(x)=0 for every x=0 and every n.
Check yourself
Check yourselfTrue or false?
If f has derivatives of every order at every point near a, then its Taylor series about a converges to f(x) for x near a.
Why
False. This is exactly Hardy’s warning. e−1/x2 (with value 0 at 0) has derivatives of all orders everywhere, its Maclaurin series is 0+0+0+⋯, and that converges to 0, not to the function. One must show Rn→0 directly.
Check yourselfChoose one
To prove that f(x)=∑f(k)(0)xk/k! for a given x, what exactly must be shown?
Why
The partial sums are f(x)−Rn, so they tend to f(x) exactly when Rn→0. Convergence of the series is not enough (it may converge to something else, as for e−1/x2); terms tending to 0 is not even enough for convergence; and f(n)(0) need not tend to 0 (for ex it is always 1).
Check yourselfEnter a number
Using the bound 1/(2n)! for the error after n non-zero terms, how many non-zero terms of the sine series guarantee ∣sin1−(partial sum)∣<10−6?
Why
(2n)!>106 first happens at 2n=10, since 8!=40320 and 10!=3628800. So 5 terms, through x9/9!.
Check yourselfTrue or false?
Lagrange’s form of the remainder, with only 0<θn<1 known, is enough to prove the binomial series for (1+x)1/2 at x=−0.4.
Why
True. The worst case bound behaves like (∣x∣/(1+x))n=(0.4/0.6)n=(2/3)n, times a power of n, which tends to 0. The argument fails only for −1<x<−21, where ∣x∣/(1+x)>1. Check it in the widget.
Hardy’s language
The reader should be careful to guard himself against supposing that the existence of all the derivatives of f(x) is a sufficient condition for the validity of Taylor’s series.
— G. H. Hardy, §152
Summary
Summary
The Taylor series of f about a converges to f(a+h) exactly when the remainder Rn→0; this must be checked directly.
For sin and cos every derivative is bounded by 1, so ∣Rn∣≤∣h∣n/n!→0: the series hold for all x.
For (1+x)m Lagrange’s form proves the series for 0≤x<1 (indeed for −21<x<1); near −1 it is too crude, and Cauchy’s form (§168) is needed.
Derivatives of all orders are not enough: e−1/x2 has Maclaurin series 0.