The mean value theorem of §126 says f(a+h)=f(a)+hf′(ξ) for some ξ between a and a+h. It is the tangent-line approximation with an honest account of its error. This lesson asks what happens if we know more derivatives: can f(a),f′(a),…,f(n−1)(a) predict f(a+h), and how large is the error? The answer, Taylor’s theorem, is the main tool of the whole first half of the chapter: power series, maxima and minima, limits of the form 0/0 and the contact of curves all come from it.
The idea
Start from the mean value theorem. It says that the gap between f(a+h) and the constant prediction f(a) is h times some value of f′ between a and a+h. We do not know which value, but if we know the largest and smallest values of f′ there, we know a range in which f(a+h) must lie.
Now use one more derivative. The natural prediction is the tangent line f(a)+hf′(a). Its error turns out to be 21h2 times some value of f′′ between a and a+h. With n derivatives, the prediction is the Taylor polynomial
which is the polynomial of degree n−1 that has the same value and the same first n−1 derivatives as f at a. Its error, the remainderRn, is hn/n! times some value of f(n) between a and a+h. So f(a+h) is trapped in a band whose width is controlled by how much f(n) varies.
ExploreThe band in which f must lie
Try this.Start with n=1: the band is f(a)+h[minf′,maxf′], the mean value theorem. Raise n to 2, 3, 4 and watch the band close in around f. Then drag h and read off where ξ sits and the ratio θn=(ξ−a)/h.
f(x)=ex, centre a=0, polynomial P0 (degree 0), remainder term of order n=1
xy
f(x)
Taylor polynomial P₀
band allowed by Lagrange's remainder
remainder R₁ at x = a + h
ξ (on the x-axis)
At x=a+h=1.5: f(x)=4.48169, P0(x)=1, remainder R1=3.482. The band at this x runs from 2.5 to 7.7225, and f(x) lies in it. Lagrange: R1=1!h1f(1)(ξ) with ξ=0.84205, so θ1=(ξ−a)/h=0.5614 (compare 1/(n+1)=0.5). §151 form: h11!R1=2.32113=f(1)(a)+η with η=1.32.
What to notice
However you set things, the blue curve never leaves the green band: that is the theorem. For small h the band is very thin, because its width is a multiple of hn. And ξ is a genuine point between a and a+h; it moves as h changes, so the theorem tells us it exists but not where.
Two cautions are already visible. The point ξ depends on h (and on n and f): it is not a fixed number. And the formula with f(n)(ξ) is useful only when we can say something about f(n) on the whole interval, usually a bound.
Precise statements
Hardy first proves the case of two derivatives, which he calls the mean value theorem of the second order, and then the general case.
TheoremMean value theorem of the second orderHardy: §150 (2)
Suppose that f′ is continuous on [a,a+h] and that f′′ exists on the open interval between a and a+h. Then
f(a+h)=f(a)+hf′(a)+21h2f′′(a+θ2h)for some 0<θ2<1.
TheoremTaylor's theorem, with Lagrange's remainderHardy: §150, the general mean value theorem
Suppose that f(n−1) is continuous on the closed interval with end points a and a+h, and that f(n) exists on the open interval. Then
for some θn with 0<θn<1. Equivalently, with b=a+h, the last term is n!(b−a)nf(n)(ξ) with ξ strictly between a and b.
For n=1 this is the mean value theorem itself. When f is only defined on [a,b], the derivatives at the end points are understood one-sidedly, as Hardy remarks.
§151 proves a second form that assumes much less: derivatives up to order nat the single point a (which forces f(n−1) to exist near a, but says nothing about f(n) anywhere else). In exchange it says less: only that the error is small compared with hn.
TheoremTaylor's theorem, with the remainder o(hn)Hardy: §151 (1), (2)
where η→0 as h→0. In the order notation of §98, f(a+h)=∑k=0nk!hkf(k)(a)+o(hn).
The two forms answer different questions. Lagrange’s form is a statement at a fixedh, and with a bound on f(n) it gives a numerical error bound. The o(hn) form is a statement about h→0 only, and it is exactly what the applications in §§153–155 need.
Proofs
The proof of Taylor’s theorem is the proof of the mean value theorem with a cleverer auxiliary function. Write b=a+h.
ProofTaylor's theorem with Lagrange's remainder
Strategy
Build a function ϕ on [a,b] that vanishes at both ends, so that Rolle’s theorem gives a point where ϕ′=0, and arrange that ϕ′(ξ)=0 says exactly what we want. The key is to expand about the moving point x and predict the fixed value f(b).
Then Fn(b)=0, and Fn(a) is exactly the remainder Rn that we want to describe.
Its derivative collapses to one term
Differentiate the general term: dxd{k!(b−x)kf(k)(x)}=k!(b−x)kf(k+1)(x)−(k−1)!(b−x)k−1f(k)(x). Summing over k=0,…,n−1, the terms cancel in pairs (the sum telescopes), leaving
Fn′(x)=−(n−1)!(b−x)n−1f(n)(x)(a<x<b).
Correct Fn so that it also vanishes at a
Let ϕ(x)=Fn(x)−(b−ab−x)nFn(a). Then ϕ(a)=Fn(a)−Fn(a)=0 and ϕ(b)=0−0=0. The hypotheses make ϕ continuous on [a,b] (it involves f,…,f(n−1)) and differentiable on (a,b) (it involves f(n) there).
Since b−ξ=0 we may divide by (b−ξ)n−1, which gives Fn(a)=n!(b−a)nf(n)(ξ). Writing b=a+h and ξ=a+θnh gives the theorem. If h<0, the same argument works on [a+h,a].
∎
For the o(hn) form we cannot use Rolle’s theorem on f(n), because f(n) may exist only at a. Hardy’s idea is to push the information about the n-th derivative at one point down through the lower derivatives, one at a time.
ProofTaylor's theorem with remainder o(hn)
Strategy
Subtract the whole Taylor polynomial of degree n. The difference g has g(0)=g′(0)=⋯=g(n)(0)=0. We show that such a g satisfies ∣g(h)∣≤ε∣h∣n near 0, for any ε>0, by applying the mean value theorem n−1 times.
Set up g
Let g(h)=f(a+h)−∑k=0nk!hkf(k)(a). Then g(k)(0)=0 for k=0,1,…,n. Because g(n)(0) exists, g(n−1) exists on some interval around 0, and so do g,g′,…,g(n−2), each continuous there. We must show g(h)/hn→0.
The top derivative is small compared with h
By the definition of the derivative, g(n−1)(h)/h=(g(n−1)(h)−g(n−1)(0))/h→g(n)(0)=0. So, given ε>0, there is a δ>0 with ∣g(n−1)(t)∣≤ε∣t∣ whenever ∣t∣<δ.
Pass the bound down, one derivative at a time
Let ∣h∣<δ. Since g(n−2)(0)=0, the mean value theorem gives g(n−2)(h)=hg(n−1)(t) with ∣t∣<∣h∣, so ∣g(n−2)(h)∣≤∣h∣⋅ε∣h∣=ε∣h∣2. Repeating with g(n−3),…,g (each vanishes at 0) gives ∣g(n−3)(h)∣≤ε∣h∣3, …, ∣g(h)∣≤ε∣h∣n.
Conclude
So ∣g(h)/hn∣≤ε for 0<∣h∣<δ, and ε was arbitrary: g(h)=o(hn). Setting η=n!g(h)/hn gives exactly the form in the theorem, with η→0.
∎
Hardy’s own version reaches the same conclusion with one-sided inequalities: he compares Fn(h) with n!hn{f(n)(a)±δ} and uses the fact that a function with positive derivative at 0 is increasing there (§122). The idea, information flowing down from f(n)(a) to f, is the same.
ExploreWhere is ξ as h→0?
Try this.With n=2, drag h slowly towards 0 and watch θ2 and η in the read-out. Repeat with n=3. Then move the centre to a=0, set n=1, and shrink h again.
f(x)=sinx, centre a=0.6, polynomial P1 (degree 1), remainder term of order n=2
xy
f(x)
Taylor polynomial P₁
band allowed by Lagrange's remainder
remainder R₂ at x = a + h
ξ (on the x-axis)
At x=a+h=1.8: f(x)=0.973848, P1(x)=1.55505, remainder R2=−0.5812. The band at this x runs from 0.83505 to 1.1485, and f(x) lies in it. Lagrange: R2=2!h2f(2)(ξ) with ξ=0.93943, so θ2=(ξ−a)/h=0.2829 (compare 1/(n+1)=0.3333). §151 form: h22!R2=−0.807219=f(2)(a)+η with η=−0.243.
What to notice
For small h, θn settles near 1/(n+1), and η (the §151 error) tends to 0. At a=0 with n=1 the rule fails: θ1 tends to 1/3≈0.577, because f′′(0)=−sin0=0. The exercise below explains both.
Worked examples and exercises
ExerciseThe sine with a remainder●○○Hardy: Examples LV, 3
Write down Taylor’s theorem for sin(x+h) with the remainder after the term in h2n−1, and deduce that the polynomial part approximates sin(x+h) with error at most ∣h∣2n/(2n)!.
Hint
The derivatives of sin cycle: sin,cos,−sin,−cos. In general sin(k)x=sin(x+21kπ).
The remainder is (−1)n(2n)!h2nsin(x+θ2nh), and ∣sin∣≤1, so the error is at most ∣h∣2n/(2n)!. The same works for cos(x+h) with cos(x+θh) in the remainder.
ExerciseTaylor's theorem for 1/x●●○Hardy: Examples LV, 2
and check the result directly, without Taylor’s theorem.
Hint
The k-th derivative of 1/x is (−1)kk!/xk+1. That gives the formula at once.
Hint
For the direct check, sum the geometric progression, and ask whether xn(x+h) lies between xn+1 and (x+h)n+1.
Solution
From the theorem. With f(t)=1/t we have f(k)(t)/k!=(−1)k/tk+1, so the terms are (−1)khk/xk+1 and the remainder is (−1)nhn/(x+θnh)n+1. The hypotheses hold because 1/t has derivatives of every order on the interval between x and x+h, which does not contain 0.
Directly. The sum of the first n terms is a geometric progression with ratio −h/x:
We need xn(x+h)=(x+θh)n+1 for some 0<θ<1. If h>0 then xn+1<xn(x+h)<(x+h)n+1; if h<0 the inequalities reverse. The function θ↦(x+θh)n+1 is continuous and runs from xn+1 to (x+h)n+1, so by the intermediate value theorem it takes the value xn(x+h) at some θ strictly between 0 and 1.
ExerciseNewton's method●●○Hardy: Examples LV, 7–8
Let ξ be an approximation to a root ξ+h of f(x)=0, with f′(ξ)=0. Use the second-order mean value theorem to show that ξ1=ξ−f(ξ)/f′(ξ) is in error by O(h2). Apply this to x2=2 starting from ξ=23, twice.
Hint
0=f(ξ+h)=f(ξ)+hf′(ξ)+21h2f′′(ξ+θh). Divide by f′(ξ).
Hint
For f(x)=x2−2 the step is ξ1=ξ−(ξ2−2)/(2ξ)=21(ξ+2/ξ).
Solution
Dividing the hint’s equation by f′(ξ) gives ξ+h=ξ−f(ξ)/f′(ξ)−21h2f′′(ξ+θh)/f′(ξ), so the root differs from ξ1 by 21h2f′′(ξ+θh)/f′(ξ). If ∣f′′∣≤M and ∣f′∣≥K>0 near a simple root, this is at most Mh2/(2K): the error is squared at each step (up to a constant), and the next steps have errors O(h4), O(h8), ….
For x2=2: ξ1=21(23+34)=1217=1.41666…, and ξ2=21(1217+1724)=408577=1.4142157…. Since 2=1.4142136…, ξ2 is correct to five decimal places, although the start 23 was correct to none.
Show that if f′(a) exists then 2hf(a+h)−f(a−h)→f′(a), and that if f′′(a) exists then h2f(a+h)−2f(a)+f(a−h)→f′′(a), as h→0. Do the converses hold?
Hint
Use the §151 form with n=1, and then with n=2, once for +h and once for −h.
Solution
With n=1: f(a±h)=f(a)±h{f′(a)+η±} where η±→0. Subtracting, f(a+h)−f(a−h)=2hf′(a)+h(η++η−); divide by 2h and let h→0.
With n=2: f(a±h)=f(a)±hf′(a)+21h2{f′′(a)+η±}. Adding, the f′(a) terms cancel: f(a+h)−2f(a)+f(a−h)=h2f′′(a)+21h2(η++η−); divide by h2.
The converses fail. For f(x)=∣x∣ at a=0 the first quotient is 0 for every h, but f′(0) does not exist. For f(x)=x∣x∣, which has f′(0)=0 but no f′′(0), the second quotient is 0 for every h.
ExerciseWhere ξ goes as h→0●●●Hardy: Examples LV, 12
Suppose that f(n+1) is continuous near a and f(n+1)(a)=0. Show that the θn of Lagrange’s form tends to 1/(n+1) as h→0.
Hint
Write f(a+h) with Lagrange’s remainder of order n and of order n+1, and compare: f(n)(a+θnh)−f(n)(a)=n+1hf(n+1)(a+θn+1h).
Hint
Apply the mean value theorem to the left side: it is θnhf(n+1)(a+θθnh).
Hence f(n)(a+θnh)−f(n)(a)=n+1hf(n+1)(a+θn+1h). By the mean value theorem the left side is θnhf(n+1)(a+θθnh) with 0<θ<1. Cancelling h,
θn=n+11⋅f(n+1)(a+θθnh)f(n+1)(a+θn+1h).
As h→0 both arguments tend to a, and by continuity numerator and denominator tend to the same non-zero number f(n+1)(a). So θn→1/(n+1).
The condition f(n+1)(a)=0 is needed (Hardy leaves it implicit). For sin at 0 with n=1, sinh=hcos(θ1h) gives cos(θ1h)=(sinh)/h=1−h2/6+…, and comparing with 1−θ12h2/2 shows θ1→1/3, as the second exploration shows.
Check yourself
Check yourselfChoose one
In f(a+h)=f(a)+hf′(a)+21h2f′′(ξ), what is ξ?
Why
Rolle’s theorem only guarantees that a suitable ξ exists in the open interval. It moves with h (drag h in the widget), and is the midpoint only by accident; for small h it is near a+h/3 when f′′′(a)=0. It is not where f′′ is largest: the remainder uses a value between the smallest and the largest.
Check yourselfTrue or false?
Taylor’s theorem says that the remainder after n terms is n!hnf(n)(a).
Why
False. That would make f equal to a polynomial of degree n. Lagrange’s form has f(n) at an unknown point a+θnh; the §151 form has f(n)(a)+η with η→0. Both say “approximately n!hnf(n)(a) for small h”, which is a different statement.
Check yourselfEnter a number
Approximate e1/2 by 1+21+81 (Taylor’s theorem for ex about 0 with n=3). The remainder is 3!h3eξ with h=21 and 0<ξ<21. Using only eξ<2, what upper bound for the error does this give?
Why
6(1/2)3⋅2=241≈0.0417. The true error is e1/2−1.625=0.0237…, comfortably below the bound. A bound on f(n) over the interval is exactly what Lagrange’s form needs.
Check yourselfChoose one
Which hypothesis is enough for f(a+h)=∑k=0nk!hkf(k)(a)+o(hn) as h→0?
Why
That is the point of §151: one derivative of order n at one point suffices. Continuity alone gives nothing beyond f(a+h)=f(a)+o(1). The stronger hypotheses also work, but they are not needed; and even infinitely many derivatives do not make the series converge to f (next lesson).
Check yourselfTrue or false?
Taylor’s theorem with Lagrange’s remainder, in the case n=1, is the mean value theorem.
Why
True. With n=1 the polynomial part is just f(a) and the remainder is hf′(a+θ1h). The proof also reduces to the usual one: ϕ(x)=f(b)−f(x)−b−ab−x{f(b)−f(a)}.
Hardy’s language
But we do not assume even the existence of the nth derivative at any point except x = a.
— G. H. Hardy, §151
Summary
Summary
Taylor polynomial: Pn−1(a+h)=∑k=0n−1k!hkf(k)(a), the polynomial matching f and its first n−1 derivatives at a.
Lagrange: f(a+h)=Pn−1(a+h)+n!hnf(n)(a+θnh), 0<θn<1. A bound on f(n) gives a bound on the error.
Proof: Rolle’s theorem applied to Fn(x)−(b−ab−x)nFn(a), where Fn(x) is the error in predicting f(b) from x.
With only f(n)(a) assumed: f(a+h)=Pn(a+h)+o(hn), proved by passing the smallness down one derivative at a time.
The point ξ moves with h; if f(n+1)(a)=0 it approaches a+h/(n+1).