Chapter VII · Additional theorems in the differential and integral calculus

Taylor's theorem by integration by parts

§167 35 min

Integrating by parts gives the remainder in Taylor's theorem as an exact integral, from which Lagrange's and Cauchy's forms both follow by the mean value theorems for integrals.

Why this matters

In §150 Taylor’s theorem was proved with Rolle’s theorem, and the error after nn terms came out as hnn!f(n)(a+θh)\frac{h^n}{n!}f^{(n)}(a + \theta h) for some unknown θ\theta between 00 and 11 (Lagrange’s form). §152 showed that this is not always good enough: for (1+x)m(1+x)^m with xx near −1-1, knowing only 0<θ<10 < \theta < 1 we cannot prove that the error tends to 00. §167 gives a second proof, by integration by parts, which produces the error exactly, as an integral with no unknown θ\theta in it. From that integral, Lagrange’s form and a new one, Cauchy’s form, both follow; and Cauchy’s form settles the binomial series in the next lesson.

The idea

Taylor’s theorem is the fundamental theorem of the calculus, f(a+h)=f(a)+∫aa+hf′(x) dxf(a + h) = f(a) + \int_a^{a+h} f'(x)\,dx, improved step by step. Integrate by parts once, moving a derivative onto f′f', and a term hf′(a)hf'(a) splits off, leaving an integral of f′′f''. Again, and 12h2f′′(a)\tfrac12 h^2 f''(a) splits off. After n−1n - 1 steps,

f(a+h)=f(a)+hf′(a)+⋯+hn−1(n−1)!f(n−1)(a)+Rn,Rn=hn(n−1)!∫01(1−t)n−1f(n)(a+th) dt.f(a+h) = f(a) + hf'(a) + \dots + \frac{h^{n-1}}{(n-1)!}f^{(n-1)}(a) + R_n, \qquad R_n = \frac{h^n}{(n-1)!}\int_0^1 (1-t)^{n-1}f^{(n)}(a + th)\,dt .

So the remainder is the area under a definite curve on [0,1][0, 1]. An area can be written as “length times a height reached by the curve” in more than one way, and the two classical forms of the remainder are two such ways.

  • Cauchy: replace the whole integrand by its value at one point t=θt = \theta. By the first mean value theorem the area is that value times 11.
  • Lagrange: keep the factor (1−t)n−1(1-t)^{n-1}, which is never negative, as a weight, and replace only f(n)(a+th)f^{(n)}(a + th) by a value f(n)(a+θh)f^{(n)}(a + \theta h). By the weighted mean value theorem this is allowed, and ∫01(1−t)n−1 dt=1/n\int_0^1(1-t)^{n-1}\,dt = 1/n gives hnn!f(n)(a+θh)\frac{h^n}{n!}f^{(n)}(a+\theta h).
Explore One remainder, two ways to average it

Try this. With log⁡(1+x)\log(1+x), n=3n = 3 and hh near −0.8-0.8, compare the two numbers in the read-out (they agree), and the two θ\theta’s. Then increase nn, and move hh to positive values.

f(x)=log⁡(1+x)f(x) = \log(1+x), remainder after n=3n = 3 terms at h=−0.8h = -0.8Rn=hn(n−1)!∫01(1−t)n−1f(n)(th) dtR_n = \dfrac{h^n}{(n-1)!}\int_0^1 (1-t)^{n-1} f^{(n)}(th)\,dt, the area of the blue region (counted negative below the axis)
t
  • integrand; its shaded area is exactly Rₙ
  • Cauchy: rectangle of width 1, same area (height = integrand at θ)
  • Lagrange: the weight (1 − t)ⁿ⁻¹ scaled by f⁽ⁿ⁾(θh), same area; it crosses the integrand at θ
Function
f(h)−∑k<nf(k)(0)k!hk=−0.489438f(h) - \sum_{k<n} \frac{f^{(k)}(0)}{k!}h^k = -0.489438; area under the curve =−0.489438= -0.489438.
Cauchy’s form holds with θ=0.7601\theta = 0.7601; Lagrange’s with θ=0.3702\theta = 0.3702. Same remainder, different points.
What to notice

The blue area is the remainder itself: the read-out computes it also as f(h)f(h) minus the Taylor polynomial, and the two agree. The ochre rectangle and the dashed purple curve both have that same area, but they touch the integrand at different points, so Cauchy’s θ\theta and Lagrange’s θ\theta are different numbers. For negative hh near −1-1 and large nn, f(n)(th)f^{(n)}(th) is huge near t=1t = 1 while the factor (1−t)n−1(1-t)^{n-1} is tiny there, and their product stays small. Lagrange’s form separates the two factors (and, knowing only 0<θ<10 < \theta < 1, must allow for the huge values of f(n)f^{(n)}); Cauchy’s form keeps them together at one point, which is what it exploits.

Precise statements

Theorem Taylor’s theorem with integral remainder Hardy: §167 (1), (2)

Let ff have continuous derivatives up to order nn on the interval between aa and a+ha + h. Then

f(a+h)=f(a)+hf′(a)+h22!f′′(a)+⋯+hn−1(n−1)!f(n−1)(a)+Rn,f(a+h) = f(a) + hf'(a) + \frac{h^2}{2!}f''(a) + \dots + \frac{h^{n-1}}{(n-1)!}f^{(n-1)}(a) + R_n,

where

Rn=hn(n−1)!∫01(1−t)n−1f(n)(a+th) dt=1(n−1)!∫aa+h(a+h−x)n−1f(n)(x) dx.R_n = \frac{h^n}{(n-1)!}\int_0^1 (1-t)^{n-1}f^{(n)}(a + th)\,dt = \frac{1}{(n-1)!}\int_a^{a+h} (a+h-x)^{n-1}f^{(n)}(x)\,dx .
Corollary Schlömilch’s form; Lagrange’s and Cauchy’s forms Hardy: §167 (3), (4)

Under the same hypotheses, for every integer pp with 1≤p≤n1 \le p \le n there is a θ\theta with 0<θ<10 < \theta < 1 such that

Rn=(1−θ)n−p f(n)(a+θh) hnp (n−1)!.R_n = \frac{(1-\theta)^{n-p}\,f^{(n)}(a+\theta h)\,h^n}{p\,(n-1)!} .

With p=np = n this is Lagrange’s form Rn=hnn!f(n)(a+θh)R_n = \frac{h^n}{n!}f^{(n)}(a + \theta h); with p=1p = 1 it is Cauchy’s form

Rn=(1−θ)n−1 f(n)(a+θh) hn(n−1)!.R_n = \frac{(1-\theta)^{n-1}\,f^{(n)}(a+\theta h)\,h^n}{(n-1)!} .

The θ\theta depends on pp (and on nn and hh).

Two remarks from Hardy. The integral form is exact: no undetermined number appears in it. On the other hand, as a proof of Lagrange’s form it is slightly weaker than the proof of §150, because it assumes that f(n)f^{(n)} is continuous, while Rolle’s theorem needs only that f(n)f^{(n)} exists. (The Rolle argument can be adapted to give the general forms too.) The name “Schlömilch’s form” for general pp is the modern one; Hardy does not name it.

Proofs

Proof The integral remainder
Strategy

Fix the end point b=a+hb = a + h and consider the error of the Taylor polynomial about the moving point xx, as a function Fn(x)F_n(x). It vanishes at x=bx = b, and its derivative telescopes to a single term. Then Fn(a)=Fn(b)−∫abFn′F_n(a) = F_n(b) - \int_a^b F_n', which is the remainder.

The error as a function of the base point

Let

Fn(x)=f(b)−f(x)−(b−x)f′(x)−⋯−(b−x)n−1(n−1)!f(n−1)(x).F_n(x) = f(b) - f(x) - (b-x)f'(x) - \dots - \frac{(b-x)^{n-1}}{(n-1)!}f^{(n-1)}(x).

Then Fn(a)=RnF_n(a) = R_n (it is the error of the expansion about aa), and Fn(b)=0F_n(b) = 0 (every term with a factor b−xb - x vanishes, and f(b)−f(b)=0f(b) - f(b) = 0).

Differentiate: the sum telescopes

For k≥1k \ge 1, ddx(b−x)kk!f(k)(x)=(b−x)kk!f(k+1)(x)−(b−x)k−1(k−1)!f(k)(x)\dfrac{d}{dx}\dfrac{(b-x)^k}{k!}f^{(k)}(x) = \dfrac{(b-x)^k}{k!}f^{(k+1)}(x) - \dfrac{(b-x)^{k-1}}{(k-1)!}f^{(k)}(x). Adding these for k=0,…,n−1k = 0, \dots, n-1, each positive term cancels the negative term of the next, and

Fn′(x)=−(b−x)n−1(n−1)!f(n)(x).F_n'(x) = -\frac{(b-x)^{n-1}}{(n-1)!}f^{(n)}(x).

Integrate

Fn′F_n' is continuous (this is where the continuity of f(n)f^{(n)} is used), so Fn(a)=Fn(b)−∫abFn′(x) dx=1(n−1)!∫ab(b−x)n−1f(n)(x) dxF_n(a) = F_n(b) - \int_a^b F_n'(x)\,dx = \dfrac{1}{(n-1)!}\displaystyle\int_a^b (b-x)^{n-1}f^{(n)}(x)\,dx.

Change the variable

Put x=a+thx = a + th, dx=h dtdx = h\,dt, so b−x=h(1−t)b - x = h(1-t), and tt runs from 00 to 11: Rn=hn(n−1)!∫01(1−t)n−1f(n)(a+th) dtR_n = \dfrac{h^n}{(n-1)!}\displaystyle\int_0^1(1-t)^{n-1}f^{(n)}(a+th)\,dt. (The argument works equally for h<0h < 0.)

The telescoping derivative is integration by parts in disguise: differentiating FnF_n is the same as integrating by parts n−1n - 1 times, as in the idea above.

Proof Schlömilch’s form, hence Lagrange’s and Cauchy’s
Strategy

Split the factor (1−t)n−1(1-t)^{n-1} as (1−t)n−p⋅(1−t)p−1(1-t)^{n-p}\cdot(1-t)^{p-1}. Treat (1−t)p−1≥0(1-t)^{p-1} \ge 0 as a weight, and apply the generalised mean value theorem of §165 to the rest.

Choose the weight

Let 1≤p≤n1 \le p \le n. On [0,1][0, 1] put ϕ(t)=(1−t)p−1≥0\phi(t) = (1-t)^{p-1} \ge 0 and g(t)=(1−t)n−pf(n)(a+th)g(t) = (1-t)^{n-p}f^{(n)}(a+th), which is continuous. Then the integral in RnR_n is ∫01gϕ\int_0^1 g\phi.

Generalised mean value theorem

By (9) of §165, ∫01gϕ=g(θ)∫01ϕ\int_0^1 g\phi = g(\theta)\int_0^1\phi for some θ\theta in [0,1][0, 1], and ∫01(1−t)p−1 dt=1/p\int_0^1(1-t)^{p-1}\,dt = 1/p. So

∫01(1−t)n−1f(n)(a+th) dt=(1−θ)n−pf(n)(a+θh)p.\int_0^1(1-t)^{n-1}f^{(n)}(a+th)\,dt = \frac{(1-\theta)^{n-p}f^{(n)}(a+\theta h)}{p}.

Multiplying by hn/(n−1)!h^n/(n-1)! gives the formula. (Hardy states 0<θ<10 < \theta < 1; the theorem as proved gives 0≤θ≤10 \le \theta \le 1, and the end points can be excluded by a slightly finer argument; nothing below depends on it.)

The two extreme cases

p=np = n: the weight is all of (1−t)n−1(1-t)^{n-1}, and 1n(n−1)!=1n!\frac{1}{n(n-1)!} = \frac{1}{n!} gives Lagrange’s form. p=1p = 1: the weight is 11, nothing is kept, and the first mean value theorem gives Cauchy’s form.

Worked examples and exercises

Exercise Two values of θ\theta ●○○ Hardy: §167

For f(x)=exf(x) = e^x, a=0a = 0, h=1h = 1 and n=2n = 2, the remainder is e−2e - 2. Find the θ\theta in Lagrange’s form and the θ\theta in Cauchy’s form (numerically for the second). Check with the widget.

Hint
Lagrange: e−2=12eθe - 2 = \tfrac12 e^{\theta}. Cauchy: e−2=(1−θ)eθe - 2 = (1-\theta)e^{\theta}.

Solution

Lagrange. eθ=2(e−2)e^\theta = 2(e - 2), so θ=log⁡{2(e−2)}=log⁡1.43656…=0.3623\theta = \log\{2(e-2)\} = \log 1.43656\ldots = 0.3623.

Cauchy. (1−θ)eθ=e−2=0.71828(1-\theta)e^\theta = e - 2 = 0.71828. The left side decreases from 11 to 00 on [0,1][0, 1] (its derivative is −θeθ-\theta e^\theta), so there is exactly one solution; bisection gives θ=0.6095\theta = 0.6095. Same remainder, different points: the θ\theta is not a property of the function but of the form chosen.

Exercise The logarithmic series at x=1x = 1 ●●○ Hardy: §167; cf. §220

Using the integral form of the remainder, show that for 0<h≤10 < h \le 1,

log⁡(1+h)=h−h22+h33−…and in particularlog⁡2=1−12+13−… .\log(1+h) = h - \frac{h^2}{2} + \frac{h^3}{3} - \dots \quad\text{and in particular}\quad \log 2 = 1 - \frac12 + \frac13 - \dots .
Hint
f(n)(x)=(−1)n−1(n−1)!/(1+x)nf^{(n)}(x) = (-1)^{n-1}(n-1)!/(1+x)^n, so Rn=(−1)n−1hn∫01(1−t)n−1(1+th)n dtR_n = (-1)^{n-1}h^n\int_0^1\frac{(1-t)^{n-1}}{(1+th)^n}\,dt.
Hint
For h>0h > 0, (1+th)n≥1(1+th)^n \ge 1.
Solution

The derivatives give f(k)(0)/k!=(−1)k−1/kf^{(k)}(0)/k! = (-1)^{k-1}/k for k≥1k \ge 1, so the Taylor polynomial is h−12h2+⋯+(−1)nhn−1/(n−1)h - \tfrac12h^2 + \dots + (-1)^{n}h^{n-1}/(n-1). By the first hint and the second, for 0<h≤10 < h \le 1,

∣Rn∣≤hn∫01(1−t)n−1 dt=hnn≤1n→0.|R_n| \le h^n\int_0^1(1-t)^{n-1}\,dt = \frac{h^n}{n} \le \frac1n \to 0 .

So the series converges to log⁡(1+h)\log(1+h), including at h=1h = 1. (Lagrange’s form would also work here: ∣Rn∣=hnn(1+θh)n≤hnn|R_n| = \frac{h^n}{n(1+\theta h)^n} \le \frac{h^n}{n}.)

Exercise The logarithmic series for −1<x<0-1 < x < 0 ●●○ Hardy: §167; cf. §168

Show that the series of the previous exercise also represents log⁡(1+h)\log(1+h) when −1<h<0-1 < h < 0, using Cauchy’s form. Why does Lagrange’s form fail to prove this for −1<h<−12-1 < h < -\tfrac12?

Hint
Cauchy’s form: Rn=(−1)n−1hn(1−θ)n−1(1+θh)nR_n = (-1)^{n-1}h^n\dfrac{(1-\theta)^{n-1}}{(1+\theta h)^n}. Write it as (−1)n−1hn(1−θ1+θh)n−111+θh(-1)^{n-1}h^n\left(\dfrac{1-\theta}{1+\theta h}\right)^{n-1}\dfrac{1}{1+\theta h}.
Hint
For −1<h<0-1 < h < 0, show 1−θ1+θh<1\dfrac{1-\theta}{1+\theta h} < 1, that is, 1−θ<1+θh1 - \theta < 1 + \theta h, that is, −θ<θh-\theta < \theta h.

Solution

Let −1<h<0-1 < h < 0 and put k=∣h∣<1k = |h| < 1. Since 0<θ<10 < \theta < 1, we have θh=−θk>−θ\theta h = -\theta k > -\theta, so 1+θh>1−θ1 + \theta h > 1 - \theta and 1−θ1+θh<1\frac{1-\theta}{1+\theta h} < 1. Also 1+θh>1−k1 + \theta h > 1 - k. Hence

∣Rn∣≤kn⋅1⋅11−k→0,|R_n| \le k^n\cdot 1\cdot\frac{1}{1-k} \to 0,

a bound that does not involve θ\theta. So the series converges to log⁡(1+h)\log(1+h).

Lagrange’s form gives only ∣Rn∣=knn(1−θk)n|R_n| = \dfrac{k^n}{n(1-\theta k)^n}, and all we know is 1−θk>1−k1 - \theta k > 1 - k, so the best bound is 1n(k1−k)n\dfrac{1}{n}\left(\dfrac{k}{1-k}\right)^n. When k>12k > \tfrac12, k1−k>1\frac{k}{1-k} > 1, and this bound tends to infinity. The remainder does tend to 00; Lagrange’s form simply cannot see it, because it throws away the factor (1−θ)n−1(1-\theta)^{n-1} that Cauchy’s form keeps.

Exercise Bounds from bounds on the nn-th derivative ●●○ Hardy: Misc. Examples VII, 4

Let FF have continuous derivatives up to order nn on [0,h][0, h], with F(0)=F′(0)=⋯=F(n−1)(0)=0F(0) = F'(0) = \dots = F^{(n-1)}(0) = 0 and A≤F(n)(x)≤BA \le F^{(n)}(x) \le B there. Show that Axnn!≤F(x)≤Bxnn!A\dfrac{x^n}{n!} \le F(x) \le B\dfrac{x^n}{n!} for 0≤x≤h0 \le x \le h, and deduce Taylor’s theorem in Lagrange’s form.

Hint
The integral form with a=0a = 0 and the vanishing derivatives gives F(x)=1(n−1)!∫0x(x−t)n−1F(n)(t) dtF(x) = \dfrac{1}{(n-1)!}\displaystyle\int_0^x(x-t)^{n-1}F^{(n)}(t)\,dt.
Hint
For the deduction, apply the result to F(x)=f(x)−f(0)−xf′(0)−⋯−xn−1(n−1)!f(n−1)(0)F(x) = f(x) - f(0) - xf'(0) - \dots - \frac{x^{n-1}}{(n-1)!}f^{(n-1)}(0), with AA and BB the least and greatest values of f(n)f^{(n)} on [0,h][0, h].

Solution

By the integral form, since all the Taylor terms of FF at 00 vanish, F(x)=1(n−1)!∫0x(x−t)n−1F(n)(t) dtF(x) = \frac{1}{(n-1)!}\int_0^x (x-t)^{n-1}F^{(n)}(t)\,dt. The weight (x−t)n−1(x-t)^{n-1} is ≥0\ge 0 on [0,x][0, x], so by (9) of §165 the integral lies between A∫0x(x−t)n−1dtA\int_0^x(x-t)^{n-1}dt and B∫0x(x−t)n−1dtB\int_0^x(x-t)^{n-1}dt, and ∫0x(x−t)n−1 dt=xn/n\int_0^x (x-t)^{n-1}\,dt = x^n/n. Dividing by (n−1)!(n-1)! gives Axn/n!≤F(x)≤Bxn/n!A x^n/n! \le F(x) \le Bx^n/n!.

For Taylor’s theorem, take FF as in the second hint: its first n−1n-1 derivatives vanish at 00 and F(n)=f(n)F^{(n)} = f^{(n)}. At x=hx = h, F(h)F(h) lies between Ahn/n!A h^n/n! and Bhn/n!B h^n/n!, so F(h)=ηhn/n!F(h) = \eta h^n/n! with A≤η≤BA \le \eta \le B. Since f(n)f^{(n)} is continuous it takes the value η\eta at some ξ=θh\xi = \theta h, and F(h)=hnn!f(n)(θh)F(h) = \frac{h^n}{n!}f^{(n)}(\theta h), which is Lagrange’s form.

Check yourself

Check yourself True or false?

For given ff, aa, hh and nn, the θ\theta in Lagrange’s form and the θ\theta in Cauchy’s form are the same number.

Why

False. Both forms equal the same remainder, but they are different expressions in θ\theta, so they are satisfied at different points. For exe^x with h=1h = 1, n=2n = 2: Lagrange’s θ≈0.362\theta \approx 0.362, Cauchy’s θ≈0.610\theta \approx 0.610. Only for n=1n = 1 do the two forms coincide.

Check yourself Choose one

In the general form Rn=(1−θ)n−pf(n)(a+θh)hnp (n−1)!R_n = \dfrac{(1-\theta)^{n-p}f^{(n)}(a+\theta h)h^n}{p\,(n-1)!}, which value of pp gives Cauchy’s form?

Why

With p=1p = 1 the weight (1−t)p−1(1-t)^{p-1} is 11, the whole integrand is replaced by its value at θ\theta, and the factor (1−θ)n−1(1-\theta)^{n-1} survives in the answer. p=np = n gives Lagrange’s form. p=0p = 0 is not allowed: the weight (1−t)−1(1-t)^{-1} has no finite integral.

Check yourself Choose one

Compared with the proof of Taylor’s theorem by Rolle’s theorem (§150), what extra hypothesis does the integration-by-parts proof use?

Why

The integral ∫01(1−t)n−1f(n)(a+th) dt\int_0^1(1-t)^{n-1}f^{(n)}(a+th)\,dt is defined (in the sense of §161) for continuous integrands, and the step Fn(a)=−∫abFn′F_n(a) = -\int_a^b F_n' needs Fn′F_n' continuous. Rolle’s theorem needs only that f(n)f^{(n)} exists. In return, the integral proof gives an exact formula. It works for h<0h < 0 as well.

Check yourself Enter a number

For f(x)=exf(x) = e^x, a=0a = 0, h=1h = 1 and n=1n = 1, find θ\theta with e−1=eθe - 1 = e^{\theta} (the forms coincide when n=1n = 1).

Why

θ=log⁡(e−1)=log⁡1.71828≈0.5413\theta = \log(e - 1) = \log 1.71828 \approx 0.5413. This is just the mean value theorem f(1)−f(0)=f′(θ)f(1) - f(0) = f'(\theta).

Hardy’s language

This proof of Taylor’s theorem has the advantage of leading to an exact formula for RnR_n, viz. (2), which does not involve an undetermined number θ\theta.

— G. H. Hardy, §167

Summary

Summary
  • If f(n)f^{(n)} is continuous, the remainder after nn terms is exactly Rn=hn(n−1)!∫01(1−t)n−1f(n)(a+th) dtR_n = \frac{h^n}{(n-1)!}\int_0^1(1-t)^{n-1}f^{(n)}(a+th)\,dt.
  • The proof: the error of the expansion about a moving point xx has a telescoping derivative; equivalently, repeated integration by parts.
  • Splitting (1−t)n−1(1-t)^{n-1} into a weight (1−t)p−1(1-t)^{p-1} and the rest, the weighted mean value theorem gives a form for each pp: p=np = n is Lagrange’s, p=1p = 1 is Cauchy’s.
  • Cauchy’s form keeps the factor (1−θ)n−1(1-\theta)^{n-1}, which beats a large 1/(1+θh)n1/(1+\theta h)^n when hh is negative; this is why it succeeds where Lagrange’s fails.