Several times in Chapters IX and X Hardy does something that looks like routine algebra: he integrates the series 1−t+t2−… term by term to get the series for log(1+x) (§220); he differentiates the exponential series term by term (§223); he lets x→ξ in ∑xn/n! term by term to see that exp is continuous. Each time he gives a special proof that the step is allowed. Appendix III explains why special proofs were needed: each step interchanges two limiting processes, and that is not always legitimate. This lesson shows what goes wrong, how to see it, and what extra condition (uniform convergence) makes it right.
(The appendix has no section numbers. The first example Hardy discusses comes from §220.)
The idea
Finite sums commute with everything: the derivative of a sum of finitely many functions is the sum of the derivatives, and the same is true of integrals and limits. An infinite series is a limit of finite sums, so “differentiate the series term by term” really says
dxdN→∞limn=1∑Nun(x)=N→∞limdxdn=1∑Nun(x),
and a derivative is itself a limit. The question is always the same: given numbers a(m,n) depending on two indices, is
m→∞lim(n→∞lima(m,n))=n→∞lim(m→∞lima(m,n))?
Write the numbers a(m,n) in a table, row m and column n. The left side goes along each row to the end, writes the row limits in a final column, and then goes down that column. The right side goes down each column first and then along the final row. Both are trying to reach the far corner of an infinite table, by different routes.
ExploreTwo routes to the corner
Try this.Start with Hardy's example (x−y)/(x+y), where x=1/m and y=1/n so that x,y→0 as m,n→∞. Read along a row towards the last column, and down a column towards the last row. Then switch to the other examples, and turn on the diagonal outline for mn/(m2+n2).
a(m,n)=x+yx−y=n+mn−m
m \ n
1
2
5
10
100
1000
10000
n → ∞
1
0
0.3333
0.6667
0.8182
0.9802
0.998
0.9998
1
2
−0.3333
0
0.4286
0.6667
0.9608
0.996
0.9996
1
5
−0.6667
−0.4286
0
0.3333
0.9048
0.99
0.999
1
10
−0.8182
−0.6667
−0.3333
0
0.8182
0.9802
0.998
1
100
−0.9802
−0.9608
−0.9048
−0.8182
0
0.8182
0.9802
1
1000
−0.998
−0.996
−0.99
−0.9802
−0.8182
0
0.8182
1
10000
−0.9998
−0.9996
−0.999
−0.998
−0.9802
−0.8182
0
1
m → ∞
−1
−1
−1
−1
−1
−1
−1
see below
Rows first, then down the last column: limm→∞limn→∞a=1. Columns first, then along the last row: limn→∞limm→∞a=−1. The two orders give different answers. Hardy’s example (3): letting y → 0 first gives 1, letting x → 0 first gives −1.
What to notice
In the first two examples the last column and the last row are different constants: the two routes end at different corners. The top-right of the table takes the colour of one limit and the bottom-left that of the other (blue 1 against ochre −1 in the first example, blue 1 against white 0 in the second), and no single value is approached when both indices are large. In the series for log2 both routes agree. For mn/(m2+n2) the routes agree (both give 0) but the diagonal stays at 21: agreement of the two repeated limits is not the same as the table settling down.
The second example is Hardy’s series ∑(xn−1−xn), whose partial sums telescope to 1−xN. For each fixed x with 0≤x<1 the sum is 1, so the sum tends to 1 as x→1. But each term tends to 0 as x→1, and the series of limits is 0+0+⋯=0. The table shows the same thing as the graphs of the partial sums.
ExploreWhere the convergence fails
Try this.With the first series, push N up and watch the graph of SN(x)=1−xN. Move x close to 1 and ask how large N must be before SN(x) is within 0.01 of the sum. Then switch to the logarithmic series and repeat.
SN(x)=∑n=1N(xn−1−xn)=1−xN
xS
partial sum S_N, N = 5
sum of the series: 1 for x < 1, 0 at x = 1
your x
S_N(1)
Series
At x=0.9: S5(x)=0.41, sum 1. At x=1: S5(1)=0. Every partial sum is continuous with SN(1)=0, but the sum jumps to 1 just left of 1. The bend in SN moves towards x=1 as N grows and never goes away.
What to notice
For the first series the drop from 1 to 0 never disappears; it only slides towards x=1. However large N is, there are x (close to 1) where SN(x) is far from 1: the convergence is not uniform, and that is what lets the two orders of limits disagree. For the logarithmic series the error is at most 1/(N+1) at every x in [0,1] at once, and the limits can be interchanged.
Precise statements
DefinitionRepeated limits and the double limit
Let a(m,n) be defined for all positive integers m,n.
The repeated limits are limm→∞limn→∞a(m,n) and limn→∞limm→∞a(m,n), when the inner and outer limits exist.
a(m,n) tends to the double limitl if for every ε>0 there is an N such that ∣a(m,n)−l∣<ε whenever m≥N and n≥N.
a(m,n)→bm as n→∞uniformly in m if for every ε>0 there is an N, the same for every m, such that ∣a(m,n)−bm∣<ε for all n≥N and all m.
Hardy draws no general theorem: he only observes that two limit operations usually commute in the cases that arise naturally, and that each case needs proof. The standard modern tool is the following theorem, which gives a sufficient condition.
TheoremInterchanging limits under uniform convergence
Suppose that (i) for each m, a(m,n)→bm as n→∞, uniformly in m; and (ii) for each n, a(m,n)→cn as m→∞. Then cn tends to a limit l as n→∞, and bm→l as m→∞. In other words, both repeated limits exist and are equal.
Proofs
ProofInterchanging limits under uniform convergence
Strategy
First show that the numbers cn converge, using Cauchy’s principle: uniformity makes different columns close to each other uniformly in m, and this closeness survives letting m→∞. Then estimate ∣bm−l∣ by going round three sides of a rectangle: from bm to a(m,N) (uniformity), from a(m,N) to cN (condition (ii)), and from cN to l.
Columns far out are uniformly close
Let ε>0 and take N from (i). If n,n′≥N then for every m,
∣a(m,n)−a(m,n′)∣≤∣a(m,n)−bm∣+∣bm−a(m,n′)∣<2ε.
So the cn form a Cauchy sequence
Let m→∞ in the last inequality, using (ii): ∣cn−cn′∣≤2ε for n,n′≥N. By Cauchy’s principle (§84), cn tends to a limit l.
Choose one good column
Given ε>0, choose N so that ∣a(m,N)−bm∣<ε for everym (by (i)) and also ∣cN−l∣<ε (possible since cn→l; take N large enough for both).
Go round the rectangle
With N fixed, (ii) gives m0 such that ∣a(m,N)−cN∣<ε for m≥m0. For such m,
∣bm−l∣≤∣bm−a(m,N)∣+∣a(m,N)−cN∣+∣cN−l∣<3ε.
So bm→l.
Where uniformity was used
The first estimate had to hold for every m with one and the same column N, because m was sent to infinity afterwards. Without uniformity, the column that makes a(m,N) close to bm may drift off to infinity as m grows, which is what happens in 1−xN.
∎
Hardy’s §220 revisited. Put x=1−1/m and let a(m,N)=∑n=1N(−1)n−1xn/n. For 0≤x≤1 the series is alternating with terms decreasing to 0, so its sum differs from the N-th partial sum by at most the next term: ∣a(m,N)−log(1+x)∣≤xN+1/(N+1)≤1/(N+1). That bound does not depend on m: the convergence is uniform. The theorem then gives limx→1log(1+x)=∑(−1)n−1/n, that is log2=1−21+31−…, which is what Hardy proved directly. For 1−xN the theorem cannot be applied, and indeed the conclusion is false: for every N the choice x=2−1/N gives 1−xN=21, so no single N works for all x<1.
Exercises
Our own exercises on the ideas of the appendix.
ExerciseCompute the repeated limits●○○
Find both repeated limits of a(m,n)=m2+n2m2. Does the double limit exist?
Solution
For fixed m, a(m,n)→0 as n→∞, so limmlimna=0. For fixed n, a(m,n)=1/(1+n2/m2)→1 as m→∞, so limnlimma=1. The double limit does not exist: if it were l, then by Exercise 5 below both repeated limits would equal l. (Directly: a(n,n)=21 while a(n,n2)→0.)
ExerciseA continuous series with a discontinuous sum●●○
Let u0(x)=x2 and un(x)=(1+x2)nx2 for n≥1, so that every term is continuous. Find ∑n=0∞un(x) for every real x, and show that the sum is discontinuous at x=0. Which two limits fail to commute?
Hint
For x=0 the series is geometric with ratio 1/(1+x2).
Solution
For x=0 every term is 0, so the sum is 0. For x=0, ∑n≥0x2(1+x2)−n=x2⋅1−1/(1+x2)1=x2⋅x21+x2=1+x2. So the sum is 1+x2 for x=0 and 0 at x=0: it tends to 1 as x→0 but equals 0 there. The limits “let x→0” and “sum to infinity” do not commute: limx→0∑un(x)=1, ∑limx→0un(x)=0. As in 1−xN, the convergence is not uniform near x=0.
ExerciseThe limit of the integrals is not the integral of the limit●●○
Let fn(x)=nx(1−x2)n on [0,1]. Show that fn(x)→0 for every x in [0,1], but ∫01fn(x)dx→21.
Hint
For 0<x<1, nrn→0 when 0<r<1 (§72). For the integral substitute u=1−x2.
Solution
At x=0 and x=1, fn(x)=0. For 0<x<1, r=1−x2 lies in (0,1) and fn(x)=x⋅nrn→0. With u=1−x2, du=−2xdx:
∫01nx(1−x2)ndx=2n∫01undu=2(n+1)n→21.
So limn∫01fn=21 but ∫01limnfn=0. The graphs are humps of height about n/2e that slide towards x=0, carrying area close to 21 with them.
ExerciseThe derivative of the limit is not the limit of the derivatives●●○
Let fn(x)=nsinnx. Show that fn(x)→0 uniformly in x, but that fn′(0) does not tend to the derivative of the limit at 0, and that fn′(π) does not tend to anything.
Solution
∣fn(x)∣≤1/n for every x, so fn→0 uniformly, and the limit function 0 has derivative 0. But fn′(x)=cosnx, so fn′(0)=1 for all n, which tends to 1=0; and fn′(π)=cosnπ=(−1)n oscillates. Uniform convergence of the functions is not enough for differentiation term by term: one needs control of the derivatives (for example, uniform convergence of fn′).
ExerciseA double limit forces the repeated limits●●○
Suppose a(m,n)→l as a double limit, and that for each m the row limit bm=limn→∞a(m,n) exists. Prove that bm→l.
Hint
Take N from the definition of the double limit, fix m≥N, and let n→∞ in ∣a(m,n)−l∣<ε.
Solution
Let ε>0 and choose N with ∣a(m,n)−l∣<ε for all m,n≥N. Fix m≥N. For every n≥N we have l−ε<a(m,n)<l+ε; letting n→∞ gives l−ε≤bm≤l+ε. So ∣bm−l∣≤ε for all m≥N, and bm→l. In particular, if both families of single limits exist, a double limit forces both repeated limits to equal it; this is why (n−m)/(n+m) and m2/(m2+n2) have no double limit.
ExerciseOperations that never commute●○○Hardy: Appendix III (2)
Show that taking the real part and multiplying by i commute only for z=0: Re(iz)=iRez only when z=0. Then show that “take the limit as n→∞” and “multiply by 3” always commute.
Solution
With z=x+iy: Re(iz)=Re(ix−y)=−y, a real number, while iRez=ix is purely imaginary. They are equal only if −y=0 and x=0, i.e. z=0. On the other hand, if sn→s then 3sn→3s (the limit of a product, §63), so lim3sn=3limsn whenever either side exists. Multiplication by a constant is continuous, and continuous operations commute with limits.
Check yourself
Check yourselfEnter a number
For a(m,n)=m2+n2m2, what is limn→∞limm→∞a(m,n)?
Why
The inner limit is over m, with n fixed: m2/(m2+n2)→1. The outer limit of the constant 1 is 1. The other order gives 0. Read the subscripts from the inside out.
Check yourselfTrue or false?
If both repeated limits of a(m,n) exist and are equal to l, then a(m,n)→l as a double limit.
Why
False. For a(m,n)=mn/(m2+n2) both repeated limits are 0, but a(n,n)=21 for every n, so ∣a(m,n)∣<41 fails for m=n however large. Check the diagonal in the grid widget.
Check yourselfChoose one
“The sum of a convergent series of continuous functions is continuous at ξ” claims that which two operations commute?
Why
Continuity at ξ of s(x)=∑un(x) means limx→ξ∑un(x)=∑un(ξ)=∑limx→ξun(x): the limit as x→ξ passes through the infinite sum. Hardy’s series ∑(xn−1−xn) at ξ=1 shows it can fail. Multiplying by x commutes with summation always.
Check yourselfChoose one
Why is the convergence of 1−xN to 1 on 0≤x<1 not uniform?
Why
Uniform convergence needs one N that works for every x at once. For each N, some x close to 1 still has an error of 21. The point x=1 is not in the interval at all, and xN→0 does hold for each fixed x<1; the failure is in the “same N for all x”.
Hardy’s language
In practice, a result obtained by assuming that two limit operations are commutative is probably true; at any rate it gives a valuable suggestion of the answer to the problem under consideration.
— G. H. Hardy, Appendix III
Summary
Summary
Term-by-term differentiation or integration, and passing a limit through an infinite sum, all interchange two limits: limmlimna(m,n) against limnlimma(m,n).
The two repeated limits can differ, as for (x−y)/(x+y) or ∑(xn−1−xn) at x=1; and they can agree without a double limit existing, as for mn/(m2+n2).
If one of the inner limits is uniform, the interchange is legitimate (proof: Cauchy’s principle plus a three-term estimate).
The partial sums 1−xN converge, but not uniformly near x=1; the logarithmic series converges uniformly on [0,1], which justifies log2=1−21+31−….