Appendices · Appendices

Double limit problems

Appendix III 40 min

Appendix III: integrating or differentiating a series term by term, or letting x→ξx \to \xi inside a sum, swaps two limits; the order can matter.

Why this matters

Several times in Chapters IX and X Hardy does something that looks like routine algebra: he integrates the series 1−t+t2−…1 - t + t^2 - \dots term by term to get the series for log⁡(1+x)\log(1 + x) (§220); he differentiates the exponential series term by term (§223); he lets x→ξx \to \xi in ∑xn/n!\sum x^n/n! term by term to see that exp⁡\exp is continuous. Each time he gives a special proof that the step is allowed. Appendix III explains why special proofs were needed: each step interchanges two limiting processes, and that is not always legitimate. This lesson shows what goes wrong, how to see it, and what extra condition (uniform convergence) makes it right.

(The appendix has no section numbers. The first example Hardy discusses comes from §220.)

The idea

Finite sums commute with everything: the derivative of a sum of finitely many functions is the sum of the derivatives, and the same is true of integrals and limits. An infinite series is a limit of finite sums, so “differentiate the series term by term” really says

ddx lim⁡N→∞∑n=1Nun(x)  =  lim⁡N→∞ ddx∑n=1Nun(x),\frac{d}{dx}\,\lim_{N\to\infty} \sum_{n=1}^N u_n(x) \;=\; \lim_{N\to\infty}\, \frac{d}{dx}\sum_{n=1}^N u_n(x),

and a derivative is itself a limit. The question is always the same: given numbers a(m,n)a(m, n) depending on two indices, is

lim⁡m→∞(lim⁡n→∞a(m,n))  =  lim⁡n→∞(lim⁡m→∞a(m,n)) ?\lim_{m\to\infty}\Bigl(\lim_{n\to\infty} a(m, n)\Bigr) \;=\; \lim_{n\to\infty}\Bigl(\lim_{m\to\infty} a(m, n)\Bigr)\,?

Write the numbers a(m,n)a(m, n) in a table, row mm and column nn. The left side goes along each row to the end, writes the row limits in a final column, and then goes down that column. The right side goes down each column first and then along the final row. Both are trying to reach the far corner of an infinite table, by different routes.

Explore Two routes to the corner

Try this. Start with Hardy's example (x−y)/(x+y)(x - y)/(x + y), where x=1/mx = 1/m and y=1/ny = 1/n so that x,y→0x, y \to 0 as m,n→∞m, n \to \infty. Read along a row towards the last column, and down a column towards the last row. Then switch to the other examples, and turn on the diagonal outline for mn/(m2+n2)mn/(m^2 + n^2).

a(m,n)=x−yx+y=n−mn+ma(m,n) = \frac{x - y}{x + y} = \frac{n - m}{n + m}
m \ n12510100100010000n → ∞
100.33330.66670.81820.98020.9980.99981
2−0.333300.42860.66670.96080.9960.99961
5−0.6667−0.428600.33330.90480.990.9991
10−0.8182−0.6667−0.333300.81820.98020.9981
100−0.9802−0.9608−0.9048−0.818200.81820.98021
1000−0.998−0.996−0.99−0.9802−0.818200.81821
10000−0.9998−0.9996−0.999−0.998−0.9802−0.818201
m → ∞−1−1−1−1−1−1−1see below
Rows first, then down the last column: lim⁡m→∞lim⁡n→∞a=1\lim_{m\to\infty}\lim_{n\to\infty} a = 1. Columns first, then along the last row: lim⁡n→∞lim⁡m→∞a=−1\lim_{n\to\infty}\lim_{m\to\infty} a = −1. The two orders give different answers.
Hardy’s example (3): letting y → 0 first gives 1, letting x → 0 first gives −1.
What to notice

In the first two examples the last column and the last row are different constants: the two routes end at different corners. The top-right of the table takes the colour of one limit and the bottom-left that of the other (blue 11 against ochre −1-1 in the first example, blue 11 against white 00 in the second), and no single value is approached when both indices are large. In the series for log⁡2\log 2 both routes agree. For mn/(m2+n2)mn/(m^2 + n^2) the routes agree (both give 00) but the diagonal stays at 12\tfrac12: agreement of the two repeated limits is not the same as the table settling down.

The second example is Hardy’s series ∑(xn−1−xn)\sum (x^{n-1} - x^n), whose partial sums telescope to 1−xN1 - x^N. For each fixed xx with 0≤x<10 \le x < 1 the sum is 11, so the sum tends to 11 as x→1x \to 1. But each term tends to 00 as x→1x \to 1, and the series of limits is 0+0+⋯=00 + 0 + \dots = 0. The table shows the same thing as the graphs of the partial sums.

Explore Where the convergence fails

Try this. With the first series, push NN up and watch the graph of SN(x)=1−xNS_N(x) = 1 - x^N. Move xx close to 11 and ask how large NN must be before SN(x)S_N(x) is within 0.010.01 of the sum. Then switch to the logarithmic series and repeat.

SN(x)=∑n=1N(xn−1−xn)=1−xNS_N(x) = \sum_{n=1}^{N} (x^{n-1} - x^n) = 1 - x^N
xS
  • partial sum S_N, N = 5
  • sum of the series: 1 for x < 1, 0 at x = 1
  • your x
  • S_N(1)
Series
At x=0.9x = 0.9: S5(x)=0.41S_{5}(x) = 0.41, sum 11. At x=1x = 1: S5(1)=0S_{5}(1) = 0. Every partial sum is continuous with SN(1)=0S_N(1) = 0, but the sum jumps to 1 just left of 1. The bend in SNS_N moves towards x=1x = 1 as N grows and never goes away.
What to notice

For the first series the drop from 11 to 00 never disappears; it only slides towards x=1x = 1. However large NN is, there are xx (close to 11) where SN(x)S_N(x) is far from 11: the convergence is not uniform, and that is what lets the two orders of limits disagree. For the logarithmic series the error is at most 1/(N+1)1/(N+1) at every xx in [0,1][0, 1] at once, and the limits can be interchanged.

Precise statements

Definition Repeated limits and the double limit

Let a(m,n)a(m, n) be defined for all positive integers m,nm, n.

  • The repeated limits are lim⁡m→∞lim⁡n→∞a(m,n)\lim_{m\to\infty} \lim_{n\to\infty} a(m, n) and lim⁡n→∞lim⁡m→∞a(m,n)\lim_{n\to\infty} \lim_{m\to\infty} a(m, n), when the inner and outer limits exist.
  • a(m,n)a(m, n) tends to the double limit ll if for every ε>0\varepsilon > 0 there is an NN such that ∣a(m,n)−l∣<ε|a(m, n) - l| < \varepsilon whenever m≥Nm \ge N and n≥Nn \ge N.
  • a(m,n)→bma(m, n) \to b_m as n→∞n \to \infty uniformly in mm if for every ε>0\varepsilon > 0 there is an NN, the same for every mm, such that ∣a(m,n)−bm∣<ε|a(m, n) - b_m| < \varepsilon for all n≥Nn \ge N and all mm.

Hardy’s examples: with x=1/mx = 1/m and y=1/ny = 1/n,

lim⁡x→0lim⁡y→0x−yx+y=1,lim⁡y→0lim⁡x→0x−yx+y=−1;\lim_{x\to0}\lim_{y\to0}\frac{x - y}{x + y} = 1, \qquad \lim_{y\to0}\lim_{x\to0}\frac{x - y}{x + y} = -1;

and, as x→1x \to 1 through values less than 11,

lim⁡x→1∑n=1∞(xn−1−xn)=1,∑n=1∞lim⁡x→1(xn−1−xn)=0.\lim_{x\to1}\sum_{n=1}^\infty (x^{n-1} - x^n) = 1, \qquad \sum_{n=1}^\infty \lim_{x\to1}(x^{n-1} - x^n) = 0.

Hardy draws no general theorem: he only observes that two limit operations usually commute in the cases that arise naturally, and that each case needs proof. The standard modern tool is the following theorem, which gives a sufficient condition.

Theorem Interchanging limits under uniform convergence

Suppose that (i) for each mm, a(m,n)→bma(m, n) \to b_m as n→∞n \to \infty, uniformly in mm; and (ii) for each nn, a(m,n)→cna(m, n) \to c_n as m→∞m \to \infty. Then cnc_n tends to a limit ll as n→∞n \to \infty, and bm→lb_m \to l as m→∞m \to \infty. In other words, both repeated limits exist and are equal.

Proofs

Proof Interchanging limits under uniform convergence
Strategy

First show that the numbers cnc_n converge, using Cauchy’s principle: uniformity makes different columns close to each other uniformly in mm, and this closeness survives letting m→∞m \to \infty. Then estimate ∣bm−l∣|b_m - l| by going round three sides of a rectangle: from bmb_m to a(m,N)a(m, N) (uniformity), from a(m,N)a(m, N) to cNc_N (condition (ii)), and from cNc_N to ll.

Columns far out are uniformly close

Let ε>0\varepsilon > 0 and take NN from (i). If n,n′≥Nn, n' \ge N then for every mm, ∣a(m,n)−a(m,n′)∣≤∣a(m,n)−bm∣+∣bm−a(m,n′)∣<2ε|a(m, n) - a(m, n')| \le |a(m, n) - b_m| + |b_m - a(m, n')| < 2\varepsilon.

So the cnc_n form a Cauchy sequence

Let m→∞m \to \infty in the last inequality, using (ii): ∣cn−cn′∣≤2ε|c_n - c_{n'}| \le 2\varepsilon for n,n′≥Nn, n' \ge N. By Cauchy’s principle (§84), cnc_n tends to a limit ll.

Choose one good column

Given ε>0\varepsilon > 0, choose NN so that ∣a(m,N)−bm∣<ε|a(m, N) - b_m| < \varepsilon for every mm (by (i)) and also ∣cN−l∣<ε|c_N - l| < \varepsilon (possible since cn→lc_n \to l; take NN large enough for both).

Go round the rectangle

With NN fixed, (ii) gives m0m_0 such that ∣a(m,N)−cN∣<ε|a(m, N) - c_N| < \varepsilon for m≥m0m \ge m_0. For such mm,

∣bm−l∣≤∣bm−a(m,N)∣+∣a(m,N)−cN∣+∣cN−l∣<3ε.|b_m - l| \le |b_m - a(m, N)| + |a(m, N) - c_N| + |c_N - l| < 3\varepsilon.

So bm→lb_m \to l.

Where uniformity was used

The first estimate had to hold for every mm with one and the same column NN, because mm was sent to infinity afterwards. Without uniformity, the column that makes a(m,N)a(m, N) close to bmb_m may drift off to infinity as mm grows, which is what happens in 1−xN1 - x^N.

Hardy’s §220 revisited. Put x=1−1/mx = 1 - 1/m and let a(m,N)=∑n=1N(−1)n−1xn/na(m, N) = \sum_{n=1}^N (-1)^{n-1}x^n/n. For 0≤x≤10 \le x \le 1 the series is alternating with terms decreasing to 00, so its sum differs from the NN-th partial sum by at most the next term: ∣a(m,N)−log⁡(1+x)∣≤xN+1/(N+1)≤1/(N+1)|a(m, N) - \log(1 + x)| \le x^{N+1}/(N+1) \le 1/(N+1). That bound does not depend on mm: the convergence is uniform. The theorem then gives lim⁡x→1log⁡(1+x)=∑(−1)n−1/n\lim_{x \to 1} \log(1 + x) = \sum (-1)^{n-1}/n, that is log⁡2=1−12+13−…\log 2 = 1 - \tfrac12 + \tfrac13 - \dots, which is what Hardy proved directly. For 1−xN1 - x^N the theorem cannot be applied, and indeed the conclusion is false: for every NN the choice x=2−1/Nx = 2^{-1/N} gives 1−xN=121 - x^N = \tfrac12, so no single NN works for all x<1x < 1.

Exercises

Our own exercises on the ideas of the appendix.

Exercise Compute the repeated limits ●○○

Find both repeated limits of a(m,n)=m2m2+n2a(m, n) = \dfrac{m^2}{m^2 + n^2}. Does the double limit exist?

Solution

For fixed mm, a(m,n)→0a(m, n) \to 0 as n→∞n \to \infty, so lim⁡mlim⁡na=0\lim_m\lim_n a = 0. For fixed nn, a(m,n)=1/(1+n2/m2)→1a(m, n) = 1/(1 + n^2/m^2) \to 1 as m→∞m \to \infty, so lim⁡nlim⁡ma=1\lim_n\lim_m a = 1. The double limit does not exist: if it were ll, then by Exercise 5 below both repeated limits would equal ll. (Directly: a(n,n)=12a(n, n) = \tfrac12 while a(n,n2)→0a(n, n^2) \to 0.)

Exercise A continuous series with a discontinuous sum ●●○

Let u0(x)=x2u_0(x) = x^2 and un(x)=x2(1+x2)nu_n(x) = \dfrac{x^2}{(1 + x^2)^n} for n≥1n \ge 1, so that every term is continuous. Find ∑n=0∞un(x)\sum_{n=0}^\infty u_n(x) for every real xx, and show that the sum is discontinuous at x=0x = 0. Which two limits fail to commute?

Hint
For x≠0x \ne 0 the series is geometric with ratio 1/(1+x2)1/(1 + x^2).

Solution

For x=0x = 0 every term is 00, so the sum is 00. For x≠0x \ne 0, ∑n≥0x2(1+x2)−n=x2⋅11−1/(1+x2)=x2⋅1+x2x2=1+x2\sum_{n\ge0} x^2 (1 + x^2)^{-n} = x^2 \cdot \dfrac{1}{1 - 1/(1 + x^2)} = x^2 \cdot \dfrac{1 + x^2}{x^2} = 1 + x^2. So the sum is 1+x21 + x^2 for x≠0x \ne 0 and 00 at x=0x = 0: it tends to 11 as x→0x \to 0 but equals 00 there. The limits “let x→0x \to 0” and “sum to infinity” do not commute: lim⁡x→0∑un(x)=1\lim_{x\to0}\sum u_n(x) = 1, ∑lim⁡x→0un(x)=0\sum \lim_{x\to0} u_n(x) = 0. As in 1−xN1 - x^N, the convergence is not uniform near x=0x = 0.

Exercise The limit of the integrals is not the integral of the limit ●●○

Let fn(x)=nx(1−x2)nf_n(x) = n x (1 - x^2)^n on [0,1][0, 1]. Show that fn(x)→0f_n(x) \to 0 for every xx in [0,1][0, 1], but ∫01fn(x) dx→12\int_0^1 f_n(x)\,dx \to \tfrac12.

Hint
For 0<x<10 < x < 1, nrn→0n r^n \to 0 when 0<r<10 < r < 1 (§72). For the integral substitute u=1−x2u = 1 - x^2.

Solution

At x=0x = 0 and x=1x = 1, fn(x)=0f_n(x) = 0. For 0<x<10 < x < 1, r=1−x2r = 1 - x^2 lies in (0,1)(0, 1) and fn(x)=x⋅nrn→0f_n(x) = x \cdot n r^n \to 0. With u=1−x2u = 1 - x^2, du=−2x dxdu = -2x\,dx:

∫01nx(1−x2)n dx=n2∫01un du=n2(n+1)→12.\int_0^1 n x (1 - x^2)^n\,dx = \frac n2 \int_0^1 u^n\,du = \frac{n}{2(n+1)} \to \frac12.

So lim⁡n∫01fn=12\lim_n \int_0^1 f_n = \tfrac12 but ∫01lim⁡nfn=0\int_0^1 \lim_n f_n = 0. The graphs are humps of height about n/2e\sqrt{n/2e} that slide towards x=0x = 0, carrying area close to 12\tfrac12 with them.

Exercise The derivative of the limit is not the limit of the derivatives ●●○

Let fn(x)=sin⁡nxnf_n(x) = \dfrac{\sin nx}{n}. Show that fn(x)→0f_n(x) \to 0 uniformly in xx, but that fn′(0)f_n'(0) does not tend to the derivative of the limit at 00, and that fn′(π)f_n'(\pi) does not tend to anything.

Solution

∣fn(x)∣≤1/n|f_n(x)| \le 1/n for every xx, so fn→0f_n \to 0 uniformly, and the limit function 00 has derivative 00. But fn′(x)=cos⁡nxf_n'(x) = \cos nx, so fn′(0)=1f_n'(0) = 1 for all nn, which tends to 1≠01 \ne 0; and fn′(π)=cos⁡nπ=(−1)nf_n'(\pi) = \cos n\pi = (-1)^n oscillates. Uniform convergence of the functions is not enough for differentiation term by term: one needs control of the derivatives (for example, uniform convergence of fn′f_n').

Exercise A double limit forces the repeated limits ●●○

Suppose a(m,n)→la(m, n) \to l as a double limit, and that for each mm the row limit bm=lim⁡n→∞a(m,n)b_m = \lim_{n\to\infty} a(m, n) exists. Prove that bm→lb_m \to l.

Hint
Take NN from the definition of the double limit, fix m≥Nm \ge N, and let n→∞n \to \infty in ∣a(m,n)−l∣<ε|a(m, n) - l| < \varepsilon.

Solution

Let ε>0\varepsilon > 0 and choose NN with ∣a(m,n)−l∣<ε|a(m, n) - l| < \varepsilon for all m,n≥Nm, n \ge N. Fix m≥Nm \ge N. For every n≥Nn \ge N we have l−ε<a(m,n)<l+εl - \varepsilon < a(m, n) < l + \varepsilon; letting n→∞n \to \infty gives l−ε≤bm≤l+εl - \varepsilon \le b_m \le l + \varepsilon. So ∣bm−l∣≤ε|b_m - l| \le \varepsilon for all m≥Nm \ge N, and bm→lb_m \to l. In particular, if both families of single limits exist, a double limit forces both repeated limits to equal it; this is why (n−m)/(n+m)(n - m)/(n + m) and m2/(m2+n2)m^2/(m^2 + n^2) have no double limit.

Exercise Operations that never commute ●○○ Hardy: Appendix III (2)

Show that taking the real part and multiplying by ii commute only for z=0z = 0: Re⁡(iz)=iRe⁡z\operatorname{Re}(iz) = i\operatorname{Re} z only when z=0z = 0. Then show that “take the limit as n→∞n \to \infty” and “multiply by 33” always commute.

Solution

With z=x+iyz = x + iy: Re⁡(iz)=Re⁡(ix−y)=−y\operatorname{Re}(iz) = \operatorname{Re}(ix - y) = -y, a real number, while iRe⁡z=ixi \operatorname{Re} z = ix is purely imaginary. They are equal only if −y=0-y = 0 and x=0x = 0, i.e. z=0z = 0. On the other hand, if sn→ss_n \to s then 3sn→3s3s_n \to 3s (the limit of a product, §63), so lim⁡3sn=3lim⁡sn\lim 3s_n = 3 \lim s_n whenever either side exists. Multiplication by a constant is continuous, and continuous operations commute with limits.

Check yourself

Check yourself Enter a number

For a(m,n)=m2m2+n2a(m, n) = \dfrac{m^2}{m^2 + n^2}, what is lim⁡n→∞lim⁡m→∞a(m,n)\lim_{n\to\infty}\lim_{m\to\infty} a(m, n)?

Why

The inner limit is over mm, with nn fixed: m2/(m2+n2)→1m^2/(m^2 + n^2) \to 1. The outer limit of the constant 11 is 11. The other order gives 00. Read the subscripts from the inside out.

Check yourself True or false?

If both repeated limits of a(m,n)a(m, n) exist and are equal to ll, then a(m,n)→la(m, n) \to l as a double limit.

Why

False. For a(m,n)=mn/(m2+n2)a(m, n) = mn/(m^2 + n^2) both repeated limits are 00, but a(n,n)=12a(n, n) = \tfrac12 for every nn, so ∣a(m,n)∣<14|a(m, n)| < \tfrac14 fails for m=nm = n however large. Check the diagonal in the grid widget.

Check yourself Choose one

“The sum of a convergent series of continuous functions is continuous at ξ\xi” claims that which two operations commute?

Why

Continuity at ξ\xi of s(x)=∑un(x)s(x) = \sum u_n(x) means lim⁡x→ξ∑un(x)=∑un(ξ)=∑lim⁡x→ξun(x)\lim_{x\to\xi}\sum u_n(x) = \sum u_n(\xi) = \sum \lim_{x\to\xi} u_n(x): the limit as x→ξx \to \xi passes through the infinite sum. Hardy’s series ∑(xn−1−xn)\sum(x^{n-1} - x^n) at ξ=1\xi = 1 shows it can fail. Multiplying by xx commutes with summation always.

Check yourself Choose one

Why is the convergence of 1−xN1 - x^N to 11 on 0≤x<10 \le x < 1 not uniform?

Why

Uniform convergence needs one NN that works for every xx at once. For each NN, some xx close to 11 still has an error of 12\tfrac12. The point x=1x = 1 is not in the interval at all, and xN→0x^N \to 0 does hold for each fixed x<1x < 1; the failure is in the “same NN for all xx”.

Hardy’s language

In practice, a result obtained by assuming that two limit operations are commutative is probably true; at any rate it gives a valuable suggestion of the answer to the problem under consideration.

— G. H. Hardy, Appendix III

Summary

Summary
  • Term-by-term differentiation or integration, and passing a limit through an infinite sum, all interchange two limits: lim⁡mlim⁡na(m,n)\lim_m\lim_n a(m, n) against lim⁡nlim⁡ma(m,n)\lim_n\lim_m a(m, n).
  • The two repeated limits can differ, as for (x−y)/(x+y)(x - y)/(x + y) or ∑(xn−1−xn)\sum(x^{n-1} - x^n) at x=1x = 1; and they can agree without a double limit existing, as for mn/(m2+n2)mn/(m^2 + n^2).
  • If one of the inner limits is uniform, the interchange is legitimate (proof: Cauchy’s principle plus a three-term estimate).
  • The partial sums 1−xN1 - x^N converge, but not uniformly near x=1x = 1; the logarithmic series converges uniformly on [0,1][0, 1], which justifies log⁡2=1−12+13−…\log 2 = 1 - \tfrac12 + \tfrac13 - \dots.