Chapter VIII · The convergence of infinite series and infinite integrals

Multiplying series of positive terms

§177 25 min

Why two convergent positive series can be multiplied term by term, collecting the products along diagonals, to give the product of their sums.

Why this matters

To multiply two polynomials we multiply every term of one by every term of the other and collect the terms of equal degree. Power series are “polynomials of infinite degree”, and we will want to multiply them in the same way: for example, to see that exey=ex+ye^x e^y = e^{x+y} from the exponential series in Chapter IX. §177 justifies this for series of positive terms. It is a direct application of Dirichlet’s theorem, and it is the model for the general case in §202.

The idea

Let ∑um\sum u_m and ∑vn\sum v_n be convergent series of positive terms with sums ss and tt. The product stst ought to be “the sum of all the products umvnu_m v_n”. There are infinitely many of these products, arranged naturally in a square array: column mm, row nn. The question is in which order to add them up.

Two orders are natural.

  • By squares. First u0v0u_0v_0; then the three products with largest index 11; then the five with largest index 22; and so on. After k+1k+1 groups we have added exactly the products with m,n≤km, n \le k, whose sum is (u0+⋯+uk)(v0+⋯+vk)(u_0 + \cdots + u_k)(v_0 + \cdots + v_k). This tends to stst, by the theorem on the limit of a product.
  • By diagonals. First u0v0u_0 v_0; then u1v0+u0v1u_1v_0 + u_0v_1; then u2v0+u1v1+u0v2u_2v_0 + u_1v_1 + u_0v_2; and so on. The kk-th group, wk=∑m+n=kumvnw_k = \sum_{m+n=k} u_m v_n, is what you get by collecting “terms of degree kk” when multiplying power series. This is the order we actually want.
Explore Adding an infinite array of products

Try this. Step kk from 00 to 77 with the diagonals, then switch to squares and compare. Back on diagonals, turn on the trap and move kk again.

The array umvn, um=2−m, vn=3−nu_m v_n,\ u_m = 2^{-m},\ v_n = 3^{-n}m=0m=1m=2m=3m=4m=5m=6m=701234567u0·v0 = 1u1·v0 = 0.5u2·v0 = 0.25u3·v0 = 0.125u4·v0 = 0.0625u5·v0 = 0.03125u6·v0 = 0.01563u7·v0 = 0.007813u0·v1 = 0.3333u1·v1 = 0.1667u2·v1 = 0.08333u3·v1 = 0.04167u4·v1 = 0.02083u5·v1 = 0.01042u6·v1 = 0.005208u7·v1 = 0.002604u0·v2 = 0.1111u1·v2 = 0.05556u2·v2 = 0.02778u3·v2 = 0.01389u4·v2 = 0.006944u5·v2 = 0.003472u6·v2 = 0.001736u7·v2 = 0.0008681u0·v3 = 0.03704u1·v3 = 0.01852u2·v3 = 0.009259u3·v3 = 0.00463u4·v3 = 0.002315u5·v3 = 0.001157u6·v3 = 0.0005787u7·v3 = 0.0002894u0·v4 = 0.01235u1·v4 = 0.006173u2·v4 = 0.003086u3·v4 = 0.001543u4·v4 = 0.0007716u5·v4 = 0.0003858u6·v4 = 0.0001929u7·v4 = 9.645 × 10⁻⁵u0·v5 = 0.004115u1·v5 = 0.002058u2·v5 = 0.001029u3·v5 = 0.0005144u4·v5 = 0.0002572u5·v5 = 0.0001286u6·v5 = 6.43 × 10⁻⁵u7·v5 = 3.215 × 10⁻⁵u0·v6 = 0.001372u1·v6 = 0.0006859u2·v6 = 0.0003429u3·v6 = 0.0001715u4·v6 = 8.573 × 10⁻⁵u5·v6 = 4.287 × 10⁻⁵u6·v6 = 2.143 × 10⁻⁵u7·v6 = 1.072 × 10⁻⁵u0·v7 = 0.0004572u1·v7 = 0.0002286u2·v7 = 0.0001143u3·v7 = 5.716 × 10⁻⁵u4·v7 = 2.858 × 10⁻⁵u5·v7 = 1.429 × 10⁻⁵u6·v7 = 7.144 × 10⁻⁶u7·v7 = 3.572 × 10⁻⁶
  • products already added (darker = larger)
  • the newest diagonal m+n=4m+n=4
Add the products by
The first 5 diagonals add up to w0+⋯+w4=2.82485w_0 + \cdots + w_{4} = 2.82485, where wk=∑m+n=k2−m3−nw_k = \sum_{m+n=k} 2^{-m}3^{-n}. Since every product is positive and the triangle m+n≤4m+n \le 4 contains the square m,n≤2m,n \le 2 and lies inside the square m,n≤4m,n\le 4: 2.52778≤2.82485≤2.894292.52778 \le 2.82485 \le 2.89429. Both bounds tend to st=3st = 3.
What to notice

Adding by squares gives exactly the product of two partial sums. Adding by diagonals gives a triangle, and every triangle sits inside the square of the same kk and contains the square of about half the size. Both squares’ sums tend to stst, so the triangles are squeezed to stst as well. Positivity is what makes “contains” imply “has a larger sum”.

Precise statements

Theorem Multiplication of positive series Hardy: §177

If u0+u1+u2+⋯u_0 + u_1 + u_2 + \cdots and v0+v1+v2+⋯v_0 + v_1 + v_2 + \cdots are convergent series of positive terms, with sums ss and tt, then the series

u0v0+(u1v0+u0v1)+(u2v0+u1v1+u0v2)+⋯u_0v_0 + (u_1v_0 + u_0v_1) + (u_2v_0 + u_1v_1 + u_0v_2) + \cdots

converges, and its sum is stst. The same is true with the brackets removed.

The series ∑wk\sum w_k, with wk=ukv0+uk−1v1+⋯+u0vkw_k = u_kv_0 + u_{k-1}v_1 + \cdots + u_0v_k, is called today the Cauchy product of the two series. More generally, since the terms are positive, Dirichlet’s theorem gives: the products umvnu_m v_n, taken in any order whatever, form a convergent series with sum stst.

Proofs

Hardy’s proof goes through the arrangement by squares and Dirichlet’s theorem.

Proof Multiplication of positive series (Hardy’s proof)
Strategy

Find one arrangement of the products whose sum is obviously stst: the arrangement by squares. Every other arrangement, in particular the one by diagonals, is a rearrangement of it, so Dirichlet’s theorem gives the same sum.

The groups of the square arrangement

The products whose largest index is exactly kk form an L-shaped group: ukv0,ukv1,…,ukvk,uk−1vk,…,u0vku_kv_0, u_kv_1, \dots, u_kv_k, u_{k-1}v_k, \dots, u_0v_k. Their sum is the difference between the products of partial sums for kk and for k−1k-1:

(u0+⋯+uk)(v0+⋯+vk)−(u0+⋯+uk−1)(v0+⋯+vk−1).(u_0 + \cdots + u_k)(v_0 + \cdots + v_k) - (u_0 + \cdots + u_{k-1})(v_0 + \cdots + v_{k-1}).

The grouped series has sum stst

So the sum of the first k+1k+1 groups telescopes to sktks_k t_k, where sks_k and tkt_k are partial sums. Since sk→ss_k \to s and tk→tt_k \to t, we get sktk→sts_k t_k \to st.

Remove the brackets

Inside each group the terms are positive, so the partial sums of the series with brackets removed increase and are squeezed between consecutive values sk−1tk−1s_{k-1}t_{k-1} and sktks_k t_k. Hence the series of single products, in the square order, converges to stst (previous lesson, second exercise).

Rearrange

The diagonal order, with brackets removed, uses each product umvnu_mv_n exactly once: it is a rearrangement of the square order. By Dirichlet’s theorem it converges to the same sum stst. Putting the brackets back (grouping consecutive terms) only picks out a subsequence of the partial sums, which has the same limit.

The widget suggests a second proof that avoids rearrangement altogether.

Proof Multiplication of positive series (by trapping)
Strategy

Squeeze the diagonal partial sums between two products of partial sums, both of which tend to stst.

Triangles between squares

Let Wk=w0+⋯+wkW_k = w_0 + \cdots + w_k, the sum of umvnu_mv_n over the triangle m+n≤km + n \le k. If m,n≤⌊k/2⌋m, n \le \lfloor k/2 \rfloor then m+n≤km + n \le k; and if m+n≤km + n \le k then m,n≤km, n \le k. So the triangle contains the square of side ⌊k/2⌋\lfloor k/2\rfloor and lies inside the square of side kk.

Positivity turns inclusion into inequality

All products are ≥0\ge 0, so a larger set of products has a larger sum: s⌊k/2⌋ t⌊k/2⌋≤Wk≤sktks_{\lfloor k/2\rfloor}\, t_{\lfloor k/2\rfloor} \le W_k \le s_k t_k.

Squeeze

Both bounds tend to stst as k→∞k \to \infty, so Wk→stW_k \to st.

Both proofs use positivity in an essential way. For series with terms of both signs the diagonal sums need not converge at all, even when both factors converge (§202).

Worked examples and exercises

Exercise ●○○ Hardy: Examples LXIX, 1

Let 0≤r<10 \le r < 1. Verify that

1+r2+r+r4+r6+r3+⋯=1+r+r3+r2+r5+r7+⋯=11−r.1 + r^2 + r + r^4 + r^6 + r^3 + \cdots = 1 + r + r^3 + r^2 + r^5 + r^7 + \cdots = \frac{1}{1-r}.
Hint
Which powers of rr occur, and how often?
Solution

Both series contain every power r0,r1,r2,…r^0, r^1, r^2, \dots exactly once: the first in the pattern “two even, one odd” (1,r2,r; r4,r6,r3; …1, r^2, r;\ r^4, r^6, r^3;\ \dots), the second in the pattern “one even, two odd”. So both are rearrangements of ∑rn\sum r^n, a convergent series of positive terms with sum 1/(1−r)1/(1-r), and Dirichlet’s theorem gives the same sum. (When r=0r = 0 all terms but the first vanish and the claim is trivial.)

Exercise Squaring the geometric series ●○○

Use the theorem to show that ∑k=0∞(k+1)rk=1(1−r)2\sum_{k=0}^{\infty} (k+1) r^k = \dfrac{1}{(1-r)^2} for 0≤r<10 \le r < 1. Deduce ∑k≥0(k+1)/2k=4\sum_{k\ge0} (k+1)/2^k = 4.

Hint
Multiply ∑rm\sum r^m by ∑rn\sum r^n and collect the diagonals.

Solution

Take um=rmu_m = r^m, vn=rnv_n = r^n. The kk-th diagonal is wk=∑m+n=krmrn=(k+1)rkw_k = \sum_{m+n=k} r^m r^n = (k+1) r^k, since there are k+1k+1 pairs with m+n=km + n = k. By the theorem ∑(k+1)rk=11−r⋅11−r\sum (k+1)r^k = \dfrac{1}{1-r}\cdot\dfrac{1}{1-r}. With r=12r = \tfrac12 this is 44.

Exercise ●●○ Hardy: Examples LXIX, 2

Show that if one of ∑un\sum u_n, ∑vn\sum v_n (both of positive terms) diverges, then so does the Cauchy product ∑wk\sum w_k, unless every term of the other series is zero.

Hint
Suppose ∑un\sum u_n diverges and vp>0v_p > 0 for some pp. Find, inside the diagonals, a copy of vpv_p times many uu’s.

Solution

Suppose ∑un\sum u_n diverges and vp>0v_p > 0. The diagonal wm+pw_{m+p} contains the product umvpu_m v_p, so WN+p=w0+⋯+wN+p≥vp(u0+u1+⋯+uN)W_{N+p} = w_0 + \cdots + w_{N+p} \ge v_p(u_0 + u_1 + \cdots + u_N), because all other products are ≥0\ge 0. The right-hand side tends to ∞\infty as N→∞N \to \infty, so the partial sums WkW_k are unbounded and ∑wk\sum w_k diverges. If instead every vn=0v_n = 0, every wk=0w_k = 0 and the product converges to 00.

Exercise ●●○ Hardy: Examples LXIX, 4

Let ∑un\sum u_n and ∑vn\sum v_n (positive terms, starting at n=1n = 1) have sums ss and tt. Let wn=∑lm=nulvmw_n = \sum_{lm = n} u_l v_m, the sum over all pairs with product nn. Show that ∑wn=st\sum w_n = st. Deduce that ∑d(n)/n2=(∑1/n2)2\sum d(n)/n^2 = (\sum 1/n^2)^2, where d(n)d(n) is the number of divisors of nn.

Hint
Each product ulvmu_lv_m occurs in exactly one wnw_n, namely the one with n=lmn = lm.

Solution

Each wnw_n is a finite sum (there are only finitely many factorisations n=lmn = lm), and every product ulvmu_lv_m appears in exactly one of them. Writing out the terms of w1,w2,w3,…w_1, w_2, w_3, \dots in turn gives an arrangement of all the products ulvmu_lv_m, each once. By Dirichlet’s theorem and the theorem of this lesson, that series converges to stst; grouping it into the blocks wnw_n does not change the sum (positive terms). So ∑wn=st\sum w_n = st.

With ul=1/l2u_l = 1/l^2 and vm=1/m2v_m = 1/m^2, ulvm=1/(lm)2=1/n2u_lv_m = 1/(lm)^2 = 1/n^2, and the number of pairs with lm=nlm = n is d(n)d(n). So wn=d(n)/n2w_n = d(n)/n^2 and ∑d(n)/n2=(∑1/n2)2\sum d(n)/n^2 = (\sum 1/n^2)^2.

Exercise ●○○ Hardy: Examples LXIX, 3

State and prove the analogous result for three convergent series of positive terms with sums rr, ss, tt, collecting umvnwpu_m v_n w_p with m+n+p=km + n + p = k.

Solution

Claim. λk=∑m+n+p=kumvnwp\lambda_k = \sum_{m+n+p=k} u_mv_nw_p gives ∑λk=rst\sum \lambda_k = rst.

Proof. First form the Cauchy product ∑cj\sum c_j of ∑um\sum u_m and ∑vn\sum v_n, so cj=∑m+n=jumvnc_j = \sum_{m+n=j} u_mv_n; it converges to rsrs by the theorem, and its terms are positive. Now form the Cauchy product of ∑cj\sum c_j and ∑wp\sum w_p: its kk-th term is ∑j+p=kcjwp=∑m+n+p=kumvnwp=λk\sum_{j+p=k} c_j w_p = \sum_{m+n+p=k} u_mv_nw_p = \lambda_k. By the theorem again it converges to (rs)t(rs)t.

Check yourself

Check yourself True or false?

If ∑um\sum u_m and ∑vn\sum v_n are convergent series of positive terms, then the products umvnu_mv_n may be added in any order whatever and always give (∑um)(∑vn)(\sum u_m)(\sum v_n).

Why

True. One order (by squares) gives stst, and by Dirichlet’s theorem every order of a positive series gives the same sum.

Check yourself Choose one

In the Cauchy product of ∑um\sum u_m and ∑vn\sum v_n (both starting at index 00), what is w3w_3?

Why

wkw_k collects the products whose indices add to kk: the kk-th diagonal of the array, with k+1k + 1 terms. The third option is the sum of the whole square m,n≤3m, n \le 3, which is w0+⋯+w3w_0 + \cdots + w_3 plus the products below the diagonal.

Check yourself Enter a number

The Cauchy product of ∑1/m!\sum 1/m! with itself has terms wk=2k/k!w_k = 2^k/k!. What is ∑k≥02k/k!\sum_{k\ge0} 2^k/k!? (Give a decimal.)

Why

By the theorem it is (∑1/m!)2=e2≈7.389(\sum 1/m!)^2 = e^2 \approx 7.389. This is the case x=y=1x = y = 1 of exey=ex+ye^x e^y = e^{x+y}, which Hardy proves in Ch. IX in exactly this way.

Check yourself True or false?

The theorem remains true, with the same proof, for any two convergent series.

Why

False. Both proofs use positivity: Dirichlet’s theorem fails for conditionally convergent series, and “a larger set of products has a larger sum” fails when products can be negative. The Cauchy product of ∑(−1)n/n+1\sum (-1)^n/\sqrt{n+1} with itself diverges (§202).

Hardy’s language

Summary

Summary
  • The products umvnu_mv_n of two series form a square array; summing by squares gives sktk→sts_kt_k \to st.
  • The Cauchy product wk=∑m+n=kumvnw_k = \sum_{m+n=k} u_mv_n sums the array by diagonals, as in multiplying polynomials.
  • For convergent positive series it converges to stst: either by Dirichlet’s theorem, or because each triangle is trapped between two squares.
  • If either factor diverges (and the other is not identically zero) the product diverges. For terms of both signs the theorem needs more care.