09 Double Series, Cauchy Products, Exponential Functions and Real-Valued Functions

Lecture from: 18.03.2024 | Video: Video ETHZ

In this lecture, we venture into the realm of double series, explore the fascinating Cauchy product of series, delve deeper into the exponential function, and lay the groundwork for our upcoming study of real-valued functions.

Double Series: Summing Over a Grid

Definition: Summing in Two Dimensions

Imagine an infinite grid of numbers, indexed by pairs of natural numbers $(i,j)$. A double sequence is precisely this: a function $a:\mathbb{N}\times \mathbb{N}\to \mathbb{R}$ (or $\mathbb{C}$). We denote the term at index $(i,j)$ as $a_{ij}$.

We can visualize a double sequence as an infinite matrix:

The challenge is: how do we sum all these infinitely many terms? There’s no single “natural” order. We can sum row by row, column by column, or even diagonally. Do these different summation methods yield the same result?

Summation Schemas and Potential Issues

Let’s consider two natural summation schemas:

1. Row-wise Summation: First sum each row to get row sums $S_i={\sum }_{j=0}^{\infty }{a}_{ij}$, and then sum the row sums: ${\sum }_{i=0}^{\infty }{S}_i={\sum }_{i=0}^{\infty }\left({\sum }_{j=0}^{\infty }{a}_{ij}\right)$.

2. Column-wise Summation: First sum each column to get column sums $U_j={\sum }_{i=0}^{\infty }{a}_{ij}$, and then sum the column sums: ${\sum }_{j=0}^{\infty }{U}_j={\sum }_{j=0}^{\infty }\left({\sum }_{i=0}^{\infty }{a}_{ij}\right)$.

Example: Different Summation Orders, Different Results

Consider the double sequence:

The matrix looks like:

• Row Sums: Each row sum is $S_i=1+(-1)+0+0+\cdots =0$ for all $i\ge 0$. Therefore, ${\sum }_{i=0}^{\infty }{S}_i={\sum }_{i=0}^{\infty }0=0$.

• Column Sums:

  • $U_0=1+0+0+0+\cdots =1$
  • $U_j=-1+1+0+0+\cdots =0$ for $j\ge 1$. Therefore, ${\sum }_{j=0}^{\infty }{U}_j=1+0+0+\cdots =1$.

We get different results depending on the summation order! This highlights that the order of summation matters for double series, and we need conditions to ensure consistent sums.

Linear Ordering of Double Sequences

To avoid ambiguity, we can consider summing the double sequence along a linear ordering.

Definition: Linear Ordering of a Double Sequence

Given a double sequence $(a_{ij}{)}_{i,j\ge 0}$, a linear ordering is a simple sequence $(b_k{)}_{k\ge 0}$ formed by arranging all terms of $(a_{ij})$ in some order. More formally, it’s a sequence $(b_k{)}_{k\ge 0}$ where there exists a bijection $\sigma :\mathbb{N}\to \mathbb{N}\times \mathbb{N}$ such that $b_k=a_{\sigma (k)}$.

For example, we could order them diagonally: $a_{00},a_{01},a_{10},a_{02},a_{11},a_{20},a_{03},\dots$

Double Series Theorem: Guaranteeing Consistent Summation

The following theorem provides conditions under which different summation methods for double series yield the same result.

Theorem (Double Series Theorem): Let $(a_{ij}{)}_{i,j\ge 0}$ be a double sequence. Assume there exists a bound $B\ge 0$ such that for all $m\in \mathbb{N}$:

Then:

1. Absolute Convergence of Row and Column Sums: For each $i\ge 0$, the row series ${\sum }_{j=0}^{\infty }{a}_{ij}$ converges absolutely to a row sum $S_i={\sum }_{j=0}^{\infty }{a}_{ij}$. For each $j\ge 0$, the column series ${\sum }_{i=0}^{\infty }{a}_{ij}$ converges absolutely to a column sum $U_j={\sum }_{i=0}^{\infty }{a}_{ij}$.

2. Absolute Convergence of Sums of Row and Column Sums: The series of row sums ${\sum }_{i=0}^{\infty }{S}_i$ and the series of column sums ${\sum }_{j=0}^{\infty }{U}_j$ are both absolutely convergent.

3. Equality of Sums: The sum of row sums equals the sum of column sums, and we denote this common sum as $S$:

   $$\sum _{i=0}^{\infty }{S}_i=\sum _{j=0}^{\infty }{U}_j=S$$

4. Convergence of Linearly Ordered Series: If $(b_k{)}_{k\ge 0}$ is any linear ordering of $(a_{ij}{)}_{i,j\ge 0}$, then the series ${\sum }_{k=0}^{\infty }{b}_k$ is absolutely convergent and converges to the same sum $S$:

   $$\sum _{k=0}^{\infty }{b}_k=S$$

Proof (Sketch):

The condition ${\sum }_{i=0}^m{\sum }_{j=0}^m|a_{ij}|\le B$ for all $m$ implies that the “quadrant sums” of absolute values are bounded. This ensures that all infinite sums (row sums, column sums, and the linearly ordered sum) converge absolutely and to the same value. The proof is conceptually similar to the proof of Dirichlet’s Theorem for rearrangements of absolutely convergent series, relying on the fact that absolute convergence allows for rearranging terms without changing the sum.

Application: Cauchy Product of Series

Defining the Product of Two Series

Given two series ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$, how do we define their “product”? If we multiply their partial sums, we get:

As $n\to \infty$, this suggests defining the product of the infinite series using the double sum of products $a_i{b}_j$. The Cauchy product formalizes this idea by arranging the terms $a_i{b}_j$ in a specific linear order.

Definition: Cauchy Product

The Cauchy product of the series ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$ is the series ${\sum }_{n=0}^{\infty }{c}_n$, where the $n$-th term $c_n$ is given by:

The first few terms of the Cauchy product are:

• $c_0=a_0{b}_0$
• $c_1=a_1{b}_0+a_0{b}_1$
• $c_2=a_2{b}_0+a_1{b}_1+a_0{b}_2$ … and so on.

Theorem: Convergence of Cauchy Product

Theorem (Cauchy Product Convergence): If both series ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$ are absolutely convergent, then their Cauchy product ${\sum }_{n=0}^{\infty }{c}_n$ is also absolutely convergent, and its sum is the product of the sums of the original series:

Proof (using Double Series Theorem):

Let $a_{ij}=a_i{b}_j$. Consider the double series ${\sum }_{i=0}^{\infty }{\sum }_{j=0}^{\infty }{a}_{ij}={\sum }_{i=0}^{\infty }{\sum }_{j=0}^{\infty }{a}_i{b}_j$.

Since ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$ are absolutely convergent, let $A={\sum }_{i=0}^{\infty }|a_i|$ and $B={\sum }_{j=0}^{\infty }|b_j|$. Then:

The condition of the Double Series Theorem is satisfied with bound $B=AB$. Therefore, we can apply the theorem.

By the Double Series Theorem, the row sums and column sums exist and are equal. Let’s consider the diagonal summation. The Cauchy product term $c_n={\sum }_{j=0}^n{a}_{n-j}{b}_j$ is precisely the sum along the diagonal $i+j=n$ in the double sum ${\sum }_{i,j}{a}_i{b}_j$. The linear ordering of terms in the Cauchy product corresponds to summing along diagonals in increasing order of $i+j$.

By the Double Series Theorem, the sum of the Cauchy product series is equal to the product of the sums of the original series:

And the Cauchy product series is absolutely convergent.

Application: Addition Theorem for the Exponential Function

Recall: Exponential Function Series

Recall the exponential function defined by the power series:

We know this series converges absolutely for all $z\in \mathbb{C}$.

Addition Theorem: Exponential of Sum is Product of Exponentials

Theorem (Addition Theorem for Exponential Function): For all $z,w\in \mathbb{C}$:

This fundamental property mirrors the behavior of real exponentials $e^{x+y}=e^x{e}^y$.

Proof (using Cauchy Product):

We want to compute $\exp (z)\exp (w)$ using the Cauchy product of the series for $\exp (z)$ and $\exp (w)$.

Let $a_n=\frac{z^n}{n!}$ and $b_n=\frac{w^n}{n!}$. Then $\exp (z)={\sum }_{n=0}^{\infty }{a}_n$ and $\exp (w)={\sum }_{n=0}^{\infty }{b}_n$.

The Cauchy product series has terms $c_n={\sum }_{j=0}^n{a}_{n-j}{b}_j={\sum }_{j=0}^n\frac{z^{n-j}}{(n-j)!}\frac{w^j}{j!}={\sum }_{j=0}^n\frac{1}{n!}\frac{n!}{(n-j)!j!}{z}^{n-j}{w}^j=\frac{1}{n!}{\sum }_{j=0}^n\left(\genfrac{}{}{0ex}{}{n}{j}\right)z^{n-j}{w}^j$.

By the Binomial Theorem, ${\sum }_{j=0}^n\left(\genfrac{}{}{0ex}{}{n}{j}\right)z^{n-j}{w}^j=(z+w{)}^n$. Therefore, $c_n=\frac{(z+w{)}^n}{n!}$.

The Cauchy product series is ${\sum }_{n=0}^{\infty }{c}_n={\sum }_{n=0}^{\infty }\frac{(z+w{)}^n}{n!}$. But this is precisely the power series definition of $\exp (z+w)$.

Thus, by the Cauchy Product Theorem (since the exponential series are absolutely convergent),

Corollary: Exponential of Integer and Rational Values

Using the Addition Theorem repeatedly, we can show:

• $\exp (m)=(\exp (1){)}^m$ for any integer $m$.
• $\exp (\frac{1}{n})=\sqrt[n]{\exp (1)}$ for any integer $n\ne 0$.
• $\exp (\frac{m}{n})=(\sqrt[n]{\exp (1)}{)}^m=(\exp (1){)}^{m/n}$ for any rational number $\frac{m}{n}$.

Defining $e=\exp (1)={\sum }_{n=0}^{\infty }\frac{1}{n!}\approx 2.71828$, we can write $\exp (r)=e^r$ for any rational number $r$. By continuity, we extend this to define $e^x=\exp (x)$ for all real numbers $x$, and $e^z=\exp (z)$ for all complex numbers $z$.

Corollary: Limit Definition of $e^z$

From the binomial expansion of $(1+\frac{z}{n}{)}^n$, we can derive the limit representation of the exponential function:

Corollary: For every $z\in \mathbb{C}$:

Proof (Sketch):

Expand $(1+\frac{z}{n}{)}^n$ using the binomial theorem:

For fixed $k$, as $n\to \infty$, $\frac{n(n-1)\dots (n-k+1)}{n^k}=\frac{n}{n}\cdot \frac{n-1}{n}\dots \frac{n-k+1}{n}\to 1\cdot 1\dots 1=1$.

Thus, as $n\to \infty$, each term in the binomial expansion approaches the corresponding term in the exponential series. A more rigorous proof involves showing that the binomial expansion converges to the exponential series.

This limit representation connects the exponential function to compound interest and provides another way to understand its nature.

Continuous Functions

We now transition to the study of functions, specifically real-valued functions. These are the workhorses of calculus and analysis, describing relationships between quantities in the real world.

Real-Valued Functions

Let $D$ be a set. We consider the set of all functions from $D$ to the real numbers $\mathbb{R}$, denoted as $R^D$. This set, $R^D$, possesses a rich algebraic structure.

Vector Space Structure of $R^D$

The set $R^D$ forms a vector space over the real numbers $\mathbb{R}$. This means we can perform vector addition and scalar multiplication on functions in $R^D$ and the results remain within $R^D$, satisfying the vector space axioms.

• Function Addition: For $f_1,f_2\in R^D$, their sum $(f_1+f_2)$ is a function in $R^D$ defined pointwise: $$(f_1+f_2)(x):=f_1(x)+f_2(x)\text{for all }x\in D$$
• Scalar Multiplication: For $f\in R^D$ and $\alpha \in \mathbb{R}$, the scalar product $(\alpha f)$ is a function in $R^D$ defined pointwise: $$(\alpha f)(x):=\alpha \cdot f(x)\text{for all }x\in D$$

For $f_1,f_2,f\in R^D$ and $\alpha \in \mathbb{R}$, these operations satisfy the properties of a vector space, such as associativity, commutativity, distributivity, and the existence of additive and multiplicative identities.

Algebra Structure of $R^D$: Pointwise Product

Beyond the vector space structure, $R^D$ is also an algebra over $\mathbb{R}$. This means we can define a pointwise product of functions, which is also a function in $R^D$:

• Pointwise Function Multiplication: For $f_1,f_2\in R^D$, their product $(f_1\cdot f_2)$ is a function in $R^D$ defined pointwise: $$(f_1\cdot f_2)(x):=f_1(x)\cdot f_2(x)\text{for all }x\in D$$

This pointwise multiplication, combined with function addition and scalar multiplication, makes $R^D$ an algebra.

Constant Functions: Identities

Within $R^D$, we have special constant functions. A constant function is one that assigns the same real value to every element in the domain $D$. Two particularly important constant functions are the zero function and the one function, denoted as $0$ and $1$ respectively:

• Zero Function ($0$): Defined by $0(x):=0$ for all $x\in D$.
• One Function ($1$): Defined by $1(x):=1$ for all $x\in D$.

These constant functions act as identities for addition and multiplication in $R^D$:

• Additive Identity: For any $f\in R^D$, $f+0=f$.
• Multiplicative Identity: For any $f\in R^D$, $f\cdot 1=f$.

Furthermore, the operations in $R^D$ (addition and multiplication) are associative and commutative, and the distributive law holds: $f_1\cdot (f_2+f_3)=f_1\cdot f_2+f_1\cdot f_3$.

However, $R^D$ is not a field in general, unless the domain $D$ has a very specific structure (e.g., if $D$ has at most one element). In general, for $|D|\ge 2$, there exist non-zero functions in $R^D$ that do not possess a multiplicative inverse. For example, if $D$ contains at least two distinct points $x_1,x_2\in D$, we can define a function $f\in R^D$ such that $f(x_1)=0$ and $f(x_2)=1$. This function $f$ is not the zero function, but it cannot have a multiplicative inverse. If it did, say $g$ is the inverse, then $(f\cdot g)(x_1)=f(x_1)\cdot g(x_1)=0\cdot g(x_1)=0$, but for a multiplicative inverse, we require $(f\cdot g)=1$, meaning $(f\cdot g)(x)=1$ for all $x\in D$. This is a contradiction.

Order Relation in $R^D$

We can define a pointwise order relation "$\le$" on $R^D$:

For functions $f,g\in R^D$, we say $f\le g$ if and only if $f(x)\le g(x)$ for all $x\in D$.

This defines a partial order on $R^D$.

Examples of $R^D$ for Different Sets $D$

• $D=\{1\}$: $R^D=R^{\{1\}}$ is essentially the set of real numbers $\mathbb{R}$ itself, as a function from $\{1\}$ to $\mathbb{R}$ is determined by its value at $1$, which is a real number. So $R^{\{1\}}\cong \mathbb{R}$.

• $D=\{1,2,\dots ,n\}$: $R^D=R^{\{1,\dots ,n\}}$ is isomorphic to ${\mathbb{R}}^n$. A function $f\in R^D$ is determined by its values $(f(1),f(2),\dots ,f(n))$, which is a vector in ${\mathbb{R}}^n$. So $R^{\{1,\dots ,n\}}\cong {\mathbb{R}}^n$.

• $D=\mathbb{N}$: $R^D=R^{\mathbb{N}}$ is the set of all real sequences. A function $f\in R^{\mathbb{N}}$ is a sequence $(f(1),f(2),f(3),\dots )$. So $R^{\mathbb{N}}$ is the space of real sequences.

Bounded Functions

We classify functions based on their boundedness. Let $f\in R^D$.

1. Bounded Above: A function $f$ is bounded above if its image set $f(D)=\{f(x)\mid x\in D\}$ is bounded above in $\mathbb{R}$. This means there exists a real number $M\in \mathbb{R}$ such that $f(x)\le M$ for all $x\in D$.

2. Bounded Below: A function $f$ is bounded below if its image set $f(D)$ is bounded below in $\mathbb{R}$. This means there exists a real number $m\in \mathbb{R}$ such that $f(x)\ge m$ for all $x\in D$.

3. Bounded: A function $f$ is bounded if its image set $f(D)$ is bounded in $\mathbb{R}$. This means there exist real numbers $m,M\in \mathbb{R}$ such that $m\le f(x)\le M$ for all $x\in D$. Equivalently, $f$ is bounded if it is both bounded above and bounded below. Also, $f$ is bounded if there exists $C\ge 0$ such that $|f(x)|\le C$ for all $x\in D$.

Monotonic Functions (for $D\subseteq \mathbb{R}$)

Now, assume the domain $D$ is a subset of the real numbers, $D\subseteq \mathbb{R}$, and consider functions $f:D\to \mathbb{R}$. We can define different types of monotonicity.

For functions $f:D\to \mathbb{R}$ where $D\subseteq \mathbb{R}$:

1. (Strongly) Monotonically Increasing: A function $f$ is (strongly) monotonically increasing if for all $x,y\in D$:

   • If $x\le y$, then $f(x)\le f(y)$. (Monotonically increasing)
   • If $x<y$, then $f(x)<f(y)$. (Strictly monotonically increasing)

2. (Strongly) Monotonically Decreasing: A function $f$ is (strongly) monotonically decreasing if for all $x,y\in D$:

   • If $x\le y$, then $f(x)\ge f(y)$. (Monotonically decreasing)
   • If $x<y$, then $f(x)>f(y)$. (Strictly monotonically decreasing)

3. (Strongly) Monotonic: A function $f$ is (strongly) monotonic if it is either (strongly) monotonically increasing or (strongly) monotonically decreasing.

Examples: Monotonicity

Example: Consider functions $f:D\to \mathbb{R},x\mapsto x^n$ for $n\in \mathbb{N}$.

• $D=\mathbb{R}$:

  • For $n$ odd (e.g., $x^3$), $f(x)=x^n$ is strictly monotonically increasing on $\mathbb{R}$.
  • For $n$ even (e.g., $x^2$), $f(x)=x^n$ is not monotonic on $\mathbb{R}$. It is monotonically decreasing on $(-\infty ,0]$ and monotonically increasing on $[0,\infty )$.

• $D=[0,\infty )$: For $D=[0,\infty )$, the function $f(x)=x^n$ is strictly monotonically increasing for all $n\ge 1$.

In the next sections, we will build upon these foundational concepts of real-valued functions to define and explore the crucial notion of continuity.

Continue here: 10 Continuity, Intermediate Value Theorem