Chapter 2 - Sequences and Series, Approaching Infinity, Divergence, Infinity, Algebra of Limits, Monotone Sequences, Weierstrass, Cauchy, Series, Summing Infinitely, Special Series and Operations

2.1 The Limit of a Sequence: Getting Closer and Closer

Imagine an infinite list of numbers, one after another: $(a_1,a_2,a_3,...)$. This is a sequence. We are interested in what happens to these numbers as we go further and further down the list – as we approach “infinity”. Do they settle down to a particular value? Do they shoot off to infinity themselves? Or do they just bounce around erratically?

We say a sequence $(a_n{)}_{n\ge 1}$ converges to a limit $l$ if the terms $a_n$ get arbitrarily close to $l$ as $n$ gets larger and larger. “Arbitrarily close” is the key idea here. No matter how small a distance you pick, eventually the terms of the sequence will be within that distance of the limit.

Here’s the formal definition, which makes “arbitrarily close” precise using $ϵ$ and $N$:

Definition 2.1.1: Convergence of a Sequence

A sequence $(a_n{)}_{n\ge 1}$ of real numbers converges to a limit $l\in \mathbb{R}$ if for every $ϵ>0$ (no matter how small), there exists a natural number $N\in \mathbb{N}$ (which might be very large) such that for all $n>N$, the absolute difference between $a_n$ and $l$ is less than $ϵ$:

$|a_n-l|<ϵ$

If such an $l$ exists, we write ${\lim }_{n\to \infty }{a}_n=l$.

Think of $ϵ$ as your “tolerance” for closeness. If you give me any tolerance $ϵ$, I can find a point $N$ in the sequence such that everything after that point is within your tolerance of the limit $l$.

Example: Consider the sequence $a_n=\frac{n}{n+1}=(\frac{1}{2},\frac{2}{3},\frac{3}{4},\frac{4}{5},...)$. It looks like these numbers are getting closer and closer to 1. Let’s prove that ${\lim }_{n\to \infty }\frac{n}{n+1}=1$.

Example: Proof of Convergence for $a_n=\frac{n}{n+1}$

We need to show that for any $ϵ>0$, we can find an $N$ such that for all $n>N$, $|\frac{n}{n+1}-1|<ϵ$.

First, simplify the expression: $|\frac{n}{n+1}-1|=|\frac{n-(n+1)}{n+1}|=|\frac{-1}{n+1}|=\frac{1}{n+1}$

We want $\frac{1}{n+1}<ϵ$. This is equivalent to $n+1>\frac{1}{ϵ}$, or $n>\frac{1}{ϵ}-1$.

By the Archimedean Principle (Corollary 1.1.7), for any $ϵ>0$, we can find a natural number $N$ such that $N>\frac{1}{ϵ}-1$.

Then, for all $n>N$, we have $n>\frac{1}{ϵ}-1$, so $n+1>\frac{1}{ϵ}$, and thus $\frac{1}{n+1}<ϵ$.

Therefore, for any $ϵ>0$, we have found an $N$ such that for all $n>N$, $|\frac{n}{n+1}-1|<ϵ$. This proves that ${\lim }_{n\to \infty }\frac{n}{n+1}=1$.

Uniqueness and Boundedness of Limits

A sequence can only converge to one limit. It can’t approach two different values at the same time.

Lemma 2.1.3: Uniqueness of Limits

The limit of a convergent sequence is unique.

Proof of Uniqueness of Limits (by contradiction)

Suppose a sequence $(a_n)$ converges to two different limits, $l_1$ and $l_2$, with $l_1\ne l_2$. Let $ϵ=\frac{|l_1-l_2|}{3}>0$.

Since $(a_n)\to l_1$, there exists $N_1$ such that for all $n>N_1$, $|a_n-l_1|<ϵ$. Since $(a_n)\to l_2$, there exists $N_2$ such that for all $n>N_2$, $|a_n-l_2|<ϵ$.

Let $N=\max (N_1,N_2)$. Then for $n>N$, both inequalities hold. Consider the distance between $l_1$ and $l_2$: $|l_1-l_2|=|(l_1-a_n)+(a_n-l_2)|\le |l_1-a_n|+|a_n-l_2|<ϵ+ϵ=2ϵ=\frac{2}{3}|l_1-l_2|$

This implies $|l_1-l_2|<\frac{2}{3}|l_1-l_2|$. If $|l_1-l_2|>0$, we can divide both sides by $|l_1-l_2|$ to get $1<\frac{2}{3}$, which is a contradiction. Therefore, our initial assumption that $l_1\ne l_2$ must be false, and the limit is unique.

If a sequence converges, it must also be bounded. It can’t go off to infinity or become arbitrarily large.

Remark 2.1.5: Boundedness of Convergent Sequences

Every convergent sequence is bounded.

Proof of Boundedness

Let $(a_n)$ be a convergent sequence with limit $l$. Choose $ϵ=1$. By definition of convergence, there exists $N$ such that for all $n>N$, $|a_n-l|<1$. This implies that for $n>N$, $|a_n|=|a_n-l+l|\le |a_n-l|+|l|<1+|l|$.

So, for $n>N$, the terms are bounded by $|l|+1$. For the first $N$ terms ($a_1,a_2,...,a_N$), they are also finite values. Therefore, we can find an overall bound $M$ by taking the maximum of the absolute values of the first $N$ terms and $|l|+1$.

Let $M=\max \{|a_1|,|a_2|,...,|a_N|,|l|+1\}$. Then $|a_n|\le M$ for all $n\ge 1$, which means the sequence is bounded.

Boundedness is a necessary condition for convergence, but it’s not sufficient. A bounded sequence doesn’t have to converge (think of the sequence $(-1{)}^n=(-1,1,-1,1,...)$ which is bounded but diverges). Convergence is a stronger condition.

2.2 Divergence: When Sequences Go Their Own Way

Not all sequences settle down to a finite limit. Some sequences diverge. Divergence simply means a sequence does not converge to a real number. There are different ways a sequence can diverge. It might oscillate, grow without bound, or behave in some other non-convergent way.

One particular type of divergence is divergence to infinity. This is when the terms of a sequence become larger and larger, exceeding any bound you can set.

Definition: Convergence to Infinity

A sequence $(a_n{)}_{n\ge 1}$ converges to infinity, written ${\lim }_{n\to \infty }{a}_n=\infty$, if for every real number $t$, there exists a natural number $N$ such that for all $n>N$, $a_n>t$.

Similarly, a sequence converges to negative infinity, written ${\lim }_{n\to \infty }{a}_n=-\infty$, if for every real number $t$, there exists a natural number $N$ such that for all $n>N$, $a_n<t$.

Example: Consider the sequence $a_n=2^n=(2,4,8,16,...)$. This sequence grows rapidly. Let’s show that ${\lim }_{n\to \infty }{2}^n=\infty$.

Example: Proof of Divergence to Infinity for $a_n=2^n$

We need to show that for any real number $t$, we can find an $N$ such that for all $n>N$, $2^n>t$.

First, we use induction to prove that $2^n\ge n+1$ for all $n\ge 1$.

• Base Case (n=1): $2^1=2\ge 1+1=2$. True.
• Inductive Step: Assume $2^k\ge k+1$ for some $k\ge 1$. We want to show $2^{k+1}\ge (k+1)+1=k+2$. $2^{k+1}=2\cdot 2^k\ge 2(k+1)=2k+2$. Since $2k+2\ge k+2$ for $k\ge 1$, we have $2^{k+1}\ge k+2$.

Now, given any real number $t$, by the Archimedean Principle, there exists an integer $N$ such that $N>t-1$.

For all $n>N$, we have $n>t-1$, so $n+1>t$. Since $2^n\ge n+1$, we have $2^n>t$.

Thus, for any $t$, we found an $N$ such that for all $n>N$, $2^n>t$. This proves ${\lim }_{n\to \infty }{2}^n=\infty$.

2.3 Limit Theorems: The Algebra of Convergence

Working directly with the $ϵ$-$N$ definition can be cumbersome. Fortunately, there are powerful theorems that let us compute limits more easily by manipulating convergent sequences algebraically. These are the Limit Laws.

Theorem: Algebraic Limit Laws

Let $(a_n{)}_{n\ge 1}$ and $(b_n{)}_{n\ge 1}$ be convergent sequences with limits $a={\lim }_{n\to \infty }{a}_n$ and $b={\lim }_{n\to \infty }{b}_n$. Then:

1. Sum Law: ${\lim }_{n\to \infty }(a_n+b_n)=a+b$
2. Product Law: ${\lim }_{n\to \infty }(a_n\cdot b_n)=a\cdot b$
3. Quotient Law: If $b\ne 0$ and $b_n\ne 0$ for all $n\ge 1$, then ${\lim }_{n\to \infty }\frac{a_n}{b_n}=\frac{a}{b}$
4. Comparison Law: If there exists $K\in \mathbb{N}$ such that $a_n\le b_n$ for all $n>K$, then $a\le b$.

These laws are incredibly useful. They tell us that limits behave nicely with basic arithmetic operations. We can find the limit of a sum by summing the limits, the limit of a product by multiplying the limits, and so on (with some conditions, like for division).

Proof of Sum Law (Theorem 2.1.8 (1))

Let $ϵ>0$. We want to show that there exists $N$ such that for all $n>N$, $|(a_n+b_n)-(a+b)|<ϵ$.

Since ${\lim }_{n\to \infty }{a}_n=a$, there exists $N_1$ such that for all $n>N_1$, $|a_n-a|<\frac{ϵ}{2}$. Since ${\lim }_{n\to \infty }{b}_n=b$, there exists $N_2$ such that for all $n>N_2$, $|b_n-b|<\frac{ϵ}{2}$.

Let $N=\max (N_1,N_2)$. Then for all $n>N$, both inequalities hold. Consider $|(a_n+b_n)-(a+b)|=|(a_n-a)+(b_n-b)|$. By the triangle inequality, $|(a_n-a)+(b_n-b)|\le |a_n-a|+|b_n-b|$. For $n>N$, we have $|a_n-a|+|b_n-b|<\frac{ϵ}{2}+\frac{ϵ}{2}=ϵ$.

Thus, for all $n>N$, $|(a_n+b_n)-(a+b)|<ϵ$, which proves ${\lim }_{n\to \infty }(a_n+b_n)=a+b$.

Example: Let’s find ${\lim }_{n\to \infty }(1+\frac{1}{n}{)}^2$.

We know ${\lim }_{n\to \infty }(1+\frac{1}{n})=1$. Using the Product Law (Theorem 2.1.8 (2)), we can write:

More generally, for any integer $b$, we can use repeated application of the product and quotient laws to show:

Example: Limit of $(1+\frac{1}{n}{)}^b$

For any integer $b\in \mathbb{Z}$, ${\lim }_{n\to \infty }(1+\frac{1}{n}{)}^b=1$.

These Limit Laws are essential tools in analysis, allowing us to break down complex limits into simpler pieces and compute them systematically. They form the basis for much of what comes next, including the study of series.

2.4 Monotone Sequences: Always Going in One Direction

A monotone sequence is one that is always either increasing or always decreasing. They are simpler to analyze than sequences that jump around.

Definition 2.2.1: Monotone Sequences

1. A sequence $(a_n{)}_{n\ge 1}$ is monotonically increasing (or non-decreasing) if $a_n\le a_{n+1}$ for all $n\ge 1$.
2. A sequence $(a_n{)}_{n\ge 1}$ is monotonically decreasing (or non-increasing) if $a_{n+1}\le a_n$ for all $n\ge 1$.
3. A sequence is monotone if it is either monotonically increasing or monotonically decreasing.

Think of a sequence walking uphill (increasing) or downhill (decreasing) – it always moves in one direction.

2.5 The Monotone Convergence Theorem (Weierstrass Theorem): Bounded and Monotone = Convergent

A fundamental result in analysis is the Monotone Convergence Theorem, also known as the Weierstrass Theorem. It provides a powerful tool for proving convergence of sequences, especially when you can show a sequence is both monotone and bounded.

Theorem 2.2.2: Monotone Convergence Theorem (Weierstrass)

1. If a sequence $(a_n{)}_{n\ge 1}$ is monotonically increasing and bounded above, then it converges to a limit $l=\sup \{a_n:n\ge 1\}$. The limit is the supremum (least upper bound) of the set of terms in the sequence.
2. If a sequence $(a_n{)}_{n\ge 1}$ is monotonically decreasing and bounded below, then it converges to a limit $l=\inf \{a_n:n\ge 1\}$. The limit is the infimum (greatest lower bound) of the set of terms in the sequence.

In simpler terms: If a sequence is going uphill and has a ceiling, it must approach that ceiling (or something below it). If it’s going downhill and has a floor, it must approach that floor (or something above it).

Proof of Weierstrass Theorem (Part 1: Increasing and Bounded Above)

Let $s=\sup \{a_n:n\ge 1\}$ be the supremum of the set of terms (which exists because the sequence is bounded above and non-empty). Let $ϵ>0$.

Since $s$ is the least upper bound, $s-ϵ$ is not an upper bound. Therefore, there must be at least one term in the sequence, say $a_N$, such that $a_N>s-ϵ$.

Since $(a_n)$ is monotonically increasing, for all $n>N$, we have $a_n\ge a_N$. Also, since $s$ is an upper bound for all terms, $a_n\le s$ for all $n$.

Combining these inequalities, for all $n>N$, we have $s-ϵ<a_N\le a_n\le s<s+ϵ$.

Thus, for all $n>N$, $|a_n-s|<ϵ$. This proves that ${\lim }_{n\to \infty }{a}_n=s=\sup \{a_n:n\ge 1\}$.

Example: Consider the sequence $x_n=(1+\frac{1}{n}{)}^n$. We can show (though it’s not immediately obvious) that this sequence is monotonically increasing and bounded above. Therefore, by the Weierstrass Theorem, it must converge. Its limit is the famous number $e\approx \mathrm{2.71828...}$.

Example: Convergence of $(1+\frac{1}{n}{)}^n$

The sequence $(1+\frac{1}{n}{)}^n$ is monotonically increasing and bounded above (though proving these facts requires some work, often using the Binomial Theorem or calculus). Therefore, by the Weierstrass Theorem, it converges. The limit is defined as Euler’s number: $e={\lim }_{n\to \infty }(1+\frac{1}{n}{)}^n$

Example: Let’s look at another sequence defined recursively: $a_1=\sqrt{2}$, $a_{n+1}=\sqrt{2+a_n}$. We can use the Weierstrass Theorem to show this converges.

Example: Recursive Sequence $a_{n+1}=\sqrt{2+a_n}$

Define $a_1=\sqrt{2}$, $a_{n+1}=\sqrt{2+a_n}$. We want to show ${\lim }_{n\to \infty }{a}_n$ exists.

1. Boundedness (above by 2): We use induction to show $a_n<2$ for all $n$.

• Base Case (n=1): $a_1=\sqrt{2}<2$.
• Inductive Step: Assume $a_k<2$. Then $a_{k+1}=\sqrt{2+a_k}<\sqrt{2+2}=\sqrt{4}=2$.

2. Monotonically Increasing: We use induction to show $a_n\le a_{n+1}$ for all $n$.

• Base Case (n=1): $a_1=\sqrt{2}\approx 1.414$, $a_2=\sqrt{2+\sqrt{2}}\approx 1.848$. So $a_1\le a_2$.
• Inductive Step: Assume $a_k\le a_{k+1}$. Then $2+a_k\le 2+a_{k+1}$. Taking square roots (which preserves order for non-negative numbers): $\sqrt{2+a_k}\le \sqrt{2+a_{k+1}}$, which means $a_{k+1}\le a_{k+2}$.

Since $(a_n)$ is monotonically increasing and bounded above, by the Weierstrass Theorem, it converges to a limit, say $L={\lim }_{n\to \infty }{a}_n$.

3. Finding the Limit: Since $a_{n+1}=\sqrt{2+a_n}$, taking the limit as $n\to \infty$ on both sides: $L=\sqrt{2+L}$. Squaring both sides: $L^2=2+L$, so $L^2-L-2=0$. Factoring: $(L-2)(L+1)=0$. Possible solutions are $L=2$ or $L=-1$. Since $a_n>0$ for all $n$, the limit must be non-negative, so $L=2$.

Therefore, ${\lim }_{n\to \infty }{a}_n=2$.

2.6 The Cauchy Criterion: Convergence Without Knowing the Limit

The definition of convergence requires knowing the limit $l$. But what if we don’t know the limit in advance? The Cauchy Criterion provides a way to determine if a sequence converges without knowing its limit. It’s based on the idea that the terms of a convergent sequence must eventually become close to each other.

Theorem 2.4.2: Cauchy Criterion for Convergence

A sequence $(a_n{)}_{n\ge 1}$ converges if and only if it is a Cauchy sequence. A sequence is Cauchy if for every $ϵ>0$, there exists an $N\in \mathbb{N}$ such that for all $n,m>N$, $|a_n-a_m|<ϵ$.

A Cauchy sequence is one where the terms eventually get arbitrarily close to each other, regardless of what the limit might be.

Proof of Cauchy Criterion (Sketch)

($\Rightarrow$) If a sequence converges, it’s Cauchy: Assume $(a_n)$ converges to a limit $A$. Then for large $n$ and $m$, both $a_n$ and $a_m$ are close to $A$. By triangle inequality, $a_n$ and $a_m$ must be close to each other.

($\Leftarrow$) If a sequence is Cauchy, it converges: (This direction is more involved and relies on the completeness of real numbers).

1. Cauchy sequences are bounded: (similar to proof that convergent sequences are bounded).
2. Use completeness to find a candidate limit: Define $b_n=\inf \{a_k:k\ge n\}$ and $c_n=\sup \{a_k:k\ge n\}$. Show that $(b_n)$ is increasing and bounded above, and $(c_n)$ is decreasing and bounded below. Both converge to limits, say $B$ and $C$.
3. Show $B=C$: Use the Cauchy condition to show that $C-B$ can be made arbitrarily small, forcing $B=C$. Let $A=B=C$.
4. Show $(a_n)\to A$: Use the fact that ${\lim }_{n\to \infty }{b}_n={\lim }_{n\to \infty }{c}_n=A$ and the Cauchy property to show that $(a_n)$ converges to $A$.

The Cauchy Criterion is a powerful theoretical tool. It’s used to prove many important results in analysis, and it provides a way to rigorously define convergence in more abstract settings where a limit might not be readily apparent.

This concludes our exploration of sequences, convergence, divergence, and the fundamental theorems that govern their behavior. These concepts are the bedrock for understanding series, continuity, differentiation, and integration, which we will explore in the following chapters.

2.7 Series: Adding Up Infinitely Many Terms

Imagine taking a sequence $(a_n{)}_{n\ge 1}$ and adding up all its terms: $a_1+a_2+a_3+\dots$. This infinite sum is called a series. Does this sum make sense? Can we actually add up infinitely many numbers and get a finite result? Sometimes yes, sometimes no.

To make sense of an infinite sum, we look at partial sums. The $n$-th partial sum $S_n$ is the sum of the first $n$ terms:

A series converges if the sequence of its partial sums $(S_n{)}_{n\ge 1}$ converges to a limit.

Definition 2.7.1: Convergence of a Series

The infinite series ${\sum }_{k=1}^{\infty }{a}_k$ converges if the sequence of partial sums $(S_n{)}_{n\ge 1}$, where $S_n={\sum }_{k=1}^n{a}_k$, converges to a limit.

If ${\lim }_{n\to \infty }{S}_n=S$, we write ${\sum }_{k=1}^{\infty }{a}_k=S$, and $S$ is called the sum of the series. If the sequence of partial sums diverges, the series diverges.

Example: The Geometric Series

Consider the geometric series ${\sum }_{k=0}^{\infty }{q}^k=1+q+q^2+q^3+\dots$. This series is fundamental in mathematics. When does it converge? And if it converges, what’s its sum?

Example: Convergence of Geometric Series

Let $S_n={\sum }_{k=0}^n{q}^k=1+q+q^2+\cdots +q^n$. Multiply $S_n$ by $q$: $q{S}_n=q+q^2+q^3+\cdots +q^{n+1}$. Subtract $q{S}_n$ from $S_n$: $S_n-q{S}_n=(1+q+\cdots +q^n)-(q+q^2+\cdots +q^{n+1})=1-q^{n+1}$. So, $(1-q)S_n=1-q^{n+1}$. If $q\ne 1$, we can solve for $S_n$: $S_n=\frac{1-q^{n+1}}{1-q}$.

Now consider the limit as $n\to \infty$. If $|q|<1$, then ${\lim }_{n\to \infty }{q}^{n+1}=0$. Thus, if $|q|<1$, ${\lim }_{n\to \infty }{S}_n={\lim }_{n\to \infty }\frac{1-q^{n+1}}{1-q}=\frac{1-0}{1-q}=\frac{1}{1-q}$.

If $|q|\ge 1$, the sequence $q^{n+1}$ does not converge to 0 (unless $q=1$, in which case $S_n=n+1$ diverges to $\infty$). Thus, if $|q|\ge 1$, the geometric series diverges (unless $q=1$ where it clearly diverges).

Conclusion: The geometric series ${\sum }_{k=0}^{\infty }{q}^k$ converges if and only if $|q|<1$, and in that case, its sum is $\frac{1}{1-q}$.

Example: The Harmonic Series

The harmonic series is ${\sum }_{n=1}^{\infty }\frac{1}{n}=1+\frac{1}{2}+\frac{1}{3}+\frac{1}{4}+\dots$. Does this series converge or diverge? It turns out it diverges, even though the terms $\frac{1}{n}$ go to zero. The terms don’t go to zero fast enough for the sum to be finite.

Example: Divergence of Harmonic Series

The harmonic series ${\sum }_{n=1}^{\infty }\frac{1}{n}$ diverges. (Proof by grouping terms, similar to Example 1.1.17(ii) in the script, showing the partial sums grow without bound).

2.8 Properties of Convergent Series: Linearity

Like limits of sequences, convergent series behave nicely with addition and scalar multiplication (linearity).

Theorem 2.7.4: Linearity of Series

Let ${\sum }_{k=1}^{\infty }{a}_k$ and ${\sum }_{j=1}^{\infty }{b}_j$ be convergent series with sums $A={\sum }_{k=1}^{\infty }{a}_k$ and $B={\sum }_{j=1}^{\infty }{b}_j$, and let $\alpha \in \mathbb{C}$ (or $\mathbb{R}$). Then:

1. Sum Law: The series ${\sum }_{k=1}^{\infty }(a_k+b_k)$ converges and ${\sum }_{k=1}^{\infty }(a_k+b_k)=A+B$.
2. Scalar Multiplication Law: The series ${\sum }_{k=1}^{\infty }(\alpha \cdot a_k)$ converges and ${\sum }_{k=1}^{\infty }(\alpha \cdot a_k)=\alpha \cdot A$.

These laws allow us to manipulate convergent series in familiar algebraic ways.

2.9 Convergence Tests: Determining if a Series Converges

We need tools to determine if a series converges without having to find its sum explicitly. The Cauchy Criterion for series is one such tool.

Theorem 2.7.5: Cauchy Criterion for Series

The series ${\sum }_{k=1}^{\infty }{a}_k$ converges if and only if for every $ϵ>0$, there exists $N\in \mathbb{N}$ such that for all $m\ge n\ge N$, $\left|{\sum }_{k=n}^m{a}_k\right|<ϵ$.

This criterion says a series converges if and only if the “tails” of the series (the sums of terms far down the sequence) become arbitrarily small.

Comparison Test: If we have a series that is “smaller” than a known convergent series, it must also converge. Conversely, if it’s “larger” than a known divergent series, it must also diverge.

Corollary 2.7.7: Comparison Test

Let ${\sum }_{k=1}^{\infty }{a}_k$ and ${\sum }_{k=1}^{\infty }{b}_k$ be series with non-negative terms ($a_k\ge 0,b_k\ge 0$).

1. If ${\sum }_{k=1}^{\infty }{b}_k$ converges and there exists $K\in \mathbb{N}$ such that for all $k>K$, $a_k\le b_k$, then ${\sum }_{k=1}^{\infty }{a}_k$ converges.
2. If ${\sum }_{k=1}^{\infty }{a}_k$ diverges and there exists $K\in \mathbb{N}$ such that for all $k>K$, $a_k\le b_k$, then ${\sum }_{k=1}^{\infty }{b}_k$ diverges.

Absolute Convergence Test: If a series of absolute values converges, then the original series must also converge. This is a powerful test because it allows us to use comparison tests (which require non-negative terms) on series with both positive and negative terms.

Theorem 2.7.10: Absolute Convergence Test

If the series ${\sum }_{k=1}^{\infty }|a_k|$ converges (we say ${\sum }_{k=1}^{\infty }{a}_k$ is absolutely convergent), then the series ${\sum }_{k=1}^{\infty }{a}_k$ also converges. Furthermore, $\left|{\sum }_{k=1}^{\infty }{a}_k\right|\le {\sum }_{k=1}^{\infty }|a_k|$.

Ratio Test and Root Test: These are practical tests that use ratios or roots of terms to determine convergence.

Theorem 2.7.17: Ratio Test

Let ${\sum }_{n=1}^{\infty }{a}_n$ be a series with $a_n\ne 0$ for all $n\ge 1$.

1. If ${\mathrm{limsup}}_{n\to \infty }\left|\frac{a_{n+1}}{a_n}\right|<1$, then the series converges absolutely.
2. If ${\mathrm{liminf}}_{n\to \infty }\left|\frac{a_{n+1}}{a_n}\right|>1$, then the series diverges.

Theorem 2.7.20: Root Test

Let ${\sum }_{n=1}^{\infty }{a}_n$ be a series.

1. If ${\mathrm{limsup}}_{n\to \infty }\sqrt[n]{|a_n|}<1$, then the series converges absolutely.
2. If ${\mathrm{limsup}}_{n\to \infty }\sqrt[n]{|a_n|}>1$, then the series diverges.

These tests provide practical methods for checking convergence of many series, especially power series and series where the terms have factorial or exponential components.

This is just the beginning of our journey into series. There’s much more to explore, including alternating series, power series, and the fascinating world of uniform convergence and its implications for calculus operations on infinite sums.

2.10 Alternating Series: A Dance of Plus and Minus

An alternating series is a series where the terms alternate in sign, like:

where $a_k\ge 0$ for all $k$. These series can converge even when the series of absolute values diverges (i.e., they can be conditionally convergent). The Alternating Series Test (Leibniz Criterion) gives us a simple condition for their convergence.

Theorem 2.7.12: Alternating Series Test (Leibniz Criterion)

Let $(a_n{)}_{n\ge 1}$ be a sequence of positive terms ($a_n\ge 0$) that is monotonically decreasing ($a_{n+1}\le a_n$) and converges to zero (${\lim }_{n\to \infty }{a}_n=0$). Then the alternating series ${\sum }_{k=1}^{\infty }(-1{)}^{k+1}{a}_k$ converges. Furthermore, the sum $S$ of the series satisfies the bounds:

$a_1-a_2\le S\le a_1$

The condition that $a_n$ decreases to zero is crucial. It ensures that the positive and negative terms “cancel out” enough for the partial sums to converge.

Proof of Alternating Series Test (Sketch)

Let $S_n={\sum }_{k=1}^n(-1{)}^{k+1}{a}_k$ be the partial sums.

1. Even Partial Sums $(S_{2n})$ are increasing and bounded above: $S_{2n+2}-S_{2n}=a_{2n+1}-a_{2n+2}=(a_{2n+1}-a_{2n+2})\ge 0$ since $a_n$ is decreasing. Also, $S_{2n}=a_1-(a_2-a_3)-(a_4-a_5)-\cdots -a_{2n}\le a_1$ since $(a_{2k}-a_{2k+1})\ge 0$.
2. Odd Partial Sums $(S_{2n+1})$ are decreasing and bounded below: $S_{2n+3}-S_{2n+1}=-a_{2n+2}+a_{2n+3}=-(a_{2n+2}-a_{2n+3})\le 0$. Also, $S_{2n+1}=(a_1-a_2)+(a_3-a_4)+\cdots +a_{2n+1}\ge a_1-a_2$ since $(a_{2k+1}-a_{2k+2})\ge 0$.
3. Both even and odd subsequences converge: By the Monotone Convergence Theorem, both $(S_{2n})$ and $(S_{2n+1})$ converge to some limits, say $L_{even}$ and $L_{odd}$.
4. Limits are equal: $S_{2n+1}-S_{2n}=a_{2n+1}\to 0$ as $n\to \infty$ (since $a_n\to 0$). Thus $L_{odd}-L_{even}=0$, so $L_{odd}=L_{even}=S$.
5. Full sequence converges: Since both even and odd subsequences converge to the same limit $S$, the entire sequence $(S_n)$ converges to $S$.
6. Error bounds: From the monotone behavior, we can deduce $S_{2n}\le S\le S_{2n+1}$, which gives the error bounds $a_1-a_2\le S\le a_1$.

Example: The Alternating Harmonic Series

The alternating harmonic series is ${\sum }_{k=1}^{\infty }\frac{(-1{)}^{k+1}}{k}=1-\frac{1}{2}+\frac{1}{3}-\frac{1}{4}+\dots$. Let’s check if it converges using the Leibniz Criterion.

Example: Convergence of Alternating Harmonic Series

Let $a_n=\frac{1}{n}$.

1. $a_n=\frac{1}{n}\ge 0$ for all $n\ge 1$. (Positive terms)
2. $a_{n+1}=\frac{1}{n+1}\le \frac{1}{n}=a_n$. (Monotonically decreasing)
3. ${\lim }_{n\to \infty }{a}_n={\lim }_{n\to \infty }\frac{1}{n}=0$. (Limit goes to zero)

All conditions of the Leibniz Criterion are satisfied. Therefore, the alternating harmonic series converges.

Its sum is actually $\ln 2\approx \mathrm{0.693...}$, which lies between $a_1-a_2=1-\frac{1}{2}=\frac{1}{2}$ and $a_1=1$, as predicted by the error bounds.

2.11 Rearrangement of Series: Order Matters (Sometimes!)

For finite sums, the order in which you add terms doesn’t matter. But for infinite series, things are more delicate. Rearranging a series means changing the order of its terms.

Absolutely Convergent Series: Order Doesn’t Matter

If a series is absolutely convergent (i.e., $\sum |a_k|$ converges), then rearranging the terms doesn’t change the sum. The sum is robust to reordering.

Theorem 2.7.16: Dirichlet's Theorem (Rearrangement of Absolutely Convergent Series)

If a series ${\sum }_{n=1}^{\infty }{a}_n$ is absolutely convergent, then any rearrangement of the series converges to the same sum.

Proof of Dirichlet's Theorem (Sketch)

1. Use the Cauchy Criterion for absolute convergence: For any $ϵ>0$, the “tail” of the absolute series sum is small.
2. Consider a rearrangement ${\sum }_{n=1}^{\infty }{a}_n^{\prime }$. Let $S_m^{\prime }$ be the $m$-th partial sum of the rearranged series and $S_m$ be the $m$-th partial sum of the original series.
3. For large enough $m$, the terms in $S_m^{\prime }$ will include all the terms in $S_N$ for some large $N$. The difference $S_m^{\prime }-S_N$ will consist of terms from the “tail” of the original series (terms with index greater than $N$).
4. Because of absolute convergence, this “tail” sum can be made smaller than $ϵ$. Thus, $S_m^{\prime }$ is close to $S_N$, which is close to the sum of the original series. This shows the rearranged series converges to the same sum.

Conditionally Convergent Series: Order Matters Greatly!

However, if a series is only conditionally convergent (converges, but not absolutely), then rearranging the terms can change the sum, or even make the series diverge! This is a surprising and counterintuitive result.

Theorem: Riemann Rearrangement Theorem

If a series ${\sum }_{n=1}^{\infty }{a}_n$ is conditionally convergent, then for any real number $L$, there exists a rearrangement of the series that converges to $L$. Furthermore, there exist rearrangements that diverge to $\infty$ or $-\infty$.

This theorem highlights the delicate nature of conditional convergence. The sum of a conditionally convergent series is not an inherent property of the terms themselves, but depends crucially on the order in which they are added.

Example: Rearranging the Alternating Harmonic Series

We saw the alternating harmonic series converges to $\ln 2$. But we can rearrange it to converge to a different value, or even diverge! The script provides an example where rearranging it makes it converge to $\frac{1}{2}\ln 2$.

2.12 The Cauchy Product of Series: Multiplying Infinite Sums

How do we multiply two infinite series? If we have ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$, what is their product? The Cauchy product provides a way to define this product.

Definition 2.7.24: Cauchy Product of Series

The Cauchy product of two series ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$ is the series ${\sum }_{n=0}^{\infty }{c}_n$, where the $n$-th term $c_n$ is given by:

$c_n={\sum }_{j=0}^n{a}_{n-j}{b}_j=a_n{b}_0+a_{n-1}{b}_1+\cdots +a_0{b}_n$

This formula arises naturally if you think about multiplying the partial sums and collecting terms with the same total index.

Theorem 2.7.23: Convergence of Cauchy Product

If both series ${\sum }_{i=0}^{\infty }{a}_i$ and ${\sum }_{j=0}^{\infty }{b}_j$ are absolutely convergent, then their Cauchy product ${\sum }_{n=0}^{\infty }{c}_n$ is also absolutely convergent, and its sum is the product of the sums of the original series:

${\sum }_{n=0}^{\infty }{c}_n=\left({\sum }_{i=0}^{\infty }{a}_i\right)\left({\sum }_{j=0}^{\infty }{b}_j\right)$

The Cauchy product works well for absolutely convergent series. However, for conditionally convergent series, the Cauchy product may not converge, or if it does, its sum might not be what you expect.

Example: Cauchy Product and the Exponential Function

The exponential function can be defined by its power series: $e^z={\sum }_{n=0}^{\infty }\frac{z^n}{n!}$. Let’s use the Cauchy product to multiply $e^w$ and $e^z$.

Example: Cauchy Product of Exponential Series

Let $a_i=\frac{w^i}{i!}$ and $b_j=\frac{z^j}{j!}$. The $n$-th term of their Cauchy product is: $c_n={\sum }_{j=0}^n{a}_{n-j}{b}_j={\sum }_{j=0}^n\frac{w^{n-j}}{(n-j)!}\frac{z^j}{j!}=\frac{1}{n!}{\sum }_{j=0}^n\frac{n!}{(n-j)!j!}{w}^{n-j}{z}^j=\frac{1}{n!}{\sum }_{j=0}^n\left(\genfrac{}{}{0ex}{}{n}{j}\right)w^{n-j}{z}^j$

By the Binomial Theorem, ${\sum }_{j=0}^n\left(\genfrac{}{}{0ex}{}{n}{j}\right)w^{n-j}{z}^j=(w+z{)}^n$.

So, $c_n=\frac{(w+z{)}^n}{n!}$. Thus, the Cauchy product series is ${\sum }_{n=0}^{\infty }\frac{(w+z{)}^n}{n!}$, which is the power series for $e^{w+z}$.

This shows that $e^w\cdot e^z=e^{w+z}$, which is a fundamental property of the exponential function, derived here using the Cauchy product of their series representations.

This concludes Chapter 2. We’ve explored sequences and series, their convergence, divergence, and important operations and tests. We’ve seen the power of the Weierstrass Theorem, the Cauchy Criterion, and the utility of Limit Laws. We’ve also touched upon the subtleties of rearranging series and multiplying infinite sums. This foundation will be crucial as we move into the realm of continuous functions and calculus in the following chapters.

Previous Chapter: Chapter 1 - Real Numbers, Euclidean Spaces, and Complex Numbers

Next Chapter: Chapter 3 - Continuous Functions, Smoothness and Limits, Continuity, Functions Without Jumps, Combining Continuous Functions, Fundamental Theorems of Continuity, Exponential and Trigonometric Functions