05 Sandwich Lemma, Cauchy Criterion, and Bolzano-Weierstrass Theorem

Lecture from: 04.03.2024 | Video: Video ETHZ

In this lecture, we will delve deeper into the properties of sequences, building upon our understanding of convergence and divergence. We will explore powerful tools like the Sandwich Lemma for determining convergence, the Cauchy Criterion for assessing convergence without knowing the limit, and the fundamental Bolzano-Weierstrass Theorem concerning subsequences of bounded sequences.

The Sandwich Lemma (Squeeze Theorem)

Statement of the Lemma

The Sandwich Lemma, also known as the Squeeze Theorem, is a valuable tool for determining the limit of a sequence by “sandwiching” it between two other sequences with known limits.

Lemma (Sandwich Lemma): Let $(a_n{)}_{n\ge 1}$, $(b_n{)}_{n\ge 1}$, and $(c_n{)}_{n\ge 1}$ be real sequences such that:

1. $(a_n{)}_{n\ge 1}$ converges to a limit $L$.
2. $(b_n{)}_{n\ge 1}$ converges to the same limit $L$.
3. There exists an index $N_0\in \mathbb{N}$ such that for all $n>N_0$, we have $a_n\le c_n\le b_n$.

Then, the sequence $(c_n{)}_{n\ge 1}$ also converges, and its limit is also $L$:

${\lim }_{n\to \infty }{c}_n={\lim }_{n\to \infty }{a}_n={\lim }_{n\to \infty }{b}_n=L$

Intuition and Justification

Imagine the sequences $(a_n)$ and $(b_n)$ as two “slices of bread” closing in on the same limit $L$. If the sequence $(c_n)$ is always “sandwiched” between them, it is forced to converge to the same limit.

Proof: Let $ϵ>0$ be given.

1. Since ${\lim }_{n\to \infty }{a}_n=L$, there exists $N_1\in \mathbb{N}$ such that for all $n>N_1$, $|a_n-L|<ϵ$. This means $L-ϵ<a_n<L+ϵ$.
2. Since ${\lim }_{n\to \infty }{b}_n=L$, there exists $N_2\in \mathbb{N}$ such that for all $n>N_2$, $|b_n-L|<ϵ$. This means $L-ϵ<b_n<L+ϵ$.
3. We are given that there exists $N_0\in \mathbb{N}$ such that for all $n>N_0$, $a_n\le c_n\le b_n$.

Let $N=\max \{N_0,N_1,N_2\}$. For all $n>N$, all three conditions hold simultaneously. Combining the inequalities:

$L-ϵ<a_n\le c_n\le b_n<L+ϵ$

From this, we can directly conclude that for all $n>N$, $L-ϵ<c_n<L+ϵ$, which is equivalent to $|c_n-L|<ϵ$.

Therefore, by definition, ${\lim }_{n\to \infty }{c}_n=L$.

Revisiting Limit Inferior and Limit Superior

Recalling Definitions

Let $(a_n{)}_{n\ge 1}$ be a real sequence. We define:

• Limit Inferior (liminf): ${\mathrm{liminf}}_{n\to \infty }{a}_n={\lim }_{n\to \infty }(\inf \{a_k\mid k\ge n\})$ Let $b_n=\inf \{a_k\mid k\ge n\}$. Then ${\mathrm{liminf}}_{n\to \infty }{a}_n={\lim }_{n\to \infty }{b}_n$.

• Limit Superior (limsup): ${\mathrm{limsup}}_{n\to \infty }{a}_n={\lim }_{n\to \infty }(\sup \{a_k\mid k\ge n\})$ Let $c_n=\sup \{a_k\mid k\ge n\}$. Then ${\mathrm{limsup}}_{n\to \infty }{a}_n={\lim }_{n\to \infty }{c}_n$.

Existence of Limit Inferior and Limit Superior

The limit inferior and limit superior always exist for any real sequence (though they may be $\pm \infty$). This existence is guaranteed by the monotonicity of the sequences $(\inf \{a_k\mid k\ge n\}{)}_{n\ge 1}$ and $(\sup \{a_k\mid k\ge n\}{)}_{n\ge 1}$.

• The sequence of infima $(b_n{)}_{n\ge 1}$ is non-decreasing (monotone increasing) because as $n$ increases, we are taking the infimum over a smaller set, which can only increase or stay the same. A bounded monotone increasing sequence converges to its supremum. If unbounded, it diverges to $+\infty$.

• The sequence of suprema $(c_n{)}_{n\ge 1}$ is non-increasing (monotone decreasing) because as $n$ increases, we are taking the supremum over a smaller set, which can only decrease or stay the same. A bounded monotone decreasing sequence converges to its infimum. If unbounded, it diverges to $-\infty$.

Therefore, the limits ${\lim }_{n\to \infty }{b}_n$ and ${\lim }_{n\to \infty }{c}_n$ always exist in $\mathbb{R}\cup \{-\infty ,+\infty \}$.

Example: Illustrating Limit Inferior and Limit Superior

Consider the sequence $a_n=(-1{)}^n+\frac{1}{n}$ for $n\ge 1$.

Let’s analyze the behavior of this sequence:

• For even $n$, $a_n=1+\frac{1}{n}$, approaching 1 from above.
• For odd $n$, $a_n=-1+\frac{1}{n}$, approaching -1 from above.

Let’s examine the sequences $b_n=\inf \{a_k\mid k\ge n\}$ and $c_n=\sup \{a_k\mid k\ge n\}$.

• For $b_n=\inf \{a_k\mid k\ge n\}$: For any $n$, the terms $a_k$ for $k\ge n$ will include terms close to -1 (when $k$ is odd and large). The infimum will therefore be close to -1 and will approach -1 as $n\to \infty$. In fact, for sufficiently large $n$, the infimum will be achieved by terms with odd indices close to $-1$. Thus, ${\mathrm{liminf}}_{n\to \infty }{a}_n={\lim }_{n\to \infty }{b}_n=-1$.

• For $c_n=\sup \{a_k\mid k\ge n\}$: Similarly, for any $n$, the terms $a_k$ for $k\ge n$ will include terms close to 1 (when $k$ is even and large). The supremum will be close to 1 and will approach 1 as $n\to \infty$. For sufficiently large $n$, the supremum will be achieved by terms with even indices close to $1$. Thus, ${\mathrm{limsup}}_{n\to \infty }{a}_n={\lim }_{n\to \infty }{c}_n=1$.

In this example:

• ${\mathrm{liminf}}_{n\to \infty }{a}_n=-1$
• ${\mathrm{limsup}}_{n\to \infty }{a}_n=1$

We can define two auxiliary sequences:

• $b_n=-1$ (constant sequence)
• $c_n=1+\frac{1}{n_g}$, where $n_g$ is the smallest even number $\ge n$. This is just to demonstrate a sequence that approaches the limsup from above, not strictly needed for calculating the limsup itself.

Properties of Limit Inferior and Limit Superior

Key Relationships and Implications

Limit inferior and limit superior possess several important properties that provide deeper insights into the behavior of sequences.

• Relationship between liminf and limsup: For any real sequence $(a_n{)}_{n\ge 1}$, we always have: ${\mathrm{liminf}}_{n\to \infty }{a}_n\le {\mathrm{limsup}}_{n\to \infty }{a}_n$

• Boundedness and limsup: A sequence $(a_n{)}_{n\ge 1}$ is bounded above if and only if ${\mathrm{limsup}}_{n\to \infty }{a}_n<\infty$.

• Boundedness and liminf: A sequence $(a_n{)}_{n\ge 1}$ is bounded below if and only if ${\mathrm{liminf}}_{n\to \infty }{a}_n>-\infty$.

• Convergence to $\infty$ and liminf: A sequence $(a_n{)}_{n\ge 1}$ converges to $\infty$ if and only if ${\mathrm{liminf}}_{n\to \infty }{a}_n=\infty$.

• Convergence to $-\infty$ and limsup: A sequence $(a_n{)}_{n\ge 1}$ converges to $-\infty$ if and only if ${\mathrm{limsup}}_{n\to \infty }{a}_n=-\infty$.

• Characterization of Convergence via liminf and limsup: A sequence $(a_n{)}_{n\ge 1}$ converges to a limit $L\in \mathbb{R}$ if and only if:

  1. It is bounded.
  2. ${\mathrm{liminf}}_{n\to \infty }{a}_n={\mathrm{limsup}}_{n\to \infty }{a}_n=L$.

Proof of the Convergence Characterization

Let’s prove the last property, which is particularly important: A bounded sequence $(a_n{)}_{n\ge 1}$ converges if and only if ${\mathrm{liminf}}_{n\to \infty }{a}_n={\mathrm{limsup}}_{n\to \infty }{a}_n$.

Proof:

• ($\Rightarrow$) If $(a_n)$ converges to $L$, then $\mathrm{liminf}{a}_n=\mathrm{limsup}{a}_n=L$. If ${\lim }_{n\to \infty }{a}_n=L$, then for any $ϵ>0$, there exists $N\in \mathbb{N}$ such that for all $n>N$, $L-ϵ<a_n<L+ϵ$. This implies that for $n>N$, $\inf \{a_k\mid k\ge n\}\ge L-ϵ$ and $\sup \{a_k\mid k\ge n\}\le L+ϵ$. Therefore, ${\mathrm{liminf}}_{n\to \infty }{a}_n\ge L-ϵ$ and ${\mathrm{limsup}}_{n\to \infty }{a}_n\le L+ϵ$. Since this holds for any $ϵ>0$, we have ${\mathrm{liminf}}_{n\to \infty }{a}_n\ge L$ and ${\mathrm{limsup}}_{n\to \infty }{a}_n\le L$. Combining with the general inequality ${\mathrm{liminf}}_{n\to \infty }{a}_n\le {\mathrm{limsup}}_{n\to \infty }{a}_n$, we get ${\mathrm{liminf}}_{n\to \infty }{a}_n={\mathrm{limsup}}_{n\to \infty }{a}_n=L$.

• ($\Leftarrow$) If ${\mathrm{liminf}}_{n\to \infty }{a}_n={\mathrm{limsup}}_{n\to \infty }{a}_n=L$, then $(a_n)$ converges to $L$. Let $L={\mathrm{liminf}}_{n\to \infty }{a}_n={\mathrm{limsup}}_{n\to \infty }{a}_n$. Since ${\lim }_{n\to \infty }(\inf \{a_k\mid k\ge n\})=L$, for any $ϵ>0$, there exists $N_1\in \mathbb{N}$ such that for all $n>N_1$, $\inf \{a_k\mid k\ge n\}>L-ϵ$. Similarly, since ${\lim }_{n\to \infty }(\sup \{a_k\mid k\ge n\})=L$, for any $ϵ>0$, there exists $N_2\in \mathbb{N}$ such that for all $n>N_2$, $\sup \{a_k\mid k\ge n\}<L+ϵ$. Let $N=\max \{N_1,N_2\}$. For all $n>N$, we have: $L-ϵ<\inf \{a_k\mid k\ge n\}\le a_n\le \sup \{a_k\mid k\ge n\}<L+ϵ$ Thus, for all $n>N$, $L-ϵ<a_n<L+ϵ$, which means $|a_n-L|<ϵ$. Therefore, ${\lim }_{n\to \infty }{a}_n=L$.

Using $ϵ$ Neighborhoods to Characterize liminf and limsup

For a bounded sequence $(a_n{)}_{n\ge 1}$, for every $ϵ>0$, there exists $N\in \mathbb{N}$ such that for all $n>N$:

$a_n\in ({\mathrm{liminf}}_{n\to \infty }{a}_n-ϵ,{\mathrm{limsup}}_{n\to \infty }{a}_n+ϵ)$

Proof:

Choose $N$ such that for all $n>N$, $b_n=\inf \{a_k\mid k\ge n\}>{\mathrm{liminf}}_{n\to \infty }{a}_n-ϵ$ and $c_n=\sup \{a_k\mid k\ge n\}<{\mathrm{limsup}}_{n\to \infty }{a}_n+ϵ$.

By definition of infimum and supremum, for all $n\ge N$, we know that $b_n=\inf \{a_k\mid k\ge n\}\le a_n\le \sup \{a_k\mid k\ge n\}=c_n$.

Combining these inequalities, we get for all $n>N$:

${\mathrm{liminf}}_{n\to \infty }{a}_n-ϵ<b_n\le a_n\le c_n<{\mathrm{limsup}}_{n\to \infty }{a}_n+ϵ$

Therefore, $a_n\in ({\mathrm{liminf}}_{n\to \infty }{a}_n-ϵ,{\mathrm{limsup}}_{n\to \infty }{a}_n+ϵ)$ for all $n>N$.

The Cauchy Criterion for Convergence

The Need for a Limit-Independent Criterion

The definition of convergence requires knowing the limit $L$ in advance. The Cauchy Criterion provides a way to determine if a sequence converges without knowing its limit. This is incredibly useful in situations where the limit is not easily guessable.

Statement of the Cauchy Criterion

Theorem (Cauchy Criterion): A real sequence $(a_n{)}_{n\ge 1}$ is convergent if and only if for every $ϵ>0$, there exists $N\in \mathbb{N}$ such that for all $m,n>N$, we have $|a_m-a_n|<ϵ$.

A sequence satisfying this condition is called a Cauchy sequence. Thus, the Cauchy Criterion states that a sequence converges if and only if it is a Cauchy sequence.

Intuition: Terms Getting Closer to Each Other

The Cauchy Criterion essentially says that a sequence converges if and only if its terms become arbitrarily close to each other as $n$ becomes large. If the terms are “crowding together,” they must be approaching some limit.

Proof of the Cauchy Criterion

($\Rightarrow$) If $(a_n)$ converges to $A$, then $(a_n)$ is a Cauchy sequence.

Assume ${\lim }_{n\to \infty }{a}_n=A$. Let $ϵ>0$. There exists $N\in \mathbb{N}$ such that for all $n>N$, $|a_n-A|<\frac{ϵ}{2}$.

For any $m,n>N$, using the triangle inequality:

$|a_m-a_n|=|(a_m-A)-(a_n-A)|\le |a_m-A|+|a_n-A|<\frac{ϵ}{2}+\frac{ϵ}{2}=ϵ$

Thus, for all $m,n>N$, $|a_m-a_n|<ϵ$, so $(a_n)$ is a Cauchy sequence.

($\Leftarrow$) If $(a_n)$ is a Cauchy sequence, then $(a_n)$ converges.

This direction is more involved and typically requires two steps:

1. Show that every Cauchy sequence is bounded. Since $(a_n)$ is Cauchy, for $ϵ=1$, there exists $N$ such that for all $m,n>N$, $|a_m-a_n|<1$. In particular, for all $n>N$, $|a_n-a_{N+1}|<1$, so $|a_n|<|a_{N+1}|+1$. Let $M=\max \{|a_1|,|a_2|,\dots ,|a_N|,|a_{N+1}|+1\}$. Then $|a_n|\le M$ for all $n$, so $(a_n)$ is bounded.

2. Use the Bolzano-Weierstrass Theorem (which we will discuss next) to show that a bounded Cauchy sequence must converge. Since $(a_n)$ is bounded, by the Bolzano-Weierstrass Theorem, it has a convergent subsequence $(a_{n_k}{)}_{k\ge 1}$ converging to some limit $A$. We now need to show that the entire sequence $(a_n)$ converges to $A$.

   Let $ϵ>0$. Since $(a_n)$ is Cauchy, there exists $N_1$ such that for all $m,n>N_1$, $|a_m-a_n|<\frac{ϵ}{2}$. Since $(a_{n_k})$ converges to $A$, there exists $K$ such that for all $k>K$, $|a_{n_k}-A|<\frac{ϵ}{2}$. Choose $k$ large enough so that $n_k>\max \{N_1,K\}$. Let $N=n_k$. Then for any $n>N$ (so $n>N_1$ and $n_k>N_1$):

   $|a_n-A|=|(a_n-a_{n_k})+(a_{n_k}-A)|\le |a_n-a_{n_k}|+|a_{n_k}-A|<\frac{ϵ}{2}+\frac{ϵ}{2}=ϵ$

   Thus, for all $n>N$, $|a_n-A|<ϵ$, so ${\lim }_{n\to \infty }{a}_n=A$.

The Bolzano-Weierstrass Theorem: Existence of Convergent Subsequences

Subsequences: Zooming in on Parts of a Sequence

Definition (Subsequence): A subsequence of a sequence $(a_n{)}_{n\ge 1}$ is a sequence of the form $(a_{\varphi (n)}{)}_{n\ge 1}$, where $\varphi :{\mathbb{N}}^{\ast }\to {\mathbb{N}}^{\ast }$ is a strictly increasing function. Essentially, we are picking out an infinite number of terms from the original sequence, keeping their original order.

Statement of the Bolzano-Weierstrass Theorem

Theorem (Bolzano-Weierstrass Theorem): Every bounded real sequence has at least one convergent subsequence.

This is a fundamental result in real analysis. It guarantees that even if a bounded sequence does not converge itself, we can always find a subsequence that does converge.

Intuition: Crowding in a Bounded Range

If a sequence is bounded, its terms are confined to a finite interval. Intuitively, if we have infinitely many terms within a bounded interval, there must be some “crowding” happening, allowing us to extract a convergent subsequence.

Proof Idea: Nested Interval Construction

The proof of the Bolzano-Weierstrass Theorem typically uses the Nested Interval Theorem and a process of repeatedly bisecting intervals. We will construct a nested sequence of closed intervals $I_1\supseteq I_2\supseteq I_3\supseteq \dots$ such that each interval $I_n$ contains infinitely many terms of the sequence $(a_n)$.

1. Start with a bounded interval: Since $(a_n)$ is bounded, there exists a closed interval $I_1=[c_1,d_1]$ that contains all terms of the sequence.

2. Bisect and choose: Bisect $I_1$ into two closed intervals of equal length. At least one of these halves must contain infinitely many terms of $(a_n)$ (because $I_1$ contains infinitely many). Let $I_2$ be such a half.

3. Repeat: Bisect $I_2$ into two closed intervals of equal length. Choose a half, call it $I_3$, that contains infinitely many terms of $(a_n)$. Continue this process inductively to obtain a nested sequence of closed intervals $I_1\supseteq I_2\supseteq I_3\supseteq \dots$.

4. Apply Nested Interval Theorem: By the Nested Interval Theorem (Cauchy-Cantor Theorem, which we will discuss next), the intersection ${\bigcap }_{n=1}^{\infty }{I}_n$ is non-empty. Since the length of $I_n$ shrinks to zero as $n\to \infty$, the intersection contains exactly one point, let’s call it $c$.

5. Construct a convergent subsequence: We can construct a subsequence $(a_{\varphi (n)})$ that converges to $c$. Choose $\varphi (1)$ such that $a_{\varphi (1)}\in I_1$. Choose $\varphi (2)>\varphi (1)$ such that $a_{\varphi (2)}\in I_2$. In general, choose $\varphi (n)>\varphi (n-1)$ such that $a_{\varphi (n)}\in I_n$. This is possible because each $I_n$ contains infinitely many terms.

   Since $a_{\varphi (n)}\in I_n$ and $c\in I_n$, we have $|a_{\varphi (n)}-c|\le \text{length}(I_n)=\frac{1}{2^{n-1}}\text{length}(I_1)$. As $n\to \infty$, $\text{length}(I_n)\to 0$, so ${\lim }_{n\to \infty }{a}_{\varphi (n)}=c$. Thus, we have found a convergent subsequence.

The Nested Interval Theorem (Cauchy-Cantor Theorem)

Statement of the Theorem

Theorem (Nested Interval Theorem, Cauchy-Cantor Theorem): Let $(I_n{)}_{n\ge 1}$ be a sequence of non-empty closed and bounded intervals in $\mathbb{R}$ such that $I_1\supseteq I_2\supseteq I_3\supseteq \dots$ (nested intervals). Then, the intersection of all intervals is non-empty:

${\bigcap }_{n=1}^{\infty }{I}_n\ne \varnothing$

Furthermore, if the lengths of the intervals $L(I_n)$ converge to 0 as $n\to \infty$, then the intersection contains exactly one point.

Proof of the Nested Interval Theorem

Let $I_n=[a_n,b_n]$, where $a_n,b_n\in \mathbb{R}$ and $a_n\le b_n$. Since $I_n\supseteq I_{n+1}$, we have $a_n\le a_{n+1}\le b_{n+1}\le b_n$.

• The sequence $(a_n{)}_{n\ge 1}$ is non-decreasing and bounded above by $b_1$ (and any $b_n$). Therefore, $(a_n)$ converges to some limit $c={\lim }_{n\to \infty }{a}_n$.
• The sequence $(b_n{)}_{n\ge 1}$ is non-increasing and bounded below by $a_1$ (and any $a_n$). Therefore, $(b_n)$ converges to some limit $d={\lim }_{n\to \infty }{b}_n$.

By the comparison law for limits, since $a_n\le b_n$ for all $n$, we have $c={\lim }_{n\to \infty }{a}_n\le {\lim }_{n\to \infty }{b}_n=d$.

Now, we show that $c\in {\bigcap }_{n=1}^{\infty }{I}_n$. For any $m\in \mathbb{N}$, and for all $n\ge m$, we have $a_m\le a_n\le b_n\le b_m$. Taking the limit as $n\to \infty$, we get $a_m\le {\lim }_{n\to \infty }{a}_n=c\le {\lim }_{n\to \infty }{b}_n=d\le b_m$. Thus, $a_m\le c\le b_m$, which means $c\in [a_m,b_m]=I_m$ for all $m\in \mathbb{N}$. Therefore, $c\in {\bigcap }_{n=1}^{\infty }{I}_n$, and the intersection is non-empty.

If ${\lim }_{n\to \infty }L(I_n)={\lim }_{n\to \infty }(b_n-a_n)=0$, then $d-c={\lim }_{n\to \infty }{b}_n-{\lim }_{n\to \infty }{a}_n={\lim }_{n\to \infty }(b_n-a_n)=0$, so $c=d$. In this case, the intersection contains exactly one point, $c$.

Example: Intervals that are not Closed or Bounded

1. Non-closed intervals: Let $X_n=(0,\frac{1}{n}]$ for $n\ge 1$. Then ${\bigcap }_{n=1}^{\infty }{X}_n=\varnothing$. Each $X_n$ is bounded but not closed at 0.

2. Non-bounded intervals: Let $X_n=[n,\infty )$ for $n\ge 1$. Then ${\bigcap }_{n=1}^{\infty }{X}_n=\varnothing$. Each $X_n$ is closed but not bounded above.

These examples highlight that both closedness and boundedness are necessary conditions for the Nested Interval Theorem to hold.

This lecture has covered essential concepts and theorems related to sequences, providing you with powerful tools for analyzing their convergence and behavior. We have seen how the Sandwich Lemma, Cauchy Criterion, Bolzano-Weierstrass Theorem, and Nested Interval Theorem play crucial roles in real analysis.

Continue here: 06 Accumulation Points, Sequences in Reals and Complex, and Series