Introduction to Linear Algebra (Gilbert Strang)

This will be my summary for: “Introduction to Linear Algebra - Gilbert Strang”.

1. Vectors and Matrices

Linear algebra is about vectors and matrices. These are the basic objects that we can add, subtract and multiply (when there shapes match).

1.1 Vectors and Linear Combinations

Linear Algebra begins with vectors $v,w$ and their linear combinations $cv+dw$. This takes you to two or more dimensions.

Linear combinations use two basic operations: 1) Scalar multiplication Scaling and 2) Adding vectors Addition.

The concept of linear combinations opens up two key questions:

1. Describe all the combinations. Which space do they fill?
2. Find the values for the coefficients which produce a specific combinations.

In ${\mathbb{R}}^2$ we can answer these questions quite simply:

1. $cv$ for all c fills an infinite line in the xy-plane. If w happens to be on that line, then combining linear combinations of v and w won’t “escape” that line. If w is independent of v then the entire xy-plane can be reached as a linear combination of v and w.
2. Two equations and two variables → go solve them.

1.2 Lengths and Angles from Dot Products

Dot Products

One of the most useful multiplications of vectors is the dot product. The dot product (aka inner product) of a two vectors $v\cdot w$ is $v_1{w}_1+v_2{w}_2+\cdots +v_n{w}_n$ (See Dot Product (Scalar Product))

The dot product $v\cdot v$ tells us the squared length: $||v|{|}^2=v_1^2+v_2^2+\cdots +v_n^2$.

Length/Magnitude

The length $||v||$ (aka magnitude) of a vector is $\sqrt{v_1^2+v_2^2+\cdots +v_n^2}$ (See Why Does Squaring and Taking the Square Root Give the Length?). A diagonal in n dimensions has the length $\sqrt{n}$.

Unit Vector

We use the words unit vector are used when the length of a vector = 1. Otherwise: $u=\frac{v}{||v||}$ gives us the unit vector. (See: Unit Vectors)

For example, $\mathbf{i}=\left(\begin{array}{c}1\\ 0\end{array}\right)$ and $\mathbf{u}=\left(\begin{array}{c}\cos \theta \\ \sin \theta \end{array}\right)$. Notice that $u$ is a unit vector because $\mathbf{u}\cdot \mathbf{u}=1$.

Perpendicular Vectors

Suppose the angle between v and w is 90°. It’s cosine is zero. $v\cdot w=0$ for perpendicular vectors.

Then we can use the Pythagorean Theorem and deduce that $v\cdot w$ must be 0. The formula is saying take the hypotenuse which is $v+w$. Now take the length of that and square it. This must be equal to the individual side lengths (vector magnitudes) squared summed up. Expanding the formula reveals that the $v\cdot w$ terms all need to be zero.

Cosine of the angle: Let us take the unit vectors $v=(1,0)$ and $u=(\cos \theta ,\sin \theta )$ then $v\cdot u=\cos \theta$. We’ll try to connect the dot product to the angle between vectors.

Angle Between Two Vectors

Let the unit vectors $v=(\cos a,\sin a)$ and $w=(\cos b,\sin b)$ Then we have $v\cdot w=\cos a\cos b+\sin a\sin b=\cos (b-a)$ (Using trigonometric identities).