Linear independence

Linear independence is a central concept in linear algebra. Two or more vectors are said to be linearly independent if none of them can be written as a linear combination of the others. On the contrary, if at least one of them can be written as a linear combination of the others, then they are said to be linearly dependent.

In the remainder of this lecture we will give a formal definition of linear independence, we will explain its meaning and we will provide some examples.

Table of contents

Linearly dependent vectors
Linearly independent vectors
Solved exercises

Linearly dependent vectors

Let us start with a formal definition of linear dependence.

Definition Let $S$ be a linear space. Some vectors $x_{1}, \dots, x_{n} \in S$ are said to be linearly dependent if and only if there exist $n$ scalars $α_{1}, \dots, α_{n}$ such that

α_{1} x_{1} + \dots + α_{n} x_{n} = 0

and at least one of the $n$ scalars $α_{1}, \dots, α_{n}$ is different from zero.

The requirement that at least one scalar be different from zero is fundamental.

First of all, without this requirement the definition would be trivial: we could always choose

α_{1} = \dots = α_{n} = 0

and obtain as a result

α_{1} x_{1} + \dots + α_{n} x_{n} = 0 \cdot x_{1} + \dots + 0 \cdot x_{n} = 0

for any set of $n$ vectors.

Secondly, if one of the coefficients of the linear combination is different from zero (suppose, without loss of generality, it is $α_{1}$ ), then we can write

x_{1} = - \frac{α _{2}}{α _{1}} x_{2} - \dots - \frac{α _{n}}{α _{1}} x_{n}

that is, $x_{1}$ is a linear combination of the vectors $x_{2}, \dots, x_{n}$ with coefficients $- \frac{α _{2}}{α _{1}}, \dots, - \frac{α _{n}}{α _{1}}$ . This fact motivates the informal definition of linear dependence we have given in the introduction above: two or more vectors are linearly dependent if at least one of them can be written as a linear combination of the others.

The assumption $α_{1} \neq = 0$ is without loss of generality because we can always change the order of the vectors and assign the first position to a vector corresponding to a non-zero coefficient (by assumption there exists at least one such vector).

Example Let $x_{1}$ and $x_{2}$ be $2 \times 1$ column vectors defined as follows.

x_{1} = 12 x_{2} = 24

The linear combination

- 2 x_{1} + x_{2}

gives as a result the zero vector because

- 2 x_{1} + x_{2} = - 212 + 24 = - 2 \cdot 1 - 2 \cdot 2 + 24 = - 2 - 4 + 24 = - 2 + 2 - 4 + 4 = 00

As a consequence, $x_{1}$ and $x_{2}$ are linearly dependent.

Linearly independent vectors

It is now straightforward to give a definition of linear independence.

Definition Let $S$ be a linear space. Some vectors $x_{1}, \dots, x_{n} \in S$ are said to be linearly independent if and only if they are not linearly dependent.

It follows from this definition that, in the case of linear independence,

α_{1} x_{1} + α_{2} x_{2} + \dots + α_{n} x_{n} = 0

implies

α_{1} = \dots = α_{n} = 0

In other words, when the vectors are linearly independent, their only linear combination that gives the zero vector as a result has all coefficients equal to zero.

Example Let $x_{1}$ and $x_{2}$ be $2 \times 1$ column vectors defined as follows.

x_{1} = 10 x_{2} = 01

Consider a linear combination of these two vectors with coefficients $α_{1}$ and $α_{2}$ :

α_{1} x_{1} + α_{2} x_{2}

This is equal to

α_{1} x_{1} + α_{2} x_{2} = α_{1} 10 + α_{2} 01 = α_{1} \cdot 1 0 + 0 α_{2} \cdot 1 = α_{1} 0 + 0 α_{2} = α_{1} + 0 0 + α_{2} = α_{1} α_{2}

Therefore, we have that

α_{1} x_{1} + α_{2} x_{2} = 0

if and only if

α_{1} α_{2} = 00

that is, if and only if $α_{1} = α_{2} = 0$ . As a consequence, the two vectors are linearly independent.

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Define the following $2 \times 1$ vectors:

A_{1} = 11 A_{2} = 21

Are $A_{1}$ and $A_{2}$ linearly independent?

Solution

Consider a linear combination with coefficients $α_{1}$ and $α_{2}$ :

α_{1} A_{1} + α_{2} A_{2} = α_{1} 11 + α_{2} 21 = α_{1} \cdot 1 α_{1} \cdot 1 + α_{2} \cdot 2 α_{2} \cdot 1 = α_{1} + 2 α_{2} α_{1} + α_{2}

Such a linear combination gives as a result the zero vector if and only if

α_{1} + 2 α_{2} = 0 α_{1} + α_{2} = 0

that is, if and only if the two coefficients $α_{1}$ and $α_{2}$ solve the system of linear equations

⎩ ⎨ ⎧ α_{1} + 2 α_{2} = 0 α_{1} + α_{2} = 0

This system can be solved as follows. From the second equation, we obtain

α_{1} = - α_{2}

which, substituted in the first equation, gives

- α_{2} + 2 α_{2} = 0

Thus, $α_{2} = 0$ and $α_{1} = 0$ . Therefore, the only linear combination of $A_{1}$ and $A_{2}$ giving the zero vector as a result has all coefficients equal to zero. This means that $A_{1}$ and $A_{2}$ are linearly independent.

Exercise 2

Let $A_{1}$ , $A_{2}$ and $A_{3}$ be $3 \times 1$ vectors defined as follows:

A_{1} = 123 A_{2} = 001 A_{3} = 002

Why are these vectors linearly dependent?

Solution

Notice that the vector $A_{3}$ is a scalar multiple of $A_{2}$ :

A_{3} = 2 A_{2}

2 A_{2} - A_{3} = 0

As a consequence, a linear combination of $A_{1}$ , $A_{2}$ and $A_{3}$ , with coefficients $α_{1} = 0$ , $α_{2} = 2$ and $α_{3} = - 1$ , gives as a result

α_{1} A_{1} + α_{2} A_{2} + α_{3} A_{3} = 0 \cdot A_{1} + 2 \cdot A_{2} - 1 \cdot A_{3} = 2 A_{2} - A_{3} = 0

Thus, there exists a linear combination of the three vectors such that the coefficients of the combination are not all equal to zero, but the result of the combination is equal to the zero vector. This means that the three vectors are linearly dependent.

Exercise 3

Let $x$ be a real number. Define the following $2 \times 1$ vectors:

A_{1} = 1 + x x A_{2} = 2 + x x

Are $A_{1}$ and $A_{2}$ linearly independent?

Solution

Take a linear combination with coefficients $α_{1}$ and $α_{2}$ :

α_{1} A_{1} + α_{2} A_{2} = α_{1} 1 + x x + α_{2} 2 + x x = α_{1} \cdot (1 + x) α_{1} \cdot x + α_{2} \cdot (2 + x) α_{2} \cdot x = α_{1} + 2 α_{2} + α_{1} x + α_{2} x α_{1} x + α_{2} x

This linear combination is equal to the zero vector if and only if

α_{1} + 2 α_{2} + α_{1} x + α_{2} x = 0 α_{1} x + α_{2} x = 0

that is, if and only if the two coefficients $α_{1}$ and $α_{2}$ solve the system of linear equations

⎩ ⎨ ⎧ α_{1} + 2 α_{2} + α_{1} x + α_{2} x = 0 α_{1} x + α_{2} x = 0

A solution to this system can be found as follows. We subtract the second equation from the first and obtain

α_{1} + 2 α_{2} = 0

α_{1} = - 2 α_{2}

By substitution into the second equation, we get

- 2 α_{2} x + α_{2} x = 0

α_{2} x = 0

Now, there are two possible cases. If $x \neq = 0$ (first case), then $α_{2} = 0$ and, as a consequence, $α_{1} = 0$ . Thus, in this case the only linear combination of $A_{1}$ and $A_{2}$ giving the zero vector as a result has all coefficients equal to zero. This means that $A_{1}$ and $A_{2}$ are linearly independent. If instead $x = 0$ (second case), then any value of $α_{2}$ will satisfy the equation

α_{2} x = 0

Choose a number different from zero and denote it by $s$ . Then, the system of linear equations will be solved by $α_{2} = s$ and $α_{1} = - 2 s$ . Thus, in this case there are infinite linear combinations with at least one coefficient different from zero that give the zero vector as a result (a different combination for each choice of $s$ ). This means that $A_{1}$ and $A_{2}$ are linearly dependent.

How to cite

Please cite as:

Taboga, Marco (2021). "Linear independence", Lectures on matrix algebra. https://www.statlect.com/matrix-algebra/linear-independence.