Linear independence

From Wikipedia, the free encyclopedia

Jump to navigation Jump to search

For linear dependence of random variables, see Covariance.

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Linear independence" – news · newspapers · books · scholar · JSTOR (January 2019) (Learn how and when to remove this template message)

Linearly independent vectors in ${\displaystyle \mathbb {R} ^{3}}$

Linearly dependent vectors in a plane in ${\displaystyle \mathbb {R} ^{3}}$.

In the theory of vector spaces, a set of vectors is said to be linearly dependent if at least one of the vectors in the set can be defined as a linear combination of the others; if no vector in the set can be written in this way, then the vectors are said to be linearly independent. These concepts are central to the definition of dimension.^[1]

A vector space can be of finite-dimension or infinite-dimension depending on the number of linearly independent basis vectors. The definition of linear dependence and the ability to determine whether a subset of vectors in a vector space is linearly dependent are central to determining a basis for a vector space.

Contents

• 1 Definition
  • 1.1 Infinite dimensions
• 2 Geometric meaning
• 3 Evaluating linear independence
  • 3.1 Vectors in R²
  • 3.2 Vectors in R⁴
  • 3.3 Alternative method using determinants
  • 3.4 More vectors than dimensions
• 4 Natural basis vectors
  • 4.1 Proof
• 5 Linear independence of basis functions
  • 5.1 Proof
• 6 Space of linear dependencies
• 7 See also
• 8 References
• 9 External links

Definition[edit]

A sequence of vectors ${\displaystyle ({\vec {v_{1}}},{\vec {v_{2}}},\dots ,{\vec {v_{k}}})}$ from a vector space V is said to be linearly dependent, if there exist scalars ${\displaystyle a_{1},a_{2},\dots ,a_{k}}$ , not all zero, such that

${\displaystyle a_{1}{\vec {v_{1}}}+a_{2}{\vec {v_{2}}}+\cdots +a_{k}{\vec {v_{k}}}={\vec {0}},}$

where ${\displaystyle {\vec {0}}}$ denotes the zero vector.

Notice that if not all of the scalars are zero, then at least one is non-zero, say ${\displaystyle a_{1}}$, in which case this equation can be written in the form

${\displaystyle {\vec {v_{1}}}={\frac {-a_{2}}{a_{1}}}{\vec {v_{2}}}+\cdots +{\frac {-a_{k}}{a_{1}}}{\vec {v_{k}}}.}$

Thus, ${\displaystyle {\vec {v_{1}}}}$ is shown to be a linear combination of the remaining vectors.

A sequence of vectors ${\displaystyle ({\vec {v_{1}}},{\vec {v_{2}}},\dots ,{\vec {v_{n}}})}$ is said to be linearly independent if the equation

${\displaystyle a_{1}{\vec {v_{1}}}+a_{2}{\vec {v_{2}}}+\cdots +a_{n}{\vec {v_{n}}}={\vec {0}},}$

can only be satisfied by ${\displaystyle a_{i}=0}$ for ${\displaystyle i=1,\dots ,n}$. This implies that no vector in the sequence can be represented as a linear combination of the remaining vectors in the sequence. In other words, a sequence of vectors is linearly independent if the only representation of ${\displaystyle {\vec {0}}}$ as a linear combination of its vectors is the trivial representation in which all the scalars ${\displaystyle a_{i}}$ are zero.^[2] Even more concisely, a sequence of vectors is linear independent if and only if ${\displaystyle {\vec {0}}}$ can be represented as a linear combination of its vectors in a unique way.

The alternate definition, that a sequence of vectors is linearly dependent if and only if some vector in that sequence can be written as a linear combination of the other vectors, is only useful when the sequence contains two or more vectors. When the sequence contains no vectors or only one vector, the original definition is used.

Infinite dimensions[edit]

In order to allow the number of linearly independent vectors in a vector space to be countably infinite, it is useful to define linear dependence as follows. More generally, let V be a vector space over a field K, and let {v_i | i∈I} be a family of elements of V. The family is linearly dependent over K if there exists a finite family {a_j | j ∈ J} of elements of K, all non-zero, such that

${\displaystyle \sum _{j\in J}a_{j}v_{j}=0.}$

A set X of elements of V is linearly independent if the corresponding family {x}_x∈X is linearly independent. Equivalently, a family is dependent if a member is in the closure of the linear span of the rest of the family, i.e., a member is a linear combination of the rest of the family. The trivial case of the empty family must be regarded as linearly independent for theorems to apply.

A set of vectors which is linearly independent and spans some vector space, forms a basis for that vector space. For example, the vector space of all polynomials in x over the reals has the (infinite) subset {1, x, x², ...} as a basis.

Geometric meaning[edit]

A geographic example may help to clarify the concept of linear independence. A person describing the location of a certain place might say, "It is 3 miles north and 4 miles east of here." This is sufficient information to describe the location, because the geographic coordinate system may be considered as a 2-dimensional vector space (ignoring altitude and the curvature of the Earth's surface). The person might add, "The place is 5 miles northeast of here." Although this last statement is true, it is not necessary.

In this example the "3 miles north" vector and the "4 miles east" vector are linearly independent. That is to say, the north vector cannot be described in terms of the east vector, and vice versa. The third "5 miles northeast" vector is a linear combination of the other two vectors, and it makes the set of vectors linearly dependent, that is, one of the three vectors is unnecessary.

Also note that if altitude is not ignored, it becomes necessary to add a third vector to the linearly independent set. In general, n linearly independent vectors are required to describe all locations in n-dimensional space.

Evaluating linear independence[edit]

Vectors in R²[edit]

Three vectors: Consider the set of vectors v₁ = (1, 1), v₂ = (−3, 2) and v₃ = (2, 4), then the condition for linear dependence seeks a set of non-zero scalars, such that

${\displaystyle a_{1}{\begin{Bmatrix}1\\1\end{Bmatrix}}+a_{2}{\begin{Bmatrix}-3\\2\end{Bmatrix}}+a_{3}{\begin{Bmatrix}2\\4\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}},}$

or

${\displaystyle {\begin{bmatrix}1&-3&2\\1&2&4\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\\a_{3}\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}}.}$

Row reduce this matrix equation by subtracting the first row from the second to obtain,

${\displaystyle {\begin{bmatrix}1&-3&2\\0&5&2\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\\a_{3}\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}}.}$

Continue the row reduction by (i) dividing the second row by 5, and then (ii) multiplying by 3 and adding to the first row, that is

${\displaystyle {\begin{bmatrix}1&0&16/5\\0&1&2/5\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\\a_{3}\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}}.}$

We can now rearrange this equation to obtain

${\displaystyle {\begin{bmatrix}1&0\\0&1\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\end{Bmatrix}}={\begin{Bmatrix}a_{1}\\a_{2}\end{Bmatrix}}=-a_{3}{\begin{Bmatrix}16/5\\2/5\end{Bmatrix}}.}$

which shows that non-zero a_i exist such that v₃ = (2, 4) can be defined in terms of v₁ = (1, 1), v₂ = (−3, 2). Thus, the three vectors are linearly dependent.

Two vectors: Now consider the linear dependence of the two vectors v₁ = (1, 1), v₂ = (−3, 2), and check,

${\displaystyle a_{1}{\begin{Bmatrix}1\\1\end{Bmatrix}}+a_{2}{\begin{Bmatrix}-3\\2\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}},}$

or

${\displaystyle {\begin{bmatrix}1&-3\\1&2\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}}.}$

The same row reduction presented above yields,

${\displaystyle {\begin{bmatrix}1&0\\0&1\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\end{Bmatrix}}={\begin{Bmatrix}0\\0\end{Bmatrix}}.}$

This shows that a_i = 0, which means that the vectors v₁ = (1, 1) and v₂ = (−3, 2) are linearly independent.

Vectors in R⁴[edit]

In order to determine if the three vectors in R⁴,

${\displaystyle \mathbf {v} _{1}={\begin{Bmatrix}1\\4\\2\\-3\end{Bmatrix}},\mathbf {v} _{2}={\begin{Bmatrix}7\\10\\-4\\-1\end{Bmatrix}},\mathbf {v} _{3}={\begin{Bmatrix}-2\\1\\5\\-4\end{Bmatrix}}.}$

are linearly dependent, form the matrix equation,

${\displaystyle {\begin{bmatrix}1&7&-2\\4&10&1\\2&-4&5\\-3&-1&-4\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\\a_{3}\end{Bmatrix}}={\begin{Bmatrix}0\\0\\0\\0\end{Bmatrix}}.}$

Row reduce this equation to obtain,

${\displaystyle {\begin{bmatrix}1&7&-2\\0&-18&9\\0&0&0\\0&0&0\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\\a_{3}\end{Bmatrix}}={\begin{Bmatrix}0\\0\\0\\0\end{Bmatrix}}.}$

Rearrange to solve for v₃ and obtain,

${\displaystyle {\begin{bmatrix}1&7\\0&-18&\end{bmatrix}}{\begin{Bmatrix}a_{1}\\a_{2}\end{Bmatrix}}=-a_{3}{\begin{Bmatrix}-2\\9\end{Bmatrix}}.}$

This equation is easily solved to define non-zero a_i,

${\displaystyle a_{1}=-3a_{3}/2,a_{2}=a_{3}/2,}$

where a₃ can be chosen arbitrarily. Thus, the vectors v₁, v₂ and v₃ are linearly dependent.

Alternative method using determinants[edit]

An alternative method relies on the fact that n vectors in ${\displaystyle \mathbb {R} ^{n}}$ are linearly independent if and only if the determinant of the matrix formed by taking the vectors as its columns is non-zero.

In this case, the matrix formed by the vectors is

${\displaystyle A={\begin{bmatrix}1&-3\\1&2\end{bmatrix}}.}$

We may write a linear combination of the columns as

${\displaystyle A\Lambda ={\begin{bmatrix}1&-3\\1&2\end{bmatrix}}{\begin{bmatrix}\lambda _{1}\\\lambda _{2}\end{bmatrix}}.}$

We are interested in whether AΛ = 0 for some nonzero vector Λ. This depends on the determinant of A, which is

${\displaystyle \det A=1\cdot 2-1\cdot (-3)=5\neq 0.}$

Since the determinant is non-zero, the vectors (1, 1) and (−3, 2) are linearly independent.

Otherwise, suppose we have m vectors of n coordinates, with m < n. Then A is an n×m matrix and Λ is a column vector with m entries, and we are again interested in AΛ = 0. As we saw previously, this is equivalent to a list of n equations. Consider the first m rows of A, the first m equations; any solution of the full list of equations must also be true of the reduced list. In fact, if 〈i₁,...,i_m〉 is any list of m rows, then the equation must be true for those rows.

${\displaystyle A_{{\langle i_{1},\dots ,i_{m}}\rangle }\Lambda =\mathbf {0} .}$

Furthermore, the reverse is true. That is, we can test whether the m vectors are linearly dependent by testing whether

${\displaystyle \det A_{{\langle i_{1},\dots ,i_{m}}\rangle }=0}$

for all possible lists of m rows. (In case m = n, this requires only one determinant, as above. If m > n, then it is a theorem that the vectors must be linearly dependent.) This fact is valuable for theory; in practical calculations more efficient methods are available.

More vectors than dimensions[edit]

If there are more vectors than dimensions, the vectors are linearly dependent. This is illustrated in the example above of three vectors in R².

Natural basis vectors[edit]

Let V = Rⁿ and consider the following elements in V, known as the natural basis vectors:

${\displaystyle {\begin{matrix}\mathbf {e} _{1}&=&(1,0,0,\ldots ,0)\\\mathbf {e} _{2}&=&(0,1,0,\ldots ,0)\\&\vdots \\\mathbf {e} _{n}&=&(0,0,0,\ldots ,1).\end{matrix}}}$

Then e₁, e₂, ..., e_n are linearly independent.

Proof[edit]

Suppose that a₁, a₂, ..., a_n are elements of R such that

${\displaystyle a_{1}\mathbf {e} _{1}+a_{2}\mathbf {e} _{2}+\cdots +a_{n}\mathbf {e} _{n}=\mathbf {0} .}$

Since

${\displaystyle a_{1}\mathbf {e} _{1}+a_{2}\mathbf {e} _{2}+\cdots +a_{n}\mathbf {e} _{n}=(a_{1},a_{2},\ldots ,a_{n}),}$

then a_i = 0 for all i in {1, ..., n}.

Linear independence of basis functions[edit]

Let ${\displaystyle V}$ be the vector space of all differentiable functions of a real variable ${\displaystyle t}$. Then the functions ${\displaystyle e^{t}}$ and ${\displaystyle e^{2t}}$ in ${\displaystyle V}$ are linearly independent.

Proof[edit]

Suppose ${\displaystyle a}$ and ${\displaystyle b}$ are two real numbers such that

${\displaystyle ae^{t}+be^{2t}=0}$

Take the first derivative of the above equation such that

${\displaystyle ae^{t}+2be^{2t}=0}$

for all values of t. We need to show that ${\displaystyle a=0}$ and ${\displaystyle b=0}$. In order to do this, we subtract the first equation from the second, giving ${\displaystyle be^{2t}=0}$. Since ${\displaystyle e^{2t}}$ is not zero for some t, ${\displaystyle b=0}$. It follows that ${\displaystyle a=0}$ too. Therefore, according to the definition of linear independence, ${\displaystyle e^{t}}$ and ${\displaystyle e^{2t}}$ are linearly independent.

Space of linear dependencies[edit]

A linear dependency or linear relation among vectors v₁, ..., v_n is a tuple (a₁, ..., a_n) with n scalar components such that

${\displaystyle a_{1}\mathbf {v} _{1}+\cdots +a_{n}\mathbf {v} _{n}=\mathbf {0} .}$

If such a linear dependence exists with at least a nonzero component, then the n vectors are linearly dependent. Linear dependencies among v₁, ..., v_n form a vector space.

If the vectors are expressed by their coordinates, then the linear dependencies are the solutions of a homogeneous system of linear equations, with the coordinates of the vectors as coefficients. A basis of the vector space of linear dependencies can therefore be computed by Gaussian elimination.

See also[edit]

• Gramian matrix
• Matroid
• Orthogonality
• Wronskian
• Multicollinearity

References[edit]

1. ^ G. E. Shilov, Linear Algebra (Trans. R. A. Silverman), Dover Publications, New York, 1977.
2. ^ Friedberg, Insel, Spence, Stephen, Arnold, Lawrence. Linear Algebra. Pearson, 4th Edition. pp. 48–49. ISBN 0130084514.CS1 maint: multiple names: authors list (link)

External links[edit]

• Hazewinkel, Michiel, ed. (2001) [1994], "Linear independence", Encyclopedia of Mathematics, Springer Science+Business Media B.V. / Kluwer Academic Publishers, ISBN 978-1-55608-010-4
• Linearly Dependent Functions at WolframMathWorld.
• Tutorial and interactive program on Linear Independence.
• Introduction to Linear Independence at KhanAcademy.

v t e Linear algebra
Basic concepts	Scalar Vector Vector space Scalar multiplication Vector projection Linear span Linear map Linear projection Linear independence Linear combination Basis Change of basis Row and column vectors Row and column spaces Orthogonality Kernel Eigenvalues and eigenvectors Outer product Inner product space Dot product Transpose Gram–Schmidt process Linear equations Fundamental theorem
Vector algebra	Cross product Triple product Seven-dimensional cross product
Multilinear algebra	Geometric algebra Exterior algebra Bivector Multivector Tensor Outermorphism
Matrices	Block Decomposition Invertible Minor Multiplication Rank Transformation Cramer's rule Gaussian elimination Determinant
Algebraic constructions	Dual Direct sum Function space Quotient Subspace Tensor product
Numerical	Floating-point Numerical stability Basic Linear Algebra Subprograms (BLAS) Sparse matrix Comparison of linear algebra libraries Comparison of numerical analysis software
Category Outline Mathematics portal Wikibook Wikiversity

Retrieved from "https://en.wikipedia.org/w/index.php?title=Linear_independence&oldid=936841438"

Categories:

• Abstract algebra
• Linear algebra

Hidden categories:

• CS1 maint: multiple names: authors list
• Articles needing additional references from January 2019
• All articles needing additional references
• Articles containing proofs

Navigation menu

Personal tools

• Not logged in
• Talk
• Contributions
• Create account
• Log in

Namespaces

• Article
• Talk

Variants

Views

• Read
• Edit
• View history

More

Search

Navigation

• Main page
• Contents
• Featured content
• Current events
• Random article
• Donate to Wikipedia
• Wikipedia store

Interaction

• Help
• About Wikipedia
• Community portal
• Recent changes
• Contact page

Tools

• What links here
• Related changes
• Upload file
• Special pages
• Permanent link
• Page information
• Wikidata item
• Cite this page

Print/export

• Create a book
• Download as PDF
• Printable version

Languages

• العربية
• Български
• Bosanski
• Català
• Čeština
• Deutsch
• Español
• Esperanto
• فارسی
• Français
• Galego
• 한국어
• Bahasa Indonesia
• Íslenska
• Italiano
• עברית
• Қазақша
• Magyar
• Nederlands
• 日本語
• Norsk bokmål
• Polski
• Português
• Русский
• Simple English
• Slovenščina
• Suomi
• Svenska
• தமிழ்
• Тоҷикӣ
• Українська
• اردو
• Tiếng Việt
• 粵語
• 中文

Edit links

• This page was last edited on 21 January 2020, at 09:49 (UTC).
• Text is available under the Creative Commons Attribution-ShareAlike License; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization.

• Privacy policy
• About Wikipedia
• Disclaimers
• Contact Wikipedia
• Developers
• Statistics
• Cookie statement
• Mobile view

• 
•