Linear Dependence and Independence in Matrix Algebra
Linear Dependence and
Independence
From: D.A. Harville,
Matrix Algebra from a Statistician’s Perspective
, Springer. Chapter 3
Linear Dependence and Independence - I
Finite set of matrices (including row and column vectors as special cases):
A
1
,…,
A
k
Linearly Dependent
There exist scalars
x
1
,…,
x
k
(not all 0) such that
x
1
A
1
+…+
x
k
A
k
=
0
Linearly Independent
x
1
A
1
+…+
x
k
A
k
=
0
iff
x
1
= … =
x
k
= 0
{
} Empty set is considered to be linearly independent
Set containing a single matrix is said to be linearly independent unless the matrix is the null matrix
0
Lemma 3.2.1. Set {A
1
,…,
A
k
} of
k
≥ 2
m
x
n
matrices is linearly dependent if at least one matrix can be written as
linear combination of the others
Linearly dependent
x
1
A
1
+…+
x
k
A
k
=
0
A
j
= -(x
1
/x
j
)
A
1
+ … + -(x
j-1
/x
j
)
A
j-1
+ -(x
j+1
/x
j
)
A
j+1
+ … + -(x
k
/x
j
)
A
k
(so long as x
j
≠ 0).
A
j
=
x
1
A
1
+ … +
x
k
A
k
-
x
1
A
1
+… + -x
j-1
A
j-1
+
A
j
+ -x
j+1
A
j+1
+ …
+ -
x
k
A
k
=
0
Linear Dependence and Independence - II
Lemma 3.2.2. Set {A
1
,…,
A
k
} of
k
≥ 2
m
x
n
matrices with
A
j
≠
0
is linearly dependent iff at least one matrix can be written as linear combination of the previous ones:
A
j
=
x
1
A
1
+ … +
x
j-1
A
j-1
Equivalently: {A
1
,…,
A
k
} is linearly independent if none of the matrices can be written in terms of previous ones.
A
j
=
x
1
A
1
+ … +
x
j-1
A
j-1
-
x
1
A
1
+… + -x
j-1
A
j-1
+
A
j
+ (0)
A
j+1
+ …
+ (0)
A
k
=
0
Let j be lowest integer such that {A
1
,…,
A
j
} ≡ lin. dep.
x
1
A
1
+ … +
x
j-1
A
j-1
+
x
j
A
j
=
0
for some (x
1
,…,x
j
) not all 0
and x
j
≠ 0
A
j
= (-
x
1
/x
j
)
A
1
+ … + (-
x
j-1
/x
j
)
A
j-1
Corollary 3.2.3. Set {A
1
,…,
A
k
} of
k
≥ 2
m
x
n
linearly independent matrices and
A
is an
m
x
n
matrix.
Then {A
1
, …,
A
k
,
A
} is linearly independent iff A ≠
x
1
A
1
+ … +
x
k
A
k
for not all
x
i
= 0
Suppose
A
=
x
1
A
1
+ … +
x
k
A
k
for not all
x
i
= 0. Then
x
1
A
1
+ … +
x
k
A
k
+ (-1)
A
=
0
{A
1
, …,
A
k
,
A
} lin. dependent
A ≠
x
1
A
1
+ … +
x
k
A
k
is only way for {A
1
, …,
A
k
,
A
} to be linearly independent
Linear Dependence and Independence - III
This discussion covers the concepts of linear dependence and independence in matrix algebra, exploring conditions for sets of matrices to be linearly dependent or independent, along with key lemmas and their implications.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Linear Dependence and Independence From: D.A. Harville, Matrix Algebra from a Statistician s Perspective, Springer. Chapter 3
Linear Dependence and Independence - I Finite set of matrices (including row and column vectors as special cases): A1, ,Ak Linearly Dependent There exist scalars x1, ,xk (not all 0) such that x1A1+ +xkAk = 0 Linearly Independent x1A1+ +xkAk = 0 iff x1= = xk = 0 { } Empty set is considered to be linearly independent Set containing a single matrix is said to be linearly independent unless the matrix is the null matrix 0 Lemma 3.2.1. Set {A1, ,Ak} of k 2 mxn matrices is linearly dependent if at least one matrix can be written as linear combination of the others Linearly dependent x1A1+ +xkAk = 0 Aj = -(x1/xj) A1+ + -(xj-1/xj) Aj-1 + -(xj+1/xj) Aj+1+ + -(xk/xj) Ak (so long as xj 0). Aj = x1A1+ + xkAk -x1A1+ + -xj-1 Aj-1 + Aj + -xj+1 Aj+1+ + -xkAk = 0
Linear Dependence and Independence - II Lemma 3.2.2. Set {A1, ,Ak} of k 2 mxn matrices with Aj 0 is linearly dependent iff at least one matrix can be written as linear combination of the previous ones: Aj = x1A1+ + xj-1Aj-1 Equivalently: {A1, ,Ak} is linearly independent if none of the matrices can be written in terms of previous ones. Aj = x1A1+ + xj-1Aj-1 Let j be lowest integer such that {A1, ,Aj} lin. dep. x1A1+ + xj-1Aj-1 + xjAj = 0 for some (x1, ,xj) not all 0 and xj 0 Aj = (-x1/xj)A1 + + (-xj-1 /xj)Aj-1 -x1A1+ + -xj-1 Aj-1 + Aj + (0)Aj+1+ + (0)Ak = 0 Corollary 3.2.3. Set {A1, ,Ak} of k 2 mxn linearly independent matrices and A is an mxn matrix. Then {A1, , Ak, A} is linearly independent iff A x1A1+ + xkAk for not all xi = 0 Suppose A = x1A1+ + xkAk for not all xi = 0. Then x1A1+ + xkAk + (-1)A = 0 {A1, , Ak, A} lin. dependent A x1A1+ + xkAk is only way for {A1, , Ak, A} to be linearly independent
Linear Dependence and Independence - III m n = + + = A A C A A Lemma 3.2.4. ,..., matrices. Let ... 1,..., x x j r 1 1 1 k j j kj k x 1 j = x A A x x C C If ,..., and ,..., are each lin. indep. sets, then ,..., lin. indep. 1 1 1 j k r r x kj r y x 1 j j = 1 j r r r r = + + = C A A x Consider scalars: ,..., : ... with y y y y x y x y 1 1 1 r j j j j j kj k j j = = = = 1 1 1 1 j j j j r y x j kj = 1 j r r = = + ... 0 + = x x x 0 C A A C C If ,..., is linearly dependent, then ,..., s.t. 0 0 ,..., linearly dependent y y y y 1 1 1 1 r r j j j j k r = = 1 1 j j r = A A x x C 0 Suppose ,..., and ,..., are each lin. indep. Let ,..., be scalars s.t. y y y 1 1 1 k r r j j = 1 j r r r r = = = = + + = = x 0 A A 0 ... 0 ... 0 y y y y x y x y x i 1 1 1 j j r j j j kj k j ij = = = = 1 1 1 1 j j j = j = = C C ,..., linearly independent since ... 0 y y 1 1 r r
