Properties of singular value decomposition
$begingroup$
Every (real) $mtimes n$ matrix $A$ of rank $r$ has an SVD
$$
A = USigma V^T
$$
Now, I have read about the following properties:
- $text{Image}(A) = text{span}{u_1,dots,u_r}$
- $text{Null space}(A) = text{span}{v_{r+1},dots,v_n}$
Maybe I am lacking some knowledge from my linear algebra courses, but how can these properties be proven?
linear-algebra svd
edited Dec 7 '18 at 11:33
Kurban
406
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
$endgroup$
add a comment |
$begingroup$
Every (real) $mtimes n$ matrix $A$ of rank $r$ has an SVD
$$
A = USigma V^T
$$
Now, I have read about the following properties:
- $text{Image}(A) = text{span}{u_1,dots,u_r}$
- $text{Null space}(A) = text{span}{v_{r+1},dots,v_n}$
Maybe I am lacking some knowledge from my linear algebra courses, but how can these properties be proven?
linear-algebra svd
edited Dec 7 '18 at 11:33
Kurban
406
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
$endgroup$
add a comment |
$begingroup$
Every (real) $mtimes n$ matrix $A$ of rank $r$ has an SVD
$$
A = USigma V^T
$$
Now, I have read about the following properties:
- $text{Image}(A) = text{span}{u_1,dots,u_r}$
- $text{Null space}(A) = text{span}{v_{r+1},dots,v_n}$
Maybe I am lacking some knowledge from my linear algebra courses, but how can these properties be proven?
linear-algebra svd
edited Dec 7 '18 at 11:33
Kurban
406
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
$endgroup$
Every (real) $mtimes n$ matrix $A$ of rank $r$ has an SVD
$$
A = USigma V^T
$$
Now, I have read about the following properties:
- $text{Image}(A) = text{span}{u_1,dots,u_r}$
- $text{Null space}(A) = text{span}{v_{r+1},dots,v_n}$
Maybe I am lacking some knowledge from my linear algebra courses, but how can these properties be proven?
linear-algebra svd
linear-algebra svd
edited Dec 7 '18 at 11:33
Kurban
406
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
edited Dec 7 '18 at 11:33
Kurban
406
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
edited Dec 7 '18 at 11:33
Kurban
406
edited Dec 7 '18 at 11:33
Kurban
406
edited Dec 7 '18 at 11:33
Kurban
406
406
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
asked Dec 7 '18 at 11:19
user3825755user3825755
1858
1858
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
Image(A) means column space of A. I will assume $mgeq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition:
begin{align}
A = U_1Sigma_1 V_1^T = underbrace{[U_1, U_2]}_Uunderbrace{begin{bmatrix} Sigma_1 & 0\0 & 0end{bmatrix}}_Sigmaunderbrace{begin{bmatrix} V_1^T \ V_2^Tend{bmatrix} }_V
end{align}
where $U in R^{mtimes m}$, $Sigma = diag(sigma_1,ldots,sigma_r)in R^{rtimes r}$, and $V in R^{ntimes n}$. Matrix U constitutes an orthonormal basis for $R^{mtimes m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{ntimes n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m or V_iV^T_i = I_n, i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $forall xin R^n$
begin{align}
Ax &= sum_i^r sigma_i(v_i^Tx)u_i \&= U_1Sigma_1V_1^Tx = U_1z\ &=sum_i^rz_iu_i
end{align} where $z_i = sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors ${u_1,ldots,u_r}$ which equivalent to say $mathcal{R}(A) = span{u_1,ldots,u_r}$. On the other hand,
null space is the vectors that are mapped to zero, i.e. if $xin mathcal{N}(A)$, then $Ax = 0$. Since the set ${u_1,ldots,u_r}$ is orthonormal,
begin{equation}
Ax = U_1z = 0
end{equation}
can be possible only if z is zero vector.
begin{align}
z = Sigma_1V_1Tx = 0 leftrightarrow V_1^Tx
end{align}
$Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span{v_1,ldots,v_r}$ which means that $$xin span{v_{r+1},ldots,v_n} = mathcal{R}(V_2)rightarrow Ax = 0$$
answered Dec 7 '18 at 12:13
KurbanKurban
406
$endgroup$
add a comment |
$begingroup$
One simple possibility is to use this form of SV decomposition of $A$:
$$A = sum_{i=1}^{r}{lambda_i u_i v_i^T}$$
Then, for an input
$$x = sum_{i=1}^{n} x_iv_i$$
It follows
$$Ax = sum_{i=1}^{r}{lambda_i x_i u_i} $$
The properties you are looking for are a direct consequence of the last relation.
answered Dec 7 '18 at 14:50
DamienDamien
59714
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3029791%2fproperties-of-singular-value-decomposition%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Image(A) means column space of A. I will assume $mgeq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition:
begin{align}
A = U_1Sigma_1 V_1^T = underbrace{[U_1, U_2]}_Uunderbrace{begin{bmatrix} Sigma_1 & 0\0 & 0end{bmatrix}}_Sigmaunderbrace{begin{bmatrix} V_1^T \ V_2^Tend{bmatrix} }_V
end{align}
where $U in R^{mtimes m}$, $Sigma = diag(sigma_1,ldots,sigma_r)in R^{rtimes r}$, and $V in R^{ntimes n}$. Matrix U constitutes an orthonormal basis for $R^{mtimes m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{ntimes n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m or V_iV^T_i = I_n, i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $forall xin R^n$
begin{align}
Ax &= sum_i^r sigma_i(v_i^Tx)u_i \&= U_1Sigma_1V_1^Tx = U_1z\ &=sum_i^rz_iu_i
end{align} where $z_i = sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors ${u_1,ldots,u_r}$ which equivalent to say $mathcal{R}(A) = span{u_1,ldots,u_r}$. On the other hand,
null space is the vectors that are mapped to zero, i.e. if $xin mathcal{N}(A)$, then $Ax = 0$. Since the set ${u_1,ldots,u_r}$ is orthonormal,
begin{equation}
Ax = U_1z = 0
end{equation}
can be possible only if z is zero vector.
begin{align}
z = Sigma_1V_1Tx = 0 leftrightarrow V_1^Tx
end{align}
$Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span{v_1,ldots,v_r}$ which means that $$xin span{v_{r+1},ldots,v_n} = mathcal{R}(V_2)rightarrow Ax = 0$$
answered Dec 7 '18 at 12:13
KurbanKurban
406
$endgroup$
add a comment |
$begingroup$
Image(A) means column space of A. I will assume $mgeq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition:
begin{align}
A = U_1Sigma_1 V_1^T = underbrace{[U_1, U_2]}_Uunderbrace{begin{bmatrix} Sigma_1 & 0\0 & 0end{bmatrix}}_Sigmaunderbrace{begin{bmatrix} V_1^T \ V_2^Tend{bmatrix} }_V
end{align}
where $U in R^{mtimes m}$, $Sigma = diag(sigma_1,ldots,sigma_r)in R^{rtimes r}$, and $V in R^{ntimes n}$. Matrix U constitutes an orthonormal basis for $R^{mtimes m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{ntimes n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m or V_iV^T_i = I_n, i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $forall xin R^n$
begin{align}
Ax &= sum_i^r sigma_i(v_i^Tx)u_i \&= U_1Sigma_1V_1^Tx = U_1z\ &=sum_i^rz_iu_i
end{align} where $z_i = sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors ${u_1,ldots,u_r}$ which equivalent to say $mathcal{R}(A) = span{u_1,ldots,u_r}$. On the other hand,
null space is the vectors that are mapped to zero, i.e. if $xin mathcal{N}(A)$, then $Ax = 0$. Since the set ${u_1,ldots,u_r}$ is orthonormal,
begin{equation}
Ax = U_1z = 0
end{equation}
can be possible only if z is zero vector.
begin{align}
z = Sigma_1V_1Tx = 0 leftrightarrow V_1^Tx
end{align}
$Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span{v_1,ldots,v_r}$ which means that $$xin span{v_{r+1},ldots,v_n} = mathcal{R}(V_2)rightarrow Ax = 0$$
answered Dec 7 '18 at 12:13
KurbanKurban
406
$endgroup$
add a comment |
$begingroup$
Image(A) means column space of A. I will assume $mgeq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition:
begin{align}
A = U_1Sigma_1 V_1^T = underbrace{[U_1, U_2]}_Uunderbrace{begin{bmatrix} Sigma_1 & 0\0 & 0end{bmatrix}}_Sigmaunderbrace{begin{bmatrix} V_1^T \ V_2^Tend{bmatrix} }_V
end{align}
where $U in R^{mtimes m}$, $Sigma = diag(sigma_1,ldots,sigma_r)in R^{rtimes r}$, and $V in R^{ntimes n}$. Matrix U constitutes an orthonormal basis for $R^{mtimes m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{ntimes n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m or V_iV^T_i = I_n, i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $forall xin R^n$
begin{align}
Ax &= sum_i^r sigma_i(v_i^Tx)u_i \&= U_1Sigma_1V_1^Tx = U_1z\ &=sum_i^rz_iu_i
end{align} where $z_i = sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors ${u_1,ldots,u_r}$ which equivalent to say $mathcal{R}(A) = span{u_1,ldots,u_r}$. On the other hand,
null space is the vectors that are mapped to zero, i.e. if $xin mathcal{N}(A)$, then $Ax = 0$. Since the set ${u_1,ldots,u_r}$ is orthonormal,
begin{equation}
Ax = U_1z = 0
end{equation}
can be possible only if z is zero vector.
begin{align}
z = Sigma_1V_1Tx = 0 leftrightarrow V_1^Tx
end{align}
$Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span{v_1,ldots,v_r}$ which means that $$xin span{v_{r+1},ldots,v_n} = mathcal{R}(V_2)rightarrow Ax = 0$$
answered Dec 7 '18 at 12:13
KurbanKurban
406
$endgroup$
Image(A) means column space of A. I will assume $mgeq n$ and rank(A) = r without loss of generality. To see these identity, distinguish compact and full svd decomposition:
begin{align}
A = U_1Sigma_1 V_1^T = underbrace{[U_1, U_2]}_Uunderbrace{begin{bmatrix} Sigma_1 & 0\0 & 0end{bmatrix}}_Sigmaunderbrace{begin{bmatrix} V_1^T \ V_2^Tend{bmatrix} }_V
end{align}
where $U in R^{mtimes m}$, $Sigma = diag(sigma_1,ldots,sigma_r)in R^{rtimes r}$, and $V in R^{ntimes n}$. Matrix U constitutes an orthonormal basis for $R^{mtimes m}$ with $$U^TU=UU^T = I_m.$$ In the same way, matrix V constitutes an orthonormal basis for $R^{ntimes n}$ with $$V^TV=VV^T=I_n.$$ On the other hand, by compact SVD, you get only orthonormal basis for the range and domain space of matrix A. So, you do not have the property $$U_iU^T_i = I_m or V_iV^T_i = I_n, i = 1,2. $$ Knowing this distinction, come back to the your question. Consider following matrix transformation for $forall xin R^n$
begin{align}
Ax &= sum_i^r sigma_i(v_i^Tx)u_i \&= U_1Sigma_1V_1^Tx = U_1z\ &=sum_i^rz_iu_i
end{align} where $z_i = sigma_i(v_i^Tx)$. So, Ax is just weighted linear combination of vectors ${u_1,ldots,u_r}$ which equivalent to say $mathcal{R}(A) = span{u_1,ldots,u_r}$. On the other hand,
null space is the vectors that are mapped to zero, i.e. if $xin mathcal{N}(A)$, then $Ax = 0$. Since the set ${u_1,ldots,u_r}$ is orthonormal,
begin{equation}
Ax = U_1z = 0
end{equation}
can be possible only if z is zero vector.
begin{align}
z = Sigma_1V_1Tx = 0 leftrightarrow V_1^Tx
end{align}
$Sigma$ is just nonzero diagonal matrix, we can ignore it. So, vector z is $0$ if and only if vector x is perpendicular to $span{v_1,ldots,v_r}$ which means that $$xin span{v_{r+1},ldots,v_n} = mathcal{R}(V_2)rightarrow Ax = 0$$
answered Dec 7 '18 at 12:13
KurbanKurban
406
answered Dec 7 '18 at 12:13
KurbanKurban
406
answered Dec 7 '18 at 12:13
KurbanKurban
406
answered Dec 7 '18 at 12:13
KurbanKurban
406
406
add a comment |
add a comment |
$begingroup$
One simple possibility is to use this form of SV decomposition of $A$:
$$A = sum_{i=1}^{r}{lambda_i u_i v_i^T}$$
Then, for an input
$$x = sum_{i=1}^{n} x_iv_i$$
It follows
$$Ax = sum_{i=1}^{r}{lambda_i x_i u_i} $$
The properties you are looking for are a direct consequence of the last relation.
answered Dec 7 '18 at 14:50
DamienDamien
59714
$endgroup$
add a comment |
$begingroup$
One simple possibility is to use this form of SV decomposition of $A$:
$$A = sum_{i=1}^{r}{lambda_i u_i v_i^T}$$
Then, for an input
$$x = sum_{i=1}^{n} x_iv_i$$
It follows
$$Ax = sum_{i=1}^{r}{lambda_i x_i u_i} $$
The properties you are looking for are a direct consequence of the last relation.
answered Dec 7 '18 at 14:50
DamienDamien
59714
$endgroup$
add a comment |
$begingroup$
One simple possibility is to use this form of SV decomposition of $A$:
$$A = sum_{i=1}^{r}{lambda_i u_i v_i^T}$$
Then, for an input
$$x = sum_{i=1}^{n} x_iv_i$$
It follows
$$Ax = sum_{i=1}^{r}{lambda_i x_i u_i} $$
The properties you are looking for are a direct consequence of the last relation.
answered Dec 7 '18 at 14:50
DamienDamien
59714
$endgroup$
One simple possibility is to use this form of SV decomposition of $A$:
$$A = sum_{i=1}^{r}{lambda_i u_i v_i^T}$$
Then, for an input
$$x = sum_{i=1}^{n} x_iv_i$$
It follows
$$Ax = sum_{i=1}^{r}{lambda_i x_i u_i} $$
The properties you are looking for are a direct consequence of the last relation.
answered Dec 7 '18 at 14:50
DamienDamien
59714
answered Dec 7 '18 at 14:50
DamienDamien
59714
answered Dec 7 '18 at 14:50
DamienDamien
59714
answered Dec 7 '18 at 14:50
DamienDamien
59714
59714
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3029791%2fproperties-of-singular-value-decomposition%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
