Linearly independence after feature embedding
$begingroup$
Problem
If feature vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$ are linearly independent, argue whether or not their embedding $psi(mathbf{x}_1), psi(mathbf{x}_2),cdots,psi(mathbf{x}_m)$ are linearly independent.
Some Thoughts
The complication here is that the choice of $psi(cdot)$ is not random and often requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable (or we could always do $psi:mathbf{x}_imapsto mathbf{0}$, but this is hardly useful). I am wondering whether this requirement could guarantee that the mapping keeps the linearly independence of original vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$.
linear-algebra machine-learning
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
$endgroup$
add a comment |
$begingroup$
Problem
If feature vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$ are linearly independent, argue whether or not their embedding $psi(mathbf{x}_1), psi(mathbf{x}_2),cdots,psi(mathbf{x}_m)$ are linearly independent.
Some Thoughts
The complication here is that the choice of $psi(cdot)$ is not random and often requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable (or we could always do $psi:mathbf{x}_imapsto mathbf{0}$, but this is hardly useful). I am wondering whether this requirement could guarantee that the mapping keeps the linearly independence of original vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$.
linear-algebra machine-learning
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
$endgroup$
$begingroup$
Could you tell us more precisely what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable"?
$endgroup$
– Federico
Dec 3 '18 at 18:01
$begingroup$
@Federico My bad. Suppose $mathbf{x}_i$'s belongs to two classes, then in original space they are not separable by a hyperplane. But after transformation, i.e. $psi(mathbf{x}_i)$'s, they could be separated by a hyperplane.
$endgroup$
– Mr.Robot
Dec 3 '18 at 18:30
1
$begingroup$
Well then it's even more severe that what I wrote. For example, all points from the first class could be mapped to the same vector $v$ and all points from the second class to another vector $w$. From the point of view of machine learning, this is quite a perfect situation, but for sure the images are not linearly independent.
$endgroup$
– Federico
Dec 3 '18 at 18:34
1
$begingroup$
It seems to me that the linear independence of the images is not really relevant for the purpose of ML, but I might be wrong.
$endgroup$
– Federico
Dec 3 '18 at 18:35
$begingroup$
@Federico Thank you. I think your example is sufficient for my purpose. Since polynomial and RBF mapping are often used, I have always been struggling with these more complicated mappings but ignore the most straightforward case.
$endgroup$
– Mr.Robot
Dec 3 '18 at 19:09
add a comment |
$begingroup$
Problem
If feature vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$ are linearly independent, argue whether or not their embedding $psi(mathbf{x}_1), psi(mathbf{x}_2),cdots,psi(mathbf{x}_m)$ are linearly independent.
Some Thoughts
The complication here is that the choice of $psi(cdot)$ is not random and often requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable (or we could always do $psi:mathbf{x}_imapsto mathbf{0}$, but this is hardly useful). I am wondering whether this requirement could guarantee that the mapping keeps the linearly independence of original vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$.
linear-algebra machine-learning
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
$endgroup$
Problem
If feature vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$ are linearly independent, argue whether or not their embedding $psi(mathbf{x}_1), psi(mathbf{x}_2),cdots,psi(mathbf{x}_m)$ are linearly independent.
Some Thoughts
The complication here is that the choice of $psi(cdot)$ is not random and often requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable (or we could always do $psi:mathbf{x}_imapsto mathbf{0}$, but this is hardly useful). I am wondering whether this requirement could guarantee that the mapping keeps the linearly independence of original vectors $mathbf{x}_1, mathbf{x}_2,cdots,mathbf{x}_m$.
linear-algebra machine-learning
linear-algebra machine-learning
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
asked Dec 3 '18 at 17:52
Mr.RobotMr.Robot
35019
35019
$begingroup$
Could you tell us more precisely what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable"?
$endgroup$
– Federico
Dec 3 '18 at 18:01
$begingroup$
@Federico My bad. Suppose $mathbf{x}_i$'s belongs to two classes, then in original space they are not separable by a hyperplane. But after transformation, i.e. $psi(mathbf{x}_i)$'s, they could be separated by a hyperplane.
$endgroup$
– Mr.Robot
Dec 3 '18 at 18:30
1
$begingroup$
Well then it's even more severe that what I wrote. For example, all points from the first class could be mapped to the same vector $v$ and all points from the second class to another vector $w$. From the point of view of machine learning, this is quite a perfect situation, but for sure the images are not linearly independent.
$endgroup$
– Federico
Dec 3 '18 at 18:34
1
$begingroup$
It seems to me that the linear independence of the images is not really relevant for the purpose of ML, but I might be wrong.
$endgroup$
– Federico
Dec 3 '18 at 18:35
$begingroup$
@Federico Thank you. I think your example is sufficient for my purpose. Since polynomial and RBF mapping are often used, I have always been struggling with these more complicated mappings but ignore the most straightforward case.
$endgroup$
– Mr.Robot
Dec 3 '18 at 19:09
add a comment |
$begingroup$
Could you tell us more precisely what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable"?
$endgroup$
– Federico
Dec 3 '18 at 18:01
$begingroup$
@Federico My bad. Suppose $mathbf{x}_i$'s belongs to two classes, then in original space they are not separable by a hyperplane. But after transformation, i.e. $psi(mathbf{x}_i)$'s, they could be separated by a hyperplane.
$endgroup$
– Mr.Robot
Dec 3 '18 at 18:30
1
$begingroup$
Well then it's even more severe that what I wrote. For example, all points from the first class could be mapped to the same vector $v$ and all points from the second class to another vector $w$. From the point of view of machine learning, this is quite a perfect situation, but for sure the images are not linearly independent.
$endgroup$
– Federico
Dec 3 '18 at 18:34
1
$begingroup$
It seems to me that the linear independence of the images is not really relevant for the purpose of ML, but I might be wrong.
$endgroup$
– Federico
Dec 3 '18 at 18:35
$begingroup$
@Federico Thank you. I think your example is sufficient for my purpose. Since polynomial and RBF mapping are often used, I have always been struggling with these more complicated mappings but ignore the most straightforward case.
$endgroup$
– Mr.Robot
Dec 3 '18 at 19:09
$begingroup$
Could you tell us more precisely what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable"?
$endgroup$
– Federico
Dec 3 '18 at 18:01
$begingroup$
Could you tell us more precisely what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable"?
$endgroup$
– Federico
Dec 3 '18 at 18:01
$begingroup$
@Federico My bad. Suppose $mathbf{x}_i$'s belongs to two classes, then in original space they are not separable by a hyperplane. But after transformation, i.e. $psi(mathbf{x}_i)$'s, they could be separated by a hyperplane.
$endgroup$
– Mr.Robot
Dec 3 '18 at 18:30
$begingroup$
@Federico My bad. Suppose $mathbf{x}_i$'s belongs to two classes, then in original space they are not separable by a hyperplane. But after transformation, i.e. $psi(mathbf{x}_i)$'s, they could be separated by a hyperplane.
$endgroup$
– Mr.Robot
Dec 3 '18 at 18:30
1
1
$begingroup$
Well then it's even more severe that what I wrote. For example, all points from the first class could be mapped to the same vector $v$ and all points from the second class to another vector $w$. From the point of view of machine learning, this is quite a perfect situation, but for sure the images are not linearly independent.
$endgroup$
– Federico
Dec 3 '18 at 18:34
$begingroup$
Well then it's even more severe that what I wrote. For example, all points from the first class could be mapped to the same vector $v$ and all points from the second class to another vector $w$. From the point of view of machine learning, this is quite a perfect situation, but for sure the images are not linearly independent.
$endgroup$
– Federico
Dec 3 '18 at 18:34
1
1
$begingroup$
It seems to me that the linear independence of the images is not really relevant for the purpose of ML, but I might be wrong.
$endgroup$
– Federico
Dec 3 '18 at 18:35
$begingroup$
It seems to me that the linear independence of the images is not really relevant for the purpose of ML, but I might be wrong.
$endgroup$
– Federico
Dec 3 '18 at 18:35
$begingroup$
@Federico Thank you. I think your example is sufficient for my purpose. Since polynomial and RBF mapping are often used, I have always been struggling with these more complicated mappings but ignore the most straightforward case.
$endgroup$
– Mr.Robot
Dec 3 '18 at 19:09
$begingroup$
@Federico Thank you. I think your example is sufficient for my purpose. Since polynomial and RBF mapping are often used, I have always been struggling with these more complicated mappings but ignore the most straightforward case.
$endgroup$
– Mr.Robot
Dec 3 '18 at 19:09
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
It depends on what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable". If you mean that any of the points $psi(mathbf{x}_i)$ can be linearly separated from the others, then the answer is false.
For instance, the points $(0,0)$, $(1,0)$ and $(0,1)$ are linearly separable in this sense, but are not linearly independent.
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
$endgroup$
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3024431%2flinearly-independence-after-feature-embedding%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
It depends on what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable". If you mean that any of the points $psi(mathbf{x}_i)$ can be linearly separated from the others, then the answer is false.
For instance, the points $(0,0)$, $(1,0)$ and $(0,1)$ are linearly separable in this sense, but are not linearly independent.
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
$endgroup$
add a comment |
$begingroup$
It depends on what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable". If you mean that any of the points $psi(mathbf{x}_i)$ can be linearly separated from the others, then the answer is false.
For instance, the points $(0,0)$, $(1,0)$ and $(0,1)$ are linearly separable in this sense, but are not linearly independent.
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
$endgroup$
add a comment |
$begingroup$
It depends on what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable". If you mean that any of the points $psi(mathbf{x}_i)$ can be linearly separated from the others, then the answer is false.
For instance, the points $(0,0)$, $(1,0)$ and $(0,1)$ are linearly separable in this sense, but are not linearly independent.
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
$endgroup$
It depends on what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable". If you mean that any of the points $psi(mathbf{x}_i)$ can be linearly separated from the others, then the answer is false.
For instance, the points $(0,0)$, $(1,0)$ and $(0,1)$ are linearly separable in this sense, but are not linearly independent.
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
answered Dec 3 '18 at 18:00
FedericoFederico
5,124514
5,124514
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3024431%2flinearly-independence-after-feature-embedding%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Could you tell us more precisely what you mean by "requires that the resulting $psi(mathbf{x}_i)$'s being linearly separable"?
$endgroup$
– Federico
Dec 3 '18 at 18:01
$begingroup$
@Federico My bad. Suppose $mathbf{x}_i$'s belongs to two classes, then in original space they are not separable by a hyperplane. But after transformation, i.e. $psi(mathbf{x}_i)$'s, they could be separated by a hyperplane.
$endgroup$
– Mr.Robot
Dec 3 '18 at 18:30
1
$begingroup$
Well then it's even more severe that what I wrote. For example, all points from the first class could be mapped to the same vector $v$ and all points from the second class to another vector $w$. From the point of view of machine learning, this is quite a perfect situation, but for sure the images are not linearly independent.
$endgroup$
– Federico
Dec 3 '18 at 18:34
1
$begingroup$
It seems to me that the linear independence of the images is not really relevant for the purpose of ML, but I might be wrong.
$endgroup$
– Federico
Dec 3 '18 at 18:35
$begingroup$
@Federico Thank you. I think your example is sufficient for my purpose. Since polynomial and RBF mapping are often used, I have always been struggling with these more complicated mappings but ignore the most straightforward case.
$endgroup$
– Mr.Robot
Dec 3 '18 at 19:09