Numpy array - duplicate removal
-2
I've data set and labels (as 2 distinct csv files). The entries are read into 2 distinct variables (as columns). I want to merge them into a 2D array and remove the duplicates, but preserve the order. Please suggest. Using "set" or "unique" didn't work.
data = np.loadtxt('raw_data.csv',delimiter=',',usecols=range(0,112),skiprows=0)
label = np.loadtxt('labels.csv',delimiter=',',usecols=range(0,112),skiprows=0)
features1 = data[:,0] ##channel 0
features1 = features1.reshape(-1,1)
labels1 = label[:,0]
python numpy duplicates
asked Nov 19 '18 at 1:16
ShachiShachi
31
add a comment |
-2
I've data set and labels (as 2 distinct csv files). The entries are read into 2 distinct variables (as columns). I want to merge them into a 2D array and remove the duplicates, but preserve the order. Please suggest. Using "set" or "unique" didn't work.
data = np.loadtxt('raw_data.csv',delimiter=',',usecols=range(0,112),skiprows=0)
label = np.loadtxt('labels.csv',delimiter=',',usecols=range(0,112),skiprows=0)
features1 = data[:,0] ##channel 0
features1 = features1.reshape(-1,1)
labels1 = label[:,0]
python numpy duplicates
asked Nov 19 '18 at 1:16
ShachiShachi
31
1
Could you give a Minimal, Complete, and Verifiable example?
– sacul
Nov 19 '18 at 1:19
I've the data on measured energy level of a set of 'n' channels, measured over 'm' instances of time. I need to check if channel 1 is occupied at time instances 1 to m. But measured energy levels are repetitive. I'm trying to use scikit-learn for this. The example mentioned above is verification for channel 0.
– Shachi
Nov 19 '18 at 4:28
add a comment |
-2
-2
-2
I've data set and labels (as 2 distinct csv files). The entries are read into 2 distinct variables (as columns). I want to merge them into a 2D array and remove the duplicates, but preserve the order. Please suggest. Using "set" or "unique" didn't work.
data = np.loadtxt('raw_data.csv',delimiter=',',usecols=range(0,112),skiprows=0)
label = np.loadtxt('labels.csv',delimiter=',',usecols=range(0,112),skiprows=0)
features1 = data[:,0] ##channel 0
features1 = features1.reshape(-1,1)
labels1 = label[:,0]
python numpy duplicates
asked Nov 19 '18 at 1:16
ShachiShachi
31
I've data set and labels (as 2 distinct csv files). The entries are read into 2 distinct variables (as columns). I want to merge them into a 2D array and remove the duplicates, but preserve the order. Please suggest. Using "set" or "unique" didn't work.
data = np.loadtxt('raw_data.csv',delimiter=',',usecols=range(0,112),skiprows=0)
label = np.loadtxt('labels.csv',delimiter=',',usecols=range(0,112),skiprows=0)
features1 = data[:,0] ##channel 0
features1 = features1.reshape(-1,1)
labels1 = label[:,0]
python numpy duplicates
python numpy duplicates
asked Nov 19 '18 at 1:16
ShachiShachi
31
asked Nov 19 '18 at 1:16
ShachiShachi
31
asked Nov 19 '18 at 1:16
ShachiShachi
31
asked Nov 19 '18 at 1:16
ShachiShachi
31
asked Nov 19 '18 at 1:16
ShachiShachi
31
31
1
Could you give a Minimal, Complete, and Verifiable example?
– sacul
Nov 19 '18 at 1:19
I've the data on measured energy level of a set of 'n' channels, measured over 'm' instances of time. I need to check if channel 1 is occupied at time instances 1 to m. But measured energy levels are repetitive. I'm trying to use scikit-learn for this. The example mentioned above is verification for channel 0.
– Shachi
Nov 19 '18 at 4:28
add a comment |
1
Could you give a Minimal, Complete, and Verifiable example?
– sacul
Nov 19 '18 at 1:19
I've the data on measured energy level of a set of 'n' channels, measured over 'm' instances of time. I need to check if channel 1 is occupied at time instances 1 to m. But measured energy levels are repetitive. I'm trying to use scikit-learn for this. The example mentioned above is verification for channel 0.
– Shachi
Nov 19 '18 at 4:28
1
1
Could you give a Minimal, Complete, and Verifiable example?
– sacul
Nov 19 '18 at 1:19
Could you give a Minimal, Complete, and Verifiable example?
– sacul
Nov 19 '18 at 1:19
I've the data on measured energy level of a set of 'n' channels, measured over 'm' instances of time. I need to check if channel 1 is occupied at time instances 1 to m. But measured energy levels are repetitive. I'm trying to use scikit-learn for this. The example mentioned above is verification for channel 0.
– Shachi
Nov 19 '18 at 4:28
I've the data on measured energy level of a set of 'n' channels, measured over 'm' instances of time. I need to check if channel 1 is occupied at time instances 1 to m. But measured energy levels are repetitive. I'm trying to use scikit-learn for this. The example mentioned above is verification for channel 0.
– Shachi
Nov 19 '18 at 4:28
add a comment |
1 Answer
1
active
oldest
votes
0
I assume the labels can have duplicates? You can use np.unique and return the unique indices, and filter the data values by them.
import numpy as np
labels = np.array(['a', 'b', 'c', 'b', 'd', 'c', 'a', 'e'])
vals = np.array([1, 2, 3, 4, 5, 6, 7, 8])
unique, unique_idx = np.unique(labels, return_index=True)
filtered_vals = vals[unique_idx]
combined = np.vstack((unique, filtered_vals))
print combined
Output
[['a' 'b' 'c' 'd' 'e']
['1' '2' '3' '5' '8']]
answered Nov 19 '18 at 1:31
pxepxe
1207
1
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53367126%2fnumpy-array-duplicate-removal%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
0
I assume the labels can have duplicates? You can use np.unique and return the unique indices, and filter the data values by them.
import numpy as np
labels = np.array(['a', 'b', 'c', 'b', 'd', 'c', 'a', 'e'])
vals = np.array([1, 2, 3, 4, 5, 6, 7, 8])
unique, unique_idx = np.unique(labels, return_index=True)
filtered_vals = vals[unique_idx]
combined = np.vstack((unique, filtered_vals))
print combined
Output
[['a' 'b' 'c' 'd' 'e']
['1' '2' '3' '5' '8']]
answered Nov 19 '18 at 1:31
pxepxe
1207
1
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
add a comment |
0
I assume the labels can have duplicates? You can use np.unique and return the unique indices, and filter the data values by them.
import numpy as np
labels = np.array(['a', 'b', 'c', 'b', 'd', 'c', 'a', 'e'])
vals = np.array([1, 2, 3, 4, 5, 6, 7, 8])
unique, unique_idx = np.unique(labels, return_index=True)
filtered_vals = vals[unique_idx]
combined = np.vstack((unique, filtered_vals))
print combined
Output
[['a' 'b' 'c' 'd' 'e']
['1' '2' '3' '5' '8']]
answered Nov 19 '18 at 1:31
pxepxe
1207
1
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
add a comment |
0
0
0
I assume the labels can have duplicates? You can use np.unique and return the unique indices, and filter the data values by them.
import numpy as np
labels = np.array(['a', 'b', 'c', 'b', 'd', 'c', 'a', 'e'])
vals = np.array([1, 2, 3, 4, 5, 6, 7, 8])
unique, unique_idx = np.unique(labels, return_index=True)
filtered_vals = vals[unique_idx]
combined = np.vstack((unique, filtered_vals))
print combined
Output
[['a' 'b' 'c' 'd' 'e']
['1' '2' '3' '5' '8']]
answered Nov 19 '18 at 1:31
pxepxe
1207
I assume the labels can have duplicates? You can use np.unique and return the unique indices, and filter the data values by them.
import numpy as np
labels = np.array(['a', 'b', 'c', 'b', 'd', 'c', 'a', 'e'])
vals = np.array([1, 2, 3, 4, 5, 6, 7, 8])
unique, unique_idx = np.unique(labels, return_index=True)
filtered_vals = vals[unique_idx]
combined = np.vstack((unique, filtered_vals))
print combined
Output
[['a' 'b' 'c' 'd' 'e']
['1' '2' '3' '5' '8']]
answered Nov 19 '18 at 1:31
pxepxe
1207
answered Nov 19 '18 at 1:31
pxepxe
1207
answered Nov 19 '18 at 1:31
pxepxe
1207
answered Nov 19 '18 at 1:31
pxepxe
1207
1207
1
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
add a comment |
1
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
1
1
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Yes, the labels can have duplicates. Thank you. It helped.
– Shachi
Nov 19 '18 at 4:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
Happy to help. If the answer solved the problem, please consider marking it as 'accepted' by clicking the green check mark.
– pxe
Nov 19 '18 at 16:56
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53367126%2fnumpy-array-duplicate-removal%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Could you give a Minimal, Complete, and Verifiable example?
– sacul
Nov 19 '18 at 1:19
I've the data on measured energy level of a set of 'n' channels, measured over 'm' instances of time. I need to check if channel 1 is occupied at time instances 1 to m. But measured energy levels are repetitive. I'm trying to use scikit-learn for this. The example mentioned above is verification for channel 0.
– Shachi
Nov 19 '18 at 4:28