reading from excel file pandas in the desired type
I am reading excel file using pandas containing 2 columns:
df
EID List of Tuples
1 [('Physics', 90)]
2 [('Physics', 80), ('Math', 70)]
3 [('Physics', 60, ('Math', 25))]
when I check df['List of Tuples'].iat[0] it gives me u"[('Physics', 90)]"
I am getting this as a unicode and not as a list of tuples.
When I df['List of Tuples'].iat[0].decode('iso-8859-1').encode('utf-8'), I get string: "[('Physics', 90)]"
I want to read/convert it as list of tuples [('Physics', 90)] instead of string or unicode.In short,I want to get rid of double quotes around each entry and read it as [('Physics', 90)] and [('Physics', 80), ('Math', 70)] and so on.
python pandas
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
add a comment |
I am reading excel file using pandas containing 2 columns:
df
EID List of Tuples
1 [('Physics', 90)]
2 [('Physics', 80), ('Math', 70)]
3 [('Physics', 60, ('Math', 25))]
when I check df['List of Tuples'].iat[0] it gives me u"[('Physics', 90)]"
I am getting this as a unicode and not as a list of tuples.
When I df['List of Tuples'].iat[0].decode('iso-8859-1').encode('utf-8'), I get string: "[('Physics', 90)]"
I want to read/convert it as list of tuples [('Physics', 90)] instead of string or unicode.In short,I want to get rid of double quotes around each entry and read it as [('Physics', 90)] and [('Physics', 80), ('Math', 70)] and so on.
python pandas
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
1
The formatting is all over the place and I can't fix from my phone, but if you're hoping to use dataframes for lists of tuples in single cells, you're opening a door to a lot of pain. That's not how pandas works. You should shoot for a structure that has scalars in cells or, IMO, drop pandas if you can't do that.
– roganjosh
Nov 21 '18 at 19:56
add a comment |
I am reading excel file using pandas containing 2 columns:
df
EID List of Tuples
1 [('Physics', 90)]
2 [('Physics', 80), ('Math', 70)]
3 [('Physics', 60, ('Math', 25))]
when I check df['List of Tuples'].iat[0] it gives me u"[('Physics', 90)]"
I am getting this as a unicode and not as a list of tuples.
When I df['List of Tuples'].iat[0].decode('iso-8859-1').encode('utf-8'), I get string: "[('Physics', 90)]"
I want to read/convert it as list of tuples [('Physics', 90)] instead of string or unicode.In short,I want to get rid of double quotes around each entry and read it as [('Physics', 90)] and [('Physics', 80), ('Math', 70)] and so on.
python pandas
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
I am reading excel file using pandas containing 2 columns:
df
EID List of Tuples
1 [('Physics', 90)]
2 [('Physics', 80), ('Math', 70)]
3 [('Physics', 60, ('Math', 25))]
when I check df['List of Tuples'].iat[0] it gives me u"[('Physics', 90)]"
I am getting this as a unicode and not as a list of tuples.
When I df['List of Tuples'].iat[0].decode('iso-8859-1').encode('utf-8'), I get string: "[('Physics', 90)]"
I want to read/convert it as list of tuples [('Physics', 90)] instead of string or unicode.In short,I want to get rid of double quotes around each entry and read it as [('Physics', 90)] and [('Physics', 80), ('Math', 70)] and so on.
python pandas
python pandas
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
edited Nov 21 '18 at 21:29
Ken Dekalb
317112
317112
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
asked Nov 21 '18 at 19:52
amanda smithamanda smith
575
575
1
The formatting is all over the place and I can't fix from my phone, but if you're hoping to use dataframes for lists of tuples in single cells, you're opening a door to a lot of pain. That's not how pandas works. You should shoot for a structure that has scalars in cells or, IMO, drop pandas if you can't do that.
– roganjosh
Nov 21 '18 at 19:56
add a comment |
1
The formatting is all over the place and I can't fix from my phone, but if you're hoping to use dataframes for lists of tuples in single cells, you're opening a door to a lot of pain. That's not how pandas works. You should shoot for a structure that has scalars in cells or, IMO, drop pandas if you can't do that.
– roganjosh
Nov 21 '18 at 19:56
1
1
The formatting is all over the place and I can't fix from my phone, but if you're hoping to use dataframes for lists of tuples in single cells, you're opening a door to a lot of pain. That's not how pandas works. You should shoot for a structure that has scalars in cells or, IMO, drop pandas if you can't do that.
– roganjosh
Nov 21 '18 at 19:56
The formatting is all over the place and I can't fix from my phone, but if you're hoping to use dataframes for lists of tuples in single cells, you're opening a door to a lot of pain. That's not how pandas works. You should shoot for a structure that has scalars in cells or, IMO, drop pandas if you can't do that.
– roganjosh
Nov 21 '18 at 19:56
add a comment |
1 Answer
1
active
oldest
votes
You might find it useful to parse these into python objects using ast which can convert string representations back into python objectd. Try something like the following (I can't replicate exactly because I don't have your data):
import ast
df['transformed_tuples'] = df['List of Tuples'].apply(ast.literal_eval)
To avoid this arising in the first place you might consider the file format you choose to read/write to, for example pickle will retain the original information (I'm assuming this has come from a pandas DataFrame that has been saved to excel).
You might also consider a tabular schema which doesn't have this irregular data type within it which would probably prove to be more stable and effective in the long run.
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53419579%2freading-from-excel-file-pandas-in-the-desired-type%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
You might find it useful to parse these into python objects using ast which can convert string representations back into python objectd. Try something like the following (I can't replicate exactly because I don't have your data):
import ast
df['transformed_tuples'] = df['List of Tuples'].apply(ast.literal_eval)
To avoid this arising in the first place you might consider the file format you choose to read/write to, for example pickle will retain the original information (I'm assuming this has come from a pandas DataFrame that has been saved to excel).
You might also consider a tabular schema which doesn't have this irregular data type within it which would probably prove to be more stable and effective in the long run.
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
add a comment |
You might find it useful to parse these into python objects using ast which can convert string representations back into python objectd. Try something like the following (I can't replicate exactly because I don't have your data):
import ast
df['transformed_tuples'] = df['List of Tuples'].apply(ast.literal_eval)
To avoid this arising in the first place you might consider the file format you choose to read/write to, for example pickle will retain the original information (I'm assuming this has come from a pandas DataFrame that has been saved to excel).
You might also consider a tabular schema which doesn't have this irregular data type within it which would probably prove to be more stable and effective in the long run.
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
add a comment |
You might find it useful to parse these into python objects using ast which can convert string representations back into python objectd. Try something like the following (I can't replicate exactly because I don't have your data):
import ast
df['transformed_tuples'] = df['List of Tuples'].apply(ast.literal_eval)
To avoid this arising in the first place you might consider the file format you choose to read/write to, for example pickle will retain the original information (I'm assuming this has come from a pandas DataFrame that has been saved to excel).
You might also consider a tabular schema which doesn't have this irregular data type within it which would probably prove to be more stable and effective in the long run.
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
You might find it useful to parse these into python objects using ast which can convert string representations back into python objectd. Try something like the following (I can't replicate exactly because I don't have your data):
import ast
df['transformed_tuples'] = df['List of Tuples'].apply(ast.literal_eval)
To avoid this arising in the first place you might consider the file format you choose to read/write to, for example pickle will retain the original information (I'm assuming this has come from a pandas DataFrame that has been saved to excel).
You might also consider a tabular schema which doesn't have this irregular data type within it which would probably prove to be more stable and effective in the long run.
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
edited Nov 21 '18 at 20:02
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
answered Nov 21 '18 at 19:57
Sven HarrisSven Harris
2,1861516
2,1861516
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53419579%2freading-from-excel-file-pandas-in-the-desired-type%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
The formatting is all over the place and I can't fix from my phone, but if you're hoping to use dataframes for lists of tuples in single cells, you're opening a door to a lot of pain. That's not how pandas works. You should shoot for a structure that has scalars in cells or, IMO, drop pandas if you can't do that.
– roganjosh
Nov 21 '18 at 19:56