BeautifulSoup tag.children only gets odd-numbered elements
I want to move the elements from one tag to another tag using the following code:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in soup.body.children:
d.append(tag)
However, this yields...
>>> d
<div><p>I wish I was bold.</p><p>me three</p><p>5</p></div>
Only the odd-numbered elements were moved. I checked soup.body.children to see what it looks like (before moving anything), and all of the tags appear to be there:
>>> list(soup.body.children)
[<p>I wish I was bold.</p>, <p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
When I iterate over list(soup.body.children), then everything works as expected:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in list(soup.body.children):
d.append(tag)
>>> d
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
Why does iterating over soup.body.children only access odd-numbered tags, but iterating over list(soup.body.children) accesses all of them?
python beautifulsoup
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
add a comment |
I want to move the elements from one tag to another tag using the following code:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in soup.body.children:
d.append(tag)
However, this yields...
>>> d
<div><p>I wish I was bold.</p><p>me three</p><p>5</p></div>
Only the odd-numbered elements were moved. I checked soup.body.children to see what it looks like (before moving anything), and all of the tags appear to be there:
>>> list(soup.body.children)
[<p>I wish I was bold.</p>, <p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
When I iterate over list(soup.body.children), then everything works as expected:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in list(soup.body.children):
d.append(tag)
>>> d
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
Why does iterating over soup.body.children only access odd-numbered tags, but iterating over list(soup.body.children) accesses all of them?
python beautifulsoup
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
Which parser are you using?
– Stephen Cowley
Nov 20 '18 at 19:33
@StephenCowleylxml, but why would the parser make a difference? I thought that once you've parsed the markup, it's all just native soup objects.
– reynoldsnlp
Nov 20 '18 at 19:38
Yeah, I wasn't sure if it mattered, but just wanted to match your configuration as close as possible.
– Stephen Cowley
Nov 20 '18 at 19:44
add a comment |
I want to move the elements from one tag to another tag using the following code:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in soup.body.children:
d.append(tag)
However, this yields...
>>> d
<div><p>I wish I was bold.</p><p>me three</p><p>5</p></div>
Only the odd-numbered elements were moved. I checked soup.body.children to see what it looks like (before moving anything), and all of the tags appear to be there:
>>> list(soup.body.children)
[<p>I wish I was bold.</p>, <p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
When I iterate over list(soup.body.children), then everything works as expected:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in list(soup.body.children):
d.append(tag)
>>> d
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
Why does iterating over soup.body.children only access odd-numbered tags, but iterating over list(soup.body.children) accesses all of them?
python beautifulsoup
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
I want to move the elements from one tag to another tag using the following code:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in soup.body.children:
d.append(tag)
However, this yields...
>>> d
<div><p>I wish I was bold.</p><p>me three</p><p>5</p></div>
Only the odd-numbered elements were moved. I checked soup.body.children to see what it looks like (before moving anything), and all of the tags appear to be there:
>>> list(soup.body.children)
[<p>I wish I was bold.</p>, <p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
When I iterate over list(soup.body.children), then everything works as expected:
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
for tag in list(soup.body.children):
d.append(tag)
>>> d
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
Why does iterating over soup.body.children only access odd-numbered tags, but iterating over list(soup.body.children) accesses all of them?
python beautifulsoup
python beautifulsoup
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
asked Nov 20 '18 at 19:28
reynoldsnlpreynoldsnlp
6661826
6661826
Which parser are you using?
– Stephen Cowley
Nov 20 '18 at 19:33
@StephenCowleylxml, but why would the parser make a difference? I thought that once you've parsed the markup, it's all just native soup objects.
– reynoldsnlp
Nov 20 '18 at 19:38
Yeah, I wasn't sure if it mattered, but just wanted to match your configuration as close as possible.
– Stephen Cowley
Nov 20 '18 at 19:44
add a comment |
Which parser are you using?
– Stephen Cowley
Nov 20 '18 at 19:33
@StephenCowleylxml, but why would the parser make a difference? I thought that once you've parsed the markup, it's all just native soup objects.
– reynoldsnlp
Nov 20 '18 at 19:38
Yeah, I wasn't sure if it mattered, but just wanted to match your configuration as close as possible.
– Stephen Cowley
Nov 20 '18 at 19:44
Which parser are you using?
– Stephen Cowley
Nov 20 '18 at 19:33
Which parser are you using?
– Stephen Cowley
Nov 20 '18 at 19:33
@StephenCowley
lxml, but why would the parser make a difference? I thought that once you've parsed the markup, it's all just native soup objects.– reynoldsnlp
Nov 20 '18 at 19:38
@StephenCowley
lxml, but why would the parser make a difference? I thought that once you've parsed the markup, it's all just native soup objects.– reynoldsnlp
Nov 20 '18 at 19:38
Yeah, I wasn't sure if it mattered, but just wanted to match your configuration as close as possible.
– Stephen Cowley
Nov 20 '18 at 19:44
Yeah, I wasn't sure if it mattered, but just wanted to match your configuration as close as possible.
– Stephen Cowley
Nov 20 '18 at 19:44
add a comment |
2 Answers
2
active
oldest
votes
When you append to the d tag in the first case, you are actually changing the size of soup.body.children as you go, since it moves the tags from soup to d.
Thus it grabs the tag at 0 and moves it to d. When it goes back for tag at 1, they have all shifted over, and it grabs the tag originally at index 2.
One way to see this in action is to actually print list(soup.body.children) in each iteration. Something like this:
for i, tag in enumerate(soup.body.children):
d.append(tag)
print(i)
print(list(soup.body.children))
print()
Output:
0 #<-- It's accessing this element
[<p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
1
[<p>me too</p>, <p>me 4</p>, <p>5</p>]
2
[<p>me too</p>, <p>me 4</p>]
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
1
Of course! That's why making a copy withlistdoesn't fail. Well done.
– reynoldsnlp
Nov 20 '18 at 19:40
add a comment |
It seems like the items being moved from 'soup' to 'd' are affected the iterator, i.e. removing them and changing the 'soup' generator as you iterate through the items.
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
children = list(soup.body.children).copy()
for tag in children:
d.append(tag)
print(d)
Creating a copy of the list solves the problem.
output
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
1
.copy()is unnecessary, sincelist()already makes a copy, as I showed in the OP.
– reynoldsnlp
Nov 20 '18 at 19:47
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400210%2fbeautifulsoup-tag-children-only-gets-odd-numbered-elements%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
When you append to the d tag in the first case, you are actually changing the size of soup.body.children as you go, since it moves the tags from soup to d.
Thus it grabs the tag at 0 and moves it to d. When it goes back for tag at 1, they have all shifted over, and it grabs the tag originally at index 2.
One way to see this in action is to actually print list(soup.body.children) in each iteration. Something like this:
for i, tag in enumerate(soup.body.children):
d.append(tag)
print(i)
print(list(soup.body.children))
print()
Output:
0 #<-- It's accessing this element
[<p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
1
[<p>me too</p>, <p>me 4</p>, <p>5</p>]
2
[<p>me too</p>, <p>me 4</p>]
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
1
Of course! That's why making a copy withlistdoesn't fail. Well done.
– reynoldsnlp
Nov 20 '18 at 19:40
add a comment |
When you append to the d tag in the first case, you are actually changing the size of soup.body.children as you go, since it moves the tags from soup to d.
Thus it grabs the tag at 0 and moves it to d. When it goes back for tag at 1, they have all shifted over, and it grabs the tag originally at index 2.
One way to see this in action is to actually print list(soup.body.children) in each iteration. Something like this:
for i, tag in enumerate(soup.body.children):
d.append(tag)
print(i)
print(list(soup.body.children))
print()
Output:
0 #<-- It's accessing this element
[<p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
1
[<p>me too</p>, <p>me 4</p>, <p>5</p>]
2
[<p>me too</p>, <p>me 4</p>]
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
1
Of course! That's why making a copy withlistdoesn't fail. Well done.
– reynoldsnlp
Nov 20 '18 at 19:40
add a comment |
When you append to the d tag in the first case, you are actually changing the size of soup.body.children as you go, since it moves the tags from soup to d.
Thus it grabs the tag at 0 and moves it to d. When it goes back for tag at 1, they have all shifted over, and it grabs the tag originally at index 2.
One way to see this in action is to actually print list(soup.body.children) in each iteration. Something like this:
for i, tag in enumerate(soup.body.children):
d.append(tag)
print(i)
print(list(soup.body.children))
print()
Output:
0 #<-- It's accessing this element
[<p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
1
[<p>me too</p>, <p>me 4</p>, <p>5</p>]
2
[<p>me too</p>, <p>me 4</p>]
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
When you append to the d tag in the first case, you are actually changing the size of soup.body.children as you go, since it moves the tags from soup to d.
Thus it grabs the tag at 0 and moves it to d. When it goes back for tag at 1, they have all shifted over, and it grabs the tag originally at index 2.
One way to see this in action is to actually print list(soup.body.children) in each iteration. Something like this:
for i, tag in enumerate(soup.body.children):
d.append(tag)
print(i)
print(list(soup.body.children))
print()
Output:
0 #<-- It's accessing this element
[<p>me too</p>, <p>me three</p>, <p>me 4</p>, <p>5</p>]
1
[<p>me too</p>, <p>me 4</p>, <p>5</p>]
2
[<p>me too</p>, <p>me 4</p>]
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
edited Nov 20 '18 at 19:48
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
answered Nov 20 '18 at 19:38
Stephen CowleyStephen Cowley
1,206518
1,206518
1
Of course! That's why making a copy withlistdoesn't fail. Well done.
– reynoldsnlp
Nov 20 '18 at 19:40
add a comment |
1
Of course! That's why making a copy withlistdoesn't fail. Well done.
– reynoldsnlp
Nov 20 '18 at 19:40
1
1
Of course! That's why making a copy with
list doesn't fail. Well done.– reynoldsnlp
Nov 20 '18 at 19:40
Of course! That's why making a copy with
list doesn't fail. Well done.– reynoldsnlp
Nov 20 '18 at 19:40
add a comment |
It seems like the items being moved from 'soup' to 'd' are affected the iterator, i.e. removing them and changing the 'soup' generator as you iterate through the items.
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
children = list(soup.body.children).copy()
for tag in children:
d.append(tag)
print(d)
Creating a copy of the list solves the problem.
output
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
1
.copy()is unnecessary, sincelist()already makes a copy, as I showed in the OP.
– reynoldsnlp
Nov 20 '18 at 19:47
add a comment |
It seems like the items being moved from 'soup' to 'd' are affected the iterator, i.e. removing them and changing the 'soup' generator as you iterate through the items.
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
children = list(soup.body.children).copy()
for tag in children:
d.append(tag)
print(d)
Creating a copy of the list solves the problem.
output
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
1
.copy()is unnecessary, sincelist()already makes a copy, as I showed in the OP.
– reynoldsnlp
Nov 20 '18 at 19:47
add a comment |
It seems like the items being moved from 'soup' to 'd' are affected the iterator, i.e. removing them and changing the 'soup' generator as you iterate through the items.
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
children = list(soup.body.children).copy()
for tag in children:
d.append(tag)
print(d)
Creating a copy of the list solves the problem.
output
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
It seems like the items being moved from 'soup' to 'd' are affected the iterator, i.e. removing them and changing the 'soup' generator as you iterate through the items.
soup = BeautifulSoup("<p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p>")
d = soup.new_tag('div')
children = list(soup.body.children).copy()
for tag in children:
d.append(tag)
print(d)
Creating a copy of the list solves the problem.
output
<div><p>I wish I was bold.</p><p>me too</p><p>me three</p><p>me 4</p><p>5</p></div>
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
answered Nov 20 '18 at 19:39
Victor 'Chris' CabralVictor 'Chris' Cabral
1,5221222
1,5221222
1
.copy()is unnecessary, sincelist()already makes a copy, as I showed in the OP.
– reynoldsnlp
Nov 20 '18 at 19:47
add a comment |
1
.copy()is unnecessary, sincelist()already makes a copy, as I showed in the OP.
– reynoldsnlp
Nov 20 '18 at 19:47
1
1
.copy() is unnecessary, since list() already makes a copy, as I showed in the OP.– reynoldsnlp
Nov 20 '18 at 19:47
.copy() is unnecessary, since list() already makes a copy, as I showed in the OP.– reynoldsnlp
Nov 20 '18 at 19:47
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53400210%2fbeautifulsoup-tag-children-only-gets-odd-numbered-elements%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Which parser are you using?
– Stephen Cowley
Nov 20 '18 at 19:33
@StephenCowley
lxml, but why would the parser make a difference? I thought that once you've parsed the markup, it's all just native soup objects.– reynoldsnlp
Nov 20 '18 at 19:38
Yeah, I wasn't sure if it mattered, but just wanted to match your configuration as close as possible.
– Stephen Cowley
Nov 20 '18 at 19:44