I am trying to access the data between >.< from following code by using Selenium in Python.
<tbody>
<tr>
<td>
<div class="answer-votes" title="Asked 8 non-wiki questions with a total score of 164. Gave 84 non-wiki answers with a total score of 337." onclick="window.location.href='/search?q=user:37181+[python]'">337</div>
<a href="/search?q=user:37181+[python]" class="post-tag" title="">python</a>
<span class="item-multiplier" title="93 posts in the python tag"><span class="item-multiplier-x">×</span> <span class="item-multiplier-count">93</span></span></td>
<td>
<div class="answer-votes" title=" Gave 4 non-wiki answers with a total score of 22." onclick="window.location.href='/search?q=user:37181+[django-templates]'">22</div>
<a href="/search?q=user:37181+[django-templates]" class="post-tag" title="">django-templates</a>
<span class="item-multiplier" title="4 posts in the django-templates tag"><span class="item-multiplier-x">×</span> <span class="item-multiplier-count">4</span></span></td>
<td>
<div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 12." onclick="window.location.href='/search?q=user:37181+[slug]'">12</div>
<a href="/search?q=user:37181+[slug]" class="post-tag" title="">slug</a>
</td>
<td>
<div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 8." onclick="window.location.href='/search?q=user:37181+[google-app-engine]'">8</div>
<a href="/search?q=user:37181+[google-app-engine]" class="post-tag" title=""><img src="//i.stack.imgur.com/vobok.png" height="16" width="18" alt="" class="sponsor-tag-img">google-app-engine</a>
</td>
</tr>
<tr>
<td>
<div class="answer-votes" title="Asked 1 non-wiki question with a total score of 89. Gave 56 non-wiki answers with a total score of 235." onclick="window.location.href='/search?q=user:37181+[django]'">235</div>
<a href="/search?q=user:37181+[django]" class="post-tag" title="">django</a>
<span class="item-multiplier" title="57 posts in the django tag"><span class="item-multiplier-x">×</span> <span class="item-multiplier-count">57</span></span></td>
<td>
<div class="answer-votes" title="Asked 1 non-wiki question with a total score of 21. Gave 1 non-wiki answer with a total score of 22." onclick="window.location.href='/search?q=user:37181+[clang]'">22</div>
<a href="/search?q=user:37181+[clang]" class="post-tag" title="">clang</a>
<span class="item-multiplier" title="2 posts in the clang tag"><span class="item-multiplier-x">×</span> <span class="item-multiplier-count">2</span></span></td>
<td>
<div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 12." onclick="window.location.href='/search?q=user:37181+[connect]'">12</div>
<a href="/search?q=user:37181+[connect]" class="post-tag" title="show all posts by this user in 'connect'">connect</a>
</td>
<td>
<div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 8." onclick="window.location.href='/search?q=user:37181+[memcached]'">8</div>
<a href="/search?q=user:37181+[memcached]" class="post-tag" title="">memcached</a>
</td>
</tr>
</tbody>
However, my program does not show the updated value of <td>
when the compiler moves to the next <td>
. Could you please guide me how I can fix this issue? Here is my code:
driver.get("https://stackoverflow.com/users/37181/alex-gaynor?tab=tags")
SMRTable = driver.find_elements_by_xpath("//*[@class='user-tags'] //td")
for i in SMRTable:
print(i.get_attribute('innerHTML'))
print(i.find_element_by_xpath("//div[@class='answer-votes']").get_attribute('innerHTML'))
print(i.find_element_by_xpath("//*[@class='post-tag']").get_attribute('innerHTML'))
print(i.find_element_by_xpath("//span[@class='item-multiplier-count']").get_attribute('innerHTML'))
print('n')
2
Answers
If you want to handle each
td
intable
you need to specify dot (context character) in the beginning of each XPath expression, e.g. replacewith
Otherwise, you will get the same values (values from the first
td
only) on each iterationAlso note that you should not use
get_attribute('innerHTML')
to get text content of node, usetext
property instead:Your code attempt was near perfect. You need to take care of a couple of additional things:
find_elements_by_xpath()
for SMRTable add the tagName which istable
..
(dot operator).//div[@class='answer-votes']
is the immediate child tag, so change it as./div[@class='answer-votes']
//*[@class='post-tag']
is always within a<a>
tag so you need to use.//a[@class='post-tag']
Your effective code will be:
Console Output: