skip to Main Content

I am trying to access the data between >.< from following code by using Selenium in Python.

<tbody>
                        <tr>                
                                <td>
    <div class="answer-votes" title="Asked 8 non-wiki questions with a total score of 164. Gave 84 non-wiki answers with a total score of 337." onclick="window.location.href='/search?q=user:37181+[python]'">337</div>
    <a href="/search?q=user:37181+[python]" class="post-tag" title="">python</a>
    <span class="item-multiplier" title="93 posts in the python tag"><span class="item-multiplier-x">×</span>&nbsp;<span class="item-multiplier-count">93</span></span></td>
                                <td>
    <div class="answer-votes" title=" Gave 4 non-wiki answers with a total score of 22." onclick="window.location.href='/search?q=user:37181+[django-templates]'">22</div>
    <a href="/search?q=user:37181+[django-templates]" class="post-tag" title="">django-templates</a>
    <span class="item-multiplier" title="4 posts in the django-templates tag"><span class="item-multiplier-x">×</span>&nbsp;<span class="item-multiplier-count">4</span></span></td>
                                <td>
    <div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 12." onclick="window.location.href='/search?q=user:37181+[slug]'">12</div>
    <a href="/search?q=user:37181+[slug]" class="post-tag" title="">slug</a>
    </td>
                                <td>
    <div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 8." onclick="window.location.href='/search?q=user:37181+[google-app-engine]'">8</div>
    <a href="/search?q=user:37181+[google-app-engine]" class="post-tag" title=""><img src="//i.stack.imgur.com/vobok.png" height="16" width="18" alt="" class="sponsor-tag-img">google-app-engine</a>
    </td>
                        </tr>
                        <tr>                
                                <td>
    <div class="answer-votes" title="Asked 1 non-wiki question with a total score of 89. Gave 56 non-wiki answers with a total score of 235." onclick="window.location.href='/search?q=user:37181+[django]'">235</div>
    <a href="/search?q=user:37181+[django]" class="post-tag" title="">django</a>
    <span class="item-multiplier" title="57 posts in the django tag"><span class="item-multiplier-x">×</span>&nbsp;<span class="item-multiplier-count">57</span></span></td>
                                <td>
    <div class="answer-votes" title="Asked 1 non-wiki question with a total score of 21. Gave 1 non-wiki answer with a total score of 22." onclick="window.location.href='/search?q=user:37181+[clang]'">22</div>
    <a href="/search?q=user:37181+[clang]" class="post-tag" title="">clang</a>
    <span class="item-multiplier" title="2 posts in the clang tag"><span class="item-multiplier-x">×</span>&nbsp;<span class="item-multiplier-count">2</span></span></td>
                                <td>
    <div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 12." onclick="window.location.href='/search?q=user:37181+[connect]'">12</div>
    <a href="/search?q=user:37181+[connect]" class="post-tag" title="show all posts by this user in 'connect'">connect</a>
    </td>
                                <td>
    <div class="answer-votes" title=" Gave 1 non-wiki answer with a total score of 8." onclick="window.location.href='/search?q=user:37181+[memcached]'">8</div>
    <a href="/search?q=user:37181+[memcached]" class="post-tag" title="">memcached</a>
    </td>
                        </tr>
                </tbody>

However, my program does not show the updated value of <td> when the compiler moves to the next <td>. Could you please guide me how I can fix this issue? Here is my code:

driver.get("https://stackoverflow.com/users/37181/alex-gaynor?tab=tags")
SMRTable = driver.find_elements_by_xpath("//*[@class='user-tags'] //td")



for i in SMRTable:

    print(i.get_attribute('innerHTML'))
    print(i.find_element_by_xpath("//div[@class='answer-votes']").get_attribute('innerHTML'))
    print(i.find_element_by_xpath("//*[@class='post-tag']").get_attribute('innerHTML'))
    print(i.find_element_by_xpath("//span[@class='item-multiplier-count']").get_attribute('innerHTML'))
    print('n')

2

Answers


  1. If you want to handle each td in table you need to specify dot (context character) in the beginning of each XPath expression, e.g. replace

    print(i.find_element_by_xpath("//div[@class='answer-votes']").get_attribute('innerHTML'))
    

    with

    print(i.find_element_by_xpath(".//div[@class='answer-votes']").get_attribute('innerHTML'))
    

    Otherwise, you will get the same values (values from the first td only) on each iteration

    Also note that you should not use get_attribute('innerHTML') to get text content of node, use text property instead:

    print(i.find_element_by_xpath(".//div[@class='answer-votes']").text)
    
    Login or Signup to reply.
  2. Your code attempt was near perfect. You need to take care of a couple of additional things:

    • While you find_elements_by_xpath() for SMRTable add the tagName which is table.
    • While printing the innerHTML as you are referring to the elements within SMRTable you have to set the reference through . (dot operator).
    • //div[@class='answer-votes'] is the immediate child tag, so change it as ./div[@class='answer-votes']
    • //*[@class='post-tag'] is always within a <a> tag so you need to use .//a[@class='post-tag']
    • Your effective code will be:

      driver.get("https://stackoverflow.com/users/37181/alex-gaynor?tab=tags")
      SMRTable = driver.find_elements_by_xpath("//table[@class='user-tags']//tr/td")
      for i in SMRTable:
          print(i.find_element_by_xpath("./div[@class='answer-votes']").get_attribute('innerHTML'))
          print(i.find_element_by_xpath(".//a[@class='post-tag']").get_attribute('innerHTML'))
          print(i.find_element_by_xpath(".//span[@class='item-multiplier-count']").get_attribute('innerHTML'))
      
    • Console Output:

      337
      python
      93
      22
      django-templates
      4
      
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search