Question 45165 in Html Question posted in
Some basic documentation can be found here.

Scrapy selector doesn't "see" an element that is present on the webpage – Html

I want to parse the following webpage:
https://mafiaworldtour.com/tournaments/2653

And I need to find the following element:

//html/body/div[1]/div/section[2]/div/div/div/div[1]/div[1]/div/div[2]/div/div[1]/div[2]/span/text()

When I search it on the webpage via inspect, it is clearly present, but
city = response.xpath('//html/body/div[1]/div/section[2]/div/div/div/div[1]/div[1]/div/div[2]/div/div[1]/div[2]/span/text()').extract_first() returns None.

What is the reason for this?

I expect to get the city Хайфа, Израиль of the tournament via xpath.

Tags:

2

Answers


  1. 0

    Using my own project retrieveCssOrXpathSelectorFromTextOrNode to fetch the full query:

    x('Хайфа, Израиль');
//body/div[@class="site-wrapper"]/div[@class="main"][@role="main"]/section[@class="page-content"]/div[@class="container"]/div[@class="tabs"]/div[@class="tab-content"]/div[@class="tab-pane fade in active "][@id="general"]/div[@class="row"]/div[@class="col-md-12"]/div[@class="table-responsive"]/div[@class="responsive-info-table"]/div[@class="row with-top-border"]/div[@class="col-md-6"]/span[@class="small_content"]

    It’s always better to have these specific XPath query’s than the one with relative path like auto-generated by chrome dev tools:

    //html/body/div[1]/div/section[2]/div/div......

    But you can remove the useless part, should be like:

    (From chrome dev tools, or firefox console):

    $x('//span[@class="small_content"]')[0].innerText

    or in your case:

    response.xpath('//span[@class="small_content"]/text()').extract_first()

    Output

    " Хайфа, Израиль"
    Login or Signup to reply.
  2. 0

    you can use both CSS selector orXPATH

    CSS selector

    response.css('.small_content::text').get()

    XPATH

    response.xpath('//span[@class="small_content"]/text()').get()
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search