Question 45804 in CSS Question posted in
Some basic documentation can be found here.

Scraping data from a website using scrapy – CSS

I am trying to scrape data from a website using scrapy. This is the css path

<div _ngcontent-amb-c25="" appcoloredmultiplier="" class="bubble-multiplier font-weight-bold" style="padding: 2px 11px; border-radius: 11px; color: rgb(52, 180, 255);"> 1.21x </div>

but I’d like to extract data between the tag which is 1.21x how do I update my code to extract the data I stated.

def parse(self, response):
# Extract game history data from the webpage
game_history_elements = response.css('div.bubble-multiplier')

# Extract the multiplier value from each game history element
game_history = [re.search(r'(d+.d+)x', element.css('::text').get()).group(1) for element in game_history_elements]

# Print the game history data
print(game_history)
Tags:

2

Answers


  1. 0 
    def parse(self, response):
    # Extract game history data from the webpage
    game_history_elements = response.css('div.bubble-multiplier::text')

    # Extract the multiplier value from each game history element
    game_history = [re.search(r'(d+.d+)x', element.get()).group(1) for element in game_history_elements]

    # Print the game history data
    print(game_history)

    try this

    Login or Signup to reply.
  2. 0

    As mentioned in the comments, you can get the text between tags using the ::text css directive in your xpath expression and then applying either the get or getall methods on the selector.

    If there is more than one div with the class bubble-multiplier, and you want the text for each of them then you would use getall(), on the other hand if there is only a single matching element or you simply only want the first, then you would use getall().

    def parse(self, response):
    game_history = response.css('div.bubble-multiplier::text').get()
    print(game_history)

    or

    def parse(self, response):
    game_history = response.css('div.bubble-multiplier::text').getall()
    print(game_history)

    You can still use getall when there is only a single match, the only difference would be that the the return value would be a list with just the one string as contents.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search