python-requests Questions

Beatifulsoup output is Json not HTML, so I cannot parse it using .find methods of bs4

December 14, 2022
Mohamed Hedeya
2 Answers

I'm trying to scrape this site. I used the following code: import requests import json from bs4 import BeautifulSoup api_url ='https://seniorcarefinder.com/Providers/List' headers= { "Content-Type":"application/json; charset=utf-8", "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:108.0) Gecko/20100101 Firefox/108.0"} body_first_page={"Services":["Independent Living","Assisted Living","Long-Term Care /…

VIEW QUESTION

Loop through multiple items with the same name in a JSON and assign to variable Python

December 9, 2022
kiestuthridge23
2 Answers

I'm trying to loop through a JSON that has multiple object keys with the same name. I need to grab them and be able to differentiate them accordingly. Below is my JSON, I need to extract the expenses, date_posted &…

VIEW QUESTION

iterating through a json elemnt after iterating through a list of jsons

December 7, 2022
Madjid
2 Answers

I have a list of jsons i made after web scraping a store website using this code : responses.append(requests.request("POST", url, data=payload, headers=headers).json()) now my issue is that each json , has a specific number of elements, and each element contains…

VIEW QUESTION

Json – How can I put this code inside a function?

December 5, 2022
Uri
2 Answers

On October 2022 I started my MSc in Data Science. I never coded before. My academical background is I achieved a Bachelor's Degree of Economics five years ago. The teacher of subject of Python put the following problem: There's an…

VIEW QUESTION

Navigate in JSON with nultiple keys

December 4, 2022
Paul Mark
2 Answers

I'm trying to get a key from a JSON from a website using the following code: import json import requests from bs4 import BeautifulSoup url = input('Enter url:') html = requests.get(url) soup = BeautifulSoup(html.text,'html.parser') data = json.loads(soup.find('script', type='application/json').text) print(data) print("####################################")…

VIEW QUESTION

How to put a dictionary in a JSON?

December 2, 2022
Murilo Souza
2 Answers

I'm working with a REST API, and I need to return a JSON with my values to it. However, I need the items of the payload variable to show all the items inside the cart_item. I have this: payload =…

VIEW QUESTION

Need help to understand the documentation of Sentry API – React native

September 9, 2022
user13603107
2 Answers

I am making an API request to the sentry API, like this: _URL = "https://sentry.io/api/0/projects/aaaa-7y/aaaa/issues/" headers = {'Authorization': 'Bearer 11111111111111111111111', 'Content-Type': 'application/json'} params = {"statsPeriod":"24h", 'cursor':'100:-1:1'} r = requests.get(url = _URL, headers = headers, params=params, verify=False) data = r.json() and…

VIEW QUESTION

Ubuntu – Two of three scripts based on bs4 not work, but theoretically should

August 27, 2022
JohnLocke
2 Answers

I prepared three little scripts, that theoretically the should do the same, but two not work properly. I'm not sure what could be wrong. I used PyCharm, and packages was installed inside projects, not globally with PIP. First script don't…

VIEW QUESTION

Mitigating TCP connection resets in AWS Fargate – Amazon web services

I am using Amazon ECS on AWS Fargate, My instances can access the internet, but the connection drops after 350 seconds. On average, out of 100 times, my service is getting ConnectionResetError: [Errno 104] Connection reset by peer error approximately…

VIEW QUESTION

Scraping Medium's clap data without Selenium – Nginx

July 22, 2022
Sefa Kalkan
2 Answers

I'm trying to scrape clap data from medium let's say this is the link. When I inspect it looks like in this photo. My code looks like this : URL = "https://medium.com/@xdxxxx4713/basic-settings-of-nginx-aeace532534f" page = requests.get(URL) soup = BeautifulSoup(page.content, 'html.parser') print(soup.prettify())…

VIEW QUESTION