Question 207052 in PHP Question posted in
Official manual can be found here.

Getting different response for same request b/w php and python

This is my first question on Stack Overflow. Someone, please help me as I am struggling to find an answer for the past 24 hours.

I tried sending a normal get request to a URL with postman and python. Both returned the same response. But when I tried sending the same get request from a PHP file, it is returning a different response, which is not intended. But all are simple get requests. I want the request made from the PHP file to return the same response as sending a request from postman or python.

Here is the code for both python and PHP.

Python:

url = "https://www.youtube.com/watch?v=Q2A5igb7pmA"
response = requests.request("GET", url)
print(response.text)

PHP:

<?php
$url = "https://www.youtube.com/watch?v=Q2A5igb7pmA"; 
$text = file_get_contents($url); 
echo $text;
?>

I am trying to build a simple YouTube scraper. Though both the responses look almost similar, if you check out the object ytInitialPlayerResponse in their responses, you can find the difference in
ytInitialPlayerResponse["streamingData"]["formats"][0]["url"]

PS: I also tried adding headers like user-agent, curl request, storing cookies, but nothing seem to work. The PHP file is hosted in Hostinger.

Tags:

1

Answers


  1. 0

    Different result wherease query looks to be the same can be caused by the
    User-Agent. Some website; for compatibility reason use to take care about it and can render different template. For example in php you can do that with:
    https://browscap.org/, https://github.com/browscap/browscap-php

    For example google search engine will return different template for old User-Agent refering to Internet Explorer.

    In python to specify User Agent in a request you can do:

    url = "https://www.youtube.com/watch?v=Q2A5igb7pmA"
headers = {'User-agent': 'My super user-agent'}
response = requests.get(url, headers=headers)

    If you need a list of existing User Agent you can use this website:
    https://explore.whatismybrowser.com/useragents/explore/

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search