Question 16618 in Laravel Question posted in
The official Laravel Documentation can be found here.

Symfony Dom-crawler not found in laravel 9

So i have this url $url = "localhost:8000/vehicles" that i want ot fetch through a cron job but the page returns html so i wanna use symfony dom crawler to get all the vehicles instead of regex

At the top of my file i added

use SymfonyComponentDomCrawlerCrawler;

To create a new instance i tried:

$crawler = new Crawler($data);

and i tried

$crawler = Crawler::create($data);

but that gives me an error, also tried adding

SymfonyComponentDomCrawlerCrawler::class,

to the service provider but when i execute the command:

composer dump-autoload it gives me the following error

In Crawler.php line 66:

  SymfonyComponentDomCrawlerCrawler::__construct(): Argument #1 ($node) must be of type DOMNodeList|DOMNode|array|string|null, IlluminateFoundationApplication given, called in C:xampphtdocsDrostMachinehandelDrostMachinehandelvendorlaravelfr   
  ameworksrcIlluminateFoundationProviderRepository.php on line 208


Script @php artisan package:discover --ansi handling the post-autoload-dump event returned with error code 1

I have no idea how to fix this.

The fucntion for fetching the url is below:

   public function handle()
    {
        $url = SettingsController::fetchSetting("fetch:vehicles");

        $ch = curl_init();

        curl_setopt($ch, CURLOPT_URL, $url);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_TIMEOUT, 10);

        $data = curl_exec($ch);

        $vehicles = $this->scrapeVehicles($data, $url);

        Log::debug($vehicles);

        curl_close($ch);
    }



    private function scrapeVehicles(string $data, string $url): array
    {
        $crawler = Crawler::create($data);
        $vehicles = $crawler->filter(".vehicleTile");

        return $vehicles;
    }

Contents of $data:

https://pastebin.com/GJ300KEv

Tags:

2

Answers


  1. 0

    Since not been tested, I’m not sure.

    Make sure you installed correct package

    composer require symfony/dom-crawler

    To initiate, use the full path. (since it’s not Laravel way(package))

    $crawler = SymfonyComponentDomCrawlerCrawler::create($data);
    Login or Signup to reply.
  2. 0

    composer require –dev "symfony/dom-crawler":"^6.3.x-dev"

    Sample crawler method
    namespace AppHttpControllers;

    use GuzzleHttpExceptionGuzzleException;
use SymfonyComponentDomCrawlerCrawler;

class CrawlerController extends Controller
{

    private $url;

    public function __construct()
    {
        $this->url = "https://www.everything5pounds.com/en/Shoes/c/shoes/results?q=&page=6";
    }

    public function index()
    {
        $client = new GuzzleHttpClient();
        try {
            $response = $client->request('GET', $this->url);
            if ($response->getStatusCode() == 200) {
                $res = json_decode($response->getBody());

                $results = $res->results;

                return $results;

                /*$results = (array)json_decode($res);


                $products = array();
                foreach ($results as $result) {
                    $product = [
                        "name" => $result["name"],
                    ];

                    array_push($products, $product);
                }

                return $products;*/

                //return $result;
                //return $this->parseContent($result);


            } else {
                return $response->getReasonPhrase();
            }
        } catch (GuzzleException $e) {
            return $e->getMessage();
        }
    }

    Parse content and store

     public function parseContent($result)
    {
        $crawler = new Crawler($result);

        $elements = $crawler->filter('.productGridItem')->each(function (Crawler $node, $i) {
            return $node;
        });

        $products = array();
        foreach ($elements as $item) {
            $image = $item->filter('.thumb .productMainLink img')->attr('src');
            $title = $item->filter('.productGridItem .details')->text();
            $price = $item->filter('.productGridItem .priceContainer')->text();

            $product = [
                "image" => 'https:' . $image,
                "title" => $title,
                "price" => $price,
            ];

            array_push($products, $product);
        }

        return $products;
    }
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search