skip to Main Content

I have a long PHP script (for scraping), It takes around 2 hours to execute, I want to run this on weekly basis, I have cron job option in Cpanel and i know how to use it, But i think it is not useful for that long scripts

I read some other answers about long PHP script, They say long PHP script should be executed via shell. I read this blog http://symcbean.blogspot.com/2010/02/php-and-long-running-processes.html But i couldn’t understand anything,

I have never worked with shell, But i can spend time to learn it, But would it be possible to run the script via shell and also on weekly basis at same time automatically

2

Answers


  1. When you run a cron job it basically runs a shell command of your choosing.
    That shell command can be php <your_script_name_here>.php
    I think anyway that cron job is the way to go (scheduled task on windows machine).

    Login or Signup to reply.
  2. If you have a script fro scrapping I recommend you to use HHVM instead of php-fpm you gonna gain lots of performance.

    And in second part you gonna need may be a worker or a job queue system to run your script via a cronjob.

    If you insert data in your data-base don’t insert row by row but create an array with all data and then insert all in one time. (there is lot of thing you can do to have best performance)

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search