I hope I am not expecting too much. Any help would be extremely useful for me, because I am stuck for days now.
I created a relatively simple wordpress plugin in php.
What is the plugin supposed to do ?
-The plugin is supposed to communicate with external api and import product data in json file.
Import is started by pressing "start import" button that is created by pluin in the wordpress menu – import products.
Example request:
curl -X GET
-H ‘X-Api-Key: [api-key]’
https://example.com/products/[PRODUCT ID]
[PRODUCT ID] is supposed to range from 1 to 10000
Plugin receives json file with product feed for every single request – every single [PRODUCT ID] Plugin creates a woocommerce product and attaches imported information.
Does the plugin work ?
- Yes and no, It imports first 100 products (in about 1min) correctly and then it just stops, sometimes resulting in error related to the request taking too much time to finish.
I know that the plugin doesn’t work because the import script is executed in the browser and gets timed out.
I also know that I should do this process in the background, split it into batches and queue execution. The thing is I tried many ways to do so but failed miserably.
I have composer and action scheduler installed.
Unfortunately everytime I try to split this into batches, use action scheduler It just doesn’t work or imports first product and stops.
I know that I’m dealing with large amount of products, I don’t have error handling, checking if imported product exists etc, but I really have to run this import once
so there is no need to make this plugin very refined. This has to run, import products and I can get rid of it.
I do have wp debug on, so my attepmts on using action scheduler didn’t create any errors or fatal errors, but it just didn’t work properly.
I have 1500MB Ram available, this is shared hosting server, I/O 1MB, 60 available processes, 88000/600000 Inodes used, 5/200GB disc space used.
I increased php maxExecutionTime to 3000, memorylimit 1536M, maxInputTime 3000, but that didn’t change anything.
I’m attaching my working code below. This is the version without my poor attemts on using action scheduler, splitting it into batches and running it in the backgroud.
I feel like this is going to be easier to read.
This code below runs in the web browser and works, but gets timed out.
I will be extremely grateful for any help with running it in the background.
Is it possible to just run this script from SSH linux terminal so it doesn’t get timed out ?
`
<?php
/**
* Plugin Name: Product Importer
* Description: Imports products from an external API
* Version: 1.0
* Author: me
* Author URI: http://www.example.com
*/
// Include the Autoscheduler library
require_once '/home/user/vendor/woocommerce/action-scheduler/action-scheduler.php';
add_action('admin_menu', 'add_import_button');
function add_import_button() {
add_menu_page('Import Products', 'Import Products', 'manage_options', 'import_products', 'import_products_page');
}
function import_products_page() {
echo '<h1>Import Products</h1>';
echo '<form method="post">';
echo '<input type="submit" name="start_import" value="Start Import">';
echo '</form>';
if (isset($_POST['start_import'])) {
import_function();
}
}
function import_function() {
$product_id = 1;
while($product_id < 10000){
$product_id++;
$api_key = 'my-api-key';
$headers = array(
'X-Api-Key: ' . $api_key,
);
$url = 'https://example.com/products/';
$product_url = $url . $product_id;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $product_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);
$product_data = json_decode($response, true);
// post array etc
// Set other product data as required.
}
}
}
`
2
Answers
OK, I managed to get it to work. I assume that it's dumb way to do this, but since it works it's fine for me. The only thing that is problematic now is the fact that the code for image import that I used previously (when code ran in browser) now makes the plugin stuck. Without it it works great and fast. Here is my current code:
This above works well. However when I add my old code for image imports it imports nothing or 1 product (without image lol) and becomes stuck. After a minute it imports the same product over and over again. Old code for image import:
And no, I don't put it after rescheduling, but after $post array. Honestly no idea why it would crash, tried smaller batches, longer wait between batches etc. I assume the code for images import is just very poorly written. Any solutions for this problem are going to be very appreciated ! :)
PS: I am totally fine with importing the first image available (there is around 6) and setting it up as thumbnail for product. I can give up on the gallery since It's going to slow down the process anyway.
EDIT: here is the fragment of json file received from api. (for better context of importing images):
etc....
One of the ways to solve this issue is using recursion where the code can run in the background. Take a look at the example below