Inserting 29000 rows via Laravel API in TimescaleDB takes 22min where inserting via Go in Influx1.7 takes 256ms – Nginx

I have to insert 29000 rows in 2 DB: TimescaleDB and Influx in my local machine (Ubuntu 20.04 8GB Ram)

When I insert into influx, it is quite fast. I do it with a Golang service that chunk data into piece of 10000 rows, and insert it into influx. it takes 256ms

But to insert into TimescaleDB, it is quite different. First, I use Laravel 8 / PHP8. Maybe not the fastest, but I’m stuck with it.

Then, I insert chunks of 100 row, with upsert() function.

This is taking 22min for 29000 rows. I use linux-valet, which use nginx server, php8, and postgres / timescaleDB 12.

I tried to replicate my production settings, in my .env, I have set

APP_ENV=production
APP_DEBUG=false

And I executed php artisan optimize

It still takes 22 min.

In production, I don’t have this issue, because I only insert 5648 rows, and it takes 123 ms for inlux, and 1sec for TimescaleDB. The only difference I guess is that in production, I still run PHP7.4

For both local and production, the timescaleDB config is the default one.

I understand Laravel is slow for bulk insert and that Go is much faster for this task.

Why is there such a difference and what should I do to make it quicker ?

EDIT: Added the piece of Laravel Code to insert in TimescaleDB:

public function store(Request $request)
    {
        ini_set('max_execution_time', -1);
        $body = $request->getContent();
        $meter = json_decode($body, true);
        $chunkSize = 100;
        $measures = $meter['Measures'];
        $chunkedRaws = collect($measures)->chunk($chunkSize);
        $ok = 0;
        $nok = 0;
        foreach ($chunkedRaws as $raws) {
            $raws = $raws->map(function ($raw) use ($meter) {
                if (!isset($raw['s'])) {
                    $source = "Pamela";
                } else {
                    $source = $raw['s'];
                }
                return new AppModelsRaw([
                    'operation_id' => $meter['OperationID'],
                    'meter_id' => $meter['ID'],
                    'conso_prod' => $meter['ConsoProd'],
                    'timestep' => $meter['Timestep'],
                    'unit' => $meter['Unit'],
                    'source' => $source,
                    'time' => Carbon::parse($raw['t'])->toIso8601ZuluString(),
                    'delta' => $raw['d'],
                ]);
            })->toArray();
            try {
                DB::disableQueryLog();
                $raws = collect($raws)->unique(function ($item) { // eliminate duplicates
                    return $item['time'] . $item['operation_id'] . $item['meter_id'] . $item['conso_prod'];
                })->toArray();
                DB::table('raws')->upsert($raws, ['time', 'operation_id', 'meter_id', 'conso_prod'], ['delta']);
//                DB::table('raws')->insert($raws);
                $ok += $chunkSize;
            } catch (PDOException $e) {
                $nok += $chunkSize;
                Log::warning($e->getMessage());
            }
        }
        return response()->json(['message' => $ok . " Raws has been inserted,n " . $nok . " Failed"], 200);
    }