skip to Main Content

I have 2 servers installed in 2 differents localisation, with a synchronisation files managed by heartbeat + drbd. They are running since few months and the date of the last main server reboot is : 2022-12-07 15:44

This night on the main server, the Mariadb service has been stopped. See below the logs :

Dec 18 06:50:14 nodegra systemd[1]: Starting Daily apt upgrade and clean activities...
Dec 18 06:50:20 nodegra pacemaker-controld[1131]:  notice: High CPU load detected: 1.370000
Dec 18 06:50:50 nodegra pacemaker-controld[1131]:  notice: High CPU load detected: 1.620000
Dec 18 06:51:20 nodegra pacemaker-controld[1131]:  notice: High CPU load detected: 1.980000
Dec 18 06:51:50 nodegra pacemaker-controld[1131]:  notice: High CPU load detected: 2.120000
Dec 18 06:52:20 nodegra pacemaker-controld[1131]:  notice: High CPU load detected: 2.210000
Dec 18 06:52:36 nodegra systemd[1]: Stopping MariaDB 10.5.15 database server...
Dec 18 06:52:36 nodegra mariadbd[1949]: 2022-12-18  6:52:36 0 [Note] /usr/sbin/mariadbd (initiated by: unknown): Normal shutdown
Dec 18 06:52:36 nodegra mariadbd[1949]: 2022-12-18  6:52:36 0 [Note] Event Scheduler: Purging the queue. 0 events
Dec 18 06:52:36 nodegra kayelles.service[1967382]: Warning: got packets out of order. Expected 2 but received 1
Dec 18 06:52:37 nodegra kayelles.service[1967382]: Warning: got packets out of order. Expected 2 but received 1
Dec 18 06:52:37 nodegra kayelles.service[1967382]: Warning: got packets out of order. Expected 2 but received 1
Dec 18 06:52:37 nodegra kayelles.service[1967382]: Warning: got packets out of order. Expected 2 but received 1
Dec 18 06:52:37 nodegra kayelles.service[1967382]: Warning: got packets out of order. Expected 15 but received 1
Dec 18 06:52:37 nodegra mariadbd[1949]: 2022-12-18  6:52:37 0 [Note] InnoDB: FTS optimize thread exiting.
Dec 18 06:52:37 nodegra kayelles.service[1967382]: db error Error: Connection was killed

I don’t find/understand the reason of the MariaDb service stop.
Is there a way to find the requester ?

2

Answers


  1. Chosen as BEST ANSWER

    Ths for your reply. Heartbeat log are empty fot this day and there is nothing special about in the pacemaker log :

    Dec 18 06:50:50 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 1.620000 Dec 18 06:51:20 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 1.980000 Dec 18 06:51:50 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 2.120000 Dec 18 06:52:20 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 2.210000 Dec 18 06:52:50 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 3.080000 Dec 18 06:53:20 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 5.630000 Dec 18 06:53:50 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 5.670000 Dec 18 06:54:20 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 6.120000 Dec 18 06:54:50 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 7.070000 Dec 18 06:55:20 nodeGRA pacemaker-controld [1131] (throttle_check_thresholds) notice: High CPU load detected: 7.360000

    I don't know what I have to check on more or that I have to set to avoid this issue again.


  2. "initiated by: unknown" is an indicator that it wasn’t a user doing SHUTDOWN and was as a result of a signal.

    Base on the close occurrence of the packmaker-control notice message I suspect it may have been the cause. Look closer at your pacemaker/heartbeat logs.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search