ISPConfig component locking apt update in Ubuntu 22.04

nah, i'd followed your very helpful tutorial when testing it first and when configuring the live servers.
all the servers are using /etc/apt/apt.conf.d/52unattended-upgrades-local so i doubt anything got overwritten..
i reckon i just didn't add the ondrej repo to the allowed origins on the live servers.. i'd have been making minimal changes to the default config options until i'd had it working for a while and i trusted what it was already doing.

 

I've got the same issue and I'm quite sure that unattended updates has nothing to do with it, because I've disabled it. See for my full description: https://forum.howtoforge.com/threads/ubuntu-is-not-updating.91091/

I also notice that the file /var/log/ubuntu-advantage.log is not updated while the "apt-get update"-process is still running. Most likely it's updated just before the "apt-get update"-process is started. I noticed that the process was started at 14:00. A transcription from that log-file:

Code:

["2023-09-04T13:00:03.453", "DEBUG", "root", "load_file", 452, "Reading file: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-04T13:00:03.453", "DEBUG", "uaclient.files.files", "read", 55, "File does not exist: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-04T14:00:02.840", "DEBUG", "uaclient.config", "parse_config", 644, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}]
["2023-09-04T14:00:02.840", "DEBUG", "root", "load_file", 452, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2023-09-04T14:00:02.844", "DEBUG", "root", "load_file", 452, "Reading file: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-04T14:00:02.844", "DEBUG", "uaclient.files.files", "read", 55, "File does not exist: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-04T14:00:02.844", "DEBUG", "root", "load_file", 452, "Reading file: /etc/os-release", {}]
["2023-09-04T14:00:02.858", "DEBUG", "uaclient.config", "parse_config", 644, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}]
["2023-09-04T14:00:02.861", "DEBUG", "root", "load_file", 452, "Reading file: /var/lib/ubuntu-advantage/status.json", {}]
["2023-09-04T14:00:02.859", "DEBUG", "root", "load_file", 452, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2023-09-04T14:00:02.864", "DEBUG", "root", "load_file", 452, "Reading file: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-04T14:00:02.864", "DEBUG", "uaclient.files.files", "read", 55, "File does not exist: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-04T14:00:02.866", "DEBUG", "root", "load_file", 452, "Reading file: /var/lib/ubuntu-advantage/private/machine-token.json", {}]
["2023-09-04T14:00:02.867", "DEBUG", "uaclient.files.files", "read", 55, "File does not exist: /var/lib/ubuntu-advantage/private/machine-token.json", {}]
["2023-09-05T19:30:02.503", "DEBUG", "uaclient.config", "parse_config", 644, "Using client configuration file at /etc/ubuntu-advantage/uaclient.conf", {}]
["2023-09-05T19:30:02.503", "DEBUG", "root", "load_file", 452, "Reading file: /etc/ubuntu-advantage/uaclient.conf", {}]
["2023-09-05T19:30:02.508", "DEBUG", "root", "load_file", 452, "Reading file: /var/lib/ubuntu-advantage/user-config.json", {}]
["2023-09-05T19:30:02.508", "DEBUG", "uaclient.files.files", "read", 55, "File does not exist: /var/lib/ubuntu-advantage/user-config.json", {}]

Next time when this happens I'll run a lsof to see the open files and try to check the exact time of the process.

 

Still working with the file move. So it could be related..

 

Installed 8 updates today without any issues. Problem is gone for a few weeks now after moving 100-monitor_system_update.inc.php.

 

just double checked on my servers, i've got 5 of them with unattended-upgrades installed, all running ubuntu 22.04.3.
they all installed updates as scheduled yesterday morning, and today, on all 5, i've manually run 'apt update' without any issue.
i haven't made any changes to, or moved, the 100-monitor_system_update.inc.php file on any of these servers.
so maybe it was just an ubuntu issue and been fixed.. i know it's a small sample group to base that on though.. will keep checking it, and report back here if it does get locked on any servers again.

 

I've got seven servers with ISPConfig installed and I'm checking them daily, but so far I haven't seen any issues with the hanging apt-get processes.
Next weekend an updated kernel will be installed and the servers will be rebooted. Let's see if this issue happens again.

 

Today I discovered another hanging apt-get process. The following processes belong to the processtree of the hanging apt-get process:

Code:

root      914684    1196  0 15:00 ?        00:00:00 /usr/sbin/CRON -f -P
root      914688  914684  0 15:00 ?        00:00:00 /bin/sh -c /usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done
root      914693  914688  0 15:00 ?        00:00:00 /bin/bash /usr/local/ispconfig/server/cron.sh
root      914694  914688  0 15:00 ?        00:00:00 /bin/sh -c /usr/local/ispconfig/server/cron.sh 2>&1 | while read line; do echo `/bin/date` "$line" >> /var/log/ispconfig/cron.log; done
root      914708  914693  0 15:00 ?        00:00:00 /bin/php -q -d disable_classes= -d disable_functions= -d open_basedir= /usr/local/ispconfig/server/cron.php
root      914844  914708  0 15:00 ?        00:00:00 sh -c while fuser /var/lib/apt/lists/lock >/dev/null 2>&1 ; do sleep 2; done; apt-get update
root      914849  914844  0 15:00 ?        00:00:01 apt-get update
_apt      914859  914849  0 15:00 ?        00:00:00 /usr/lib/apt/methods/https
_apt      914860  914849  0 15:00 ?        00:00:00 /usr/lib/apt/methods/http
_apt      914973  914849  0 15:00 ?        00:00:00 /usr/lib/apt/methods/gpgv

I've attached a file with the output of

Code:

lsof -nP

of those processes.

In the logfiles I can't find anything special that could cause this, except that after the "apt-get"-process started, the logfile "ubuntu-advantage.log" has not been updated anymore.

I'll keep the process running till then next Friday in case someone wants some additional information.

 

I've got this on another server now too, so in total two servers have got those hanging apt-get processes. Attached I've got the output of "lsof -nP" of the pstree of those processes, starting with cron.
I noticed that on syslog I found the following lines:

Code:

Sep 29 07:00:01 mysql systemd[1]: Starting Update APT News...
Sep 29 07:00:01 mysql systemd[1]: Starting Update the local ESM caches...
Sep 29 07:00:02 mysql systemd[1]: apt-news.service: Deactivated successfully.
Sep 29 07:00:02 mysql systemd[1]: Finished Update APT News.
Sep 29 07:00:02 mysql systemd[1]: esm-cache.service: Deactivated successfully.
Sep 29 07:00:02 mysql systemd[1]: Finished Update the local ESM caches.

Apparently apt-get is also started by another process.