Please, can you give some guidelines or basic steps to troubleshoot this errors? They are recurrent but not occurs every day. Thanks a lot.
Are those capacities for the disk where backups are written? Is it mounted from somewhere? How big are the backups? Do they fit in the remaining 69% or 52%? Try checking logs that have entries at about the time the backups run.
Depends on what Operating system you have installed on that host. Maybe you should start with https://forum.howtoforge.com/threads/please-read-before-posting.58408/
Thanks @Taleman . I guess you miss the full report of my system. Please have it, and sorry not to follow the guidelines. Code: ##### SERVER ##### IP-address (as per hostname): ***.***.***.*** [WARN] could not determine server's ip address by ifconfig [INFO] OS version is Debian GNU/Linux 12 (bookworm) [INFO] uptime: 07:31:57 up 14 days, 13:04, 1 user, load average: 2.01, 1.89, 2.07 [INFO] memory: total used free shared buff/cache available Mem: 3.7Gi 1.7Gi 358Mi 242Mi 2.2Gi 2.0Gi Swap: 0B 0B 0B [INFO] systemd failed services status: UNIT LOAD ACTIVE SUB DESCRIPTION ● clamav-daemon.service loaded failed failed Clam AntiVirus userspace daemon LOAD = Reflects whether the unit definition was properly loaded. ACTIVE = The high-level unit activation state, i.e. generalization of SUB. SUB = The low-level unit activation state, values depend on unit type. 1 loaded units listed. [INFO] ISPConfig is installed. ##### ISPCONFIG ##### ISPConfig version is 3.2.12p1 ##### VERSION CHECK ##### [INFO] php (cli) version is 8.2.22 [INFO] php-cgi (used for cgi php in default vhost!) is version 8.2.22 ##### PORT CHECK ##### [WARN] Port 8080 (ISPConfig) seems NOT to be listening [WARN] Port 143 (IMAP server) seems NOT to be listening [WARN] Port 993 (IMAP server SSL) seems NOT to be listening [WARN] Port 110 (POP3 server) seems NOT to be listening [WARN] Port 995 (POP3 server SSL) seems NOT to be listening [WARN] Port 465 (SMTP server SSL) seems NOT to be listening [WARN] Port 22 (SSH server) seems NOT to be listening ##### MAIL SERVER CHECK ##### [WARN] I found no "submission" entry in your postfix master.cf [INFO] this is not critical, but if you want to offer port 587 for smtp connections you have to enable this. [WARN] I found no "smtps" entry in your postfix master.cf [INFO] this is not critical, but if you want to offer SSL for smtp (not TLS) connections you have to enable this. ##### RUNNING SERVER PROCESSES ##### [INFO] I found the following web server(s): Unknown process (nginx:) (PID 2493) [INFO] I found the following mail server(s): Postfix (PID 1401) [WARN] I could not determine which pop3 server is running. [WARN] I could not determine which imap server is running. [INFO] I found the following ftp server(s): PureFTP (PID 1591385) ##### LISTENING PORTS ##### (only () Local (Address) [anywhere]:8081 (2493/nginx:) [localhost]:25 (1401/master) [localhost]:53 (1585907/named) [localhost]:53 (1585907/named) ***.***.***.***:53 (1585907/named) ***.***.***.***:53 (1585907/named) [localhost]:11211 (844/memcached) [anywhere]:3306 (1591303/mariadbd) [localhost]:953 (1585907/named) [localhost]:953 (1585907/named) [anywhere]:1022 (899/sshd:) [anywhere]:443 (2493/nginx:) [anywhere]:21 (1591385/pure-ftpd) [anywhere]:80 (2493/nginx:) *:*:*:*::*:8081 (2493/nginx:) *:*:*:*::*:53 (1585907/named) *:*:*:*::*:53 (1585907/named) *:*:*:*::*9400:ff:fea9:5:53 (1585907/named) *:*:*:*::*9400:ff:fea9:5:53 (1585907/named) *:*:*:*::*:3306 (1591303/mariadbd) [localhost]022 (899/sshd:) *:*:*:*::*:953 (1585907/named) *:*:*:*::*:953 (1585907/named) *:*:*:*::*:443 (2493/nginx:) *:*:*:*::*:53 (1585907/named) *:*:*:*::*:53 (1585907/named) *:*:*:*::*:25 (1401/master) *:*:*:*::*:21 (1591385/pure-ftpd) *:*:*:*::*:80 (2493/nginx:) ##### IPTABLES ##### Chain INPUT (policy ACCEPT) target prot opt source destination f2b-pure-ftpd 6 -- [anywhere]/0 [anywhere]/0 multiport dports 21 f2b-sshd 6 -- [anywhere]/0 [anywhere]/0 multiport dports 22 Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain f2b-pure-ftpd (1 references) target prot opt source destination RETURN 0 -- [anywhere]/0 [anywhere]/0 Chain f2b-sshd (1 references) target prot opt source destination RETURN 0 -- [anywhere]/0 [anywhere]/0 ##### LET'S ENCRYPT ##### acme.sh is installed in /root/.acme.sh/acme.sh
On ISPConfig Debian 12 systems log files are in /var/log -directory and subdirectories there. Examine log files that have lines at the time the backups run. Maybe backups exhaust memory? What shows command Code: dmesg -T | grep 'killed process' If that is the case, add swap to your system, 4GB or maybe 8GB.
I have check system log at the time of the backup fail, and found nothing. Nothing found. Also nothing found about killed processes. I use borg for backups. Is there a way of checking the backup log? Backups should log to somewhere, since there is an option about log retention on backup tab of server configuration. I have checked the last ISPConfig manual and found nothing about backup logs, the mentioned option of log retention still undocumented in the manual. Thanks.
borgbackup manual says it logs to standard out. So, how are you using borgbackup? I have until now assumed you backup by using the ISPConfig panel, Sites tab and there Backups.
Both are true, I am using borgbackup on the ISPConfig panel. I have configured backups tu use borg at server configuration level.
Thanks @till, I can try this, but if I have understood, it will launch a new backup, which maybe will fail, maybe will success. I can (and will) try it, but what I am trying to find is a log of the backup process to check an error that has already occurred. Can this be done? Is such log generated in any way?
I have not written the backup plugin, so I can't tell if it wites a log besides the normal ispconfig.log file.
Sorry, I am using ispconfig for couple years and didn't manage troubleshooting very well. As far I can see, ispconfig.log file is the same that system log in ISPConfig panel. The only thing I found there is a message saying that backup has failed. So, what I am doing is to raise the debug levels to debug, and wait some days for a new backup fail. I expect to see the reason then. Thanks a lot guys for your advise.
After raising the log level to debug, I still seeing the backup failed message, but no other messages about the reason of the fail . I am a bit stucked with this, cause it seems that there is no specific/more detailed log for backups. Maybe a bit of information about how backups are launched, to find and investigate the php file that handles backups could help me. In addition, where can I find information about the basics on how ISPConfig works internally? Thanks!
Here you can find information about the logging functions of Borgbackup: https://borgbackup.readthedocs.io/en/stable/usage/general.html#logging and here links to Borgbackup communities: https://www.borgbackup.org/support/free.html Personally, I think the Borgbackup option in ISPConfig needs some attention. Question is if the original community-maintainer would take over this job, or if anyone else is interested in working on it. Right now the situation with this feature is unsatisfying (imho) since it is also missing the possibility to backup/restore mails.
Thanks for the links, but they imply changes on borg command line, so I will need to dig into ISPConfig to find where where and how borg is invoked, and so check if the logs are being sent to anywhere or not :|. Any help on this? Thanks in advance!
You can find the borg backup code in this file: https://git.ispconfig.org/ispconfig/ispconfig3/-/blob/develop/server/lib/classes/backup.inc.php However, I don't use Borgbackup and thus I cannot give you a detailed step-by-step help on how to resolve the backup problems you are experiencing. If I would be you, I would expand/modify the backup.inc.php file to redirect the output to a log file or try to create a backup from the command line directly. It also shouldn't hurt to read the Borgbackup docs since it may contain information on how to resolve backup problems with Borg. In the worst-case, I would switch the backup mode to the traditional (tar archives) mode. It may be the least efficient and economic backup method, but it works at least.
