Postfix stopped working

Discussion in 'Server Operation' started by Tomislav Aurednik, May 19, 2021.

  1. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Today coworker woke me up with critical problem : mails simply stopped working. No matter what I tried, I always got the same message : no space left on device. This is c/p from log file

    May 19 07:29:55 pro1 postgrey[1661]: FATAL: ERROR: can't create DB environment: No space left on device (dbdir: /var/lib/postgrey uid/gid: 117,117)
    May 19 07:29:56 pro1 postfix/tlsmgr[2287]: error: open database /var/lib/postfix/smtpd_scache.db: No space left on device
    May 19 07:29:56 pro1 postfix/tlsmgr[2287]: error: open database /var/lib/postfix/smtp_scache.db: No space left on device
    May 19 07:29:57 pro1 amavis[2296]: (!!)TROUBLE in pre_loop_hook: BDB can't create db env. at /var/lib/amavis/db: No space left on device, No space left on device. at (eval 92) line 312.

    Of course that was not a case, there is plenty of space on all volumes. Except one : /dev/mapper/vg--ssd-lv--root is 100% full and I have no idea what to do with it.
    No configuration was not changed for quite some time now. When I moved operations on another (mirror) server, everything works smoothly. But my concerns are that if I move everything back to the first server the problem will repeat.

    I already googled, but could not find any usable solution. Any idea would be appreciated.
     
  2. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    remove some files from /dev/mapper/vg--ssd-lv--root (maybe mounted to / so you can start with /var/log/ ? )
     
  3. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Filesystem Size Used Avail Use% Mounted on
    udev 7.9G 0 7.9G 0% /dev
    tmpfs 1.6G 9.1M 1.6G 1% /run
    /dev/mapper/vg--ssd-lv--root 19G 18G 0 100% /
    tmpfs 7.9G 39M 7.8G 1% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 7.9G 0 7.9G 0% /sys/fs/cgroup
    /dev/vdc1 20G 1.7G 17G 10% /var/log
    /dev/vda1 938M 105M 786M 12% /boot
    tmpfs 1.6G 0 1.6G 0% /run/user/1000

    log is separate mount, and ls of root reveals nothing. I am lost... but there's something strange. Although should be 1 GB free it shows 100% usage. Something seem to be locked.
     
  4. florian030

    florian030 ISPConfig Developer ISPConfig Developer

    the / partition is full. just check all folders on / (i.e. du -chs folder/*)
     
  5. Tomislav Aurednik

    Tomislav Aurednik Member HowtoForge Supporter

    Still have to find another 2-3 GB, but at least I managed to free something: /var/lib/fail2ban/fail3ban.sqlite was over 2 GB... Thanks for help.
     
  6. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    Code:
    Filesystem                        Size  Used  Avail  Use%   Mounted on
    /dev/mapper/vg--ssd-lv--root 19G   18G    0       100%    /
    
    The root partition is on the small side for an e-mail server. If you need to keep partition that small, monitor the disk usage.
     

Share This Page