What's your routine for maintaining server security?

Discussion in 'Server Operation' started by schwim, Apr 17, 2023.

  1. schwim

    schwim Member HowtoForge Supporter

    And how much of that is automated?

    I'm running a Debian 11 server with ISPConfig, ISPProtect and the ban daemon. I'm not a host of any sort, I'm just a guy that needs a couple domains hosted and likes to give free web hosting to friends and family that need it. I'm currently down to just a single server so I'm not managing multiple machines and none of my people have complex hosting needs, just sites, databases and email basically.

    In a setup like mine, what would competent monitoring look like in your eyse? Is there a daily, weekly, etc routine that you would expect to perform to keep the machine reasonably safe and secure?

    I want to strike a balance between not living for the machine and keeping everyone's sites running without hiccup.

    Your thoughts would be most welcome. Thanks for your time!
     
  2. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    For web servers I guess running apt update && apt -y upgrade daily should suffice which can be set via unattended upgrades.
     
    schwim and till like this.
  3. till

    till Super Moderator Staff Member ISPConfig Developer

    - Install updates regularly with apt or dnf, depending on your Linux distribution.
    - Update ISPConfig, when updates get released.
    - Ensure that you have a daily server backup.

    And the most likely reason for a system getting hacked is not that the OS gets hacked, its more likely that one of your websites gets hacked or that the password of a mail account gets guessed (e.g. when the same password is used for other services online and one of these services got hacked). So keep your website CMS updated and do not use the same password for multiple services.
     
    schwim and ahrasis like this.
  4. Taleman

    Taleman Well-Known Member HowtoForge Supporter

    I suggest installing logwatch and pflogsum, then set them to e-mail the daily log summaries to you to read daily. Learn what is normal on your server, so you spot when unusual happens.
     
    schwim likes this.
  5. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    regarding updates / upgrades... you can do them manually. even have a regular schedule for it.. but like everything else.. life gets in the way.... you don't have time.. something more important to do.. internet's down.... forget...
    unattended upgrades is a good idea.... get's around this issue.. just like auto-update on wordpress / plugins...
    however. by default.. they run on their own schedules.. with no warning that something might be updated.. no auto-backup prior to updating.. and no idea what might be updated.. if anything breaks... it leads to confusion.. and time spent working out what was changed..

    i don't want to find out that something broke with no warnings.. or get woekn up by alerts in the middle of the night.. so my suggestion would be to make sure that any auto-updating is confined to a strict time/day/date schedule. yes, it means an important update might take a day or two longer to get installed... but if it went wrong.. i know exactly what sort of update is happening at that time, and where to look to find out what went wrong, i'd rather that than spend a weekend chasing down what went wrong when some unknown type of update tried updating some random bit of software at some random point in time.

    also, if running multiple servers... good time synchronisation and centralised logging... if something goes wrong between multiple servers, the time sync makes it easy to track the issue across them all in sequence.. and centralised logging means i only have to check in one place. eg the ELK stack or opensearch or similar..
     
    schwim likes this.
  6. schwim

    schwim Member HowtoForge Supporter

    Thank you all very much for the help guys, I really appreciate it!

    I use a program called FreeFileSync run with task scheduler on my local machine to connect to the site directories nightly and run an incremental backup. Is there a system that you prefer for backups, perhaps something baked into the panel?

    This is the latest howto I can find for using those applications. Is there any post or howto you're aware of that goes through the process of install and setup for what you're suggesting?

    That's why I've chosen not to automate upgrades to this point. I have broken the OS a few times in the past due to running the upgrade process and worry that I would miss the server going down until someone being hosted on the machine contacted me to let me know.
     
  7. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    I have monit to inform me via email if anything goes wrong with my servers. I also subscribe to some free yet reliable online services that check if my sites down.
     
    schwim likes this.
  8. schwim

    schwim Member HowtoForge Supporter

    Would you mind sharing an example config file for your monit install? I ask because I'm struggling to successfully set up the additional services(missing pids, etc) from the deb 10 howto: https://www.howtoforge.com/tutorial/server-monitoring-with-munin-and-monit-on-debian/
     
  9. ahrasis

    ahrasis Well-Known Member HowtoForge Supporter

    The tutorial already covers them all but mine is as follows:
    Code:
    set daemon 60
    set logfile /var/log/monit/monit.log
    set mailserver localhost
    set mail-format { from: [email protected] }
    set alert [email protected]
    set ssl options {
        version: auto
        verify: enable
        selfsigned: reject
    }
    set httpd port 8082 and
        use address 0.0.0.0
        SSL ENABLE
        PEMFILE /usr/local/ispconfig/interface/ssl/ispserver.pem
        allow username:'pa$$word'
    
    check process SSH with pidfile /var/run/sshd.pid
       start program "/usr/sbin/service ssh start"
       stop program "/usr/sbin/service ssh stop"
       if failed port 22 protocol ssh then restart
       if 5 restarts within 5 cycles then timeout
    
    check host OnlineStatus with address 1.1.1.1
       if failed icmp type echo count 3 with timeout 10 seconds
          then exec "/bin/bash /usr/share/router-reboot"
    
    check file IspSpeedTest path /usr/share/ispspeedtest.txt
       if content = "reboot"
          then exec "/bin/bash /usr/share/router-reboot"
    
    check file ispserver.pem path /usr/local/ispconfig/interface/ssl/ispserver.pem
       if failed permission 600 then exec "/bin/chmod 600 /usr/local/ispconfig/interface/ssl/ispserver.pem"
    
    # Ubuntu Jammy 22.04 - Change this according to OS version
    check directory DefaultPHP path /etc/php/8.1/fpm
       if changed timestamp then exec "/usr/bin/update-alternatives --set php /usr/bin/php8.1 && /usr/bin/update-alternatives --set php-fpm.sock /run/php/php8.1-fpm.sock"
    check directory DefaultPHPCGI path /etc/php/8.1/cgi
       if changed timestamp then exec "/usr/bin/update-alternatives --set php-cgi /usr/bin/php-cgi8.1"
    
    check process php5.6-fpm with pidfile /run/php/php5.6-fpm.pid
       start program = "/usr/sbin/service php5.6-fpm start"
       restart program = "/usr/sbin/service php5.6-fpm restart"
       stop program = "/usr/sbin/service php5.6-fpm stop"
       if failed unixsocket /var/run/php/php5.6-fpm.sock then restart
    
    check process php7.2-fpm with pidfile /run/php/php7.3-fpm.pid
       start program = "/usr/sbin/service php7.2-fpm start"
       restart program = "/usr/sbin/service php7.2-fpm restart"
       stop program = "/usr/sbin/service php7.2-fpm stop"
       if failed unixsocket /var/run/php/php7.2-fpm.sock then restart
    
    check process php7.3-fpm with pidfile /run/php/php7.3-fpm.pid
       start program = "/usr/sbin/service php7.3-fpm start"
       restart program = "/usr/sbin/service php7.3-fpm restart"
       stop program = "/usr/sbin/service php7.3-fpm stop"
       if failed unixsocket /var/run/php/php7.3-fpm.sock then restart
    
    check process php7.4-fpm with pidfile /run/php/php7.4-fpm.pid
       start program = "/usr/sbin/service php7.4-fpm start"
       restart program = "/usr/sbin/service php7.4-fpm restart"
       stop program = "/usr/sbin/service php7.4-fpm stop"
       if failed unixsocket /var/run/php/php7.4-fpm.sock then restart
    
    check process php8.0-fpm with pidfile /run/php/php8.0-fpm.pid
       start program = "/usr/sbin/service php8.0-fpm start"
       restart program = "/usr/sbin/service php8.0-fpm restart"
       stop program = "/usr/sbin/service php8.0-fpm stop"
       if failed unixsocket /var/run/php/php8.0-fpm.sock then restart
    
    check process php8.1-fpm with pidfile /run/php/php8.1-fpm.pid
       start program = "/usr/sbin/service php8.1-fpm start"
       restart program = "/usr/sbin/service php8.1-fpm restart"
       stop program = "/usr/sbin/service php8.1-fpm stop"
       if failed unixsocket /var/run/php/php8.1-fpm.sock then restart
    
    check process php8.2-fpm with pidfile /run/php/php8.2-fpm.pid
       start program = "/usr/sbin/service php8.2-fpm start"
       restart program = "/usr/sbin/service php8.2-fpm restart"
       stop program = "/usr/sbin/service php8.2-fpm stop"
       if failed unixsocket /var/run/php/php8.2-fpm.sock then restart
    
    check process Nginx with pidfile /var/run/nginx.pid
       start program = "/usr/sbin/service nginx start"
       stop program = "/usr/sbin/service nginx stop"
       if failed host 127.0.0.1 port 80 then restart
    
    check process MySQL with pidfile /var/run/mysqld/mysqld.pid
       group database
       start program = "/usr/sbin/service mysql start"
       stop program = "/usr/sbin/service mysql stop"
       if failed host 127.0.0.1 port 3306 then restart
       if 5 restarts within 5 cycles then timeout
    
    check process PureFTPD-MySQL with pidfile /var/run/pure-ftpd/pure-ftpd.pid
       start program = "/usr/sbin/service pure-ftpd-mysql start"
       stop program = "/usr/sbin/service pure-ftpd-mysql stop"
       if failed port 21 protocol ftp then restart
       if 5 restarts within 5 cycles then timeout
    
    check process postfix with pidfile /var/spool/postfix/pid/master.pid
       group mail
       start program = "/usr/sbin/service postfix start"
       stop program = "/usr/sbin/service postfix stop"
       if failed port 25 protocol smtp then restart
       if 5 restarts within 5 cycles then timeout
    
     
    Last edited: Apr 18, 2023
  10. schwim

    schwim Member HowtoForge Supporter

    Thanks very much for that. The tutorial gives the following: "#check process mysql with pidfile /var/lib/mysql/example.pid" when I should have been looking at /var/run/mysqld/mysqld.pid, as shown in your config file. That's where I was getting confused. Thanks for the clarification!
     
  11. schwim

    schwim Member HowtoForge Supporter

    Is this to resolve a broken SSL cert when monit runs? Is using this cert allow you to bypass the part of the tutorial that has you create the cert specifically for monit?
     
  12. nhybgtvfr

    nhybgtvfr Well-Known Member HowtoForge Supporter

    sort of.. the part which specifies the cert file for monit to use itself is in the 'set httpd' port section, starting with 'PEMFILE'

    the check file part is to check the file permissions for that pem file.. ispconfig doesn't care too much about it, it will run with 644.
    monit is fussier.. it will refuse to start/restart if the cert file doesn't have permissions 600..
    so the next line resets the permission on the file after every cert renewal
     
    ahrasis likes this.

Share This Page