Spamassassin 100% CPU

Hi,

after a server crash I decided to install everyting from roots, so I installed a ISPConfig on a brand new Ubuntu Hardy server, following the "perfect server" setup.

Short after creating some new users, I lost the control of the server: 100% CPU, 100% memory, 100% swap. The most of the problem was due to the process "spamassassin" which was eating all the resources.

After rebooting the server, gained access to it and quickly disabled Postfix. Doing that, the load stepped down from the former 20 to 1,7 and I could then configure ISPConfig not to use SpamAssassin.

Well, not pleasant. I prefere to have all the emails, for all the hosted domains, filtered.

So, what can I do? And, how can I do to have filtering back at a fair load?

Thanks.

 

Take a look here: http://www.howtoforge.com/forums/showthread.php?t=21911&highlight=spamd+ispconfig

 

No way,

I did

apt-get install spamassassin

changed /root/ispconfig/isp/conf/spamassassin.rc.master

:0fw
* < 256000
| /usr/bin/spamc
/root/ispconfig/isp/conf/spamassassin.rc.master

Then

touch /home/admispconfig/ispconfig/.run


But...

syslog:May 18 12:12:29 homeland spamd[23012]: Use of uninitialized value in numeric ge (>=) at /usr/lib/perl/5.8/DB_File.pm line 276.
syslog:May 18 12:12:29 homeland spamd[23012]: Use of uninitialized value in numeric gt (>) at /usr/lib/perl/5.8/DB_File.pm line 280.
syslog:May 18 12:12:29 homeland spamd[23012]: Deep recursion on subroutine "DB_File::AUTOLOAD" at /usr/lib/perl/5.8/DB_File.pm line 235.
root@homeland:/var/log# ls /root/isp

I'm using an ubuntu server hardy

 

Look here what i found on a spam email header:

X-Virus-Status: Failed
X-Virus-Report: /home/admispconfig/ispconfig/tools/clamav/bin/clamdscan error 2
X-Virus-Checker-Version: clamassassin 1.2.4 with clamdscan / WARNING: Can't parse the configuration file..


What is wrong? Mail, obviously, don't get tagged as spam

 

Please copy /root/ispconfig/isp/conf/spamassassin.rc.master to /root/ispconfig/isp/conf/customized_templates/spamassassin.rc.master. Afterwards, you must update all your users in ISPConfig so that the new template will be used to rewrite each user's spamassassin.rc file.

 

I did as you told, but the email stopped to fall in the user mailbox.

So, I tried to delete two users in that domain and recreate them:

I can't. It says the already exists, even if the do not appear in the user manager.

I did a grep
on

/etc/passwd, /home/admispconfig/ispconfig/users, /etc/postfix/virtusertabl

and they only appear in /etc/passwd.

Then I checked isp_isp_user table and I found them there, there is also their directory unders "users".

The recycle bin keeps telling me

"The system is currently updating the configuration files."

I modified some other users, rebooted ispconfig, rebooted the servers, but nothing. Any ideas?

Thanks

 

Well,

problem solved. After a while ISPConfig wrote the new configuration.

I enabled spamassassin and then...well...still 100% of resources eaten, even using clamscand, following the instructions found following the link you pointed me.

The processor of the server is quite fair:

rocessor : 0
vendor_id : AuthenticAMD
cpu family : 15
model : 55
model name : AMD Athlon(tm) 64 Processor 3700+
stepping : 2
cpu MHz : 2199.743
cache size : 1024 KB

and 2 GB of ram.

Any suggestions?

Thanks

 

Is it really ClamAV and SpamAssassin that are eating up your resources? You can run

Code:

top

to find out.

 

Mem: 2061840k total, 2047520k used, 14320k free, 152k buffers
Swap: 2104504k total, 2104504k used, 0k free, 3620k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10144 root 20 0 4000m 1.9g 360 R 9.1 94.8 0:19.81 spamd


Well, it seems it's spamd. It is started as I launch /etc/init.d/spamassassin.

It goes even higher with cpu utilization, but it's difficult to snapshot cause the server freezes after a while.

 

Are there any errors related to spamassassin in your logs?
What's the output of

Code:

spamassassin --lint

?

 

spamassassin --lint gives no output

I have grepped /var/log

egrep -R -i spamassassin /var/log/*

but I found only messages from dpkg from the install procedure, like

/var/log/dpkg.log:2008-05-18 12:10:16 status installed spamassassin 3.2.4-1ubuntu1

 

Can you try

Code:

:0fw
* < 256000
| /usr/bin/spam[B][COLOR="Red"]d[/COLOR][/B]

instead?

 

I did it, but the problem didn't solved.

spamd is consuming between 80% to 100%, filling up all the swap. The cpu usage is not so high, the load about 9 but the server become unusable.

Any ideas?

 

You could try to update the DB_File module using the Perl shell:

Code:

perl -MCPAN -e shell
install DB_File

 

No way.

I upgraded the perl module and setup maxchildren=1, nice=15 for spamd.

Now it crunched between 75%-90% of memory and 10% on CPU. After about one hour the server killed spamd due to lack of resources.

Anyway, during the working time, it seemd not to tag any spam.

 

I also noticed this in the mail headers:


X-Virus-Status: Failed
X-Virus-Report: /home/admispconfig/ispconfig/tools/clamav/bin/clamdscan error 2
X-Virus-Checker-Version: clamassassin 1.2.4 with clamdscan / WARNING: Can't parse the configuration file.

 

What's the output of

Code:

/home/admispconfig/ispconfig/tools/clamav/bin/clamscan -V

?

What's in /home/admispconfig/ispconfig/tools/clamav/etc/clamd.conf?

 

/home/admispconfig/ispconfig/tools/clamav/bin/clamscan -V
ClamAV 0.93/7319/Mon Jun 2 02:37:30 2008

cat /home/admispconfig/ispconfig/tools/clamav/etc/clamd.conf
##
## Example config file for the Clam AV daemon
## Please read the clamav.conf(5) manual before editing this file.
##


# Comment or remove the line below.
#Example

# Uncomment this option to enable logging.
# LogFile must be writable for the user running the daemon.
# Full path is required.
#LogFile /tmp/clamd.log

# By default the log file is locked for writing - the lock protects against
# running clamd multiple times (if want to run another clamd, please
# copy the configuration file, change the LogFile variable, and run
# the daemon with --config-file option). That's why you shouldn't uncomment
# this option.
#LogFileUnlock

# Maximal size of the log file. Default is 1 Mb.
# Value of 0 disables the limit.
# You may use 'M' or 'm' for megabytes (1M = 1m = 1048576 bytes)
# and 'K' or 'k' for kilobytes (1K = 1k = 1024 bytes). To specify the size
# in bytes just don't use modifiers.
#LogFileMaxSize 2M

# Log time with an each message.
#LogTime

# Use system logger (can work together with LogFile).
#LogSyslog

# Enable verbose logging.
#LogVerbose

# This option allows you to save the process identifier of the listening
# daemon (main thread).
#PidFile /var/run/clamd.pid

# Path to a directory containing .db files.
# Default is the hardcoded directory (mostly /usr/local/share/clamav,
# it depends on installation options).
#DataDirectory /var/lib/clamav

# The daemon works in local or network mode. Currently the local mode is
# recommended for security reasons.

# Path to the local socket. The daemon doesn't change the mode of the
# created file (portability reasons). You may want to create it in a directory
# which is only accessible for a user running daemon.
LocalSocket /home/admispconfig/ispconfig/temp/clamd

# Remove stale socket after unclean shutdown.
#FixStaleSocket

# TCP port address.
#TCPSocket 3310

# TCP address.
# By default we bind to INADDR_ANY, probably not wise.
# Enable the following to provide some degree of protection
# from the outside world.
#TCPAddr 127.0.0.1

# Maximum length the queue of pending connections may grow to.
# Default is 15.
#MaxConnectionQueueLength 30

# When activated, input stream (see STREAM command) will be saved to disk before
# scanning - this allows scanning within archives.
#StreamSaveToDisk

# Close the connection if this limit is exceeded.
#StreamMaxLength 10M

# Maximal number of a threads running at the same time.
# Default is 5, and it should be sufficient for a typical workstation.
# You may need to increase threads number for a server machine.
#MaxThreads 10

# Thread (scanner - single task) will be stopped after this time (seconds).
# Default is 180. Value of 0 disables the timeout. SECURITY HINT: Increase the
# timeout instead of disabling it.
#ThreadTimeout 500

# Maximal depth the directories are scanned at.
MaxDirectoryRecursion 15

# Follow a directory symlinks.
# SECURITY HINT: You should have enabled directory recursion limit to
# avoid potential problems.
#FollowDirectorySymlinks

# Follow regular file symlinks.
#FollowFileSymlinks

# Do internal checks (eg. check the integrity of the database structures)
# By default clamd checks itself every 3600 seconds (1 hour).
#SelfCheck 600

# Execute a command when virus is found. In the command string %v and %f will
# be replaced by the virus name and the infected file name respectively.
#
# SECURITY WARNING: Make sure the virus event command cannot be exploited,
# eg. by using some special file name when %f is used.
# Always use a full path to the command.
# Never delete/move files with this directive !
#VirusEvent /usr/local/bin/send_sms 123456789 "VIRUS ALERT: %f: %v"

# Run as selected user (clamd must be started by root).
# By default it doesn't drop privileges.
User admispconfig

# Initialize the supplementary group access (for all groups in /etc/group
# user is added in. clamd must be started by root).
#AllowSupplementaryGroups

# Don't fork into background. Useful in debugging.
#Foreground

# Enable debug messages in libclamav.
#Debug

##
## Mail support
##

# Uncomment this option if you are planning to scan mail files.
ScanMail 1

##
## Archive support
##


# Comment this line to disable scanning of the archives.
ScanArchive 1


# By default the built-in RAR unpacker is disabled by default because the code
# terribly leaks, however it's probably a good idea to enable it.
#ScanRAR


# Options below protect your system against Denial of Service attacks
# with archive bombs.

# Files in archives larger than this limit won't be scanned.
# Value of 0 disables the limit.
# WARNING: Due to the unrarlib implementation, whole files (one by one) in RAR
# archives are decompressed to the memory. That's why never disable
# this limit (but you may increase it of course!)
# ArchiveMaxFileSize 10M
MaxScanSize 10M

# Archives are scanned recursively - e.g. if Zip archive contains RAR file,
# the RAR file will be decompressed, too (but only if recursion limit is set
# at least to 1). With this option you may set the recursion level.
# Value of 0 disables the limit.
ArchiveMaxRecursion 5

# Number of files to be scanned within archive.
# Value of 0 disables the limit.
ArchiveMaxFiles 1000

# Use slower decompression algorithm which uses less memory. This option
# affects bzip2 decompressor only.
#ArchiveLimitMemoryUsage

##
## Clamuko settings
## WARNING: This is experimental software. It is very likely it will hang
## up your system !!!
##

# Enable Clamuko. Dazuko (/dev/dazuko) must be configured and running.
#ClamukoScanOnLine

# Set access mask for Clamuko.
ClamukoScanOnOpen 1
ClamukoScanOnClose 1
ClamukoScanOnExec 1

# Set the include paths (all files in them will be scanned). You can have
# multiple ClamukoIncludePath options, but each directory must be added
# in a seperate option. All subdirectories are scanned, too.
ClamukoIncludePath /home
#ClamukoIncludePath /students

# Set the exclude paths. All subdirectories are also excluded.
#ClamukoExcludePath /home/guru

# Limit the file size to be scanned (probably you don't want to scan your movie
# files )
# Value of 0 disables the limit. 1 Mb should be fine.
ClamukoMaxFileSize 1M

# Enable archive support. It uses the limits from clamd section.
# (This option doesn't depend on ScanArchive, you can have archive support
# in clamd disabled).
# ClamukoScanArchive

 

Hm, looks ok...

 

Well, no idea? It seems that for the period spamd is running, it doesn't filter any emails, so I receive spam anyway.

Can we recap some steps? What can we do more? Before upgrading OS and ISPConfig, spamassassin was working nicely.