HELP!! High Scan Times on messages

[jrmontg]

I am having very high scan times on my messages

Average scan time/msg (secs): 41.8507

How do I fix this?

[Donboy]

You can try some of these...

http://www.expresshosting.net/howto/spamassassin-performance-tuning.html

[w0ls0n]

WOW. I have never seen a scan time of 41 seconds. Thats just huge. I take it that you're getting thosands of messages a day or this is a really old box?

[Donboy]

When I have had this problem, it was usually related to NS lookups taking too long.

[lordofgore]

what hardware do you have?

[w0ls0n]

Hi,

I average about 3-5 seconds per message on one server

AMD Athlon(tm) 64 Processor 3400+
512MB RAM
200GIG IDE Hard Drive
OS is FreeBSD 6.3-PRERELEASE

I have another box I help admin and scan times are 3-5 seconds

Athlon 3500+ 64-bit CPU
1 GIG RAM
80 GIG IDE Hard Drive
OS is FreeBSD 5.4-STABLE

[Taz]

Hi,

I average about 3-5 seconds per message on one server

AMD Athlon(tm) 64 Processor 3400+
512MB RAM
200GIG IDE Hard Drive
OS is FreeBSD 6.3-PRERELEASE

I have another box I help admin and scan times are 3-5 seconds

Athlon 3500+ 64-bit CPU
1 GIG RAM
80 GIG IDE Hard Drive
OS is FreeBSD 5.4-STABLE

Care to say how many messages per 24hr period those servers are getting?

Reason I ask. I have a server averaging a scan time of 20-30 seconds per message in a 24hr period. This server is a dual CPU box, opertons, that is processing 25k+ emails per 24 hrs.

[w0ls0n]

You never answered my first question.

[Taz]

You never answered my first question.

What question was that?

I'm a thread highjacker here, not the originator of the thread, hehe :)

[w0ls0n]

I have never seen a scan time of 41 seconds. Thats just huge. I take it that you're getting thousands of messages a day or this is a really old box?

[Taz]

I have never seen a scan time of 41 seconds. Thats just huge. I take it that you're getting thousands of messages a day or this is a really old box?

I am averaging 25k-30k SMTP sessions in a 24hr period. The box is not really old. Dual Opteron 244 with 3GB of ECC Reg on Tyan server board. It also runs around 28 websites but the CPU time of the httpd is very minimal. Server has been in service for 1.25 years as of today.

[w0ls0n]

Ok there are some things you need to do. I can bet you money that over half if not 75% of the emails you get are spam. What you need to do is upgrade your box to a more stable version of qmail:

http://goodcleanemail.com/index.php?option=com_content&task=view&id=21&Itemid=1

Make sure you enable validrcptto

Then enable jgreylist

http://qmail.jms1.net/scripts/jgreylist.shtml

Now a warning; If you have your users sending mail via port 25, your best bet first BEFORE you enable validrcptto, jgreylist, etc is to get your users off sending on port 25. Have them start using smtp via ssl:

http://goodcleanemail.com/index.php?option=com_content&task=view&id=73&Itemid=1

Now the reason why is because when your users are sending via port 25, it is doing all the checks before sending: rbls, validrcptto, jgreylist, etc. What you can do eventually is disable smtp-auth on port 25 and start enabing all those types of stuff.

Hope that made sense.

[Taz]

Actually the percentage is considerably more than 75% - it's more like 98%. On average the amount of mail that is not spam is less than 1000 per 24 hours.

The validrcptto will be extremely useful since I get a tremendous amount of spam for users that do not exist.

Non of my users send mail via smtp. Webmail only. I run a small hosting and develoment business and don't offer roaming SMTP support.

I am going to look into the upgrade of the whole system later when I eventually have an email server that is seperate from the web server. For now I don't want to touch too much of a production system that is working.

Poking around, I don't think the Bayes filter is working. using sa-learn --dump majic gives me an error.

I install Spamassissin via ports and hence it is running as user root. I need to change that to user qscand and get the bayes filter working properly.

[w0ls0n]

Hi,

Well bayes will hlep mark spams but thats all. The only way your load times and scan times are going to go down is by enabling what I have discussed with you above. It would be a better idea to have a separate mail server and web server if load/scan times are getting high. Let me know if you need any help, I am available for hire.

[Taz]

Thanks for the offer. I now see I got a lot of underlying issues with spamassassin that I did not think I had.

I have found two "auto_whitelist" files. Both files seem to be updated. And some really cooky stuff going on with one of the whitelist files. ls -alh shows it being 8.1GB file but a du -h shows considerably less. This particular folder has bayes files within it, the other folder does not have bayes information. This stuff is in /var/spool/qmailscan/.spamassassin

The other whitelist files is in the root (/root/.spamassassin) folder - the user that spamd is running as. It has no bayes files.

[w0ls0n]

Hello,

Look at your local.cf and see if that gives you any clues and then look at

http://goodcleanemail.com/index.php?option=com_content&task=view&id=12&Itemid=1