Why do I want Greylisting? Why does the UTM Greylist everything not rejected?

Greylisting is a great way to reduce spam -- ...

After a successful delivery, the result is kept in what you have called a whitelist for a week (I think) after the last successful delivery.

...Greylisting's retained memory of previous senders had been enhanced to a year...

Maintainer: Micha Lenk 

GREYLIST_RETRY_HOST_CLEANUP = ${lookup pgsql{DELETE FROM greylist_retry_hosts \

                              WHERE (($tod_epoch - stamp) > (60*60*24*30))}{1}{1}}

Bruce, a new thread in the German Forum got me to thinking about this conversation over four years ago, so I did some more research and testing.  They all speak English and it's easier for me to respond in English.  I posted the following there:

I haven't recommend greylisting for years, but others whom I respect do, so I'm experimenting with it again.  I saw a 2018 study where roughly 2/3 of greylisted emails were finally delivered, with 1/3 not retried - one assumes those were from spammers.  One of my clients that uses greylisting saw only 56% retried successfully so far in June.

I had thought that the SMTP Proxy used only the triad of sending IP, sender and recipient, but I realize now that it also uses the subject.  This means that greylisting occurs after DATA, so that's after rejections for RBL, rDNS/HELO, local Blacklists, Recipient verification and SPF.  I also see ctasd reports 'unknown' in the line above the greylisted message, so we know that the temporary rejection occurs after the anti-spam tests that would result in rejection have been passed.  The advantage is that malware scans, which are expensive, are skipped unless the message is resent and accepted.

There are situations where Exceptions for greylisting should be made such as addresses to which orders are sent where there's a cut-off time.  Also, mailing services like Constant Contact will use a different IP virtually every time a greylisted email is resent. 

I would appreciate your comments/corrections here or on the German thread.

Cheers - Bob

On implementation:   

You have identified the control methods.  You can turn it on or off globally or by target domain.  You can also turn it off with an exception.   In this respect, UTM will do whatever you want it to do.

On the complications:

The world's legitimate mail is coming from fewer and fewer sources:  Gmail, Outlook.com, Proofpoint, Mimecast, Cisco, etc.    These organizations are so large that you cannot keep track of the IP addresses that they are using, and it is possible that the IP address will change on every delivery attempt.    Users in this forum complained that they were seeing 24-hour delays from Outlook.com with graylisting enabled.   Other posts have indicated that Sophos maintains an internal exception list for these big organizations, and that they updated their list to correct the Outlook.com problem.   The best solution would be to have exceptions defined using SPF syntax, so you can say "bypass graylisting for anything coming from Outlook.com servers".   (Not because they are immune from bad behavior by their clients, but they are not going to be scared away by graylisting so it is useless as a defense for messages coming from them.)

On my UTM configuration, I have never tried to use it.   On another environment with weak spam filtering, I have turned off graylisting and seen no significant change in spam levels.  So I am a skeptic.

On data analysis:

The SMTP logs are rather difficult to parse into coherent data.   I have just completed a redesign of my log parsing tools while chasing the antispam check failure errors that are mentioned in a new post that I started this morning.  It should be possible to collect data about messages that are not retried when graylisting is off, but it will require significant effort.   Someone can send me a PM if you want my code (which uses a SQL database).