Why do some emails get logged in mail manager and some don't?

If processed by the proxy, they should be in the Mail Manager SMTP Log Tab. Did you enable any of the display filters (checkboxes)? Are they older emails that have already been purged (see Email Protection > Mail Manager > Configuration)?

In a nutshell, the raw logs are just text files.  The Mail Manager GUI does not use these, it uses a backend database that the RAW log data is read into.

I haven't heard of this before. Can you paste the lines from the SMTP log that represent an email that was not in Mail Manager?
Cheers - Bob

Finally found another example (this one actually came to me). It seems to think it's a problem on the sender's end, but temporarily adding there sender to a whitelist let's the email in.

2016:01:25-10:11:23 kingarch exim-in[10293]: 2016-01-25 10:11:23 [67.231.148.255] F=<noreply.myaccountviewonline@lpl.com> R=<jcooper@kingarch.com> Verifying recipient address with callout
2016:01:25-10:11:24 kingarch exim-in[10293]: 2016-01-25 10:11:24 1aNinb-0002g1-2X ctasd reports 'Unknown' RefID:str=0001.0A020206.56A63B1C.00BE:SCGSTAT1808257,ss=1,re=-4.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=1024
2016:01:25-10:11:24 kingarch exim-in[10293]: 2016-01-25 10:11:24 1aNinb-0002g1-2X Greylisting: Greylisted 67.231.148.255
2016:01:25-10:11:24 kingarch exim-in[10293]: [1\33] 2016-01-25 10:11:24 1aNinb-0002g1-2X H=mx0a-0017a501.pphosted.com [67.231.148.255]:1567 F=<noreply.myaccountviewonline@lpl.com> temporarily rejected after DATA: Temporary local problem, please try again!

I'm happy to send you more of the log, but I'd rather not post it all in a forum. Can I somehow send you the log snippet directly?

Thanks,

Jeff

This appears to be the greylisting process at work, which will be reflected in the raw log, but not in the Mail Manager Log. Do you understand what greylisting is and how it works? Emails from an unknown address will be temporarily rejected by the UTM. A properly configured and valid sending MTA (email server) will then retry after a certain interval (set on the sending server) and the UTM will accept the retry, adding the address to its' greylisting good list. I'm thinking that you may not be leaving enough time for the sending email server to retry (generally can be anywhere from seconds to 15 minutes).

If you find there are some valid senders whose MTAs never retry, you just need to create an exception rule to skip greylisting for those sender addresses/domains.

Found another possible culprit, or perhaps a contributing factor. For years now, I was under the impression I needed DNAT on for SMTP along with the email filtering to it could pass scanned incoming emails to our internal server after scanning. I just found a thread relating to something else (community.sophos.com/.../73422) which indicated this was not necessary and may in fact cause problems.

I wonder if this confused the utm by making it unsure if it needed to scan a message or just send it to the server to deal with.

I turned it off. Emails still flowing both directions, so we'll see what happens.

Thanks for your help with the greylisting.

Jeff

Hello Scott,

How can you add a greylisting rule skip?

Regards,

DeltaSM

With an Exception.

Cheers - Bob