This is more of a "Help the next guy out" post.
So, I was thinking that my email was unusually quiet for the past couple of days, so I finally got around to checking things out, and discovered that I hadn't received any email for 3 days!!!. Did the usual Troubleshooting:
- Check Exchange Services
- Check Databases mounted
- Perform the Exchange Server Connectivity test. This test was kind of a Smoking Gun pointing at the Sophos. In the output, I was able to see my custom Headers and Server reply name in the results, so I know It's able to receive emails OK.
It all passed, so I went into the Mail section of my Sophos XG and found that for the past 3 days ALL inbound email was being bounced with the following error in the SMTP headers:
A message that you sent could not be delivered to one or more of its
recipients. This is a permanent error. The following address(es) failed:
SanatizedAddress@SanatizedDomain.com
all hosts for 'SanatizedDomain' have been failing for a long time (and retry time not reached)
So, I rebooted the Sophos, tried again, and got the same response. I spent about 2 hours troubleshooting, testing, and resorting to Google.... Well, it was actually Bing, but that doesn't matter. I came across a little article about MTA troubleshooting, and hidden down in one of the steps was the following:
Is Exim DB have cached the recipient server as non-routable?
Exim has an internal logic to retry & deliver the mails to the mail server. The number of retries gets reduced if the mail server is unreachable for a longer time and the mails are continuously received for that domain. Exim may try to send the emails once a day if the mail server is unreachable for more time. So, the admin can delete the Exim database by running the command below to deliver all those mails immediately when the mail server is available to receive the mails.
rm -rf /var/spool/output/db/
So, I figured, what the heck. I took a system backup of the XG, logged onto the physical console, and performed the command. Rebooted XG, MAIL FLOWING AGAIN!!
And while I was poking around the console, I found a setting for the LCD to scroll automatically. I kinda like that so it's not a static screen all the time, so I enabled that too. Why isn't that toggle anywhere in the GUI? That would have been handy. (set lcd auto-scroll on)
But, if anyone starts searching for this particular issue online, hopefully it helps you resolve your problem a lot quicker, or possibly point you in the right direction for whatever system is protecting your email.