Advisory: Reflexion - Delayed delivery of outbound and inbound email

Good morning, we spent hours yesterday with our own email servers because we were not notified via secondary systems email address that there was an issue at Reflexion. This took three (3) technicians out of handling critical client events and kept our Senior Engineer and me up until 11:00 PM monitoring the situation and determining if MX Records needed to be re-assigned.

If we continue both as a user and reseller of Reflexion will be determined by whether or not better notification of an issue resulting in the system being down, in any way, is advised via alternate notification. This could be emails to a gmail account (or other non focussed via Reflexion) or even SMS.

Thank you

Bruce G. Kreeger, President / CTO

Clarity Technologies Group, LLC

7/31/19

[Update]01:00 EDT

• We are still experiencing a delay in the delivery of inbound email.
  
  
  • The inbound emails will be queued and processed and delivered as quickly as possible taking into account the delay.
    
    
  • No email has been lost, and all queued email should be delivered over the next few hours.
    
    
  • We sincerely apologize for any inconvenience this has caused and will provide an update once the queues return to normal.

[Update] 7/31/2019 - 7:15AM EDT

As of 6:30 AM EDT, Reflexion services continue to process the backlog of email that was delayed as a result of the outage yesterday.

• No email was lost, and we expect all queued email to be delivered over the next few hours.

• Our infrastructure engineering teams continue to monitor and are actively working to resolve the issue and restore regular mail flow as quickly as possible.

• Reflexion Outbound mail is delivering without issue.

• Again, we sincerely apologize for the inconvenience this has caused and will provide an update once the queues return to normal or there is additional information to share.

Up at 5a to deal with this issue again here on west coast.  Already getting emails from my customers at 4:30a replying to emails I sent late last night.  Sent email tests to two customers at 5:18a and as of 5:54a they are not delivered nor are they in the queue waiting to be delivered.  I've moved 4 customers off of Reflexion last night bypassing via MX records.  If the delay is still 45-60+ minutes for inbound emails I will need to spend all morning updating MX records.  

Also, there is still no communication regarding the actual issue.  How is that possible?  Who is in charge of Reflexion and why is there such poor communication even as of this morning?

If your update was at 7:15a eastern and you expect the queues to be caught up within a few hours then we are approaching or have surpassed this timing.  What's the status of the queues?  How long is it taking for emails to be delivered?  How long are the delays?  This type of actionable information allows me to make decisions.  All of your updates so far are generic and making our job more difficult.  I can't expect my customers to wait an hour or who knows how long for emails which are normally delivered within seconds.  In their eyes they rightfully will believe the system is still not working.

Now 6:10a PST and multiple test emails to customers not delivered and not showing in queues.  Approaching one hour.  They are receiving some emails albeit one here and there and who knows when those were actually sent to them.

Engineering is sending more generic updates this morning.  Where is management with any communication?  What was the reason for Sophos purchasing Reflexion only to abandon it?  I was a huge fan of Sophos and Reflexion but now it looks like Reflexion has major infra issues and Sophos has shown that during a crisis they have no clue how to communicate.

I've been trying to get ahold of support at Reflexion all morning... No one or 1 person in queue, then it just dies and says we are unavailable. What are you doing if you are not available to help the clients who suffered at your hands yesterday? Today is not the day to let people sleep in, you still have work to do. 

For anyone looking for actionable information.  I sent test emails from O365 (with Reflexion filtering turned OFF in both directions) to Reflexion customers at 5:18a.  They are not even showing up in the queues as of 7a Pacific.  That's 1.75 hour delay at a minimum.  I'm in process of updating MX records to skip Reflexion as this is not an acceptable delay.  I've left the outbound connector alone for now as they appear to be timely.

This matches what we're seeing too.  Messages that happen to reach Reflexion are being delivered and outbound seems to be working.  But Reflexion isn't accepting most of the incoming mail that's queued up for it.  Trying to find a common denominator.

SophosSupport your system is NOT healthy like you claim in system status. NOT HEALTHY.

Backlogged messages from yesterday are slowly catching up, but there are still major delays with inbound and outbound messages. Please fix this ASAP.