Advisory: Reflexion - Delayed delivery of outbound and inbound email

Hi All,

My sincere apologies for any delays and inconvenience experienced.

Our team is actively working on resolving this incident and the following links will be updated accordingly when new information is available:

• Reflexion Status Page
• Advisory: Reflexion - Delays in the delivery of outbound and inbound email

I will report back with any other information that I receive.

Regards,

Also, in Reflexion inbound AND outbound are throwing errors:

This is an example of an outbound message in the reporting within RTC for a client of ours:

ReasonERROR: Message is not accepted by the Internet delivery queue

So, is mail really being queued on the outbound for delivery or is the error reporting wrong?

Or should we be rerouting our send connectors too?

We're working with our development team to provide an ETA. We will share this information ASAP.

We are seeing a significant amount of undeliverable messages coming back from mail servers attempting to send to reflexion. I am not at all confident that reflexion is even queuing anything at this point:

Error Type: SMTP
  Connection to recipients server failed.
  Error: Host name: 208.70.211.233, message: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respondTried 6 time(s)

SOPHOS - we need you to tell us more than refer to a vague status update. If email is being rejected or not accepted we need to know that so we can assess the impact and take action as needed.

7/30/2019
6:15pm EDT - Reflexion is currently experiencing the following:

• We are currently experiencing a delay in the processing of inbound and outbound email.
• Delivery of outbound email flow is processing but at a slower rate as the delivery queues are catching up and working to return to normal levels.
• We are still experiencing a delay in delivery of inbound email and outbound email.
• Engineering is actively working to resolve mail flow service as quickly as possible. We will provide an update as soon as we have additional information.

https://community.sophos.com/kb/en-us/134470

7/30/2019
6:15pm EDT - Reflexion is currently experiencing the following:

• We are currently experiencing a delay in the processing of inbound and outbound email.
• Delivery of outbound email flow is processing but at a slower rate as the delivery queues are catching up and working to return to normal levels.
• We are still experiencing a delay in delivery of inbound email and outbound email.
• Engineering is actively working to resolve mail flow service as quickly as possible. We will provide an update as soon as we have additional information.

https://community.sophos.com/kb/en-us/134470

We are noticing a 3 hour gap from about 10:30PST to 1:30PST in our inbound message log.  This isnt just emails not being delivered, this emails are straight up missing from the inbound logs entirely.  What happened to these emails?  Are they lost?  Also all of our Outbound messages are showing an error state saying ERROR: Message is not accepted by the Internet delivery queue.  Will these emails need to be resent by all of our users or will Reflexion process these once the delays have been resolved?

You do realize that saying the same thing over and over is not an "update," right?

At this point there are only two possibilities.  Either Sophos does not WANT to tell us what is going on, or Sophos does NOT KNOW what is going on.  Both are equally troubling.

Good Evening,

Any update? I see the system status has turned green for some reason. Maybe because I see like 4 emails got delivered! Now just cough up the thousands our customers are waiting on. 

We didn't even mention how this is effecting the MSPs that actually use your product. You realize we missed nearly all ticket submissions to us today? Who knows if we'll ever get em!

At this point, seems like a waste of energy to hope this gets resolved.

Here's the MOST sad part. We just put all our client on the central platform for AV. The plan was to move them over to that, IF THEY EVER GET THE MIGRATION PLATFORM FINISHED, like they promised. 

Why pull all your resources off a product so populated with dedicated customers? You should have used your resources to make a way off this platform before you left it to die. 

Sean

7/30/19

9:00PM EDT - Reflexion is still experiencing a delay in the processing of inbound and outbound email.

Delivery of outbound email flow is processing but at a slower rate as the delivery queues are catching up and working to return to normal levels.

We are still experiencing a delay in delivery of inbound email, and our infrastructure engineering team is actively working to resolve the issue and restore mail flow as quickly as possible.

We are working as fast as possible to resolve the issue. We sincerely apologize for the inconvenience this has caused and will provide an update as soon as we have additional information.

7/30/19

[Update]11PM EDT - All Reflexion services have been restored and all messages queued in Reflexion have been processed.

• The inbound and outbound emails that have been queued are now being received, processed, and delivered as quickly as possible.
• No email was lost, and all queued email should be delivered over the next few hours.
• Again, we sincerely apologize for the inconvenience this has caused and will provide a final update once the queues return to normal.

I am sorry but this is utter BS!  Emails were lost, every customer account has reported emails that bounced back, never were received, and some that came through with blank message bodies...  A lot of email was lost in this severe screw up on Sophos' end.  Completely unacceptable!  Then the communication from Sophos....  Even worse!

NEW inbound emails are NOT being queued in Reflexion.  I'm sending test email after test email  to my Reflexion customers and none of them show up anywhere queued inbound in Reflexion.  They are sitting my outgoing queue of O365 servers in a 'delayed' state.  I currently have all Reflexion routing turned off on my side so I can test and communicate.  I've sent over half a dozen emails to different customers and none of them receive the email nor are they visible in the queue waiting to be delivered.  It is NOT fixed.

Unfortunately, email was lost. I'm sure MSPs will lose customers. I'm even more sure Sophos will lose customers. 

Losing day all around 

7/31/19

[Update]01:00 EDT

• We are still experiencing a delay in the delivery of inbound email.
  
  
  • The inbound emails will be queued and processed and delivered as quickly as possible taking into account the delay.
    
    
  • No email has been lost, and all queued email should be delivered over the next few hours.
    
    
  • We sincerely apologize for any inconvenience this has caused and will provide an update once the queues return to normal.