[NC-52120] Daily report - Reports are received every day at different time - bug

As you were told earlier, the reports will start at the time you wish, with the criteria you wish. The fact that the reports arrive at different times will be a factor of how busy the unit is at the time it is preparing the report.

Control plane should never have priority over data plane.

Stuart

Stuart,

a suggestion:

since the daily report correspond to the previous day, XG can create the report at midnight when no one is connected of before the scheduled time to send the report and at the schedule time send the report. This was one of the things that many customers experienced and it is the "business card" of XG for trial customers too.

Many times, why the daily report arrives at different time each day?

You is how it works today but it is not a nice way to present Sophos to new customers.

Regards

12 hours to produce a report?

If the delivery time is 9:00 but XG sends when it can, it is unprofessional! In the daily report settings, put that the report will be sent starting from this hours. Some delays can occur due to the XG resource usage or something similar.

I am a consultant now, but when I was working as a System Admin, I read daily reports always at the same time.

Luk, can you send some screenshots of the Daily Report configuration please.

 I made an assumption regarding the report being produced at midnight, given it was a daily report. I certainly havent seen the issue of delayed reporting that you are talking of here.

Stuart

Also, the email should be sent at 11 and not sometime in 12:xx range

Thanks, and these reports are being mailed to local recipients or through intermediate MTA's?

External SMTP server.

Luk, I have kicked off an investigation, but going back to your original screenshot it shows reports being delivered to a mailbox at 1-2 hrs after the 'send at' time. Could it be external delivery delays that are also additive??

I do not think so, as failed logins, appliance restart, backup configuration are not delayed at all.

I tried with local XG smtp settings. Same thing!

Hi Luk,

Circling back to this issue.

This is an example of terminological inexactitude :) In other words "Send e-Mail at" doesn't mean what you (we) think it means.

The report scheduling system looks for reports being scheduled every hour and if it finds a report has been scheduled it starts to produce that report. The report can take variable amounts of time depending upon amount of data, system load, other resource constraints like CPU and memory.

So "Send e-Mail at" actually means "Begin producing report at"

I am going to work with engineering to understand how we can best resolve this UI language issue - I doubt we will resolve this before GA as I don't think its as easy as simply changing the words. Unless you think a change of wording and documentation would suffice?

I know this has been something you've reported for a while now, so thank you for your patience, we will resolve the confusion. However, as it stands the reports will still be received at variable times depending on the circumstances above, so my best advice at this time is if you want to see a report at a specific time, schedule it 2 hours earlier than you need it.

Unknown said:

I am going to work with engineering to understand how we can best resolve this UI language issue - I doubt we will resolve this before GA as I don't think its as easy as simply changing the words. Unless you think a change of wording and documentation would suffice?

I know this has been something you've reported for a while now, so thank you for your patience, we will resolve the confusion. However, as it stands the reports will still be received at variable times depending on the circumstances above, so my best advice at this time is if you want to see a report at a specific time, schedule it 2 hours earlier than you need it.

PMStuart this is an example of XG bad design. I think that you should investigate to re-engineer the process and make sure we receive the report at scheduled time and changing the work is not a nice workaround. Time is time. So, all the time, we need to explain to customers: "yes, it is by design. As you can read, the report starts to be calculated at that time and you will receive the report each day at different time." This is already being explained to them.

Now: think about how to fix the issue properly (no workaround) but as I showed, the report is from the day before, so you can:

• really generate the report at the time of the scheduled time and send it within 1/3 minutes OR
• generate the report at midnight and save it somewhere in the /var/log or /var/reports and send it at the scheduled time

Up to you. In my case, I am receiving reports even after more than 1 hour, there are more than something that you need to re-engineer as I have 3/5 IPs at home and few rules. If extracting logs from the DB requires so much time...I am sure you need to go back to design phase and understand how to built a new reporting module. This is not a bad idea, as logging and reporting (at the moment) are the worst topic on XG.

With UTM 9 SG650 with 4000 users, daily report arrives within 3 minutes. Just to let you know some differences.

Thanks

I have to agree with Luk. There is something seriously wrong with the database if it can't generate simple reports within a few minutes. A correctly sized system shouldn't be so overwhelmed that it can't generate a report promptly. If there is a cron job that starts a report every hour on the scheduled time, maybe you guys can change the cron timing and start the report 15-20 minutes before the gui calls for it for daily reports and then queue it for mail at the scheduled time. Just throwing ideas here since a KB article saying a 9am report doesn't mean 9am, it could be 9 or 10 or whatever depending on system load is not something an end user wants to hear.

Thanks for your efforts on this by the way.

I have to agree with Luk. There is something seriously wrong with the database if it can't generate simple reports within a few minutes. A correctly sized system shouldn't be so overwhelmed that it can't generate a report promptly. If there is a cron job that starts a report every hour on the scheduled time, maybe you guys can change the cron timing and start the report 15-20 minutes before the gui calls for it for daily reports and then queue it for mail at the scheduled time. Just throwing ideas here since a KB article saying a 9am report doesn't mean 9am, it could be 9 or 10 or whatever depending on system load is not something an end user wants to hear.

Thanks for your efforts on this by the way.

To be honest, I prefer that they fix the problem at source. As you said, if XG cannot generate reports in few minutes, something must be revisited or built from scratch.

I was going to post about central reporting but alas; they just need to fix it on box.

Something so simple yet nobody seems to see the point of a scheduled report?  If it's this big of an issue, maybe rip out on-box reports all together and use that space/horsepower for log archives, verbose logging, etc.  Force everyone to central for advanced metrics and reporting.  I know this isn't a solution...just my frustrated opinion.