This discussion has been locked.

You can no longer post new replies to this discussion. If you have a question you can start a new discussion

POP3S scanning does not work sometimes - BUG

Hi,

POP3 scanning is not working anymore (at least 1 month now). I was able to filter the spam email (by only changing header because at the moment no more action is possible on IMAP/POP3 scanning. Vote http://feature.astaro.com/forums/330219-sophos-xg-firewall/suggestions/10614834-pop3-imap-more-scanning-option).

Since the last month and more, the email are not downloaded anymore. No logs inside XG and Outlook stacks on Downloading Emails. See screnshot.

If I connect without the XG, POP3 is working without problem.

Luk

This thread was automatically locked due to age.

0 IanMorehouse over 9 years ago

Hi Luk,
I have the opposite to you and I don't think your issue is a bug, I have managed to get my pop3s imap/s email policy working, but 90% of the mail is classed as spam by the XG. Sometimes the daily report shows the clean mail sender and receiver and other days no. Tomorrow I will review the received mail to see which one is clean?

Extra info.

I have incoming and outgoing policies. The incoming policy doesn't show any traffic and the outgoing policy traffic count appears to be too low. I have changed some of the rules behind the mail policy to see if that will help identify the potential spam.

The only mail scanned is the pops. Imaps is classified but not scanned.

Ian,

home UTM 9.x running in ESXi 6 e3-1275v2

AP55c and AP10 (courtesy Astaro)

Three other UTMs, SUM and SFM in hibernation

XG 15.x MR3 in hibernation
- 0 lferrara over 9 years ago in reply to IanMorehouse
  
  Thank you Ian.
  My imap scanning is working with no issue.
  Are you spam email get blocked or only headers are changed?
  My spam counters increase correcly.
  
  Happy Christmas.
  
  Luk
  - 0 IanMorehouse over 9 years ago in reply to lferrara
    
    Hi Luk,
    My spam counters increase. The mail is not blocked or changed. I put different comments into the various spam filters to see if the messages were being processed.
    I cannot get the mail proxy function working, the policy only works with the NAT (MASQ).
    
    Even the pop3s message taged as spam in the XG reports does not get its message header changed and is delivered.
    
    Since I made the changes earlier this morning, nothing is appearing in the logs, so nothing is being scanned which explains the isses in the earlier parts of this post.
    
    More updates. I found the cause of the failing scan, my interpretation of a selection tag - if spam, so how do you test for spam? I have found a bug in my opinion in that you cannot select any of the existing networks or elements, you have to create new ones which seems a litle silly.
    
    Merry Christmas and a Happy New Year.
    
    Ian,
    
    home UTM 9.x running in ESXi 6 e3-1275v2
    
    AP55c and AP10 (courtesy Astaro)
    
    Three other UTMs, SUM and SFM in hibernation
    
    XG 15.x MR3 in hibernation
0 IanMorehouse over 9 years ago

Hi Luk,
further fiddling. I rebuilt two of my email accounts on MBP and the Mac mini. On the MBP both are now imaps. The impas traffic is sort of scanned if you believe the log file because the headers are changed. I can't see any sign of that in the messages. Some messages have the start of the body changed, but again that is not consistent. The report shows mail trends of large numbers eg about 60 messages delivered and scanned, the mail application shows one pops message.
There appears to be an issue with the priority of data updates to the XG logging system, mine can take up to 3 minutes before and receipted mail message shows in the reports even when toggling between reports.

The pops traffic on one other account is scanned, that shows up in the reports.

The mail setup is very strange.
I will let this current mail configuration settle for a couple of days so that all the daily tasks and reports are updated.

Ian,

home UTM 9.x running in ESXi 6 e3-1275v2

AP55c and AP10 (courtesy Astaro)

Three other UTMs, SUM and SFM in hibernation

XG 15.x MR3 in hibernation
- 0 lferrara over 9 years ago in reply to IanMorehouse
  
  Thank you Ian.
  In my situation, the same policy scans IMAP and POP3. Imap is working while pop3 is not. Logs does not give further information. I created another policy rule to allow pop3 otherwise I cannot use the other mailbox.
  It is a bug, for sure!
  
  Luk
  - 0 IanMorehouse over 9 years ago in reply to lferrara
    
    Hi Luk,
    I have let my XG stabalise eg not fiddled for two reporting periods.
    My POP3/s is being scanned and subject line being altered.
    My IMAP/s accounts are being scanned with the first couple of line in the message being altered sometimes.
    I am seeing reports of senders and recipients int eh reports from both imap and pop3 accounts.
    There is a bug in the scanning process
    1/. according to the XG I receive many hundreds of imap email a day.
    2/. the log shows them being rewritten
    3/. the log also shows one message as being accepted.
    
    My theory, but you might not agree having more qualifications on this product. For imap scanning my impression is that the messages is passed from rule to rule and treated differently in each rule and at the end it is passed to the user, not necessarily rejected or accepted. Why I say this is because the log indicates one message (by time stamp) but many rewrites with the odd accept, whereas the pop3/s only has one entry in the log accept or rewrite.
    A couple more days of observation is required., I also think the logging process has some holes because I see messages reported in say yesterday's daily report that arrived the day before.
    
    Ian M
    
    Ian,
    
    home UTM 9.x running in ESXi 6 e3-1275v2
    
    AP55c and AP10 (courtesy Astaro)
    
    Three other UTMs, SUM and SFM in hibernation
    
    XG 15.x MR3 in hibernation
  - 0 IanMorehouse over 9 years ago in reply to IanMorehouse
    
    Hey Luk,
    are your pop3 users in a group (clientless or real)?
    
    Ian,
    
    home UTM 9.x running in ESXi 6 e3-1275v2
    
    AP55c and AP10 (courtesy Astaro)
    
    Three other UTMs, SUM and SFM in hibernation
    
    XG 15.x MR3 in hibernation
  - 0 lferrara over 9 years ago in reply to IanMorehouse
    
    Ian,
    Thank you for your answer.
    The BAP is not applied to any user but it is applied from LAN to WAN as IMAP (which works). I tried to change POP3 rule to match one user but same behaviour.
    
    Luk
  - 0 IanMorehouse over 9 years ago in reply to lferrara
    
    Hi Luk,
    I have number of issues with the scanning processes, so I am going to start a new thread with all the failed scan results.
    
    Ian,
    
    home UTM 9.x running in ESXi 6 e3-1275v2
    
    AP55c and AP10 (courtesy Astaro)
    
    Three other UTMs, SUM and SFM in hibernation
    
    XG 15.x MR3 in hibernation
0 lferrara over 9 years ago

After MR2, POP3s works again with no changes made on Pop3 server or XG rule.

They fix something!
- 0 IanMorehouse over 9 years ago in reply to lferrara
  
  No change on my XG. The mail function still has trouble parsing then passing messages with many attachments such as daily UTM reports. If I want messages with multiple attachments I need to use the utm connection.
  
  Ian,
  
  home UTM 9.x running in ESXi 6 e3-1275v2
  
  AP55c and AP10 (courtesy Astaro)
  
  Three other UTMs, SUM and SFM in hibernation
  
  XG 15.x MR3 in hibernation
  - 0 lferrara over 9 years ago in reply to IanMorehouse
    
    Ian,
    
    POP3s yesterday and today is not working. Only few mails are downloaded and then it stops downloading....I rejected the solution. The bug still exists.
  - 0 IanMorehouse over 9 years ago in reply to lferrara
    
    Hi Luk,
    
    I posted a story in this thread yesterday and it just vanished, something I did wrong at my end.
    
    A couple of observations from my experience with email on the XG as a home user with the mail servers being hosted by various ISPs.
    
    1/. You have to be patient with the XG mail processing system if you allow about 5 minutes or so after you receive the initial message header the remainder of the message appears.
    
    2/. if you are too impatient and try to force the mail you will break your mail client, done that too many times.
    
    3/. the issue about downloading messages means you have to keep the receiving laptop/PC alive until all the messages have been delivered.
    
    4/. the XG mail process scans each attachment and includes it as a spam message in the mail reports.
    
    I thought that my mail issues were caused by a slower CPU and system, was using a i3-3220T 2.8ghz, 8gb 1600 ram and an SSD. So I upgraded to a i3-6100 3.6ghz with faster memory 8ghz 2400 (pair of 4gb) and a SSD.. This did not make any difference what so ever.
    
    Now the XG advertises a fast path process for realtime data handling eg webbrowsing, media throughput and relegates things like mail scanning to the much slower full scanning path.
    
    To improve mail handling I suspect the XG processes will have to be tweaked so that more CPU cycles are provided to the various full path processes. This also implies that the system is single threaded which defeats the purpose of installing a multi-core CPU.
    
    Ian,
    
    home UTM 9.x running in ESXi 6 e3-1275v2
    
    AP55c and AP10 (courtesy Astaro)
    
    Three other UTMs, SUM and SFM in hibernation
    
    XG 15.x MR3 in hibernation