Why the link can't be blocked?

I am very tired because UTM 9 can't block the link successfully.

Following is the link

https://scratch.mit.edu/explore/projects/all "

I tested follow Regular Expression :

https://scratch.mit.edu/explore/projects/[all] "

" https://scratch.mit.edu/explore/projects/[a-zA-Z0-9] "

When I use policy test, it shows blocked. However I try the user computers (total 10 different computers), it can be passed. 

Include the last "Lego" Game link, this is the second link that can't be blocked.

 

Please help!

  • Do you have access to support, or are you a home user?   You may have found a bug that Support needs to investigate.

    I have reproduced your symptoms.

  • In reply to DouglasFoster:

    I think I repeated your problem because I repeated your mistake.

    Escape all of the "/" characters -- replace them with "\/" -- and see if it fixes the problem.   I do not have time to repeat my tests right now.

  • In reply to DouglasFoster:

    DouglasFoster

    I think I repeated your problem because I repeated your mistake.

    Escape all of the "/" characters -- replace them with "\/" -- and see if it fixes the problem.   I do not have time to repeat my tests right now.

     

    Hi DouglasFoster,

    Thanks for reply!

      I test this Regular Expression:

    "^https?://scratch.mit.edu\/explore\/projects\/[A-Za-z0-9] "

    and 

    "^https?:\/\/scratch.mit.edu\/explore\/projects\/[A-Za-z0-9] " 

    Same as before, the policy test shows blocked. But the user's computer can be passed too.

    Also, I found a new problem. I open the live log and go to scratch.mit.edu by chrome. Strangely, the live log will not show anything about it but it will show other website status.

    Thanks a lot.

  • In reply to DouglasFoster:

     

    DouglasFoster

    Do you have access to support, or are you a home user?   You may have found a bug that Support needs to investigate.

    I have reproduced your symptoms.

     

    Hi DouglasFoster,

    I am a school IT staff in Hong Kong.

    thanks

  • In reply to Perry Pong:

    Hi Perry,

    Does the following work?

    ^https?://scratch.mit.edu/explore/projects/

    If not, show us the line from the Web Filtering log file where a URL was not blocked.

    Cheers - Bob

  • In reply to BAlfson:

    Hi Bob,

    Sorry, it still not work.

    Admin Console:

    User Computer :

    Log :

    2018:12:08-11:15:35 httpproxy[5763]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="CONNECT" srcip="10.77.192.90" dstip="151.101.2.133" user="" ad_domain="" statuscode="200" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="24217" request="0xdc862000" url="https://api.scratch.mit.edu/" referer="" error="" authtime="0" dnstime="2" cattime="140" avscantime="0" fullreqtime="61958215" device="0" auth="0" ua="" exceptions="" category="111" reputation="neutral" categoryname="Education/Reference"
    2018:12:08-11:15:35 httpproxy[5763]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="CONNECT" srcip="10.77.192.90" dstip="151.101.194.133" user="" ad_domain="" statuscode="200" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="3233490" request="0xe1069800" url="https://scratch.mit.edu/" referer="" error="" authtime="0" dnstime="2" cattime="96" avscantime="0" fullreqtime="62455812" device="0" auth="0" ua="" exceptions="" category="111" reputation="neutral" categoryname="Education/Reference"

     

    But the log can match "scratch" word , which has two lines only. Also, the live log window show above information around 30 second. Is this reasonable?

    Thanks Bob

  • In reply to Perry Pong:

    The lines you show from the log are not accesses you said you wanted to block.  The pictures are too small to see, but I suspect that the accesses are not being seen by Web Filtering or that you need to block other URLs than the one you've identified.

    Cheers - Bob

  • I should have seen this before.

    The site is https.

    I assume that you are not using decrypt-and-scan (https inspection).

    Without https inspection, the FQDN is in the unencrypted part of the packet, but the path and query string are encrypted.    It cannot match what it cannot see.

    You have to enable https inspection for your regex to work.     

    I used to be a big fan of https inspection, but less so now.   It solves some problems but it creates others.   I intend to post a comprehensive explanation of my views sometime soon.

  • In reply to DouglasFoster:

    DouglasFoster

    I should have seen this before.

    The site is https.

    I assume that you are not using decrypt-and-scan (https inspection).

    Without https inspection, the FQDN is in the unencrypted part of the packet, but the path and query string are encrypted.    It cannot match what it cannot see.

    You have to enable https inspection for your regex to work.     

    I used to be a big fan of https inspection, but less so now.   It solves some problems but it creates others.   I intend to post a comprehensive explanation of my views sometime soon.

     

     

    Hi DouglasFoster,

    Yes, you are correct. I haven't use the decrypt-and-scan. This function will affect the NOD32 and always prompt the warning windows. As before,I have post this in here too. T_T.

    Thanks!

  • In reply to BAlfson:

    BAlfson

    The lines you show from the log are not accesses you said you wanted to block.  The pictures are too small to see, but I suspect that the accesses are not being seen by Web Filtering or that you need to block other URLs than the one you've identified.

    Cheers - Bob

     

     

    Hi Bob,

    I re-insert the image.

    Proxy Test Tools:

    User's Computer :

     

    About the log, I don't know why that's like the LEGO URL. Even I open the live log window and go to the URL, it will not show the URL in the live log. I refresh the URL 3 Times and after around 30 second, live log show above two log data only. Does it wants to set anything?

    I have already rebooted the firewall, but the live log has the same status.

    Please Help!

  • In reply to Perry Pong:

    Without HTTPS inspection, UTM only logs one entry for the session, with method="CONNECT", and only at the end of the session.   The size="value" field of the log entry is the total of all traffic seen during the session.   Live Log is probably not helping because you are opening the page, which starts a session, and then expecting a log entry to appear.   You may be able to make the live log entry appear by navigating away from the page, to something unrelated.

  • In reply to DouglasFoster:

    Sharp eye, Doug!  I admit to scanning too often without looking closely.

    Perry, if you're not using decrypt and scan for HTTPS traffic, you won't be able to block traffic inside the encrypted tunnel, just as Doug commented twice above.

    Cheers - Bob

  • In reply to BAlfson:

    BAlfson

    Sharp eye, Doug!  I admit to scanning too often without looking closely.

    Perry, if you're not using decrypt and scan for HTTPS traffic, you won't be able to block traffic inside the encrypted tunnel, just as Doug commented twice above.

    Cheers - Bob

     

     
    Hi Bob and DouglasFoster
     
    Thanks you for your's big HELP!
     
    About the decrypt and scan function, does the CA download from Web Protection|Filtering Options|HTTPS CAs and install to all client computers before enabling the decrypt and scan function?
     
    Again , thank you for help!
  • In reply to Perry Pong:

    Yes, Perry, if you don't install the Proxy CA in all computers, they will get a certificate warning.

    Cheers - Bob

  • In reply to BAlfson:

    BAlfson

    Yes, Perry, if you don't install the Proxy CA in all computers, they will get a certificate warning.

    Cheers - Bob

     

     
    Hi all, 
     
    When enable the "decrypt and scan" function, it has a new problem as follows.  I already install the Cert download from "Web Protection|Filtering Options|HTTPS CAs".
     
    Web Protection Setting:
     
    ERROR:
    For Example: hk.yahoo.com
     
     
    If I am not switching back to "URL filtering only", the error will keep in here.
     
    Please help!!