This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Regarding website filtering

 could you pls explain how to block uncategories wesites for  particular users



This thread was automatically locked due to age.
Parents
  • Hi, Kiran, and welcome to the UTM Community!

    There is a trick with "Uncategorized" as there is a Category named "Uncategorized" and there are sites that are "Uncategorized" because no Category has been determined for the site.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • I have to quibble Bob.  I just rechecked and I have only one type of Uncategorized in the administrative interface for web proxy

    Perhaps you were remembering that there are options for both "Uncategorized" and "Categorization Failure".  The latter would mean that the Categorization database is not reachable.

  • Assuming that you're not looking at Sophos at home because I haven't seen that but there should be a catergory selector for "Uncategorized websites" at the bottom of the Categories tab in your filter action.  This is where you set whether to allow or block uncategorized sites.

    However I'd be wary of doing that big bang so to speak.  You'll block a lot of legitimate sites.  Just run a few reports on activity logged against the Uncategorized category and you'll see just how many sites are still in that default category.

  • Again, guys, there is a sub-category "Uncategorized" that you can select when you define a new Category on the 'Categories' tab in 'Web Filtering Options'.  This is an assigned category, not the lack of any category assignment.

    At the bottom of the Categories section of a Filter Action, you can select to Allow 'Uncategorized websites'.  These are sites that have had no category assigned.  They haven't even been assigned 'Uncategorized'.  Perhaps this is what you mean as "categorization failure," Doug?

    At least, that's how I've understood and used these concepts - I'd be happy to have my knife sharpened though!

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • cheers Bob

    been that long since we built our category list that I'd forgotten about that   :)

  • Well Bob, that is probably a bug and it probably has existed since...  at least 9.0 and maybe even before that.  :)

    Although I cannot be sure without testing, I believe that if you create a Category Group containing "uncategorized" it will do nothing.

  • I don't think it's a bug, Michael, unless Sophos doesn't have an "Uncategorized" category in it's SXL database like CFF does.  I guess that's what you're telling me - my understanding no longer applies to the UTM as CFF is no longer used?

    Hmmmm, in my lab, I still get 0 when I do:

    cc get http use_sxl_urid

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • AFAIK both CFF and SXL have the same thing.  A single "response from the server" that is a hexadecimal number (0xFF) that maps to the the word "Uncategorized".  AFAIK, the underlying database does not have an "explicitly defined" Uncategorized that is seperate from the "I have no information" Uncategorized.  Or if it does, I don't think that gets passed back to the client in either CFF or SXL.  Regardless the httpproxy only has a single Uncategorized concept.  The fact that there are two ways to configure it in the UI is a problem and I don't know what the system does if there is a conflict.  I suspect the one in the Category Group is ignored.

    Of course, I could be wrong.

    cc get http use_sxl_urid
    cc get http sc_local_db

    If use_sxl_urid is 1 then it will use SXL
    If use_sxl_urid is 0 AND sc_local_db is none then it will use CFFS
    If use_sxl_urid is 0 AND sc_local_db is mem|disk then it will use CFF with a local database


    Fun Fact:
    We are currently updating the TrustedSource SDK that the SXL servers use to the latest version.
    In an upcoming UTM release we will be updating the TrustedSource SDK that the local CFF database uses.
    We are not going to update the SDK that the CFFS servers use.

    Therefore SXL users will get very slightly better categorization automatically as soon as we finish the SXL server.  Local db will get very slightly better categorization when they upgrade to the new version.  Anyone still using CFFS (which is anyone on 9.0 or earlier, or anyone who has manually changed their settings) will have the same categorization.
    By "slightly better" I mean less than 0.01% improvement.  :)

  • Just revisited this to understand your point.   I was burned on the sub-category / super-category distinction.   My "Uncategorized" sub-category is in the "web filtering problems" super-category, and only super-categories appear in the Filter Action menus.

    In the logs, I have all of the folllowing CategoryCode/Category pairs from the web logs:

    9998 - Uncategorized

    9998,9998 - Uncategorized,Uncategorized

    9999 - Categorization failed

    The "9998,9998" entry is the most common.   For three days of logs:

    Category#   Log Entries
    9998                 94
    9998,9998   52,239
    9999                 43

    (out of 1,668,457 total log entries)

    Many of these entries appear to be categorized sites that become uncategorized when the path gets long, and McAfee shows most of them with valid categories.   It would help to know which Uncategorized code is which.

    Support had led me to believe that the new search engine was going to be a fix for Uncategorized sites.   Michael Dunn's comment is discouraging, as it suggests that there may be no fix.

  • That seems odd to me.  I just double-checked, I have no categorization problems with URLS that are 5K in length (for most of the internet 4K is the practical limit).

    I would not expect to see sites becoming uncategorized, or having failures, due to the length of the URL.

    Can you please give some examples from the log?

    Can you please tell me the output of

    cc get http sc_local_db
    cc get http use_sxl_urid

    Does the UTM need to use an upstream/parent proxy?

Reply
  • That seems odd to me.  I just double-checked, I have no categorization problems with URLS that are 5K in length (for most of the internet 4K is the practical limit).

    I would not expect to see sites becoming uncategorized, or having failures, due to the length of the URL.

    Can you please give some examples from the log?

    Can you please tell me the output of

    cc get http sc_local_db
    cc get http use_sxl_urid

    Does the UTM need to use an upstream/parent proxy?

Children