Location of Web Categories List File

I have been parsing the webfilter log data to create master lists for the pairs ID/Name, Category/CategoryName, and AppID/Application.  The motivation was to reduce the size of the log data, by eliminating unnecessary duplication, when I load the logs into a SQL database.   For the same reason, I generate my own index numbers so that the long text strings for Profile, Filter Action, and UA (UserAgent) can be replaced with a number.   For completeness, I also created a number-to-description table to hold HTTP status codes, since the log data only contains numbers.

I have noticed that the ID/Name pairing appears in other log files.   Each log file uses an independent set of ID codes, so that the aggregate is (or appears to be) globally unique.

By adding new entries to the code-to-description tables as they occur in the logs, I avoid the need to anticipate additions that Sophos may implement in the future.   But in case my parsing fails, I always do my code-to-description lookups using an outer join.   I would rather have a missing description field than to accidentally omit a log entry from a report due to an inner join lookup where a code-to-description translation is missing.

Not knowing the Sophos position on whether this is proprietary information, I am reluctant to post what I have.   But you should be able to construct a similar list using a similar process.

Hi Douglas,

Thank you very much for your input, seems like you have done quite a bit of impressive work on pruning the amount of data created by logs.

My query was just one of curiosity. I spend a lot of time poring through log files generated by our UTMs and like to have a good grasp on what every field means. I do appreciate that the logs display both Category ID and Category Name (e.g. category="154", categoryname="Web Ads). For my own requirements/peace of mind, I thought there would be a file somewhere which would list the Category IDs and their corresponding Category Names.

Best regards and thanks again,

John P

Hi Douglas,

Thank you very much for your input, seems like you have done quite a bit of impressive work on pruning the amount of data created by logs.

My query was just one of curiosity. I spend a lot of time poring through log files generated by our UTMs and like to have a good grasp on what every field means. I do appreciate that the logs display both Category ID and Category Name (e.g. category="154", categoryname="Web Ads). For my own requirements/peace of mind, I thought there would be a file somewhere which would list the Category IDs and their corresponding Category Names.

Best regards and thanks again,

John P