Important note about SSL VPN compatibility for 20.0 MR1 with EoL SFOS versions and UTM9 OS. Learn more in the release notes.

Sophos Firewall: CLI Troubleshooting Tools

Disclaimer: This information is provided as-is for the benefit of the Community. Please contact Sophos Professional Services if you require assistance with your specific environment.


Overview

This Recommended Reads provides some of the basic troubleshooting tools that can be used from Advanced Shell of Sophos.

Accessing CLI

To access the Shell, SSH into Sophos and select option 5. Device Management and 3. Advanced Shell. For detailed instructions, refer to this KBA "Sophos Firewall: How to SSH to the firewall."

Commands

  • bwmon

This command displays the bandwidth of every interface in Sophos. By default, it’s shown in Kbytes/s. You may press the following buttons on the keyboard to show different values and change the output.

'+' increases sampling time by 100ms
'-' decreases sampling time by 100ms
'u’ cycle: bytes, bits, packets, errors
't' cycle: current rate, max, sum since start, average for last 30s



  • showfw 

This command will show the current Firmware version available in SFLoader.



  • nslookup 

This command can be used to resolve a DNS name. Using the "-q" option, you can specify which types of DNS records you want returned in the results.

For example, nslookup –q=mx sophos.com - would return the MX (Mail Exchanger) records for the Sophos.com domain.

The DNS record types that can be requested are:

A – Host record for an IPv4 address
AAAA – Host record for an IPv6 address
CNAME – An alias for another DNS name
MX – Mail Exchanger
TXT – Used to hold data for things like SPF (Sender Policy Framework), DKIM (DomainKeys Identified Mail), and more
PTR – A pointer to a canonical name

  • top/psmon

 This command provides a dynamic real-time view of a running system. It can show system summary information and a list of processes or threads currently being managed by the kernel. Top/psmon provides an ongoing look at processor activity in real time. It displays a listing of the most CPU-intensive tasks on the system and can provide an interactive interface for manipulating processes. It can sort the tasks by CPU, memory, and runtime usage.

PID - Process Id. The task's unique process ID periodically wraps, though never restarting at zero. In kernel terms, it’s a dispatchable entity defined by a 'task_struct'.
PR - Priority. The scheduling priority of the task. If you see ‘RT' in this field, it means the task is running under 'real time' scheduling priority.
NI - Nice Value. The nice value of the task. A negative nice value means priority, whereas a positive nice value means lower priority. Zero in this field means priority won’t be adjusted in determining a task's dispatchability.
VIRT - Virtual Memory Size (KiB). The total amount of virtual memory used by the task. It includes all code, data, and shared libraries, plus pages that have been swapped out and pages that have been mapped but not used.
RES - Resident Memory Size (KiB). The non-swapped physical memory a task has used.
SHR - Shared Memory Size (KiB). The amount of shared memory available to a task, not all of which is typically resident. It simply reflects memory that could be potentially shared with other processes.
S - Process Status. The status of the task can be one of the following:
   D = uninterruptible sleep
   R = running
   S = sleeping
   T = traced or stopped
   Z = zombie
%CPU - CPU Usage. The task's share of the elapsed CPU time since the last screen update is expressed as a percentage of total CPU time.
%MEM - Memory Usage (RES). A task's currently used share of available physical memory.
TIME+ - CPU Time (hundredths). Total CPU time the task has used since it started. The same as 'TIME', but reflecting more granularity through hundredths of a second.
COMMAND - Command name or Command line. Show the command line used to start a task or the name of the associated program. You can toggle between the command line and name with "c", which is both a command-line option and an interactive command.



  • free 

This command shows available and used RAM on the device.

  • Mem shows how much RAM is currently free or used.
  • Buffers indicate how much data is cached into the unused RAM.
  • Swap shows you the total available swap and how much is free or used.
  • All units on above output are in bytes.



  • tcpdump 

This command is a packet capture tool that allows interception and capture of packets passing through a network interface, making it useful for understanding and troubleshooting network layer problems. It helps monitor packet flow coming on the interface, response for each packet, packet drop, and ARP information.

tcpdump [-n] [-i interface] [-w file] [-r file] [-s0] [-X] [-v] [expression]

-n Show IP addresses rather than hostnames
-i Listen on interface
-w Write to file in pcap format
-r Read from pcap file
-s0 Show the whole packet that doesn’t truncate
-X Show both hex and ASCII (helpful for looking at clear text errors)
-v Show verbose output (use -vv and -vvv for more verbose output)
[expression] You can use expressions to filter your query. See below for the most commonly used.

  • host [ip | hostname] - filter for a specific host
  • port [port number] - filter for a specific port
  • ICMP - only show ICMP packets (ping and traceroute for example)
  • and [not] [or] - you can use multiple expressions together with
  • these keywords.

not (‘!=’ or ‘not’) Negation
and (‘&&’ or ‘and’) Concatenation
or (‘||’ or ‘or’) Alternation

Examples

To print all packets from host 192.168.1.69 arriving at or departing on eth1 (shows only IP addresses and verbose output)
tcpdump –eni eth1 host 192.168.1.69

To print all ICMP packets arriving at or departing on any interfaces
tcpdump –eni any icmp

To print all packets from host 192.168.0.121 or 192.168.0.135, excluding traffic on port 22 and port 4444 arriving or departing on eth5 (important to exclude packets of the WebAdmin-session and SSH itself)
tcpdump –eni eth5 host 192.168.0.121 or 192.168.0.135 and not port 22 and not port 4444

To print all packets to and from port 80 arriving at or departing on eth0
tcpdump –eni eth0 port 80

  • drppkt 

This command displays the packets dropped by firewall rules. It’ll provide connection details and details of the packets the device processes. This will help administrators to troubleshoot errant firewall rules. You can also filter the dropped packets.

drppkt host 10.10.10.1 and port 21 (This will show all dropped packets for host 10.10.10.1 and port 21)

How to view traffic of a ...

drppkt command

Example
specific host

drppkt host <ipaddress>

drppkt host 10.10.10.1

specific source host

drppkt src host <ipaddress>

drppkt src host 10.10.10.1

specific destination host

drppkt dst host <ipaddress>

drppkt dst host 10.10.10.1

specific network

drppkt net <network address>

drppkt net 10.10.10

specific source network

drppkt src net <network address>

drppkt src net 10.10.10

specific destination network

drppkt dst net <network address>

drppkt dst net 10.10.10

specific port

drppkt port <port-number>

drppkt port 21

specific source port

drppkt src port <port-number>

drppkt src port 21

specific destination port

drppkt dst port <port-number>

drppkt dst port 21

specific host for the particular port

drppkt host <ipaddress> and port <port-number>

drppkt host 10.10.10.1 and port 21

the specific host for all the ports except SSH

drppkt host <ipaddress> and port not <port-number>

drppkt host 10.10.10.1 and port not 22

specific protocol

drppkt proto ICMP

drppkt proto UDP

drppkt proto TCP

drppkt arp

 
  • conntrack 

This command/tool is used to list the connections in Sophos. It will also help you identify the firewall rule ID through which this packet was processed in the Sophos. In the first example of the captured conntrack, reply-sport is 3128, the HTTP proxy port for the Sophos device from where it was taken. It means that traffic was captured and processed by the web Proxy of the Sophos device. Also, look at the fwid, which indicates the firewall rule processing the connection.

proto=tcp proto-no=6 timeout=10765 state=ESTABLISHED orig-src=10.160.24.195 orig-dst=172.217.13.195 orig-sport=53498 orig-dport=443 packets=9 bytes=2243 reply-src=10.160.24.1 reply-dst=10.160.24.195 reply-sport=3128 reply-dport=53498 packets=9 bytes=6501 [ASSURED] mark=0x8001 use=2 id=1756024256 masterid=0 fwid=2 policytype=1 user=0 luserid=0 usergp=0 webfltid=1 hotspotid=0 hotspotuserid=0 hb_src=0 hb_dst=0 dnat_done=0 proxy_flags=0 icapid=0 appfltid=1 appid=100 catid=29 appcatid=5 ips=3 ips_nfqueue=0 ips_maxsesbytes=1 inmark=0x0 brdevinindex=0 devinindex=5 devoutindex=0 devin=Port1 devout= inzone=1 outzone=2 bwid=0 upclass=0:0 dnclass=0:0 sslvpnid=0 snatid=1 cluster_node=0 gwoff=0 ctflags=0x4200840a mmflags=0x10802200 dropfix=0 src_mac=00:00:00:00:00:00 dst_mac=00:00:00:00:00:00 vlan_id=0 diffserv=0 current_state[0]=7 current_state[1]=7

In this second example, you can see that fwid is zero but the connection was still processed, which in most cases means that it was a system(Sophos) generated traffic.

proto=tcp proto-no=6 timeout=10799 state=ESTABLISHED orig-src=10.20.20.63 orig-dst=84.39.152.32 orig-sport=46340 orig-dport=80 packets=7 bytes=1352 reply-src=84.39.152.32 reply-dst=10.20.20.63 reply-sport=80 reply-dport=46340 packets=8 bytes=941 [ASSURED] mark=0x8001 use=1 id=971057744 masterid=0 fwid=0 policytype=0 user=0 luserid=0 usergp=0 webfltid=0 hotspotid=0 hotspotuserid=0 hb_src=0 hb_dst=0 dnat_done=0 proxy_flags=0 icapid=0 appfltid=0 appid=0 catid=0 appcatid=0 ips=0 ips_nfqueue=0 ips_maxsesbytes=0 inmark=0x0 brdevinindex=0 devinindex=0 devoutindex=6 devin= devout=Port2 inzone=0 outzone=0 bwid=0 upclass=0:0 dnclass=0:0 sslvpnid=0 snatid=0 cluster_node=0 gwoff=0 ctflags=0x0 mmflags=0x2000 dropfix=0 src_mac=00:00:00:00:00:00 dst_mac=00:00:00:00:00:00 vlan_id=0 diffserv=0 current_state[0]=8 current_state[1]=8

Conntrack command can be used for multiple purposes, but for this specific guide, we’ll only use this command is syntax conntrack -L, which would show or list the connections. Some common parameters and options for conntrack are:

-s, --orig-src ip Source address from the original direction
-d, --orig-dst ip Destination address from the original direction
-r, --reply-src ip Source addres from reply direction
-q, --reply-dst ip Destination address from reply direction
-p, --protonum proto Layer 4 Protocol, e.g. 'tcp'
-P, --not-protonum notproto Except Protonum(Do not use this option with 'p')
-f, --family proto Layer 3 Protocol, eg. 'ipv6'
-t, --timeout timeout Set timeout
-u, --status status Set status, e.g. ASSURED
-b, --buffer-size Netlink socket buffer size

i.e. conntrack -L -s 192.168.24.195 -d 8.8.8.8 will show connections for which Src IP is 10.160.24.195 and destination is 8.8.8.8.

proto=icmp proto-no=1 timeout=29 orig-src=10.160.24.195 orig-dst=8.8.8.8 type=8 code=0 id=1 packets=8 bytes=480 reply-src=8.8.8.8 reply-dst=10.20.20.63 type=0 code=0 id=1 packets=8 bytes=480 mark=0x8001 use=2 id=1529445248 masterid=0 fwid=2 policytype=1 user=0 luserid=0 usergp=0 webfltid=1 hotspotid=0 hotspotuserid=0 hb_src=0 hb_dst=0 dnat_done=0 proxy_flags=0 icapid=0 appfltid=1 appid=0 catid=0 appcatid=0 ips=3 ips_nfqueue=100 ips_maxsesbytes=0 inmark=0x0 brdevinindex=0 devinindex=5 devoutindex=6 devin=Port1 devout=Port2 inzone=1 outzone=2 bwid=0 upclass=0:0 dnclass=0:0 sslvpnid=0 snatid=1 cluster_node=0 gwoff=0 ctflags=0x200000a mmflags=0x10902200 dropfix=0 src_mac=00:00:00:00:00:00 dst_mac=00:00:00:00:00:00 vlan_id=0 diffserv=0 current_state[0]=8 current_state[1]=8


The description of different Conntrack fields is mentioned in this table:

fwid Firewall rule id
idp IPS policy id
webfiltid Web filter policy id
appfiltid Application filter policy id
snatid Source natting policy id
svp sslvpn policy id
bwid Bandwidth policy id
luserid Live user id
user User id
usergp Usergroup id
hotspotid ID of Hotspot from which traffic is passing
hotspotuserid Hotspot User id
appid Application id
appcatid Application category id
catid Web category id
inzone zone (lan/wan/dmz) id of in interface
outzone zone id of out interface
brinindex bridgein device index
devinindex in device index
devoutindex out device index
iif[2] Storing interface index for routing cache
ctflags

Flags -1 (ctflags)

mmflags Flags-2 (mmflags)
scanflags Flags-3 (scanflags)
proxy_flags Flags-4 (proxy flags)
route_dst[2] Caching route in conntrack (both directions)
current_state[2] System snapshot number (for state change)
classid[2] Bandwidth classes for traffic shaping (QoS)
hb_src Heartbeat health for source
hb_dst

Heartbeat health for destination

diffserv DSCP marking (as per firewall rule)
Do you have an idea or suggestion regarding our Documentation, Knowledgebase, or Videos? Please visit our User Assistance forum on the Community to share your feedback! https://community.sophos.com/community-chat/f/user-assistance-feedback


Revamped RR Added Overview & Table of Contents Added Horizontal Lines Corrected Grammar & Font Size
[edited by: Erick Jan at 11:26 AM (GMT -7) on 25 Sep 2023]