Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 9 replies
  • Subscribers 2 subscribers
  • Views 6976 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

AWS Quickstart - Enable outbound ports

Andrew Shepherd

Hi there,

 

I'm running the AWS Quickstart cloudformation templates for Sophos UTM 9 for an evaluation of the product as a transparent web proxy. This all works fine and is quite straight forward.

 

An additional requirement is to enable outbound traffic for logging, which is to be sent to our monitoring tool datadog.

For this we must enable outbound tcp connections on ports 10514 and 10516 to intake.logs.datadoghq.com (ref datadog network config for logs collection)

I've created and enabled the following firewall rule (permissive obviously for debugging)

Source: Any

Service: 10514 / 10516

Destination: Any

Action: Allow

Advanced/Log Traffic: Enabled

 

I've tested connectivity from the tester EC2 instance and it shows the outbound connection as permitted however it does not work as the telnet client hangs pending connectivity.

 

 

aws.amazon.com/.../
34 package(s) needed for security, out of 54 available
Run "sudo yum update" to apply all updates.
[ec2-user@ip-10-100-101-170 ~]$ telnet intake.logs.datadoghq.com 10514
Trying 52.206.235.241...

<hangs />

 

I've checked the AWS security group and route table for the OGWs - all protocols are permitted and 0.0.0.0/0 routes to the internet.

Telneting to 80/443 for the domain works, so DNS is resolving correctly.

With the exception of web filtering configuration there has been any other modification to the Quickstart CF templates.

 

Questions

  • Is the above the correct configuration for enabling outbound tcp traffic on a specific port? 
  • Is there any other configuration required within Sophos UTM? 
  • Are there any AWS config changes that might be required?

 

Any suggestions or assistance would be much appreciated



This thread was automatically locked due to age.
  • 0

    Hi Andrew and welcome to the UTM Community!

    What do you learn from doing #1 in Rulz (last updated 2019-04-17).  Alone among the logs, the Firewall Live Log presents abbreviated information in a format easier to read quickly.  Usually, you can't troubleshoot without looking at the corresponding line from the full Firewall log file.  Please post one line corresponding to one blocked above.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • 0 in reply to BAlfson

    Hi Bob,

     

    thanks very much for the advice

    Here are the complete logs for the same issue. I've regenerated these just now

     

     

    2019:05:10-21:33:54 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="52.207.149.246" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="47594" dstport="10514" tcpflags="SYN"
    2019:05:10-21:34:57 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="52.207.149.246" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="47594" dstport="10514" tcpflags="SYN"
    2019:05:10-21:36:03 sophos-eval-i-0cf50e286810fd54b ulogd[13156]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="02:8f:27:37:7f:48" srcip="10.100.101.170" dstip="54.89.13.167" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="35060" dstport="10514" tcpflags="SYN"
    2019:05:10-21:36:34 sophos-eval-i-0cf50e286810fd54b ulogd[13156]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="02:8f:27:37:7f:48" srcip="10.100.101.170" dstip="54.89.13.167" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="35060" dstport="10514" tcpflags="SYN"
    2019:05:10-21:37:08 sophos-eval-i-0cf50e286810fd54b ulogd[13156]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="02:8f:27:37:7f:48" srcip="10.100.101.170" dstip="54.89.13.167" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="35060" dstport="10514" tcpflags="SYN"
    2019:05:10-21:38:14 sophos-eval-i-0cf50e286810fd54b ulogd[13156]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="02:8f:27:37:7f:48" srcip="10.100.101.170" dstip="54.209.130.218" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="42006" dstport="10514" tcpflags="SYN"

     

    and for completeness the same as viewed in the UI

     

    thanks very much for any suggestions you can make

  • 0

    How about pictures of the Edits of your firewall rule and of the related Service definition(s)?  Also, please copy the line from the Firewall log file corresponding to the one at 23:03:10 in the picture in your first post.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • 0 in reply to BAlfson

    certainly Bob. Here we go

    Firewall rule

     

    (We're using ANY for source and dest while debugging and will be changing these to appropriate definitions once the issue is understood)

     

    Service definition

     

    Here are the archived firewall logs corresponding to the live log entries in my original post

     

    2019:05:09-23:03:10 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2001" severity="info" sys="SecureNet" sub="packetfilter" name="Packet dropped" action="drop" fwrule="60002" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="54.210.188.168" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="54158" dstport="10514" tcpflags="SYN"
    2019:05:09-23:03:12 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2001" severity="info" sys="SecureNet" sub="packetfilter" name="Packet dropped" action="drop" fwrule="60002" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="54.210.188.168" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="54158" dstport="10514" tcpflags="SYN"
    2019:05:09-23:05:13 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="1" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="3.85.250.140" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="35538" dstport="10514" tcpflags="SYN"
    2019:05:09-23:05:51 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2001" severity="info" sys="SecureNet" sub="packetfilter" name="Packet dropped" action="drop" fwrule="60002" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="52.6.186.36" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="34942" dstport="10516" tcpflags="SYN"
    2019:05:09-23:05:52 sophos-eval-i-056817dbcef599d8f ulogd[14045]: id="2001" severity="info" sys="SecureNet" sub="packetfilter" name="Packet dropped" action="drop" fwrule="60002" initf="egw0" outitf="eth0" srcmac="06:bd:c3:1a:49:14" srcip="10.100.101.170" dstip="52.6.186.36" proto="6" length="60" tos="0x10" prec="0x00" ttl="253" srcport="34942" dstport="10516" tcpflags="SYN"

     

    many thanks for you help

  • 0 in reply to Andrew Shepherd

    I don't see any reason for the first two blocks in your first post, Andrew.  The IP is in the USA and the blocked packets should have qualified for your firewall rule #1.  There are no blocks in your most-recent post, so I will assume that the instance was rebooted or that the old firewall rule was deleted and recreated.  Apparently, the configuration daemon didn't initially construct the code properly from the configuration data base.  Does it work now as you expected?

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
  • 0 in reply to BAlfson

    Bob,

     

    I'm very sorry I was not clear in my original post. The first two blocks happened before the firewall rule was implemented. The connection reported as accepted ( 23:05:13) was after the firewall rule was implemented. 

    The issue is that when the firewall rule is operational and, while connections are being reported as accepted in the UTM firewall logs, the connections themselves are not actually successful. the telnet client I'm using to validate connectivity is hanging

     

    I'm testing connectivity from a host on the network managed by UTM using the telnet client

    The telnet command (telnet intake.logs.datadoghq.com 10514) is proven and works on other networks not managed by UTM

    Running telnet intake.logs.datadoghq.com 80  (or 443) from a host on the network managed by UTM works as the domain has been whitelisted in the web filtering polices (ports 80+443 are open on the destination hosts but cannot be used for my purposes) Incorrect. I'm not sure I why I thought this was working

     

    UTM on AWS config

    As I'm using UTM on AWS I've connected to the swarm worker that is managing these transactions to check if the firewall rules I'm setting are reflected in the iptables. 

    when i enable and disable the firewall rule listed previous (src: ANY, service 10514, dest ANY) I can see this iptables entry being added and removed accordingly

    # iptables -L

    <snip />

    Chain USR_FORWARD (1 references)
    target prot opt source destination
    LOGACCEPT tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:10514 LOGMARK match 1

    <snip />

     

    I've limited experience with iptables, nevermind iptables with the complexity of those managed by UTM so I can't diagnose what going on. Nevertheless it looks like a rule is in place.

    I've included the full 

     

    Outbound connectivity to Internet via AWS internet gateway

    To sanity check if there is some issue with outbound traffic to 10514 via the internet gateway I've duplicated the EC2 configuration for a UTM worker but using a vanilla Amazon Linux AMI, rather than the Sophos UTM work AMI. So all AWS networking and security config is retained, the instance runs in the same VPC and subnet as the as the UTM worker, but UTM is not involved and does not evaluate the traffic.

    In this case when running telnet intake.logs.datadoghq.com 10514 then connectivity is established.

     

    What this suggests to me is that the issue may be something to do with or something on the UTM works instance.

     

    I've attempted to connect to 10514 from the UTM work but this doesn't work. I presume this is because the firewall rules don't apply to the UTM workers themselves. My naive reading of the iptables would also suggest this is the case as the 10514 firewall applies to the FORWARD chain, not the OUTPUT chain, so won't exempt connection attempts to 10514 that are made locally on the UTM worker.

     

     

     

    Sorry all this is a lot to digest. I'm continuing to work through to the issue. Any suggestions are welcome.

     

    kind regards

     

     

     

     

    UTM iptables

    It's probably not relevant but for the sake of completeness here are the iptables when the firewall rule is enable.

    I'd expect most of this is vanilla as, aside from the firewall rule, I've only implemented steps outlined in the AWS Quickstart for UTM

    Sophos UTM (AWS)
    (C) Copyright 2000-2018 Sophos Limited and others. All rights reserved.
    Sophos is a registered trademark of Sophos Limited and Sophos Group.
    All other product and company names mentioned are trademarks or registered
    trademarks of their respective owners.

    For more copyright information look at /doc/astaro-license.txt
    or www.astaro.com/.../astaro-license.txt

    NOTE: If not explicitly approved by Sophos support, any modifications
    done by root will void your support.

    <W> (i-0cf50e286810fd54b) loginuser@sophos-eval:/home/login > sudo -i
    <W> (i-0cf50e286810fd54b) sophos-eval:/root # iptables -L
    Chain INPUT (policy DROP)
    target prot opt source destination
    ACCEPT all -- anywhere anywhere
    ACCEPT all -- anywhere !base-address.mcast.net/4 CONFIRMED match
    CONFIRMED all -- anywhere anywhere ctstate RELATED
    HA_IN all -- anywhere anywhere
    LOCKOUT all -- anywhere anywhere
    PSD_MATCH all -- anywhere anywhere
    SANITY_CHECKS all -- anywhere anywhere
    AUTO_INPUT all -- anywhere anywhere
    USR_INPUT all -- anywhere anywhere
    LOGDROP all -- anywhere anywhere LOGMARK match 60001

    Chain FORWARD (policy DROP)
    target prot opt source destination
    ACCEPT all -- anywhere !base-address.mcast.net/4 CONFIRMED match
    RELATED_FWD all -- anywhere anywhere ctstate RELATED
    PSD_MATCH all -- anywhere anywhere
    AUTO_FORWARD all -- anywhere anywhere
    USR_FORWARD all -- anywhere anywhere
    LOGDROP all -- anywhere anywhere LOGMARK match 60002

    Chain OUTPUT (policy DROP)
    target prot opt source destination
    LOGDROP tcp -- !loopback/8 anywhere tcp spts:1024:65535 dpt:webadmin LOGMARK match 60005
    LOGDROP tcp -- anywhere db_host.local tcp dpt:4472 owner UID match loginuser
    LOCAL_RESTAPI tcp -- anywhere anywhere tcp dpt:exlm-agent
    LOCAL_RESTAPI tcp -- anywhere anywhere tcp dpt:dashpas-port
    ACCEPT all -- anywhere anywhere
    ACCEPT all -- anywhere !base-address.mcast.net/4 CONFIRMED match
    CONFIRMED all -- anywhere anywhere ctstate RELATED
    CONFIRMED all -- anywhere anywhere -m condition --condition "OUTPUT_ACCEPT_ALL" owner UID match root owner GID match root
    HA_OUT all -- anywhere anywhere
    SANITY_CHECKS all -- anywhere anywhere
    AUTO_OUTPUT all -- anywhere anywhere
    USR_OUTPUT all -- anywhere anywhere
    LOGDROP all -- anywhere anywhere LOGMARK match 60003

    Chain AUTO_FORWARD (1 references)
    target prot opt source destination
    LOGDROP tcp -- anywhere 169.254.169.254 tcp spts:tcpmux:65535 dpt:http

    Chain AUTO_INPUT (1 references)
    target prot opt source destination
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:ssh
    LOGDROP tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:ssh LOGMARK match 60004
    CONFIRMED tcp -- anywhere anywhere tcp spts:1024:65535 dpt:webadmin
    LOGDROP tcp -- anywhere anywhere tcp spts:1024:65535 dpt:webadmin LOGMARK match 60005
    CONFIRMED udp -- anywhere anywhere udp spt:bootps dpt:bootpc
    CONFIRMED tcp -- 10.15.4.0/24 anywhere tcp spts:domain:65535 dpt:domain
    CONFIRMED udp -- 10.15.4.0/24 anywhere udp spts:domain:65535 dpt:domain
    CONFIRMED tcp -- anywhere anywhere match-set NoxRMviV7p0XjnMyYzNYpQ src tcp spts:tcpmux:65535 dpt:http-alt
    CONFIRMED icmp -- anywhere anywhere icmptype 8 code 0
    LOGDROP tcp -- anywhere anywhere tcp spts:tcpmux:65535 multiport dports smtp,smtps,submission
    CONFIRMED udp -- 240.0.4.5 anywhere udp spts:tcpmux:65535 dpts:tcpmux:65535
    CONFIRMED udp -- 240.0.8.5 anywhere udp spts:tcpmux:65535 dpts:tcpmux:65535
    CONFIRMED gre -- anywhere anywhere
    CONFIRMED all -- anywhere anywhere mark match 0x40000/0x40000

    Chain AUTO_OUTPUT (1 references)
    target prot opt source destination
    CONFIRMED tcp -- anywhere ec2-23-20-91-175.compute-1.amazonaws.com tcp dpt:https
    LOGDROP tcp -- anywhere 169.254.169.254 tcp spts:tcpmux:65535 dpt:http ! owner UID match root
    CONFIRMED udp -- anywhere anywhere udp spt:bootpc dpt:bootps
    CONFIRMED tcp -- anywhere anywhere tcp spts:domain:65535 dpt:domain
    CONFIRMED udp -- anywhere anywhere udp spts:domain:65535 dpt:domain
    CONFIRMED udp -- anywhere anywhere udp spt:domain dpts:domain:65535
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:http
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:https
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:ldap
    CONFIRMED udp -- anywhere anywhere udp spts:tcpmux:65535 dpt:ldap
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:ldaps
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:http-alt
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:ftp
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:3840
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:webadmin
    CONFIRMED icmp -- anywhere anywhere icmptype 8 code 0
    CONFIRMED udp -- anywhere anywhere udp spts:1024:65535 multiport dports 33000:34000,44444:55555
    CONFIRMED udp -- anywhere anywhere match-set z+qc1DEoGvOS+9Df3kmVKA dst udp spts:ntp:65535 dpt:ntp
    CONFIRMED tcp -- anywhere anywhere tcp spts:tcpmux:65535 multiport dports smtp,smtps,submission
    CONFIRMED udp -- anywhere 240.0.4.5 udp spts:tcpmux:65535 dpts:tcpmux:65535
    CONFIRMED udp -- anywhere 240.0.8.5 udp spts:tcpmux:65535 dpts:tcpmux:65535
    CONFIRMED tcp -- anywhere 10.100.1.37 tcp spts:tcpmux:65535 dpt:personal-agent
    CONFIRMED tcp -- anywhere 10.100.2.64 tcp spts:tcpmux:65535 dpt:personal-agent
    CONFIRMED gre -- anywhere anywhere
    CONFIRMED tcp -- anywhere anywhere tcp spts:1024:65535 dpt:https
    CONFIRMED tcp -- anywhere a104-123-101-156.deploy.static.akamaitechnologies.com tcp spts:tcpmux:65535 dpt:https owner UID match dehydrated owner GID match dehydrated
    CONFIRMED udp -- anywhere anywhere udp spts:tcpmux:65535 dpt:syslog

    Chain GEOIP_OUT (0 references)
    target prot opt source destination

    Chain GEOIP_REJECT (0 references)
    target prot opt source destination
    REJECT tcp -- anywhere anywhere reject-with tcp-reset
    REJECT all -- anywhere anywhere reject-with icmp-port-unreachable

    Chain HA_IN (1 references)
    target prot opt source destination

    Chain HA_OUT (1 references)
    target prot opt source destination

    Chain INVALID_PKT (0 references)
    target prot opt source destination
    NFLOG all -- anywhere anywhere LOGMARK match 60007 nflog-prefix "INVALID_PKT: "
    DROP all -- anywhere anywhere

    Chain LOCAL_RESTAPI (2 references)
    target prot opt source destination
    ACCEPT all -- anywhere anywhere owner UID match root
    ACCEPT all -- anywhere anywhere owner UID match loginuser
    DROP all -- anywhere anywhere

    Chain LOCKOUT (1 references)
    target prot opt source destination

    Chain LOGACCEPT (1 references)
    target prot opt source destination
    ACCEPT all -- anywhere anywhere ADDRTYPE match src-type BROADCAST
    ACCEPT all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST
    NFLOG all -- anywhere anywhere nflog-prefix "ACCEPT: "
    CONFIRMED all -- anywhere anywhere

    Chain LOGDROP (10 references)
    target prot opt source destination
    DROP all -- anywhere anywhere ADDRTYPE match src-type BROADCAST
    DROP all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST
    NFLOG all -- anywhere anywhere nflog-prefix "DROP: "
    DROP all -- anywhere anywhere

    Chain LOGREJECT (0 references)
    target prot opt source destination
    REJECT all -- anywhere anywhere ADDRTYPE match src-type BROADCAST reject-with icmp-port-unreachable
    REJECT all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST reject-with icmp-port-unreachable
    NFLOG all -- anywhere anywhere nflog-prefix "REJECT: "
    REJECT all -- anywhere anywhere reject-with icmp-port-unreachable

    Chain MULTIPATH_DROP (0 references)
    target prot opt source destination

    Chain PSD_ACTION (0 references)
    target prot opt source destination

    Chain PSD_MATCH (2 references)
    target prot opt source destination

    Chain RELATED_FWD (1 references)
    target prot opt source destination
    CONFIRMED all -- anywhere anywhere

    Chain SANITY_CHECKS (2 references)
    target prot opt source destination

    Chain STRICT_TCP_DROP (0 references)
    target prot opt source destination
    DROP all -- anywhere anywhere

    Chain STRICT_TCP_STATE (0 references)
    target prot opt source destination

    Chain USR_FORWARD (1 references)
    target prot opt source destination
    LOGACCEPT tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:10514 LOGMARK match 1

    Chain USR_INPUT (1 references)
    target prot opt source destination

    Chain USR_OUTPUT (1 references)
    target prot opt source destination
    <W> (i-0cf50e286810fd54b) sophos-eval:/root # exit
    logout
    <W> (i-0cf50e286810fd54b) loginuser@sophos-eval:/home/login > exit
    logout
    Connection to 34.247.33.64 closed.
    ➜ ~ ssh -i /Users/andrew.shepherd/Downloads/mvfsophos-evaleu-west-1.pem loginuser@34.247.33.64
    Last login: Sun May 12 19:40:07 2019 from 151.22.9.51.dyn.plus.net


    Sophos UTM (AWS)
    (C) Copyright 2000-2018 Sophos Limited and others. All rights reserved.
    Sophos is a registered trademark of Sophos Limited and Sophos Group.
    All other product and company names mentioned are trademarks or registered
    trademarks of their respective owners.

    For more copyright information look at /doc/astaro-license.txt
    or www.astaro.com/.../astaro-license.txt

    NOTE: If not explicitly approved by Sophos support, any modifications
    done by root will void your support.

    <W> (i-0cf50e286810fd54b) loginuser@sophos-eval:/home/login > telnet intake.logs.datadoghq.com 10514
    Trying 35.174.76.62...
    ^C
    <W> (i-0cf50e286810fd54b) loginuser@sophos-eval:/home/login >

     

    nat table

    <W> (i-0cf50e286810fd54b) sophos-eval:/root # iptables -L -t nat
    Chain PREROUTING (policy ACCEPT)
    target prot opt source destination
    AUTO_PRE all -- anywhere anywhere
    USR_PRE all -- anywhere anywhere
    LOAD_BALANCING all -- anywhere anywhere

    Chain INPUT (policy ACCEPT)
    target prot opt source destination

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination
    AUTO_OUTPUT all -- anywhere anywhere
    USR_OUTPUT all -- anywhere anywhere

    Chain POSTROUTING (policy ACCEPT)
    target prot opt source destination
    AUTO_POST all -- anywhere anywhere
    USR_POST all -- anywhere anywhere

    Chain AUTO_OUTPUT (1 references)
    target prot opt source destination

    Chain AUTO_POST (1 references)
    target prot opt source destination

    Chain AUTO_PRE (1 references)
    target prot opt source destination
    ACCEPT tcp -- anywhere anywhere tcp spts:1024:65535 dpt:webadmin ADDRTYPE match dst-type LOCAL

    Chain LOAD_BALANCING (1 references)
    target prot opt source destination

    Chain USR_OUTPUT (1 references)
    target prot opt source destination

    Chain USR_POST (1 references)
    target prot opt source destination

    Chain USR_PRE (1 references)
    target prot opt source destination
    <W> (i-0cf50e286810fd54b) sophos-eval:/root #

     

    mangle table

    <W> (i-0cf50e286810fd54b) sophos-eval:/root # iptables -L -t mangle
    Chain PREROUTING (policy ACCEPT)
    target prot opt source destination
    TPROXY_HOOK tcp -- anywhere anywhere
    TPROXY_HOOK icmp -- anywhere anywhere

    Chain INPUT (policy ACCEPT)
    target prot opt source destination
    CONNMARK all -- anywhere anywhere ctstate NEW CONNMARK or 0x80000

    Chain FORWARD (policy ACCEPT)
    target prot opt source destination
    CONNMARK all -- anywhere anywhere ctstate NEW CONNMARK or 0x100000
    CONNMARK all -- anywhere anywhere ctstate NEW CONNMARK or 0x80000

    Chain OUTPUT (policy ACCEPT)
    target prot opt source destination
    CONNMARK all -- anywhere anywhere ctstate NEW CONNMARK or 0x80000
    CONNMARK all -- anywhere anywhere ctstate NEW mark match 0x80000/0x80000 CONNMARK or 0x40000

    Chain POSTROUTING (policy ACCEPT)
    target prot opt source destination

    Chain AFC_CLUSTER_POSTROUTING (0 references)
    target prot opt source destination

    Chain AFC_DETECT (0 references)
    target prot opt source destination

    Chain AFC_EXCEPTIONS_ALL (1 references)
    target prot opt source destination

    Chain AFC_EXCEPTIONS_IN (0 references)
    target prot opt source destination
    AFC_EXCEPTIONS_ALL all -- anywhere anywhere

    Chain AFC_EXCEPTIONS_OUT (0 references)
    target prot opt source destination
    CONNMARK tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:https owner UID match root CONNMARK xset 0x1319/0xffff
    CONNMARK tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:http owner UID match root CONNMARK xset 0x1319/0xffff

    Chain APTP_ALERT (0 references)
    target prot opt source destination

    Chain APTP_BLOCK (0 references)
    target prot opt source destination

    Chain APTP_EXCEPTIONS (0 references)
    target prot opt source destination

    Chain CLUSTER_INPUT (0 references)
    target prot opt source destination

    Chain FLOW_MONITOR (0 references)
    target prot opt source destination
    CONNMARK all -- anywhere anywhere CONNMARK and 0xffffff

    Chain GEOIP_DROP (0 references)
    target prot opt source destination
    DROP all -- anywhere anywhere

    Chain GEOIP_FORWARD (0 references)
    target prot opt source destination

    Chain GEOIP_IN (0 references)
    target prot opt source destination

    Chain HOTSPOT_CUTOFF_POST (0 references)
    target prot opt source destination

    Chain HOTSPOT_CUTOFF_PRE (0 references)
    target prot opt source destination

    Chain POLICY_ROUTING_OUT (0 references)
    target prot opt source destination

    Chain POLICY_ROUTING_PRE (0 references)
    target prot opt source destination

    Chain SANITYCHECK_FORWARD (0 references)
    target prot opt source destination

    Chain SANITYCHECK_IN (0 references)
    target prot opt source destination

    Chain TPROXY_DIVERT (1 references)
    target prot opt source destination
    MARK all -- anywhere anywhere MARK set 0x40000
    ACCEPT all -- anywhere anywhere

    Chain TPROXY_DIVERT_HTTP (3 references)
    target prot opt source destination
    CONNMARK all -- anywhere anywhere socket CONNMARK or 0x40000
    TPROXY tcp -- anywhere anywhere TPROXY redirect 0.0.0.0:18080 mark 0x40000/0xffffffff

    Chain TPROXY_HOOK (2 references)
    target prot opt source destination
    TPROXY_DIVERT all -- anywhere anywhere ctstate RELATED,ESTABLISHED connmark match 0x40000/0x40000
    TPROXY_HOOK_HTTP tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:http ADDRTYPE match dst-type !LOCAL
    TPROXY_HOOK_HTTP tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:https ADDRTYPE match dst-type !LOCAL

    Chain TPROXY_HOOK_HTTP (2 references)
    target prot opt source destination
    TPROXY_DIVERT_HTTP all -- anywhere passthrough.fw-notify.net
    RETURN all -- anywhere passthrough.fw-notify.net
    TPROXY_DIVERT_HTTP tcp -- anywhere anywhere match-set NoxRMviV7p0XjnMyYzNYpQ src tcp spts:tcpmux:65535 dpt:http
    TPROXY_DIVERT_HTTP tcp -- anywhere anywhere match-set NoxRMviV7p0XjnMyYzNYpQ src tcp spts:tcpmux:65535 dpt:https
    <W> (i-0cf50e286810fd54b) sophos-eval:/root #

     

    d

  • 0 in reply to Andrew Shepherd

    Bob,

     

    From trying to understand the iptable rules as best I can I think that the majority of the action is happening in Filter table FORWARD chain. 

    The Chains for Filter/FORWARD match with the firewall log entries generated by the LOGACCEPT Chain invocation (with LOGMARK match 1 corresponding to the firewall rule number)

     

    Chain FORWARD (policy DROP)
    target prot opt source destination
    ACCEPT all -- anywhere !base-address.mcast.net/4 CONFIRMED match
    RELATED_FWD all -- anywhere anywhere ctstate RELATED
    PSD_MATCH all -- anywhere anywhere
    AUTO_FORWARD all -- anywhere anywhere
    USR_FORWARD all -- anywhere anywhere
    LOGDROP all -- anywhere anywhere LOGMARK match 60002


    Chain USR_FORWARD (1 references)
    target prot opt source destination
    LOGACCEPT tcp -- anywhere anywhere tcp spts:tcpmux:65535 dpt:10514 LOGMARK match 1

    Chain LOGACCEPT (1 references)
    target prot opt source destination
    ACCEPT all -- anywhere anywhere ADDRTYPE match src-type BROADCAST
    ACCEPT all -- anywhere anywhere ADDRTYPE match dst-type BROADCAST
    NFLOG all -- anywhere anywhere nflog-prefix "ACCEPT: "
    CONFIRMED all -- anywhere anywhere

     

    In reading this and looking at the logs it does appear as through UTM is accepting the packets and forwarding them on, though why it's not working is still unclear.

    I'm sure it's something very small that I've missed that is causing the issue.

     

    Here is a summary of the testing so far to help me identify further test cases and lines of enquiry

    Case Description Command Result Conclusion
    1 Connect from Laptop telnet intake.logs.datadoghq.com 10514 Success Destination service working as described
    2

    Connect from AWS

    Using VPC/EC2 unrelated to UTM

    		 telnet intake.logs.datadoghq.com 10514 Success

    Working from AWS.

    No issues with existing, non-UTM AWS config
    3

    Connection from UTM on AWS Ref Config

    test host -> OGW -> UTM worker -> Internet Gateway

    		 telnet intake.logs.datadoghq.com 10514  Failure  
    4

    Connection from UTM on AWS Ref Config

     

    telnet www.google.com 80 

    (Domain whitelisted for testing)

    		 Success

    Telnet works from test host albeit using UTM auto-managed rules and config for http
    5

    Connection from UTM on AWS Ref Config

    EC2 Inst (Cloned UTM) -> Internet Gateway

    		 telnet intake.logs.datadoghq.com 10514  Success

    AWS Routetable and security groups defined in UTM on AWS  Reference model allow traffic to 10514

    Connectivity works when UTM software/AMI not involved

    Destination service does not discriminate against traffic from UTM on AWS Reference infrastructure

    Note:

    When connectivity to destination service has worked I've entered data and confirmed it is received separately, to be certain everything is working.

     

    Question:

    What could be possible differences in behaviour of telnet for cases 4+5? Why would telnet work in #5 but not #4 given that ports 80 and 10514 are both open? Firewall rules for port 80 are part of UTM automanaged config. Is there something additional required to replicate this for user defined ports such as 10514?

  • 0 in reply to Andrew Shepherd

    Hi Bob.

     

    Well I'm pretty sure the issue is user error. I had not setup NAT for any of the firewall rules. I've done masquerading NAT for the moment and that seems to be working fine.

    I now need to understand this a little better and tune it so that only desired traffic is NATed.

     

     

    many thanks for your assistance with this. It's greatly appreciated

     

    Andrew

     

  • 0

    If you only want to allow the Datadog ports use an SNAT instead of a masq rule.  Start with a Service Group "Datadog" containing Service definitions for 10514 and 10516.  Then a NAT rule like 'SNAT : Any -> Datadog -> Any : from Internal (Address)'.

    You can make a similar rule for the Web Surfing group.  As #2 in Rulz (last updated 2019-04-17) clarifies, the automatic rules created by the configuration daemon (based on the WebAdmin databases) for Web Filtering preclude the need for a second NAT rule.

    All that said, your masqing solution will work fine.

    Cheers - Bob

     
    Sophos UTM Community Moderator
    Sophos Certified Architect - UTM
    Sophos Certified Engineer - XG
    Gold Solution Partner since 2005
    MediaSoft, Inc. USA
Unfiltered HTML