Strange routing problem - Hosted in ESXi environment

Posted the ESX diagram.

So, above, SERVER(my windows machine) can access internet, can access XG etc - BUT cannot reach remote computers across IPSEC connection of XG.

My laptop, which also gets DHCP and gateway from XG, can access the remote compuetrs across IPSEC tunnel of XG.

The remote computers can access the Server(in esxi). Tried with both other hosted linuxes (3cx and Gira in above pic)

Packet capture doesnot capture packets (XG). 

Any update?

Summarizing - All the hosts in the ESXi server in which XG is installed cannot use IPSEC tunnel of XG.

XG silently drops any packet which has originated inside esxi and is to be routed via the tunnel.

XG itself can route other traffic via tunnel which has originated outside the ESXi.

Hello Nitin,

we have instaled many XG virtual appliances with the vmware standard switch and the distributed vmware switch and all installation route absolutely correctly.

Regards

alda

I would not have noticed it (all is working properly). I noticed it when I found out that I cannot access a computer which was on the other side of tunnel when I was RDPing into the windows server which was hosted with XG.

Do you have any IPSEC tunnel in your installation? 

My XG is inside ESXi, and has no issue in routing traffic when the traffic has originated outside ESXi.

For traffic originating inside the ESXI (inside the same switch), XG is failing ONLY for traffic which was to be routed via IPSEC tunnel. 

All traffic which passes through XG, and for traffic which is generated inside ESXi(but not via tunnel) eg - web, and others have no issue.

I narrowed it down to XG, as I see that my windows server is sending the traffic to XG, but XG just drops that traffic in blackhole ONLY if it was to be routed via the tunnel. All other traffic (like web etc) is visible and is routed correctly.

I will do packet capture tonight between the XG and the windows server, to confirm this.

Till now all is based on traceroutes and XG capabilities. Now that I have vsphere distributed switch, much more powerful diagnostics available.

Hello Nitin,

we use IPsec tunnels and SSL VPN remote access too and both work correctly and without problems. I think you have poorly defined IP networks that can communicate through an IPsec tunnel. I think so based on your description of behavior. I would focus attention in this direction. Sometimes it is good to draw a network diagram on paper and from another point of view to assess the real state of affairs.

Regards

alda

Hello Alda

Here is the diagram (on text)

Site 1

192.168.82.0/24

Windows Server on site 1 = 192.168.82.97

Gateway is 192.168.82.1

This site can ping 192.168.39.249

My Site

192.168.39.0/24

Windows server on my site = 192.168.39.249

Gateway is 192.168.39.1 (XG)

This site cannot ping 192.168.39.249

The IPSEC tunnel is between these two sites.

Here is the packet capture -

000c 2951 1daa 000c 2920 b357 0800 4500
003c 6378 0000 7e01 dd9d c0a8 5261 c0a8
27f9 0800 40c7 000b 0c8a 6162 6364 6566
6768 696a 6b6c 6d6e 6f70 7172 7374 7576
7761 6263 6465 6667 6869

I dont know much about packet capture - it says destination is 192.168.39.249, and source is 192.168.82.97, which i must be reading wrong.

Can you please read above packet capture.

Thank you

Apologies - My ping was on from 192.168.82.97.

So PCAP is wrong.

Correct PCAP to follow...

Replying to myself - using pktcap is esxi seems to capture only the packets coming into the port of the VM.

Is there a way to capture packets which are going OUT of the port.

Ok - Confirmed it. XG is dropping it.

Here is the packet capture from my internal machine using XG (192.168.39.1) as GW.

(switchport below is XG switchport)

pktcap-uw --switchport 67117068 --proto 0x01 --capture PortOutput

000c 2920 b357 000c 2951 1daa 0800 4500
003c f52a 0000 8001 49eb c0a8 27f9 c0a8
5261 0800 3a81 0001 12da 6162 6364 6566
6768 696a 6b6c 6d6e 6f70 7172 7374 7576
7761 6263 6465 6667 6869

So, 192.168.39.249(my site) is pinging 192.168.82.97 (site1). XGs NIC is receiving this ping (as the capture is from ESXi), and then putting this packet in blackhole because there is no trace of this packet inside XG.

What is so special in this configuration? How XG even knows this packet is originating from ESXi?

Hello Nitin,

do you have (by any chance) exchanged internal and external port in your XG config?

AND/OR did you put both physical NICs into the same switch?

BTW, and not meant to be impolite:

I would NEVER state things like these when troubleshooting is going on;

"Ofcourse I have checked the usual suspects such as default gateway etc - I am ok in the basics of networking - so usual suspects taken care of)."

This has the assumption, that we should not take care about these questions, although something really goes wrong in your setup.

First thing I teach my apprentices is "Ask questions, never make assumptions!"

BTW, and not meant to be impolite:

I would NEVER state things like these when troubleshooting is going on;

"Ofcourse I have checked the usual suspects such as default gateway etc - I am ok in the basics of networking - so usual suspects taken care of)."

This has the assumption, that we should not take care about these questions, although something really goes wrong in your setup.

First thing I teach my apprentices is "Ask questions, never make assumptions!"