Runing Sophos UTM virtualized or on dedicated hardware?

widdde said:

Hi!
I have a computer with a Core I7 2600K, 32GB ram, 4 Intel nics, and I'm planning to run Sophos UTM on this as my primary firewall (home use with some internal and some public servers behind).

I think it is quite a waste to use the hardware as a firewall only, I don't think Sophos need an I7 with 32GB ram, or am I wrong?

I have a little thought to run ESXI on the machine to have some more vm:s running on it in parallell with Sophos UTM.

I think the hardware can handle a one or two vm:s alongside a Sophos install..?

My biggest concern are the security, how secure is it to run the firewall virtualized? For some reason my heart screams a little when I'm thinking to run it virtualized, because
a dedicated approach should be more secure, because you don't expose a virtualization platform to the net as you do with a virtualized approach.

But is it common that hackers find holes on the virtualization platform and can gain access to the internal lans? How safe are virtualization platforms nowadays?

I don't think it is an uncommon approach on companies to run it virtualized, but maybe I'm wrong. This is for personal use in a home environment, but I still want secure networks.

I may run it virtualized if you say that it is safe.. :P

For your information: I have a 250/100Mbps fiber connection to my house.

Thanks in advance! 

Widde,

You can run Sophos SG UTM and XG Firewall virtualized, but in your situation. To do this, you would have to allow the unfiltered traffic onto your network in order to filter it in a virtualized environment.  This is complicated and expensive for most home environments as it requires managed switches for VLANs and robust servers for virtualization.

For most home settings, all you need to do is deploy the UTM/Firewall on a small firewall appliance in between your fiber gateway and your internal network.  The device required is very simple computer with two or more ethernet ports.  You can purchase a firewall computer with RAM and SSD installed, or you can purchase a barebones systems and add your own RAM/storage.  In most cases, you don't need a more than 8GBs of RAM or 64GB of storage.  If you have an old computer laying around, you can use that too, but be warned that UTM and XG installers wipe the hard drive during installation.

On your fiber connection, I would recommend a Protectli Vault.  It includes multiple ports for organizing your LAN, virtualization, Wi-Fi, and other networks.  Each port will become an interface.  Each interface can have a default LAN on which you can add layers of VLANs.  In the default Gateway mode, your UTM will be the network router, and through it, you can create, manage, and combine all the interfaces, LANs, or VLANs.  You can do a lot of other things too.  If you prefer to use your existing router as the network manager, you can set the UTM in a Bridge mode.

I moved from the Sophos SG UTM to the Sophos XG Firewall.  It was easy because they use the same appliances.  My XG Firewall is in a ZBOX CI325 Nano with 4GB RAM and a 32GB SSD connected to an Arris SurfBoard SB6190 cable modem to a on a 350Mbps cable connection.  The Zbox only has two ports WAN and LAN.  The only thing on the LAN is an Apple Time Machine in bridge mode.  This gives me two networks, the default LAN for my internal wired/wireless/backup network and Guest Wi-Fi on a separate VLAN.

Hi David and thanks for your response!

Well, I'm not an ordinary home user, I already have 2x 24 ports managed switches and 4 x 8 ports managed switches.

I currently run Sophos UTM on hardware where I can't get full speed on my 250/100 Mbps connection with IPS enabled.

The speed drops about 100Mbps to 150Mbps, this is because snort is single threaded and therefor I planned to beef upp
the cpu to a Core I7 (but run it in a virtualized environment). 

I'm already running a couple of vlans in my home and will continue to do this after the change to other hardware.

From my fiber connection I will connect ESXI on a dedicated network card that only the firewall will use and all other network will reside on
another network card, the lan side, in the virtualized firewall.

My concern is if ESXI will be a security issue because the firewall isn't the first physical box, but if ESXI doesn't listen on the interface where I plug internet in

maybe it is a secure solution?

widdde said:

Well, I have googled a lot today and alot of people run firewalls virtualized, where do you get the information that it is super risky and why is it super risky when

the hypervisor doesn't listen to the net, just the firewall vm will do this.

Regarding snort, IPS is snort in the UTM. I am running IPS on my current UTM and the IPS implementation in the UTM is running snort.

Well, If I'm paying for 250Mbps I want to utilize all the bandwidth, don't you want to use all the available bandwidth with your connection? 
You don't know what I do on my connection or why I want more than 100Mbps or how many users we are that is sharing this connection. My family is backing up their computers to my servers once a day and It can be large files and therefor I want more than 100Mbps. I don't understand why you think I should downgrade to 100Mbps just to fit my current box when I have a beefier computer that can run the UTM. That is just silly, I'm sorry.

But it is not the question here, the question is if and how safe it is to run the firewall virtualized and when googling, it is not that insecure as you say what I can read.
So if you can direct me to the information where it says it is super risky, I'm really glad to read that.

I never said there is nothing wrong with 250Mbps.  As I said, I have a 350Mbps connection, but I don't always use the entire bandwidth.  I have another location with a 500Mbps and 1G connection which is similar to your setup and needs.  Knowing how you are using your setup helps to better help you.  I interpreted what you typed as a separate deployment of Snort.

Regardless, any internet traffic coming to your NIC (physical or virtual) is a potential point of entry.  The NIC itself can be attacked.  Your server OS, CPU, and RAM constantly communicate with your NICs.  From there, the hacker can gain access to the bus and RAM and other parts of your entire physical machine and virtualization setup.  Every other port on the server is a potential attack surface as well.  Simply plugging in a USB stick could take dow your entire server.  The virtual UTM would not be able to detect nor prevent an attack on the hypervisor or on the physical server itself because the UTM is isolated in a virtual environment.  IF the UTM was installed directly in a firewall appliance, the UTM would protect the host computer itself.

The other ports in to and out of your other NICs (virtual and physical) on that machine can also be attack points.  Any laptops, smartphones, tablets, or any other device (leaving your network and coming back to it on the wireless or wired side) can each be a carrier of malware that will enter your network from the internal Wi-Fi or switching.  Any new device you add to your internal network can be a new attack vector from within.  Depending on your particular setup, Your UTM may or may not be able to detect or deter this from hurting you internally.  This particular UTM setup will only ensure the internet traffic coming in and going out is clean.

The only way to prevent this kind of attack is to deploy a physical UTM or firewall in front of the virtual server or use security software on the physical host itself.  Sophos makes an agent that sits between BIOS and the OS/Hypervisor to protect the physical server as well as the OS, the hypervisor, and the virtual machines within; but you have to purchase this.  I have covered many datacenter with virtualization.  Your virtual UTM will do nothing to protect its physical machine or the hypervisor hosting the virtual machines within.

Okey, thanks for your response.
I know that every device in the network and devices that is carried out of the network can be potential dangerous, but somewhere we have to draw a line. This is my home network and I can't be that paranoid with phones carried in and out of the network and so on.

But I understand your point to not virtualize Sophos and it was this answer I was looking for. I think I will run UTM on dedicated hardware. Right now it is running on a box with a J1900 celeron (I think it is 2Ghz per core) and 8 GB ram. As I have understood IPS can run multiple instances of snort and when I do a benchmark test of my speed I can't get more than 150Mbps on that particular connection, but when surfing on other sites while doing a test I can get more bandwidth because another core will take care of that connection. Am i right?

It felt wrong to virtualize the firewall, but I wanted more information regarding this issue and exactly how insecure it is to expose the hypervisor. 

I have 2 servers running ESXI and both of them are behind my dedicated UTM today, and I think I will keep it that way. Maybe I use my Core I7 with yet another ESXI install and run some vm:s on it and keep my J1900 celeron box to continue to run Sophos on it (dedicated)

Dude, that was all FUD.  VM escape attacks are rare and you are not going to be targeted by these.  Just run it virtualized and be done with it.  VLAN your switch if you cannot physically segment and you will be fine.

Okey, I see, thanks for your response. 
I don't know what to think. Half the net seem to say it is totally ok to run it virtualized, and the rest
says it is super insecure.

My switches will be behind the lan of the Sophos UTM, the ESXI box will have one of its nic direct connected to
the fiber box (to the net) and this network card will only be used by Sophos UTM VM. On the lan side of the UTM
i will divide my network in some more VLANS..

(Fiber box (internet)) -> ESXI box with a nic connected to Sophos UTM VM -> Lan side with x VLANS -> Switches that have VLAN configurations -> servers and clients on different vlans. (The UTM has the rules which computer can talk to which server and so on so all traffic will go through the UTM between the VLANS)

I can see why people think this can be dangerous, because the ESXI box with it's OS is connected physically to the net running a virtual firewall,

but if nothing in ESXI host is listening on the card that is connected to the WAN NIC in my ESXI, how insecure is that?

darrellr said:

Dude, that was all FUD.  VM escape attacks are rare and you are not going to be targeted by these.  Just run it virtualized and be done with it.  VLAN your switch if you cannot physically segment and you will be fine.

 

I completely agree!

i run this on my home lab on 3 locations, all virtual and in vmware, We even have 100s of virtual UTM at customer sites. So dedicated HW at home for UTM is so last year, IF you have a big server for that - my oppinion :-)

But it takes at little more experience with vmware, but if your physical server has two nics, there is no need to go down that vlan road :-)

I needed to go with this CPU to make IPS / snort fully support my 300MB connection :-)

Thanks for your response!

My server has 4 physical Intel NICS. One of them will be used only by the Sophos UTM. (And regarding the CPU I will only run one or two small VM:s on the server
and Sophos UTM of course)
Unfortuneately I can't pass through the network card to the Sophos VM, because my Core I7 2600k does not have Vt/d (the k model does not have this)

Will this be an issue?

widdde said:

Thanks for your response!

 

My server has 4 physical Intel NICS. One of them will be used only by the Sophos UTM.
Unfortuneately I can't pass through the network card to the Sophos VM, because my Core I7 2600k does not have Vt/d (the k model does not have this)

Will this be an issue?

 

Absolutely not, just create a new vSwitch in VMWARE and add the physical nic to it, after that, create a new nic in the SOPHOS VM thats assignes to the new vswitch port group :-)

VMDirectPath is only needed in specialt setups ex. graphics VDI aso.

Thank you!
Will try this in my labbox first! :)

widdde said:

Okey, thanks for your response.
I know that every device in the network and devices that is carried out of the network can be potential dangerous, but somewhere we have to draw a line. This is my home network and I can't be that paranoid with phones carried in and out of the network and so on.

But I understand your point to not virtualize Sophos and it was this answer I was looking for. I think I will run UTM on dedicated hardware. Right now it is running on a box with a J1900 celeron (I think it is 2Ghz per core) and 8 GB ram. As I have understood IPS can run multiple instances of snort and when I do a benchmark test of my speed I can't get more than 150Mbps on that particular connection, but when surfing on other sites while doing a test I can get more bandwidth because another core will take care of that connection. Am i right?

It felt wrong to virtualize the firewall, but I wanted more information regarding this issue and exactly how insecure it is to expose the hypervisor. 

I have 2 servers running ESXI and both of them are behind my dedicated UTM today, and I think I will keep it that way. Maybe I use my Core I7 with yet another ESXI install and run some vm:s on it and keep my J1900 celeron box to continue to run Sophos on it (dedicated)

 

David Birdsall said:

DarrellR claims that VMWare attacks are rare.  If that is so, why is WMWare, Sophos, Symantec, Palo Alto, and so many major security vendors in a rush to ensure they have virtualization covered?  Well, it's because virtualization is the easiest and fastest way to deploy malware, specifically ransomware.  It's so easy to do.  Any Keyboard/Mouse/Video or connection can allow malware to inject into your VMWare because it connects to the VM holding the UTM, not to the UTM.  Any USB, network, or optical drive connection can also provide the same problems in the same ways.  All these vector I mentions can be in one single virtual desktop client.  Your shared resources of CPU, RAM, and storage systems NAS/SAN can also be a vector for injection.  The list goes on and on.

 

Virtualization is covered because it sells agents.  FUD sells agents.  Do you have any evidence that virtualization escapes are NOT rare?  What the hell does virtualization have to do with malware/ransomware spreading?  Where are you seeing this as an attack vector?  The MOST likely method of infection will be phishing or drive-bys on websites.  Some consumer gateways have vulnerabilities that leads to c2 infections.  But vm escapes or attacks on the hypervisor from a guest machine just don't happen.  Imagine the fallout if those were common.  Hosting companies would be shutting down en mass.   VMware would be out of business.  Amazon would have to shut down AWS.  Seriously.  This is FUD, pure and simple (at least until it is not).  For a home user, dedicating all that hardware to a UTM is a waste of resources.  They are far more likely to visit a shady p0rn site or get hit with malvertising or have an asteroid fall on their house than they are to be compromised by a vm escape attack these days.

Again, what you are recommending is solid for large business, but SMB and definitely home, this is good enough.

David Birdsall said:

DarrellR claims that VMWare attacks are rare.  If that is so, why is WMWare, Sophos, Symantec, Palo Alto, and so many major security vendors in a rush to ensure they have virtualization covered?  Well, it's because virtualization is the easiest and fastest way to deploy malware, specifically ransomware.  It's so easy to do.  Any Keyboard/Mouse/Video or connection can allow malware to inject into your VMWare because it connects to the VM holding the UTM, not to the UTM.  Any USB, network, or optical drive connection can also provide the same problems in the same ways.  All these vector I mentions can be in one single virtual desktop client.  Your shared resources of CPU, RAM, and storage systems NAS/SAN can also be a vector for injection.  The list goes on and on.

 

Virtualization is covered because it sells agents.  FUD sells agents.  Do you have any evidence that virtualization escapes are NOT rare?  What the hell does virtualization have to do with malware/ransomware spreading?  Where are you seeing this as an attack vector?  The MOST likely method of infection will be phishing or drive-bys on websites.  Some consumer gateways have vulnerabilities that leads to c2 infections.  But vm escapes or attacks on the hypervisor from a guest machine just don't happen.  Imagine the fallout if those were common.  Hosting companies would be shutting down en mass.   VMware would be out of business.  Amazon would have to shut down AWS.  Seriously.  This is FUD, pure and simple (at least until it is not).  For a home user, dedicating all that hardware to a UTM is a waste of resources.  They are far more likely to visit a shady p0rn site or get hit with malvertising or have an asteroid fall on their house than they are to be compromised by a vm escape attack these days.

Again, what you are recommending is solid for large business, but SMB and definitely home, this is good enough.

The only thing im worried about is attacks from the outside, from the net. This because the hypervisor is physically connected to the net instead of the utm.

I want a really secure lan, thats why im asking and i can of course run a separate box for utm but as said, feels like a waste.. But the security is prio one (from the net, not from lan side)

As stated, many of us run this way without issue.  Like David said, it ultimately comes down to your comfort level, but I really doubt you are harboring the droids they are looking for in your home network.

darrellr said:

Virtualization is covered because it sells agents.  FUD sells agents.  Do you have any evidence that virtualization escapes are NOT rare?  What the hell does virtualization have to do with malware/ransomware spreading?  Where are you seeing this as an attack vector?  The MOST likely method of infection will be phishing or drive-bys on websites.  Some consumer gateways have vulnerabilities that leads to c2 infections.  But vm escapes or attacks on the hypervisor from a guest machine just don't happen.  Imagine the fallout if those were common.  Hosting companies would be shutting down en mass.   VMware would be out of business.  Amazon would have to shut down AWS.  Seriously.  This is FUD, pure and simple (at least until it is not).  For a home user, dedicating all that hardware to a UTM is a waste of resources.  They are far more likely to visit a shady p0rn site or get hit with malvertising or have an asteroid fall on their house than they are to be compromised by a vm escape attack these days.

Again, what you are recommending is solid for large business, but SMB and definitely home, this is good enough.

Darrel,

You're describing the exact opposite threat of what Im describing.  Im taking about injections from peripheral device to physical host to host OS to hypervisor to VM.  Im not talking about "virtualization escape" attacks in the opposite direction.  The attack Im describing is more common in home environments than in larger settings because large organizations protect their servers directly from both physical access as well as remote access.

In your recommended setup, you're only covering a couple of network ports.  You're not covering any other port on the server.  This leaves the host server unprotected, as a whole.  The home user is much more likely to plug something into another port on the server and infect the host.  The host OS, hypervisor, vSphere, vSwitch, and other virtualization organization programs are not virtualized.  They are standard software running in the host OS.  Once the host OS or virtualization control is infected, the VMs are easy prey.  But really... all the malware would need to do in your setup is to shut down your two physical ports or shut down your hypervisor or shut down the server or encrypt it.  The malware would not need to "escape" any virtualization, and your virtualized firewall can't see it nor do anything about it.

In most virtualization today, the ports between clients and VMs are encrypted.  Your email, banking, social networking, text messaging, and many other common home communications are also encrypted between the internet and home network device.  If your virtual UTM is not scanning encryption connections, your VMs and home network are still at risk of infection.  Yes, the UTM will scan the header information of encrypted packets coming through it, but it's probably not going to flag packets coming from within the network.  How many home users are going to take the time to deal with the necessary settings and certificates to enable encrypted scanning on web, email, and other protocols?  all it would take is a home to get infected from a bad link in an email or social network post.  Then, that device can infect one or more of your VMs as well as other devices on the network.  Now, the home virtualization user has to either go and rebuild that VM form a snapshot they believe is clean or deploy another product to protect their VMs.  The home user must also deploy another product to clean and protect the devices they can access.

Data centers get hit all the time.  The reason why you don't feel it happening to you is because good data centers and virtualization designers deploy redundancy to immediately replace the current infected/corrupted VM with a clean copy running in parallel or active standby, instead of taking the time to clean the infection/corruption.  Home virtualization users don't commonly employ this simple protection.

So yes, of course, I agree that a home user deploying the hardware, software, and licensing to host a virtual UTM to protect their network is a waste.  It is simpler, cheaper, and more efficient to deploy a simple firewall appliance for under $500.

I do not feel this thread is going to help the OP any more and we are simply more likely to confuse the issue.  You and I will not come to an agreement on this.  I will consider this conversation done.

darrellr said:

I do not feel this thread is going to help the OP any more and we are simply more likely to confuse the issue.  You and I will not come to an agreement on this.  I will consider this conversation done.

Darrel,

No, of course not.  We're not going to agree.   These recent replies of yours tell me that you're just trying create an argument and win it.  You're not trying to help make sure Widde or anyone else viewing this issue can learn about options and the pros/cons/issues associated with them.

Widde understands the difference of running the UTM/Firewall in a physical firewall appliance vs virtualized.  Don't put him down or anyone else by confusing that fact with your argument.   Widde is asking to hear the advantages and disadvantages of both, so are some of the other people coming to view this issue.

You're stuck on claiming a virus isn't going to escape the VM and get out while being protected by a virtualized UTM.  I'm trying to discuss this with you to understand how the virus got on the VM in the first place while being protected by your virtual UTM, but you're avoiding that explanation.  Is it because that explanation would negate your point of a virtual UTM?

Im not here just to debate of how viruses got past your virtual UTM the first time but won't "escape" a second time in your design.  Im here discussing overall cost and maintenance over time.  Im discussing how complicated the setup and troubleshooting will be for both.  Im addressing how many products one would need in both setups.  Im also addressing what issues each stops will create or solve.

I still stand by my recommendation, and Widde understands why:  For less than $500, Sophos SG UTM or XG Firewall can be deployed in Protecli 6-port Vault firewall appliance or something similar at the "head" of the network.  The internet gateway would be connected to the default WAN port, and the internal network on the default LAN port.  If you wanted to scan everything coming in and out of servers too, they can be placed on any of the other ports.  There would be no need to purchase or maintain software and licenses for virtualization as all of the security and virtualization software can be free: Sophos SG UTM, Sophos XG Firewall, Sophos Home, Oracle VM VirtualBox.

Yes I understand both of you.

One question here: You are linking to virtualbox that runs on Windows or Mac, I am talking about VMWare ESXi that is a dedicated virtualization platform.. I would never run UTM in Virtual box in Windows.. That I see as a big security risk because all traffic will go through Windows first..

As said, I am talking about ESXi which also is free for personal use and the question regarded running UTM on Esxi.

Without wading into this too much, you will be fine on ESXi.

Think along the lines of Ip addressing and routable/non routable ip addresses.

The only entry from outside is via a router or UTM, not ESXi etc. If they compromise your UTM/router eg via poor configuration etc, effectively they could get on your internal protected resources at which time it doesn't matter whether it's virtual or not.

You will be fine with vlans or seperate nic's and virtulised. Heck, all my internet/internal traffic runs down the same port channel cable separated by vlans/ACL's. I don't use separate nic's for internet/lan etc.
Traffic coming from the internet on 8.x.x.x simply ain't going to cross over into 10.1.1.0/24 etc unless I allow it to. Now, if the UTM became compromised eg weak encrypted password for admin etc, then I may have an issue but that lies with the UTM, not the hypervisor, nic's etc

Been running like this since the dawn of ESXi 4. It's not a problem although I would advise that you do use proper seperation eg your vm's aren't on the same network as your hypervisor.

Use the usual stuff, eg strong encrypted passwords, UTM admin interface not accessible from internet, lockouts and all the functions the UTM offers etc and you will be fine.

Larger, corporate networks have their own issues eg attacks from within etc and as such need further levels of separation, resilience, monitoring etc
As mentioned, take a cost/risk based approach to it and for a SOHO, SMB, it's fine. Everybody values their data but there does come a time when the effort/cost may not be worth it.

Thank you for your involvement in this hot topic! :D

Of course I will separate utm admin on a separate vlan (my managing vlan) and never ever expose utm admin on the net side.

I have separated servers from clients as well as IOT-devices, they all have their own vlans and are routed in the UTM.
Servers can not access clients, some clients can access some servers and so on. :)

But right now the UTM is on a separate, pretty slow, box and will maybe move it to a virtualized one. Have not decided how to do yet.

I think that is what David was missing, that there is no underlying user OS like VirtualBox running under windows or linux with ESXi.  It is, and only is, the hypervisor.

On another note, though, do keep a watch on the KAISER issue with Intel CPUs.  The more I look at the code the more concerned I am getting about the possibility of vm escape.  I believe patches will be available for platforms like ESXi before public exploits are available, but if that comes to pass, be prepared to patch.  At this time, it appears that reading from ring3 (user space) to ring0 (privileged kernel space) may be possible in some scenarios.  Combined with other attacks, though could be dangerous.  NOT because you are running in a virtualized environment, though, but because of local privilege escalation.  In your case, they would need to compromise some VM on your hypervisor, then read privileged memory long enough to get meaningful information, then potentially use that information to escalate privileges (potentially up to the hypervisor).  Again, though, this is not a malware spreading issue or botnet, this is a hands on keyboard security issue that would require you to be specifically targeted and have other exploitable vulnerabilities exposed (which is kind of what a next gen UTM helps to protect).

For review:  hxxps://www.theregister.co.uk/2018/01/02/intel_cpu_design_flaw/ and other sites, shouldn't be too hard to find info.

The link you posted was very interesting! I have missed this issue during the weekends. Sounds VERY serious.

I have just ordered an AMD Ryzen 7, thank god for that ;)

But the UTM is running on a pretty old J1900 celeron and will probably be affected..?  Myabe it is better to switch to my Ubiquiti Edgerouter until this hole is fixed.?