IPsec VPN: best policy for encryption?

"Best encryption" usually means "hardest to intercept", not "best performance".   So you need to decide what you want.  Better security requires more CPU, so you need to determine if your perceived performance problem is because of a CPU limitation.

For security:

• 3DES is considered barely adequate these days.   AES-128 is a minimum and AES-256 is preferred.
• MD5 and SHA1 are considered broken, and you should use one of the SHA2 variants.

But you also need to address MTU.   When a packet enters a VPN tunnel, it receives an envelope that makes the packet bigger.   If the new packet is too big for the outbound interface, it has to be fragmented.   Fragmentation costs you CPU overhead at both ends, consumes bandwidth because of extra packet overhead, and introduces latency,   To prevent this problem, you need to change the MTU on the INSIDE interfaces, so that the maximum packet size plus the VPN overhead still fits in one outbound packet.

 To test this, try ping.   In Windows syntax the option -l nnnn says to use packets of size nnnn.   -f says to prevent packet fragmentation -- if the packet is too big, the ping will fail.   Find the largest value for -l that does not fail, and set your inside MTU to not more than that value.   I think some of the overhead is variable, so a value somewhat less than the maximum is probably wise.

Another option is to switch from TCP to UPD.   TCP guarantees error-free lossless communication, but adds bandwidth and latency to do so.   UTM doesn't care if an occasional packet gets lost, so it is faster because of less overhead and less latency.   Streaming services work best on UDP.   I use TCP, although for an RDP session, an occasional dropped packet might be as inconsequential as a screen blip or jumpy mouse movements.

Ciao Marco,

Agreed with Doug, but I prefer AES 128 PFS to 256 as I read somewhere 1-to-2 years ago that there's a vulnerability in AES 256 - I don't remember where I saw that, so if anyone else can point us at that or at another article that debunks the one I saw, I'd be glad to hear of it.

3DES is not only not that secure, but it's also more resource intensive than AES.

Depending on the processors, you might be able to use Intel's hardware acceleration by selecting AES 128 GCM:

Cheers - Bob

I did not mention the PFS setting in my first post, but it is also important.

My current understanding is that DH groups 1-5 are considered broken, I think because of increases in processor speed for brute force attacks.   So choose DH 14 or higher.