I’ve always run a VPN server at home for as long as I can remember, so that both my laptop and phone can connect to home when necessary. Many, many, many years ago I’d switched from PPTP to SSTP VPNs for my laptop, as getting a clean HTTPS connection out of a corporate firewall was quite straight forward, where as PPTP was generally blocked. For the iPhone/iPad though, I’d switched them to L2TP/IPSec. This was working quite well until a few major releases ago, Apple changed something in the negotiation part of establishing an L2TP VPN connection.

More specifically, they changed the ciphers used in both the IKE phase 1 (ISAKMP) and IKE phase 2 (IPSec) security associations. When two hosts attempt to form a security association, they first negotiate a main mode (phase 1) security association, and then a quick mode security association (phase 2) for the actual data encryption). There’s a huge amount of info around explaining how that works.

Anyway, that wasn’t working anymore. I spent many nights trying to resolve the issue and ultimately gave up – it’s not like L2TP is the future of VPN protocols anyway. So instead I decided to move it all to IKEv2, which was only slightly less painful to configure.

Just on the L2TP topic though. Routing and Remote Access (the Windows Server VPN product), and Windows in general to be honest, are both really unhelpful when it comes to IPSec configuration and debugging. With logging, both file and audit, registry settings, and PowerShell, I was ultimately not able to get the iPhone and RRAS happily talking to each other. I was able to get a Main Mode security association formed – i.e. they agreed on common ciphers/parameters during the ISAKMP negotiation, but then they were unable to get through Phase 2. In an attempt to see more debugging info (I was already looking at packet captures of the exchange), I ran up a Cisco Catalyst 8000V virtual router on a KVM server and pointed the VPN traffic towards that (in this case UDP 500 (IKE) and UDP 4500 (NAT-T)). I configured the Cisco VPN Server, and with the trusty debug crypto isa sa and debug crypto ipsec sa commands, was able to see the exchange in more detail.

Interestingly, I still couldn’t get it to work even with a Cisco router. I did get more insight into the failure though. The router was ultimately finding a match of ciphers and parameters, but was then reporting an invalid set of flags being received from the phone. Given the phone’s behaviour was largely out of my control, it was time to park that issue and find an alternative. There may have been more I could do, but I’d largely lost interest at this point and L2TP wasn’t really the requirement, just a secure VPN.

So – the alternative – I finally switched the configuration across to IKEv2 tunnels. There was again many a packet capture and log reviewed to get this working, so I’ll save you the detail and just give you the working configuration.

Steps:

I used a Windows 2022 server running RRAS and NPS. RRAS was configured in a more or less standard manner – IKEv2 dial-in ports, no pre-shared key set, and a DHCP relay agent configured to point to the DHCP server on my LAN.

My local Certificate Authority (CA) had issued a certificate to the VPN server for the external FQDN (host/domain name) that I would use to access the VPN server, and in addition to the Server Authentication Extended Key Usage, I also had the IKE intermediate key usage applied. When RRAS starts, it looks for a certificate with the IKE intermediate key usage to use for IKEv2.

The NPS server was configured with a network policy that provided PEAP and Smart Card/Certificate authentication. They were both configured to validate against my CA certificate. I issued a user certificate to my dial-in user, and loaded both the CA certificate, and the user certificate onto the iPhone.

I lowered the MTU on the RRAS server external interface to accommodate the ESP header used in the IPSec traffic.

I modified the ciphers used by the RRAS server to be a set of ciphers with which the iPhone would be satisfied.

I added UDP 4500 to the Windows Firewall (NAT-T), as my VPN server was behind a NAT device. That also needed a registry setting to enable NAT-T communications.

I configured the IKEv2 VPN profile on the iPhone to use the external FQDN of the VPN server (the one in the certificate for both the VPN serrver and the remote ID fields of the VPN. The local ID was set to the UPN of the authenticating user, and the authentication was set to certificate using that user’s certificate.

That was it – very little needed to configure, but an epic pain trying to work it out.

Set-VpnServerConfiguration -TunnelType ikev2 -CustomPolicy -AuthenticationTransformConstants SHA256128 -CipherTransformConstants AES256 -DHGroup Group14 -EncryptionMethod AES256 -IntegrityCheckMethod SHA256 -PFSGroup None -SALifeTimeSeconds 3600 -SADataSizeForRenegotiation 102400

New-ItemProperty -Path HKLM:\SYSTEM\CurrentControlSet\Services\PolicyAgent -Name AssumeUDPEncapsulationContextOnSendRule -Value 2 -PropertyType DWORD

netsh int ipv4 set subint "Ethernet" mtu=1350 store=persistent

Let me know if you get stuck, but that’s all you need.

~ Mike