EdgeRouter X SFP IPSec/LT2P VPN "Remote server not responding..."

VPN
1

Hello all,

Background: I am a part-time administrator for IT at a small business. Currently off-site and can’t be back on-site for several months. I’m generally tech savvy but not an IT professional (not my full time job).

Config: Ubiquiti EdgeRouter X SFP, configured it to host an IPSec/LT2P VPN with pre-shared key for our remote employees to use for remote access to our on-site servers and equipment. We use the built-in VPN client in Windows 10/11.

Problem: One of our employees started seeing an error when attempting to connect (on Windows 10). Paraphrasing: “Network between your PC and VPN server couldn’t be established because the server isn’t responding…”, with the remainder of the message suggesting possible causes (firewalls, NAT, router, etc.).

The employee experienced this problem for a week while we went back and fourth troubleshooting. My VPN access worked fine during this time, but after a week also failed in the same way (Windows 11).

Obviously these symptoms are vague, but does anyone have suggestions for what could cause this? Everything has worked fine for 6+ months, and no changes have been made to the router or network config recently.

2

Hello! Thanks for posting on r/Ubiquiti!

This subreddit is here to provide unofficial technical support to people who use or want to dive into the world of Ubiquiti products. If you haven’t already been descriptive in your post, please take the time to edit it and add as many useful details as you can.

Please read and understand the rules in the sidebar, as posts and comments that violate them will be removed. Please put all off topic posts in the weekly off topic thread that is stickied to the top of the subreddit.

If you see people spreading misinformation, trying to mislead others, or other inappropriate behavior, please report it!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

3

Might be helpful to get to the ERX logs during that timeframe. Should give you more info.

Are you using static IP? Or dynamic DNS? Could it have been an issue where the IP changed? Are there other services you can test to ensure the ERX is responding (ping, ssh, web)?

I haven’t played with ERX in a while; do you have individual users setup for L2TP? Is there a max concurrent connections limit?

Also IPsec can be tricky depending on where you are connecting from. Some home routers or hotspots may not pass IPsec through and the client will try over TCP using NAT traversal. There should be a setting for that on the ERX you can try.

I saw this as well, explains how to log from swan

https://community.ui.com/questions/ERX-setting-up-a-L2TP-VPN-The-L2TP-VPN-server-did-not-respond/c5048354-2547-4f01-a8c5-08b181da397d

4

Did your ISP switch to CGNAT?
May not match the timing of observations if you could get in after the first person had an issue.

5

Thanks for the reply!

I did think about grabbing the ERX logs, but didn’t go through with it before I lost access via the VPN. Planning to get myself into the network tomorrow via another method in order to debug, and this will be my first step.

The ISP-assigned address is dynamic, but I have the ERX setup with NOIP for DDNS. This was one of the first things I checked because we’ve had issues with it before. The DDNS hostname and external IP still match and neither work.

I figured it was a problem with the initial employee’s network or router messing with the VPN connection, but after it stopped working for me, this just didn’t seem as plausible. My home network config hasn’t changed in months. I also tried to connect through the hotspot on my phone with the same result.

I will investigate the link that you sent when time permits. Thanks!

6

Out of curiosity, do you think that error applies to only network-related issues, or might it also correspond to authentication issues? Part of the process of adding the VPN to the built in VPN client in Windows is running the following command in PowerShell:

Set-VpnConnection -Name "Company Name" -AuthenticationMethod MSCHAPv2,CHAP

I learned that this had to be done in order for the VPN to work upon initial setup. Something to do with Microsoft not allowing the correct security protocol by default, so this command changes it. I get a different authentication error when the command isn’t run, but wondering if a Windows update perchance changed how the Windows VPN client works and is causing the error.

7

I ran the command

sudo swanctl --log

and then attempted to connect to the VPN. I saw in the output that a packet was received from my external IP (yay!), so this seems to eliminate the possibility that the issue exists anywhere but within the ERX.

I’m going to look over the error logs, but I don’t understand a majority of what I am seeing. I’d like to post them here, but am obviously not going to do that without first removing personal details. Suggestions welcome.

8

Not aware of what CGNAT is, but I did (and possibly still do) suspect an ISP blocking issue. Here’s what makes this more weird too. My boss just tried the VPN and it’s working fine for him. Still doesn’t work for me and the original person.

9

Verify NOIP is still updating correctly then dig into those logs. Should give you some clues. Good luck.

10

Like you said no config changed. I’m not aware of any updates that would change that, I use it weekly for a very similar setup.

With windows 10 they obscured some of those advanced settings, hence you using PowerShell to tweak the connection. You can still get to those settings if you want to check them.

Open the network and internet settings, go to the VPN tab and click “change adapter options” in the top right corner. You’ll see your VPN interface there. Look at the properties>security tab.

The only thing checked on mine is CHAP version 2… but I’m connecting to a USG so I’m not sure which option is necessary for ERX.

11

Setting ms chap fixed mine. You can go into network adapters, right click go into properties and click a mschap checkbook.

12

Nice. Also What do you see in system logs?

cat /var/log/messages | grep ‘ipsec|l2tp’

Post what you can with redacted public IP, usernames and passwords or keys.

13

Copy that. I’ve already bypassed NOIP in my debugging by just using the known-good external IP.

Will update with whatever I find in the ERX logs tomorrow if I can.

14

Okay, that’s what I figured.

When I first deployed the VPN, I was manually changing those settings in the control panel to get it working, but needed to streamline the setup process to document it for less tech-savvy users. I found that I could run that PowerShell command to make it work, which is a lot easier than clicking around in menus.

15

Thanks for the suggestion. Unfortunately MS CHAP is already checked in order for the VPN to work. And unchecking it does not fix the issue, which appears to be on the ERX end.

16

Doing the cat command you mentioned doesn’t bring up any results. I’ll have to dig into that one. I did look at the output for /var/log/messages and didn’t see any recent entries or anything that changed when I tried to connect my VPN.

Here is the complete output of the swanctl command, with hopefully everything important removed. This is everything printed from pressing ‘Connect’ on my PC to the connection timing out on my PC.

05[NET] received packet: from <my-ip>[500] to <vpnserver-ip>[500] (408 bytes)

05[ENC] parsed ID_PROT request 0 [ SA V V V V V V V V ]

05[ENC] received unknown vendor ID: <vendor-id>

05[IKE] received MS NT5 ISAKMPOAKLEY vendor ID

05[IKE] received NAT-T (RFC 3947) vendor ID

05[IKE] received draft-ietf-ipsec-nat-t-ike-02\n vendor ID

05[IKE] received FRAGMENTATION vendor ID

05[ENC] received unknown vendor ID: <vendor-id>

05[ENC] received unknown vendor ID: <vendor-id>

05[ENC] received unknown vendor ID: <vendor-id>

05[IKE] <my-ip> is initiating a Main Mode IKE_SA

05[ENC] generating ID_PROT response 0 [ SA V V V ]

05[NET] sending packet: from <vpnserver-ip>[500] to <my-ip>[500] (136 bytes)

10[NET] received packet: from <my-ip>[500] to <vpnserver-ip>[500] (228 bytes)

10[ENC] parsed ID_PROT request 0 [ KE No NAT-D NAT-D ]

10[IKE] remote host is behind NAT

10[ENC] generating ID_PROT response 0 [ KE No NAT-D NAT-D ]

10[NET] sending packet: from <vpnserver-ip>[500] to <my-ip>[500] (212 bytes)

15[NET] received packet: from <my-ip>[4500] to <vpnserver-ip>[4500] (76 bytes)

15[ENC] parsed ID_PROT request 0 [ ID HASH ]

15[CFG] looking for pre-shared key peer configs matching <vpnserver-ip>...<my-ip>[192.168.0.125]

15[CFG] selected peer config "remote-access"

15[IKE] IKE_SA remote-access[129] established between <vpnserver-ip>[<vpnserver-ip>]...<my-ip>[192.168.0.125]

15[IKE] DPD not supported by peer, disabled

15[ENC] generating ID_PROT response 0 [ ID HASH ]

15[NET] sending packet: from <vpnserver-ip>[4500] to <my-ip>[4500] (76 bytes)

12[NET] received packet: from <my-ip>[4500] to <vpnserver-ip>[4500] (444 bytes)

12[ENC] parsed QUICK_MODE request 1 [ HASH SA No ID ID NAT-OA NAT-OA ]

12[IKE] received 3600s lifetime, configured 0s

12[IKE] received 250000000 lifebytes, configured 0

12[ENC] generating QUICK_MODE response 1 [ HASH SA No ID ID NAT-OA NAT-OA ]

12[NET] sending packet: from <vpnserver-ip>[4500] to <my-ip>[4500] (204 bytes)

02[NET] received packet: from <my-ip>[4500] to <vpnserver-ip>[4500] (60 bytes)

02[ENC] parsed QUICK_MODE request 1 [ HASH ]

02[CFG] unable to install policy <vpnserver-ip>/32[udp/l2f] === <my-ip>/32[udp/l2f] out (mark 0/0x00000000) for reqid 61, the same policy for reqid 18 exists

02[CFG] unable to install policy <my-ip>/32[udp/l2f] === <vpnserver-ip>/32[udp/l2f] in (mark 0/0x00000000) for reqid 61, the same policy for reqid 18 exists

02[CFG] unable to install policy <vpnserver-ip>/32[udp/l2f] === <my-ip>/32[udp/l2f] out (mark 0/0x00000000) for reqid 61, the same policy for reqid 18 exists

02[CFG] unable to install policy <my-ip>/32[udp/l2f] === <vpnserver-ip>/32[udp/l2f] in (mark 0/0x00000000) for reqid 61, the same policy for reqid 18 exists

02[IKE] unable to install IPsec policies (SPD) in kernel

02[KNL] deleting policy <vpnserver-ip>/32[udp/l2f] === <my-ip>/32[udp/l2f] out failed, not found

02[KNL] deleting policy <my-ip>/32[udp/l2f] === <vpnserver-ip>/32[udp/l2f] in failed, not found

02[KNL] deleting policy <vpnserver-ip>/32[udp/l2f] === <my-ip>/32[udp/l2f] out failed, not found

02[KNL] deleting policy <my-ip>/32[udp/l2f] === <vpnserver-ip>/32[udp/l2f] in failed, not found

02[IKE] sending DELETE for ESP CHILD_SA with SPI 0377937e

02[ENC] generating INFORMATIONAL_V1 request 4081609149 [ HASH D ]

02[NET] sending packet: from <vpnserver-ip>[4500] to <my-ip>[4500] (76 bytes)

13[NET] received packet: from <my-ip>[4500] to <vpnserver-ip>[4500] (76 bytes)

13[ENC] parsed INFORMATIONAL_V1 request 918370530 [ HASH D ]

13[IKE] received DELETE for ESP CHILD_SA with SPI 0377937e

13[IKE] CHILD_SA not found, ignored

10[NET] received packet: from <my-ip>[4500] to <vpnserver-ip>[4500] (92 bytes)

10[ENC] parsed INFORMATIONAL_V1 request 3801245624 [ HASH D ]

10[IKE] received DELETE for IKE_SA remote-access[129]

10[IKE] deleting IKE_SA remote-access[129] between <vpnserver-ip>[<vpnserver-ip>]...<my-ip>[192.168.0.125]

Sheesh that’s long…sorry for the spam.

17

Ahhh nice hack. Ok just wanted to make sure you knew where to look.

18

Reboot the edge router. See if it works after.

Then update the UI firmware. I suspect the included strongswan is an old version.

Found a thread detailing similar issue with a bug fix in strongswan version 5.5.0.

Seems the policies are not getting cleared and conflicting? We’ll see.

19

I rebooted the ERX outside of business hours and now my VPN connection is working. I will grab logs of a connect/disconnect with it working. Have not had the original person test it. I do not trust that this won’t happen again. Have yet to update any firmware due to general wariness around updating things and the risk of causing more issues. Will update soon.