Site to Site VPN With Customers

Those of you that leverage site to site VPNs for supporting customers:

-What is your deployment model?

-How do you handle customers that do not agree to the security mechanisms or routing that you have in place?

-How do you handle IP address overlap and associated NAT policy when public IP addresses cannot be used inside the tunnel?

-Do you keep this on prem or cloud?

Ideally the solution is no site to site VPN with customers and a more modern solution in favor of a remote access tool on each device that is supported is implemented, but not all customers allow server to internet directly for tunneling apps, mostly for the right reasons.

EDIT: More context. We are a healthcare IT provider of software and services to hospitals where the customer or we deploy virtual and/or physical VMs/Servers into their network. We use a variety of remote access solutions including site to site VPN. We access our customers via the standard Linux and Windows remote access ports (RDP, SSH).

No matter what comments you get on this … please remind the remote site to WRITE MEM.

I don’t know how vpns we had at the hospital go down because of this stupid command.

I wrestled with this for years and the simplest solution I came to was to supply them with a CPE managed by us. Then just have them point a route on their firewall at your box. I usually then point a default route back at them and specifically route anything back to me, ideally via BGP. The NAT part isn’t necessary if you have public IP space you can use for customer facing services to avoid address overlap. Any outbound NAT that’s needed is all handled by me.

That way I own the kit on both ends of the VPN. So I know it’s patched, I know the crypto is secure, if I need to change the peer address or something, I have the ability to do that myself I don’t have to engage 100s of customers with varying kit and tech abilities who will guaranteed find a way to break things any time something needs changing.

Our jump boxes selectively have routes via single customer DMZs then across a tunnel- it’s pretty restricted to icmp, ssh, esp and http/ssl though bad things could tunnel I suppose- enough to get to a jump box or network device at the customer though.

There is a lot of NAT at play.
Edit to add we need to two factor to get to our jump boxes too - last thing we want is to be the weakest link.

I try to be as friendly to our customers as I can:

  • I don’t need admin on your domain. I don’t generally need ANY login on your assets.

  • I don’t really need remote access. I just need to be able to exchange data on a couple of ports. You are absolutely right to want to limit my access.

  • I don’t need SMB, SSH, Netbios, etc. There’s an optional product that I could use RDP to support, but if you don’t want that, I’m OK. I’ll need a phone number of someone I can talk through troubleshooting, but I promise it won’t be often. That’s probably easier for me anyway. The usual fix is just a reboot.

  • I kind of like ping / ICMP echo open, but if you don’t want that, fine.

  • I’ve already NATed my side into a routable /28. If you prefer I go smaller to just a couple of hosts, OK.

  • I usually just need a half dozen hosts on your side. I don’t want your side to be a full /24 (Class C). I really don’t want a /16 (Class B). I really, really don’t want these to be RFC 1918 ranges.

  • I hope you have some kind of process for patching your systems. Patches and updates are highly unlikely to affect my stuff. Please do these at your convenience and don’t even bother to ask me. I don’t like vendors who forbid people to patch, so I’m not going to be that guy.

Happened across your post just now. I am a dev on an opensource project that almost certainly can do the things you want/need… Dunno if you are interested in a response from some dev of some random opensource project or not but it’s probably able to solve what you need.

Deployment model - simple binary you ask the customer to run or send them an enrollment token that ends up making a strong identity for them. They can then tell you what services they want you to access and you configure them through your own centralized controller. Clients for all major OS - windows/mac are ‘easiest’ but linux is pretty simple if you’re familiar. A rendezvous device is needed - often it’s something out on the open internet but you can keep it totally on-net if you are. Sounds like you want one on the internet though from your post.

I can’t speak to when customers won’t agree to your security mechanisms - seems like a non-starter to me? :slight_smile:

IP address hasn’t been an issue. You can “effectively” nat to whatever you need. You can use an IP address space or just make up your own DNS names (the clients are basically private, authenticated DNS). You can send traffic from your local machine and it offloads from within the target network. Often i run the agent right on the target machine and only need to trust the host OS only. You can intercept public ip addresses if you want with this solution too but it’s usually a bad idea. if you know what you’re doing though - it works. big post about it if interested here

On prem or on cloud is up to you. Works best when you treat EVERY network like it’s hostile and just pretend you’re on the open internet anywhere you go but if you want to trust a local network - well you can…

This is not a site to site vpn. It’s a zero trust overlay (i know - zero trust such marketing BS right? lol)… It’s modern, centralized access control, mtls between all network nodes (before leaving the overlay), e2e by default (chacha20/poly-1305 - libsodium), continous auth capabilities, mfa on windows (currently only really on windows). All you need outbound internet and you have an incredibly secure overlay. it’s literally designed to act like it’s on an open, unsecure network but I can understand if some of your clients won’t allow it out of the gate. They can always turn off your access by just stopping the local entry point if they ever get squirrely. All they need is outbound to one or two addresses, port 80/443 and you can do all this.

Anyway, I don’t know if this post will get downvoted / removed. Since it’s all open source and FOSS and seems to be a solution to what you’re looking for I thought it’d be fine and to post this and it seems like it would do what you want so i took a chance and wrote all this up… It’s https://openziti.github.io/ if you want to check the project out. I dropped a link to the discourse post if you want to ask other questions there/here. I made a subreddit for it but - it’s pretty barren still. You could ask there too :slight_smile:

I hope this is helpful and welcomed - if not well dang… I’m sorry :confused:

This sounds like you’re asking as an MSP that wants to manage or monitor remote systems and services on customer networks. Ideal solutions would have something onsite with a reporting tool that restricts access to just the services/systems that need to be monitored and only reports data out to specific collection servers over specific secure ports. This usually works for customers that don’t like remote access to their network.

If you are tunneling/adding remote access to healthcare vendors what sort of HIPAA regulations do you face? Serious question, working for a MSSP that does SMB and Ent and we just landed our first Healthcare provider. We are worried about tying anything of ours to them with bringing on greater legal liability.

This is so true and the most ideal solution. While I wished this was possible with our team, the hospitals that we deal with are very picky about having their own solution in place and it’s a battle between winning a deal and doing it right and we all know how that works.

What solution are you using for 2FA/MFA on your jump boxes?

I can’t speak to all of the intricacies with HIPAA as that gets into legal and compliance with variability on a per state basis, however there are great distinctions between taking in PHI vs. accessing PHI. Also, it depends if you are a covered entity vs. business associate. We remotely access the customers site as to not have to worry about taking data on site and dealing with the “data at rest” portions of HIPAA. We fall under the business associate portion.

At least you’re using VPN. I moved from healthcare to Media and all their vendors want to use teamviewer as a support tool…

right, i figured/hoped you were only in the “access” portion and not rest cuz f that. so even if we are the hardware provider/configuration we still are only associate even with a “permanent” connection in the way of VPN or Teamviewer host or whathaveyou

Edit: depending on local jurisdiction that i should check

teamviewer has it’s place…in a dumpster.