Beginner guide: How to secure your self-hosted services

VPN
1

Hi guys,

I decided to write this little guide following a bunch of posts about people having their things published without any form of protection on the web.

​

I hope this helps many gain a little insight in to what they’re actually doing.

Note: This will be a work-in-progress at first. Any feedback is welcome!

Important: This guide is aimed at beginners, so I won’t go too much in-depth and mostly rely on common sense and (fairly) easy to implement solutions. I will make a more advanced guide later on.

​

READ ME FIRST:

Holy shit this thing blew up in less then a day.

Upon multiple requests this guide will be continued on github and I will update Github changes here on a regular basis. Please see https://github.com/justSem/r-selfhosted-security/tree/main/beginners-guide

Contributors are welcome! Please send a PM if you wish to do so

​

First: What’s going on?

Recently posts have been showing up about people finding others’ exposed dashboards or even fully unprotected services such as Heimdall, Pihole, Calibre, you name it. People expose it all on the public web, often without even knowing they’re doing so.

To some this might seem innocent, but it’s not. Even if you’re not a specific target to anyone, there a lots of automated bots and botnets out there who just scan the entire internet for exposed services like yours in order to exploit those.

​

So what are the dangers of this exactly?

Those services you’re hosting are exposing a lot of your private info. I’ll list a few examples of things I come across.

  • I once came across a fully open Calibre instance, upon browsing through it I found out that this particular person configured Calibres mail settings using their GMail details, just a little tinkering exposed their full GMail username and password
  • People tend to use their full names, or even full address info, etc. in things like Nextcloud, maybe even things like Pihole or Heimdall. This will make you a target for (automated) phishing campaigns. If those services are publicly accessible you can easily assume that someone has already got his hands on your info.

So this all might seem innocuous to some, or some might even utter the: But I have nothing to hide - kind of phrase. But think about why most people are self-hosting in the first place. Privacy is most likely a big part of that, and now you’re putting that out on the web for everyone to see?

In example: Big data, botnets, hackers, etc. can build an extensive profile based on this kind of info:

  • One could sift through your Calibre service to find out what things you read.
  • One could sift through your Pihole logs to find out what you do on the web.
  • One could search through your Plex, Jellyfin, or others to find out what things you like to watch.

This kind of info is especially useful for things like Phishing campaigns. The more familiar and polished a phishing mail is, the more likely you’ll fall for it. And you will be targeted. No-one’s exempt.

​

Another danger is the case where people have a set-and-forget mentality, which leads them to never updating their services. In that case your service will get hacked at some point which might result in anything from your device being abused as cryptominer, to your connection being abused for malicious traffic, your devices being enslaved into a botnet or an actual human hacker who might have even more sinister intents.

​

How do I know if I’m publicly exposing services?

There are a few indicators which will easily tell you:

  • Did you ever follow a guide that told you to port-forward something?
  • Do you proxy or forward your services using a reverse proxy? (i.e. Nginx proxy manager)
  • Can you access your services from anywhere (i.e. from your phone) without any extra effort like a VPN.

​

I’m not sure, how do I check?

There are plenty of tools that will freely tell you if you’re hosting something. First you’ll need to know your public IP. Some site like https://whatismyipaddress.com/ will tell you.

Please realise you might have a number of different IP addresses dependent on if your provider provides you with both IPv4 and/or IPv6. Your public IPv4 address will be the same for all devices in your network, but your IPv6 address will be different per device!

The following tools might give you an insight in the ports you have opened publicly:

  • Shodan https://shodan.io - Shodan does it’s own scanning but will not per-say reveal everything as it does not tend to scan every single open port at any given time. Some IP addresses might not even be listed in Shodan.
  • Yougetsignal https://www.yougetsignal.com/tools/open-ports/ - Chances are that if you’ve been port forwarding you’ve been using a tool like this to actually verify if the port you’ve configured is accessible.

​

I’m still unsure and I want to scan it all, how do I do that?

This section is slightly more advanced, but if you can selfhost then you can do this too!

First you’ll need a device that does not host any of your services and a different internet connection. (Your phone’s 4G or a neighbours WiFi will do).

You’ll need a port scanning tool, in this case I’ll use nmap which is available for practically all linux distributions, macOS and Windows.

If you’re using Windows you can download nmap here: https://nmap.org/download.html

If you’re using a Debian based distro (Debian, Ubuntu, Mint, etc.) you can install nmap using sudo apt install nmap

If you’re using a Redhat based distro (Redhat, Fedora, CentOS, etc.) you can install nmap using sudo dnf install nmap

If you’re using macOS you can install nmap using Homebrew ( https://brew.sh ) by issuing brew install nmap

One you’ve got nmap setup, make sure you’re using a different internet connection and then issue:

nmap -v -T4 -sV -A -p 1-65535 my.public.ip.address

This will take a while as it’ll scan all available TCP ports. It’ll also try to determine what’s running on an open port it finds (-sV flag) as well as some additional detection (-A flag)

​

Okay, so I do got open ports, what do I do?

Firstly, you’ll have to close them. It’s most likely that you’ll do this in your router. If you’re unsure then I’d suggest you check the guide that you used to setup your service in order to determine what steps you took to expose it to the internet in the first place.

​

So now my ports are closed, but I can’t access service xyz from remote anymore. What do I do?

It’s understandable you want to access your services from anywhere, but there are more secure methods for this then simply exposing this.

There are a number of steps you can take which’ll be listed in order from most secure to least.

  • Use a VPN
    • Setting up a VPN like Wireguard is easy and secure. WireGuard has support for all major devices and it’ll allow you to access your entire network from anywhere.
    • Sidenote: You’ll have to port forward WireGuard from your router, this is to be expected. But exposing a VPN service to the public internet is way more secure then exposing an unsecured service.
  • Use port-forwarding with specific IPs
    • This is a feature some routers might not support. But you can utilize a whitelist of IPs that can access your service.
  • Using Cloudflare’sArgo tunnel
    • By using Cloudflare’s Argo tunnel you don’t have to open any ports, but instead your webserver will build up a vpn-like connection to cloudflare, over which your webserver will be reachable to cloudflare. Your users then access your service through cloudflare without any risk for you due to exposed ports.
  • Utilizing a security CDN like CloudFlare
    • Using services like CloudFlare prevents an attacker from learning your actual IP address (unless said IP address can be accessed somehow through your service of course). Additionally CloudFlare actively filters out bots and malicious traffic. Depending on your tier with them you have more granular control and can choose to block entire countries from accessing your site.
  • Use a reverse proxy with an authentication frontend
    • One could utilize a platform like Authelia or Keycloak to secure public-facing services.
  • Use a reverse proxy and utilize access-lists
    • A thing one could do with a reverse proxy like nginx is the usage of access lists. By using the allow directive in the nginx config you can restrict entire services or subfolders to specific IP addresses.

​

I’ve read this all, but I still keep wanting to do the things I do. Any tips?

  • Be aware of what info you expose using the services you expose to the internet.
  • CHANGE DEFAULT PASSWORDS! This cannot be said enough, exposing services is one thing, but not changing passwords is like giving out your credit card to complete strangers and hoping they’ll bring it back to you.

​

General recommendations

These might be duplicates of parts above, but it’s useful to sum them up:

  1. Expose only what’s really needed: Why would your service need to be open to the internet?
  2. Change default passwords: You don’t give your credit card to strangers either, do you?
  3. Use common sense: You can’t magically access something you host at home without exposing something to the public internet.
  4. Use 2FA wherever you can. Any form of 2FA is better then nothing. Most services support OTP (Google Authenticator/Authy/Yubico Auth) these days and the more advanced ones even support Webauthn (Yubikeys or any other hardware token)

​

To-do parts:

  • !Extend on how-tos in building Wireguard, Nginx and NAT access lists!<

​

Changelog:

  • !Added Clouflare’s Argo Tunnel!<

  • !Added 2FA and Cloudflare; Clarified requirement for separate connection for nmap.!<

  • !Initial guide!<

2

Very good article!

Am I correct that securing a Nextcloud instance with Wireguard or Authelia would prevent the use of sharing files with friends, or is there a way around that?

3

Good article. Just two thoughts - doing nmap scanning on the WAN interface from the LAN side might give inconsistent results depending on the firewall implementation. Best to scan from the outside. IMO, using 2FA on exposed services should be enough to negate the need to use VPNs or front-end authentication.

Also - best to keep all self-hosted software up to date and don’t use any software that is no longer maintained.

4

Another important point is to monitor that the security measures you’ve put in place are actually working. Set up alerts to notify you of failures. An example: I once set up fail2ban to monitor a bunch of services and found that it would stop working after about a day. As it turned out, one of the services was deleting its log file at some semi-regular interval and not replacing it with a new log right away. Fail2ban complained that the log file was not found, and stopped working - not just for that one service but for all the others too. If I hadn’t been monitoring it, I wouldn’t have known. Sure it worked when I tested it at the time of setting it up, but it’s important to make sure it continues to work.

5

One more thing: use badass passwords. Bitwarden is your friend :wink:

6

Nice write up and I think you’re off to a good start! Bookmarked it.

Been planning to do something similar concerning networking and os setup slowly moving up to the service components. Are you planning to cover that as well?

7

Just curious, what is everyone’s consensus on just using in service 2FA, like Nextcloud’s built in 2FA using google authenticate?

Nice article.

8

For those people who just blatantly open ports and have no idea the consequences, I feel this guide is too general/hard for them.

I have a colleague I sent this because I can’t be bothered to try and secure his personal shit and it’s above his understanding.

Now, should these people not be self hosting ? Yes. However I feel the guide will need to be expanded as the OP described before it can help some other less technical inept.

9

Good start, but I’d also recommend adding something about HTTP vs HTTPS for services exposed using HTTP. I understand you mentioned reverse proxy, 2FA, and authentication middleware but if it is all sent in plain text it’s not that much better than nothing - perhaps if you have 2FA with protection against replays, you are ok, but you’d still be giving people your username/pass.

10

I feel like UPnP on the router deserves an honorable mention here. Allowing devices to open ports on your router seems like a particularly bad idea. Very nice summary!

11

Hidden Services with authenticated clients are also a great option, especially in I2P where you can hide the existence of the hidden service behind an Encrypted LeaseSets so only people with the key can discover the real LeaseSet. Not only is there no open port, for everyone but authorized clients, there isn’t even an address, and it’s no more difficult(sometimes easier) than the methods you described in your post.

Disclaimer: I am an I2P developer.

12

Great article! One though, core to securing a self hosted environment is to secure the system doing said hosting. Shameless plug to a guide I put together: GitHub - imthenachoman/How-To-Secure-A-Linux-Server: An evolving how-to guide for securing a Linux server..

13

access your service through cloudflare [Argo] without any risk for you due to exposed ports

Unfamiliar with cloudflare argo but I was curious how it would magically protect you from any and all attacks. Their documentation says:

You can think of Argo Tunnel as a virtual P.O. box. It lets someone send you packets without knowing your real address.

Having a PO box does not prevent anyone from sending you anthrax, it just means they don’t know where you live. Which is great if you are where you live and you don’t want that anthrax person to walk up to your house, but in IP space, it doesn’t as easily translate to a physical address and via this tunnel they can still directly send traffic to the service so I do not see any security advantage whatsoever. Host a vulnerable application (DVWA for example) via this tunnel and see if you can still exploit it.

Hiding your IP is also completely useless because people scan the whole public IP space many times a day. (I.e. that anthrax sender is already knocking on your door, they don’t need to find you specifically they just check everyone’s doors.) If you host something vulnerable, it will be found that way no matter if you “hide your IP”. Hiding your IP is useful if you want law enforcement to have to get a warrant for Cloudflare before they get a warrant for your ISP, or if you are a DDoS target, which if you self-host regular personal projects, you generally are neither. (And note that DoS still passes through this tunnel and e.g. Nextcloud categorically ignores DoS, including pre-auth; it’s definitely not an uptime guarantee.)

Also, running more software is more attack surface as people like to point out in this thread. Before, you were running nginx; now, you are running nginx and Cloudflare’s closed-source software. If you get real benefit from it, sure why not? But I really don’t see this as a good trade-off. Also, you’re leaking all your metadata to Cloudflare now.

Finally, another point Cloudflare makes in the post you linked:

traffic through Argo Tunnel gets a performance boost

This is certifiably rubbish. Before, your packets would go directly from you to your service. If it’s on your LAN, it would be microseconds; if it was in the same geographical region, it would be milliseconds. Now, it needs to go to Cloudflare first before it can go anywhere else. Even if they brought this overhead down to a handful of milliseconds by being geographically almost everywhere so the extra leg is short, it’s never going to have a negative ping time.

14

But exposing a VPN service to the public internet is way more secure then exposing an unsecured service.

I don’t get this.

How is a VPN service that is exposed to the Internet more secure than a webserver?

I understand there is a chance of zero-day attack or similar vulnerability in the code, but surely that applies equally to VPN software as well? If I expose a presumably well-tested, well-developed service, use HTTPS where applicable and don’t use default passwords, what’s the difference?

15

I haven’t seen anyone mention https://tailscale.com. It’s the easiest way that I’ve found to set up a VPN, and is totally free for personal use. It’s how I access all of my self hosted stuff when I’m out and about.

16

And here I thought I was looking solid with nginx proxy manager.

17

How much should one trust service log-in pages like grafana or jellyfin?

I have services that I want friends and family to access without the “hassle” of a VPN.

18

There are even bigger risks than those presented here. A lot of these tools let you run scripts on download completion etc. or have other holes for remote code execution.

Once an attacker gets access to this it’s game over: they can do much more than just get your name for phishing purposes.

The fact is this software was always written to be run on a local network and not on the public internet.

19

One thing I would add about the reverse proxy (i.e. Nginx) along with Cloudflare Proxying is to blacklist all the IPs that are not Cloudflare’s. Better if done using iptables, in order avoid flooding the proxy itself with useless stuff

20

Regarding the port scans, I know that this has been called out in other comments, but, there’s another angle I think is missing.

If you are performing a port scan, and performing it from an external source (such as a VPS or something) - PLEASE READ THE TERMS OF SERVICE AND ACCEPTABLE USE POLICY OF YOUR PROVIDER. THEY WILL ALMOST CERTAINLY STATE SOMETHING ABOUT PORT SCANNING AND WHETHER IT IS ALLOWED OR NOT.

Many larger providers, such as AWS, GCP, for example, do not allow port scans. Minimally your scan won’t complete, you may get an email from a security team, or up to your account being closed.

Secondly, I think somebody else already mentioned, but if you are performing a scan of your network (to your external IP) from inside your network, depending on your router and its configuration, you may end up with some wonky or false results. This depends on how it handles port forwards, how the bridge is set up, how the routes are configured, and if the router is able to perform hairpin NAT.

One other thing I want to point out - your point about nginx access rules. Nginx is a wonderful piece of software. I use it as a reverse proxy to many of my internal services.
In addition to the allow directive you posted, nginx also supports HTTP Auth, aka, auth_basic
Link: Nginx Docs, ngx_http_auth_basic_module

While auth basic is “alright” - it’s got plenty of attack surface. And, if you are not using https, your credentials will be sent in plain text. Very bad. SSL makes it a tad better. But in any case, auth_basic can stop most basic skids and crawlers.

My recommendation to this point is as follows:

  1. Use nginx as a reverse proxy for your services
  2. Use Let’s Encrypt to get certificates for nginx, and configure SSL/TLS (Conf Examples/Generators: Here and Here - I personally prefer the Mozilla generator, but whatever suites you.)
  3. Configure nginx to use access rules, based on IP AND http auth (they can be combined Nginx docs )
  4. ???
  5. Profit