Is It Time For You to Set Up Tailscale ACLs?

If you’re a lone Tailscale user like me, there’s a good chance that you have no pressing need to set up Tailscale’s access control lists (ACLs). Until quite recently, I didn’t feel there was much reason to lock anything down.

Pretty much every computer I own has been running Tailscale for more than a year now. They could all ping each other. In fact, most of them are on the same LAN, and they could ping each other before I had Tailscale. Tailscale already locked them down a bit more thoroughly for me. Why lock them down any more?

Then I started using Tailscale SSH

As soon as I started enabling Tailscale SSH, I needed to set up some access controls. I wanted to emulate my previous setup.

My desktop and two laptops had their own SSH private keys, and their matching public keys were distributed to all my other machines. That meant these three computers could connect to any computer I own.

"ssh": [
        // I don't actually use this rule anymore!
      {
          "action": "accept",
          "src":    ["tag:workstation"],
          "dst":    ["tag:server", "tag:workstation"],
          "users":  ["autogroup:nonroot", "root"],
      },
  ],

I gave those three devices a tag of workstation, and I stuck a server tag on everything else. Then I set up an ssh rule in Tailscale to allow any workstation to ssh into any server or any other workstation.

So far, so good. This configuration does happen on Tailscale’s Access Controls tab, but it isn’t in the acls section of the file. At this point, my Tailnet was still wide open.

I got worried when I added my public web server to my Tailnet

I have a tiny Digital Ocean droplet running nginx hosting patshead.com, briancmoses.com, and butterwhat.com. I always said I should install Tailscale out there, but my web server droplet has been running an outdated operating system for a while, so I knew I would be creating a fresh VPS at some point.

I finally did that. I spun up one of the new $4 per month droplets, copied my nginx config over, and installed Tailscale. I am super excited about this because it means I don’t even have to have an ssh port open to the Internet on my web server.

However, this means that a scary server that I don’t personally own that is sitting out there listening for connections on the Internet is connected directly to my Tailnet. Yikes!

Tagging all your machines for use in ACLs is hard!

It isn’t hard because you have to click on every machine to add tags. It is challenging because choosing names for your tags is easy to goof up!

My original decision that a workstation would connect to a server and never the other way around was too simple. It wasn’t the right way for me to break things down, and as I started adding more tags, I wasn’t able to easily set things up the way I wanted.

I’ve been doing my best to make sure my Tailscale nodes don’t have any services open on their physical network adapters. My workstations are mostly locked down well, and I moved things like my Octoprint virtual machine behind the NAT interface of KVM instead of being bridged to my LAN.

Even so, I have two servers at home that need to be accessible from outside my Tailnet. My NAS shares video to my Fire TV devices just in case I need to watch Manimal, and I have lots of unsafe devices around the house that need to connect to my Home Assistant server.

This seemed easy. I immediately tagged my NAS, my Home Assistant server, and my public web server with a tag of dmz.

Tailscale Network access controls

What was the problem with this?

I want my workstations to be able to see everything. I want my servers to be able to communicate with each other, but I don’t want my servers in the dmz to be able to connect to my internal servers or workstations.

This all seemed simple and smart until I realized that everything in my dmz already had a server tag. I also very quickly realized that my Home Assistant server listening to my LAN is much less threatening than my web server listening to the public Internet. One of those should be on an even more restricted tag!

Where did I actually land?

I have four main tags now:

workstation
server-ts
server-dmz
server-external

My personal workstations can connect to anything. Machines tagged server-ts can connect to machines tagged server-ts and server-dmz, while the server-dmz servers can only talk to other server-dmz machines.

  "acls": [
    {"action": "accept", "src": ["tag:workstation"],   "dst": ["*:*"]},
    {"action": "accept", "src": ["tag:server-ts"],     "dst": ["tag:server-ts:*", "tag:server-dmz:*", "autogroup:internet:*"]},
    {"action": "accept", "src": ["tag:server-dmz"],     "dst": ["tag:server-dmz:*"]},
    {"action": "accept", "src": ["tag:blogdev"],      "dst": ["tag:blogprod:22"]},
    {"action": "accept", "src": ["nas"],              "dst": ["seafile:*"]},
    {"action": "accept", "src": ["autogroup:shared"], "dst": ["tag:shared:22,80,443"]},
  ],

These are all my ACLs as of writing this. There are a couple of more specific rules there that I didn’t talk about yet.

There’s a rule there that allows one of my virtual machines here at home to publish content to my public web server.

My NAS is in the dmz, so I had to give it its own rule to allow it to connect to my Seafile Pi*. My NAS syncs extra copies of some of my data for use as a local backup!

I goobered up my exit nodes!

I am more than a little embarrassed by how many times I had to go back and forth between desks to figure out why the exit node on my GL.iNet Mango stopped passing traffic to the Internet.

The Mango had a tag that allowed it to access the exit node. If I took that tag away, it couldn’t ping the exit node. I’d add it back, and while it could ping the exit node, it couldn’t route any farther. If I dropped the original ACL that leaves everything wide open, the Mango could route traffic just fine. What was going wrong?!

It seems like I had this idea in my head that Tailscale’s ACLs only applied to Tailscale nodes and addresses. I didn’t immediately realize that I had to explicitly allow access to the Internet or even other subnets I might be routing!

{"action": "accept", "src": ["tag:server-ts"], "dst": ["tag:server-ts:*", "tag:server-dmz:*", "autogroup:internet:*"]},

I just had to add autogroup:internet to the allowed destinations for the appropriate tag. Duh!

Don’t think too hard before implementing your ACLs

This is especially true if you are down here at my scale with a couple dozen nodes and only a few shared nodes. Just drop some tags on things and set up some access controls that allow nodes access to what they need.

You probably won’t set things up optimally. I know I didn’t on my first try, and I am already seeing things I’d like to do differently. Even if my initial attempt left things more open than I might like, it was still a huge win just because it blocked my public web server from connecting to the rest of my Tailnet. Any other improvements are minor by comparison.

If money and other people’s livelihoods are on the line, maybe you should spend some time having meetings and planning things out on whiteboards. It only takes a few seconds to switch back to the single default ACL that leaves your Tailnet wide open, so if you do find a problem, you can at least revert your changes quickly and easily!

Tailscale SSH is affected by Tailscale network ACLs!

This seems obvious, but I wasn’t positive that this would be the case! Tailscale seems to always make the best possible default choices, and that got me thinking that it might be the case that Tailscale’s own SSH server would ignore the ACLs if the connection were allowed in the ssh section of the access control configuration.

This does not seem to be the case. If you want to use Tailscale SSH, then your networking ACLs have to allow it. To be clear, I think this was the correct thing for Tailscale to do.

I am Using Tailscale SSH, and Maybe You Should Too!

Shared nodes are allowed access by default

I wasn’t sure about this. The default single ACL just has one line that allows everyone access to everything. The first thing you do when designing your own ACLs is delete that entry. At that point nobody has access to anything, so I assumed I would need to add a line similar to this:

{"action": "accept", "src": ["autogroup:shared"], "dst": ["tag:shared:*"]},

We tested this. This wasn’t necessary, but I figured it would be a good idea to lock down my shared nodes just a bit, so I wound up using this ACL:

{"action": "accept", "src": ["autogroup:shared"], "dst": ["tag:shared:22,80,443"]},

It is a bit lazy. Three people need access to ports 80 or 443 on the Seafile server, and Brian needs SSH access to rsync files to his blog. It gets the job done.

I did test out removing ports 80 and 443 from this ACL, and I watched the connections on my Seafile server. All the Tailscale IP addresses that I didn’t own dropped off the netstat list, and when I put those ports back in the ACLs, everyone connected back up immediately.

I am sure the documentation explains this, but I doubt I am the only one who likes to see things work in practice just to make sure!

Forgetting you have Tailscale ACLs configured makes troubleshooting a real challenge!

This happened to me yesterday! A friend sent me a GL.iNet GL-AXT1800 router to help him get his identical router to pass local traffic through a Tailscale exit node.

I installed the ancient OpenWRT Tailscale package, replaced the binaries with the official Tailscale static ARM binaries, ran tailscale up, and it gave me the URL to open to authenticate this new node. Everything went smoothly, except I couldn’t ping any of my other Tailscale devices!

Derp. Since I didn’t remember to put any tags on my new Tailscale device, it wasn’t matching any of my Tailscale ACLs, so it couldn’t actually connect to anything!

This was a simple mistake, but I walked back and forth between two desks and rebooted the GL.iNet router at least twice before remembering that I even configured any Tailscale ACLs in the first place!

Conclusion

If you’re just a home gamer like I am, you probably don’t need to worry about Tailscale ACLs. If you have one or more nodes on your Tailnet that have services running on the open Internet, you may want to lock things down a bit. It would be a real bummer if someone managed to crack open your public web server, because they might be able to ride Tailscale past your other routers and firewalls.

One of the awesome things about Tailscale is that I have absolutely no idea what you’re doing with it. You might just be one person sharing a Minecraft server with some friends. You might be sharing a couple of servers with business partners like I am. You might even be managing a massive and complicated Tailnet at a giant corporation.

You and your Minecraft server probably don’t need to worry about ACLs, but if you are in a position where you should be thinking about tightening up your access controls, I hope my thoughts have been helpful!