I am excited about the new Tailscale Funnel feature. I have been asking for an option like this ever since the very first time anyone at Tailscale asked me for my opinion. What the folks at Tailscale have done isn’t exactly what I asked for. Funnels are better than the feature I was envisioning, and they’re only worse in a few minor ways.

My wish was to be able to click a button in the Tailscale admin console to have an IPv4 address assigned to a machine on my Tailnet. The use case floating in my mind was that I could leave a clone of my Digital Ocean droplet running on my homelab server. If something went sideways with Digital Ocean, I could immediately point Cloudflare away from the old server and toward my new Tailscale IP address and have all our blogs back up and on the Internet with very little effort.

Tailscale has done a good job with their Funnel. I only need to touch the admin console once to set up the ACLs to allow my machines to create funnels. Everything else is handled on the nodes that want traffic to be funneled their way. That’s awesome!

How is this different than the feature I’ve been patiently hoping for over the last year or two? Tailscale Funnel can only tunnel TLS connections. It can forward traffic to a web server on your Tailnet just fine, but I don’t expect you can use a Funnel to let the entire world connect to your Minecraft server.

I am OK with that.

• Introducing Tailscale Funnel

What functionality did I replace (and upgrade!) with Tailscale Funnel?

Brian Moses and I have some rather simple infrastructure set up so we can both push new blog posts to butterwhat.com, and they will be automatically published. The blog posts are stored in a private repository on Gitlab.com. I have a virtual machine here on my homelab server that regularly pulls the latest changes down and publishes if there are any changes.

That virtual server also runs continuous previews of the jekyll blog so that Brian can see what he’s working on.

Instead of connecting to Gitlab every two minutes looking for changes, I figured I could configure a webhook on that repository and Gitlab could just tell me when there is a change.

Setting up the Funnel and webhook was easy!

I have never done any of this before. I was chatting about my progress on our Discord server, and it sure looks like it took me less than 30 minutes. That even includes tearing down most of what I saw in the webhook tutorial I was following so I could reconfigure things so the webhook server wouldn’t run as root. I also goofed up and accidentally configured the webhook server to listen for Github instead of Gitlab, and I had to spend a few minutes scratching my head before I noticed.

Aside from the Tailscale funnel, the star of the show is the little webhook server available in Ubuntu’s apt repositories. I didn’t really know how I was going to listen for Gitlab’s connection and kick off a shell script. The answer wound up being quite simple!

I followed a nice tutorial on how to use Gitlab webhooks. The tutorial tells you how to install and configure webhook, how to configure webhook for Github, and how to configure Github to call your webhook. There are links to sample configurations for Gitlab as well.

Tailscale’s documentation is always fantastic, but the documentation for the Funnel is a little sparse. That is OK. Funnel is a new feature, and it isn’t even a production feature.

To use a funnel, you need to enable HTTPS certificates in your Tailscale admin console, allow funnels in your ACLs, and then turn on the funnel at the client. It was the second step there that I wasn’t sure about. I was wondering if I could enable funnels based on ACL tags, and it turns out that you can.

1
2
3
4
5
6

"nodeAttrs": [
  {
      "target": ["tag:blogdev"],
      "attr":   ["funnel"],
  },
],

The webhook server defaults to listening on port 9000. I had to run two commands to pass that through to the funnel:

1
2

root@butterwhat:~# tailscale serve / proxy 9000
root@butterwhat:~# tailscale serve funnel on

I was going to leave out my webhook config file. I just pasted in the sample from the documentation and edited a few lines. I may as well include it, so here it is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

[
  {
    "id": "butterwhat-gitlab",
    "execute-command": "/home/pat/bin/webhook.sh",
    "command-working-directory": "/home/pat",
    "pass-arguments-to-command":
    [
      {
        "source": "payload",
        "name": "user_name"
      }
    ],
    "response-message": "Executing redeploy script",
    "trigger-rule":
    {
      "match":
      {
        "type": "value",
        "value": "footballislifebutalsodeath",
        "parameter":
        {
          "source": "header",
          "name": "X-Gitlab-Token"
        }
      }
    }
  }
]

It has been slightly modified to protect my little webhook server.

• Deploy from Github/Gitlab to server using Webhook

Did I do a good job with my blog publishing script updates?

Not really! I did set things up so that the pull-and-publish job only runs one process at a time. The bad news is that the wrong thing will happen if we manage to push another commit before the job finishes. We are a small team. The odds of this happening are not exactly high.

When I started setting this up, I assumed I would be able to completely eliminate the publish loop. Then I remembered that the script is smart enough to only publish blog posts with timestamps in the past. The loop still needs to run to eventually publish blog posts in the future.

Instead of pulling from Gitlab and publishing every three minutes, it is now doing so only once an hour. That seems much more reasonable, though I am certain I can better optimize this.

The webhook over the funnel works great, but it is the wrong solution for us!

Our blog-publishing infrastructure seems out of place today. It is not only possible to have Github or Gitlab publish our Jekyll and Octopress blogs directly every time we push to the appropriate repo. It was possible to do this when Butter, What?! was born, but we had a virtual machine publishing previews of upcoming posts anyway. Making that machine rsync to our web server didn’t add much in the way of friction.

Brian Moses and I set up our collaboration on Butter, What?! before we heard of Tailscale, and possibly even before Tailscale released a product. We used a private repo on Gitlab so that we would both easily have access to the Markdown files.

I am excited about trying out Tailscale Funnel today to streamline our process, but it also made me realize that using Gitlab for this is an artifact left over from the days when we didn’t use Tailscale.

The Butter, What?! test server is on my Tailnet, and I already share it with Brian. He could just as easily be pushing directly to a repo on that server. We don’t need to host anything on the Internet. We don’t need to be punching a hole to our development box for Gitlab to send us a notification.

Everyone at Butter, What?! is all-in on Tailscale. I don’t like to redo things that are already working, but the idea of moving the blog repo onto our Tailnet is in my mind now.

I might be most excited about Tailscale serve

The new serve command is awesome. It seems to be new, and it looks like the whole thing arrived with the Funnel update.

I am not entirely certain of everything you can accomplish with this new TCP forwarding and proxy system. It seems to have plenty of seemingly artificial limits so far, but even so, it sure looks like it will come in handy. Right now you can’t point the proxy at a different machine on the local network. It can only proxy localhost.

You can use the proxy to wrap an HTTP server on the Tailscale node in TLS. This seems neat. You won’t have to figure out how to correctly install your certificates after running tailscale cert. As far as I can tell, the Tailscale proxy will just use the generated certificates.

Why is this a big deal? I already know how to set up my certificates with nginx or apache. Why would I need Tailscale to do it for me?

Octoprint doesn’t run as root, so it can’t run its embedded web server on port 80 or port 443. I also have no idea what web server software it uses or how to configure it. I can just run tailscale cert octoprint.humpback-rooster.ts.net then tailscale serve / proxy 5000 on my Octoprint server, and it will wrap my Octoprint instance up in some sweet encrypted TLS.

I don’t need to know how my appliances configure their web servers. I don’t have to worry about an update wiping out my TLS config changes. I can just wrap them in TLS without installing any extra software on the node, and I think that is awesome. I can set up that proxy in less time than it would take me to figure out that Octoprint isn’t using nginx or apache.

Forget the funnels. The real gem that was made available to me this week is the proxy built into tailscale serve!

Conclusion?!

I am excited. Did I already say that? I really am. Tailscale just keeps making my life easier. Tailscale makes it easier to keep all my machines connected and safely off the public Internet. Then Tailscale made it really easy to let friends and associates connect to my private servers. Then Tailscale started managing 90% of my SSH keys for me.

Now they’re letting me poke tiny holes in my mesh firewall and even stick a TLS proxy in front of both my public and private web servers. This is all fantastic stuff, and I expect Tailscale will just continue to become even more awesome and convenient!

• Introducing Tailscale Funnel
• Tailscale Funnel Documentation
• I am Using Tailscale SSH, and Maybe You Should Too!
• Making My Life Easier With Tailscale