Permission denied (publickey) is never a firewall problem

I spent an evening taking SSH off the public internet.

The box is an 8GB Hetzner instance in Helsinki, fresh Ubuntu. The plan was ordinary: a non-root sudo user, key-only auth, root login disabled, ufw dropping everything inbound except 80 and 443. Then Tailscale, so the machine joins a private network and SSH listens somewhere the public internet can’t reach.

That last part is the one genuinely nice trick in the setup. A ufw rule can be scoped to a network interface, and Tailscale gives the box a second interface — tailscale0, alongside the public-facing eth0. So instead of “allow SSH from anywhere,” you can say:

sudo ufw allow in on tailscale0 to any port 22 proto tcp

Allow port 22, but only through the private door. Then delete the public rules and port 22 stops existing as far as the internet is concerned.

I tested it before deleting anything — new terminal, SSH over the tailnet, in. The old session stayed open the whole time, because the one rule of SSH hardening is that you never disable your current way in until you’ve proven the new way works.

So I proved it. Then I closed the public door, went to bed, and the next day it stopped working.

johndoe@hetzner-box: Permission denied (publickey).

The most attractive suspect is the thing you just built

My first thought was that I’d locked myself out of my own machine.

It’s a reasonable thought. I had just spent an evening rearranging the only path to the box, and somewhere in there was an interface-scoped firewall rule I’d written for the first time in my life, plus a network overlay I’d installed four hours earlier. Of course that’s what broke. Nothing else had changed.

So I opened the Hetzner web console — the virtual keyboard and monitor that plugs into the machine underneath SSH, the firewall, and all networking — and started poking at ufw.

I shouldn’t have. The error message had already ruled all of it out.

Two gates

Getting into a server over SSH means passing two checks in sequence, and the useful thing about them is that they fail differently.

Gate 1 is the network: can my packets reach sshd at all? That’s firewalls, routing, Tailscale, the port, whether I typed the right host.

Gate 2 is authentication: now that we’re talking, does sshd accept my key?

Permission denied (publickey) is a message from sshd. For sshd to send it, sshd had to receive my connection, negotiate a session, look at what I offered, and decide to reject it. That’s a full round trip. The server heard me and answered.

If Tailscale had been down, I’d have gotten Connection timed out — packets going nowhere. If the firewall rule were wrong, same thing. If sshd had died, Connection refused — the machine answering on behalf of a service that isn’t there.

I got none of those. I got the one failure that is only reachable after passing gate 1. The tailnet was up, the routing worked, and ufw was passing traffic on tailscale0 exactly as written. The entire evening’s work was functioning perfectly, and the error said so in its first word.

I spent an hour not reading it.

The false lead

The console did turn up something. Here’s what /home/johndoe/.ssh looked like:

drwxrwxr-x  2 johndoe johndoe  .ssh
-rw-rw-r--  1 johndoe johndoe  authorized_keys

775 and 664. Both group-writable. The key itself was present and correct, owned by johndoe.

This is a real bug. SSH refuses to trust an authorized_keys that anyone other than the owner can write to — if your group can append a line to that file, your group can log in as you. So sshd ignores the file. Not warns about it: ignores it. And from the client’s side, “I ignored your key file” and “I rejected your key” produce the same six words.

I’d created the file by hand and my umask handed it group-write. ssh-copy-id would have set the permissions correctly on its own.

chmod 700 /home/johndoe/.ssh
chmod 600 /home/johndoe/.ssh/authorized_keys

No reload needed — sshd re-checks permissions on every connection.

Still Permission denied (publickey).

This is the expensive kind of wrong theory: one that’s also a genuine bug. I found something broken and I fixed it, which feels exactly like progress right up until the moment nothing changes. If those permissions had been fine, I’d have discarded the theory in thirty seconds and moved on.

What it actually was

An hour in, I finally ran the thing I should have run second.

ssh -v johndoe@hetzner-box

debug1: Will attempt key: /Users/johndoe/.ssh/id_rsa
debug1: Will attempt key: /Users/johndoe/.ssh/id_ecdsa
debug1: Will attempt key: /Users/johndoe/.ssh/id_ed25519
...
debug1: No more authentication methods to try.
johndoe@hetzner-box: Permission denied (publickey).

Every one of them came back no pubkey loaded. Those files don’t exist. Here’s what’s actually in ~/.ssh on my Mac:

hetzner-box-key
hetzner-box-key.pub
github-personal
work-vpn
side-project

The key was sitting right there. SSH never tried it.

SSH does not search ~/.ssh for keys. It walks a fixed list of default names — id_rsa, id_ecdsa, id_ed25519, a couple of others — and offers whichever ones exist. My key is called hetzner-box-key. It’s not on the list, so as far as the client was concerned, I had no keys at all. It wasn’t rejecting my key. It was arriving empty-handed and getting turned away, which looks identical.

Which leaves the question of why it ever worked. It worked because I’d loaded the key into ssh-agent while setting the box up, and the agent offers whatever it’s holding regardless of filename. Then my Mac restarted. The agent came back empty. SSH fell back to the default-names list, found nothing, and had nothing left to send.

That’s the whole bug. A reboot and a filename.

ssh -i ~/.ssh/hetzner-box-key johndoe@hetzner-box

Straight in. -i names the identity file explicitly, which confirmed the key had been fine the entire time.

The permanent fix:

Host hetzner-box
    HostName hetzner-box
    User johndoe
    IdentityFile ~/.ssh/hetzner-box-key 
    IdentitiesOnly yes ~/.ssh/config

IdentitiesOnly yes matters more than it looks. Without it, SSH will still throw every other key it can find at the server before getting to the one you named. With a .ssh full of keys, you can exhaust the server’s MaxAuthTries and get rejected for too many failed attempts while holding the correct key.

The login banner closed the loop:

Last login: Mon Jul  6 00:47:21 2026 from 100.124.x.x

That’s my MacBook’s Tailscale address. The connection came in over the tailnet, through tailscale0, with public SSH closed at both the ufw and Hetzner firewall layers. Exactly as designed. It had been working the whole time.

The part worth keeping

Categorize the error before you form a theory about it. SSH failures split cleanly, and the split costs nothing:

Half the system gets eliminated before you’ve spent a minute in it. And once you’re at gate 2, ssh -v answers the only question that matters — which keys did the client actually offer — in its first ten lines.

The layer I’d just built was the one layer the error had already exonerated. It was also the only place I looked.