Proxmox Host (hetzner-main)

hetzner-main is a Hetzner dedicated server running Proxmox VE 9 on Debian 13. It runs every VM in the Server List, and it is the SSH jump host for all of them.

Public address	136.243.40.234/26 on vmbr0 (gateway 136.243.40.193), IPv6 2a01:4f8:212:8d2::2/64
VM network	10.0.0.254/24 on vmbr1 (the gateway for every VM)
vSwitch to Amergin	192.168.100.1/24 on enp6s0.4000
Proxmox node name	proxmox (the Terraform files depend on this name)
SSH alias	proxmox-host, as root
Playbook	ansible/proxmox-host/setup-proxmox-host.yml

The host itself is configured by Ansible. On a fresh install, this playbook runs first: it creates the network bridges and the VM template that Terraform needs.

What the playbook does

Each part of the playbook has its own tag, so you can run it on its own.

Tag	Make target	What it does
bridges	setup-proxmox-host-bridges	Writes /etc/network/interfaces: vmbr0 holds the public IP on enp6s0, and vmbr1 is the internal VM bridge with no physical port. Uses commit-confirm. Only needed on a freshly installed host.
template	part of setup-proxmox-host	If VMID 9000 doesn't exist yet, downloads the Debian 13 generic cloud image and creates debian-13-template (VMID 9000) from it, with a cloud-init drive, a serial console and the QEMU agent turned on. The image is not customised; each VM's playbook installs qemu-guest-agent itself.
vswitch	part of setup-proxmox-host	Creates the VLAN 4000 interface enp6s0.4000 (192.168.100.1/24, MTU 1400), turns on IP forwarding, and clamps TCP MSS to 1360 on that interface. See Networking.
ssh-hardening	part of setup-proxmox-host	Key-only SSH (PasswordAuthentication no, PermitRootLogin prohibit-password). Doesn't touch authorized_keys.
fail2ban	part of setup-proxmox-host	Bans an IP for 1 hour after 5 failed SSH logins within 10 minutes.
unattended-upgrades	part of setup-proxmox-host	Automatic security updates.
backup	setup-proxmox-host-backup	Mounts the Hetzner Storage Box over CIFS and creates the nightly vzdump job. See Backups.
firewall	setup-proxmox-host-firewall	Writes and loads the nftables ruleset below. Uses commit-confirm.
robot-firewall	setup-proxmox-host-robot-firewall	Pushes rules to Hetzner's Robot edge firewall. Don't run it: see below.
wazuh-agent	enroll-wazuh-proxmox-host	Installs and enrolls the Wazuh agent.

make setup-proxmox-host runs the setup, vswitch and template tags. It requires ADMIN_SSH_IPS and PUBLIC_INTERFACE to be set, even though those tags don't use them. The backup tasks are also tagged setup, so this target sets up the backup job too, with the Hetzner Storage Box credentials. make setup-proxmox-host-backup runs only the backup part.

Firewall

The firewall tag writes /etc/nftables.conf. The ruleset is:

• Inbound to the host: SSH (port 22) only from the addresses in ADMIN_SSH_IPS. All traffic from the VM network (10.0.0.0/24) and from the vSwitch (192.168.100.0/24) is allowed. Everything else is dropped. The Proxmox web UI (port 8006) is not opened on the public interface.
• Port forwarding: ports 80 and 443 on the public IP are forwarded (DNAT) to the webserver VM, 10.0.0.14. No other VM receives inbound traffic from the internet.
• Outbound: VMs reach the internet through NAT (masquerade) on the public interface. Traffic between vmbr1 and the vSwitch is forwarded in both directions.

NAT and the port forward are IPv4 only.

The UI is also published at proxmox.abair.ie: the webserver's nginx proxies that name to 10.0.0.254:8006. So the Proxmox UI is reachable over the internet through the webserver, behind its own login. The other way in is an SSH tunnel:

ssh -L 8006:10.0.0.254:8006 proxmox-host   # then open https://localhost:8006

Terraform uses the same kind of tunnel. See Provisioning with Terraform.

The firewall targets need two variables:

Variable	Used for
ADMIN_SSH_IPS	Comma-separated list (no spaces) of IPs allowed to SSH in, e.g. ADMIN_SSH_IPS=1.2.3.4,5.6.7.8
PUBLIC_INTERFACE	The interface that carries the public IP. This is the bridge vmbr0, not the physical NIC. Confirm it with ip route show default.

Hetzner Robot firewall

The robot-firewall tag would turn on Hetzner's edge firewall for this server through the Robot API. It locked the host out of SSH once, during the rebuild on 2026-08-26. That firewall is currently off, and the host's own nftables is the only filter. Don't run make setup-proxmox-host-robot-firewall until the tag is fixed.

Commit-confirm

The bridges and firewall tags both change the only network path into the host. A mistake could lock everyone out of a remote machine with no console. So both use the same safety net:

1. The current config is saved.
2. The new config is applied.
3. A systemd timer is scheduled to restore the saved config after 5 minutes.
4. You confirm from a new SSH connection, which proves you can still get in. The old connection may already have dropped.

Tag	Confirm with
bridges	ssh proxmox-host touch /root/.network-switch-confirmed
firewall	ssh proxmox-host touch /etc/nftables.confirmed

If you don't confirm in time, the old config comes back.

caution

For the firewall, the automatic revert only affects the rules that are currently loaded. The new rules stay in /etc/nftables.conf, and nftables loads that file at boot. If the timer reverts the firewall, fix or restore /etc/nftables.conf before the host next reboots.

Rebuilding hetzner-main from scratch

The repo can rebuild the whole host after a fresh OS install. The full runbook, including the manual steps, is in progress/fresh-install-hetzner-main.md in the repo. ansible/proxmox-host/README.md also covers it. In outline:

1. Install Debian 13 with Hetzner's installimage, using the hostname proxmox, then install Proxmox VE 9 from its apt repository.
2. make setup-proxmox-host-bridges, then confirm.
3. make setup-proxmox-host ADMIN_SSH_IPS=... PUBLIC_INTERFACE=vmbr0.
4. make setup-proxmox-host-firewall ADMIN_SSH_IPS=... PUBLIC_INTERFACE=vmbr0, then confirm.
5. Check that the backup job exists (pvesh get /cluster/backup). Step 3 creates it.
6. make provision PROXMOX_JUMP_HOST=proxmox-host to create the VMs. Then clear the old host keys with ssh-keygen -R for each VM IP.
7. Configure the VMs. docker-build goes first, and its registry data must be restored before the other VMs can pull images.
8. Restore stateful data (Plausible, app .env files, piper models, and so on) from the nightly backups with the restore script. Restore scealai as a whole VM.

make bootstrap-hetzner-main chains steps 3, 6 and 7 into one command. It doesn't handle the registry restore that step 7 needs, so on a real rebuild run the steps one at a time.

Admin-only

These commands can wipe or cut off the whole host. Only admins run them.