Every target below runs inside the tooling container (see Getting Started). Most fetch secrets from Vaultwarden, so run eval "$(make unlock-bw)" first in the same terminal. The last column shows which Vaultwarden items each target reads.
Every target here changes production directly. There's no staging environment. If you're not an admin, open a pull request and ask an admin to run the target. See Making Changes.
These run on your own machine, not inside the container.
| Target | What it does |
|---|
docker-build | Builds the tooling image infra-automation:local. A bare make does the same. |
docker-shell | Opens a shell in the tooling container with the same mounts. Use it for the one-time bw login, or to run terraform/ansible-playbook by hand. |
unlock-bw | Unlocks the vault. Use it as eval "$(make unlock-bw)". |
| Target | What it does | Secrets |
|---|
provision | terraform apply -auto-approve for the hetzner-main VMs. Needs PROXMOX_JUMP_HOST=proxmox-host. | Proxmox Terraform Token |
wait | provision, then a 30-second wait | as above |
setup-all | wait, then site.yml --tags setup,gitea-runner on every hetzner-main VM | Docker Registry Password, Gitea Runner Registration Token, Cloudflare DNS Token, Webserver htpasswd admin, Grafana Admin Password, API Gateway Admin Secret |
update-all | site.yml --tags update on every hetzner-main VM (no Terraform) | same, minus the Gitea token |
bootstrap-hetzner-main | setup-proxmox-host, provision, wait, setup-all | all of the above |
Before running provision, wait, setup-all or bootstrap-hetzner-main, read the state warning.
Proxmox host
The firewall targets need ADMIN_SSH_IPS and PUBLIC_INTERFACE. See Proxmox Host.
| Target | What it does | Secrets |
|---|
setup-proxmox-host-bridges | Network bridges (commit-confirm). Fresh installs only. | |
setup-proxmox-host | SSH hardening, fail2ban, upgrades, vSwitch, VM template, backup job | Hetzner Storage Box |
setup-proxmox-host-firewall | nftables (commit-confirm) | |
setup-proxmox-host-robot-firewall | Hetzner Robot firewall. Don't run. Needs HETZNER_SERVER_IP. | Hetzner Robot Webservice |
setup-proxmox-host-backup | Storage Box mount and the nightly vzdump job | Hetzner Storage Box |
hetzner-main VMs
| Target | What it does | Secrets |
|---|
setup-docker-build / update-docker-build | docker-build base setup and GitHub runner. Pass TARGET_GITHUB_TOKEN=<token> when registering a runner. | Docker Registry Password, ABAIR Github Runner |
setup-gitea-runner / update-gitea-runner | Gitea Actions runner on docker-build | Docker Registry Password, Gitea Runner Registration Token (setup only) |
deploy-docker-registry | Redeploys the registry stack only | |
setup-webserver / update-webserver | nginx, certificates, sites from networking/nginx-config/ | Cloudflare DNS Token, Webserver htpasswd admin |
setup-api-gateway / update-api-gateway | API gateway stack | API Gateway Admin Secret, Docker Registry Password, Webserver htpasswd admin, note api-gateway .env.secrets |
setup-services-user / update-services-user | services-user base setup | Docker Registry Password, Webserver htpasswd admin |
deploy-services-user-apps | Deploys every services-user app | Docker Registry Password, notes <app> .env |
setup-services-staff / update-services-staff | services-staff base setup | Docker Registry Password, Webserver htpasswd admin |
deploy-services-staff-apps | Deploys every services-staff app except Plausible | Docker Registry Password, notes <app> .env |
deploy-analytics-plausible | Deploys Plausible on services-staff | note analytics-plausible .env |
setup-synthesis-legacy / update-synthesis-legacy | synthesis-legacy stack (see its known issue) | Webserver htpasswd admin |
Amergin
| Target | What it does | Secrets |
|---|
setup-amergin-host | Full host setup: NVIDIA, Docker, vSwitch, monitoring, watchdog | Webserver htpasswd admin |
update-amergin-host | Re-applies persistence mode, UFW, the daily Docker clean-up, Alloy, DCGM and the watchdog | Webserver htpasswd admin |
setup-streaming-recognition / update-… | Triton streaming ASR | Gitea Automation Bot Token |
setup-fotheidil-recognition / update-… | asr-diarisation and marian-cpr. Deploy before fotheidil-api, which uses its network. | Gitea Automation Bot Token, Huggingface Token Amergin (pyannote's diarisation models only) |
setup-fotheidil-api / update-… | Fotheidil API | Gitea Automation Bot Token, Fotheidil Supabase Key, Fotheidil Supabase Service Role Key, ABAIR Subtitles Gateway Shared Secret |
setup-batch-recognition / update-… | batch-asr and batch-marian-cpr | Gitea Automation Bot Token |
setup-predictive-text / update-… | Predictive text | Gitea Automation Bot Token |
update-predictive-text-model | Reloads the predictive-text model only | |
setup-synthesis / update-synthesis | Speech synthesis | Gitea Automation Bot Token |
setup-testing-user / update-testing-user | GPU testing sandbox | note users.json |
Secops
| Target | What it does | Secrets |
|---|
setup-secops-host | Base host. Use ANSIBLE_USER=root on the first run. ADMIN_SSH_IPS is optional. | |
setup-secops-host-firewall | nftables (commit-confirm) | |
setup-secops-host-robot-firewall | Broken. Don't run. | Hetzner Robot Webservice |
setup-wazuh / update-wazuh | Wazuh manager, indexer and dashboard | Wazuh Admin Password, Wazuh Dashboard Internal Password, Wazuh API Password, Wazuh Agent Registration Password |
wazuh-dashboard | SSH tunnel to the Wazuh dashboard (see Secops) | |
setup-gitea-secops / update-gitea-secops | Gitea | Gitea DB Password, Gitea Admin Account, Cloudflare DNS Token, note Gitea .env |
setup-observability-secops / update-observability-secops | Loki, Prometheus, Grafana, Uptime Kuma | Grafana Admin Password, Webserver htpasswd admin, Cloudflare DNS Token, note users.json |
setup-rocketchat-secops / update-rocketchat-secops | Rocket.Chat | Rocket.Chat Admin Password, Cloudflare DNS Token |
setup-secops-backup | Borg backup | Borg Passphrase - secops |
Vaultwarden
| Target | What it does | Secrets |
|---|
setup-vaultwarden-host | Base host. Use ANSIBLE_USER=root on the first run. | |
setup-vaultwarden-host-firewall | nftables (commit-confirm) | |
lock-root-vaultwarden | Removes root's SSH access. One-way. | |
setup-vaultwarden / update-vaultwarden | Vaultwarden, nginx and the certificate | Vaultwarden Admin Token, Cloudflare DNS Token |
setup-vaultwarden-backup | Borg backup | Borg Passphrase - vaultwarden |
Wazuh agents
Each target installs and enrolls the Wazuh agent on one host. Run setup-wazuh first. Every target reads the Wazuh Agent Registration Password.
enroll-wazuh-webserver, enroll-wazuh-api-gateway, enroll-wazuh-services-user, enroll-wazuh-services-staff, enroll-wazuh-synthesis-legacy, enroll-wazuh-docker-build, enroll-wazuh-scealai, enroll-wazuh-proxmox-host, enroll-wazuh-amergin, enroll-wazuh-secops-host, enroll-wazuh-vaultwarden-host
Puca
| Target | What it does | Secrets |
|---|
setup-puca PUCA_PAT=<token> | Full Puca setup, using your own GitHub token for the private recorder repo | Webserver htpasswd admin |
setup-puca-john | The same, with the token from Vaultwarden | Github Puca PAT, Webserver htpasswd admin |
Both prompt for a sudo password (-K). See Puca → Access for where to run them from.
Variables you can pass
| Variable | Used by |
|---|
PROXMOX_JUMP_HOST | provision and anything that depends on it. Set it to proxmox-host. |
ADMIN_SSH_IPS | The Proxmox, Secops and Vaultwarden firewall targets. Comma-separated, no spaces. |
PUBLIC_INTERFACE | The Proxmox firewall. Use vmbr0. |
HETZNER_SERVER_IP | setup-proxmox-host-robot-firewall |
STORAGEBOX_HOST, STORAGEBOX_USER | Backup targets. Default: u348035.your-storagebox.de / u348035. |
TARGET_GITHUB_TOKEN | setup-docker-build: a fresh GitHub runner registration token |
ANSIBLE_USER | First run of setup-secops-host / setup-vaultwarden-host (root) |
PUCA_PAT | setup-puca |