Skip to main content

Secops

secops-host is a Hetzner AX42 dedicated server (135.181.120.84) for security and operations tools. It is deliberately separate from production: it has no vSwitch into the private network and is reached only over the internet. If production is compromised, the SIEM, the code repositories and the alerting chat aren't compromised with it.

Address135.181.120.84 (alias secops-host, user ansible-control)
Playbooksansible/secops/
ServicesWazuh, Gitea, Rocket.Chat, and the monitoring stack

Every service runs in Docker and listens on 127.0.0.1 only. Where a service is public, nginx on the host terminates TLS in front of it. Each service playbook issues its own copy of the abair.ie / *.abair.ie wildcard certificate with Certbot (Cloudflare DNS-01).

Base host​

make setup-secops-host (setup-secops-host.yml) sets up:

  • an ansible-control user with passwordless sudo;
  • key-only SSH (PasswordAuthentication no, PermitRootLogin prohibit-password);
  • fail2ban (5 failed SSH logins in 10 minutes gives a 1-hour ban);
  • unattended upgrades.

On the very first run, only root has access, so run make setup-secops-host ANSIBLE_USER=root. After that, leave ANSIBLE_USER off.

Firewall​

make setup-secops-host-firewall writes an nftables ruleset and uses the same commit-confirm process as the Proxmox host. Confirm with:

ssh secops-host sudo touch /etc/nftables.confirmed
PortAllowed from
22anywhere (unless ADMIN_SSH_IPS is set)
443Cloudflare's IPv4 ranges, fetched live on each run
443136.243.40.234: all traffic from the hetzner-main VMs, e.g. the webserver's relays and the Gitea runner
443134.226.0.0/16 (TCD), for direct Git LFS pushes to gitea-direct.abair.ie
1514, 1515, 55000the Wazuh agents: hetzner-main 136.243.40.234, Amergin 5.9.9.57, Vaultwarden 91.99.99.28

Everything else is dropped, including port 80. All rules are IPv4 only. The allowed sources come from the extra_allow_rules variable in the playbook.

Port 22 is shared with Gitea

Git over SSH to Gitea also uses port 22 (see Gitea). If you set ADMIN_SSH_IPS to lock SSH down to admins, everyone else loses git clone over SSH.

Docker-published ports skip this firewall

These rules only filter services listening on the host itself. A container that publishes a port directly is reachable regardless of nftables. That's why every container here binds to 127.0.0.1 and sits behind nginx.

The robot-firewall tag (Hetzner's edge firewall) is known to be broken; don't run it. It is the same code that locked the Proxmox host out on 2026-08-26.

Wazuh​

A single-node Wazuh 4.14.6 deployment (manager, indexer and dashboard) in /opt/wazuh-stack, set up with make setup-wazuh.

  • Agents connect on ports 1514 (events) and 1515 (enrollment). nginx on the host passes these through as plain TCP to the manager on 127.0.0.1, and only accepts the three production IPs listed above.
  • The dashboard has no public access at all. Reach it through an SSH tunnel:
    ssh -L 8443:127.0.0.1:443 secops-host
    Then open https://localhost:8443 and log in as admin (Vaultwarden: Wazuh Admin Password). There's one shared account and no MFA; see Admin Services.
  • Which hosts have agents, and how to enroll them, is covered on Monitoring and Security.

make wazuh-dashboard runs the same ssh -L command, but inside the tooling container, so the tunnel may not reach your own localhost. If it doesn't work, run the ssh command above directly.

Gitea​

Gitea 1.26.4 with Postgres 16, in /opt/gitea-stack, set up with make setup-gitea-secops. It hosts ABAIR's source code and models under the ABAIR organisation.

HostnameCloudflareUsed for
gitea.abair.ieproxiedThe web UI and HTTPS Git for everyone
gitea-direct.abair.ieDNS only (not proxied)Git over SSH; the Gitea Actions runner; large Git LFS pushes. Over HTTPS it only works from TCD and hetzner-main.

gitea-direct exists because Cloudflare cuts off long-running connections (the Actions runner long-polls for jobs) and limits upload size (which broke the first LFS push). Both names point straight at this server; the webserver isn't involved.

Settings:

  • two-factor authentication is required for every user;
  • registration is closed and sign-in is needed to view anything;
  • Git LFS is on;
  • Actions are on;
  • email notifications go out through mailbox.org SMTP, with settings from the Secure Note Gitea .env.

Git over SSH uses the host's real port 22: git clone git@gitea-direct.abair.ie:ABAIR/<repo>.git. The host's git user hands every connection to the Gitea container through a small wrapper (/usr/local/bin/gitea-serv). A systemd path unit keeps that user's authorized_keys in sync with the keys users add in Gitea.

Deploy playbooks authenticate to Gitea as a read-only bot account (Vaultwarden: Gitea Automation Bot Token), not a personal account.

Rocket.Chat​

Rocket.Chat with MongoDB 8.0, in /opt/rocketchat-stack, set up with make setup-rocketchat-secops. It serves alerts.abair.ie, which the webserver relays here (see Networking). A Watchtower container updates Rocket.Chat to the latest image every day at 04:00. Two-factor authentication is opt-in per user; the Community edition can't enforce it.

Monitoring stack​

Loki, Prometheus, Grafana and Uptime Kuma, in /opt/logging-stack, set up with make setup-observability-secops. Grafana is at corais.abair.ie and Uptime Kuma at status.abair.ie; the webserver relays both here. How metrics and logs get in, and how dashboards and access are set up, is covered on Monitoring and Security.

Backups​

A nightly Borg backup at 22:00 of /opt and /var/lib/docker/volumes, to the Hetzner Storage Box. The Gitea Postgres database and the Rocket.Chat MongoDB are dumped first. See Backups and Recovery.

Make targets​

TargetWhat it does
setup-secops-hostBase host (users, SSH, fail2ban, upgrades)
setup-secops-host-firewallnftables (commit-confirm)
setup-secops-host-robot-firewallBroken; don't run
setup-wazuh / update-wazuhWazuh stack
setup-gitea-secops / update-gitea-secopsGitea stack
setup-rocketchat-secops / update-rocketchat-secopsRocket.Chat stack
setup-observability-secops / update-observability-secopsLoki, Prometheus, Grafana, Uptime Kuma
setup-secops-backupBorg backup
enroll-wazuh-secops-hostWazuh agent on this host (reports to the local manager)
wazuh-dashboardSSH tunnel to the Wazuh dashboard

Some secrets are only written by the setup tag, not update. These include the Wazuh passwords and the Rocket.Chat admin password. To rotate one of them, re-run the setup-* target.