Skip to main content

Networking and Ingress

Networks​

NetworkRangeWhat's on it
hetzner-main VM network (vmbr1)10.0.0.0/24All hetzner-main VMs. The gateway is the Proxmox host at 10.0.0.254.
Amergin vSwitch (VLAN 4000)192.168.100.0/24Proxmox host 192.168.100.1, Amergin 192.168.100.2
Public internetProxmox host 136.243.40.234, Amergin 5.9.9.57, Secops 135.181.120.84, Vaultwarden 91.99.99.28
Lab network (TCD)Puca 10.7.160.28, setanta, the NAS

The VMs have no public addresses. They reach the internet through NAT on the Proxmox host, so outbound traffic from any VM comes from 136.243.40.234. Other servers' firewalls use that address when they need to allow "hetzner-main".

How a web request gets in​

  1. Cloudflare provides DNS for abair.ie. Most hostnames are proxied through Cloudflare (orange cloud).
  2. The request reaches the Proxmox host's public IP. Its firewall forwards ports 80 and 443, and nothing else, to the webserver VM.
  3. nginx on the webserver terminates TLS with the shared wildcard certificate and matches the hostname to a vhost in networking/nginx-config/.
  4. nginx forwards the request to the back end: a VM on 10.0.0.0/24, Amergin over the vSwitch, or Secops over the internet.

Two hostnames skip the webserver completely. Their DNS points straight at their own server, which terminates TLS itself:

  • gitea.abair.ie and gitea-direct.abair.ie go to Secops.
  • vault.abair.ie goes to Vaultwarden.

Public hostnames​

Every vhost below is a file in networking/nginx-config/. Every one redirects HTTP to HTTPS.

Apps on services-user (10.0.0.18)​

HostnamePortNotes
abair.ie, www.abair.ie, dev.abair.ie3001abair.ie and www.abair.ie set a Content-Security-Policy and other security headers. www.abair.ie/api2/ goes to synthesis-legacy.
preview.abair.ie4001
litreoir.abair.ie3002
fotheidil.abair.ie3003Uploads up to 100 MB
mileglornanog.abair.ie3005
meallanoige.abair.ie3009mao.abair.ie and xn--meallanige-mbb.abair.ie (meallanóige) redirect here
webreader.abair.ie3010
aac-editor.abair.ie3013
mao-portal.abair.ie3014
leighdom.abair.ie3015Uploads up to 64 MB; returns a JSON error if too large
bat-mirialta.abair.ie8001
giob-geab.abair.ie8003
geabaire-api.abair.ie, api.geabaire.abair.ie4041Only /v1/ is proxied

Apps on services-staff (10.0.0.19)​

HostnamePortNotes
translations.abair.ie3006
backoffice.abair.ie3007
backoffice-api.abair.ie4000Uploads up to 1 GB, 120 s timeouts
cul.abair.ie3004
irishfst.abair.ie6789
analytics.abair.ie8002Plausible

API and speech services​

HostnameGoes to
api.abair.ie/v4/ → API gateway 10.0.0.17:4004; /v4/docs and / → docs site 10.0.0.17:5100; /v3/ → legacy abair-api on 10.0.0.18:4040. See API Gateway.
streaming-asr.abair.ieTriton on Amergin, 192.168.100.2:8080 (WebSocket, 24-hour timeouts)
fotapi.abair.ieFotheidil API on Amergin, 192.168.100.2:4040 (uploads up to 100 MB)
recognition.abair.ie/v3-5/ → batch ASR on Amergin, 192.168.100.2:8002. / → the legacy 10.0.0.8:6060. 6 MB upload limit.
synthesis.abair.ie/api/, /piper/, /lts/ → synthesis-legacy (10.0.0.21 ports 8002, 8001, 2029); /piper-new-lts/ → legacy 10.0.0.5:9001; /audio → static files on the webserver

Sites hosted on Secops​

These are proxied by the webserver to Secops's public IP over HTTPS. Secops terminates TLS again with its own copy of the wildcard certificate.

HostnameService
corais.abair.ieGrafana (plus the Loki and Prometheus push endpoints)
status.abair.ieUptime Kuma. / redirects to /status/main.
alerts.abair.ieRocket.Chat

Because nginx connects to a bare IP address, these vhosts set proxy_ssl_verify on, proxy_ssl_server_name on and proxy_ssl_name <hostname>. nginx doesn't verify upstream certificates by default. Without these settings it would connect without checking who it's talking to.

Infrastructure and other sites​

HostnameGoes to
proxmox.abair.ieProxmox web UI, 10.0.0.254:8006
registry.abair.ieDocker registry on docker-build, 10.0.0.12:5000 (no upload size limit)
scealai.abair.ie, gramadoir.abair.iescealai VM, 10.0.0.6
lara.abair.iescealai VM, 10.0.0.6
mail.abair.ie (plus autodiscover, autoconfig)legacy services VM, 10.0.0.2
old.abair.ielegacy services VM 10.0.0.2:3000; /api2/ → synthesis-legacy
screenreader.abair.ie, srv.abair.iestatic files on the webserver

Legacy upstreams​

These routes still point at machines outside the managed estate:

UpstreamUsed by
10.0.0.2 (legacy services)mail.abair.ie, old.abair.ie
10.0.0.5 (old synthesis VM)synthesis.abair.ie/piper-new-lts/
10.0.0.8:6060 (old recognition proxy)recognition.abair.ie/
10.0.0.6 (scealai)scealai.abair.ie, gramadoir.abair.ie, lara.abair.ie

If one of those machines is switched off, its routes will fail.

TLS certificates​

  • The webserver holds one Let's Encrypt certificate for abair.ie, *.abair.ie, *.geabaire.abair.ie and *.scealai.abair.ie, issued with Certbot's Cloudflare DNS-01 challenge (Cloudflare DNS Token). A new public hostname needs no certificate work of its own.
  • Secops and Vaultwarden each issue their own abair.ie / *.abair.ie certificate the same way.
  • www.srv.abair.ie is two levels deep, so the wildcard doesn't cover it.

DNS​

DNS records are managed by hand in the Cloudflare dashboard. Nothing in the repo creates them. A new public site needs a proxied record pointing at the Proxmox host's public IP. See Adding an App.

The Amergin vSwitch​

Amergin connects to hetzner-main over a Hetzner vSwitch (VLAN 4000). The vSwitch is routed, not bridged into the VM network:

Proxmox hostAmergin
Interfaceenp6s0.4000enp4s0.4000
Address192.168.100.1/24192.168.100.2/24
MTU14001400
RoutingIP forwarding on; VMs send 192.168.100.0/24 traffic through their normal gateway, 10.0.0.254Static route to 10.0.0.0/24 through 192.168.100.1

Hetzner vSwitches need MTU 1400, but the VM network uses 1500. Bridging the two would silently drop large packets, such as audio and video uploads to Fotheidil. Instead the Proxmox host routes between them and clamps TCP MSS to 1360 on the vSwitch interface, so TCP connections never send packets too big to cross it.

The Proxmox host's firewall lets everything through between the VM network and the vSwitch. Access to Amergin's ports is controlled on Amergin itself: every service binds only to 192.168.100.2, and UFW lists the allowed callers. See the port table.

Firewalls​

ServerFirewallInbound rules
Proxmox hostnftablesSSH from admin IPs. Everything from the VM network and the vSwitch. 80/443 forwarded to the webserver.
webserverUFW22, 80, 443 from anywhere
api-gatewayUFW22 from anywhere; 4004 and 4005 from the webserver
services-userUFW22 from anywhere; all ports from the webserver
services-staffUFW22 from anywhere; all ports from the webserver and the API gateway
docker-build, synthesis-legacynone
AmerginUFW22 from anywhere; each service port from its listed callers
SecopsnftablesSee Secops → Firewall
Vaultwardennftables22, 80, 443
PucaUFW22 from listed lab addresses only

The Proxmox host's firewall already blocks every inbound port except 80 and 443 (which go to the webserver). So the VMs aren't reachable from the internet whatever their own firewall says. The VM firewalls only matter for traffic between VMs.

UFW doesn't filter Docker-published ports

Docker writes its own iptables rules for published container ports (ports: "3002:3000"), and those rules take effect before UFW's. So on the VMs, a published container port is reachable from anywhere on the private network, whatever UFW says. For example, the API gateway's docs site on 5100 works through the webserver even though UFW has no rule for it. On these hosts, the real controls are the Proxmox host firewall and, on Amergin, binding ports to the vSwitch address only. On Secops and Vaultwarden, every container binds to 127.0.0.1 for the same reason.