Skip to main content

Services Staff

services-staff (VMID 119, 10.0.0.19) runs internal staff tools, the Plausible analytics server, and the language back-end services that the API gateway calls. It is set up the same way as services-user: each app is a Docker Compose stack in /opt/apps/<app>/.

Playbookansible/hetzner-main/setup-services-staff.yml
App definitionsansible/hetzner-main/services-staff-apps/<app>/
Make targetssetup-services-staff, update-services-staff, deploy-services-staff-apps, deploy-analytics-plausible, enroll-wazuh-services-staff
SecretsDocker Registry Password, Webserver htpasswd admin, and one Secure Note per app that needs a .env

Apps​

App (directory)Public hostnamePort.env from VaultwardenNotes
backofficebackoffice.abair.ie3007yes
backoffice-servicesbackoffice-api.abair.ie4000yesUploads up to 1 GB.
translations-editortranslations.abair.ie3006noUses the :latest tag.
culcul.abair.ie3004yesUses the :latest tag. Resolves mail.abair.ie to the legacy services VM (10.0.0.2).
irishfstirishfst.abair.ie6789noIrish part-of-speech tagger. The grammar is built into the image. The API gateway also calls it directly.
bunamointernal only9000noBuNaMo Irish noun-form API.
gramsrvinternal only10001noA checker for the genitive and relative clauses. Calls irishfst and bunamo on this VM.
gramadoirinternal only10002noAn Gramadóir grammar checker (a TypeScript reimplementation), with a gramadoir-pos sidecar that runs the original Perl part-of-speech tagger.
analytics-plausibleanalytics.abair.ie8002yesPlausible Community Edition v3.2.0 with Postgres 16 and ClickHouse 24.12. Deployed separately, see below.

"Internal only" means there is no public hostname. The API gateway reaches the service directly at 10.0.0.19:<port>.

Two different "gramadóir" services

The public gramadoir.abair.ie points at the unmanaged scealai VM, which runs a separate, older service. The gramadoir container here has no public hostname and is only reached through the API gateway.

How apps are deployed​

make deploy-services-staff-apps works exactly like it does on services-user. For every app in staff_apps, it copies the app directory and fetches <app> .env for apps in staff_apps_with_env (currently cul, backoffice and backoffice-services). Then it runs docker compose pull and docker compose up -d.

The deploy user on this VM is authorized with the SSH keys of both CI runners on docker-build, GitHub and Gitea. If the Gitea runner's key doesn't exist yet, it is skipped. Run make update-services-staff again once it does.

Plausible analytics​

Plausible holds real data (Postgres and ClickHouse), so it has its own tag and target and isn't part of deploy-services-staff-apps:

make deploy-analytics-plausible

This fetches the analytics-plausible .env note, copies the stack (including the ClickHouse tuning files in clickhouse/), fixes the ownership of plausible-data/, and starts the stack. The data directories (postgres-data/, clickhouse-data/, clickhouse-logs/, plausible-data/) were moved over from the legacy services VM by hand. They aren't in the repo, and Ansible doesn't create them.

Base setup and firewall​

The base setup is the same as on services-user: Docker, the deploy user, the registry login and Alloy. Docker restarts on every setup or update run. UFW (setup only) allows:

  • SSH from anywhere;
  • all ports from the webserver (10.0.0.14);
  • all ports from the API gateway (10.0.0.17), for irishfst, bunamo, gramsrv and gramadoir.

Everything else is denied.

Resource history​

  • Raised from 2 CPUs and 2 GB to 4 CPUs and 4 GB on 2026-07-22, when the VM stalled under memory pressure.
  • Memory raised again, to 8 GB, on 2026-08-17, before Plausible moved here.
  • Both changes were made cold on the Proxmox host (qm set 119 ... and a reboot), then copied into services-staff.tf.