Skip to main content

transitioning

Updating CI/CD Pipeline of Existing Projects​

We are transitioning to a faster and more secure CI/CD pipeline using Github Runners. Rather than building images externally on Github and pushing them to our Docker Hub, we will be building those same images on a dedicated VM in our Hetzner server, then pushing locally to the Docker Hub.

This workflow provides a number of advantages, including:

  1. No private keys copied and pasted to Github - a major security improvement.
  2. Bypasses the 100MB limit on uploads proxied through Cloudflare - allows us to proxy all traffic.
  3. Faster build times - resources and build caches are localized on our servers.

Steps to Update​

You will need to complete three steps to migrate to this new pipeline:

  1. Edit CI/CD Github Actions on the project
  2. Edit Secrets and Variables on Github
  3. Push a small change and check the Action has been successful

1. Edit CI/CD Github Actions​

Edit your ci.yml and cd.yml files in the .github/workflows/ folder in the root of your project

ci.yml

name: Node.js CI

on:
  pull_request:
    branches: [main]

jobs:
  build:
    runs-on: self-hosted
    environment: prod

    steps:
      - uses: actions/checkout@v4
      - name: Use Node.js LTS
        uses: actions/setup-node@v4
        with:
          node-version: 18.16.0
      - run: |
          npm ci

cd.yml

name: Publish Docker image

on:
  push:
    branches:
      - main

jobs:
  build_and_push:
    name: Build & Push to Internal Registry
    runs-on: self-hosted
    environment: prod
    steps:
      - name: Check out the repo
        uses: actions/checkout@v4
      - name: Log in to Internal Docker Registry
        uses: docker/login-action@v3
        with:
          registry: ${{ vars.DOCKER_REGISTRY }}
          username: ${{ vars.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Extract metadata (tags, labels) for Docker
        id: meta
        uses: docker/metadata-action@v5
        with:
          images: ${{ vars.DOCKER_REGISTRY }}/${{ vars.PROJECT_NAME }} # Directly push to your Services VM

      - name: Build and push Docker image
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          build-args: |
            NEXT_PUBLIC_SUPABASE_URL=${{ vars.NEXT_PUBLIC_SUPABASE_URL }}
            NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY=${{ vars.NEXT_PUBLIC_SUPABASE_PUBLISHABLE_KEY }}
            NEXT_PUBLIC_FOTHEIDIL_API_URL=${{ vars.NEXT_PUBLIC_FOTHEIDIL_API_URL }}
            FOTHEIDIL_INTERNAL_API_URL=${{ vars.FOTHEIDIL_INTERNAL_API_URL }}
            NEXT_PUBLIC_AUTH_BASE_URL=${{ vars.NEXT_PUBLIC_AUTH_BASE_URL }}
            NEXT_PUBLIC_SITE_URL=${{ vars.NEXT_PUBLIC_SITE_URL }}

  update_services_vm:
    name: Update Services VM Container
    needs: "build_and_push"
    runs-on: self-hosted
    environment: prod
    steps:
      - name: Trigger App Restart on Services VM
        run:
          ssh -o StrictHostKeyChecking=no -p 22102 ${{ vars.USERNAME }}@${{ vars.HOST
          }} "bash ${{ vars.DEPLOY_SCRIPT_PATH }}"

2. Edit Secrets/Variables​

On the Github page for your app, go to:

Settings -> Secrets and variables -> Actions

Delete:

  • SSH_PRIVATE_KEY

Update:

  • DOCKER_REGISTRY -> 10.0.0.2:5000
  • HOST -> 10.0.0.2
  • PORT -> 22102

3. Push a small change and check the Action has been successful​

Make a small edit that can easily be checked in production. Commit and push/merge to main. Check that the action has been successful.

Email sloanjo@tcd.ie with any errors.