Skip to content

Upgrading#

Ticket Board is built from source on the host — there are no published images. Upgrading means git pull, rebuild, and let Compose orchestrate the migrate-then-restart sequence. Schema migrations are forward-only by convention; rollback is by restore from backup.

This page covers the routine upgrade path, the rollback path, and the specifics of cross-major-version Postgres moves (which require more care than the app itself).

Before you start#

A safe upgrade has four prerequisites:

  1. A recent backup. Run make backup and confirm the file exists, is non-empty, and is no older than the changes you want to risk. See Backup and restore.
  2. A clean working tree on the host. Local edits to docker-compose.yml, Caddyfile, or backend source will block a git pull --ff-only. Commit, stash, or extract them to an override file.
  3. A maintenance window. Upgrades involve a brief outage while containers recreate. Plan for 1–2 minutes for a no-migration upgrade, longer for any release with audit_event or tool_invocation_log index work.
  4. Release notes read. Skim the CHANGELOG (or git log since your current ref) for breaking changes, new required env vars, or migration warnings.

Routine upgrade procedure#

The happy path is short — five commands and a smoke test.

# 1. Take a backup.
cd /opt/ticket-board
make backup

# 2. Pull the new release.
git fetch origin
git pull --ff-only origin main

# 3. Check for new env vars in .env.example that are missing from .env.
comm -23 \
    <(grep -oE '^[A-Z_]+=' .env.example | sort -u) \
    <(grep -oE '^[A-Z_]+=' .env | sort -u)
# Empty output means no new keys. Otherwise, add the missing names with values.

# 4. Rebuild images and recreate containers.
docker compose build --pull
docker compose up -d

# 5. Verify.
docker compose ps
curl -s http://localhost/readyz | python3 -m json.tool

docker compose up -d after a rebuild will:

  1. Notice that the backend and frontend images changed and queue them for recreation.
  2. Run the migrate service first (it depends on Postgres being healthy, which it already is).
  3. Wait for migrations to complete successfully before starting the new backend.
  4. Recreate the frontend once the backend is healthy.
  5. Caddy stays up across the whole sequence unless its config file changed.

The result is a brief unavailability window — typically 20–40 seconds — during which the proxy returns 502s while the backend recreates. The session table survives, so logged-in users do not have to log in again unless their session expires during the window.

Drain in-flight webhooks first

Webhook deliveries are in-process (no durable queue in v1). A backend container that's terminated mid-delivery loses any queued attempts not yet committed. If you have a busy webhook subscription and want to be careful, wait for webhook_delivery rows from the last few minutes to settle before upgrading.

Validating the upgrade#

The release smoke checklist exists for exactly this moment. Walk it after every upgrade, even minor ones:

If any step fails, stop and triage before declaring the upgrade complete. A failed smoke step is your best signal that the upgrade introduced a regression while you can still roll back from a known-good backup with minimal data loss.

For zero-downtime confidence the checklist won't give you, watch docker compose logs backend for the first few minutes after an upgrade. The ticket_board.startup line logs every resolved setting (with secrets redacted), so you can confirm you're running the version and configuration you intended.

Rollback#

Schema migrations are forward-only by convention. Alembic technically supports downgrade, and the migration scripts implement it, but rolling a production database backward through DDL is the most error-prone path in the stack — especially if any post-migration writes happened.

The safer rollback path:

  1. Stop the stack. docker compose down. This does not delete pgdata.
  2. Restore the pre-upgrade backup. Follow Backup and restore — Full restore from a logical dump. If the upgrade just happened, the data drift is small.
  3. Check out the previous code. git checkout <previous-tag-or-sha>.
  4. Rebuild and restart. docker compose build && docker compose up -d.
  5. Run the smoke checklist on the old version to confirm the rollback worked.

If the upgrade was an emergency rollback within minutes and you took a backup right before, this is straightforward. If you skipped step 1 of the upgrade procedure (the backup) and want to roll back, see Cold rollback without a fresh backup below.

Cold rollback without a fresh backup#

If no backup exists from the moment before the upgrade, you have three options ordered by preference:

  1. Find a recent enough backup. Last night's daily dump is usually fine — a homelab user generates very little data per day. Accept the data loss for the rows written after the dump.
  2. Run alembic downgrade against the current DB. Works only if every migration since the previous version has a correct downgrade step and no post-migration writes broke the assumptions. The migrate service supports this manually:
docker compose run --rm \
    --entrypoint "/bin/sh -c 'pip install --quiet psycopg2-binary && python -m alembic -c /app/alembic.ini downgrade <revision>'" \
    migrate

Identify <revision> by listing the migration history with the same entrypoint override (the migrate service's default entrypoint runs upgrade head, so you have to override it explicitly):

docker compose run --rm \
    --entrypoint "/bin/sh -c 'pip install --quiet psycopg2-binary && python -m alembic -c /app/alembic.ini history'" \
    migrate
  1. Reverse the migration by hand. Last resort. Connect with docker compose exec postgres psql ... and write the inverse SQL yourself. Document what you did in audit_event so the trail is intact.

Do not skip the backup step in the upgrade procedure. It costs seconds; the alternatives cost hours.

Cross-major-version Postgres upgrades#

Major-version Postgres upgrades (16 → 17, etc.) require pg_upgrade or a logical dump-and-restore. The pgdata directory format is not portable across major versions.

The recommended path:

# 1. Take a logical dump from the old version.
docker compose up -d postgres
make backup
docker compose down

# 2. Copy the old volume aside (Docker has no rename verb — make a sibling).
docker volume create ticket-board_pgdata_pg16
docker run --rm \
    -v ticket-board_pgdata:/from \
    -v ticket-board_pgdata_pg16:/to \
    alpine sh -c 'cp -a /from/. /to/'

# 3. Remove the old volume so the new Postgres image initializes a fresh one.
docker volume rm ticket-board_pgdata

# 4. Edit docker-compose.yml to pin the new major version.
#    services.postgres.image: postgres:17-alpine

# 5. Bring up Postgres on the new version — it creates an empty pgdata.
docker compose up -d postgres

# 6. Restore the dump.
docker compose exec -T postgres \
    psql -U "$POSTGRES_USER" -d "$POSTGRES_DB" \
    < backups/<your-pre-upgrade-dump>.sql

# 7. Bring up the rest of the stack.
docker compose up -d

Keep ticket-board_pgdata_pg16 around until you've confirmed the new stack works end-to-end. It's the only path back if the upgrade goes wrong — remove the new ticket-board_pgdata, copy ticket-board_pgdata_pg16 back over it (using the same docker run pattern), and pin the image back to postgres:16-alpine to bring the old version online.

Do not attempt to run a Postgres 17 container against a Postgres 16 pgdata directory — it will refuse to start and may leave the directory in a state that's harder to recover.

Upgrading the application without touching Postgres#

For application-only upgrades (no schema migration in the release), the sequence is even simpler:

make backup    # always
git pull --ff-only origin main
docker compose build --pull backend frontend
docker compose up -d backend frontend

This recreates only the application containers; Postgres and Caddy stay running. The migrate service runs as part of up -d and exits cleanly with no work to do.

If the release notes confirm no schema change, you can skip the migrate rerun, but it costs nothing — Alembic is idempotent at head.

Upgrading Caddy or its configuration#

The Caddyfile is bind-mounted into the proxy container. After editing it:

# Validate the syntax first.
docker compose exec proxy caddy validate --config /etc/caddy/Caddyfile

# Reload without restarting (graceful — does not drop in-flight requests).
docker compose exec proxy caddy reload --config /etc/caddy/Caddyfile

caddy reload is graceful and zero-downtime. It applies the new config without dropping in-flight connections. Use it instead of docker compose restart proxy whenever possible.

If you change the proxy image (e.g. pin to a different Caddy version), you have to recreate the container:

docker compose up -d proxy

The caddy_data volume preserves ACME certificates across recreations, so this does not trigger a re-issuance.

Upgrade dry-run on a copy#

For high-risk upgrades — major version bumps, big data-model changes — practice the upgrade on a copy first:

  1. Take a backup.
  2. On a separate host (or a different Compose project on the same host using -p ticket-board-test), restore the backup into a fresh stack.
  3. Run the upgrade against the copy.
  4. Walk the smoke checklist.
  5. If it passes, repeat against production.

docker compose -p ticket-board-test up -d creates a parallel stack with isolated volume names (ticket-board-test_pgdata, etc.) and no port conflict (override HTTP_PORT in a separate .env for the test project). This is the safest way to find out what breaks before your live data is on the line.