Please use a larger screen to view this site.
Technical reference for the helldivers.bot infrastructure layer: where the app runs, how a change reaches it, how the images are built, how the app starts, and which environment variables it reads.
graph LR
subgraph App["Application Container"]
direction TB
SERVER["server.js"] --> |"instrumentation.js"| REG["register()"]
REG --> MIG["migrateOrWait"]
MIG --> |"advisory lock"| DEPLOY["prisma migrate deploy"]
DEPLOY --> |"seed hash changed"| SEED["seed.mjs"]
SEED --> WORKER["Worker Thread"]
DEPLOY --> |"seed unchanged"| WORKER
WORKER --> |"setTimeout loop"| POLL["Poll Official API"]
end
style App fill:#0f1a0f,stroke:#22c55e,color:#4ade80
Two Docker Swarm stacks run the app. Both stacks have the same shape and the same stack name, helldiversbot.
| Production | Staging | |
|---|---|---|
| Cluster | Docker Swarm on Hetzner | Docker Swarm on three Raspberry Pi nodes |
app service | 2 replicas | 2 replicas |
cloudflared service | 2 replicas | 2 replicas |
| Database | One Postgres database outside the stack, shared by both app replicas | One Postgres database outside the stack, shared by both app replicas |
UPDATE_INTERVAL | 20 seconds | 20 seconds |
| Image pin | ghcr.io/elfensky/helldiversbot:vX.Y.Z | ghcr.io/elfensky/helldiversbot:sha-<commit> |
| Deployed from branch | deploy/production | deploy/staging |
Each stack has a compose.yaml under deploy/. Arcane Git Sync reads that file from the CI-owned branch and applies it. The stack is read-only in the Arcane interface, and nobody runs docker stack deploy by hand. A rollback is a re-pin on the git side. The image: line on develop is not what runs, because CI writes the real pin on the deploy branch.
Each stack has its own Cloudflare Tunnel. The cloudflared replicas dial out to Cloudflare, so the stacks publish no ports. Both replicas register as connectors on the same tunnel. They reach the app at http://app:3000 on the internal overlay network. Swarm spreads new connections over the healthy app replicas. The hostname routing is set in the Cloudflare Zero Trust dashboard, not in the compose files.
Every app replica runs the cron thread. Each poll first claims the single worker_heartbeat lease row (src/update/lease.mjs). Postgres serializes the claim, so one replica holds the lease and polls the official API. The other replicas answer 200 {role:'standby'}. A dead holder is replaced within the 60 s lease TTL, with no election. The poller state (prev_events, last_season_observed) lives in the same row, so a handover neither misses nor repeats a transition.
The lease, not the placement, picks the poller. For this reason:
docker service scale needs no other change.WORKER_ENABLED=false is an optional opt-out for one replica. It is not needed to scale.Both stacks use order: start-first. A new task starts and must pass the image health check before an old task stops. The old and the new build therefore serve side by side for a short time. A request for a /_next/static chunk of the new build can reach an old replica and get a plain-text 404. A later request that reaches a new replica gets the 200, and Cloudflare caches it with cache-control: public, max-age=31536000, immutable. This was observed on a staging rollover.
Every task also applies pending migrations before it serves (see Migrate on start in Section 1). A migration must stay backward compatible for one release, because of the side-by-side window.
v1.2.3 tag on main
-> Build: Release
Test & Build (verify the tagged commit)
Build App Image (amd64) + Build App Image (arm64) -> Publish App Manifest
Build Migrate Image (amd64) + (arm64) -> Publish Migrate Manifest
Create GitHub Release (body from CHANGELOG.md)
Pin Production -> force-pushes deploy/production
-> Arcane Git Sync applies deploy/production
vX.Y.Z tag on main starts Build: Release.ubuntu-latest and arm64 on ubuntu-24.04-arm. Publish App Manifest joins the legs as :<tag>, :production and :latest. The migrate image follows the same steps.deploy/production/compose.yaml and force-pushes it to the CI-owned branch deploy/production.v0.98.1).Rollback is a manual dispatch of Build: Release with the tag of an earlier release. A dispatch skips the builds and runs only Pin Production.
develop runs Check: CI. When Check: CI succeeds, Build: Staging starts.<version>-dev.<sha7> through the APP_VERSION build argument. For this reason every staging build is a distinct version for the stale-tab reload and for the error tracker.:staging and :sha-<commit>. It builds the migrate image only when migration-related files changed.sha-<commit> into deploy/staging/compose.yaml and force-pushes it to deploy/staging.docker-compose.yml in the repository root is a single-host compose file for local or self-hosted use. It is not the production deployment, and it is not what either Swarm stack runs. See the docker-compose.yml subsection of Section 1.
The project uses two Dockerfiles. One image serves and migrates: the app image applies pending migrations on start. Dockerfile.migrate builds the manual and emergency image.
Images: ghcr.io/elfensky/helldiversbot-migrate:staging (Build: Staging) and :<tag>, :production, :latest (Build: Release)
Purpose: Manual and emergency one-shot. It runs prisma migrate deploy and the seed once and exits. It never stays alive. The app image runs the same two steps on start.
Build process:
node:26-alpinetini via apk (init system for zombie process prevention)WORKDIR /apppackage.json to /tmp as a version reference, so npm never reads it as the manifest. Copy package-lock.json, prisma/, prisma.config.mjs, and the two modules the seed imports (src/validators/isValidSeason.mjs, src/shared/enums/events.mjs)package.json in /app with the project's ranges for five packages only (prisma, @prisma/client, @prisma/adapter-pg, dotenv, zod) and the project's overrides. Run npm install next to the project lockfile, so every version, transitive ones included, keeps the locked version and integrity hash. No full npm cinpx prisma generate to produce the client, then remove the /tmp reference and the npm cache in the same layerUSER node/sbin/tini --npx prisma migrate deploy && node --experimental-strip-types prisma/seed/seed.mjs — runs migrations first, then seeds historical season data from JSON files in prisma/seed/seasons/. The seed uses upserts and is idempotent (safe to re-run on every deploy). It compares each file's season against the seasons already in h1_season and upserts only the missing ones, so a deploy that adds one season file writes one season rather than re-running all of them (set FORCE_SEED=true on the migrate container to re-upsert everything — only true, 1 or yes count, so false correctly means off). If no season files exist, the seed exits gracefully.The seasons/*.json files are produced by prisma/seed/fetch-seasons.mjs — a one-shot refresh script developers run on their workstations to pull the latest state from the official HD1 API. Developers then commit the updated files to git. The fetch script never captures the currently-active season (its get_snapshots response is a mid-war partial by definition), so --to defaults to currentSeason - 1 and is clamped to that maximum even if specified higher. Refresh the files periodically via node prisma/seed/fetch-seasons.mjs --force to keep disk data in parity with the API's occasional "closing frame" writes after a season ends. See prisma/seed/readme.md for the full workflow.
The minimal manifest comes from the project manifest with a one-line Node script:
COPY package.json /tmp/project-package.json
COPY package-lock.json ./
RUN node -e "const p = require('/tmp/project-package.json'), all = { ...p.dependencies, ...p.devDependencies }; \
const deps = Object.fromEntries(['prisma', '@prisma/client', '@prisma/adapter-pg', 'dotenv', 'zod'].map((n) => [n, all[n]])); \
require('fs').writeFileSync('package.json', JSON.stringify({ name: 'migrate', version: '1.0.0', private: true, type: 'module', dependencies: deps, overrides: p.overrides }))" && \
npm install --no-audit --no-fund && \
POSTGRES_URL=postgresql://dummy npx prisma generate
Prisma 7 CLI no longer auto-loads .env files. The prisma.config.mjs file imports dotenv/config to handle local env loading; in Docker, POSTGRES_URL comes from the environment. The Swarm stacks set POSTGRES_URL_FILE and read the value from a secret (see the *_FILE convention in Section 5).
This keeps the migrate image small — it carries only the Prisma CLI, not the entire application dependency tree.
Images: ghcr.io/elfensky/helldiversbot:staging and :sha-<commit> (Build: Staging); :<tag>, :production and :latest (Build: Release). Both are multi-arch manifests of an amd64 and an arm64 image.
Purpose: Runs the Next.js standalone server. Before it serves, it applies pending migrations, and the seed when its data changed (see Migrate on start).
Build stages:
| Stage | Base | What it does |
|---|---|---|
base | node:26 | Debian (glibc) build base, so Sharp's prebuilt glibc binary matches the runtime |
deps | base | Runs npm ci from lockfile; fails explicitly if lockfile is absent |
builder | base | Copies node_modules from deps, copies source, runs npx prisma generate then npm run build |
migrate-tools | node:26-slim | Mirrors Dockerfile.migrate (Prisma CLI, schema engine, seed, generated client), then deletes PGlite and the query compilers of other providers; copied to /app/migrate |
runner | node:26-slim | Copies only .next/standalone, .next/static, public and /app/migrate; runs as a non-root numeric user |
Build inputs (builder stage):
NEXT_PUBLIC_DEPLOY_ENV, APP_VERSION, NEXT_PUBLIC_SENTRY_DSN and NEXT_PUBLIC_VAPID_PUBLIC_KEY. Next.js inlines the NEXT_PUBLIC_* values into the client bundle, so they must exist before npm run build. All four are public values.APP_VERSION replaces the package.json version in next.config.mjs. Build: Staging passes <version>-dev.<sha7>. Build: Release passes nothing, so the package.json version stays.sentry_auth_token, sentry_url, sentry_org and sentry_project are BuildKit secrets, not build arguments. A build argument lands in the provenance attestation that is published next to the image. A secret exists only for the one RUN that reads it. See "next.config.mjs Sentry settings" in Section 4 for what the build does with them.RUN uses cache mounts for the npm download cache and for .next/cache. The cache lives in BuildKit storage and never in the image.Runner stage details:
ARG NODE_ENV=production — overridable at build time. Build: Staging and Build: Release both pass NODE_ENV=production.node:26-slim (Debian, glibc). The slim image has a shell and apt, so its surface is larger; the app does not call a shell.65532:65532 (USER 65532:65532). A numeric id needs no /etc/passwd lookup.--chown=65532:65532, so .next stays writable for the image optimizer's runtime cache. /app/migrate stays root-owned: the app user can read and run the tooling, not change it.org.opencontainers.image.source, org.opencontainers.image.licenses, org.opencontainers.image.title, version, descriptionENTRYPOINT of its own. The node:26-slim entrypoint script runs the command. The migrate image above keeps its init wrapper.["node", "server.js"] (the Next.js standalone output file)EXPOSE 3000, ENV PORT=3000, ENV HOSTNAME="0.0.0.0", ENV MIGRATE_ROOT=/app/migrate, ENV CHECKPOINT_DISABLE=1 (the Prisma CLI runs on every start and makes no update check)fetch of /api/healthcheck every 30s, timeout 5s, start period 900s, 3 retries. Failures inside the start period do not count, and the first healthy answer ends it. Each call also refreshes the replica's app_instance row after the response (at most once per 20 s per process); a failed write never changes the 200/503 answer.next/standalone stay in the image. The app does not serve them over HTTP.helldiversbot
image: ghcr.io/elfensky/helldiversbot:staging
platform: linux/amd64
env_file: .env.development
ports: 127.0.0.1:50001:3000
restart: unless-stopped
healthcheck: Node fetch of /api/healthcheck every 60s, timeout 10s, 3 retries, 300s start_period
This file runs one container on one host. It uses the :staging image, reads its variables from a local env file, and has no replicas, no cloudflared and no database. The compose healthcheck replaces the image healthcheck. Neither Swarm stack uses it; see Deployment topology.
The port binding 127.0.0.1:50001:3000 deliberately limits exposure to the host loopback interface. External traffic must arrive through a reverse proxy (e.g., nginx or Caddy) on the host.
Check: Docker Smoke uses a separate compose file. It builds the app image from the working tree and starts it with a Postgres container.
Every app process runs migrateOrWait() (src/update/migrateOnStart.mjs) from instrumentation.js, before the worker starts and before Next serves a request.
pg session takes pg_advisory_lock(599000001). Replicas take turns, so Prisma's own fixed 10 s migrate lock never contends.prisma migrate deploy, then the seed. The session ends afterwards, which releases the lock on success and on failure.prisma/seed/seed.mjs plus prisma/seed/seasons/*.json differs from the hash in the seed_state table. A failed seed stores nothing, so the next attempt seeds again.migrateOrWait() never throws. A failure logs [migrate] failed: …, reports to GlitchTip with stage migrate-on-start, and retries after 30 s, 60 s, 2 min, then every 5 min.register() resolves. A migrating task stays Starting, and Swarm's start-first update keeps the old tasks serving.MIGRATE_ROOT names the folder with the Prisma CLI and the prisma/ tree. The image sets /app/migrate; the default is the process cwd.Every change reaches develop and main through a pull request. Two workflows build and publish images. Nine others check code, clean the registry, or refresh data.
graph LR
subgraph Staging["STAGING PIPELINE"]
PUSH["Merge to develop"] --> CI["Check: CI<br/><small>lint · typecheck · test · build · smoke</small>"]
CI -->|"green only"| DETECT["Detect Migration Changes"]
CI -->|"green only"| BUILD_APP["Build App Image<br/><small>amd64 and arm64 legs</small>"]
DETECT -->|"migration files changed"| BUILD_MIG["Build Migrate Image<br/><small>amd64 and arm64 legs</small>"]
BUILD_APP --> MERGE_APP["Publish App Manifest<br/><small>:staging and :sha-commit</small>"]
BUILD_MIG --> MERGE_MIG["Publish Migrate Manifest<br/><small>:staging</small>"]
MERGE_APP --> BUMP["Bump Staging Tag<br/><small>force-push deploy/staging</small>"]
BUMP --> ARC_S["Arcane Git Sync"]
end
subgraph Production["PRODUCTION PIPELINE"]
TAG["Push version tag<br/><small>v*.*.* on main</small>"] --> VERIFY["Test and Build<br/><small>the tagged commit</small>"]
VERIFY --> BUILD_PROD["Build App Image<br/><small>amd64 and arm64 legs</small>"]
VERIFY --> BUILD_MIGP["Build Migrate Image<br/><small>amd64 and arm64 legs</small>"]
BUILD_PROD --> MERGE_PROD["Publish App Manifest<br/><small>:tag, :production, :latest</small>"]
BUILD_MIGP --> MERGE_MIGP["Publish Migrate Manifest"]
MERGE_PROD --> RELEASE["Create GitHub Release"]
MERGE_MIGP --> RELEASE
MERGE_PROD --> PIN["Pin Production<br/><small>force-push deploy/production</small>"]
MERGE_MIGP --> PIN
PIN --> ARC_P["Arcane Git Sync"]
end
style Staging fill:#1c1917,stroke:#f59e0b,color:#fbbf24
style Production fill:#0f1a0f,stroke:#22c55e,color:#4ade80
The repository has 11 workflow files in .github/workflows/ and two composite actions, .github/actions/verify (lint, typecheck, unit tests, build) and .github/actions/docker-publish (Buildx build and push to GHCR).
| File | name: | Trigger | Purpose |
|---|---|---|---|
build-release.yml | Build: Release | Push of a v*.*.* tag; manual dispatch with an existing tag | Verify the tag, build and publish the app and migrate images, create the GitHub Release, pin deploy/production |
build-staging.yml | Build: Staging | Check: CI completes on develop (workflow_run); manual dispatch | Build and publish the staging images, pin deploy/staging, remove untagged images |
check-ci.yml | Check: CI | Pull request or push on main and develop | Job Test & Build: lint, typecheck, unit tests, build, then smoke tests. Job Visual Regression: committed visual baselines |
check-codeql.yml | Check: CodeQL | Pull request or push on main and develop; Mondays 06:00 UTC | CodeQL analysis (job Analyze) |
check-dependencies.yml | Check: Dependencies | Pull request on main and develop | Dependency review (job Review Dependencies) |
check-docker-smoke.yml | Check: Docker Smoke | Pull request on main | Build the app image from the working tree, start it with an empty Postgres, run the smoke tests |
check-migrate-arm64.yml | Check: Migrate Image (arm64) | Pull request on develop and main that touches the migrate image inputs | Build Dockerfile.migrate for linux/arm64 and run it |
check-version.yml | Check: Version Bump | Pull request on main that touches src/**, prisma/**, public/** or package.json | Fail when the package.json version equals the version on main; skipped for Dependabot |
cleanup-ghcr.yml | Cleanup: GHCR | Mondays 04:17 UTC; manual dispatch (dry run by default) | Keep the newest 3 sha-* and the newest 3 v* images per package; delete untagged leftovers; never delete floating tags |
scheduled-pagespeed.yml | Scheduled: Pagespeed | Mondays 00:00 UTC, Fridays 06:00 UTC; manual dispatch | Render a PageSpeed Insights SVG for https://helldivers.bot and commit it to the standalone metrics branch |
scheduled-seed-refresh.yml | Scheduled: Seed Refresh | Mondays 03:00 UTC; manual dispatch | Run prisma/seed/fetch-seasons.mjs; when new completed seasons appear, push chore/seed-refresh and open a PR on develop |
The name: of Check: CI is load-bearing. build-staging.yml matches its workflow_run trigger on that name, not on the file name. The two change together.
Every image build runs one leg per architecture on a runner of that architecture: amd64 on ubuntu-latest, arm64 on ubuntu-24.04-arm. QEMU emulation is not used, because prisma generate fails under emulated aarch64. Each leg pushes a per-architecture tag. A Publish Manifest job joins the two tags and checks that both platforms exist.
build-staging.yml)Trigger: Successful completion of Check: CI on a push to develop (workflow_run), or manual workflow_dispatch.
The build starts only after Check: CI is green, so a red CI never ships an image. workflow_run resolves the workflow file from the default branch (develop), so an edit takes effect only after it is merged there. Every checkout pins TESTED_SHA, the commit that Check: CI tested, because a workflow_run checkout otherwise defaults to the branch tip, which may have moved on. A second run queues behind the first (cancel-in-progress: false), because Bump Staging Tag force-pushes deploy/staging.
| Job | Condition | Result |
|---|---|---|
| Detect Migration Changes | Always | Compares the commit pinned on deploy/staging with the tested commit; outputs whether migrate inputs changed |
| Build Migrate Image | Prisma files, prisma.config.mjs, package.json, package-lock.json or Dockerfile.migrate changed; or the image is missing; or a manual dispatch | helldiversbot-migrate:staging after Publish Migrate Manifest |
| Build App Image | Always | Two legs, with NODE_ENV=production, NEXT_PUBLIC_DEPLOY_ENV=staging and APP_VERSION=<version>-dev.<sha7>; the Sentry credentials are BuildKit secrets |
| Publish App Manifest | After both legs | helldiversbot:staging and helldiversbot:sha-<commit> |
| Bump Staging Tag | After Publish App Manifest | Writes sha-<commit> into deploy/staging/compose.yaml and force-pushes deploy/staging |
| Cleanup Untagged Images | After the builds, unless the app build was skipped | Deletes untagged versions older than 30 minutes with snok/container-retention-policy |
Registry auth: secrets.GITHUB_TOKEN with packages: write. The cleanup job uses secrets.GHCR_CLEANUP_TOKEN, a classic personal access token with delete:packages. The default token cannot delete user-account packages.
build-release.yml)Trigger: Version tags matching v*.*.*, or a manual dispatch with an existing tag. A dispatch skips the builds and runs only Pin Production (see Rollback in Deployment topology).
| Job | Needs | Result |
|---|---|---|
| Test & Build | --- | Runs the verify composite action against the tagged commit |
| Build App Image | Test & Build | Two legs, with NODE_ENV=production and NEXT_PUBLIC_DEPLOY_ENV=production |
| Publish App Manifest | Build App Image | helldiversbot:<tag>, :production, :latest |
| Build Migrate Image | Test & Build | Two legs; always built, because every tag needs a versioned migrate image |
| Publish Migrate Manifest | Build Migrate Image | helldiversbot-migrate:<tag>, :production, :latest |
| Create GitHub Release | Both manifests | The release body is the CHANGELOG.md section of the version, extracted with awk, plus the image pull commands |
| Pin Production | Both manifests (or a dispatch) | Checks that both images exist, writes the tag into deploy/production/compose.yaml, force-pushes deploy/production |
main requires the Test & Build check to merge, so release code has normally passed. A tag can still point at any commit, and Check: CI does not run on tags. For this reason Build: Release repeats the same gate on the tagged commit. It shares the steps through a composite action, not a reusable workflow, because a reusable workflow renames the check to <caller> / Test & Build and the required status check matches the exact name.
Version: the image tags come from the tag name. Build: Release passes no APP_VERSION, so the app reports the package.json version.
Registry auth: secrets.GITHUB_TOKEN with packages: write. The release job has contents: write to create the GitHub Release, so no personal access token is needed.
graph TD
REG["register()"] -->|"SENTRY_DSN set"| SENTRY["load sentry.server.config"]
REG -->|"NEXT_RUNTIME=nodejs"| INIT["initializeHelldivers1Api()"]
SENTRY --> INIT
INIT --> ENV["initializeEnvironmentVariables()"]
ENV -->|"required variable missing"| CRASH1["💥 Process crash"]
ENV -->|"core vars present"| MIG["migrateOrWait()<br/><small>advisory lock · migrate · seed</small>"]
MIG -->|"failure: log, report, retry"| MIG
MIG --> WORKER["initializeWorker()"]
WORKER -->|"WORKER_ENABLED=false"| WEB["Web only<br/><small>no worker thread</small>"]
WORKER -->|"Worker spawned"| POLL["Worker polling loop<br/><small>setTimeout(doWork, interval)</small>"]
WORKER -->|"Spawn failed"| CRASH3["💥 Process crash"]
POLL -->|"thread exits outside shutdown"| CRASH4["💥 exit code 1"]
style REG fill:#1e293b,stroke:#3b82f6,color:#60a5fa
style CRASH1 fill:#2d1b1b,stroke:#ef4444,color:#f87171
style CRASH3 fill:#2d1b1b,stroke:#ef4444,color:#f87171
style CRASH4 fill:#2d1b1b,stroke:#ef4444,color:#f87171
style POLL fill:#0f1a0f,stroke:#22c55e,color:#4ade80
Entry point: src/instrumentation.js — Next.js calls register() automatically on server startup via the instrumentation hook. The Node.js steps live in src/instrumentation.node.js.
register() [src/instrumentation.js]
│
└── NEXT_RUNTIME === 'nodejs'
├── SENTRY_DSN set → import sentry.server.config.js → Sentry.init() for server runtime
│
└── initializeHelldivers1Api() [src/instrumentation.node.js]
│
├── Step 1: initializeEnvironmentVariables() [src/shared/utils/initializeEnv.mjs]
│ ├── hydrateFileSecrets() → <KEY>_FILE populates <KEY> [src/shared/utils/hydrateFileSecrets.mjs]
│ ├── validateDatabase() → POSTGRES_URL ← REQUIRED, throws
│ ├── validateUpdates() → UPDATE_KEY, UPDATE_INTERVAL ← REQUIRED, throws (PORT optional)
│ ├── checkAnalytics() → UMAMI_SITE_ID, SENTRY_DSN, SENTRY_AUTH_TOKEN ← OPTIONAL, warns
│ ├── checkAuth() → BETTER_AUTH_SECRET + 7 auth vars ← OPTIONAL, warns (partial = throws)
│ └── Returns { auth: boolean, analytics: boolean }
│
├── Step 2: migrateOrWait() [src/update/migrateOnStart.mjs]
│ ├── skipped under Vitest (VITEST set or NODE_ENV=test)
│ ├── takes the advisory lock, runs prisma migrate deploy, then the seed when its data changed
│ └── never throws: a failure logs, reports and retries after 30 s, 60 s, 2 min, then every 5 min
│
└── Step 3: initializeWorker() [src/shared/utils/initializeWorker.mjs]
├── WORKER_ENABLED=false (also 0, no, off) → logs, returns true, spawns no thread
├── Resolves worker path:
│ ├── development: path.resolve(process.cwd(), 'public/workers/cron.js')
│ └── production: path.resolve('/app/public/workers/cron.js')
├── new Worker(workerPath)
├── worker.postMessage({ key, interval, port })
├── Attaches message/error/exit handlers; an exit outside a shutdown calls process.exit(1)
├── Registers SIGINT/SIGTERM handlers that terminate the worker before exit
└── false → throw (crashes the process)
Three failures crash the process. An environment error from Step 1 throws. A falsy return from initializeWorker() throws. A worker thread that exits outside a shutdown calls process.exit(1), because the container health check is a database ping and a dead thread would otherwise leave a healthy task that never polls. Since Next.js does not catch errors thrown from register(), a throw ends the process. The orchestrator restarts the container: restart: unless-stopped in docker-compose.yml, the Swarm restart policy in the stacks. The underlying problem (missing env var, bad database) then gets a chance to be resolved.
A migration failure does not crash the process. migrateOrWait() retries, and Next holds requests until register() resolves (see Migrate on start in Section 1).
src/instrumentation.js also exports:
export const onRequestError =
process.env.SENTRY_DSN ? Sentry.captureRequestError : () => {};
Next.js calls this hook for errors that occur inside Server Components, middleware, and proxied routes — errors that do not surface through the normal React error boundary. This is the server-side equivalent of global-error.jsx. The DSN-presence gate means localhost reports too when SENTRY_DSN is set in .env.development; environments stay distinguishable via the environment tag.
The project uses the Sentry SDK (@sentry/nextjs) but targets a self-hosted GlitchTip instance rather than Sentry SaaS. GlitchTip is Sentry-protocol-compatible and supports error aggregation and performance traces but not session replay.
| File | Runtime | Role |
|---|---|---|
sentry.server.config.js | Node.js | Sentry.init() called on server startup via instrumentation.js |
src/instrumentation-client.js | Browser | Sentry.init() called when a page loads in the browser |
src/app/api/glitchtip/route.js | Node.js | Client tunnel — proxies Sentry envelopes to GlitchTip, bypassing ad blockers |
{
dsn: process.env.SENTRY_DSN, // server: SENTRY_DSN, client: NEXT_PUBLIC_SENTRY_DSN
environment: // "development" | "staging" | "production"
process.env.NEXT_PUBLIC_DEPLOY_ENV ||
process.env.DEPLOY_ENV ||
process.env.NODE_ENV,
release: process.env.NEXT_PUBLIC_APP_VERSION, // the version tag on every event
dataCollection: { // server value; keeps the v10 scope
httpBodies: ['incomingRequest'],
databaseQueryData: false,
stackFrameVariables: false,
},
tracesSampleRate: 1.0, // server: 100% of requests traced; client: see below
debug: false,
}
Trace sampling differs by runtime. The server (sentry.server.config.js) samples 100% of requests. The client (src/instrumentation-client.js) samples 10% in production and 100% in every other environment.
Client-only additions: tunnel: '/api/glitchtip' (ad blocker bypass), dataCollection: { httpBodies: [] } (the browser sends no HTTP bodies), and a beforeSend hook that calls guardedReload('chunk') when the event is a chunk-load error from a stale tab.
The server config also sets serverName from SENTRY_SERVER_NAME. Without it the SDK uses the container hostname, which changes on every redeploy.
Server init lives in sentry.server.config.js and is loaded from src/instrumentation.js only when SENTRY_DSN is set — no production gate, so localhost reports too. Client init lives in src/instrumentation-client.js and runs on every page load whenever NEXT_PUBLIC_SENTRY_DSN is set.
The /api/glitchtip route proxies Sentry envelope payloads to GlitchTip's ingest endpoint. This prevents ad blockers from blocking error reports since the SDK sends to the app's own domain. The tunnel extracts the DSN's public key and forwards to https://<host>/api/<project>/envelope/?sentry_key=<key>&sentry_version=7.
Three levels of error isolation:
global-error.jsx — last-resort boundary wrapping the entire React tree. Renders its own <html>/<body> since the root layout is unavailable. Reuses the shared RouteError component.error.jsx — at root (src/app/error.jsx) and archives (src/app/archives/error.jsx). Thin wrappers around RouteError that catch errors within the layout.ComponentErrorBoundary — React class component wrapping Galaxy Map, Regions, Stats, Dashboard, and Timeline. Shows inline "failed to load" + retry button. Layout containers stay outside boundaries to preserve CSS grid/flex structure on error.Each boundary calls the shared reportError(error, { boundary }) helper (src/shared/utils/observability.mjs) so the user-visible failure reaches GlitchTip with the boundary name as context. ComponentErrorBoundary also passes the React componentStack for attribution. The same helper is used at every API-route 5xx path; calling it with a falsy error is a no-op so wiring it next to existing errorResponse(5xx, ...) sites is safe.
The Sentry browser SDK's global window.onerror / unhandledrejection handlers continue to catch uncaught client errors automatically — reportError exists for everything that gets try/catch'd or routed through an error boundary, which the global handler can't see.
src/shared/utils/observability.mjs exposes one function:
export function reportError(error, context = {}) {
if (!error) return;
const { level, ...extra } = context;
Sentry.captureException(error, { level, extra });
}
Call sites pass a route and stage tag for API routes (e.g. { route: '/api/h1/update', stage: 'season' }) or a boundary tag for React error boundaries. The level field maps to Sentry severity — used at update/route.js to mark the closing-pass season-fetch as a warning instead of an error since the code path is explicitly non-fatal.
The tunnel route (/api/glitchtip) is deliberately not wired with reportError; capturing there would self-loop when GlitchTip itself is the failing upstream.
Caught errors flowing through src/shared/utils/tryCatch.mjs — the project's canonical error-handling pattern, used at ~30+ call sites — are also auto-reported with source: 'tryCatch' and level: 'warning'. The explicit reportError(...) calls at 5xx sites still fire at the default error level. Sentry groups by stack-trace fingerprint so duplicate events for the same throw roll up into one issue, while the severity tag keeps caught-and-recovered errors visually distinct from user-visible failures in the inbox.
The service worker at src/sw.js runs in a separate worker context with no DOM, so it can't import the Sentry browser SDK directly. The push and notificationclick handlers are wrapped with try/catch (synchronous) and .catch() (on the async chains inside event.waitUntil); failures postMessage a structured { type: 'sw-error', error: { message, name, stack }, context } envelope to all controlled clients via self.clients.matchAll(). The receiving end lives in src/shared/utils/swErrorBridge.mjs — registered from instrumentation-client.js, it reconstructs the Error and calls reportError(err, { source: 'sw', ...context }). When no clients are open (push received with all tabs closed) the SW falls back to console.error, visible from chrome://serviceworker-internals during triage. Errors are re-thrown after the postMessage so the SW event loop still sees the rejection — original behavior preserved.
public/workers/cronLogic.js POSTs to GLITCHTIP_HEARTBEAT_URL after each successful (response.ok) poll of /api/h1/update. Fire-and-forget — heartbeat fetch failures are swallowed and don't affect the poll loop. GlitchTip's uptime monitor flips red when heartbeats stop, which surfaces: worker thread death, sustained 5xx from the API route, or DB unavailability. The worker reads the URL from process.env (Node worker_threads inherits the parent's env by default), so no message passing is needed.
Umami v3 analytics use a same-origin proxy to bypass ad blockers:
/stats.js and /recorder.js are route handlers (src/app/stats.js/route.js, src/app/recorder.js/route.js, shared code proxyUmamiScript in src/shared/utils/umami.mjs). Each fetches script.js or recorder.js from UMAMI_SITE_URL with no visitor headers, so visitor cookies, the auth session cookie included, never reach the Umami host. A Next.js rewrite would forward them./api/send → /api/umami (Next.js rewrite) → umami.lav.ren/api/send (API route proxy)/api/record and /api/websites/:id/recorder are Next.js rewrites to umami.lav.ren, set in next.config.mjs and not read from UMAMI_SITE_URL: rewrites are resolved at build time, and that variable exists only at runtime. The recorder calls them with credentials: "omit", so the browser attaches no cookie. RecorderScript renders the recorder only for browsers without the umami.disabled opt-out flag. Replays and heatmaps stay inert until they are turned on for the site in Umami.The proxy writes the visitor's IP from Cloudflare's cf-connecting-ip into the hit as payload.ip, so Umami uses the visitor, not this server, for its cookieless session hash and its location. Without it, the relay's second pass through Cloudflare labels every hit with the server's IP and country. The app sets no Content-Security-Policy header and the production edge sets one. Every Umami request (script, send, recorder) is same-origin, so 'self' in script-src and connect-src covers it and no Umami domain needs an allowance.
Env vars: UMAMI_SITE_URL (domain without protocol, e.g., umami.lav.ren), UMAMI_SITE_ID (website UUID).
The withSentryConfig wrapper (imported from @sentry/nextjs/config) controls build-time behavior. It runs only when SENTRY_AUTH_TOKEN is set:
export default process.env.SENTRY_AUTH_TOKEN ?
withSentryConfig(finalConfig, {
authToken: process.env.SENTRY_AUTH_TOKEN,
org: process.env.SENTRY_ORG,
project: process.env.SENTRY_PROJECT,
sentryUrl: process.env.SENTRY_URL,
sourcemaps: { deleteSourcemapsAfterUpload: true },
errorHandler: (err) => {
throw err;
},
})
: finalConfig;
sourceMappingURL comments from .next/static. The image ships no browser maps..next/standalone stay in the image. The app does not serve them over HTTP.silent option. It hid two upload failures in 2026-07 and 2026-08 (#496). Upload errors must print in the build log.errorHandler rethrows each Sentry step error (release creation, upload, finalize), so next build exits non-zero. The Build App Image leg fails, and Pin Production cannot pin a build without its maps (#661).The authToken, org, project, and sentryUrl env vars are only used during builds.
Variables are checked at startup by initializeEnvironmentVariables() in src/shared/utils/initializeEnv.mjs. The function uses progressive enhancement: core variables throw on missing (app cannot function), while auth and analytics variables warn and degrade gracefully. Returns { auth: boolean, analytics: boolean } so the startup log shows which features are active.
Validation rules:
BETTER_AUTH_SECRET absent → all auth disabled (warn). BETTER_AUTH_SECRET present but other auth vars missing → throws (partial config is a misconfiguration).SENTRY_DSN set without SENTRY_AUTH_TOKEN → warns about degraded source maps.*_FILE conventionA key that the running server reads can also come from a file. Set <NAME>_FILE to a file path, and hydrateFileSecrets() (src/shared/utils/hydrateFileSecrets.mjs) reads the file and sets <NAME> from its trimmed content. A <NAME> that is already set wins over the file. An unreadable file logs a warning and is skipped, and the normal validation then reports the missing variable. The Swarm stacks use this with POSTGRES_URL_FILE and UPDATE_KEY_FILE, which point at mounted Swarm secrets.
The convention has three limits:
SENTRY_DSN stays a plain variable. src/instrumentation.js reads it before the file values load.NEXT_PUBLIC_*, the SENTRY_* upload keys) cannot come from a file, because the hydration step never runs during a build.Every key below is in .example.env. That file is the source for the descriptions.
| Variable | Required | Category | Description |
|---|---|---|---|
POSTGRES_URL | Yes | Database | PostgreSQL connection string |
UPDATE_KEY | Yes | Updates | Bearer token for /api/h1/update — used by the worker to authenticate its polling requests |
UPDATE_INTERVAL | Yes | Updates | Polling interval in seconds (e.g., "20"); passed to the worker thread |
WORKER_ENABLED | No | Updates | "false" makes this replica web-only: no worker thread starts and its update route answers 403. Not needed to scale: every replica runs the worker and the worker_heartbeat lease lets one of them poll |
PORT | No | Updates | Server port; defaults to 3000; passed to the worker so it knows which port to poll |
BUCKET_SIZE | No | Updates | Bucket size in seconds for the h1_status, h1_statistic and h1_event_progress upserts; default 900 (15 min). A smaller value gives finer archives and more storage. Changing it mixes granularities in the history |
NEXT_PUBLIC_SITE_URL | No | Site | Public site origin (e.g. https://helldivers.bot); inlined at build time so self-hosters can serve under their own domain. Defaults to https://helldivers.bot (src/config/site.mjs). |
UMAMI_SITE_ID | No | Analytics | Umami website tracking ID; if absent, Umami script not loaded and server-side tracking skipped |
UMAMI_SITE_URL | No | Analytics | Umami instance domain, without protocol or port; used in server-side fetch calls |
SENTRY_DSN | No | Error tracking | Sentry DSN for server-side error reporting (points to GlitchTip); if absent, error tracking disabled |
NEXT_PUBLIC_SENTRY_DSN | No | Error tracking | Sentry DSN for client-side error reporting (same DSN, exposed to browser); a build argument, inlined at build time |
SENTRY_AUTH_TOKEN | No | Error tracking | GlitchTip API token for source map uploads; if absent, withSentryConfig build plugin skipped |
SENTRY_URL | No | Error tracking | GlitchTip instance URL; used by withSentryConfig for source map uploads |
SENTRY_ORG | No | Error tracking | GlitchTip organization slug (the slug, not the display name); used for source map uploads |
SENTRY_PROJECT | No | Error tracking | GlitchTip project slug (the slug, not the display name); used for source map uploads |
SENTRY_SERVER_NAME | No | Error tracking | server_name tag on server events and the replica name in the lease. Unset falls back to the container hostname, which is new on every redeploy. Swarm sets it to the node hostname and task slot |
GLITCHTIP_HEARTBEAT_URL | No | Error tracking | GlitchTip uptime heartbeat endpoint; worker POSTs after each successful update |
NEXT_PUBLIC_DEPLOY_ENV | No | Error tracking | Inlined at next build; CI sets staging or production. Sentry environment tag (falls back to DEPLOY_ENV, then NODE_ENV). Only production makes robots.txt allow crawling. |
DEPLOY_ENV | No | Error tracking | Server-only fallback for the Sentry environment tag when rebuilding to update NEXT_PUBLIC_DEPLOY_ENV isn't an option. |
BETTER_AUTH_SECRET | No (all-or-none) | Auth | BetterAuth session encryption secret; 128+ chars recommended. If absent, all auth features disabled. If present, all other auth vars are required |
BETTER_AUTH_URL | If auth | Auth | Base URL for BetterAuth (e.g., http://localhost:3000; production: https://helldivers.bot) |
AUTH_DISCORD_ID | If auth | Auth | Discord OAuth application client ID |
AUTH_DISCORD_SECRET | If auth | Auth | Discord OAuth application client secret |
AUTH_GITHUB_ID | If auth | Auth | GitHub OAuth application client ID |
AUTH_GITHUB_SECRET | If auth | Auth | GitHub OAuth application client secret |
AUTH_GOOGLE_ID | If auth | Auth | Google OAuth application client ID |
AUTH_GOOGLE_SECRET | If auth | Auth | Google OAuth application client secret |
VAPID_PUBLIC_KEY | No (all-or-none) | Push | Web Push VAPID public key; if absent, push notifications disabled |
VAPID_PRIVATE_KEY | No | Push | Web Push VAPID private key |
VAPID_SUBJECT | No | Push | VAPID contact identifier (mailto: email) |
NEXT_PUBLIC_VAPID_PUBLIC_KEY | No | Push | Client-side VAPID public key (same value as VAPID_PUBLIC_KEY); a build argument, inlined at build time |
MIGRATE_ROOT | No | Docker | Folder with the Prisma CLI and the prisma/ tree for migrate-on-start; defaults to the process cwd, the image sets /app/migrate |
FORCE_SEED | No | Docker | Makes the seed re-upsert every season file even when all are in the database. Only true, 1 or yes enable it; anything else, including false, leaves it off |
LEGACY_POSTGRES_URL | No | Scripts | Restored pg_dump of the pre-cleanup schema; read by scripts/backfill-h1-tables.mjs only |
SEED_TEST_API_KEY_HASH | No | Tests | SHA-256 hex digest of the smoke-test API key. The seed stores it, and the seed also runs on app start, so it creates a full-access key: never set it in a deployed environment |
SEED_TEST_API_KEY | No | Tests | Plaintext key behind that digest; the smoke tests send it. CI sets both; never reuse them |
TEST_SERVER_URL | No | Tests | Server the smoke tests call; default http://localhost:3000 |
SMOKE_ALLOW_SKIP | No | Tests | 1 or true turns an unreachable smoke-test server into a skip (local use only) |
Two more variables are not in .example.env. Node and Next.js set them, and nobody sets them by hand:
NODE_ENV is development for npm run dev and production in the images. It picks the worker path (public/workers/cron.js from the working directory in development, /app/public/workers/cron.js otherwise) and is the last fallback for the Sentry environment tag.NEXT_RUNTIME is set by Next.js and decides which Sentry config and init steps run.NEXT_PUBLIC_APP_VERSION is generated by next.config.mjs at build time from APP_VERSION or the package.json version. Do not set it.
Local development connects directly to the host machine:
postgresql://user:pass@127.0.0.1:5432/dbname
Inside Docker, the app container reaches the host's PostgreSQL via the special Docker DNS name:
postgresql://user:pass@host.docker.internal:5432/dbname
docker-compose.yml reads its values from a local env file that is not checked into version control. The Swarm stacks pass the connection string as a secret through POSTGRES_URL_FILE.
The app is an installable Progressive Web App. PWA assets are static files that ship with the build — no build-time generation step is needed.
Source: public/site.webmanifest (a static file, served at /site.webmanifest)
Linked in <head> via metadata.manifest in src/app/layout.jsx. Contains start_url, scope, orientation: portrait, display: standalone, theme and background colors, and two icons in public/favicons/ with purpose: "any maskable".
20 portrait PNG images in public/splash/ covering all current iPhone and iPad device sizes. Generated once via npx pwa-asset-generator with solid #282828 background + centered logo. Linked via <link rel="apple-touch-startup-image"> tags with device-specific media queries in layout.jsx.
To regenerate after logo changes:
npx pwa-asset-generator public/images/logo_square.png public/splash \
--splash-only --background "#282828" --padding "30%" --type png
# Then delete landscape images (width > height) since orientation is portrait-only
headers() in next.config.mjs sets these headers:
| Path | Cache-Control |
|---|---|
/favicons/* | public, max-age=86400, immutable (1 day) |
/fonts/* | public, max-age=31536000, immutable (1 year) |
/icons/* | public, max-age=31536000, immutable (1 year) |
/images/* | public, max-age=31536000, immutable (1 year) |
/svgs/* | public, max-age=31536000, immutable (1 year) |
/workers/* | public, max-age=86400, immutable (1 day) |
/sw.js | no-cache, no-store, must-revalidate |
HTML page routes (not the paths above, /api/*, /_next/* or /profile*) | public, s-maxage=30, stale-while-revalidate=60 |
The service worker is built from src/sw.js into public/sw.js by serwist build, which npm run build runs after next build. The built file is git-ignored. It is never HTTP-cached, so a browser detects a new version on every navigation. skipWaiting and clientsClaim in src/sw.js then activate it. The s-maxage header on page routes lets Cloudflare collapse concurrent visitors into one origin request per 30 seconds. Browsers still revalidate normally.
Every path also gets X-Frame-Options: DENY, X-Content-Type-Options: nosniff, Referrer-Policy: strict-origin-when-cross-origin and a Strict-Transport-Security header.