Dev and prod workflow

Tempest runs as two Docker Compose projects on this machine. They use the same Dockerfile and the same repository, and they do not share a database, photos, backups, or assistant state. Which stack the public name serves is in deploy.md. The documentation site is a third project, described under Docs site.

The two stacks

flowchart TB
  devStack ~~~ prodStack
  subgraph devStack [tempest-dev]
    direction TB
    lan["Browser on the LAN, port 8000"] --> devWeb["web: Django runserver, live source mount"]
    devWeb --> devDb["PostgreSQL in data/dev"]
    devAssistant["assistant: run_assistant"] --> devDb
  end
  subgraph prodStack [tempest-prod]
    direction TB
    prodWeb["web: gunicorn, code baked into the image"] --> prodDb["PostgreSQL in data/prod"]
    prodAssistant["assistant: run_assistant"] --> prodDb
    prodSched["scheduler at 03:30 Europe/London"] --> prodDb
  end

	Dev	Prod
Compose file	`compose.dev.yaml`	`compose.prod.yaml`
Settings	`tempest.settings.dev`	`tempest.settings.prod`
Env file	`.env.dev`	`.env.prod`
Image tag	`tempest:dev`	`tempest:prod`
What serves the code	The working tree, mounted on `/app`	The files copied into the image at build time
Web process	`migrate`, then `runserver` on port 8000	`scripts/start-web.sh`: `migrate`, `collectstatic`, gunicorn
Published port	`8000` on the host	None. Other containers reach web as `tempest-prod-web-1:8000` on the `npm_default` network
Scheduler	None	`run_scheduler`: `daily_maintenance` at 03:30 Europe/London
Assistant	`run_assistant`, state in `data/dev/assistant`	`run_assistant`, state in `data/prod/assistant`. It does not join the public proxy network
Mail	Printed in the web container log	Sent with the SMTP settings in `.env.prod`
Debug	On. `/static/` is `no-cache`	Off. Secure cookies, HSTS, and `X-Forwarded-Proto` so Django treats the proxy's HTTPS as HTTPS

Start or restart one stack from /home/ajoyner/tempest:

docker compose -f compose.dev.yaml up -d
docker compose -f compose.prod.yaml up -d

.env.dev and .env.prod are not in git. Copy .env.example if one is missing. Their values stay in those files. Do not copy them into chat, commits, or other files.

/spike/ answers only when TEMPEST_ENABLE_SPIKE is true. It is a metadata check, not part of the product.

Docs site

compose.docs.yaml is the Compose project tempest-docs. It serves this docs/ directory with MkDocs Material. It uses its own image, tempest-docs:dev, built from Dockerfile.docs and requirements-docs.txt. It does not use the app image, the database, or either env file. Stopping tempest-dev or tempest-prod leaves it running.

The repository is mounted at /docs. mkdocs serve listens on port 8090 with live reload. A save to a page or to mkdocs.yml refreshes the browser. A change to requirements-docs.txt or Dockerfile.docs needs a rebuild:

docker compose -f compose.docs.yaml up -d --build

On the LAN the site is http://192.168.10.20:8090. The public name, and what a proxy in front of it must allow, is in deploy.md.

The pages in the site nav are the nav list in mkdocs.yml. A file under docs/ that is missing from that list is left out of the nav, and MkDocs warns about it in the container log. The type size for the header, navigation, and page text is the root size in docs/stylesheets/extra.css.

What `docker build` puts in the image

compose.prod.yaml builds Dockerfile and tags the result tempest:prod. Dev builds the same Dockerfile as tempest:dev, then hides it with the source mount, so the dev image's copy of the code is unused while that mount is in place.

The image contains Python 3.12, the PostgreSQL 16 client tools, and the Python packages from requirements.txt, including Django, gunicorn, WhiteNoise, Pillow, pillow-heif, and requests. It contains the Python apps, templates, static sources, and scripts. It leaves out .git, data/, the env files, and staticfiles/. The PostgreSQL client is there so backup and restore use the same major version as the database. The process runs as uid 1000, matching the host user who owns data/.

These run when the production web container starts, not during the build:

python manage.py migrate --noinput
python manage.py collectstatic --noinput into /app/staticfiles, using WhiteNoise's hashed, gzipped storage
gunicorn on 0.0.0.0:8000, 3 workers, timeout 120 seconds (GUNICORN_WORKERS and GUNICORN_TIMEOUT in .env.prod)

The scheduler is the same image with a different command. It does not serve the site. At 03:30 Europe/London it runs daily_maintenance: a backup kept for 14 days, purge of Trash and old deletion markers, purge of accounts that are due, lifting of expired suspensions, and pruning of the forecast cache, applied operations (60 days), and old revisions (90 days). If the last run is more than 25 hours overdue it runs shortly after the container starts. Backups go to data/prod/backups. See restore.md.

Only data is mounted into prod containers:

data/prod/postgres on the database container
data/prod/media and data/prod/backups on web and the scheduler
data/prod/assistant on the assistant, with media and backups as well

.env.prod is passed in as the container environment. It is not a file inside the image.

Changing code

On dev, a Python change is picked up by runserver. Templates, CSS, and the JS modules are read from the mount. A change to requirements.txt or the Dockerfile still needs a rebuild:

docker compose -f compose.dev.yaml build web
docker compose -f compose.dev.yaml up -d --no-deps web assistant

On prod, editing the working tree does not change the running site. Rebuild the image, then recreate the containers so they use it. build without a following up -d leaves the old containers on the previous image. The steps are in deploy.md.

To reload environment variables for one service without recreating the database:

docker compose -f compose.dev.yaml up -d --no-deps --force-recreate web

Use compose.prod.yaml for production. Omit --no-deps and Compose recreates the database container as well. The files in data/ remain; the recreate is an extra restart, not a wipe. Recreate assistant as well when the change is CURSOR_API, the assistant tools, or requirements.txt. The worker does not reload because web migrated.

Tests

docker compose -f compose.dev.yaml exec web python manage.py test
docker run --rm -v "$PWD":/w -w /w node:22-alpine node tests/js/grid.test.mjs
docker run --rm -v "$PWD":/w -w /w node:22-alpine node tests/js/kit.test.mjs
docker run --rm -v "$PWD":/w -w /w node:22-alpine node tests/js/assistant.test.mjs
docker run --rm --network host -v "$PWD":/w -w /w mcr.microsoft.com/playwright/python:v1.49.0-noble \
  sh -c "pip install -q --break-system-packages playwright==1.49.0 && python tests/e2e/offline_check.py <email> <password>"
docker run --rm --network host -v "$PWD":/w -w /w mcr.microsoft.com/playwright/python:v1.49.0-noble \
  sh -c "pip install -q --break-system-packages playwright==1.49.0 && python tests/e2e/assistant_check.py <email> <password>"
docker run --rm --network host -v "$PWD":/w -w /w mcr.microsoft.com/playwright/python:v1.49.0-noble \
  sh -c "pip install -q --break-system-packages playwright==1.49.0 && python tests/e2e/kit_check.py <email> <password>"

The offline check drives Chromium and WebKit through sign-in, offline capture, reopen, reconnect and a gated app update against the dev stack. The assistant check uses WebKit at an iPhone 15 size. It expects the dev assistant worker to be running with ASSISTANT_SESSION=assistant.fake.FakeSession, and the account to have both AI Trip Assistant and AI Assistant switched on. The kit check adds an item offline, packs it on a trip, and opens Kit Bag at an iPhone 15 size. Live Cursor calls are separate: TEMPEST_LIVE_CURSOR=1 python manage.py test assistant.test_live inside the assistant container.