clone

one developer profile for links and private tcp services, reached peer to peer over iroh with no open ports

main
devsite / deploy
README.md

Deploying dev.site

The control plane runs on Fly: one machine, one volume, SQLite on it. fly.toml and Dockerfile in the repository root are the whole deployment.

One machine is deliberate. A Fly volume attaches to a single machine in a single region, and nothing here wants a second writer. Since the heartbeat went away the database is read-mostly — a few hundred bytes per account, and a handful of writes per user per lifetime — so this shape has a lot of room in it.

The durable identity boundaries

OIDC issuer and subjects. Accounts are keyed by the verified (issuer, subject) pair. Shoo is the default adapter and derives its default client_id and pairwise subjects from DEVSITE_PUBLIC_ORIGIN, so changing that origin after anyone has signed in still orphans those identities. Another public OIDC client can be selected with the DEVSITE_OIDC_* settings below without changing the account or session code.

The capability signing key. Every daemon pinned its public half at devsite login and refuses capabilities signed by anything else. Lose it and every daemon in the world stops working, with no way to recover but re-running devsite login everywhere.

So it is a Fly secret rather than a file on the volume: encrypted at rest, injected into the machine's environment, and not tied to the survival of one volume on one host. The server prefers DEVSITE_SIGNING_KEY when it is set and falls back to DEVSITE_STATE_DIR when it is not, which is what keeps local development unchanged.

One-time setup

fly apps create devsite

# --yes answers the "every volume is pinned to one physical host, make two"
# warning, which cannot prompt without a terminal. One volume is the intended
# shape here: a second would be a second SQLite file on a second machine, both
# taking writes and neither aware of the other. Redundancy needs LiteFS, not
# another volume.
fly volumes create devsite_data --region sjc --size 1 --yes

# 32 random bytes as 64 hex characters. Generate it here, keep a copy somewhere
# you would still have after losing this laptop, and never commit it.
fly secrets set DEVSITE_SIGNING_KEY="$(openssl rand -hex 32)"

To carry an existing key over instead of generating one:

fly secrets set DEVSITE_SIGNING_KEY="$(xxd -p -c 64 data/state/capability_signing.key)"

Check it against what daemons pinned before trusting it — fly logs prints the public half at boot, and devsite status prints what the daemon expects.

The hostname

The first deploy provisions a shared v4 and a dedicated v6 by itself, so there is usually nothing to allocate:

fly ips list                      # the addresses to point DNS at

At Cloudflare, on the dev.site zone, add A and AAAA records for the apex pointing at those addresses, DNS only — grey cloud, not proxied. Fly needs to answer the ACME challenge on the hostname itself; proxying breaks issuance and puts a second TLS terminator in front of a site whose whole argument is about who can see what.

fly certs add dev.site
fly certs show dev.site           # until it reports Ready

Deploying

fly deploy

Fly builds the image remotely. You do not need a local Docker daemon. The build checks and compiles the web source. It then builds the server and copies both outputs into the image.

A deploy replaces the machine, which means a few seconds where the site is down. That is the cost of one volume, and it is the right trade at this size.

Backups

Fly snapshots volumes daily, five days' retention. That covers the machine dying; it does not cover "I want the database on my laptop", which is worth having:

fly ssh console -C "sqlite3 /data/devsite.db '.backup /data/backup.db'"
fly sftp get /data/backup.db ./devsite-backup.db

The signing key is not on the volume and is not in these backups. Keep it separately — it is the part that cannot be regenerated.

Configuration

Set in fly.toml, except the secret. See Authentication providers for replacing Shoo or implementing a non-OIDC adapter.

Variable Where Notes
DEVSITE_PUBLIC_ORIGIN fly.toml https://dev.site. See above.
DEVSITE_OIDC_ISSUER optional Token issuer; defaults to https://shoo.dev.
DEVSITE_OIDC_AUTHORIZATION_ENDPOINT optional Defaults to <issuer>/authorize.
DEVSITE_OIDC_TOKEN_ENDPOINT optional Defaults to <issuer>/token.
DEVSITE_OIDC_JWKS_URI optional Defaults to <issuer>/.well-known/jwks.json.
DEVSITE_OIDC_CLIENT_ID optional Public client id; defaults to origin:<public origin>.
DEVSITE_OIDC_SCOPES optional Space-separated; defaults to openid and must include it.
DEVSITE_OIDC_ALGORITHMS optional Allowed asymmetric JWT algorithms; defaults to Shoo's ES256.
DEVSITE_SIGNING_KEY fly secrets Hex. Never in the repository.
IROH_SERVICES_API_SECRET optional secret Iroh Services API key for scoped relay tokens.
DEVSITE_RELAY_URLS optional Comma-separated custom relay URLs. The n0 preset is the default.
DEVSITE_BIND Dockerfile 0.0.0.0:8080, matching internal_port.
DEVSITE_DB Dockerfile /data/devsite.db, on the volume.
DEVSITE_STATE_DIR Dockerfile Unused while the secret is set.
DEVSITE_WEB_ROOT Dockerfile /app/web, baked into the image.

What Fly sees

Fly terminates TLS for the control plane, so it sees profile metadata and API calls — the same position Cloudflare was in before. It never sees private service traffic: that goes client-to-daemon over an Iroh relay, end-to-end encrypted, and the control plane is not even told where the daemon is.

Caching

Static files and the HTML shell revalidate; /api/ responses are no-store.

Cloudflare still holds the dev.site zone, but only as DNS — the A and AAAA records point straight at Fly, unproxied.