The README says what runs now (the private server, litestream), how private repos' pages are built and served and to whom, the ssh commands for passkeys, sessions and tokens, how passkeys, cookies and tokens work, HTTPS clones with a token, and how identity.db is backed up, replicated and restored. It drops "pages for private repos" from what isn't there yet, notes Node 24 for the container, and lists /login/ and /private/ among the URLs. The guide and Caddyfile.example follow.
- commit
- d3d8e6e71b950198b28854ef7629a0196cfc86ff
- parent
- f4a7674039d42a19a70c6460ac2816fbbddcd339
- author
- cat <admin@dev.site>
- kind
- docs risk score 0.1 of 3, Jev's estimate
| M | README.md | 189 | +++++++++++++++++++++++------- |
| M | guide/ | 3 | +- |
| M | server/ | 13 | ++- |
@@ -4,11 +4,13 @@ A personal forge built from plain git. The server is the setup from
44 [Git on the Server](https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server):55 a `git` user, SSH keys, and bare repositories. Pushes run two hooks, and an66 [11ty](https://www.11ty.dev/) build writes static pages in the style of7-[stagit](https://git.suckless.org/). There is no database: the state is files8-in the bare repos and a directory of HTML that any static file server can9-serve. Two small Node servers sit next to it for what static files can't do:10-one streams live page updates, the other serves repo sites on their own11-hostnames (see [What runs](#what-runs)).7+[stagit](https://git.suckless.org/). The state is files in the bare repos,8+a directory of HTML that any static file server can serve, and one SQLite9+file of web sign-ins (passkeys, sessions, tokens) for private repos' pages.10+Small Node servers sit next to it for what static files can't do: one11+streams live page updates, one serves private repos' pages to their12+readers, one serves repo sites on their own hostnames (see [What13+runs](#what-runs)).1214 1315 It adds what stagit doesn't have, borrowing from [args.io](https://args.io):1416 @@ -93,13 +95,15 @@ forge comment, approve ─── ssh ──────────────>
9395 | Process | Why |9496 | --- | --- |9597 | sshd | git over ssh. Every key is forced through `forge shell <name>`. |96-| Caddy | Serves `.forge/www` as plain files, read-only HTTPS clones of public repos (`git http-backend` through fcgiwrap; nothing else inside a bare repo is served), and `go get` meta tags; proxies `/events/` and, with a pages domain, sites. |98+| Caddy | Serves `.forge/www` as plain files, read-only HTTPS clones (`git http-backend` through fcgiwrap; nothing else inside a bare repo is served; a private repo takes a token), and `go get` meta tags; proxies `/events/`, every path `www` has no file for (to the private server) and, with a pages domain, sites. |9799 | `forge build` | Runs 11ty after each push, in the background and one repo at a time, then swaps the new pages into `www/`. Unchanged pages are hard links to the last build's. First it adds the push's commits to the repo's commit-graph (see [Cleaning up](#cleaning-up)). |98-| `lib/events-server.ts` | Tails `.forge/events.jsonl`, which hooks and builds append to, and streams it as server-sent events at `/events/<owner>/<repo>/`. Open pages say who pushed and swap in fresh content when the build finishes. Without it pages still work; their status line says offline. Private repos stream nothing. |100+| `lib/events-server.ts` | Tails `.forge/events.jsonl`, which hooks and builds append to, and streams it as server-sent events at `/events/<owner>/<repo>/`. Open pages say who pushed and swap in fresh content when the build finishes. Without it pages still work; their status line says offline. A private repo streams only to its readers, signed in. |101+| `lib/private-server.ts` | Serves private repos' pages to their readers, signed in with a passkey or holding a token, the sign-in pages (`/login/`, `/private/`), and the 404 of every page that isn't there; answers Caddy whether a token may clone a private repo over HTTPS (see [Private pages](#private-pages)). |99102 | `lib/pages-server.ts` | Only with a pages domain. Serves each site on its own hostname the way Cloudflare Pages does (`_headers`, `_redirects`, pretty URLs, the nearest `404.html` or a single-page-app fallback) and redirects old `/sites/...` paths there. Runs as the caddy user, since deploys are untrusted content. |100103 | `forge backup` | Daily, from a loop in `start.sh`. Snapshots the forge with restic into a bucket, then checks the snapshots and test-restores the new one (see [Backups and moving hosts](#backups-and-moving-hosts)). |101104 | `forge mirror` | After builds, in the background: syncs the forge's own pages to a bucket with rclone, for the edge to serve while the forge is down. |102105 | `forge lake` | After builds, in the background: adds what is new in the event log, build metrics, review logs and CI notes to the lake, and writes the insights exports (see [The lake and insights](#the-lake-and-insights)). |106+| `forge identity replicate` | With backups set up: [litestream](https://litestream.io) copies `.forge/identity.db` to the backup bucket as it changes (see [identity.db](#identitydb-backed-up-as-it-changes)). |103107 104108 Without a pages domain, Caddy serves sites straight from `/sites/...` and105109 `/previews/...` on the forge's own origin, and `_headers` and `_redirects` don't@@ -114,8 +118,10 @@ compressed, and are sent as attachments (see [Releases](#releases)).
114118 115119 ### Identity and permissions116120 117-`/srv/git/.forge/keys` holds `name ssh-ed25519 AAAA...` lines and is the only118-source of identity. `forge keys sync` generates two files from it:121+`/srv/git/.forge/keys` holds `name ssh-ed25519 AAAA...` lines and is the122+root of identity: a user is a name there, and everything else (web sign-ins123+included, below) belongs to one. `forge keys sync` generates two files from124+it:119125 120126 - `authorized_keys`, which forces every key through `forge shell <name>`, so121127 hooks know who pushed;@@ -152,15 +158,20 @@ have it, and Forgejo toggles the same file. `forge repo new` adds it, and
152158 `forge repo new --private`, and the imports below.153159 154160 A private repo can be read by its owner, its collaborators and the names in155-`<repo>.git/forge/readers`, over ssh only. To anyone else it isn't there: a156-clone, fetch, archive, push or ssh command gets the same answer, byte for157-byte, as for a repo that doesn't exist, however the path is spelled. It has158-no pages, no entry on the index, owner pages or `repos.json`, and no live159-events. Its sites and previews aren't served unless `git config160-forge.publicsites true`, and its releases aren't served at all; readers get161-them over ssh (see [Releases](#releases)). Pushes still record reviews, and `forge reviews`162-reads them from the review logs (without CI). The secret check still runs,163-since a secret stays in the history when the repo goes public later.161+`<repo>.git/forge/readers`: over ssh, on the web once they sign in (see162+[Private pages](#private-pages)), and over HTTPS with a token. To anyone163+else it isn't there: a clone, fetch, archive, push or ssh command gets the164+same answer, byte for byte, as for a repo that doesn't exist, however the165+path is spelled, and so does every page of it. It has no entry on the166+index, owner pages or `repos.json`, its pages and live events go to its167+readers only, and the pages mirror never sees it. Its sites and previews168+aren't served unless `git config forge.publicsites true`, and its releases169+aren't served publicly; readers get them over ssh (see170+[Releases](#releases)). Pushes record reviews and print their links, which171+work for its readers, and `forge reviews` reads them from the review logs172+(without CI). The secret check still runs, since a secret stays in the173+history when the repo goes public later; Jev isn't asked about private174+repos.164175 165176 ```sh166177 ssh git@git.example.com readers dami/notes add ana # rm to take it back; no argument lists them@@ -170,15 +181,77 @@ ssh git@git.example.com ls # yours, and the private o
170181 ```171182 172183 The owner and collaborators flip a repo, the way Forgejo does. `private`173-removes the flag, the repo's pages, its entries on the index, owner pages174-and `repos.json`, and its served sites, previews and their hostnames. Then175-it starts a mirror sync, which deletes them from the bucket. Clones, caches176-and copies people already have can't be recalled. `public` first runs the177-secret check over the whole history: every commit on a branch or tag, and178-the old review revisions that review pages show, not only what a push adds.179-If it finds something, the repo stays private unless you pass `--force`.180-Then it adds the flag, links the sites and previews again from their refs,181-and rebuilds the pages.184+removes the flag, the repo's public pages, its entries on the index, owner185+pages and `repos.json`, and its served sites, previews and their186+hostnames, and builds its pages again into the private root. Then it starts187+a mirror sync, which deletes them from the bucket. Clones, caches and copies188+people already have can't be recalled. `public` first runs the secret check189+over the whole history: every commit on a branch or tag, and the old review190+revisions that review pages show, not only what a push adds. If it finds191+something, the repo stays private unless you pass `--force`. Then it adds192+the flag, removes its private pages, links the sites and previews again193+from their refs, and rebuilds the pages into `www`. A flag added or removed194+by hand moves the pages at the repo's next build, and `forge build` drops195+any left in the wrong place.196+197+### Private pages198+199+A private repo's pages are built like any repo's, into200+`.forge/private/www` instead of `www`, with the same layout: pages, search201+index, `code.db`. They link the same hashed assets, which the build puts in202+`www/assets` (they are the forge's scripts and styles, nothing of the203+repo), and their header marks the repo private. Caddy serves a file of204+`www` when there is one and hands every other path to205+`lib/private-server.ts`, so it needs no list of private repos. That server206+serves a private repo's page to someone who may read it (the rule207+`can_read` in `bin/forge` applies, kept in step with `lib/access.ts` by a208+scenario) and is still a user, signed in with a passkey or showing a token,209+with the headers Caddy gives pages (ETag, 304s, byte ranges, which code210+search reads `code.db` with) except `Cache-Control: private, no-cache`.211+Not signed in, not allowed, or a page that isn't there: the same 404, byte212+for byte, which every missing page of the forge gets, public or not.213+`/private/` lists the private repos you can read.214+215+Web credentials live in one SQLite file, `.forge/identity.db` (WAL mode,216+only git reads it), and are made over ssh only, so there are no passwords217+or emails, and losing them is fixed by running ssh again:218+219+```sh220+ssh git@git.example.com passkey [name] # a link that adds a passkey: once, within 10 minutes221+ssh git@git.example.com passkeys [rm <id>]222+ssh git@git.example.com sessions [rm <id>|--all]223+ssh git@git.example.com token [--repo dami/notes] [--ttl 30d] [name] # printed once224+ssh git@git.example.com tokens [rm <id>]225+```226+227+- **Passkeys** ([SimpleWebAuthn](https://simplewebauthn.dev) 14). The228+ relying party is the forge's origin, `FORGE_BASE_URL`, whose host is the229+ RP ID; responses from any other origin are refused. The link carries a230+ one-time code in its fragment, so it reaches no log, and binds the new231+ passkey to your ssh user. Sign-in on `/login/` needs no user name: the232+ browser offers the forge's passkeys, which are discoverable, and user233+ verification is required. On Node 24.7 and later (the container) ML-DSA-44234+ is offered first, so a passkey can be post-quantum. After `passkeys rm`,235+ the next page you open signed in tells your password manager which236+ passkeys still work (the WebAuthn Signal API), so it stops offering the237+ removed one; it also signs out the sessions that passkey made.238+- **Sessions** are a `__Host-forge` cookie (Secure, HttpOnly, SameSite=Lax)239+ that lasts 30 days from its last use. Sign-out is a POST from the forge's240+ page, with a CSRF token. The events server streams a private repo to a241+ page whose session may read it.242+- **Tokens** are for one repo or for every repo you can read when they are243+ used, for 30 days unless `--ttl` (at most 365d). Send one as244+ `Authorization: Bearer <token>`, or as the password of basic auth:245+ `git clone https://dami:<token>@git.example.com/dami/notes.git`, a246+ credential helper, or the token alone as the user name.247+248+identity.db keeps only SHA-256 hashes of cookies, tokens and enrollment249+codes, and every credential stops working when its user's keys go (`forge250+user rm` deletes them too). Its SQL is the plain subset Turso runs as well.251+HTTPS clones of a repo without `git-daemon-export-ok` go through Caddy's252+`forward_auth`: the private server answers 401 without a token (so git sends253+one) and, with one, 200 or the same 404 for a private repo and a missing254+one; only then does `git http-backend` run, with `GIT_HTTP_EXPORT_ALL`.182255 183256 ### Reviews184257 @@ -619,10 +692,11 @@ What it costs, measured on an M-series Mac and in the image (linux/arm64):
619692 620693 ## Hosting621694 622-The forge is one container (`Dockerfile`): sshd for git, Caddy for pages,623-sites, and HTTPS clones (with fcgiwrap for `git http-backend`), the events and624-pages servers, and tailcat. `server/container/start.sh` starts them all. All625-state is in `/srv/git`, a volume.695+The forge is one container (`Dockerfile`, on Node 24 LTS): sshd for git,696+Caddy for pages, sites, and HTTPS clones (with fcgiwrap for `git697+http-backend`), the events, private and pages servers, litestream, and698+tailcat. `server/container/start.sh` starts them all. All state is in699+`/srv/git`, a volume.626700 627701 ### Home machine behind a public edge (how src.rest runs)628702 @@ -669,9 +743,10 @@ port 22 doesn't work on shared addresses. DNS for `src.rest` and
669743 670744 `server/provision.sh` turns a fresh Debian 13 or Ubuntu 24.04 box into the671745 forge directly (git, sshd, Node, Caddy with automatic TLS, and systemd units672-for the events and pages servers). It needs Node 22.18 or newer, which runs673-the TypeScript sources as they are; those distros ship older ones, so install674-a current Node first:746+for the events, private and pages servers). It needs Node 22.18 or newer,747+which runs the TypeScript sources as they are, and 24 LTS (24.7 or later) is748+best, as in the container, for post-quantum passkeys; those distros ship749+older ones, so install a current Node first:675750 676751 ```sh677752 rsync -a --exclude node_modules --exclude demo/root --exclude demo/work ./ root@HOST:/opt/forge/@@ -766,8 +841,8 @@ Then point existing clones at the forge:
766841 `git remote set-url origin git@git.example.com:dami/<repo>.git`.767842 768843 A public repo is public everywhere: pages, sites, releases, HTTPS clones and769-the pages mirror. A private one is only read over ssh for now; pages for it, behind a770-login, come later (see [Private repos](#private-repos)).844+the pages mirror. A private one is read over ssh, on its pages once you sign845+in, and over HTTPS with a token (see [Private repos](#private-repos)).771846 772847 ## Backups and moving hosts773848 @@ -790,6 +865,8 @@ Everything in `/srv/git` that can't be rebuilt:
790865 collaborators, readers, and anything else kept there;791866 - `.forge/keys`, `.forge/config`, and `.forge/www/hosts/`, which hostname each792867 site has;868+- `.forge/identity.db`, as SQLite copies it (`VACUUM INTO`): one consistent869+ file, never the live one with its WAL;793870 - `.hostkeys/`, so clients see the same ssh host key, and `.tailcat/`, so a794871 restored box keeps its tunnel address and the edge finds it;795872 - the lake (`.forge/lake`): the catalog, copied with SQLite's `VACUUM INTO`@@ -802,8 +879,8 @@ Everything in `/srv/git` that can't be rebuilt:
802879 below, so state added later is backed up by default.803880 804881 Left out, because a restore rebuilds it or every boot rewrites it: the pages in805-`.forge/www`, the extracted sites and releases in `.forge/store` (both are806-git refs),882+`.forge/www` and `.forge/private`, the extracted sites and releases in883+`.forge/store` (both are git refs),807884 `.forge/cache`, `authorized_keys`, `allowed_signers`, `git-shell-commands/`,808885 hooks, `.forge/secrets` and `.forge/backup.env` (both written from `forge.env`809886 at boot), and logs, locks and scratch files.@@ -851,6 +928,32 @@ BACKUP YET` or `BACKUPS ARE OFF`. The daily run prints it to the container log
851928 (`docker logs forge | grep backup:`). To hear about trouble, run `--status`852929 from cron on another machine: `ssh slate docker exec forge forge-admin backups --status`.853930 931+### identity.db, backed up as it changes932+933+Sign-ins and tokens change all day, so a daily snapshot isn't enough for934+`.forge/identity.db`. With backups set up, the container runs `forge935+identity replicate`: [litestream](https://litestream.io) v0.5 copies each936+change to the backup bucket within seconds, under `litestream/identity`937+(`FORGE_LITESTREAM_PATH`), with the same keys. Its copy holds no secrets938+(identity.db has only hashes and public keys), so it isn't encrypted the939+way restic's is.940+941+```sh942+docker exec forge forge-admin identity restore # as it is now943+docker exec forge forge-admin identity restore --at 2026-09-01T10:00:00Z # as it was then (the last week)944+```945+946+Stop the forge's servers before restoring by hand, since they hold the file947+open (`docker compose stop forge`, then `docker compose run --rm forge948+forge-admin identity restore ...`); the file it replaces is kept as949+`identity.db.before-restore`. A box that boots without identity.db (a new950+one) takes it from the replica before anything makes an empty one, whose951+replication would bury the replica's; if the replica can't be read then,952+nothing replicates until the next boot. `forge restore` takes whichever of953+the backup's copy and the replica changed last (every write, deletions954+too, stamps the file), which is litestream's unless replication had955+stopped. Without backups set up, identity.db stays local.956+854957 ### The pages mirror855958 856959 After a build, `forge mirror` syncs the forge's own pages (not sites, previews@@ -898,6 +1001,7 @@ What you set up yourself, once:
8981001 FORGE_MIRROR_KEY_ID=<access key id>8991002 FORGE_MIRROR_SECRET=<secret access key>9001003 # optional: FORGE_BACKUP_AT=03:30 FORGE_BACKUP_KEEP_DAILY=7 FORGE_BACKUP_KEEP_WEEKLY=5 FORGE_MIRROR_DELAY=301004+ # FORGE_LITESTREAM_PATH=litestream/identity9011005 ```9021006 9031007 At every boot the container writes these to `.forge/backup.env`, which@@ -915,8 +1019,9 @@ What you set up yourself, once:
9151019 `forge server init` and `forge keys sync` write, hooks in every repo, pinned9161020 review revisions, extracted sites, public repos' release downloads, with9171021 the mtimes they had, so the mirror uploads none again) and runs `forge918-build`. It refuses a forge919-that already has repos. The host's own `.forge/config` wins over the backup's,1022+build`. identity.db is the backup's copy or litestream's replica, whichever1023+changed last (see [above](#identitydb-backed-up-as-it-changes)). It refuses1024+a forge that already has repos. The host's own `.forge/config` wins over the backup's,9201025 since it is written for the domain that host serves, and users from both are9211026 kept. `forge restore <snapshot> --to <dir>` only unpacks the files, to look at9221027 them or to copy one repo back by hand. Any machine with restic and the@@ -1098,6 +1203,8 @@ may be missing needs an explicit check or `or ''`.
10981203 | `/<owner>/<repo>/releases/download/<tag>/<file>` | a release file; `latest/` for the newest release's |10991204 | `/<owner>/<repo>/insights/` | charts from the lake; `insights.json` and Parquet files next to it |11001205 | `/insights/` | the same for the whole forge, and a table by repo |1206+| `/login/` | sign in with a passkey, add one (`#enroll=<code>`), sign out |1207+| `/private/` | the private repos you can read |11011208 | `/sites/<owner>/<repo>/<name>/` | a deployed site |11021209 11031210 ## Not yet@@ -1105,5 +1212,3 @@ may be missing needs an explicit check or `or ''`.
11051212 - Commenting and approving from the web pages; for now both come from the11061213 terminal.11071214 - Bugs or issues in the args.io style.1108-- Pages for private repos, behind a login. For now private repos are read1109- over ssh only.@@ -272,7 +272,7 @@ log sha256:6cc79019... 4 lines
272272 <figcaption><b>fig. 9</b>or a plain VPS</figcaption>273273 </figure>274274 </div>275-<p class="prose">Either way, Caddy serves the pages directory as plain files and hands on only three things: clones to <code>git http-backend</code>, <code>/events/</code> to the events server and, with a pages domain, sites to the pages server. The pages server runs as the caddy user, because deploys are untrusted content. A public repo is public everywhere: pages, sites and clones. A private one (no <code>git-daemon-export-ok</code>) is read over ssh only, by its owner, collaborators and readers, and has no pages.</p>275+<p class="prose">Either way, Caddy serves the pages directory as plain files and hands on only four things: clones to <code>git http-backend</code>, <code>/events/</code> to the events server, every path the pages directory has no file for to the private server and, with a pages domain, sites to the pages server. The pages server runs as the caddy user, because deploys are untrusted content. A public repo is public everywhere: pages, sites and clones. A private one (no <code>git-daemon-export-ok</code>) is read by its owner, collaborators and readers only: over ssh, on its pages, which are built into a root of their own and served by the private server once they sign in with a passkey, and over HTTPS with a token. To anyone else every page of it is the 404 of a page that isn't there. Passkeys and tokens come from ssh (<code>ssh git@host passkey</code>, <code>token</code>), and live, as hashes, in one SQLite file that litestream copies to the backup bucket as it changes.</p>276276 </section>277277 278278 <section class="commit" id="cli" aria-labelledby="cli-h">@@ -331,7 +331,6 @@ log sha256:6cc79019... 4 lines
331331 <header><p class="ref"><b>09</b> not yet</p><h2 id="not-yet-h"><a href="#not-yet">What isn't there</a></h2></header>332332 <ul class="gaps">333333 <li><b>Issues.</b> Bugs and tasks in the args.io style don't exist yet.</li>334-<li><b>Pages for private repos.</b> Private repos are read over ssh only for now; pages for them, behind a login, come later.</li>335334 <li><b>Notifications.</b> Nothing emails you. Open pages update themselves, and that is all.</li>336335 <li><b>Buttons.</b> Pages only show things. Landing, deploying and administering happen through git, ssh and <code>forge</code>.</li>337336 <li><b>CI on the server.</b> By design: it runs where the code is written, and the server checks signatures.</li>@@ -1,7 +1,9 @@
11 # Pages, plus clones: git clone https://git.example.com/dami/greet.git. Git's22 # paths, smart and dumb, go to git http-backend, which serves only repos with33 # git-daemon-export-ok (not private ones); nothing else inside a bare repo is4-# served. forge server caddyfile prints the full config.4+# served. A path www has no file for goes to the private server (private5+# repos' pages, sign-in, the 404). forge server caddyfile prints the full6+# config, which also lets a token clone a private repo (forward_auth).57 git.example.com {68 @git path_regexp git ^/[A-Za-z0-9_-][^/]*/[^/]+\.git/(HEAD|info/refs|objects/info/(alternates|http-alternates|packs)|objects/[0-9a-f]{2}/([0-9a-f]{38}|[0-9a-f]{62})|objects/pack/pack-([0-9a-f]{40}|[0-9a-f]{64})\.(pack|idx)|git-upload-pack)$79 handle @git {@@ -22,10 +24,17 @@ git.example.com {
2224 handle @sites {2325 redir https://pages.example.com{uri}2426 }25- handle {27+ @public file {28+ root /srv/git/.forge/www29+ try_files {path} {path}/30+ }31+ handle @public {2632 root * /srv/git/.forge/www2733 file_server2834 }35+ handle {36+ reverse_proxy 127.0.0.1:809337+ }2938 }3039 3140 # Set FORGE_SITES_URL=https://pages.example.com/sites/{owner}/{repo}/{site}/ and