1# forge23A personal forge built from plain git. The server is the setup from4[Git on the Server](https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server):5a `git` user, SSH keys and bare repositories. Pushes run two hooks, and an6[11ty](https://www.11ty.dev/) build writes static pages in the style of7[stagit](https://git.suckless.org/). The state is files in the bare repos,8a directory of HTML that any static file server can serve, and one SQLite9file of web sign-ins (passkeys, sessions, tokens) for private repos' pages.10Small Node servers sit next to it for what static files can't do: one11streams live page updates, one serves private repos' pages to their12readers, one serves repo sites on their own hostnames (see [What13runs](#what-runs)).1415It adds what stagit doesn't have, borrowing from [args.io](https://args.io):1617- **Every branch is browsable.** Log, files and blobs for each branch, plus a18 refs page with ahead and behind counts.19- **Reviews are stacked diffs.** Any pushed branch is a review. A branch built20 on another review's branch is detected as the next layer of a stack. A21 review page is its commits' diffs in order, plus a combined diff.22- **Revisions are recorded per push.** Force-pushing a review records a new23 revision. Its page shows what changed: the diff between the two tips, or24 after a rebase, each commit paired with its earlier version (`git range-diff`).25- **Discussion from the terminal, or the page.** `forge comment` and26 `forge approve` add comments (on a line or the whole review) and27 approvals, which the review page shows; signed in, the page adds them28 too. Approvals are keyed by tree like CI, and `main` can require them29 from someone other than the author, as many and from whom30 `.forge/approvals` says for the files a change touches. `forge land`31 checks both and fast-forwards.32- **Agent reviews, signed like CI.** `forge critique` runs the reviewer the33 target branch names (`.forge/review`) on your machine, posts what it finds34 as comments, and signs what is still open into a git note. An approval35 rule can ask for one with nothing blocking.36- **CI runs on your machine.** `forge attest` runs `.forge/ci` in a clean37 worktree. It signs the result with the SSH key you push with and stores it38 as a git note. Pages show the result for each commit, and `main` can require39 a verified pass to accept a push. CI can report numbers (bundle size,40 test count) that a repo's insights chart by commit.41- **Sites ship with repos.** Docs, Docusaurus builds or anything static is42 committed to `refs/sites/<name>` and served from the forge. Each review43 branch gets its own preview. The wiki is markdown in `wiki/`.44- **Releases are signed files.** `forge release publish v1.0 <file>...`45 puts a tag's binaries on `refs/releases/v1.0` with a `SHA256SUMS` signed46 by your key; the forge checks both and serves the files.47- **Keyboard navigation and clean URLs**, e.g. `/cat/greet/tree/main/greet.sh/`.48- **Blame and history.** A file on the default branch shows who last changed49 each line and every commit that changed it, each with the review it50 landed in.51- **Search and syntax highlighting.** `/` searches a repo's commits, reviews,52 wiki and file paths from a static index, and file contents from a SQLite53 index the browser reads in byte ranges; code is highlighted at build time.54- **Insights.** The forge's facts (pushes, builds, reviews, CI results) are55 kept in a [DuckLake](https://ducklake.select/), and static pages chart56 them for the forge and for each repo.5758The [guide](guide/index.html) explains all of this with diagrams. It is a59static site in `guide/`, declared in `.forge/sites`, so the forge serves it60next to this repo after `forge site deploy guide`.6162## Try it6364```sh65npm install66sh demo/make-demo.sh # throwaway forge in demo/root, real hooks over file:// pushes67python3 -m http.server 4817 -d demo/root/.forge/www68```6970The demo creates two users and a repo with:7172- a landed review and a two-layer stack, with both layers rewritten once,73 and a description, comments and approvals on it (one comment outdated by74 the rewrite, one approval stale);75- a contribution from the second user whose CI fails and then passes, and which `main`76 turns away because nobody approved it;77- a draft that grows, then is rebased onto a newer `main` and reworked;78- an abandoned branch, a wiki and a docs site with a preview;79- a release of `v0.1`: the script and a source tarball;80- a standalone site that the second user owns;81- code in JavaScript, Go, CSS, Python and markdown fences, and a review of a82 1,200-line file with long lines.8384Its clock is fixed: commits get set dates, and the hooks stamp reviews with85`FORGE_NOW` (an override for the demo and tests) instead of the wall clock.8687## How it fits together8889```90your machine server (git user, /srv/git)91------------ ---------------------------92git push ─────────── ssh, forced command ────> forge shell <user> -> git-shell93forge attest -> refs/notes/ci (signed) pre-receive: who may push what; CI and94 approval gates on main95forge site deploy -> refs/sites/<name> post-receive: record review revisions,96 refs/site-previews/... extract sites and releases, rebuild pages97forge release publish -> refs/releases/<tag>98 11ty build -> /srv/git/.forge/www (static)99 events.jsonl -> live page updates100forge comment, approve ─── ssh ──────────────> comment, approve: next to the review log101review page, signed in ─── HTTPS ────────────> the same, through the private server102```103104### What runs105106| Process | Why |107| --- | --- |108| sshd | git over ssh. Every key is forced through `forge shell <name>`. |109| Caddy | Serves `.forge/www` as plain files, read-only HTTPS clones (`git http-backend` through fcgiwrap; nothing else inside a bare repo is served; a private repo takes a token) and `go get` meta tags, with security headers (HSTS, nosniff, a referrer policy and a CSP against framing) on the forge's own pages; proxies `/events/`, every path `www` has no file for (to the private server) and, with a pages domain, sites. |110| `forge build` | Runs 11ty after each push, in the background and one repo at a time, then swaps the new pages into `www/`. Unchanged pages are hard links to the last build's. First it adds the push's commits to the repo's commit-graph (see [Cleaning up](#cleaning-up)). |111| `lib/events-server.ts` | Tails `.forge/events.jsonl`, which hooks and builds append to, and streams it as server-sent events at `/events/<owner>/<repo>/`. Open pages say who pushed and swap in fresh content when the build finishes. Without it pages still work; their status line says offline. A private repo streams only to its readers, signed in. At most 2,000 streams are open for people not signed in, and 16 from one address. |112| `lib/private-server.ts` | Serves private repos' pages to their readers, signed in with a passkey or holding a token, the sign-in pages (`/login/`, `/private/`), and the 404 of every page that isn't there; takes comments and approvals posted from review pages (see [Comments and approvals](#comments-and-approvals)); answers Caddy whether a token may clone a private repo over HTTPS (see [Private pages](#private-pages)). |113| `lib/pages-server.ts` | Only with a pages domain. Serves each site on its own hostname the way Cloudflare Pages does (`_headers`, `_redirects`, pretty URLs, the nearest `404.html` or a single-page-app fallback) and redirects old `/sites/...` paths there. Runs as the caddy user, since deploys are untrusted content. |114| `forge backup` | Daily, from a loop in `start.sh`. Snapshots the forge with restic into a bucket, then checks the snapshots and test-restores the new one (see [Backups and moving hosts](#backups-and-moving-hosts)). |115| `forge mirror` | After builds, in the background, at most hourly: syncs the forge's own pages, less those of single files and folders, to a bucket with rclone, for the edge to serve while the forge is down. |116| `forge lake` | After builds, in the background: adds what is new in the event log, build metrics, review logs and CI notes to the lake, and writes the insights exports (see [The lake and insights](#the-lake-and-insights)). |117| `forge identity replicate` | With backups set up: [litestream](https://litestream.io) copies `.forge/identity.db` to the backup bucket as it changes (see [identity.db](#identitydb-backed-up-as-it-changes)). |118119Without a pages domain, Caddy serves sites straight from `/sites/...` and120`/previews/...` on the forge's own origin, and `_headers` and `_redirects` don't121apply. `forge server caddyfile <address> [<pages-domain>]` prints the Caddy122config for either setup. It serves assets, whose names carry a hash of their123contents, with `Cache-Control: public, max-age=31536000, immutable`. It124serves pages, and the JSON and text indexes they fetch, with `no-cache`, so125browsers revalidate them with the ETag and Last-Modified Caddy sends and get126a 304 when nothing changed. Release downloads revalidate the same way, aren't127compressed and are sent as attachments (see [Releases](#releases)).128`demo/serve.ts` sends the same headers.129130### Identity and permissions131132`/srv/git/.forge/keys` holds `name ssh-ed25519 AAAA...` lines and is the133root of identity: a user is a name there, and everything else (web sign-ins134included, below) belongs to one. `forge keys sync` generates two files from135it:136137- `authorized_keys`, which forces every key through `forge shell <name>`, so138 hooks know who pushed;139- `allowed_signers`, so the same keys verify CI attestations, signed site140 deploys and release checksums (namespaces `forge-ci`, `forge-release` and141 `git`).142143Repos live at `/srv/git/<owner>/<repo>.git`, and `<owner>` owns them. Anyone144can read a public repo; a private one, only the people below. An owner's145page, `/<owner>/`, is there only while they have a public repo, so a user's146name alone isn't published. The owner and147anyone listed in `<repo>.git/forge/collaborators` can push anything. Other148users who can read it can push branches and previews under their own name149(`fox/...`) and add CI notes, which must fast-forward. Nobody can force-push150`main` unless `forge.allowforce` is set.151152```sh153ssh git@git.example.com collaborators cat/notes add fox # rm to take it back; no argument lists them154ssh git@git.example.com who cat/notes # its owner, collaborators and readers155```156157Only the owner changes a repo's collaborators; its collaborators can list158them, and no one else can. The name must be a user of the forge. `who` is159for anyone who can read the repo, so a private repo's is the same `no repo`160as a missing one's to everyone else. The operator has the same lists for161any repo: `forge collaborators <owner>/<repo> [add|rm <user>]` and `forge162readers`.163164#### People165166A user is added by the operator, with their SSH public key, or invited by167a link that lets them give it themselves:168169```sh170forge user add fox 'ssh-ed25519 AAAA... fox@laptop'171forge user invite fox [--ttl 7d] [--repo cat/notes --as reader|collaborator]172forge user invites [rm fox] # invites not taken up yet; rm makes a link dead173forge users # everyone: keys, passkeys and tokens; then the invites174```175176`invite` holds the name (nobody can be added or invited as it meanwhile)177and prints `https://<domain>/login/#invite=<code>`, which works once, for 7178days unless `--ttl` (at most 30d). The page it opens takes one SSH public179key (`ssh-*`, `ecdsa-*` or `sk-*`, on one line, one ssh-keygen reads and180nobody on the forge has already), then does what `user add` does: the line181in keys, the home directory, `keys sync`. With `--repo`, the new user is182that repo's reader or collaborator too. Then the same page offers to add a183passkey, as the link `ssh passkey` prints does. The code is in the URL's184fragment and identity.db keeps only its hash, like enrollment codes. The185posts come only from the forge's own page (Origin), at most 10 a minute from186one address with a live invite's code, one taking a key at a time. The code187is checked first: without a live invite every post gets the same answer,188whatever its key, so the page can't be asked whether a key has an account,189and posts with dead codes aren't counted. `user add` checks a key the same190way, so no key adds a second line to keys.191192#### Admins193194The forge's admins are the users named in `forge.admins` in the server's195git config, `/srv/git/.gitconfig` (the git user's own): one value each, or196several names split by spaces or commas. Owning repos doesn't make anyone197an admin. Set it on the server, as git:198199```sh200git config --file /srv/git/.gitconfig --add forge.admins cat # in the container:201ssh slate docker exec -u git forge git config --file /srv/git/.gitconfig --add forge.admins cat202git config --file /srv/git/.gitconfig --get-all forge.admins # the list; --unset-all forge.admins empties it203```204205An admin runs the people commands above over ssh, from their own machine,206the same as on the server:207208```sh209ssh git@git.example.com invite fox [--ttl 7d] [--repo cat/notes --as reader|collaborator]210ssh git@git.example.com invites [rm fox]211ssh git@git.example.com users212```213214They also set the home page's motd, the quiet line under its header215(the server's form is `forge motd`):216217```sh218ssh git@git.example.com motd "notes, tools and games" # one line of plain text, 72 characters at most219ssh git@git.example.com motd # print it220ssh git@git.example.com motd --clear # no line at all221ssh git@git.example.com motd --default # back to "a personal forge on plain git"222```223224It is kept in `.forge/motd` (no file is the default, an empty one no line),225which backups carry, and setting it rebuilds the home page.226227An admin's ssh session finds them in a `git-shell-commands` of their own,228`.forge/admin/git-shell-commands` (`forge server init` writes it), so to229anyone else they don't exist: git-shell answers `fatal: unrecognized230command`, with the exit status of any command that isn't there, and231`help` doesn't list them. A change to `forge.admins` counts from the next232ssh command on.233234`ssh git@git.example.com new <name>` creates a repo under your name, and so235does pushing to one that doesn't exist yet: `forge shell` sees the push before236git does, creates the repo the same way (same name rules, hooks installed),237prints `created cat/notes` and lets the push go on.238239```sh240git push git@git.example.com:cat/notes.git main241```242243The first branch pushed becomes the default if it isn't `main`. A push to a244missing repo under someone else's name, or a clone or fetch of one, fails245as before.246247`ssh git@git.example.com rm cat/notes --yes` takes a repo of yours off the248forge (without `--yes` it says what would go); collaborators can't, and249`forge repo rm <owner>/<repo>` does the same as root. Its clones, pages in250either root, sites, previews, hostnames, release downloads and insights go251at once, as do its rows in the lake, so a new repo of the same name starts252with no history. The index and owner pages are rebuilt before the command253returns, and a mirror sync within 30 seconds deletes its pages. The bare repo waits in254`.forge/trash/<owner>/<repo>.git.<time>` for `FORGE_TRASH_DAYS` (14) days,255backed up like the rest of the forge. `forge repo restore <owner>/<repo>`256puts back the newest removal if the name is free, and `forge gc` deletes it257after that.258259### Private repos260261A repo is public while its bare repo has a `git-daemon-export-ok` file. That262is git's own flag: `git daemon` and `git http-backend` serve only repos that263have it, and Forgejo toggles the same file. `forge repo new` adds it, and264`--private` leaves it out: `ssh git@git.example.com new --private <name>`,265`forge repo new --private` and the imports below.266267A private repo can be read by its owner, its collaborators and the names in268`<repo>.git/forge/readers`: over ssh, on the web once they sign in (see269[Private pages](#private-pages)), and over HTTPS with a token. To anyone270else it isn't there: a clone, fetch, archive, push or ssh command gets the271same answer, byte for byte, as for a repo that doesn't exist, however the272path is spelled, and so does every page of it. It has no entry on the273index, owner pages or `repos.json`. Its pages and live events go to its274readers only, and the pages mirror never sees it. Its sites and previews275aren't served unless `git config forge.publicsites true`. Its releases276are, to its readers only, next to its pages and over ssh (see277[Releases](#releases)). Pushes record reviews and print their links, which278work for its readers, and `forge reviews` reads them from the review logs279(without CI). The secret check still runs, since a secret stays in the280history when the repo goes public later; Jev isn't asked about private281repos.282283```sh284ssh git@git.example.com readers cat/notes add fox # rm to take it back; no argument lists them285ssh git@git.example.com who cat/notes # who reads it, and who may push it286ssh git@git.example.com private cat/greet287ssh git@git.example.com public cat/notes [--force]288ssh git@git.example.com ls # yours, and the private ones you may read289```290291The owner and collaborators flip a repo, the way Forgejo does. `private`292removes the flag, the repo's public pages, its entries on the index, owner293pages and `repos.json`, and its served sites, previews and their294hostnames. It moves its release downloads into the private root (and295their files into the private store, see [Releases](#releases)) and builds296its pages again there. Then it starts a mirror sync within 30 seconds,297which deletes them from the bucket. Clones, caches and copies298people already have can't be recalled. `public` first runs the secret check299over the whole history: every commit on a branch or tag, and the old review300revisions that review pages show, not only what a push adds. If it finds301something, the repo stays private unless you pass `--force`. Then it adds302the flag, removes its private pages, links the sites, previews and303release downloads again from their refs, and rebuilds the pages into304`www`. A flag added or removed305by hand moves the pages at the repo's next build, and `forge build` drops306any left in the wrong place.307308### Private pages309310A private repo's pages are built like any repo's, into311`.forge/private/www` instead of `www`, with the same layout: pages, search312index, `code.db`, release downloads, insights exports. The build writes313them in `.forge/private/build.<pid>` and swaps them in from there, so314they are never outside `.forge/private`, which only git can read, even315mid-build. They link the same hashed assets, which the build puts in316`www/assets` (they are the forge's scripts and styles, nothing of the317repo), and their header marks the repo private. Caddy serves a file of318`www` when there is one and hands every other path to319`lib/private-server.ts`, so it needs no list of private repos. That server320serves a private repo's page to someone who may read it (the rule321`can_read` in `bin/forge` applies, kept in step with `lib/access.ts` by a322scenario) and is still a user, signed in with a passkey or showing a token,323with the headers Caddy gives pages (ETag, 304s, byte ranges, which code324search reads `code.db` with) except `Cache-Control: private, no-cache`.325It serves the repo's release downloads as attachments, as Caddy sends a326public repo's. Not signed in, not allowed or a page that isn't there: the same 404, byte327for byte, which every missing page of the forge gets, public or not.328`/private/` lists the private repos you can read.329330Web credentials live in one SQLite file, `.forge/identity.db` (WAL mode,331only git reads it), and are made over ssh only, so there are no passwords332or emails, and losing them is fixed by running ssh again:333334```sh335ssh git@git.example.com passkey [name] # a link that adds a passkey: once, within 10 minutes336ssh git@git.example.com passkeys [rm <id>]337ssh git@git.example.com sessions [rm <id>|--all]338ssh git@git.example.com token [--repo cat/notes] [--ttl 30d] [name] # printed once339ssh git@git.example.com tokens [rm <id>]340```341342- **Passkeys** ([SimpleWebAuthn](https://simplewebauthn.dev) 14). The343 relying party is the forge's origin, `FORGE_BASE_URL`, whose host is the344 RP ID; responses from any other origin are refused. The link carries a345 one-time code in its fragment, so it reaches no log, and binds the new346 passkey to your ssh user. Sign-in on `/login/` needs no user name: the347 browser offers the forge's passkeys, which are discoverable, and user348 verification is required. On Node 24.7 and later (the container) ML-DSA-44349 is offered first, so a passkey can be post-quantum. After `passkeys rm`,350 the next page you open signed in tells your password manager which351 passkeys still work (the WebAuthn Signal API), so it stops offering the352 removed one; it also signs out the sessions that passkey made. The353 server keeps no challenges until they are used: each one carries its354 expiry and an HMAC, and is bound to the browser that asked for it by a355 `__Host-forge-challenge` cookie that lasts five minutes, so a flood of356 requests can't push out anyone's sign-in. Each address may ask for357 options 20 times a minute.358- **Sessions** are a `__Host-forge` cookie (Secure, HttpOnly, SameSite=Lax)359 that lasts 30 days from its last use. Sign-out is a POST from the forge's360 page, with a CSRF token. The events server streams a private repo to a361 page whose session may read it.362- **Tokens** are for one repo or for every repo you can read when they are363 used, for 30 days unless `--ttl` (at most 365d). Send one as364 `Authorization: Bearer <token>`, or as the password of basic auth:365 `git clone https://cat:<token>@git.example.com/cat/notes.git`, a366 credential helper or the token alone as the user name.367368identity.db keeps only SHA-256 hashes of cookies, tokens, enrollment369and invite codes, and every credential stops working when its user's keys go (`forge370user rm` deletes them too). Its SQL is the plain subset Turso runs as well.371HTTPS clones of a repo without `git-daemon-export-ok` go through Caddy's372`forward_auth`: the private server answers 401 without a token (so git sends373one) and, with one, 200 or the same 404 for a private repo and a missing374one; only then does `git http-backend` run, with `GIT_HTTP_EXPORT_ALL`.375376### Reviews377378`post-receive` appends each push of a non-default branch to379`<repo>.git/forge/reviews/<id>` and pins the tip under `refs/forge/keep/` so380old revisions survive. The generator works out the rest from refs:381382- **Stack:** the parent is the nearest other review whose tip lies inside this383 branch.384- **Layer:** the commits between the parent (or `main`) and the tip.385- **Status:** merged if the tip is in `main`; closed if the branch was deleted;386 draft if the branch is `wip/...` or pushed with `-o draft`.387388Push options set metadata: `forge push -t "Title" -d "Description" --draft --target <branch>`.389The description is one line (`\n` makes a line break, and so does a real390newline in the argument) and renders as markdown under the title, sanitized391like a README. Each push that creates or updates a review prints its link,392`review #7: https://git.example.com/cat/greet/reviews/7/ (on #6 cat/flags)`,393from `FORGE_BASE_URL` (just the path if that isn't set).394395### Comments and approvals396397Both are written from the terminal over the forge's ssh account, or from398the review page once you are signed in (see [On the web](#on-the-web)399below), and the pages rebuild in the background, so an open review page400updates live.401402```sh403forge reviews # open and draft reviews: CI, approvals404forge checkout 7 # fetch the branch, switch to a tracking branch405forge comment -m "Looks good" # the current branch's review406forge comment 7 greet.sh:12 # a line comment; the message from $EDITOR or stdin407forge approve 7408forge land 7 # what the server gates check, then fast-forward409ssh git@git.example.com land cat/greet 7 # the forge lands it itself; no clone needed410```411412- **Comments** are files, `<repo>.git/forge/comments/<review>/<n>`: header413 lines (time, user, the review's tip and revision, and for a line comment the414 path, the new-side line number and that line's text), a blank line, then the415 markdown message (at most 32 KiB). Any registered user can comment, or on a416 private repo anyone who can read it. Once the review gets a new revision,417 older comments show as outdated and still quote the line they were about.418 Line comments on the current revision also appear under their line in the419 combined diff. A comment made from a checkout belongs to the revision420 checked out, and one from a page to the revision the page shows.421- **Approvals** are lines in the review log, `approve <ts> <user> <sha>422 <tree>`. Like CI results they are keyed by tree, so rewording or423 reordering commits keeps them; if the tree changes, the page shows424 "approved r1, changed since". Authors can't approve their own review.425- **Gate:** `git config forge.requireapproval true` in a bare repo means a426 push that moves the default branch needs approvals of the new tip's tree427 from someone other than that review's author: one, or what the approval428 rules (below) say. It works alongside `forge.requireci`.429- **Landing:** `forge land [<id>|<branch>]` checks that every commit between430 the target and the branch has a verified passing attestation and, if the431 repo requires it, the approvals the rules ask for (`--force` turns the local checks into432 warnings; the server's gates still apply). It refuses anything that isn't a433 fast-forward, pushes the branch to its target (the target of the stack's434 lowest layer), then deletes the landed branch and the stack layers below435 it.436- **Landing on the server:** `ssh git@host land <owner>/<repo> <id|branch>`,437 or `forge land --server` from a clone, lands a review without fetching438 anything, for agents or a phone. The forge resolves the review's tip and439 target from the review logs and refs, then pushes inside itself, from the440 bare repo to the bare repo, as you: the tip to the target and deletions of441 the branch and its landed layers, in one atomic push with leases. So442 `pre-receive` applies exactly the rules any push meets (who may move the443 default branch, `forge.requireci`, `forge.requireapproval`, fast-forward444 only unless `forge.allowforce`), and `post-receive` does what it does after445 one: the review logs record the deleted branches, their previews go, an446 event is appended and the pages rebuild. If it refuses, nothing changes.447448`forge reviews`, `checkout` and `land` read `/<owner>/<repo>/reviews.json`,449which the page build writes; `ssh ... land` reads the review logs, so it can450land a review pushed a moment ago. The ssh commands live in451`git-shell-commands/`; on an existing server, run `forge server init` again to452add them (the container does that on every boot).453454#### Approval rules455456`.forge/approvals` on the default branch says how much approval a change to457it needs, by the files the change touches (`lib/rules.ts`):458459```460# paths rule461.forge/ bin/forge 2462lib/identity.ts lib/login.ts 1 from fox463lib/ 1 + agent464*.md guide/ wiki/ none465@mechanical none466* 1467```468469- **Matching.** Each changed file takes the first line it matches. A470 pattern without a slash matches a file's name in any directory; with one,471 from the top. `*` is any run of characters but `/`, `**` any run at all,472 `?` one character, and a trailing `/` everything under that directory.473 `@mechanical` matches files Jev judged mechanical (the ones review pages474 fold), from its stored answers; a file with none isn't. A file no line475 matches needs 1, as it would without the file. A rename counts both its476 paths, so moving a file out of a guarded directory needs that rule.477- **Rules.** `none`, or a number of approvals, optionally `from` a478 comma-separated list of people (that many of them), optionally `+ agent`:479 a verified agent review of the tree with no blocking finding open (see480 [Agent reviews](#agent-reviews)). A change needs every rule its files481 took. A line the parser can't read (or `3 from fox`, which can never be482 met) is an error the gate reports with its line, and holds.483- **Whose rules.** The default branch's, as they were before the push: a484 review can't loosen its own, so a change to `.forge/approvals` lands under485 the old ones (and `.forge/` above makes that need two).486- **Where they show.** The gate (`pre-receive`, with487 `forge.requireapproval`) prints what is missing. `forge land` asks the488 forge (`ssh ... approvals`) for the same verdict before it pushes. A live489 review's page starts with a **to land** panel: CI, each rule its files490 take, who approved, and its latest agent review. Without491 `forge.requireapproval` the panel says approval is optional there.492493#### On the web494495Signed in with a passkey (see [Private pages](#private-pages)), a review496page ends in a comment box and an Approve button. The combined diff497puts a + beside the line under the pointer, which opens a box for a498comment on that line. Pages are static files, the same for everyone, so499these start hidden and do nothing without JavaScript: a script asks500`/login/whoami` (never cached) who is signed in and shows them to a user,501without Approve for the review's author. A page posts the tip it shows,502like `forge comment` from a checkout: if the review has moved on since it503was built, a comment goes to the revision you read, and an approval is504refused. Until the rebuilt page arrives, a sent comment shows as pending,505as typed; the build renders and sanitizes its markdown, as for any other.506507The posts go to the private server, `POST508/<owner>/<repo>/reviews/<id>/comment` (fields `body`, `at`, `line` as509`<path>:<n>`) and `.../approve` (`at`), as a form or JSON, and it runs510`forge shell-cmd comment|approve` as you with the message on stdin, the511command ssh runs. So a comment file, an approve line, the rules and the512messages are the same either way, byte for byte. A session's post must513come from the forge's own origin and carry its CSRF token (the page gets it514from `/login/whoami`); a token works as `Authorization: Bearer` without515one, for curl, and a token for one repo only there. Basic auth doesn't516post, since browsers send it by themselves. A private repo you can't read517answers 404, the same as a missing one. Each user may post 20 times a518minute. Caddy sends these posts to the private server before it looks in519`www`, so public reviews, whose pages are plain files, take them too.520521```sh522curl -H "Authorization: Bearer $token" -d 'body=Looks good' -d 'line=greet.sh:12' \523 https://git.example.com/cat/greet/reviews/7/comment524```525526### CI attestations, and what they prove527528`forge attest [--stack]` checks out each commit into a temporary worktree and529runs `.forge/ci`. If a `flake.nix` exists, it runs inside `nix develop`. It530then writes a statement like this and signs it with531`ssh-keygen -Y sign -n forge-ci`:532533```534forge-attestation 1535commit 2aa8af0...536tree 58945d0...537script .forge/ci 99a0d5e...538result pass539exit 0540duration 41s541runner cat@laptop542system Darwin 25.5.0 arm64543env host544log sha256:... 120 lines545|ok greet --name fox546-----BEGIN SSH SIGNATURE-----547```548549`runner` is your forge name and a made-up name for the machine:550`git config forge.user`, else the name `allowed_signers` gives your signing551key; and `git config forge.machine`, else two words and two hex digits552picked once from a hash of the hostname and a random salt, then kept in your553global git config. Never the hostname or your local account, since the554statement is published; site deploys and releases name their builder the555same way (`Built-By`). Pages don't show either: they show who signed.556557The statement is appended to `refs/notes/ci`, and `forge push` merges and558pushes those notes with the branch. Results are keyed by **tree**, so a559reworded or reordered commit with an identical tree keeps its result without560re-running.561562What a verified attestation proves: the holder of a registered key claims that563this exact tree passed this exact script on a machine they describe. Tampering564is caught; changing `result fail` to `result pass` breaks the signature. A565dishonest key holder is not caught. To make the claim stronger:566567- keep the key in hardware (a Secure Enclave key via568 [Secretive](https://github.com/maxgoedjen/secretive), or a FIDO `sk-` key)569 so it is bound to one device;570- use Nix so the environment is pinned by `flake.lock`, which is recorded in571 the statement;572- have a second person run `forge attest` on their machine. Attestations from573 several signers stack up on the same tree.574575`.forge/ci` can also report numbers about the commit: bundle size, test576count, benchmark times. It writes them to the file named by `$FORGE_METRICS`,577a line each, as a name, a number and an optional unit:578579```sh580echo "bundle_kb $(du -k dist/app.js | cut -f1) KB" >>"$FORGE_METRICS"581echo "tests $(grep -c '^ok' test.log)" >>"$FORGE_METRICS"582```583584They go into the statement as `metric bundle_kb 412 KB` lines, before the585log, so they are signed with the result. A name is lowercase letters, digits586and `_ . -`, starting with a letter; a unit is up to 16 letters, `%` or `/`.587`forge attest` leaves out, with a warning, a line that isn't one, a name588seen before and anything past the 100th. Like the result, they are what the589signer's machine said. The repo's insights page charts each one by commit on590the default branch (see [Insights pages](#insights-pages)).591592To require a verified pass before `main` moves, set593`git config forge.requireci tip` (or `all`, for every new commit) in the bare594repo.595596### Agent reviews597598`forge critique [<id>|<branch>]` reviews a review with an agent, on your599machine, the way `forge attest` runs CI (`lib/critique.ts`):6006011. It reads the reviewer, `.forge/review`, and its rules,602 `.forge/review.md`, from the review's **target** branch, so a change603 can't bring its own reviewer or loosen the rules.6042. It runs the reviewer in a clean worktree of the review's pushed tip,605 with the diff on stdin. The first time that is everything since the606 target. After that, while the last reviewed tip is still in the branch,607 only what changed since. Files Jev judged mechanical are left out.6083. It posts each finding as a comment, with `agent <model>` and609 `severity blocking|note` in its header: the page marks them **agent** and610 **blocking**. A finding on a file the review doesn't change, or past a611 file's end, goes on the whole review or the file's first line, with a612 warning.6134. It signs a statement into `refs/notes/review` on the tip, with614 `ssh-keygen -Y sign -n forge-review`, and pushes it: what it reviewed,615 which reviewer and rules (by blob), the model, how many findings and how616 many blocking ones are still **open**, carried over from earlier runs.617 It exits 1 while any is.618619```620forge-review 1621review cat/greet#7622commit 2aa8af0...623tree 58945d0...624since 3c1e2aa...625scope incremental626script .forge/review 99a0d5e...627rules .forge/review.md 1b2c3d4...628model claude-opus-5-5629reviewer cat@gentle-yak-2e630started 2026-11-04T10:00:00Z631findings 1632open 0633|{"id":"2aa8af0-1","severity":"note","path":"lib/a.ts","line":12,"body":"..."}634|{"resolved":"9f8e7d6-1","by":"agent"}635-----BEGIN SSH SIGNATURE-----636```637638The reviewer is any program. It gets the diff on stdin, and639`FORGE_REVIEW_ID`, `FORGE_REVIEW_BASE` and `FORGE_REVIEW_TIP` (the commits640the diff is between), `FORGE_REVIEW_SCOPE` (`full` or `incremental`),641`FORGE_REVIEW_RULES` (a file with `.forge/review.md`, or empty),642`FORGE_REVIEW_OPEN` (the open blocking findings, a JSON object a line),643`FORGE_REVIEW_SKIPPED` (the paths left out) and `FORGE_REVIEW_FIX` (1 with644`--fix`). It prints JSON objects, a line each; anything else is ignored:645646```647{"model":"claude-opus-5-5"}648{"severity":"blocking","path":"lib/a.ts","line":12,"body":"markdown"}649{"severity":"note","body":"about the whole change"}650{"resolved":"9f8e7d6-1"}651```652653At most 50 findings a run, each up to 4,000 characters. `resolved` names an654open finding the change dealt with. A reviewer with Claude Code, for one:655656```sh657#!/bin/sh658# .forge/review: Claude Code, headless and read-only, in the worktree of the tip.659diff=$(cat)660{661 echo "Review this change. The project's rules:"662 cat "${FORGE_REVIEW_RULES:-/dev/null}"663 echo "Findings still open from earlier runs (print {\"resolved\":id} for any the change fixes):"664 cat "$FORGE_REVIEW_OPEN"665 echo "Answer with JSON objects only, one a line:"666 echo '{"severity":"blocking"|"note","path":"...","line":N,"body":"markdown"}'667 echo "The diff:"668 printf '%s\n' "$diff"669} | claude -p --model claude-opus-5-5 --allowedTools Read Grep Glob670echo '{"model":"claude-opus-5-5"}'671```672673- **Again on the same tree** does nothing but say what is open; `--full`674 reviews everything again.675- **Dismissing.** `--resolve <id>,...` signs a statement without those676 findings, naming you as who resolved them, without running the reviewer677 (on the same tree) or along with it.678- **Fixes.** With `--fix` the reviewer may change files in the worktree;679 what it changed becomes a commit on `<you>/fix-<id>`, pushed as a review680 stacked on this one. The review itself is untouched.681- **What it proves.** Like an attestation: the key holder claims this682 reviewer, with these rules, said this about this tree. The review page's683 **to land** panel shows the latest one, and an approval rule's `+ agent`684 wants one of the tip's tree, verified, with `open 0`. Keys may sign them685 once `forge keys sync` (which `forge server init` and the container's686 boot run) writes `forge-review` into `allowed_signers`.687688### Sites689690A site deploy is a commit on `refs/sites/<name>` whose tree is the built691output, so git provides what nzip builds by hand:692693- content-addressed storage and dedup across deploys;694- history, with revert by pushing an older deploy;695- provenance through commit trailers (`Source-Commit`, `Build`, `Built-By`),696 plus an SSH signature when `gpg.format=ssh` is set.697698Declare sites in `.forge/sites`:699700```701# name output build command702docs build npm ci && npm run build703```704705`forge site deploy docs` builds from a clean worktree of `HEAD` and pushes the706result. On `main` it goes to `refs/sites/docs`. On any other branch it becomes707that review's preview, `refs/site-previews/docs/<branch>`. The build gets708`FORGE_SITE_BASE`, the path it will be served from, so Docusaurus can use709`baseUrl: process.env.FORGE_SITE_BASE`. `forge site push <dir> <name>`710publishes a directory you already have.711712`post-receive` extracts each tree once into `.forge/store/<tree>/`, then points713`www/sites/<owner>/<repo>/<name>` at it with a symlink. Only the owner can714deploy `refs/sites/*`.715716Deleting a branch deletes its previews: `post-receive` removes717`refs/site-previews/*/<branch>`, the served link and the preview's hostname,718so the sites and review pages stop listing it. Previews of merged branches719stay until their branch is deleted. Their extracted files stay in the store720until `forge gc` (below).721722### Releases723724A release is a commit on `refs/releases/<tag>` whose tree is the release's725files plus `SHA256SUMS` and `SHA256SUMS.sig`, the way a site deploy is a726commit whose tree is the built output.727728```sh729git tag v1.0730forge release publish v1.0 dist/greet-linux-amd64 dist/greet-1.0.tar.gz731forge release verify --url https://git.example.com/cat/greet/releases/download/v1.0/ \732 ~/Downloads/greet-linux-amd64 # SHA256SUMS and its .sig next to it733forge release ls # tag, file, size, sha256734forge release rm v1.0735```736737`publish` writes `SHA256SUMS` in `sha256sum`'s format, so `sha256sum -c`738reads it, and signs it for the release with `ssh-keygen -Y sign -n739forge-release`, using the key that signs CI results. What it signs is740`SHA256SUMS` behind a short header naming the repo and tag:741742```743forge-release 1744repo cat/greet745tag v1.0746<the bytes of SHA256SUMS>747```748749so the signature holds for that release only: v1.0's files can't pass for750v1.1's, nor a copy in another repo for the original. The signature is751`SHA256SUMS.sig`. Then it makes a commit with trailers `Tag`,752`Source-Commit` (the tag's commit) and `Built-By`, signed like a site753deploy, and pushes the commit, the tag and your CI notes in one atomic754push. The commit has no parent, so publishing under the tag again replaces755the files and the old ones aren't kept reachable.756757`verify` checks files against `SHA256SUMS`, and its signature for the758release they say they are from: `--repo` and `--tag`, else `--url`, else759the path they were saved under if it is a download path (`wget -x`), else760the remote for the repo. It says which release it verified (`ok761greet-linux-amd64: matches SHA256SUMS of cat/greet v1.0, signed by cat`).762The forge's keys come from `--signers <file>`, or `ssh <host> signers` with763`--host`, `forge.allowedsigners`, `forge.host` or the remote.764765`pre-receive` takes a release from the owner and collaborators only, like a766tag, and only if:767768- the tag exists once the push lands (it may come in the same push), and the769 trailers name it and its commit;770- the tree is plain files at the top level, named with letters, digits and771 `. _ + -`, none over `git config forge.releasemaxsize` (256m unless set;772 git's k, m and g work);773- `SHA256SUMS` lists every file once with its sha256, and nothing else;774- `SHA256SUMS.sig` is a `forge-release` signature of it for this repo and775 the ref's tag, by the key of the repo's owner or a collaborator: another776 registered user's won't do, even pushed by the owner;777- with `git config forge.releaserequireci true`, the tag's tree has a778 verified passing attestation, as `forge.requireci` asks of `main`779 (`forge attest v1.0` before publishing).780781The secret check skips releases. It reads branches and tags, and a782release's source is its tag.783784`post-receive` extracts the tree once into `.forge/store/<tree>/`, as for a785site (a private repo's into a store of its own, below), and hard-links its786files into787`/<owner>/<repo>/releases/download/<tag>/` (a `/` in the tag becomes `~`).788`latest/` next to them holds the files of the newest tag that has a789release, so `/cat/greet/releases/download/latest/greet-linux-amd64` keeps790working from one release to the next. Hard links, not a symlink as for a791site: rclone skips symlinks, and the [mirror](#the-pages-mirror) copies792these. The page build links them into the repo's new pages before it swaps793them in. Caddy serves them uncompressed. It sends `no-cache`, since a tag794can be published again, and `Content-Disposition: attachment`, since they795are anyone's files on the forge's own origin. Deleting the ref removes them.796797Clones and fetches don't see `refs/releases` (`uploadpack.hideRefs`), so798nobody downloads binaries with the source; pushes do, so `publish` can799replace and `rm` delete them. Blobs over 16M are stored without deltas800(`core.bigFileThreshold`). The releases page lists each tag's files, with801size and sha256, whether `SHA256SUMS.sig` verifies and whose key signed it,802and the CI result of the tag's commit.803804A private repo's releases go where its pages are, the private root805(`.forge/private/www`), with the same layout, never `www`, so the mirror806doesn't see them and `forge.publicsites` doesn't change that. The private807server gives them to its readers, signed in or showing a token (see808[Private pages](#private-pages)) as Caddy gives a public repo's809(attachments, uncompressed, byte ranges) but `Cache-Control: private,810no-cache`, and anyone else gets the 404 of a file that isn't there.811812Their files aren't extracted into `.forge/store` either, but into813`.forge/private/store/<tree>/`, and linked from there. `.forge/store` is814readable by the caddy user, which serves sites from it; nothing serves a815release tree from a store, but a private repo's files shouldn't sit where816anything but the forge can read them, so they live under `.forge/private`,817which only git can read (mode 700). Both stores are under `.forge`, so the818downloads stay hard links on one filesystem. A repo that goes private takes819its downloads to the private root at once, linked from the private store820(extracted there), and one that goes public brings them back to `www`,821linked from `.forge/store` (extracted there again); `forge gc` frees the822trees left in the store it no longer uses.823824Readers can also get them over ssh, and to anyone else these commands825answer as for a missing repo:826827```sh828curl -fO -u fox:<token> https://git.example.com/cat/notes/releases/download/v1.0/tool # token --repo cat/notes829ssh git@git.example.com release ls cat/notes830ssh git@git.example.com release get cat/notes v1.0 tool > tool # also SHA256SUMS, SHA256SUMS.sig831forge release verify --repo cat/notes --tag v1.0 tool832```833834A verified download proves what a verified attestation does, about these835bytes and the release they name: the holder of a key the forge knows836vouched for this `SHA256SUMS` as that repo's tag. The forge took it only837from an owner's or collaborator's key; `verify` checks the key against the838forge's keys file, and leaves who administers the repo to that.839840### Search and highlighting841842`/`, or the search button in the header, opens a search over the repo's843commits (subject, body, sha, author), reviews (number, title, branch,844author), wiki pages, file paths and the contents of the files on the845default branch. On the index and owner pages it searches repo names and846descriptions. There is no server: each build writes847`/<owner>/<repo>/search.json` (and `/repos.json`), which the page fetches the848first time the search opens, and matching runs in the browser. Commit bodies849are cut to 240 characters and wiki pages to 2,000, to keep the index small.850`t` still finds files by name on the current branch.851852Every section reads a query the same way (`site/assets/query.ts`): terms853split on whitespace, in any order, and a quoted phrase (`"fix the search"`)854is one term, spaces and all, that must appear as written, except that a run855of whitespace in it matches any run of whitespace: `"fix the"` finds856`fix the`, and `"fix the"` finds `fix the`. Every term must857match, ignoring case, and results mark each match, a phrase as a whole. A858quote opens a phrase only at the start of a term (in `it's"` it is just a859character). An unclosed quote runs to the end of the query as a phrase, so a860phrase narrows as you type it, and empty quotes are no term. So861`fix search` finds "Search fix for the box" and `"fix the search"` doesn't.862`node test/search-phrases.ts` runs the same code in Node, for863`test/scenarios/search-phrases.sh`. Pages load `search.ts` as a module; it864and `codesearch.ts` import `./query.ts`, which a build rewrites to that865file's content-hashed name.866867#### Code search868869File contents are in a second index per repo, `/<owner>/<repo>/code.db`,870which `search.json` points at (`"code"`). It is a SQLite database with an871[FTS5](https://sqlite.org/fts5.html) trigram index, and the browser never872downloads it whole: the official [sqlite-wasm](https://sqlite.org/wasm) build873opens it through a small read-only VFS (`site/assets/codesearch.ts`) whose874reads are HTTP range requests, so a query fetches only the pages it touches.875Any static server that honors `Range` works: Caddy, `demo/serve.ts` and the876pages mirror on R2 through the edge, whose fallback passes the header on.877878Once the query has a word of three characters (a trigram can't match less), it879goes to a worker, and a Code section lists each line that has every word,880ignoring case, as `path:line`, linked to that line of the file's page (the881source page for markdown). Its words are the terms the other sections read,882so a quoted phrase (`"import fs from"`) is one literal string, spaces and883all, in which a run of whitespace matches any run, tabs and alignment too;884a word inside another goes, since a line with the longer one has it.885Words of three characters or more are looked up886in the index; shorter ones only filter the lines it finds. A phrase is887looked up by its pieces between whitespace, those of three characters or more,888since no trigram may span whitespace that can be any length: all of them889within `NEAR` distance of the phrase's length plus 32 characters per gap,890and the lines found are checked with any run of whitespace for a space. A891phrase without such a piece (`"x = 1"`) is looked up as written, so it finds892only single-spaced lines. It loads nothing893until then: the first code query starts the worker, which loads894`sqlite3.wasm` and sqlite-wasm's script, both content-hashed assets. Results895come in path order; the first look reads 12 chunks, and "more" reads up to896100. The index finds chunks with every word, and their words can be on897different lines, so a first look that finds chunks but no line reads on, once,898up to 100.899900What goes in: every file on the default branch except binary files (a NUL in901the first 8,000 bytes, as git decides), files over the 256K highlight cap,902lockfiles (`package-lock.json`, `yarn.lock`, `go.sum`, `Cargo.lock`, ...),903minified files and source maps, images, symlinks and submodules. Files are904stored as chunks of eight lines (`file`, `chunk` and the FTS5 table `code` over905the chunks' text, with `detail=full`, which substring queries need); the906browser finds the line inside a chunk itself. The build writes 16K pages,907merges the index into one segment (`optimize`) and `VACUUM`s, so every table908is its root page over its leaves, laid out in order.909910Each round trip counts, since SQLite reads one page at a time and waits for911it. So the worker names most of a query's pages before SQLite asks: it opens912the file by fetching both ends at once (the schema, every table's root and913the file list are at the front; FTS5's term index is at the back), looks up914where each term's doclist lives in that term index and fetches those pages915together, then fetches the pages of the matching chunks together. A query916looks up every third trigram of each word (and the last), within `NEAR`917distance, rather than all of them, and checks the lines for the exact words.918The words of a query are looked up in the same round trips as one word.919If the index is rebuilt while a page is open, the next query notices the new920ETag and reopens it.921922Measured with `node test/code-search-bench.ts bench <state>`, which serves a923built forge with `demo/serve.ts` and runs the same worker code in Node:924925| Repo | Indexed text | `code.db` | Cold query | Next queries |926| --- | --- | --- | --- | --- |927| this repo, 180 files | 1.0M | 4.2M | 3 round trips, 5 to 13 requests, 240K to 432K | 1 or 2 round trips, up to 160K |928| the same, two or three words (`Cache-Control private`, `fs readFileSync json`) | | | 3 round trips, 9 to 16 requests, 416K to 480K | 1 or 2 round trips, up to 128K |929| an 11M tree (eleventy, markdown-it, highlight.js, typescript's `tsc.js` and more), 662 files | 4.4M | 16.1M | 3 round trips, 6 to 12 requests, 290K to 400K | 1 or 2 round trips, 80K to 160K |930931Of a cold query, 128K is the file's head and 64K its tail, read once per932page. An index under 256K is read whole on the first query. The first code933search also costs the assets, once and then cached for good: `sqlite3.wasm`934849K (394K gzipped) and sqlite-wasm's script 628K (171K gzipped). Building the935index took about 0.3s for this repo and 1.5 to 2s for the 11M tree.936937The index is cached in `.forge/cache/code/<owner>/<repo>/` by the default938branch's tree id and hard-linked into the pages, so a push that leaves that939tree alone (a review branch, an empty commit) keeps the very same file. A940new tree starts from the previous index and only deletes and inserts the941files whose blob changed: about 0.35s for one changed file in the 11M tree.942Don't compress `code.db` in transit: a range of a compressed response is not943a range of the file. `forge server caddyfile` compresses `.wasm` and scripts944but not the index.945946Files, diffs and fenced code in markdown are highlighted at build time with947[highlight.js](https://highlightjs.org/), so pages need no script for it. The948language comes from the file name, or the `#!` line of an extensionless949script; unknown types and files over 256K stay plain rather than being950guessed at. Each diff hunk's old and new sides are highlighted as whole text,951so strings and comments that span lines mostly come out right. Highlighted952files are cached in `.forge/cache/highlight/` by blob id, so pushing a new953branch doesn't redo files that other branches already have. Token colors are954CSS variables (`--tok-*`) in `style.css`.955956Mermaid diagrams in markdown (` ```mermaid ` fences) are drawn at build957time too, as inline SVG, by [Pele](https://pele.run), so pages ship no958diagram script. `style.css` colors them through Pele's `--pele-*`959variables, which follow light and dark mode without drawing them again.960There is no browser at build time to measure text in, so labels are sized961from a table of the sans font's character widths in `lib/diagrams.ts`962(`lib/text-widths.sh` makes it from the font). One wider than a phone is963drawn a second time, laid out for a phone's width (often top to bottom),964and the stylesheet shows that one in a narrow column. A diagram that965doesn't parse shows its source and the reason instead. Markdown is anyone's966who can push a repo: Pele escapes a diagram's text and filters its links967and styles. Then the SVG goes through a sanitizer of its own that keeps968only the shapes, text and links Pele draws with (no images, no `url()`),969while raw SVG in the markdown itself is still dropped.970971## The lake and insights972973The forge's facts are append-only, but they don't last: `events.jsonl` is974cut to its last 1,000 lines once it passes 1 MB, `metrics.jsonl` isn't backed975up, and review logs and CI notes are spread over the repos. `forge lake`976gathers them into a [DuckLake](https://ducklake.select/) (DuckDB 1.5.6977through `@duckdb/node-api`): its catalog is a SQLite file,978`.forge/lake/catalog.sqlite`, and its data Parquet files in979`.forge/lake/data`, or in a bucket (below).980981```sh982forge lake # add what is new; builds run it by themselves983forge lake --dry-run # say what it would add984forge lake query "SELECT repo, count(*) FROM events WHERE type = 'push' GROUP BY repo"985forge metrics # build history, from the lake as well as metrics.jsonl986```987988Tables: `events` (pushes, comments, approvals, builds), `builds` and989`build_repos` (each line of `metrics.jsonl`), `reviews` (each line of a990review log: created, revisions, approvals, deleted), `review_ends` (landed991or closed, and when) and `ci` (each attestation: commit, tree, result,992duration, runner, signer; `ci.parquet` leaves the runner out), `ci_metrics`993(each metric an attestation reported, with its id) and `commits` (each994commit on a default branch: time, author, subject and tree). A review counts as landed when its branch is995deleted with its tip in the default branch, as `forge land` leaves it; one996whose tip reaches the default branch while the branch stays counts from997when the lake saw it.998999- **Incremental.** Each source remembers how far it was read, in the same1000 transaction as its rows, so a run that dies adds nothing and a second run1001 adds nothing new. For `events.jsonl` that is a byte offset plus the1002 file's inode, a hash of its first line and hashes of the last lines read:1003 when the build cuts the file into a new one, the next run finds the last1004 line it read and goes on from there. If the cut went past it (more than1005 1,000 lines came in between runs), it says so, and only those lines are1006 missing. Review logs are read again whole when they change, CI notes when1007 their ref moves.1008- **One writer.** A DuckDB file can't be written by two processes, and1009 DuckLake over SQLite turned away half of 120 concurrent inserts with1010 "database is locked". So `forge lake` holds `.forge/lake.lock`; a second1011 one waits for it (`FORGE_LAKE_WAIT` seconds, 300) or gives up and says1012 so. Builds start it the way they start the mirror, `FORGE_LAKE_DELAY`1013 seconds (10) after the last one; `FORGE_LAKE=0` stops that.1014- **New columns.** A table that gains a column in a newer forge gets it1015 when the lake is next opened to write, empty in the rows it had.1016- **Small writes.** DuckLake keeps inserts under 10 rows in the catalog1017 rather than writing a Parquet file for each; about once a day1018 (`FORGE_LAKE_CHECKPOINT` hours) a run checkpoints, which writes them out1019 and merges small files.1020- **Read-only queries.** `forge lake query` attaches the lake read-only,1021 runs only `SELECT`, `EXPLAIN` and `PRAGMA` statements, and can't touch1022 files other than the lake's data. `--json` and `--csv` change the output.1023- **Offline.** DuckDB downloads extensions at first use. The image installs1024 them at build time (`node lib/lake.ts install`, which checks they are the1025 versions this was tried with), so the container never does. Elsewhere1026 they go to `~/.cache/duckdb/extensions` the first time.10271028Data on R2 instead of the volume: put `FORGE_LAKE_DATA=s3://<bucket>/lake`1029and `FORGE_LAKE_ENDPOINT=https://<account-id>.r2.cloudflarestorage.com` in1030`forge.env` (the `FORGE_BACKUP_*` keys are used unless `FORGE_LAKE_KEY_ID`1031and `FORGE_LAKE_SECRET` are set). Set it before the first run; the catalog1032stays on the volume either way. This was tried against `rclone serve s3`,1033not R2 itself.10341035### Insights pages10361037`/insights/` covers the forge, and each repo has `/<owner>/<repo>/insights/`1038(the Insights tab, `i`): stat tiles, then build time, pushes, time to land,1039revisions per review, CI results and CI duration. A repo whose CI reports1040metrics also gets a chart for each, its value at each commit on the default1041branch (the latest 400): a commit takes the value from an attestation of its1042tree, so a reworded commit keeps it, and a verified, passing one comes1043first. The forge's page has none, since a metric's name means something1044only in its repo. The pages are static and1045built once; their script (`insights.js`, on these pages only) fetches1046`insights.json` and draws the charts as SVG, light and dark, with a tooltip1047on hover and keyboard focus and a table of each chart's numbers. After each1048run, `forge lake` writes that JSON and the rows behind it as Parquet1049(`events`, `builds`, `reviews`, `ci`, `ci_metrics`) into `.forge/lake/export` and straight1050into the pages, so they update without a build, and the mirror picks them1051up. A repo's exports sit in its page directory, which builds keep. DuckDB1052reads the Parquet files over HTTP in byte ranges:10531054```sh1055duckdb -c "SELECT result, count(*) FROM 'https://src.rest/insights/ci.parquet' GROUP BY result"1056```10571058Private repos are in no public insights: the forge's exports leave their1059facts out, and `www` has no page of theirs. A private repo's own insights1060page and exports are in its pages in the private root, which `forge lake`1061keeps fresh the same way, for its readers only (see [Private1062pages](#private-pages)); the mirror never sees them. Making a repo private1063removes the forge's exports from `www` at once; the next run writes them1064without it. Making it public again publishes its facts from while it was1065private as well, as its whole history becomes public then. The builds1066exports give a build's reason as `push` or `other`, since the others would1067say when a repo went private or away.10681069The page has no query console. [DuckDB-Wasm](https://duckdb.org/docs/api/wasm/overview)1070could run SQL over those files in the browser, but its build is 34.2 MiB1071(7.7 MiB gzipped, 5.1 MiB with brotli) to query exports of a few kilobytes,1072and its npm package is 149 MB.10731074What it costs, measured on an M-series Mac and in the image (linux/arm64):10751076| | |1077| --- | --- |1078| `@duckdb/node-api` | the native DuckDB library: 61M in the image (71M for x64; npm also fetches the musl build, which the image drops) |1079| Extensions | 80M in the image: ducklake 30.7M, sqlite 29.6M, httpfs 18.9M |1080| Image | about 143M more, all told |1081| Disk | the demo's lake: 225K. 20,000 events and 5,000 builds (3.3M of JSON lines): 406K, a 180K catalog and Parquet |1082| Time | 20,000 events and 5,000 builds in 0.18s; a run after a push 0.1 to 0.5s, plus about half a second to start Node and load DuckDB; the demo's first run 0.7s |1083| Exports | the demo's: insights.json 1.5K, each Parquet file 2 to 4K; with 300 builds, insights.json 8.5K and builds.parquet 9K |10841085## Hosting10861087The forge is one container (`Dockerfile`, on Node 24 LTS): sshd for git,1088Caddy for pages, sites and HTTPS clones (with fcgiwrap for `git1089http-backend`), the events, private and pages servers, litestream and1090tailcat. `server/container/start.sh` starts them all. All state is in1091`/srv/git`, a volume. Caddy, rclone, litestream and tailcat are pinned1092releases the Dockerfile builds from source; the rest comes from Debian.1093Caddy is on the edge's line, 2.11, where Debian has 2.6.10941095### Home machine behind a public edge (how src.rest runs)10961097```1098internet ── :22, :443 ──> Fly edge (deploy/edge) ── tailcat (WireGuard) ──> forge container on slate1099 TLS ends here accepts only the publishes no ports;1100 edge's client key the host stays closed1101```11021103[tailcat](https://github.com/tailscale/tailcat) gives the edge and the container a1104point-to-point WireGuard tunnel using Tailscale's data plane without a tailnet:1105no accounts, no control plane, nothing on your own tailnet. The container's1106tailcat server keeps its key on the volume (stable address) and only accepts1107the edge's client key (`FORGE_TAILCAT_ALLOW`). The edge is stateless; its key1108and the container's address are Fly secrets (`TAILCAT_CLIENT_KEY`,1109`FORGE_TAILCAT_ADDR`).11101111On the edge, Caddy takes the public HTTP port and passes everything through1112the tunnel. When slate doesn't answer (the tunnel or the box is down, a1113timeout, or a 502, 503 or 504), Caddy serves GET and HEAD requests for the1114forge's own pages from the [pages mirror](#the-pages-mirror) instead, with1115`X-Forge-Offline: mirror` and `Cache-Control: no-store`. A page the copy1116hasn't got, one it leaves out like a file's, or one that isn't there at all,1117is a 404: the forge's not-found page, saying the forge is offline and the1118page isn't in its offline copy. Sites, clones, live1119updates and pushes need slate; they get a 503, or for ssh no connection,1120until it's back. Caddy checks slate every 10 seconds, so only the first1121request after it goes away waits. `deploy/edge/fly.toml` sets `FORGE_DOMAIN`1122(only that host falls back) and `FORGE_MIRROR_URL`, the mirror's public URL;1123without it there is no fallback. With the Fly secret `FORGE_MIRROR_TOKEN`1124set, the edge sends it to the bucket as `X-Forge-Mirror-Token`, so a rule in1125front of the bucket can refuse requests without it.11261127On slate, everything lives in `/srv/forge` (`src/`, `data/`, `forge.env`):11281129```sh1130deploy/slate/deploy.sh # ship HEAD (git archive), docker compose up -d --build1131ssh slate docker exec forge forge-admin user add fox 'ssh-ed25519 AAAA...'1132ssh slate docker exec forge forge-admin user invite fox --repo cat/notes --as collaborator # a link instead1133ssh slate docker exec forge forge-admin users1134```11351136`deploy.sh [--force] [<commit>]` ships a commit, `HEAD` by default, exported1137with `git archive`: uncommitted edits in the checkout stay behind. It wants1138a verified passing CI attestation for it (`forge verify`), as `main` on the1139forge does with `forge.requireci`, and `--force` ships one without. The1140commit running on slate is in `/srv/forge/src/DEPLOYED`.11411142The edge (`deploy/edge`) needs a dedicated IPv4 on Fly, because raw TCP on1143port 22 doesn't work on shared addresses. DNS for `src.rest` and1144`pages.src.rest` points at the edge's addresses, DNS-only on Cloudflare1145(SSH can't go through Cloudflare's proxy). To rotate the edge key:1146`tailcat genkey --client --key=edge.private.json`, put its nodekey in1147`/srv/forge/forge.env`, its file in the `TAILCAT_CLIENT_KEY` secret and redeploy both.11481149### A plain VPS instead11501151`server/provision.sh` turns a fresh Debian 13 or Ubuntu 24.04 box into the1152forge directly (git, sshd, Node, Caddy with automatic TLS, and systemd units1153for the events, private and pages servers). It needs Node 22.18 or newer,1154which runs the TypeScript sources as they are. Node 24 LTS (24.7 or later) is1155best, as in the container, for post-quantum passkeys. Those distros ship1156older ones, so install a current Node first:11571158```sh1159rsync -a --exclude node_modules --exclude demo/root --exclude demo/work ./ root@HOST:/opt/forge/1160scp ~/.ssh/id_ed25519.pub root@HOST:/tmp/me.pub1161ssh root@HOST sh /opt/forge/server/provision.sh git.example.com cat /tmp/me.pub [pages.example.com]1162```11631164Either way: every key in `authorized_keys` is forced through `forge shell`, so1165administer as root (`forge-admin ...` in the container, or1166`sudo -u git /opt/forge/bin/forge ...` on a VPS), never by logging in as1167`git`. Back up with `forge backup` ([Backups and moving1168hosts](#backups-and-moving-hosts)). Builds run in the background after each1169push (log: `.forge/build.log`); `forge build` rebuilds everything.11701171### Cleaning up11721173Every distinct deploy is extracted once into `.forge/store/<tree>/` (a1174private repo's releases into `.forge/private/store/<tree>/`), and the old1175tree stays when a newer deploy takes its link. `forge gc` removes trees in1176`.forge/store` that no `www/sites` or `www/previews` link points at1177(hostnames point at those links) and no public repo's `refs/releases/*` ref1178names, and trees in `.forge/private/store` that no private repo's release1179ref names (release files are hard links, which name no tree), so a repo1180that flipped leaves nothing in the store it left. It also removes hostnames1181whose deploy is gone, previews of review branches that no longer exist1182and the downloads of releases whose ref was deleted without the hooks, in1183`www` or the private root. A deploy extracts its tree before it links it, and1184touches a tree it reuses, so store entries changed in the last 10 minutes1185are kept (`--age <minutes>`), which makes gc safe to run while people push. `--dry-run` lists1186what it would remove. A release published again leaves its old files in the1187repo's packs, unreachable: gc prunes nothing. Removed repos leave the trash1188after `FORGE_TRASH_DAYS` (14) days.11891190Then it maintains every repo:11911192- `git repack -d --geometric=2 --write-midx --write-bitmap-index`: packs1193 are merged only until each is at least twice the size of the next, so a1194 run rewrites what is new rather than the whole repo, and loose objects go1195 into the new pack. A multi-pack index with a reachability bitmap serves1196 clones and fetches from all the packs at once. It deletes only packs it1197 rewrote and prunes nothing, so pushes can land meanwhile.1198- the commit-graph, rewritten as one layer with changed-path Bloom filters1199 for every commit (filters it already has are reused);1200- `git pack-refs --all`, since every review revision pins a loose1201 `refs/forge/keep/<sha>`.12021203Between runs, each build after a push adds a small commit-graph layer for the1204new commits (about 20 ms), so `git log -- <path>` and blame skip commits1205that don't touch the path. That happens in the background build, not while1206the pusher waits.12071208Run it daily:12091210```sh1211# the host's crontab, for the container121217 4 * * * docker exec forge forge-admin gc1213# on a VPS: crontab -u git -e1214PATH=/usr/local/bin:/usr/bin:/bin121517 4 * * * /opt/forge/bin/forge gc1216```12171218Settings forge gives every repo, when it is created and again on1219`forge server init` (which the container runs on every boot): its hooks,1220`receive.advertisePushOptions true` for review metadata,1221`transfer.hideRefs refs/forge`, so clones, fetches and pushes don't see the1222pinned revisions (the build reads them directly, and pushes to them are1223refused), `uploadpack.hideRefs refs/releases`, so clones and fetches don't1224download release binaries but pushes still reach them, `core.bigFileThreshold122516m`, so big blobs are stored without deltas, and1226`commitGraph.changedPathsVersion 2`, Bloom filters that hash non-ASCII paths1227the same on every platform. The dumb protocol's `info/refs` still lists1228`refs/forge` and `refs/releases`; git clients use the smart protocol.12291230### Moving repos over12311232```sh1233git config --global forge.host git@git.example.com1234forge import https://github.com/you/thing.git # default branch + tags1235forge import --branches --desc "..." <any-git-url> # every branch1236forge import --private <url> # lands as a private repo1237```12381239Imports keep the source's default branch and don't turn existing branches into1240reviews. To move every public repo from a GitHub account:12411242```sh1243gh repo list you --limit 200 --json url,description,isPrivate \1244 --jq '.[] | select(.isPrivate | not) | [.url, (.description // "")] | @tsv' |1245while IFS="$(printf '\t')" read -r url desc; do forge import --desc "$desc" "$url"; done1246```12471248For the private ones, run the same loop with `select(.isPrivate)` and1249`forge import --private`. The clone runs on your machine, with your GitHub1250credentials.12511252Then point existing clones at the forge:1253`git remote set-url origin git@git.example.com:cat/<repo>.git`.12541255A public repo is public everywhere: pages, sites, releases, HTTPS clones and1256the pages mirror. A private one is read over ssh, on its pages once you sign1257in and over HTTPS with a token (see [Private repos](#private-repos)).12581259## Backups and moving hosts12601261`forge backup` snapshots the forge with [restic](https://restic.net) into an1262S3-compatible bucket, Cloudflare R2 for src.rest. Snapshots are encrypted with1263a password you keep, and deduplicated, so a day's snapshot of a forge that1264barely changed costs kilobytes. In the container one runs every day at1265`FORGE_BACKUP_AT` (UTC, default `03:30`), or soon after boot if that day's1266hasn't run, so a box that was off catches up; a failed one is retried hourly.12671268### What a backup holds12691270Everything in `/srv/git` that can't be rebuilt:12711272- every bare repo, `<owner>/<repo>.git`, with its refs (releases too, and so1273 their files, which restic stores once however many backups hold them),1274 its config, its `git-daemon-export-ok` if it has one (so a private repo1275 comes back private) and its whole `forge/` directory: review logs,1276 comments, collaborators, readers and anything else kept there;1277- `.forge/keys`, `.forge/config` and `.forge/www/hosts/`, which hostname each1278 site has;1279- `.forge/identity.db`, as SQLite copies it (`VACUUM INTO`): one consistent1280 file, never the live one with its WAL;1281- `.hostkeys/`, so clients see the same ssh host key, and `.tailcat/`, so a1282 restored box keeps its tunnel address and the edge finds it;1283- the lake (`.forge/lake`): the catalog, copied with SQLite's `VACUUM INTO`1284 while no `forge lake` runs, never the live file, and checked; and its data1285 files, as hard links. It can't be rebuilt, since the files it read are cut1286 or left out. Data on R2 stays in its bucket: the lake keeps snapshots for1287 40 days and deletes a file a week after the last one that used it, so a1288 catalog from any kept backup still finds its files;1289- `.ssh/`, and anything else under `/srv/git` or `.forge/` that isn't left out1290 below, so state added later is backed up by default.12911292Left out, because a restore rebuilds it or every boot rewrites it: the pages in1293`.forge/www` and `.forge/private`, the extracted sites and releases in1294`.forge/store` and private repos' releases in `.forge/private/store` (all1295are git refs),1296`.forge/cache`, `authorized_keys`, `allowed_signers`, `git-shell-commands/`,1297hooks, `.forge/secrets` and `.forge/backup.env` (both written from `forge.env`1298at boot), and logs, locks and scratch files.12991300### Consistent while people push13011302A backup doesn't read the live repos. It stages a copy first, per repo: the1303review logs, comments and config; then the refs, read once by git; then the1304objects, as hard links (free, since git never changes an object file once1305written), loose ones before packs. Git writes objects before the refs and logs1306that name them, so everything the copied refs and logs mention is in the copy,1307even mid-push or mid-repack. Each staged repo must pass `git fsck1308--connectivity-only`, or is staged again. Nothing waits on a lock: pushes,1309comments and `forge gc` carry on during a backup.13101311### Every run checks itself13121313After uploading, a backup131413151. forgets and prunes old snapshots: `--keep-daily 7 --keep-weekly 5` by1316 default (`FORGE_BACKUP_KEEP_DAILY`, `FORGE_BACKUP_KEEP_WEEKLY`; a daily of 01317 keeps only the newest snapshot of each week);13182. runs `restic check`, and about once a week also reads a fifth of the stored1319 data, a different fifth each week;13203. restores the new snapshot into a scratch directory and drills it: the same1321 files as staged, a clean full `git fsck` in every repo and the ref count1322 each live repo had.13231324A failure goes to `.forge/backup.log`, makes the command exit non-zero and is1325recorded in `.forge/backup.status`. A backup also refuses a forge with no1326repos, or with under half the repos of the last backup (`--force` overrides):1327retention keeps one snapshot a day, and a new box backing itself up before1328its restore must not push the good ones out.13291330```sh1331docker exec forge forge-admin backups # health, then the snapshots1332docker exec forge forge-admin backups --status # health only; exits 1 if failed, stale (36h) or off1333docker exec forge forge-admin backup # one now1334docker exec forge forge-admin backup --dry-run # stage it and say what would be added1335```13361337The health line is `last good backup: 5h ago ...` with the drill and check1338results, or, loudly, `LAST BACKUP FAILED`, `BACKUPS ARE STALE`, `NO GOOD1339BACKUP YET` or `BACKUPS ARE OFF`. The daily run prints it to the container log1340(`docker logs forge | grep backup:`). To hear about trouble, run `--status`1341from cron on another machine: `ssh slate docker exec forge forge-admin backups --status`.13421343### identity.db, backed up as it changes13441345Sign-ins and tokens change all day, so a daily snapshot isn't enough for1346`.forge/identity.db`. With backups set up, the container runs `forge1347identity replicate`: [litestream](https://litestream.io) v0.5 copies each1348change to the backup bucket within seconds, under `litestream/identity`1349(`FORGE_LITESTREAM_PATH`), with the same keys. Its copy holds no secrets1350(identity.db has only hashes and public keys), so it isn't encrypted the1351way restic's is.13521353```sh1354docker exec forge forge-admin identity restore # as it is now1355docker exec forge forge-admin identity restore --at 2026-09-01T10:00:00Z # as it was then (the last week)1356```13571358Stop the forge's servers before restoring by hand, since they hold the file1359open (`docker compose stop forge`, then `docker compose run --rm forge1360forge-admin identity restore ...`); the file it replaces is kept as1361`identity.db.before-restore`. A box that boots without identity.db (a new1362one) takes it from the replica before anything makes an empty one, whose1363replication would bury the replica's. If the replica can't be read then,1364nothing replicates until the next boot. `forge restore` takes whichever of1365the backup's copy and the replica changed last (every write, deletions1366too, stamps the file), which is litestream's unless replication had1367stopped. Without backups set up, identity.db stays local.13681369### The pages mirror13701371After a build, `forge mirror` syncs the forge's own pages to a second bucket1372with rclone, in the background, for the edge to serve while slate is down1373(see [above](#home-machine-behind-a-public-edge-how-srcrest-runs)). It holds1374public content only, the pages the forge serves anyway, and public repos'1375release downloads. It never replaces the copy with an empty forge.13761377The copy only has to keep the forge readable for a while, so it leaves out1378the pages there is one of per file. They were nine in ten of its files, and1379since every page links the stylesheet by its hash, each style change made1380the next sync upload all of them again. What `lib/mirror.ts` puts in:13811382| In the copy | Left out |1383|---|---|1384| `/`, owner pages, `/topics/`, `/insights/` with its exports, `/design/`, `/assets/` (scripts, styles, fonts), `repos.json` | `/sites/`, `/previews/`, `/hosts/`: sites need slate |1385| each repo's home (with its README), `log/`, `commit/`, `reviews/` and each review, `releases/` and their downloads, `refs/`, `sites/`, `wiki/`, `insights/`, `atom.xml`, `search.json`, `code.db`, `reviews.json`, branch menus, `files/<branch>.txt` | `/<owner>/<repo>/blob/**`: file pages |1386| `tree/<branch>/`, each branch's top-level file list | `tree/<branch>/<folder>/**`: folder pages |1387| the images a page in the copy shows, a README's or a wiki page's | the rest of `raw/**` |13881389A page left out gets the edge's offline 404. Whatever else is in the bucket1390goes at the next sync: pages from before they were left out (once), and1391those of a repo that went private or away.13921393**When it syncs.** Each build waits `FORGE_MIRROR_DELAY` seconds (600) and1394only the newest goes on, so a burst of pushes makes one sync. Then it waits1395until `FORGE_MIRROR_MIN_INTERVAL` seconds (3600) have passed since the last1396sync started; a build meanwhile takes its place, so there is one sync1397pending at most, and it runs when the interval ends. A sync that started1398after a build has its pages already, and that build's wait ends there. A1399repo that goes private, or is removed, makes a sync within 30 seconds1400whatever the interval, so its pages leave the copy then. `forge mirror` by1401hand syncs now. One sync at a time (`.forge/mirror.lock`); the background1402ones log to `.forge/mirror.log`.14031404A sync is three rclone steps. The pages on the list go by checksum, since1405a rebuilt page that came out the same has a new mtime and shouldn't be1406uploaded again; that reads every file. Then what else is in the bucket is1407deleted, except the downloads. The downloads, which can be big and never1408change in place, are synced on their own by size and mtime, rclone's1409default. Their mtime is their release commit's, so an unchanged binary isn't1410read. Uploads skip rclone's HEAD of each object afterwards (`--s3-no-head`):1411a page is one PUT, whose response has its MD5, and the next sync compares1412checksums with the ETags in the bucket's listing. The edge serves1413downloads from the copy like any file, as attachments.14141415### Setting it up14161417What you set up yourself, once:141814191. **Two R2 buckets**: `forge-backup`, private, and `forge-pages`, with public1420 access on, through a custom domain such as `mirror.src.rest` (best) or its1421 `r2.dev` URL (rate-limited, fine for an offline copy).14222. **An R2 API token** with Object Read & Write on both buckets (or a token1423 each). Note its access key ID and secret, and the account's S3 endpoint,1424 `https://<account-id>.r2.cloudflarestorage.com`.14253. **A restic password**, e.g. `openssl rand -hex 32`. **Put it in your1426 password manager, off slate, before the first backup.** Snapshots are1427 encrypted with it: without it nobody can read them, not you and not1428 Cloudflare, and a dead slate would take the only copy with it. Keep all of1429 `forge.env` there too; a new box needs it.14304. **`/srv/forge/forge.env` on slate**, then `deploy/slate/deploy.sh`:14311432 ```sh1433 FORGE_BACKUP_REPOSITORY=s3:https://<account-id>.r2.cloudflarestorage.com/forge-backup1434 FORGE_BACKUP_PASSWORD=<the restic password>1435 FORGE_BACKUP_KEY_ID=<access key id>1436 FORGE_BACKUP_SECRET=<secret access key>1437 FORGE_MIRROR_S3=https://<account-id>.r2.cloudflarestorage.com/forge-pages1438 FORGE_MIRROR_KEY_ID=<access key id>1439 FORGE_MIRROR_SECRET=<secret access key>1440 # optional: FORGE_BACKUP_AT=03:30 FORGE_BACKUP_KEEP_DAILY=7 FORGE_BACKUP_KEEP_WEEKLY=5 FORGE_MIRROR_DELAY=600 FORGE_MIRROR_MIN_INTERVAL=36001441 # FORGE_LITESTREAM_PATH=litestream/identity1442 ```14431444 At every boot the container writes these to `.forge/backup.env`, which1445 only git and root can read.14465. **The edge**: set `FORGE_MIRROR_URL` in `deploy/edge/fly.toml` to the1447 bucket's public URL (`FORGE_DOMAIN` is set already), then `fly deploy`. Its1448 secrets are unchanged: `TAILCAT_CLIENT_KEY` and `FORGE_TAILCAT_ADDR`.14496. **Check it**: `docker exec forge forge-admin backup`, then `backups` and1450 `forge-admin mirror` for the first sync instead of waiting for a push.14511452### Restoring, or moving to another host14531454`forge restore [<snapshot>|latest]` brings a forge back into an empty1455`/srv/git`. It restores the files and rebuilds what a backup leaves out:1456what `forge server init` and `forge keys sync` write, hooks in every repo,1457pinned review revisions, extracted sites and release downloads (a private1458repo's in the private root, extracted into `.forge/private/store` again),1459with the mtimes they had, so the mirror uploads none again. Then it runs1460`forge build`. identity.db is the backup's copy or litestream's replica, whichever1461changed last (see [above](#identitydb-backed-up-as-it-changes)). It refuses1462a forge that already has repos. The host's own `.forge/config` wins over the backup's,1463since it is written for the domain that host serves, and users from both are1464kept. `forge restore <snapshot> --to <dir>` only unpacks the files, to look at1465them or to copy one repo back by hand. Any machine with restic and the1466password can read the snapshots too: `restic -r <repository> restore latest1467--target <dir>`, then `forge restore --from <dir>` on the new forge.14681469**A new slate**, or any box with Docker. This is also how the forge moves:14701471```sh1472# Docker, this repo in /srv/forge/src, forge.env from your password manager1473# in /srv/forge/forge.env, an empty /srv/forge/data1474cd /srv/forge/src/deploy/slate1475docker compose build1476docker compose run --rm forge forge-admin restore latest1477docker compose up -d1478docker exec forge forge-admin backups1479```14801481Turn the old forge off first, since it has the same tailcat key. The restored1482key keeps the tunnel address, so the edge finds the new box by itself, with1483no change on Fly, and serves the offline copy until then. If the forge already1484booted on the new box, run `docker exec forge forge-admin restore latest`, then1485`docker compose restart forge`, so sshd and tailcat load the restored keys.14861487**A plain VPS**: provision it as above (that installs restic and rclone too),1488write the `FORGE_BACKUP_*` lines into `/srv/git/.forge/backup.env` (quote1489values with spaces or `$` in them), restore as git and point DNS at it:14901491```sh1492ssh root@HOST 'install -o git -g git -m 600 /dev/stdin /srv/git/.forge/backup.env' <backup.env1493ssh root@HOST sudo -u git /opt/forge/bin/forge restore latest1494# optional: keep the ssh host key clients know, or they see a changed key once1495ssh root@HOST 'cp /srv/git/.hostkeys/ssh_host_* /etc/ssh/ && systemctl restart ssh'1496# daily backups, from crontab -u git -e1497PATH=/usr/local/bin:/usr/bin:/bin149830 3 * * * /opt/forge/bin/forge backup1499```15001501## Developer setup15021503```sh1504ln -s /path/to/forge/bin/forge ~/bin/forge1505git config --global forge.signingkey ~/.ssh/id_ed25519.pub # .pub uses ssh-agent1506ssh git@git.example.com signers > ~/.config/forge/allowed_signers1507git config --global forge.allowedsigners ~/.config/forge/allowed_signers1508git config --global forge.host git@src.rest # for forge import1509```15101511Day to day:15121513```sh1514git switch -c cat/topic && git commit ...1515forge attest --stack # CI for every commit on top of main1516forge push -t "Title" # branch + notes; becomes a review1517forge land # CI and approval checks, then fast-forward main1518forge land --server # or let the forge land it, with its gates only1519git tag v1.0 && forge release publish v1.0 dist/* # files for a tag, signed1520```15211522For a stack, `git rebase -i --update-refs` keeps the layers in sync, and1523`forge push --stack` pushes every layer, lowest first.15241525Working on forge itself: after `sh demo/make-demo.sh`, `npm run dev` serves1526the demo forge at http://localhost:8080/ (`npm run dev -- <port>` for1527another) and rebuilds it into `_site/` whenever a template, asset, `lib/` or1528`eleventy.config.ts` changes, reloading open tabs. Each build is a fresh 11ty1529process, since a running one can't pick up changed TypeScript modules. It1530only reads `demo/root`: no incremental build, and caches go to `.cache/dev/`.1531Live updates have no event stream there, so the status line stays empty.15321533The scenarios (`test/scenarios/`, one behavior each, end to end in a1534throwaway forge) run with `sh test/run.sh [word...]`, as CI does. They need1535git, ssh-keygen and Node, and use Caddy and litestream where they are1536installed; HTTP goes through `test/http.ts`, not curl. To run them where the1537forge runs, in the container as root (Debian's sh, a git user that restores1538are handed to, Caddy and litestream, no curl):15391540```sh1541docker build -t forge:test . # once, and after Dockerfile or package changes1542sh test/container.sh [word...] # this checkout's code, the image's node_modules1543```15441545`FORGE_CI_CONTAINER=1 forge attest` has `.forge/ci` build that image and run1546them there too, after the host's run; it is off by default, since the1547first build takes minutes.15481549## Reading pages15501551The home page has the motd (above), then three panels. **profiles**: each1552owner with public repos, how many and when they were last active.1553**recent**: the last eight things that happened on public repos, newest1554first: landed reviews, pushes to a default branch, releases (a tag or1555release pushed) and site deploys, each linking to what it names1556(`lib/recent.ts`). Landed reviews come from the review logs; the rest1557from `.forge/events.jsonl`, whose push lines carry the default branch's new1558tip as `head`. A push whose head is a review's tip is that review landing,1559and shows once, as landed. Every build redoes the home page, and open1560copies of it swap in the new one when the build is done, so the list is1561as fresh as the last push. Its times stay relative in the browser. The1562events log is cut and left out of backups, so after a restore the list has1563only landed reviews until the next pushes. **topics**: the shelves (below)1564across every public repo; one owner's links to that shelf on their page1565(`/cat/#games`, which opens on its tab), and one that several owners1566share to `/topics/#games`, every public repo on its shelf, each naming its1567owner. Private repos, and owners with only private ones, are never on1568any of it.15691570Each owner's page, and `/topics/`, show repos as rows on shelves by category,1571newest first. A repo's category is one line in the bare repo's `category`1572file, next to `description` (`printf 'games\n' > /srv/git/cat/greet.git/category`).1573A repo without one goes on the shelf for what Jev says it is (a web app, a1574library), and failing that on "other". Links above the shelves show one shelf1575alone, only repos with a site or sort by name.15761577A review page reads top to bottom: the stack, the layer's commits, then each1578commit's diff. While you read the diffs, a bar sticks to the top with the1579layers below and above and the previous and next commit. Every file folds1580(click its header). Its Viewed box folds it and counts toward "3 of 7 files1581viewed". Viewed marks are kept in the browser's localStorage, keyed by repo,1582path and a hash of that file's diff, so they survive a force-push that leaves1583the file alone and clear when the file changes.15841585Above the diffs, the discussion: approvals, then comments oldest first. Signed1586in, it ends in a box to comment in, with Approve beside it (Ctrl or Cmd and1587Enter sends), and on the combined diff a + follows the pointer down the lines:1588click it for a box under that line (on a phone, tap the line, then the +). A1589comment you send waits on the page, as you typed it, until the rebuilt page1590swaps in a moment later with it rendered; what you are typing outlasts that1591swap, and a reload. Signed out, or without JavaScript, the page shows none of1592this.15931594"What changed since rN" compares a revision with the one before. If both were1595pushed on the same base, it is the diff between the two tips. If the branch1596was rebased, that diff would include everything that landed meanwhile, so1597each commit is paired with its earlier version instead and labeled1598unchanged, rewritten, added or dropped.15991600Review lists nest stacks, each layer under the one below it, and a revised1601review links its latest change (`r2`). Times are written in UTC and shown1602relative ("3h ago", then "Sep 1") by the browser, with the local time on1603hover, so static pages don't go stale. On phones, long code lines wrap unless1604you turn it off with the Wrap button, which is remembered.16051606A file on the default branch has **blame** and **history** buttons. Blame1607puts the commit that last changed each line, and when, in a column before1608the line numbers, one band per run of lines from a commit; clicking it1609opens that commit. History lists the commits that changed the file, newest1610first (the latest 200), each with the review it landed in. Both come from1611the file's `blame.json` beside its page (`lib/blame.ts`), fetched the1612first time either is asked for, so a file adds no pages. A build computes1613it with `git blame` and `git log` for the files commits touched since the1614last one, and keeps the rest in `$FORGE_CACHE/blame/`. Blame stays on from1615file to file once turned on.16161617Keyboard shortcuts (`?` lists them):16181619| Key | Does |1620| --- | --- |1621| `/` | search |1622| `t` or `p` | find a file on this branch |1623| `j` `k`, `Enter` or `o` | move through a list, open the row |1624| `n` `N` | next or previous commit on a review, file elsewhere |1625| `[` `]` | layer below or above in a stack |1626| `x` | fold or unfold the file in view |1627| `v` | mark the file in view as viewed |1628| `h` `g` | repo home, all repositories |1629| `f` `l` `b` `r` `w` `s` `e` `i` | code, commits, branches, reviews, wiki, sites, releases, insights |1630| `u` | up a level |16311632## Templates16331634Pages are Nunjucks templates in `site/`. The data layer in `lib/` reads git1635with plain `git` commands and hands 11ty flat page lists (`forge.pages.logs`,1636`.reviews`, `.blobs`, ...), one item per page. Large diffs and files are1637rendered by filters when a page is written, so the build never holds all the1638markup at once.16391640Every page extends `site/_includes/base.njk` and fills its blocks: `title`, `content`,1641`head` (the page's description and its OpenGraph tags, so links pasted into1642chat apps preview well; `og:url` needs `FORGE_BASE_URL`), `scripts` (the diff1643pages add `diffs.js`) and `heading`. Front matter shared by many templates,1644like the pagination alias and permalink, is in directory data files1645(`site/site.11tydata.json`, `site/repo/repo.11tydata.json`), so a repo page's1646front matter is just the list it paginates:16471648```njk1649---1650pagination:1651 data: forge.pages.commits1652---1653{% extends "base.njk" %}1654{% import "macros.njk" as m with context %}1655{% block content %}1656...1657{% endblock %}1658```16591660`site/assets/style.css` holds what every page needs: the design tokens1661(every color, size, weight and font), layout, lists, code views. The rules1662for the look, and for changing it, are in [DESIGN.md](DESIGN.md); the1663controls are macros in `site/_includes/ui.njk`, and `/design/` shows both.1664CSS for one feature sits next to its markup in a `{% css %}` block, in the1665template, include or macro that renders it (the discussion, the search1666overlay, the shortcuts dialog, the review bar, review lists, the wiki).1667From those blocks, 11ty's bundle plugin gives each page one file of the1668blocks it rendered, `/assets/<hash>.css`, shared by pages of a kind. Import1669`macros.njk` with context, or blocks in macros don't reach the bundle.16701671Assets ship under content-hashed names (`/assets/style.<hash>.css`,1672`ui.<hash>.js`, sqlite-wasm), so browsers can cache them for good;1673the stylesheet's `url(./fonts/open-sans.woff2)` becomes the font's hashed name. The same1674sources give the same names, and pages' fingerprints include them, so1675unchanged pages are still hard-linked. A build keeps the files of the last1676five code versions (listed in `assets/versions.json`), because pages built by1677older code stay live for a while: a full `forge build` swaps one repo at a1678time, and a repo whose build fails keeps its old pages.16791680The templates render without 11ty too. `lib/render-page.ts` sets up plain1681Nunjucks with the same options, filters and bundle tags, so a server can1682render a page on demand:16831684```ts1685const render = pageRenderer();1686const html = render('repo/commit.njk', { forge, p });1687```16881689`node lib/render-page.ts check <www>` renders every page that way and1690compares it with 11ty's; CI runs it on the demo. With `FORGE_STRICT=1` (CI1691sets it), printing an undefined or null value is an error, so a value that1692may be missing needs an explicit check or `or ''`.16931694| URL | Page |1695| --- | --- |1696| `/<owner>/<repo>/` | open reviews, recent commits, README |1697| `/<owner>/<repo>/log/<branch>/` | log for a branch (`/` in branch names becomes `~`) |1698| `/<owner>/<repo>/tree/<branch>/<path>/` | directory |1699| `/<owner>/<repo>/blob/<branch>/<path>/` | file |1700| `/<owner>/<repo>/commit/<sha>/` | commit |1701| `/<owner>/<repo>/reviews/<id>/` | stacked diff view; `diff/` combined, `r<n>/` what changed since the revision before |1702| `/<owner>/<repo>/wiki/<page>/` | wiki |1703| `/<owner>/<repo>/reviews.json` | the review list as data, for `forge reviews` |1704| `/<owner>/<repo>/code.db` | the default branch's code index, read by code search in byte ranges |1705| `/<owner>/<repo>/sites/` | deploys and previews |1706| `/<owner>/<repo>/releases/` | release notes per tag, and each release's files |1707| `/<owner>/<repo>/releases/download/<tag>/<file>` | a release file; `latest/` for the newest release's |1708| `/<owner>/<repo>/insights/` | charts from the lake; `insights.json` and Parquet files next to it |1709| `/insights/` | the same for the whole forge, and a table by repo |1710| `/login/` | sign in with a passkey, add one (`#enroll=<code>`), join from an invite (`#invite=<code>`), sign out; `whoami` says who is signed in, for review pages |1711| `/private/` | the private repos you can read |1712| `/sites/<owner>/<repo>/<name>/` | a deployed site |17131714## Not yet17151716- Bugs or issues in the args.io style.