The events server took any number of open streams. Now at most FORGE_EVENTS_MAX (2,000) are open for people not signed in to a private repo's stream, and FORGE_EVENTS_PER_IP (16) from one address; past either it answers 503 with Retry-After. The private, events and pages servers give a request's headers 10 seconds and the whole request 30 to arrive (a response, such as the event stream, may run on). Caddy has no rate-limit module in the container's build, so per-address limits in Caddy are left for later. Scenario: the per-address and overall caps, and a closed stream freeing its place.
- commit
- 56690dd0e278f446809ace61f4728dce4b13c321
- parent
- 473a10f84180dbd2c0dc654aa8e72a2d525c9a7f
- author
- cat <admin@dev.site>
- ci
- ✓ passed in 117s, signed by cat
- kind
- feat risk score 2.0 of 3, Jev's estimate
- review
- #30 Security: sign-in, invites, owner pages, lake, caps and the edge
ci attestation
forge-attestation 1 type ci commit 56690dd0e278f446809ace61f4728dce4b13c321 tree 7357db113f007572ec399d1f61d41cc42c740a28 script .forge/ci 8d2000a416f8160bcf710d98b72af9eb76287f58 result pass exit 0 started 2026-10-19T07:48:27Z duration 117s runner (not shown here; refs/notes/ci has it) system Darwin 25.5.0 arm64 env host log sha256:377eb4afedd405a1454a32919c59106cfa51228feb53c075fc59b1415f60fe62 319 lines |pushed wip/i18n |d7a9fdf running .forge/ci: greet.sh: say what it is | | sh: /bin/sh | | ok greet |d7a9fdf pass in 0s, signed |pushed main |remote: review #4: /dami/greet/reviews/4/ |pushed wip/i18n |ba42465 running .forge/ci: wave at the world | | sh: /bin/sh | | FAIL greet : got 'hello, world o/', want 'hello, world' |ba42465 fail in 0s, signed |forge: 1 commit(s) did not pass |remote: review #6: /dami/greet/reviews/6/ |pushed ana/wave |1741831 running .forge/ci: update the test for the wave | | sh: /bin/sh | | ok greet |1741831 pass in 0s, signed |remote: review #6: /dami/greet/reviews/6/ |pushed ana/wave |warning: You appear to have cloned an empty repository. |site home: deployed 75b4460 -> /sites/ana/dotfiles/home/ |created ana/notes |remote: review #1: /ana/notes/reviews/1/ |pushed ana/todo |landing #1 A todo list: ana/todo -> main |8bb308e running .forge/ci: web, server and Python versions of greet | | sh: /bin/sh | | ok greet |8bb308e pass in 0s, signed |pushed main |152956c running .forge/ci: messages: notes for the later regions | | sh: /bin/sh | | ok greet |152956c pass in 0s, signed |remote: review #7: /dami/greet/reviews/7/ |pushed dami/regions |created dami/forge |warning: You appear to have cloned an empty repository. |site guide: mkdir -p guide/fonts && cp site/assets/fonts/*.woff2 guide/fonts/ |site guide: deployed c6c1c52 -> /sites/dami/forge/guide/ |(node:21796) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built ana/dotfiles in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.aPbDrS/wt/demo/root/.forge/www |(node:21957) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built ana/notes in 0s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.aPbDrS/wt/demo/root/.forge/www |(node:22128) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built dami/forge in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.aPbDrS/wt/demo/root/.forge/www |(node:22292) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built dami/greet in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.aPbDrS/wt/demo/root/.forge/www |(node:22993) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built - in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.aPbDrS/wt/demo/root/.forge/www |235 of 235 pages render the same without 11ty |guide: 8 files, links ok |ok
Signature: good, cat
| M | README.md | 2 | +- |
| M | lib/ | 31 | +++++++++++++++++++++--- |
| M | lib/ | 5 | ++-- |
| M | lib/ | 4 | ++-- |
| A | test/ | 39 | ++++++++++++++++++++++++++++++ |
@@ -99,7 +99,7 @@ review page, signed in ─── HTTPS ────────────> the
9999 | sshd | git over ssh. Every key is forced through `forge shell <name>`. |100100 | Caddy | Serves `.forge/www` as plain files, read-only HTTPS clones (`git http-backend` through fcgiwrap; nothing else inside a bare repo is served; a private repo takes a token) and `go get` meta tags, with security headers (HSTS, nosniff, a referrer policy and a CSP against framing) on the forge's own pages; proxies `/events/`, every path `www` has no file for (to the private server) and, with a pages domain, sites. |101101 | `forge build` | Runs 11ty after each push, in the background and one repo at a time, then swaps the new pages into `www/`. Unchanged pages are hard links to the last build's. First it adds the push's commits to the repo's commit-graph (see [Cleaning up](#cleaning-up)). |102-| `lib/events-server.ts` | Tails `.forge/events.jsonl`, which hooks and builds append to, and streams it as server-sent events at `/events/<owner>/<repo>/`. Open pages say who pushed and swap in fresh content when the build finishes. Without it pages still work; their status line says offline. A private repo streams only to its readers, signed in. |102+| `lib/events-server.ts` | Tails `.forge/events.jsonl`, which hooks and builds append to, and streams it as server-sent events at `/events/<owner>/<repo>/`. Open pages say who pushed and swap in fresh content when the build finishes. Without it pages still work; their status line says offline. A private repo streams only to its readers, signed in. At most 2,000 streams are open for people not signed in, and 16 from one address. |103103 | `lib/private-server.ts` | Serves private repos' pages to their readers, signed in with a passkey or holding a token, the sign-in pages (`/login/`, `/private/`), and the 404 of every page that isn't there; takes comments and approvals posted from review pages (see [Comments and approvals](#comments-and-approvals)); answers Caddy whether a token may clone a private repo over HTTPS (see [Private pages](#private-pages)). |104104 | `lib/pages-server.ts` | Only with a pages domain. Serves each site on its own hostname the way Cloudflare Pages does (`_headers`, `_redirects`, pretty URLs, the nearest `404.html` or a single-page-app fallback) and redirects old `/sites/...` paths there. Runs as the caddy user, since deploys are untrusted content. |105105 | `forge backup` | Daily, from a loop in `start.sh`. Snapshots the forge with restic into a bucket, then checks the snapshots and test-restores the new one (see [Backups and moving hosts](#backups-and-moving-hosts)). |@@ -14,12 +14,20 @@
1414 // git-daemon-export-ok at each event, not only from the event's flag, and1515 // the session and the right to read are checked again too, so making a1616 // repo private, removing a reader or signing out takes effect at once.17+//18+// Streams stay open, so there are only so many: at most ANON_MAX for19+// people not signed in to a private repo's stream, and PER_IP_MAX from any20+// one address (lib/limits.ts clientIp). Past either, a 503 with Retry-After.1721 import http from 'node:http';1822 import fs from 'node:fs';1923 import path from 'node:path';2024 import { fileURLToPath } from 'node:url';2125 import { canRead, isPublic as flagged, repoDir, users } from './access.ts';2226 import { Identity, cookieValue } from './identity.ts';27+import { clientIp, timeouts } from './limits.ts';28+29+export const ANON_MAX = Number(process.env.FORGE_EVENTS_MAX) || 2000;30+export const PER_IP_MAX = Number(process.env.FORGE_EVENTS_PER_IP) || 16;2331 2432 export interface ForgeEvent {2533 t: number;@@ -38,6 +46,7 @@ interface Client {
3846 repo: string | null;3947 // Watching a private repo: the session that may.4048 session: string | null;49+ ip: string;4150 res: http.ServerResponse;4251 ping: NodeJS.Timeout;4352 }@@ -83,9 +92,16 @@ export function serveEvents(state: string, port: number, root = path.dirname(sta
8392 const dir = repoDir(root, repo);8493 return !!s && !!dir && !flagged(dir) && users(state).has(s.user) && canRead(dir, s.user);8594 };95+ // Open streams by address, and those without a session.96+ const perIp = new Map<string, number>();97+ let anon = 0;8698 const drop = (c: Client): void => {99+ if (!clients.delete(c)) return;87100 clearInterval(c.ping);88- clients.delete(c);101+ const n = (perIp.get(c.ip) ?? 1) - 1;102+ if (n > 0) perIp.set(c.ip, n);103+ else perIp.delete(c.ip);104+ if (!c.session) anon--;89105 };90106 const broadcast = (ev: ForgeEvent): void => {91107 if (ev.private || !isPublic(root, ev.repo)) {@@ -132,7 +148,7 @@ export function serveEvents(state: string, port: number, root = path.dirname(sta
132148 };133149 fs.watchFile(file, { interval: 300 }, poll);134150 135- return http.createServer((req, res) => {151+ return timeouts(http.createServer((req, res) => {136152 const m = /^\/events\/(?:([A-Za-z0-9_.-]+)\/([A-Za-z0-9_.-]+)\/?)?$/.exec(new URL(req.url ?? '/', 'http://x').pathname);137153 const repo = m?.[1] ? `${m[1]}/${m[2]}` : null;138154 const secret = cookieValue(req.headers.cookie);@@ -142,14 +158,21 @@ export function serveEvents(state: string, port: number, root = path.dirname(sta
142158 res.writeHead(404).end();143159 return;144160 }161+ const ip = clientIp(req);162+ if ((perIp.get(ip) ?? 0) >= PER_IP_MAX || (!session && anon >= ANON_MAX)) {163+ res.writeHead(503, { 'Retry-After': '60', 'Cache-Control': 'no-store' }).end();164+ return;165+ }145166 res.writeHead(200, { 'Content-Type': 'text/event-stream; charset=utf-8', 'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no' });146167 res.write('retry: 5000\n');147168 // Connected and idle: nothing to show until someone pushes.148169 res.write('data: \n\n');149- const client: Client = { repo, session, res, ping: setInterval(() => res.write(': ping\n\n'), 20000) };170+ const client: Client = { repo, session, ip, res, ping: setInterval(() => res.write(': ping\n\n'), 20000) };150171 clients.add(client);172+ perIp.set(ip, (perIp.get(ip) ?? 0) + 1);173+ if (!session) anon++;151174 req.on('close', () => drop(client));152- }).listen(port, '127.0.0.1', () => console.log(`events on 127.0.0.1:${port}, following ${file}`));175+ })).listen(port, '127.0.0.1', () => console.log(`events on 127.0.0.1:${port}, following ${file}`));153176 }154177 155178 // Run directly: node lib/events-server.ts@@ -15,6 +15,7 @@ import path from 'node:path';
1515 import { fileURLToPath } from 'node:url';1616 import { Hosts, hostsConfig, parseTarget } from './hosts.ts';1717 import type { HostsConfig } from './hosts.ts';18+import { timeouts } from './limits.ts';1819 1920 const TYPES: Record<string, string> = {2021 html: 'text/html; charset=utf-8', htm: 'text/html; charset=utf-8', css: 'text/css; charset=utf-8',@@ -290,7 +291,7 @@ export function servePages(cfg: HostsConfig, port: number): http.Server {
290291 }291292 292293 const suffix = cfg.pagesDomain ? `.${cfg.pagesDomain}` : null;293- return http.createServer((req, res) => {294+ return timeouts(http.createServer((req, res) => {294295 try {295296 if (req.method !== 'GET' && req.method !== 'HEAD') {296297 res.writeHead(405, { Allow: 'GET, HEAD' }).end();@@ -349,7 +350,7 @@ export function servePages(cfg: HostsConfig, port: number): http.Server {
349350 if (!res.headersSent) res.writeHead(500).end();350351 else res.destroy();351352 }352- }).listen(port, '127.0.0.1', () => console.log(`pages on 127.0.0.1:${port} for *${suffix ?? ' (paths only)'}, from ${cfg.www}`));353+ })).listen(port, '127.0.0.1', () => console.log(`pages on 127.0.0.1:${port} for *${suffix ?? ' (paths only)'}, from ${cfg.www}`));353354 }354355 355356 // Run directly: node lib/pages-server.ts@@ -38,7 +38,7 @@ import { INVITE_EXPIRED, Identity, SESSION_TTL, TOKEN_PREFIX, cookieValue, csrfT
3838 import { canRead, isPublic, readable, repoDir, users } from './access.ts';3939 import { DOWNLOAD, DOWNLOAD_HEADERS, resolve, sendFile } from './static.ts';4040 import { Login, LoginError, agentName, browserId, challengeCookie, newBrowserId, readBody, relyingParty } from './login.ts';41-import { Window, clientIp } from './limits.ts';41+import { Window, clientIp, timeouts } from './limits.ts';4242 import { pageRenderer, type Render } from './render-page.ts';4343 import { assets } from './assets.ts';4444 import { summarize } from './repo.ts';@@ -509,7 +509,7 @@ export function privateHandler(cfg: PrivateConfig): http.RequestListener {
509509 }510510 511511 export function servePrivate(cfg: PrivateConfig, port: number): http.Server {512- return http.createServer(privateHandler(cfg)).listen(port, '127.0.0.1', () => console.log(`private pages on 127.0.0.1:${port}, from ${cfg.priv}`));512+ return timeouts(http.createServer(privateHandler(cfg))).listen(port, '127.0.0.1', () => console.log(`private pages on 127.0.0.1:${port}, from ${cfg.priv}`));513513 }514514 515515 // Run directly: node lib/private-server.ts@@ -0,0 +1,39 @@
1+# The events server holds only so many streams open: at most2+# FORGE_EVENTS_PER_IP from one address and FORGE_EVENTS_MAX in all for3+# anyone not signed in (here 2 and 3); past either, a 503 with4+# Retry-After. A stream that closes frees its place.5+. "$(dirname "$0")/../lib.sh"6+user dami7+repo dami/open8+: >"$FORGE_ROOT/.forge/events.jsonl"9+# Another scenario may take the free port first: then try another.10+tries=011+while :; do12+ port=$(free_port)13+ FORGE_STATE=$FORGE_ROOT/.forge FORGE_EVENTS_PORT=$port FORGE_EVENTS_PER_IP=2 FORGE_EVENTS_MAX=3 node "$T_HOME/lib/events-server.ts" >"$T/events.log" 2>&1 &14+ pid=$!15+ n=016+ until grep -q "^events on 127.0.0.1:$port" "$T/events.log" || ! kill -0 "$pid" 2>/dev/null || [ "$n" -ge 100 ]; do sleep 0.1; n=$((n + 1)); done17+ if kill -0 "$pid" 2>/dev/null && grep -q '^events on' "$T/events.log"; then break; fi18+ kill "$pid" 2>/dev/null || true19+ tries=$((tries + 1))20+ [ "$tries" -lt 5 ] || fail "the events server did not start"21+done22+T_PIDS="$T_PIDS $pid"23+es=http://127.0.0.1:$port/events24+# Streams from an address, held open for a few seconds.25+hold() { http -sN --max-time 4 -H "X-Forge-Client-IP: $1" "$es/" >/dev/null & T_PIDS="$T_PIDS $!"; }26+code() { http -s -o /dev/null -w '%{http_code}' --max-time 1 -H "X-Forge-Client-IP: $1" "$es/" || true; }27+hold 192.0.2.128+hold 192.0.2.129+sleep 130+eq "$(code 192.0.2.1)" 50331+hold 192.0.2.232+sleep 133+# Three open: no more for anyone not signed in.34+eq "$(code 192.0.2.3)" 50335+http -s -i --max-time 1 -H 'X-Forge-Client-IP: 192.0.2.3' "$es/" | tr -d '\r' >"$T/res" || true36+has "$T/res" '^Retry-After: 60$'37+# Once they close, there is room again.38+sleep 439+eq "$(code 192.0.2.1)" 200