ssh private and repo rm started the index and owner build in the background and then the mirror's sync within 30 seconds, while that build could still be waiting on build.lock, so the listing could linger in the offline copy. They build the index before returning now (rebuild_index), unless builds are off. repo rm deleted only the repo's exports, so its rows stayed in the lake and came back in the exports of a new repo under the same name: lib/lake.ts forget takes them out of every table, with the lake's note of where it read the repo's sources, under the lake's lock. Scenarios: the listing is gone when the command returns, and a reused name starts with no history.
- commit
- 0573918c003ef590d27ecf8b263d254e423aa824
- parent
- 51330a11d4f4f0236a1ee7b446832936c7f0beaa
- author
- cat <admin@dev.site>
- ci
- ✓ passed in 140s, signed by cat
- kind
- fix risk score 1.9 of 3, Jev's estimate
- review
- #30 Security: sign-in, invites, owner pages, lake, caps and the edge
ci attestation
forge-attestation 1 type ci commit 0573918c003ef590d27ecf8b263d254e423aa824 tree 8a9151a43acf42df7bf29e26ae2c6903ab767327 script .forge/ci 8d2000a416f8160bcf710d98b72af9eb76287f58 result pass exit 0 started 2026-10-19T07:53:10Z duration 140s runner (not shown here; refs/notes/ci has it) system Darwin 25.5.0 arm64 env host log sha256:6e35b6ca4dd8b63a340b2894153a6d83d4e678e51476e73b3ceac9bbcb83a127 322 lines |pushed wip/i18n |8588d76 running .forge/ci: greet.sh: say what it is | | sh: /bin/sh | | ok greet |8588d76 pass in 0s, signed |pushed main |remote: review #4: /dami/greet/reviews/4/ |pushed wip/i18n |b063e4e running .forge/ci: wave at the world | | sh: /bin/sh | | FAIL greet : got 'hello, world o/', want 'hello, world' |b063e4e fail in 0s, signed |forge: 1 commit(s) did not pass |remote: review #6: /dami/greet/reviews/6/ |pushed ana/wave |dc1d6d6 running .forge/ci: update the test for the wave | | sh: /bin/sh | | ok greet |dc1d6d6 pass in 0s, signed |remote: review #6: /dami/greet/reviews/6/ |pushed ana/wave |warning: You appear to have cloned an empty repository. |site home: deployed 010e7b1 -> /sites/ana/dotfiles/home/ |created ana/notes |remote: review #1: /ana/notes/reviews/1/ |pushed ana/todo |landing #1 A todo list: ana/todo -> main |48d3b91 running .forge/ci: web, server and Python versions of greet | | sh: /bin/sh | | ok greet |48d3b91 pass in 0s, signed |pushed main |a361e9c running .forge/ci: messages: notes for the later regions | | sh: /bin/sh | | ok greet |a361e9c pass in 0s, signed |remote: review #7: /dami/greet/reviews/7/ |pushed dami/regions |created dami/forge |warning: You appear to have cloned an empty repository. |site guide: mkdir -p guide/fonts && cp site/assets/fonts/*.woff2 guide/fonts/ |site guide: deployed d2699c6 -> /sites/dami/forge/guide/ |(node:18266) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built ana/dotfiles in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.e0bXzn/wt/demo/root/.forge/www |(node:18443) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built ana/notes in 0s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.e0bXzn/wt/demo/root/.forge/www |(node:18634) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built dami/forge in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.e0bXzn/wt/demo/root/.forge/www |(node:18816) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built dami/greet in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.e0bXzn/wt/demo/root/.forge/www |(node:19555) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built - in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.e0bXzn/wt/demo/root/.forge/www |235 of 235 pages render the same without 11ty |guide: 8 files, links ok |ok
Signature: good, cat
| M | README.md | 4 | ++- |
| M | bin/ | 35 | +++++++++++++++++++++++----- |
| M | lib/ | 37 | +++++++++++++++++++++++++++++ |
| A | test/ | 38 | ++++++++++++++++++++++++++++++ |
| A | test/ | 25 | ++++++++++++++++++++ |
@@ -239,7 +239,9 @@ as before.
239239 forge (without `--yes` it says what would go); collaborators can't, and240240 `forge repo rm <owner>/<repo>` does the same as root. Its clones, pages in241241 either root, sites, previews, hostnames, release downloads and insights go242-at once, and a mirror sync within 30 seconds deletes its pages. The bare repo waits in242+at once, as do its rows in the lake, so a new repo of the same name starts243+with no history. The index and owner pages are rebuilt before the command244+returns, and a mirror sync within 30 seconds deletes its pages. The bare repo waits in243245 `.forge/trash/<owner>/<repo>.git.<time>` for `FORGE_TRASH_DAYS` (14) days,244246 backed up like the rest of the forge. `forge repo restore <owner>/<repo>`245247 puts back the newest removal if the name is free, and `forge gc` deletes it@@ -1473,9 +1473,9 @@ cmd_repo() {
14731473 # trash like any other state. Everything that served it goes now, the way14741474 # going private takes a repo's public side away: its pages in either root14751475 # (with its release downloads), its sites, previews and their hostnames,1476-# its insights exports and its caches. The index, owner pages and1477-# repos.json are rebuilt without it, and the mirror syncs within 30 seconds,1478-# which deletes its pages from the bucket.1476+# its insights exports, its rows in the lake and its caches. The index,1477+# owner pages and repos.json are rebuilt without it before this returns, and1478+# the mirror syncs within 30 seconds, which deletes its pages from the bucket.14791479 repo_rm() {14801480 owner=${1%%/*} name=${1#*/}14811481 valid_name "$owner" && valid_name "$name" && [ "$owner/$name" = "$1" ] && [ -f "$FORGE_ROOT/$1.git/HEAD" ] || die "no repo $1"@@ -1495,7 +1495,8 @@ repo_rm() {
14951495 rm -rf "${FORGE_WWW:?}/$1" "${FORGE_PRIVATE_WWW:?}/$1" "${FORGE_WWW:?}/sites/$1" "${FORGE_WWW:?}/previews/$1" \14961496 "${FORGE_STATE:?}/lake/export/$1" "${FORGE_STATE:?}/cache/repos/$1.json" "${FORGE_STATE:?}/cache/code/$1"14971497 rm -f "$FORGE_WWW"/insights/*.json "$FORGE_WWW"/insights/*.parquet1498- rebuild - removed1498+ lake_forget "$1" || say "forge: the lake still has $1's rows ($FORGE_STATE/lake.log)"1499+ rebuild_index removed14991500 mirror_kick soon15001501 echo "removed $1; forge repo restore $1 brings it back within ${FORGE_TRASH_DAYS:-14} days (the trash: ${trash#"$FORGE_ROOT"/}/$name.git.$t)"15011502 }@@ -1807,6 +1808,16 @@ rebuild() {
18071808 fi18081809 }18091810 1811+# rebuild_index <reason>: the index and owner pages, built before this1812+# returns (unless builds are off). For a repo gone private or removed, so1813+# its listing is gone from www before mirror_kick soon syncs the mirror: a1814+# build in the background could still be waiting on build.lock by then.1815+rebuild_index() {1816+ [ "${FORGE_NO_BUILD:-0}" != 1 ] || return 01817+ FORGE_BUILD_REASON=$1 "$FORGE_BIN" build - >>"$FORGE_STATE/build.log" 2>&1 ||1818+ say "forge: the index build failed ($FORGE_STATE/build.log); forge build - again"1819+}1820+18101821 # forge jev [owner/repo]: ask Jev about what the repo's pages show and store18111822 # the answers (lib/jev-fill.ts). Without TYPESAFE_API_KEY it does nothing,18121823 # and a failure never stops the build that follows. Private repos have no@@ -1999,7 +2010,7 @@ redeploy_sites() {
19992010 20002011 # private <owner>/<repo>: only its owner, collaborators and readers can see it20012012 # from now on. Its pages go, the index, owner pages and repos.json are2002-# rebuilt without it, its sites, previews and their hostnames go unless2013+# rebuilt without it before this returns, its sites, previews and their hostnames go unless20032014 # forge.publicsites, and the mirror syncs within 30 seconds, which deletes20042015 # them from the bucket. Its pages are built again into the private root,20052016 # for its readers, and its release downloads move there. The events server@@ -2027,7 +2038,7 @@ sc_private() {
20272038 # The forge's insights count public repos only; they are written again20282039 # without this one after the build.20292040 rm -f "$FORGE_WWW"/insights/*.json "$FORGE_WWW"/insights/*.parquet2030- rebuild - private2041+ rebuild_index private20312042 rebuild "$1" private20322043 mirror_kick soon20332044 echo "$1 is private: only ${1%%/*}, its collaborators and its readers (readers $1 add <user>) can clone it, and see its pages once they sign in."@@ -2985,6 +2996,18 @@ lake_env() {
29852996 esac29862997 }29872998 2999+# lake_forget <owner>/<repo>: a removed repo's rows out of the lake (lib/lake.ts3000+# forget), under the lake's lock, if there is a lake.3001+lake_forget() {3002+ [ -f "$FORGE_STATE/lake/catalog.sqlite" ] || return 03003+ (3004+ lake_env3005+ take_lock "$FORGE_STATE/lake.lock" "${FORGE_LAKE_WAIT:-300}" || exit 13006+ trap 'rm -rf "$FORGE_STATE/lake.lock"' EXIT3007+ node "$FORGE_HOME/lib/lake.ts" forget "$1" >>"$FORGE_STATE/lake.log" 2>&13008+ )3009+}3010+29883011 cmd_lake() {29893012 load_config29903013 lake_env@@ -17,6 +17,7 @@
1717 // node lib/lake.ts query <sql> [--json|--csv] read-only, no file access1818 // node lib/lake.ts install install the DuckDB extensions (the Dockerfile runs it)1919 // node lib/lake.ts stage <dir> a consistent copy of the lake for a backup20+// node lib/lake.ts forget <owner>/<repo> take a removed repo's rows out (forge repo rm)2021 import fs from 'node:fs';2122 import os from 'node:os';2223 import path from 'node:path';@@ -228,6 +229,33 @@ interface SourceState extends Offset {
228229 229230 const offsetOf = (s: SourceState | undefined): Offset | null => (s ? { ino: s.ino, pos: s.pos, head: s.head, tail: s.tail } : null);230231 232+// A removed repo (forge repo rm): its rows in every table, and where the233+// lake read its sources up to, so a new repo under the same name starts234+// with none of them, and its exports with none of the removed one's235+// history. events.jsonl and metrics.jsonl were read past its lines236+// already, so they don't come back. Returns how many rows went.237+export async function forget(lake: Lake, slug: string): Promise<number> {238+ const s = lit(slug);239+ const where: Record<string, string> = {240+ events: `repo = ${s}`, builds: `scope = ${s}`, build_repos: `repo = ${s}`, reviews: `repo = ${s}`,241+ review_ends: `repo = ${s}`, commits: `repo = ${s}`, ci: `repo = ${s}`,242+ sources: `source IN (${['main', 'commits', 'notes'].map((k) => lit(`${k} ${slug}`)).join(', ')}) OR starts_with(source, ${lit(`review ${slug} `)})`,243+ };244+ let n = 0;245+ await lake.run('BEGIN TRANSACTION');246+ try {247+ for (const [t, w] of Object.entries(where)) {248+ if (t !== 'sources') n += Number((await lake.rows(`SELECT count(*) FROM lake.${t} WHERE ${w}`))[0]?.[0] ?? 0);249+ await lake.run(`DELETE FROM lake.${t} WHERE ${w}`);250+ }251+ await lake.run('COMMIT');252+ } catch (e) {253+ await lake.run('ROLLBACK').catch(() => {});254+ throw e;255+ }256+ return n;257+}258+231259 // Add what is new in every source. With dry, the same reads and inserts run232260 // and are rolled back, so it says what a run would add.233261 export async function ingest(lake: Lake, { dry = false, now = Date.now() }: { dry?: boolean; now?: number } = {}): Promise<Summary> {@@ -558,6 +586,15 @@ async function main(argv: string[]): Promise<void> {
558586 const sql = rest.filter((a) => a !== '--json' && a !== '--csv').join(' ');559587 if (!sql.trim()) throw new LakeError('usage: forge lake query "<sql>" [--json|--csv]');560588 process.stdout.write(await query(cfg, sql, fmt));589+ } else if (cmd === 'forget') {590+ if (rest.length !== 1 || !/^[^/]+\/[^/]+$/.test(rest[0])) throw new LakeError('usage: node lib/lake.ts forget <owner>/<repo>');591+ if (!fs.existsSync(cfg.catalog)) return;592+ const lake = await Lake.open(cfg, { write: true });593+ try {594+ console.log(`lake: forgot ${plural(await forget(lake, rest[0]), 'row')} of ${rest[0]}`);595+ } finally {596+ lake.close();597+ }561598 } else if (cmd === 'install') {562599 await install(cfg);563600 } else if (cmd === 'stage') {@@ -0,0 +1,38 @@
1+# forge repo rm takes the repo's rows out of every lake table, and the2+# lake's note of where it read the repo's sources up to. So a private repo3+# removed and a new public one made under its name don't share a history:4+# the new one's insights have only its own facts.5+. "$(dirname "$0")/../lib.sh"6+user dami7+user ana8+repo --private dami/greet9+mkdir -p "$FORGE_ROOT/dami/greet.git/forge" && echo ana >"$FORGE_ROOT/dami/greet.git/forge/collaborators"10+clone dami dami/greet "$T/w/d"11+at "$T/w/d" commit "secret plans"12+push_as dami "$T/w/d" origin main13+clone ana dami/greet "$T/w/a"14+at "$T/w/a" commit "idea"15+push_as ana "$T/w/a" origin HEAD:ana/idea16+build dami/greet17+ok forge lake18+for t in events build_repos reviews commits; do19+ [ "$(lq "SELECT count(*) FROM $t WHERE repo = 'dami/greet'")" -gt 0 ] || fail "no $t rows to forget"20+done21+[ "$(lq "SELECT count(*) FROM builds WHERE scope = 'dami/greet'")" -gt 0 ] || fail "no builds to forget"22+ok forge repo rm dami/greet23+for t in events build_repos reviews review_ends commits ci; do24+ eq "$(lq "SELECT count(*) FROM $t WHERE repo = 'dami/greet'")" 025+done26+eq "$(lq "SELECT count(*) FROM builds WHERE scope = 'dami/greet'")" 027+eq "$(lq "SELECT count(*) FROM sources WHERE source LIKE '% dami/greet' OR source LIKE '% dami/greet %'")" 028+29+repo dami/greet30+clone dami dami/greet "$T/w/n"31+at "$T/w/n" commit "fresh start"32+push_as dami "$T/w/n" origin main33+build dami/greet34+ok forge lake35+eq "$(lq "SELECT string_agg(subject, ',') FROM commits WHERE repo = 'dami/greet'")" "fresh start"36+eq "$(lq "SELECT count(*) FROM reviews WHERE repo = 'dami/greet'")" 037+has "$FORGE_WWW/dami/greet/insights/insights.json" '"pushes":1,'38+lacks "$FORGE_WWW/dami/greet/insights/insights.json" 'ana/idea|secret plans'@@ -0,0 +1,25 @@
1+# Going private or being removed takes a repo off the index, its owner's2+# page and repos.json before the command returns, not in a build in the3+# background: the mirror's sync within 30 seconds (mirror_kick soon) must4+# not find the listing still there.5+. "$(dirname "$0")/../lib.sh"6+user dami7+repo dami/greet8+repo dami/other9+for r in greet other; do10+ clone dami "dami/$r" "$T/w/$r"11+ at "$T/w/$r" commit "first"12+ push_as dami "$T/w/$r" origin main13+done14+build15+has "$FORGE_WWW/repos.json" '"dami/greet"'16+# Builds in the background from here, as on the server.17+export FORGE_NO_BUILD=018+ok ssh_as dami private dami/greet19+lacks "$FORGE_WWW/repos.json" '"dami/greet"'20+lacks "$FORGE_WWW/index.html" 'dami/greet'21+lacks "$FORGE_WWW/dami/index.html" 'href="/dami/greet/"'22+has "$FORGE_WWW/repos.json" '"dami/other"'23+ok ssh_as dami rm dami/other --yes24+lacks "$FORGE_WWW/repos.json" '"dami/other"'25+missing "$FORGE_WWW/dami"