clone

a personal forge on plain git: static pages, stacked-diff reviews, signed local ci, repo sites and releases

Remove repos into a trash that gc empties

cat committed 091641c
ssh rm <owner>/<repo> --yes lets a repo's owner take it off the forge,
and forge repo rm does the same as root. The bare repo moves to
.forge/trash/<owner>/<repo>.git.<time>, and everything that served it
goes at once, the way going private takes a repo's public side away:
its pages in either root with their release downloads, its sites,
previews and hostnames, its insights exports and caches; the index,
owner pages and repos.json are rebuilt, and the mirror's next sync
deletes its pages.

forge repo restore puts back the newest removal while the name is free,
and forge gc deletes trash older than FORGE_TRASH_DAYS (14). Without
--yes, rm only says what would go; collaborators can't remove a repo,
and a private one you can't read isn't there to remove.
commit
091641c558375ac17aa8a7d466b71dbdb055ee94
parent
fcf11b25f343ae303ae9b78e2d6412c63cacf98e
author
cat <admin@dev.site>
kind
feat risk score 1.7 of 3, Jev's estimate
3 files changed, +154 -4
MREADME.md13++++-
Mbin/forge88+++++++++++++++++++++++++++++-
Atest/scenarios/repo-rm.sh57+++++++++++++++++++
M README.md +12 -1
@@ -149,6 +149,16 @@ The first branch pushed becomes the default if it isn't `main`. A push to a
149149 missing repo under someone else's name, or a clone or fetch of one, fails150150 as before.151151 152+`ssh git@git.example.com rm dami/notes --yes` takes a repo of yours off the153+forge (without `--yes` it says what would go); collaborators can't, and154+`forge repo rm <owner>/<repo>` does the same as root. Its clones, pages in155+either root, sites, previews, hostnames, release downloads and insights go156+at once, and the mirror's next sync deletes its pages. The bare repo waits in157+`.forge/trash/<owner>/<repo>.git.<time>` for `FORGE_TRASH_DAYS` (14) days,158+backed up like the rest of the forge: `forge repo restore <owner>/<repo>`159+puts back the newest removal if the name is free, and `forge gc` deletes it160+after that.161+152162 ### Private repos153163 154164 A repo is public while its bare repo has a `git-daemon-export-ok` file. That
@@ -795,7 +805,8 @@ the private root. A deploy extracts its tree before it links it, and
795805 touches a tree it reuses, so store entries changed in the last 10 minutes796806 are kept (`--age <minutes>`), which makes gc safe to run while people push. `--dry-run` lists797807 what it would remove. A release published again leaves its old files in the798-repo's packs, unreachable: gc prunes nothing.808+repo's packs, unreachable: gc prunes nothing. Removed repos leave the trash809+after `FORGE_TRASH_DAYS` (14) days.799810 800811 Then it maintains every repo:801812
M bin/forge +85 -3
@@ -24,6 +24,8 @@
2424 # forge keys sync2525 # forge repo new [--private] <owner>/<name> [description]2626 # forge repo import [--private] <owner>/<name> <url> [description] (server-side fetch)27+# forge repo rm <owner>/<name> into the trash, with everything that served it28+# forge repo restore <owner>/<name> back from the trash (the newest removal)2729 # forge build [owner/repo]2830 # forge jev [owner/repo] ask Jev about new commits and reviews (needs TYPESAFE_API_KEY)2931 # forge metrics [n] [owner/repo] recent builds and where their time went
@@ -983,7 +985,7 @@ cmd_server_init() {
983985 # FORGE_SITES_URL=/sites/{owner}/{repo}/{site}/984986 # FORGE_PREVIEWS_URL=/previews/{owner}/{repo}/{site}/{branch}/985987 EOF986- for c in new ls whoami signers site-url reviews comment approve approvals land private public readers release passkey passkeys sessions token tokens help; do988+ for c in new ls whoami signers site-url reviews comment approve approvals land private public readers release rm passkey passkeys sessions token tokens help; do987989 printf '#!/bin/sh\nexec "%s" shell-cmd %s "$@"\n' "$FORGE_BIN" "$c" >"$FORGE_ROOT/git-shell-commands/$c"988990 chmod +x "$FORGE_ROOT/git-shell-commands/$c"989991 done
@@ -1251,7 +1253,11 @@ cmd_repo() {
12511253 repo_import "$1" "$2" "${3:-}" $private12521254 return12531255 fi1254- [ "$sub" = new ] && [ $# -ge 1 ] || die "usage: forge repo new [--private] <owner>/<name> [description] | repo import [--private] <owner>/<name> <url> [description]"1256+ case $sub:$#:$private in1257+ rm:1:) load_config; repo_rm "$1"; return ;;1258+ restore:1:) load_config; repo_restore "$1"; return ;;1259+ esac1260+ [ "$sub" = new ] && [ $# -ge 1 ] || die "usage: forge repo new [--private] <owner>/<name> [description] | repo import [--private] <owner>/<name> <url> [description] | repo rm|restore <owner>/<name>"12551261 load_config12561262 owner=${1%%/*} name=${1#*/}12571263 valid_name "$owner" && valid_name "$name" && [ "$owner/$name" = "$1" ] || die "invalid repo: $1"
@@ -1267,6 +1273,55 @@ cmd_repo() {
12671273 say "created $owner/$name${private:+ (private)}"12681274 }12691275 1276+# repo_rm <owner>/<repo>: take a repo off the forge. The bare repo moves to1277+# .forge/trash/<owner>/<repo>.git.<time>, where forge repo restore finds it1278+# and forge gc deletes it after FORGE_TRASH_DAYS (14) days; backups keep the1279+# trash like any other state. Everything that served it goes now, the way1280+# going private takes a repo's public side away: its pages in either root1281+# (with its release downloads), its sites, previews and their hostnames,1282+# its insights exports and its caches. The index, owner pages and1283+# repos.json are rebuilt without it, and the mirror's next sync deletes its1284+# pages from the bucket.1285+repo_rm() {1286+ owner=${1%%/*} name=${1#*/}1287+ valid_name "$owner" && valid_name "$name" && [ "$owner/$name" = "$1" ] && [ -f "$FORGE_ROOT/$1.git/HEAD" ] || die "no repo $1"1288+ trash=$FORGE_STATE/trash/$owner1289+ mkdir -p "$trash"1290+ t=$(date -u +%Y%m%dT%H%M%SZ)1291+ mv "$FORGE_ROOT/$1.git" "$trash/$name.git.$t"1292+ # The time it was removed, which gc counts from.1293+ touch "$trash/$name.git.$t"1294+ for l in "$FORGE_WWW/sites/$1"/* "$FORGE_WWW/previews/$1"/*/*; do1295+ if [ -L "$l" ]; then deploy_site "${l#"$FORGE_WWW"/}" "$ZERO"; fi1296+ done1297+ rm -rf "${FORGE_WWW:?}/$1" "${FORGE_PRIVATE_WWW:?}/$1" "${FORGE_WWW:?}/sites/$1" "${FORGE_WWW:?}/previews/$1" \1298+ "${FORGE_STATE:?}/lake/export/$1" "${FORGE_STATE:?}/cache/repos/$1.json" "${FORGE_STATE:?}/cache/code/$1"1299+ rm -f "$FORGE_WWW"/insights/*.json "$FORGE_WWW"/insights/*.parquet1300+ rebuild - removed1301+ mirror_kick1302+ echo "removed $1; forge repo restore $1 brings it back within ${FORGE_TRASH_DAYS:-14} days (the trash: ${trash#"$FORGE_ROOT"/}/$name.git.$t)"1303+}1304+1305+# repo_restore <owner>/<repo>: put back the newest removal of a repo, if the1306+# name is free, with this forge's hooks, and build its pages again.1307+repo_restore() {1308+ owner=${1%%/*} name=${1#*/}1309+ valid_name "$owner" && valid_name "$name" && [ "$owner/$name" = "$1" ] || die "invalid repo: $1"1310+ [ ! -e "$FORGE_ROOT/$1.git" ] || die "$1 exists; remove or rename it first"1311+ from=$(ls -d "$FORGE_STATE/trash/$owner/$name.git."* 2>/dev/null | sort | tail -n 1)1312+ [ -n "$from" ] || die "no removed $1 in the trash"1313+ mkdir -p "$FORGE_ROOT/$owner"1314+ mv "$from" "$FORGE_ROOT/$1.git"1315+ repo_setup "$FORGE_ROOT/$1.git"1316+ (1317+ export GIT_DIR="$FORGE_ROOT/$1.git"1318+ OWNER=$owner NAME=$name1319+ if sites_public "$GIT_DIR"; then redeploy_sites; fi1320+ )1321+ rebuild "$1" restored1322+ echo "restored $1 (removed ${from##*.git.})"1323+}1324+12701325 # can_read <git-dir> <user>: anyone may read a public repo; a private one,12711326 # its owner and the names in its forge/collaborators and forge/readers.12721327 can_read() {
@@ -1763,6 +1818,21 @@ sc_readers() {
17631818 esac17641819 }17651820 1821+# rm <owner>/<repo> --yes: its owner removes it (repo_rm); collaborators1822+# can't. Without --yes it only says what would go. A repo you can't read1823+# isn't there (open_repo).1824+sc_rm() {1825+ case $#:${2:-} in 1: | 2:--yes) ;; *) die "usage: rm <owner>/<repo> --yes" ;; esac1826+ open_repo "$1"1827+ [ "$user" = "${1%%/*}" ] || die "only ${1%%/*} can remove $1"1828+ if [ "${2:-}" != --yes ]; then1829+ echo "rm $1 --yes takes it off the forge: clones over ssh and HTTPS, its pages, sites, previews and release downloads."1830+ echo "It goes to the trash, where the forge's admin can restore it for ${FORGE_TRASH_DAYS:-14} days."1831+ exit 11832+ fi1833+ repo_rm "$1"1834+}1835+17661836 # release ls <owner>/<repo> | release get <owner>/<repo> <tag> <file>: for17671837 # anyone who may read the repo, so a private repo's releases, which the web17681838 # gives its readers only once they sign in or show a token, are there for
@@ -1857,6 +1927,7 @@ cmd_shell_cmd() {
18571927 public) sc_public "$@" ;;18581928 readers) sc_readers "$@" ;;18591929 release) sc_release "$@" ;;1930+ rm) sc_rm "$@" ;;18601931 passkey | passkeys | sessions | tokens) credentials "$c" "$@" ;;18611932 token)18621933 # A token for one repo needs a repo you can read, or it isn't there.
@@ -1869,7 +1940,7 @@ cmd_shell_cmd() {
18691940 [ -z "$r" ] || open_repo "$r"18701941 credentials token "$@"18711942 ;;1872- *) printf 'commands:\n new [--private] <name> [description] create %s/<name>\n ls list your repos, and private ones you may read\n signers keys that may sign CI results (for forge verify)\n site-url <owner>/<repo> <site> [branch] where a site deploy is served\n reviews <owner>/<repo> open and draft reviews (forge reviews)\n land <owner>/<repo> <id|branch> land a review here, with the same gates as a push\n comment, approve, approvals used by forge comment, approve and land\n private <owner>/<repo> only its owner, collaborators and readers can see it\n public <owner>/<repo> [--force] anyone can, once its whole history passes the secret check\n readers <owner>/<repo> [add|rm <user>] who may read it while it is private\n release ls <owner>/<repo> its releases: tag, file, size, sha256\n release get <owner>/<repo> <tag> <file> a release file, on stdout\n passkey [name] a one-time link that adds a passkey, to sign in on the web\n passkeys [rm <id>] your passkeys\n sessions [rm <id>|--all] where you are signed in on the web; rm signs one out, --all all\n token [--repo <owner>/<repo>] [--ttl 30d] [name] a token for curl and HTTPS clones, shown once\n tokens [rm <id>] your tokens\na push to %s/<name>.git creates that repo when it is missing\n' "$user" "$user" ;;1943+ *) printf 'commands:\n new [--private] <name> [description] create %s/<name>\n ls list your repos, and private ones you may read\n signers keys that may sign CI results (for forge verify)\n site-url <owner>/<repo> <site> [branch] where a site deploy is served\n reviews <owner>/<repo> open and draft reviews (forge reviews)\n land <owner>/<repo> <id|branch> land a review here, with the same gates as a push\n comment, approve, approvals used by forge comment, approve and land\n private <owner>/<repo> only its owner, collaborators and readers can see it\n public <owner>/<repo> [--force] anyone can, once its whole history passes the secret check\n readers <owner>/<repo> [add|rm <user>] who may read it while it is private\n release ls <owner>/<repo> its releases: tag, file, size, sha256\n release get <owner>/<repo> <tag> <file> a release file, on stdout\n rm <owner>/<repo> --yes remove a repo of yours (its owner restores it from the trash)\n passkey [name] a one-time link that adds a passkey, to sign in on the web\n passkeys [rm <id>] your passkeys\n sessions [rm <id>|--all] where you are signed in on the web; rm signs one out, --all all\n token [--repo <owner>/<repo>] [--ttl 30d] [name] a token for curl and HTTPS clones, shown once\n tokens [rm <id>] your tokens\na push to %s/<name>.git creates that repo when it is missing\n' "$user" "$user" ;;18731944 esac18741945 }18751946
@@ -2585,6 +2656,17 @@ cmd_gc() {
25852656 if [ -d "$FORGE_WWW/hosts" ]; then25862657 if [ "$dry" = 1 ]; then node "$FORGE_HOME/lib/hosts.ts" prune --dry-run; else node "$FORGE_HOME/lib/hosts.ts" prune; fi25872658 fi2659+ # Removed repos (forge repo rm) stay in the trash FORGE_TRASH_DAYS days.2660+ days=${FORGE_TRASH_DAYS:-14}2661+ case $days in '' | *[!0-9]*) die "gc: FORGE_TRASH_DAYS takes whole days" ;; esac2662+ if [ -d "$FORGE_STATE/trash" ]; then2663+ set --2664+ [ "$days" = 0 ] || set -- -mtime +"$((days - 1))"2665+ for x in $(find "$FORGE_STATE/trash" -mindepth 2 -maxdepth 2 -name '*.git.*' "$@"); do2666+ [ "$dry" = 1 ] || rm -rf "$x"2667+ echo "$did trash ${x#"$FORGE_STATE"/trash/}"2668+ done2669+ fi25882670 # Candidates before links: a deploy that links a tree after this looked25892671 # has touched it first, so it can't be a candidate.25902672 n=0 kb=0
A test/scenarios/repo-rm.sh +57 -0
@@ -0,0 +1,57 @@
1+# forge repo rm and ssh rm take a repo off the forge: clones, pages, sites2+# and previews go, the bare repo waits in the trash, forge repo restore3+# brings back the newest removal, and gc empties the trash after4+# FORGE_TRASH_DAYS. Only the owner may rm over ssh, and a private repo you5+# can't read isn't there to remove.6+. "$(dirname "$0")/../lib.sh"7+user dami8+user ana9+repo dami/greet10+mkdir -p "$FORGE_ROOT/dami/greet.git/forge" && echo ana >"$FORGE_ROOT/dami/greet.git/forge/collaborators"11+clone dami dami/greet "$T/w/d"12+at "$T/w/d" commit "first"13+at "$T/w/d" git switch -q -c dami/topic14+at "$T/w/d" commit "topic"15+ok push_as dami "$T/w/d" origin main dami/topic "$(site_commit "$T/w/d" docs):refs/sites/docs" "$(site_commit "$T/w/d" preview):refs/site-previews/docs/dami/topic"16+build17+exists "$FORGE_WWW/dami/greet/index.html"18+exists "$FORGE_WWW/sites/dami/greet/docs/index.html"19+exists "$FORGE_WWW/previews/dami/greet/docs/dami~topic/index.html"20+has "$FORGE_WWW/repos.json" '"dami/greet"'21+22+# Without --yes it says what would go, and nothing does.23+fails_with "takes it off the forge" ssh_as dami rm dami/greet24+exists "$FORGE_ROOT/dami/greet.git/HEAD"25+# Collaborators can't; a private repo you can't read looks missing.26+fails_with "only dami can remove dami/greet" ssh_as ana rm dami/greet --yes27+repo --private dami/secret28+fails_with "^forge: no repo dami/secret$" ssh_as ana rm dami/secret --yes29+fails_with "^forge: no repo dami/nothing$" ssh_as ana rm dami/nothing --yes30+31+ok ssh_as dami rm dami/greet --yes32+missing "$FORGE_ROOT/dami/greet.git"33+missing "$FORGE_WWW/dami/greet"34+missing "$FORGE_WWW/sites/dami/greet"35+missing "$FORGE_WWW/previews/dami/greet"36+fails_with "does not appear to be a git repo" fetch_ssh dami dami/greet37+eq "$(ls "$FORGE_ROOT/.forge/trash/dami" | grep -c '^greet\.git\.')" 138+build -39+lacks "$FORGE_WWW/repos.json" '"dami/greet"'40+41+# The name is free again; restore won't overwrite what took it.42+repo dami/greet43+fails_with "dami/greet exists" forge repo restore dami/greet44+ok forge repo rm dami/greet45+eq "$(ls "$FORGE_ROOT/.forge/trash/dami" | grep -c '^greet\.git\.')" 246+# The newest removal comes back: the empty repo, not the first one.47+ok forge repo restore dami/greet48+exists "$FORGE_ROOT/dami/greet.git/hooks/pre-receive"49+fails git --git-dir="$FORGE_ROOT/dami/greet.git" rev-parse -q --verify refs/heads/main50+eq "$(ls "$FORGE_ROOT/.forge/trash/dami" | grep -c '^greet\.git\.')" 151+52+# gc keeps the trash for FORGE_TRASH_DAYS, then empties it.53+ok forge gc --age 054+eq "$(ls "$FORGE_ROOT/.forge/trash/dami" | grep -c '^greet\.git\.')" 155+ok env FORGE_TRASH_DAYS=0 "$FORGE" gc --age 056+output | grep -q '^removed trash dami/greet\.git\.' || fail "gc removed no trash"57+eq "$(ls "$FORGE_ROOT/.forge/trash/dami" | grep -c '^greet\.git\.' || true)" 0