.forge/ci writes "name number [unit]" lines to $FORGE_METRICS; forge attest signs them into the statement as metric lines. The lake keeps them in ci_metrics, and commits gains the tree, so a repo's insights page charts each metric on its default branch, a reworded commit keeping its values. A lake made before a column existed gets it when next opened to write.
- commit
- 242833e9b6329682e1969af0591105b196d08a39
- parent
- c3443814e355e88b193b53919496daa1c04ac2fc
- author
- cat <admin@dev.site>
- ci
- ✓ passed in 116s, signed by cat
- kind
- feat risk score 1.7 of 3, Jev's estimate
- review
- #33 CI metrics, approval rules, agent reviews, blame
ci attestation
forge-attestation 1 type ci commit 242833e9b6329682e1969af0591105b196d08a39 tree 33d472363855d04e97ff234f865dd314ae430337 script .forge/ci 3dbc91920fbe9105a57a13da4f17861aa3500f04 result pass exit 0 started 2026-11-04T04:55:52Z duration 116s runner (not shown here; refs/notes/ci has it) system Darwin 27.2.0 arm64 env host log sha256:619a37813eb3db1e02940472c6d89804a11edd56a2d349c94ebabd143f88d201 328 lines |pushed wip/i18n |8a36d94 running .forge/ci: greet.sh: say what it is | | sh: /bin/sh | | ok greet |8a36d94 pass in 0s, signed |pushed main |remote: review #4: /dami/greet/reviews/4/ |pushed wip/i18n |3b25da7 running .forge/ci: wave at the world | | sh: /bin/sh | | FAIL greet : got 'hello, world o/', want 'hello, world' |3b25da7 fail in 0s, signed |forge: 1 commit(s) did not pass |remote: review #6: /dami/greet/reviews/6/ |pushed ana/wave |3fa0c29 running .forge/ci: update the test for the wave | | sh: /bin/sh | | ok greet |3fa0c29 pass in 0s, signed |remote: review #6: /dami/greet/reviews/6/ |pushed ana/wave |warning: You appear to have cloned an empty repository. |site home: deployed f127773 -> /sites/ana/dotfiles/home/ |created ana/notes |remote: review #1: /ana/notes/reviews/1/ |pushed ana/todo |landing #1 A todo list: ana/todo -> main |48df1dc running .forge/ci: web, server and Python versions of greet | | sh: /bin/sh | | ok greet |48df1dc pass in 0s, signed |pushed main |d6b97a1 running .forge/ci: messages: notes for the later regions | | sh: /bin/sh | | ok greet |d6b97a1 pass in 0s, signed |remote: review #7: /dami/greet/reviews/7/ |pushed dami/regions |created dami/forge |warning: You appear to have cloned an empty repository. |site guide: mkdir -p guide/fonts && cp site/assets/fonts/*.woff2 guide/fonts/ |site guide: deployed f4d7813 -> /sites/dami/forge/guide/ |(node:27647) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built ana/dotfiles in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.tQeXMx/wt/demo/root/.forge/www |(node:27820) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built ana/notes in 0s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.tQeXMx/wt/demo/root/.forge/www |(node:28006) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built dami/forge in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.tQeXMx/wt/demo/root/.forge/www |(node:28187) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built dami/greet in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.tQeXMx/wt/demo/root/.forge/www |(node:28902) ExperimentalWarning: stripTypeScriptTypes is an experimental feature and might change at any time |(Use `node --trace-warnings ...` to show where the warning was created) |built - in 1s -> /var/folders/b4/mv8w4c8j0zj0_mldppxdbdw00000gn/T/forge-ci.tQeXMx/wt/demo/root/.forge/www |235 of 235 pages render the same without 11ty |guide: 8 files, links ok |ok
Signature: good, cat
| M | README.md | 32 | +++++++++++++++++-- |
| M | bin/ | 26 | +++++++++++++++- |
| M | demo/ | 3 | ++ |
| M | lib/ | 11 | ++++++- |
| M | lib/ | 31 | ++++++++++++++++++- |
| M | lib/ | 26 | ++++++++++++---- |
| M | lib/ | 26 | ++++++++++++--- |
| M | lib/ | 8 | +++++ |
| A | site/ | bin | |
| M | site/ | 11 | ++++++ |
| M | site/ | 36 | ++++++++++++++++++++- |
| A | test/ | 51 | ++++++++++++++++++++++++++++++ |
@@ -524,6 +524,23 @@ dishonest key holder is not caught. To make the claim stronger:
524524 - have a second person run `forge attest` on their machine. Attestations from525525 several signers stack up on the same tree.526526 527+`.forge/ci` can also report numbers about the commit: bundle size, test528+count, benchmark times. It writes them to the file named by `$FORGE_METRICS`,529+a line each, as a name, a number and an optional unit:530+531+```sh532+echo "bundle_kb $(du -k dist/app.js | cut -f1) KB" >>"$FORGE_METRICS"533+echo "tests $(grep -c '^ok' test.log)" >>"$FORGE_METRICS"534+```535+536+They go into the statement as `metric bundle_kb 412 KB` lines, before the537+log, so they are signed with the result. A name is lowercase letters, digits538+and `_ . -`, starting with a letter; a unit is up to 16 letters, `%` or `/`.539+`forge attest` leaves out, with a warning, a line that isn't one, a name540+seen before and anything past the 100th. Like the result, they are what the541+signer's machine said. The repo's insights page charts each one by commit on542+the default branch (see [Insights pages](#insights-pages)).543+527544 To require a verified pass before `main` moves, set528545 `git config forge.requireci tip` (or `all`, for every new commit) in the bare529546 repo.@@ -832,7 +849,9 @@ Tables: `events` (pushes, comments, approvals, builds), `builds` and
832849 `build_repos` (each line of `metrics.jsonl`), `reviews` (each line of a833850 review log: created, revisions, approvals, deleted), `review_ends` (landed834851 or closed, and when) and `ci` (each attestation: commit, tree, result,835-duration, runner, signer; `ci.parquet` leaves the runner out). A review counts as landed when its branch is852+duration, runner, signer; `ci.parquet` leaves the runner out), `ci_metrics`853+(each metric an attestation reported, with its id) and `commits` (each854+commit on a default branch: time, author, subject and tree). A review counts as landed when its branch is836855 deleted with its tip in the default branch, as `forge land` leaves it; one837856 whose tip reaches the default branch while the branch stays counts from838857 when the lake saw it.@@ -852,6 +871,8 @@ when the lake saw it.
852871 one waits for it (`FORGE_LAKE_WAIT` seconds, 300) or gives up and says853872 so. Builds start it the way they start the mirror, `FORGE_LAKE_DELAY`854873 seconds (10) after the last one; `FORGE_LAKE=0` stops that.874+- **New columns.** A table that gains a column in a newer forge gets it875+ when the lake is next opened to write, empty in the rows it had.855876 - **Small writes.** DuckLake keeps inserts under 10 rows in the catalog856877 rather than writing a Parquet file for each; about once a day857878 (`FORGE_LAKE_CHECKPOINT` hours) a run checkpoints, which writes them out@@ -875,12 +896,17 @@ not R2 itself.
875896 876897 `/insights/` covers the forge, and each repo has `/<owner>/<repo>/insights/`877898 (the Insights tab, `i`): stat tiles, then build time, pushes, time to land,878-revisions per review, CI results and CI duration. The pages are static and899+revisions per review, CI results and CI duration. A repo whose CI reports900+metrics also gets a chart for each, its value at each commit on the default901+branch (the latest 400): a commit takes the value from an attestation of its902+tree, so a reworded commit keeps it, and a verified, passing one comes903+first. The forge's page has none, since a metric's name means something904+only in its repo. The pages are static and879905 built once; their script (`insights.js`, on these pages only) fetches880906 `insights.json` and draws the charts as SVG, light and dark, with a tooltip881907 on hover and keyboard focus and a table of each chart's numbers. After each882908 run, `forge lake` writes that JSON and the rows behind it as Parquet883-(`events`, `builds`, `reviews`, `ci`) into `.forge/lake/export` and straight909+(`events`, `builds`, `reviews`, `ci`, `ci_metrics`) into `.forge/lake/export` and straight884910 into the pages, so they update without a build, and the mirror picks them885911 up. A repo's exports sit in its page directory, which builds keep. DuckDB886912 reads the Parquet files over HTTP in byte ranges:@@ -270,6 +270,29 @@ tree_passed() {
270270 END { exit !found }'271271 }272272 273+# ci_metrics <file>: the statement's metric lines from what .forge/ci wrote274+# to $FORGE_METRICS, a line each: a name, a number and an optional unit275+# ("bundle_kb 412", "tests 318", "p50 41.5 ms"). A name is lowercase276+# letters, digits and _ . -, starting with a letter, at most 64 characters;277+# a unit is up to 16 letters, % or /. A line that isn't one, a name seen278+# before and anything past the 100th are left out, with a warning.279+ci_metrics() {280+ LC_ALL=C awk '281+ function warn(m) { printf "forge: FORGE_METRICS: %s\n", m > "/dev/stderr" }282+ { sub(/\r$/, ""); sub(/^[ \t]+/, ""); sub(/[ \t]+$/, "") }283+ $0 == "" || /^#/ { next }284+ {285+ bad = NF < 2 || NF > 3 || length($1) > 64 || $1 !~ /^[a-z][a-z0-9_.-]*$/ ||286+ $2 !~ /^-?([0-9]+\.?[0-9]*|\.[0-9]+)([eE][-+]?[0-9]+)?$/ ||287+ (NF == 3 && (length($3) > 16 || $3 !~ /^[A-Za-z%\/]+$/))288+ if (bad) { warn("line " NR " is not \"name number [unit]\"; left out"); next }289+ if ($1 in seen) { warn($1 " twice; the first is kept"); next }290+ if (++n > 100) { if (n == 101) warn("more than 100 metrics; the rest are left out"); next }291+ seen[$1] = 1292+ print "metric " $1 " " $2 (NF == 3 ? " " $3 : "")293+ }' "$1"294+}295+273296 attest_one() {274297 c=$1 key=$2 force=$3 clean=${4:-0}275298 sha=$(git rev-parse "$c")@@ -295,7 +318,7 @@ attest_one() {
295318 { (296319 set +e297320 cd "$ci_wt" || exit298- FORGE_CI=1 FORGE_COMMIT=$sha $run </dev/null321+ FORGE_CI=1 FORGE_COMMIT=$sha FORGE_METRICS=$tmp/metrics $run </dev/null299322 echo $? >"$tmp/code"300323 ) 2>&1 | tee "$tmp/log" | sed 's/^/ | /' >&2; } || true301324 code=$(cat "$tmp/code" 2>/dev/null || echo 255)@@ -316,6 +339,7 @@ attest_one() {
316339 echo "runner $(whoami_at)"317340 echo "system $(uname -srm)"318341 echo "env $envdesc"342+ [ ! -f "$tmp/metrics" ] || ci_metrics "$tmp/metrics"319343 echo "log sha256:$(sha256 "$tmp/log") $(wc -l <"$tmp/log" | tr -d ' ') lines"320344 tail -n "${FORGE_LOG_LINES:-60}" "$tmp/log" | sanitize | sed 's/^/|/'321345 } >"$tmp/statement"@@ -113,6 +113,9 @@ cat >.forge/ci <<'EOF'
113113 set -e114114 echo "sh: $(command -v sh)"115115 sh test.sh116+# Numbers about the commit, which the insights page charts.117+echo "tests $(grep -c . tests.txt)" >>"$FORGE_METRICS"118+echo "script_bytes $(wc -c <greet.sh | tr -d ' ') B" >>"$FORGE_METRICS"116119 EOF117120 chmod +x .forge/ci118121 commit "add tests and a local CI script" "CI runs on your own machine: forge attest signs the result@@ -9,7 +9,7 @@ import { createHash } from 'node:crypto';
99 import fs from 'node:fs';1010 import os from 'node:os';1111 import path from 'node:path';12-import type { Attestation, CiStatus, Verification } from './types.ts';12+import type { Attestation, CiStatus, Metric, Verification } from './types.ts';1313 1414 export const NAMESPACE = 'forge-ci';1515 const BLOCK = /^forge-attestation 1\n[\s\S]*?^-----END SSH SIGNATURE-----$/gm;@@ -23,11 +23,19 @@ export function parseNote(text: string): Attestation[] {
2323 const payload = block.slice(0, at);2424 const fields: Record<string, string> = {};2525 const log: string[] = [];26+ const metrics: Metric[] = [];2627 for (const line of payload.split('\n').slice(1)) {2728 if (line.startsWith('|')) {2829 log.push(line.slice(1));2930 continue;3031 }32+ // metric <name> <number> [<unit>], as forge attest writes them.33+ const m = /^metric ([a-z][a-z0-9_.-]{0,63}) (\S+)(?: ([A-Za-z%/]{1,16}))?$/.exec(line);34+ if (m) {35+ const value = Number(m[2]);36+ if (Number.isFinite(value) && !metrics.some((x) => x.name === m[1])) metrics.push({ name: m[1], value, unit: m[3] ?? null });37+ continue;38+ }3139 const sp = line.indexOf(' ');3240 if (sp > 0) fields[line.slice(0, sp)] = line.slice(sp + 1);3341 }@@ -46,6 +54,7 @@ export function parseNote(text: string): Attestation[] {
4654 script: fields.script,4755 logHash: fields.log,4856 log,57+ metrics,4958 verification: { ok: false, signer: null, reason: 'not checked' },5059 });5160 }@@ -22,11 +22,13 @@ import { listRepos } from './lake-sources.ts';
2222 import { lit, type Lake, type LakeConfig } from './lake.ts';2323 import type { RepoActivityExport } from './activity.ts';2424 25-export const FILES = ['insights.json', 'activity.json', 'events.parquet', 'builds.parquet', 'reviews.parquet', 'ci.parquet'];25+export const FILES = ['insights.json', 'activity.json', 'events.parquet', 'builds.parquet', 'reviews.parquet', 'ci.parquet', 'ci_metrics.parquet'];2626 // How far back the activity exports go: a year's grid, and some.2727 const DAYS = 400;2828 // The most points a chart gets; older ones are only in the Parquet files.2929 const CAP = 1000;30+// The most points a metric's chart gets: there can be many metrics.31+const METRIC_CAP = 400;3032 3133 const inList = (col: string, xs: string[]): string => (xs.length ? `${col} IN (${xs.map(lit).join(', ')})` : 'false');3234 const n = (v: unknown): number | null => (v == null ? null : Number(v));@@ -84,6 +86,7 @@ async function exportScope(lake: Lake, s: Scope, publicRepos: string[]): Promise
8486 reviews: reviews.map(([repo, id, c, e, how, revs, appr]) => [repo, n(id), n(c), n(e), how ?? null, n(revs), n(appr)]),8587 ci: ci.map(([t, d, p]) => [n(t), n(d), p ? 1 : 0]),8688 };89+ if (s.slug) out.metrics = await metrics(lake, s.slug);8790 if (!s.slug) {8891 const per = await rows(`8992 WITH p AS (SELECT repo, count(*) FILTER (type = 'push') AS pushes, max(t) FILTER (type = 'push') AS last FROM lake.events WHERE ${s.where} GROUP BY repo),@@ -107,6 +110,9 @@ async function exportScope(lake: Lake, s: Scope, publicRepos: string[]): Promise
107110 UNION ALL SELECT repo, review, kind, ts, "user", NULL, sha, NULL, NULL, NULL, NULL, observed FROM lake.review_ends WHERE ${s.where}108111 ORDER BY repo, review, ts`);109112 await copy('ci.parquet', `SELECT repo, "commit", tree, result, exit, started, duration_s, system, env, signer, verified FROM lake.ci WHERE ${s.where} ORDER BY started`);113+ await copy('ci_metrics.parquet', `114+ SELECT m.repo, m."commit", m.tree, m.name, m.value, m.unit, c.started, c.result, c.signer, c.verified115+ FROM (SELECT * FROM lake.ci_metrics WHERE ${s.where}) m JOIN lake.ci c USING (id) ORDER BY c.started, m.name`);110116 out.files = FILES.slice(2).map((f) => [f, fs.statSync(path.join(tmp, f)).size]);111117 if (s.slug) fs.writeFileSync(path.join(tmp, 'activity.json'), JSON.stringify(await repoActivity(lake, s.slug)) + '\n');112118 fs.writeFileSync(path.join(tmp, 'insights.json'), JSON.stringify(out) + '\n');@@ -120,6 +126,29 @@ async function exportScope(lake: Lake, s: Scope, publicRepos: string[]): Promise
120126 fs.rmSync(old, { recursive: true, force: true });121127 }122128 129+// A repo's metrics on its default branch, for a chart each: [name, unit,130+// points], by name, where a point is [commit time (ms), value, commit,131+// subject], oldest first. A commit gets the value of an attestation of its132+// tree (or of itself, for commits the lake has no tree of), the verified133+// and passing one first, then the latest; its unit is the latest point's.134+// Commits made the same second go in the order they were attested.135+async function metrics(lake: Lake, slug: string): Promise<[string, string | null, [number, number, string, string][]][]> {136+ const got = await lake.rows(`137+ WITH p AS (138+ SELECT m.name, m.value, m.unit, k.sha, k.t, k.subject, c.started139+ FROM lake.ci_metrics m JOIN lake.ci c USING (id)140+ JOIN lake.commits k ON k.repo = m.repo AND (k.tree = m.tree OR (k.tree IS NULL AND k.sha = m."commit"))141+ WHERE m.repo = ${lit(slug)} AND k.t IS NOT NULL142+ QUALIFY row_number() OVER (PARTITION BY m.name, k.sha ORDER BY c.verified DESC, c.result = 'pass' DESC, c.started DESC NULLS LAST) = 1),143+ q AS (SELECT *, row_number() OVER (PARTITION BY name ORDER BY t DESC, started DESC NULLS LAST, sha) AS i FROM p)144+ SELECT name, arg_max(unit, t), list([epoch_ms(t)::VARCHAR, value::VARCHAR, sha, left(subject, 100)] ORDER BY t, started NULLS FIRST, sha)145+ FROM q WHERE i <= ${METRIC_CAP} GROUP BY name ORDER BY name`);146+ return got.map(([name, unit, pts]) => [147+ String(name), unit == null ? null : String(unit),148+ (pts as string[][]).map(([t, v, sha, subj]) => [Number(t), Number(v), sha, subj ?? ''] as [number, number, string, string]),149+ ]);150+}151+123152 const since = (): string => {124153 const now = Number(process.env.FORGE_NOW) * 1000 || Date.now();125154 return lit(new Date(now - DAYS * 86400000).toISOString().slice(0, 10));@@ -348,17 +348,29 @@ export interface CiRow {
348348 verified: boolean;349349 }350350 351+export interface CiMetricRow {352+ id: string;353+ repo: string;354+ commit: string | null;355+ tree: string | null;356+ name: string;357+ value: number;358+ unit: string | null;359+}360+351361 // The tip of a repo's CI notes, so a run skips repos whose notes didn't move.352362 export const notesTip = (dir: string): string | null => tryRun(dir, ['rev-parse', '-q', '--verify', 'refs/notes/ci^{commit}'])?.trim() || null;353363 354364 // The attestations in a repo's refs/notes/ci that aren't in `known` (ids are355365 // a hash of the signed statement and its signature, so a note that gained a356366 // block only adds that block). Signatures are checked as they are read.357-export function ciRows(slug: string, dir: string, known: Set<string>, verifier: Verifier): CiRow[] {367+// Each one's metrics come with it, keyed by the same id.368+export function ciRows(slug: string, dir: string, known: Set<string>, verifier: Verifier): { rows: CiRow[]; metrics: CiMetricRow[] } {358369 const list = tryRun(dir, ['ls-tree', '-r', 'refs/notes/ci']);359- if (!list) return [];370+ if (!list) return { rows: [], metrics: [] };360371 const blobs = list.split('\n').filter(Boolean).map((l) => l.split(/\s+/)).filter((f) => f[1] === 'blob').map((f) => f[2]);361372 const out: CiRow[] = [];373+ const metrics: CiMetricRow[] = [];362374 const seen = new Set(known);363375 for (const b of catFileBatch(dir, blobs).values()) {364376 for (const a of parseNote(b.data.toString('utf8'))) {@@ -373,9 +385,10 @@ export function ciRows(slug: string, dir: string, known: Set<string>, verifier:
373385 started: a.started ? ts(Date.parse(a.started)) : null, duration_s: dur ? Number(dur[1]) : null,374386 runner: a.runner ?? null, system: a.system ?? null, env: a.env ?? null, signer: v.signer, verified: v.ok,375387 });388+ for (const m of a.metrics) metrics.push({ id, repo: slug, commit: a.commit ?? null, tree: a.tree ?? null, name: m.name, value: m.value, unit: m.unit });376389 }377390 }378- return out;391+ return { rows: out, metrics };379392 }380393 381394 export interface CommitRow {@@ -384,15 +397,16 @@ export interface CommitRow {
384397 t: string | null;385398 author: string | null;386399 subject: string | null;400+ tree: string | null;387401 }388402 389403 // The commits reachable from tip and not from since (all of them without390404 // it), first parent or not: every commit the default branch took in.391405 export function commitRows(slug: string, dir: string, tip: string, since: string | null): CommitRow[] {392- const out = tryRun(dir, ['log', '-z', '--no-color', '--format=%H%x1f%at%x1f%an%x1f%s', tip, ...(since ? [`^${since}`] : []), '--']);406+ const out = tryRun(dir, ['log', '-z', '--no-color', '--format=%H%x1f%at%x1f%an%x1f%T%x1f%s', tip, ...(since ? [`^${since}`] : []), '--']);393407 if (!out) return [];394408 return out.split('\0').filter(Boolean).map((rec) => {395- const [sha, at, author, subject] = rec.replace(/^\n/, '').split('\x1f');396- return { repo: slug, sha, t: ts(Number(at) * 1000), author: author || null, subject: subject ?? null };409+ const [sha, at, author, tree, subject] = rec.replace(/^\n/, '').split('\x1f');410+ return { repo: slug, sha, t: ts(Number(at) * 1000), author: author || null, subject: subject ?? null, tree: tree || null };397411 });398412 }@@ -28,7 +28,7 @@ import { Verifier } from './attest.ts';
2828 import { isAncestor, tryRun } from './git.ts';2929 import {3030 buildRows, ciRows, commitRows, eventRow, listRepos, notesTip, readNew, reviewEnd, reviewLogs, reviewRows,31- type BuildRepoRow, type BuildRow, type CiRow, type CommitRow, type EventRow, type Offset, type Read, type ReviewEnd, type ReviewRow,31+ type BuildRepoRow, type BuildRow, type CiMetricRow, type CiRow, type CommitRow, type EventRow, type Offset, type Read, type ReviewEnd, type ReviewRow,3232 } from './lake-sources.ts';3333 import { exportInsights, publishInsights } from './insights.ts';3434 @@ -87,10 +87,15 @@ const TABLES: Record<string, [string, string][]> = {
8787 reviews: [['repo', 'VARCHAR'], ['review', 'INTEGER'], ['kind', 'VARCHAR'], ['ts', 'TIMESTAMP'], ['user', 'VARCHAR'], ['rev', 'INTEGER'], ['sha', 'VARCHAR'], ['base', 'VARCHAR'], ['tree', 'VARCHAR'], ['branch', 'VARCHAR'], ['title', 'VARCHAR']],8888 review_ends: [['repo', 'VARCHAR'], ['review', 'INTEGER'], ['kind', 'VARCHAR'], ['ts', 'TIMESTAMP'], ['user', 'VARCHAR'], ['sha', 'VARCHAR'], ['observed', 'BOOLEAN']],8989 // Commits on each repo's default branch, as it moved: its author's time90- // (the day it counts for on activity grids) and subject.91- commits: [['repo', 'VARCHAR'], ['sha', 'VARCHAR'], ['t', 'TIMESTAMP'], ['author', 'VARCHAR'], ['subject', 'VARCHAR']],90+ // (the day it counts for on activity grids), subject and tree (which CI91+ // results and metrics are keyed by; commits added before it was kept92+ // have none).93+ commits: [['repo', 'VARCHAR'], ['sha', 'VARCHAR'], ['t', 'TIMESTAMP'], ['author', 'VARCHAR'], ['subject', 'VARCHAR'], ['tree', 'VARCHAR']],9294 // refs/notes/ci: one row an attestation.9395 ci: [['id', 'VARCHAR'], ['repo', 'VARCHAR'], ['commit', 'VARCHAR'], ['tree', 'VARCHAR'], ['result', 'VARCHAR'], ['exit', 'INTEGER'], ['started', 'TIMESTAMP'], ['duration_s', 'INTEGER'], ['runner', 'VARCHAR'], ['system', 'VARCHAR'], ['env', 'VARCHAR'], ['signer', 'VARCHAR'], ['verified', 'BOOLEAN']],96+ // The metrics in each attestation (.forge/ci writes them to97+ // $FORGE_METRICS): one row a metric, id the attestation's in ci.98+ ci_metrics: [['id', 'VARCHAR'], ['repo', 'VARCHAR'], ['commit', 'VARCHAR'], ['tree', 'VARCHAR'], ['name', 'VARCHAR'], ['value', 'DOUBLE'], ['unit', 'VARCHAR']],9499 // Where each source was read up to (lib/lake-sources.ts), written in the95100 // same transaction as its rows.96101 sources: [['source', 'VARCHAR'], ['ino', 'BIGINT'], ['pos', 'BIGINT'], ['head', 'VARCHAR'], ['tail', 'VARCHAR'], ['at', 'TIMESTAMP']],@@ -153,6 +158,14 @@ export class Lake {
153158 for (const [t, cols] of Object.entries(TABLES)) {154159 await lake.run(`CREATE TABLE IF NOT EXISTS lake.${t} (${cols.map(([c, ty]) => `${ident(c)} ${ty}`).join(', ')})`);155160 }161+ // Columns added since a lake was made: at the end of their table,162+ // empty in the rows it already has.163+ const have = new Set((await lake.rows(`SELECT table_name || ' ' || column_name FROM information_schema.columns WHERE table_catalog = 'lake'`)).map((r) => String(r[0])));164+ for (const [t, cols] of Object.entries(TABLES)) {165+ for (const [c, ty] of cols) {166+ if (!have.has(`${t} ${c}`)) await lake.run(`ALTER TABLE lake.${t} ADD COLUMN ${ident(c)} ${ty}`);167+ }168+ }156169 await lake.run('COMMIT');157170 }158171 } catch (e) {@@ -238,7 +251,7 @@ export async function forget(lake: Lake, slug: string): Promise<number> {
238251 const s = lit(slug);239252 const where: Record<string, string> = {240253 events: `repo = ${s}`, builds: `scope = ${s}`, build_repos: `repo = ${s}`, reviews: `repo = ${s}`,241- review_ends: `repo = ${s}`, commits: `repo = ${s}`, ci: `repo = ${s}`,254+ review_ends: `repo = ${s}`, commits: `repo = ${s}`, ci: `repo = ${s}`, ci_metrics: `repo = ${s}`,242255 sources: `source IN (${['main', 'commits', 'notes'].map((k) => lit(`${k} ${slug}`)).join(', ')}) OR starts_with(source, ${lit(`review ${slug} `)})`,243256 };244257 let n = 0;@@ -273,6 +286,7 @@ export async function ingest(lake: Lake, { dry = false, now = Date.now() }: { dr
273286 const redo: [string, number][] = [];274287 const ends: ReviewEnd[] = [];275288 const ci: CiRow[] = [];289+ const ciMetrics: CiMetricRow[] = [];276290 const commits: CommitRow[] = [];277291 278292 const jsonl = <T>(name: string, file: string, parse: (line: string) => T | null, take: (row: T) => void): void => {@@ -351,8 +365,9 @@ export async function ingest(lake: Lake, { dry = false, now = Date.now() }: { dr
351365 known.get(String(r))!.add(String(id));352366 }353367 }354- const rows = ciRows(repo.slug, repo.dir, known.get(repo.slug) ?? new Set(), verifier);368+ const { rows, metrics } = ciRows(repo.slug, repo.dir, known.get(repo.slug) ?? new Set(), verifier);355369 ci.push(...rows);370+ ciMetrics.push(...metrics);356371 if (rows.length) sum.touched.add(repo.slug);357372 next.set(nkey, { ino: 0, pos: 0, head: notes, tail: [] });358373 }@@ -378,6 +393,7 @@ export async function ingest(lake: Lake, { dry = false, now = Date.now() }: { dr
378393 await lake.insert('reviews', reviews, scratch);379394 await lake.insert('review_ends', ends, scratch);380395 await lake.insert('ci', ci, scratch);396+ await lake.insert('ci_metrics', ciMetrics, scratch);381397 await lake.insert('commits', commits, scratch);382398 const names = [...next.keys()];383399 for (let i = 0; i < names.length; i += 500) {@@ -67,6 +67,12 @@ export interface Verification {
6767 reason?: string;6868 }6969 70+export interface Metric {71+ name: string;72+ value: number;73+ unit: string | null;74+}75+7076 export interface Attestation {7177 payload: string;7278 signature: string;@@ -82,6 +88,8 @@ export interface Attestation {
8288 script?: string;8389 logHash?: string;8490 log: string[];91+ // What .forge/ci wrote to $FORGE_METRICS, in its order.92+ metrics: Metric[];8593 object?: string;8694 via?: string | null;8795 verification: Verification;Binary file.
@@ -31,6 +31,7 @@
3131 .insights svg .s1 { fill: var(--viz-1); }3232 .insights svg .fail { fill: var(--viz-fail); }3333 .insights svg .dot { stroke: var(--raised); stroke-width: 2; }34+.insights svg .line { fill: none; stroke: var(--viz-1); stroke-width: 1.5; stroke-linejoin: round; }3435 .insights svg .hair { stroke: var(--muted); stroke-width: 1; shape-rendering: crispEdges; }3536 .insights svg .on { filter: brightness(1.15); }3637 .insights svg .mark-label { fill: var(--fg); paint-order: stroke; stroke: var(--raised); stroke-width: 3px; stroke-linejoin: round; }@@ -50,6 +51,7 @@
5051 .insights pre.cmd { margin: 0; overflow-x: auto; white-space: pre; padding: var(--space-3); border-top: var(--hairline) solid var(--border); font-size: var(--text-sm); }5152 @media (forced-colors: active) {5253 .insights svg .s1, .insights svg .fail { fill: CanvasText; }54+ .insights svg .line { stroke: CanvasText; }5355 }5456 {% endcss %}5557 <div class="insights" data-insights="{{ base }}insights.json"{% if forgeWide %} data-forge="1"{% endif %}>@@ -70,6 +72,14 @@
7072 <figure class="panel" data-chart="ci"><figcaption class="panel-head"><h3>ci results</h3><span class="grow">attestations a <span class="ins-bucket">day</span>, passed and failed</span></figcaption><div class="plot"></div><details class="table"><summary>table</summary></details></figure>7173 <figure class="panel" data-chart="ciTime"><figcaption class="panel-head"><h3>ci duration</h3><span class="grow">seconds per run of <code>.forge/ci</code></span></figcaption><div class="plot"></div><details class="table"><summary>table</summary></details></figure>7274 </div>75+{% if not forgeWide %}76+{# A chart a metric (insights.ts draws them from insights.json's metrics),77+ shown once there is one. #}78+<section id="ins-metrics" hidden>79+{{ ui.heading('metrics', null, 'what .forge/ci reported for each commit on ' + p.repo.defaultBranch) }}80+<div class="charts"></div>81+</section>82+{% endif %}7383 {% if forgeWide %}7484 {% call ui.panel('<b>by repo</b>') %}7585 <table class="list nav" id="ins-repos">@@ -86,6 +96,7 @@
8696 <tr><td class="mono nowrap"><a href="{{ base }}builds.parquet">builds.parquet</a></td><td class="main">each page build: time spent, pages written</td></tr>8797 <tr><td class="mono nowrap"><a href="{{ base }}reviews.parquet">reviews.parquet</a></td><td class="main">each review's lifecycle: created, revisions, approvals, landed or closed</td></tr>8898 <tr><td class="mono nowrap"><a href="{{ base }}ci.parquet">ci.parquet</a></td><td class="main">each CI attestation: commit, tree, result, duration, system, signer</td></tr>99+<tr><td class="mono nowrap"><a href="{{ base }}ci_metrics.parquet">ci_metrics.parquet</a></td><td class="main">each metric an attestation reported: name, value, unit, with its commit and result</td></tr>89100 {% endcall %}90101 <p class="desc">DuckDB reads them over HTTP in byte ranges, so a query fetches only what it needs:</p>91102 <pre class="cmd">duckdb -c "SELECT result, count(*), median(duration_s) FROM '{{ forge.config.baseUrl or 'https://<this forge>' }}{{ base }}ci.parquet' GROUP BY result"</pre>@@ -21,6 +21,8 @@
2121 reviews: ReviewRow[];2222 ci: [number, number | null, number][];2323 repos?: [string, number, number, number | null, number, number, number, number | null][];24+ // [name, unit, [[commit time, value, commit, subject]...]], a repo's only.25+ metrics?: [string, string | null, [number, number, string, string][]][];2426 }2527 2628 // ---------------------------------------------------------------- formats@@ -29,6 +31,8 @@
2931 const secs = (s: number): string => (s < 10 ? `${Math.round(s * 100) / 100}s` : s < 60 ? `${r1(s)}s` : s < 3600 ? `${Math.floor(s / 60)}m ${Math.round(s % 60)}s` : `${r1(s / 3600)}h`);3032 const hours = (h: number): string => (h < 1 ? `${Math.round(h * 60)}m` : h < 48 ? `${r1(h)}h` : `${r1(h / 24)}d`);3133 const count = (n: number): string => (n >= 1e4 ? `${Math.round(n / 1e3)}K` : n >= 1e3 ? `${r1(n / 1e3)}K` : String(n));34+ // A metric's value: whole numbers grouped, others to three significant digits.35+ const num = (v: number): string => (Number.isInteger(v) ? v.toLocaleString() : Math.abs(v) >= 1000 ? Math.round(v).toLocaleString() : String(Number(v.toPrecision(3))));3236 const plural = (n: number, w: string): string => `${count(n)} ${w}${n === 1 ? '' : 's'}`;3337 const when = (ms: number): string => new Date(ms).toLocaleString(undefined, { month: 'short', day: 'numeric', hour: '2-digit', minute: '2-digit' });3438 const dayOf = (ms: number): string => new Date(ms).toLocaleDateString(undefined, { month: 'short', day: 'numeric', timeZone: 'UTC' });@@ -212,8 +216,9 @@
212216 rows: [string, string, string?][];213217 }214218 215- // Dots over time, one per fact. The last one carries its value.216- function dots(plot: HTMLElement, list: Dot[], yFmt: (v: number) => string): void {219+ // Dots over time, one per fact. The last one carries its value. A line220+ // joins them, oldest first, for a metric's values commit by commit.221+ function dots(plot: HTMLElement, list: Dot[], yFmt: (v: number) => string, joined = false): void {217222 const max = Math.max(...list.map((d) => d.v));218223 const f = frame(plot, max, yFmt);219224 const a = Math.min(...list.map((d) => d.t));@@ -221,6 +226,10 @@
221226 const pad = (b - a) * 0.03 || 3600e3;222227 const x = (t: number): number => PAD.left + ((t - (a - pad)) / (b - a + 2 * pad)) * f.plotW;223228 xLabels(f, timeTicks(a - pad, b + pad, Math.max(2, Math.floor(f.plotW / 90))).map((k) => ({ x: x(k.at), label: k.label })));229+ if (joined && list.length > 1) {230+ const by = [...list].sort((p, q) => p.t - q.t);231+ svgEl('path', { class: 'line', d: by.map((d, i) => `${i ? 'L' : 'M'}${x(d.t).toFixed(1)},${f.y(d.v).toFixed(1)}`).join('') }, f.svg);232+ }224233 const marks: Mark[] = list.map((d) => {225234 const c = svgEl('circle', { class: `dot ${d.fail ? 'fail' : 's1'}`, cx: x(d.t), cy: f.y(d.v), r: 4 }, f.svg);226235 return { cx: x(d.t), cy: f.y(d.v), els: [c], head: d.head, rows: d.rows };@@ -439,6 +448,29 @@
439448 table(fig, ['started', 'took', 'result'], () => [...runs].reverse().map(([t, s, p]) => [when(t), secs(s!), p ? 'passed' : 'failed']));440449 });441450 451+ // A chart a metric, made here since the page doesn't know their names.452+ const box = root.querySelector<HTMLElement>('#ins-metrics');453+ const grid = box?.querySelector('.charts');454+ if (box && grid && d.metrics?.length) {455+ box.hidden = false;456+ for (const [name, unit, pts] of d.metrics) {457+ const fig = el('figure', 'panel');458+ fig.dataset.chart = `metric:${name}`;459+ const head = el('figcaption', 'panel-head');460+ head.append(el('h3', 'mono', name), el('span', 'grow', unit ? `in ${unit}, ` : ''));461+ head.lastElementChild!.append(plural(pts.length, 'commit'));462+ const det = el('details', 'table');463+ det.appendChild(el('summary', '', 'table'));464+ fig.append(head, el('div', 'plot'), det);465+ grid.appendChild(fig);466+ const fmt = (v: number): string => num(v) + (unit ? ` ${unit}` : '');467+ chart(`metric:${name}`, (f, plot) => {468+ dots(plot, pts.map(([t, v, sha, subj]) => ({ t, v, head: `${sha.slice(0, 7)} ${subj}`, rows: [[fmt(v), name], [when(t), 'committed']] })), num, true);469+ table(f, ['commit', 'subject', 'committed', name], () => [...pts].reverse().map(([t, v, sha, subj]) => [sha.slice(0, 7), subj, when(t), fmt(v)]));470+ });471+ }472+ }473+442474 const repos = root.querySelector('#ins-repos tbody');443475 if (repos && d.repos) {444476 for (const [slug, pushes, builds, buildMs, landed, ci, pass, last] of d.repos) {@@ -0,0 +1,51 @@
1+# Metrics: .forge/ci writes "name number [unit]" lines to $FORGE_METRICS,2+# forge attest signs them into the statement, and the lake keeps a row a3+# metric in ci_metrics. A repo's insights.json charts each one by commit on4+# its default branch, matched by tree, so a commit reworded after it was5+# attested keeps its values.6+. "$(dirname "$0")/../lib.sh"7+user dami8+repo dami/greet9+clone dami dami/greet "$T/w/d"10+mkdir -p "$T/w/d/.forge"11+cat >"$T/w/d/.forge/ci" <<'EOF'12+#!/bin/sh13+n=$(git rev-list --count HEAD)14+{15+ echo "files $n"16+ echo "bundle_kb 41.5 KB"17+ echo "Not A Metric"18+ echo "files 9"19+} >"$FORGE_METRICS"20+EOF21+chmod +x "$T/w/d/.forge/ci"22+at "$T/w/d" commit "ci reports metrics"23+ok at "$T/w/d" "$FORGE" attest24+has "$T_LOG" 'FORGE_METRICS: line 3 is not "name number \[unit\]"; left out'25+has "$T_LOG" 'FORGE_METRICS: files twice; the first is kept'26+note=$(git -C "$T/w/d" notes --ref=ci show HEAD)27+case $note in *"metric files 128+metric bundle_kb 41.5 KB29+log sha256:"*) ;; *) fail "the statement lacks its metric lines: $note" ;; esac30+# Reworded after the attestation: same tree, another commit.31+git -C "$T/w/d" commit -q --amend -m "ci reports metrics, reworded"32+ok push_as dami "$T/w/d" origin main refs/notes/ci33+ok forge lake34+eq "$(lq "SELECT string_agg(name || '=' || value || coalesce(unit, ''), ';' ORDER BY name) FROM ci_metrics")" 'bundle_kb=41.5KB;files=1.0'35+eq "$(lq "SELECT count(*) FROM ci_metrics m JOIN ci USING (id) WHERE ci.verified AND m.tree = ci.tree")" 236+eq "$(lq "SELECT count(*) FROM commits WHERE tree IS NOT NULL")" 137+# A second commit: the chart has two points.38+at "$T/w/d" commit "another file"39+ok at "$T/w/d" "$FORGE" attest40+ok push_as dami "$T/w/d" origin main refs/notes/ci41+build dami/greet42+ok forge lake43+j=$FORGE_WWW/dami/greet/insights/insights.json44+has "$j" '"metrics":\[\["bundle_kb","KB",\[\[[0-9]+,41.5,"[0-9a-f]{40}","[a-z ,]+"\],\[[0-9]+,41.5,'45+has "$j" '\["files",null,\[(\[[0-9]+,[12],"[0-9a-f]{40}","[a-z ,]+"\],?){2}\]\]'46+has "$j" '\[[0-9]+,1,"[0-9a-f]{40}","ci reports metrics, reworded"\]'47+has "$j" '\[[0-9]+,2,"[0-9a-f]{40}","another file"\]'48+[ "$(head -c 4 "$FORGE_WWW/dami/greet/insights/ci_metrics.parquet")" = PAR1 ] || fail "ci_metrics.parquet isn't Parquet"49+# The forge's insights have no metrics chart: names mean something per repo.50+lacks "$FORGE_WWW/insights/insights.json" '"metrics"'51+has "$FORGE_WWW/dami/greet/insights/index.html" 'id="ins-metrics" hidden'