clone

push a directory of html from the terminal and get a four-character url back. a deno cli on cloudflare

main
nzip / ARCHITECTURE.md
ARCHITECTURE.md 12.9K rendered
1# nzip architecture23nzip is a Deno CLI backed by a Cloudflare Worker, R2 object storage, and D14relational state. The CLI and Worker share one runtime-neutral package so both5sides canonicalize and hash manifests identically.67## Components89```mermaid10flowchart TB11 CLI["Deno CLI"] -- "bearer API at control host" --> W["Worker"]12 SHORT(["GET control/2a3f"]) -- "308" --> SITE["https://2a3f.site-domain/"]13 V(["visitor"]) -- "GET site hostname" --> C{"browser cache"}14 C -- "fresh" --> RESP["public response"]15 C -- "miss or revalidate" --> W16 W --> RESP17 W --> R2[("R2 blobs")]18 W --> D1[("D1 state")]19```2021- `shared/` defines manifests, hashing, target parsing, limits, media types, and22 API contracts.23- `cli/` bundles local files, uploads missing blobs, commits sites, and manages24 owner operations.25- `worker/` serves public sites and implements authenticated management,26 security telemetry, notifications, cleanup, and delivery retries.2728## Push protocol2930A push is a stateless three-step exchange; the manifest is the state carried31between steps:32331. `POST /api/push/prepare` sends the manifest and returns missing blob hashes.342. `PUT /api/blob/{sha256}` uploads only missing blobs, with bounded35 concurrency. The Worker re-hashes each body and rejects mismatches or36 oversized blobs.373. `POST /api/push/commit` verifies every blob, resolves or allocates the38 address, applies TTL and password policy, repoints the site, and appends39 retained history atomically. TTL resolution is explicit request, existing40 site, vault default, then the 14-day global fallback; the response identifies41 the winning layer.4243Content-addressed blobs deduplicate across every site. The last ten pushes per44site remain addressable for revert operations.4546## App-origin reservation4748`POST /api/apps` reserves a named address before content exists. The reservation49is separate from the live `sites` row and participates in allocation forever.50This gives lofi a stable origin before `credentialOrigins`, WebAuthn RP IDs,51service-worker scope, OPFS, or IndexedDB state are configured. A reservation52without content serves a small non-indexed placeholder. Deploying creates or53updates the ordinary site row at the reserved address; deletion and expiry may54remove that row but never the reservation tombstone.5556The CLI's tracked `nzip.app.json` stores only framework, target, address, origin,57and build configuration. The owner bearer token remains solely in the mode-060058CLI config. `app deploy` runs lofi's Deno build at the origin root, validates the59build identity and PWA artifacts, and sends the generated CSP with the commit so60the Worker can mirror it as an HTTP response header. Sync-node configuration is61application data and remains orthogonal to hosting.6263## Serving and caching6465The configured control origin owns authenticated APIs, notification enrollment,66and legacy short paths. A legacy `/<address>` request receives a permanent `308`67to `https://<address>.<site-domain>/`; the control origin never serves artifact68bytes. The wildcard route accepts only exact four-character lowercase69hexadecimal hostnames. Management and notification routes are not dispatched on70artifact hosts.7172Each artifact hostname is a distinct browser origin. Origin-scoped storage and73service-worker control therefore stay within one site, and nzip's `__Host-`74unlock cookie is host-only. Artifact responses also disable `document.domain`75relaxation. Until `SITE_DOMAIN` is registered in the Public Suffix List, an76artifact can still set a parent-domain cookie or deliberately choose that parent77as its WebAuthn RP ID. Applications that require isolation must use host-only78cookies and their exact hostname as the RP ID.7980Serving normally requires one D1 read for site state and two R2 reads for the81manifest and file. Public responses use `ETag` revalidation and a 60-second82browser cache entry. Every Worker response also sends83`Cloudflare-CDN-Cache-Control: no-store`: the exact custom domain and wildcard84route currently share an unsafe hostname-agnostic edge cache key, so an edge85entry must never be replayed on a sibling site origin.8687Public responses retain site cache tags and purge hooks for a future host-safe88edge-cache design. Protected responses and errors are never cached.8990Site roots always use `/`. Directory paths redirect to the trailing-slash form91so relative asset paths resolve correctly, and explicit `index.html` paths92redirect to their containing directory. For migration compatibility, an93unresolved `/<address>/...` asset path on that address's hostname falls back to94the corresponding root-relative asset. A real file at the prefixed path always95wins.9697## Site feedback9899Feedback turns a password-protected page into a review surface without giving the page any100server code. It is opt-in per site (`feedback` on push or policy) and refused for unprotected101sites, so it never becomes an open write endpoint on a public URL.102103```mermaid104sequenceDiagram105 participant A as Agent (CLI/MCP)106 participant W as Worker107 participant P as Phone108 participant V as Reviewer's browser109 A->>W: push with feedback=true (password-protected)110 A->>W: POST /api/sites/{t}/feedback/request111 W->>P: "Feedback requested" notification (link only)112 V->>W: GET site, unlock, load /__feedback.js113 V->>W: POST /__feedback {items, submit?}114 W->>A: one-shot callback POST on submit (no note text)115 A->>W: GET /api/sites/{t}/feedback116```117118- **Visitor routes.** `/__feedback` and `/__feedback.js` are reserved on site hostnames and run119 after the password gate, so only someone holding the unlock cookie can read or write notes. Writes120 must be same-origin JSON: the `Origin` and `Sec-Fetch-Site` headers must match and the content121 type must be `application/json`, on top of the `SameSite=Lax` host-only cookie. `RL_FEEDBACK`122 throttles writes per client and site. Responses are `no-store`.123- **Data model.** Notes live in `site_feedback`, keyed by site and a page-chosen item key, capped at124 200 items of 4,000 characters per site. Each note records the push sequence it was written125 against. Rows cascade with the site, so a reallocated address never inherits old notes.126- **Review rounds.** An owner request stamps `feedback_requested_at`, clears127 `feedback_submitted_at`, and queues a notification through the normal outbox with a link-only128 body. A request may arm a one-shot HTTPS `callbackUrl` (for example a webhook bound to the129 requesting agent); the submit that claims it, inside the same transaction, POSTs the site,130 timestamps, and note count, never note text, with one retry, and clears it. The URL is never131 returned or logged. A visitor submit stamps `feedback_submitted_at`. Notes saved after the request belong to the132 current round; the helper script shows older notes as placeholders.133- **Trust.** Note text is visitor input. Owner tooling returns it as data; agents must not treat it134 as instructions.135136## Storage model137138Core publishing state:139140| table | purpose |141| ------------------ | --------------------------------------------------------------------- |142| `vaults` | named slots, descriptions, and optional default TTL |143| `vault_defaults` | temporary/permanent lifecycle selection |144| `app_reservations` | permanent address and alias tombstones for application origins |145| `sites` | live address, manifest, expiry, password, app CSP, and feedback state |146| `pushes` | retained per-site manifest history |147| `site_feedback` | opt-in visitor notes per site and item, cascading with the site |148149Security and notification state:150151| table | purpose |152| ----------------------------- | ---------------------------------------------------- |153| `security_probes` | deduplicated scanner/address observations |154| `security_signals` | rate-limit and suspicious-hit confirmations |155| `security_incidents` | alert lifecycle, severity, and suppression state |156| `security_notifications` | durable alert email outbox |157| `notification_pairing_window` | owner-controlled enrollment deadline |158| `notification_devices` | pending, approved, active, and revoked device claims |159| `notification_events` | bounded notification payloads and click targets |160| `notification_deliveries` | leased delivery attempts, retries, and outcomes |161162An R2 object is deleted only when no live site or retained history entry163references it and it is older than 24 hours. This prevents concurrent garbage164collection from deleting an in-flight push.165166## Notification flow167168Notification setup separates enrollment, owner approval, and subscription169attachment. Delivery is also asynchronous: the owner request persists an event170and per-device attempts before background work contacts a Web Push provider.171172```mermaid173flowchart TB174 OWNER["owner: notify pair"] --> WINDOW[("10-minute window")]175 PHONE["phone browser"] -- "enroll" --> PENDING[("pending device")]176 WINDOW --> PENDING177 PENDING --> CODE["display pairing code"]178 CODE --> APPROVE["owner approves code"]179 APPROVE --> APPROVED[("approved claim")]180 APPROVED -- "activate" --> PWA["installed PWA"]181 PWA -- "attach subscription" --> ACTIVE[("active device")]182```183184```mermaid185flowchart TB186 OWNER["owner CLI"] -- "notify send" --> API["notification API"]187 API --> EVENT[("event + deliveries")]188 ACTIVE[("active devices")] --> DRAIN["scheduled drain"]189 EVENT --> DRAIN190 DRAIN --> PUSH["Web Push provider"]191 PUSH --> SW["service worker"]192 SW --> NOTICE["system notification"]193 NOTICE -- "tap" --> TARGET["validated click target"]194 TARGET --> SITE["root or pinned site"]195```196197Pairing codes cannot approve themselves; the approval path always requires the198owner bearer token. Subscription endpoints must match the configured199provider-origin allowlist. Notification click targets are resolved at tap time200and open only the deployment root or the unchanged site manifest recorded with201the event.202203## Background work204205The five-minute schedule evaluates enumeration windows and drains notification206deliveries. The daily schedule expires sites, garbage-collects content, prunes207telemetry and notification state, and sends the security activity digest.208209Durable outboxes separate request acceptance from email and Web Push delivery.210Delivery workers use leases, bounded retry schedules, and terminal outcomes so211overlapping cron executions remain safe.212213## Observability214215```mermaid216flowchart TB217 RESP["Worker response"] --> CLASSIFY["classify request"]218 CLASSIFY --> SAMPLE{"scanner sample"}219 SAMPLE -- "selected" --> LOGS["Workers Logs"]220 SAMPLE -- "not selected" --> DROP["no event"]221 PROBES[("D1 probe windows")] --> CRON["5-minute evaluator"]222 CRON --> WINDOW["incident event"]223 CRON --> OUTBOX[("alert outbox")]224 LOGS --> DASH["Observability"]225 WINDOW --> DASH226 PROBES --> METRICS["D1 row metrics"]227```228229Workers Logs provide sampled request-level context. D1 probe windows remain230bounded but unsampled for automatic incident decisions. D1 row metrics are the231direct storage-budget signal.232233## Free-tier design target234235nzip is designed so a small personal deployment can stay inside Cloudflare's236included usage. This is a design target, not a guarantee: quotas are237account-wide and Cloudflare may change them.238239| resource | bounding strategy | usage reference |240| ------------- | ------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------- |241| Workers | short path; every browser miss executes the Worker | [Workers limits](https://developers.cloudflare.com/workers/platform/limits/) |242| Workers Logs | invocation logs off; deterministic identity sample | [Workers Logs pricing](https://developers.cloudflare.com/workers/observability/logs/workers-logs/#pricing) |243| D1 | deduplicated windows, write caps, and seven-day pruning | [D1 pricing](https://developers.cloudflare.com/d1/platform/pricing/) |244| R2 | content deduplication, history cap, and daily GC | [R2 pricing](https://developers.cloudflare.com/r2/pricing/) |245| Alert email | one verified Email Routing destination | [Email Service pricing](https://developers.cloudflare.com/email-service/platform/pricing/) |246247Probe persistence happens after the public response and catches its own errors.248Exhausting the telemetry budget reduces detection coverage until quotas reset249but does not stop ordinary site serving.