Expand the Worker diagram with cache, rate limits, logs, scheduled incident evaluation, D1 alert state, and verified-destination email. Document alert signals, operational monitoring, free-tier budgets, quota behavior, and setup ordering.
- commit
- a4aadccb0874a30b35ee681fb396aa619ff497ea
- parent
- 66cfb8308a8cee55391cece624b432809cde01de
- author
- Dami <dami.etoile@gmail.com>
| M | README.md | 159 | +++++++++++++++++++++--------- |
| M | worker/ | 30 | +++++- |
@@ -2,8 +2,8 @@
22 33 **Push a directory of HTML from the terminal. Get a four-character URL back.**44 5-_Deno CLI → Cloudflare Worker → R2 + D1. Zero-cost static site hosting with6-optional expiration and/or password protection._5+_Deno CLI → Cloudflare Worker → R2 + D1. Free-tier-oriented static site hosting6+with optional expiration, password protection, and enumeration alerts._77 88 [](https://jsr.io/@nzip/cli)99 @@ -108,11 +108,21 @@ alias.
108108 ## Architecture109109 110110 ```mermaid111-flowchart LR112- CLI["nzip<br/><i>Deno CLI</i>"] -- "https + bearer" --> W["Worker<br/><i>self-hosted</i>"]113- W --> R2[("R2<br/>blob/{sha256}<br/>manifest/{sha256}")]114- W --> D1[("D1<br/>vaults · sites · pushes")]115- V(["visitor"]) -- "GET /2a3f" --> W111+flowchart TB112+ CLI["nzip<br/><i>Deno CLI</i>"] -- "push API<br/>bearer token" --> W113+ V(["visitor"]) -- "GET /2a3f" --> C{"Workers Cache<br/>60s public TTL"}114+ C -- "HIT" --> V115+ C -- "MISS / BYPASS" --> W["nzip Worker<br/><i>serve · API · security</i>"]116+ W -- "cacheable response<br/>tagged by site" --> C117+118+ W --> RL["Rate-limit bindings<br/>enumeration · unlock · telemetry"]119+ W --> R2[("R2<br/>blobs · manifests")]120+ W --> D1[("D1<br/>site metadata<br/>probe windows · incidents")]121+ W -. "sampled structured events" .-> LOGS["Workers Logs<br/>security.request"]122+123+ CRON(["Cron triggers<br/>5 min · daily"]) --> W124+ W -- "incident + digest" --> EMAIL["Email Routing binding<br/>verified destination"]125+ EMAIL --> INBOX(["operator inbox"])116126 ```117127 118128 A push is a stateless three-step protocol, and the manifest itself is the state:@@ -126,10 +136,13 @@ A push is a stateless three-step protocol, and the manifest itself is the state:
126136 TTL/password policy, repointing the site, and appending history127137 128138 Serving is one D1 read (address → manifest, expiry, password) and two R2 reads,129-with `ETag` revalidation and a 60-second tiered Workers Cache. Cache hits skip130-Worker execution and the D1/R2 reads entirely. Public responses are tagged per131-site and purged when a push, revert, deletion, TTL, or password policy changes;132-protected responses and errors are never cached. Metadata lives in three tables:139+with `ETag` revalidation and a 60-second Workers Cache. Cloudflare checks this140+cache before invoking the Worker, so a hit skips Worker execution and the D1/R2141+reads. Public responses are tagged per site and purged when a push, revert,142+deletion, TTL, or password policy changes; protected responses and errors are143+never cached.144+145+Content metadata lives in three tables:133146 134147 | table | keys |135148 | -------- | -------------------------------------------------------------------------------- |@@ -137,16 +150,94 @@ protected responses and errors are never cached. Metadata lives in three tables:
137150 | `sites` | address (0–65535), vault, alias, current manifest, expiry, password hash/version |138151 | `pushes` | per-site history (seq, manifest, note), capped at 10, powers `revert` |139152 153+Security alert state is deliberately separate:154+155+| table | purpose |156+| -------------------- | ------------------------------------------------------------------------ |157+| `security_probes` | deduplicated five-minute scanner/address observations; retained 7 days |158+| `security_signals` | rate-limit confirmations, deduplicated by scanner and window |159+| `security_incidents` | open/closed state, severity, suppression timestamps, and incident totals |160+140161 **GC safety rule:** an R2 object is deleted only if no live site or retained141162 history entry references it _and_ it's older than 24 hours, so an in-flight push142163 can never lose objects to a concurrent sweep.143164 165+## Security and observability166+167+The public address space is intentionally small and human-friendly, so nzip168+treats enumeration as something to observe and rate-limit rather than pretending169+four hex characters are a secret. The two telemetry paths serve different jobs:170+171+| path | signal and retention |172+| ---------------------------- | ------------------------------------------------------------------------------------ |173+| Workers Logs | structured `security.request` events for a deterministic 1% scanner-identity sample |174+| D1 five-minute probe windows | bounded, unsampled bare-address observations used for automatic incident decisions |175+176+Log events classify bare addresses, missing address assets, unlock attempts, bad177+API requests, and other scanner-shaped paths. They include the response result,178+derived vault/site slots (for example, `/0123` → vault `0x0`, site `0x123`),179+country, colo, ASN, and an HMAC-derived `scanner_id`. Raw client IPs are neither180+logged nor stored. Automatic invocation logs are disabled, successful site181+assets are omitted, paths are length-bounded, and blocked floods are sampled a182+second time to control log volume.183+184+The five-minute evaluator opens or escalates an enumeration incident on these185+signals:186+187+| severity | signal |188+| --------- | -------------------------------------------------------------------------------------- |189+| warning | one scanner tries 20 distinct addresses in 5 minutes |190+| warning | 128 addresses from 10 scanners in 5 minutes, with at least 90% misses |191+| confirmed | an enumeration request reaches the 429 limiter |192+| confirmed | a live hit after 8-address/sequence evidence, or any live hit during an open incident |193+194+Duplicate email is suppressed unless severity increases, volume doubles, a live195+site is hit for the first time, or a new vault is targeted after 30 minutes.196+Active incidents summarize at most hourly and resolve after three quiet windows197+(15 minutes). A daily activity digest is sent only when probes occurred. Alerts198+use a Worker email binding restricted to one verified destination; the199+owner-authenticated `/api/security/test-alert` endpoint validates delivery.200+201+Inspect sampled events under **Workers & Pages → nzip → Observability**, filtering202+on `event = "security.request"`. Each event's `sample_rate` records its sampling203+factor for aggregate estimates. Watch D1 row metrics as the direct free-tier204+budget signal for alert storage, and Worker request/log counts for the overall205+service budget.206+207+## Free-tier design target208+209+nzip is designed so a small personal deployment can stay inside Cloudflare's210+included free usage without disabling security or observability. This is a211+design target, not a promise: account-wide traffic and storage count toward the212+same quotas, Cloudflare can change its limits, and a sufficiently distributed213+attack can exhaust a daily quota.214+215+| resource | how nzip keeps usage bounded | current included usage to watch |216+| -------- | ---------------------------- | ------------------------------- |217+| Workers | short request path; cache checked before invocation | [100,000 requests/day; 10 ms CPU/invocation](https://developers.cloudflare.com/workers/platform/limits/) |218+| Workers Cache | public content only, 60-second TTL, tag purge on mutation | [no separate request allowance](https://developers.cloudflare.com/workers/platform/pricing/); cached Worker requests still count toward Workers usage |219+| Workers Logs | invocation logs off; deterministic 1% identity sample; successful assets omitted | [200,000 log events/day with 3-day retention](https://developers.cloudflare.com/workers/observability/logs/workers-logs/#pricing) |220+| D1 | probe rows deduplicated by scanner/address/window, capped per scanner/location, pruned after 7 days | [5M rows read/day, 100K written/day, 5 GB](https://developers.cloudflare.com/d1/platform/pricing/) |221+| R2 Standard | content-addressed deduplication, retained-history cap, daily GC | [10 GB-month, 1M Class A and 10M Class B operations/month; free egress](https://developers.cloudflare.com/r2/pricing/#free-tier) |222+| Alert email | restricted to a verified Email Routing destination; no arbitrary-recipient sending | [verified-destination sends are free on all plans](https://developers.cloudflare.com/email-service/platform/pricing/) |223+224+On the Workers Free plan, exceeding included daily usage does not create an225+overage bill. D1 queries fail until the quota resets; Worker request-limit226+behavior follows the route's configured fail-open/fail-closed mode. Probe227+persistence runs after the response and catches its own errors, so an exhausted228+alert-storage budget does not stop normal site serving; it does reduce detection229+coverage until the quota resets. Cache hits reduce execution and storage reads,230+but they do not remove the request from the Workers request quota. The telemetry231+rate limiter is best-effort and local to a Cloudflare location; deduplication and232+seven-day pruning provide the durable bounds, while a distributed attack can233+still consume the daily D1 allowance.234+144235 ## Repo layout145236 146237 ```text147238 shared/ manifest canonicalization, hashing, addressing (imported by both sides)148239 cli/ the nzip command (Deno, no framework), published as jsr:@nzip/cli149-worker/ Cloudflare Worker: serving + API + GC cron (wrangler)240+worker/ Cloudflare Worker: serving, cache, API, security alerts, and GC (wrangler)150241 ```151242 152243 `shared/` is the contract: canonical JSON serialization lives in exactly one@@ -199,6 +290,7 @@ npx wrangler r2 bucket create nzip-content
199290 npx wrangler d1 create nzip # paste id into wrangler.local.jsonc200291 npx wrangler d1 execute nzip --remote --file schema.sql201292 openssl rand -hex 32 | npx wrangler secret put NZIP_TOKEN --config wrangler.local.jsonc293+# configure and verify the alert destination; see worker/setup.md202294 npx wrangler deploy --config wrangler.local.jsonc203295 nzip auth --server https://share.example.com # use your actual PUBLIC_BASE204296 ```@@ -230,39 +322,10 @@ npx wrangler dev # local R2/D1; token in .dev.v
230322 nzip auth --server http://localhost:8787 --token dev-token-local-only231323 ```232324 233-Test the GC cron: `npx wrangler dev --test-scheduled`, then234-`curl "http://localhost:8787/__scheduled?cron=0+4+*+*+*"`.235-236-### Security observability237-238-Workers Logs are enabled by default for structured scan telemetry, with239-automatic invocation logs disabled to keep volume and cost bounded. The Worker240-retains complete request sequences for a deterministic 1% sample of scanner241-identities. Events are emitted as `security.request` and include the request242-class, bounded path, response result, derived vault/site slots for four-hex243-addresses, country, Cloudflare colo, ASN, and an HMAC-derived `scanner_id`; raw244-client IPs are never logged.245-246-Inspect these events under **Workers & Pages → nzip → Observability**, filtering247-on `event = "security.request"`. The `sample_rate` field records the sampling248-factor for aggregate estimates. Normal successful asset requests are omitted,249-while bare address probes, unlock attempts, bad API requests, invalid paths, and250-missing address assets are eligible.251-252-Rejected `429` floods receive a second 1% per-request sample after the identity253-sample. This keeps the allowed probe sequence intact while preventing a single254-blocked client from producing unbounded log volume; each event's `sample_rate`255-reflects the effective rate.256-257-### Enumeration email alerts258-259-The Worker also evaluates bounded, unsampled bare-address observations every260-five minutes and sends incident email through the free Email Routing binding.261-The primary warning is 20 distinct addresses from one scanner in five minutes; a262-distributed 128-address sweep from 10 scanners also warns when at least 90%263-miss. A limiter rejection or a suspicious live-address hit promotes the incident264-to confirmed. Re-alerting is immediate when severity increases or volume265-doubles, otherwise limited to an hourly summary, and an incident resolves after266-15 quiet minutes. An activity digest is also sent daily, but only when probes267-occurred. See [`worker/setup.md`](worker/setup.md#free-security-alert-email) for268-setup and delivery validation.325+Test scheduled handlers with `npx wrangler dev --test-scheduled`, then invoke326+either cron locally:327+328+```sh329+curl "http://localhost:8787/__scheduled?cron=*/5+*+*+*+*" # alert evaluation330+curl "http://localhost:8787/__scheduled?cron=0+4+*+*+*" # GC + digest331+```@@ -2,6 +2,12 @@
22 33 Everything here happens once. Day-to-day content pushes never touch wrangler.44 5+The default architecture targets the Workers Free plan. It requires no paid6+Email Sending subscription: alerts use Email Routing to one verified destination.7+Free-tier limits are account-wide, so review the8+[budget notes](../README.md#free-tier-design-target) before exposing a busy or9+multi-tenant deployment.10+511 ```sh612 cd worker713 @@ -27,7 +33,12 @@ npx wrangler d1 execute nzip --remote --file schema.sql
2733 openssl rand -hex 32 # keep this — it's your CLI token2834 npx wrangler secret put NZIP_TOKEN --config wrangler.local.jsonc2935 30-# 6. first deploy36+# 6. alert destination (click the verification link before deploying)37+npx wrangler email routing enable example.com38+npx wrangler email routing addresses create operator@example.com39+# Edit send_email and ALERT_EMAIL_* in wrangler.local.jsonc as shown below.40+41+# 7. first deploy3142 npx wrangler deploy --config wrangler.local.jsonc3243 ```3344 @@ -67,9 +78,20 @@ at least 90% misses. A rate-limit hit or a suspicious live-address hit confirms
6778 the incident. Duplicate email is suppressed unless severity increases or volume6879 doubles, active incidents summarize hourly, and three quiet windows (15 minutes)6980 resolve the incident. Probe rows are deduplicated by scanner/address, capped at70-30 per scanner per minute, contain no raw IP, and are pruned after seven days. A71-daily activity digest is sent only when probes occurred in the preceding 2472-hours.81+30 per scanner per minute in each Cloudflare location, contain no raw IP, and are82+pruned after seven days. A daily activity digest is sent only when probes83+occurred in the preceding 24 hours.84+85+### Operational checks after deployment86+87+- **Workers Metrics:** request volume and errors; cache hits reduce execution and88+ storage reads but still count as Worker requests.89+- **Workers Observability:** filter `event = "security.request"`; Free retains90+ Workers Logs for three days.91+- **D1 Metrics → Row Metrics:** rows written is the main enumeration-telemetry92+ budget; Free currently includes 100,000 written rows per day.93+- **Email Routing:** the destination must remain verified. A successful test94+ endpoint response means Cloudflare accepted the message for delivery.7395 7496 `routes[0].pattern` and `vars.PUBLIC_BASE` are required user-provided values.7597 `PUBLIC_BASE` is the origin printed in share URLs and the server URL passed to