Everything Cloudflare-specific at the edge now has a stand-in when the binding is absent: the lease door keeps token buckets in memory, the client's address comes from the header CLIENT_IP_HEADER names or not at all, and the custom hostname map is optional. A proxy that issues certificates on demand can ask /.well-known/bindws/hostname whether a hostname is ours. The helpers live in edge.ts, since workerd refuses a constant exported from the entry module. wrangler.celld.jsonc carries the config with only the keys celld accepts; scripts/check-celld.mjs, in npm run typecheck, keeps it in step with wrangler.jsonc. CI runs the conformance suite, files included, against celld dev on every push. docs/16 walks through MinIO, deploy, a node, Caddy with on-demand TLS, custom domains, and the promise: a notice on that page and in the README before any host is dropped. Verified here with celld 0.4.0: celld dev and a fleet node on MinIO both pass the suite, the R2 binding serves media from the bucket, a KV-mapped hostname routes to its relay.
- commit
- ce48db48534dd2f84ed9c04edef7c8a99bca0304
- parent
- e3185729a817b4645d6d0b727444a4066e187fe4
- author
- Dami <dami.etoile@gmail.com>
| M | .github/ | 33 | +++++++ |
| M | .gitignore | 1 | + |
| M | README.md | 3 | +- |
| M | docs/ | 2 | +- |
| M | docs/ | 2 | +- |
| M | docs/ | 10 | ++- |
| M | docs/ | 1 | + |
| A | docs/ | 146 | ++++++++++++++++++++++++++++++ |
| M | package.json | 2 | +- |
| A | scripts/ | 71 | +++++++++++++++ |
| M | src/ | 13 | ++- |
| A | src/ | 83 | +++++++++++++++++ |
| M | src/ | 36 | ++----- |
| M | src/ | 18 | ++- |
| A | test/ | 59 | ++++++++++++ |
| A | test/ | 62 | +++++++++++++ |
| A | wrangler.celld.jsonc | 43 | +++++++++ |
@@ -15,3 +15,36 @@ jobs:
1515 - run: npm ci1616 - run: npm run typecheck1717 - run: npm test18+ # The same Worker on celld (docs/16-hosting-without-cloudflare.md): the19+ # black-box suite, files included, against `celld dev`. A red run here20+ # means the host without Cloudflare broke; fix it or say so in the doc21+ # before dropping it.22+ celld:23+ runs-on: ubuntu-latest24+ env:25+ CELLD_VERSION: v0.4.026+ steps:27+ - uses: actions/checkout@v728+ - uses: actions/setup-node@v729+ with:30+ node-version: 2231+ cache: npm32+ - run: npm ci33+ - name: Install celld34+ run: |35+ curl -fsSL https://celld.dev/install.sh -o install.sh36+ sh install.sh37+ echo "$HOME/.local/bin" >> "$GITHUB_PATH"38+ echo "$PWD/node_modules/.bin" >> "$GITHUB_PATH"39+ - name: Start celld dev40+ run: |41+ celld dev wrangler.celld.jsonc --port 9876 > celld.log 2>&1 &42+ for i in $(seq 1 60); do43+ curl -sf -o /dev/null http://127.0.0.1:9876/ && break44+ sleep 145+ done46+ curl -sf -H 'accept: application/nostr+json' http://127.0.0.1:9876/47+ - run: CLAIM=1 RELAY_URL=ws://127.0.0.1:9876 npm run test:conformance48+ - name: celld log49+ if: failure()50+ run: cat celld.log@@ -26,7 +26,7 @@ That answers with a relay at a memorable name, open to anyone for 14 days. Claim
2626 2727 <p align="center"><img src="docs/img/console.png" alt="The console's People tab" width="720"></p>2828 29-It runs on Cloudflare Workers with one Durable Object per name, SQLite inside it and R2 beside it.29+It runs on Cloudflare Workers with one Durable Object per name, SQLite inside it and R2 beside it. The same code runs without Cloudflare, on [celld](https://celld.dev) with an S3 bucket of your own; that path is checked in CI and [documented](docs/16-hosting-without-cloudflare.md).3030 3131 ## Docs3232 @@ -48,6 +48,7 @@ It runs on Cloudflare Workers with one Durable Object per name, SQLite inside it
4848 4949 - [Architecture](docs/10-architecture.md): routing, the object, storage, alarm, jobs, identity, groups.5050 - [Hosting bind.ws](docs/11-hosting-bindws.md): run your own on your own domain, and watch it through logs, traces and metrics.51+- [Hosting without Cloudflare](docs/16-hosting-without-cloudflare.md): the same Worker on celld, with your own bucket and proxy, and what "supported" promises.5152 - [Develop and extend](docs/12-develop-extend.md): layout, tests, adding methods and NIPs, the console.5253 - [Costs and margins](docs/15-costs-and-margins.md): what a relay costs the host, what fuel charges, the weekly check against the bitcoin price.5354 @@ -18,7 +18,7 @@ client ──wss/https──▶ worker ──host→object──▶ durable obje
1818 1919 ## Routing and states2020 21-`src/index.ts` maps the hostname to a name. `<name>.<domain>` goes to the object `getByName(name)`. The apex serves the landing page and `POST /lease`. Reserved names and invalid names redirect to the apex. Any other hostname is looked up in the `HOSTS` KV namespace, which maps a custom hostname to a relay name, with a one-minute cache per isolate; a miss is cached too. In `wrangler dev`, `<name>.localhost` plays the same role. The worker stamps every forwarded request with `x-relay-name` and with `x-relay-ip` from Cloudflare's `cf-connecting-ip`, overwriting anything a client sent.21+`src/index.ts` maps the hostname to a name. `<name>.<domain>` goes to the object `getByName(name)`. The apex serves the landing page and `POST /lease`. Reserved names and invalid names redirect to the apex. Any other hostname is looked up in the `HOSTS` KV namespace, which maps a custom hostname to a relay name, with a one-minute cache per isolate; a miss is cached too. In `wrangler dev`, `<name>.localhost` plays the same role. The worker stamps every forwarded request with `x-relay-name` and with `x-relay-ip`, overwriting anything a client sent: from Cloudflare's `cf-connecting-ip`, or, on a host without Cloudflare, from the header `CLIENT_IP_HEADER` names, which the operator's proxy sets. What the edge provides, and what stands in when there are no Cloudflare bindings, is `edge.ts`; the whole Worker also runs on celld, [Hosting without Cloudflare](16-hosting-without-cloudflare.md).2222 2323 A relay is in one of three states:2424 @@ -5,7 +5,7 @@ audience: developer
55 66 # Hosting bind.ws77 8-How to run your own copy on your own domain. You need a Cloudflare account with Workers Paid, a zone for the domain and `wrangler` logged in.8+How to run your own copy on your own domain. You need a Cloudflare account with Workers Paid, a zone for the domain and `wrangler` logged in. To run it on your own machines instead, with no Cloudflare in the path, see [Hosting without Cloudflare](16-hosting-without-cloudflare.md).99 1010 ## Configure1111 @@ -10,6 +10,7 @@ audience: developer
1010 ```1111 src/1212 index.ts worker entry: hostname → object, POST /lease13+ edge.ts what the edge provides, with and without Cloudflare: the client's address, the lease limit, the hostname map1314 names.ts valid names, reserved names, lease names1415 pull.ts copy another relay in: NIP-77 as the initiator, one round per connection1516 jobs.ts the alarm's job list: pulls, backfills, rebroadcasts, once or standing@@ -43,10 +44,13 @@ src/
4344 ui.ts the shared look4445 test/4546 *.test.ts Durable Object tests in workerd, one file per feature46- conformance/ black-box suite for any relay URL47+ conformance/ black-box suite for any relay URL, files included4748 scripts/4849 dev-signer.mjs seed.mjs junk.mjs zaptest.mjs shot.mjs4950 build-signer.mjs signer-entry.js bundle nostr-tools for the console's remote signing51+ check-celld.mjs wrangler.celld.jsonc stays in step with wrangler.jsonc52+wrangler.jsonc the Worker on Cloudflare53+wrangler.celld.jsonc the same Worker on celld (docs/16)5054 ```5155 5256 ## Run the tests@@ -63,7 +67,9 @@ The conformance suite needs a claimed relay. Against a dev server:
6367 CLAIM=1 RELAY_URL=ws://dev.localhost:8787 npm run test:conformance6468 ```6569 66-CI runs typecheck and the object tests on every push. Work lands on `main` directly, in small commits that each typecheck on their own; a red check on `main` is fixed forward with the next commit.70+CI runs typecheck and the object tests on every push, and, in a second job, the conformance suite against the Worker on `celld dev` ([Hosting without Cloudflare](16-hosting-without-cloudflare.md)). Work lands on `main` directly, in small commits that each typecheck on their own; a red check on `main` is fixed forward with the next commit.71+72+The entry module `src/index.ts` exports the handler and the object and nothing else: workerd refuses an exported constant there, and the test pool does not, so a helper that tests import lives in its own module.6773 6874 ## Add a management method6975 @@ -16,6 +16,7 @@ Any request with `Accept: application/nostr+json` answers the NIP-11 document, w
1616 | `/` | GET | none | the landing page | 200 |1717 | `/lease` | POST | NIP-98 optional | `{ name, url, console, expires_at, days, holder?, claim }`; a signature reserves the claim for that key | 201; 401 bad signature; 429 rate limited, five a minute per address, 60 overall; 503 no free name |1818 | `/favicon.svg` | GET | none | the icon | 200 |19+| `/.well-known/bindws/hostname?domain=<host>` | GET, on any host | none | whether the hostname is one of ours: the apex, a valid name under it, or a mapped custom hostname; no body. For a proxy issuing certificates on demand ([Hosting without Cloudflare](16-hosting-without-cloudflare.md)) | 200 ours; 404 not |1920 2021 ## Relay page and information2122 @@ -0,0 +1,146 @@
1+---2+title: Hosting without Cloudflare3+audience: developer4+---5+6+# Hosting without Cloudflare7+8+How to run bind.ws on your own machines, with nothing from Cloudflare in the path. The Worker is the same code; [celld](https://celld.dev) runs it, an S3-compatible bucket you own holds everything, and a reverse proxy of yours answers TLS. bind.ws itself runs on Cloudflare ([Hosting bind.ws](11-hosting-bindws.md)); this is the other supported way, and the section at the end says what "supported" means.9+10+## What runs where11+12+celld is Deno's self-hosted, distributed implementation of Durable Objects and Workers, Apache-2.0. Each relay is a cell: a named server with its own SQLite database, replicated into the bucket. Nodes coordinate through the bucket alone, with conditional writes, so there is no control plane and no consensus service. The R2 binding that holds media is served from the same bucket, under `r2/bindws-media/`, and the KV namespace that maps custom hostnames lives there too.13+14+| | On Cloudflare | On celld |15+|---|---|---|16+| The relay objects | Durable Objects | cells, same code, same SQLite |17+| Media, dumps, imports | R2 | the R2 binding on the fleet bucket |18+| TLS and names | Universal SSL, wildcard route | your proxy, one of two ways below |19+| Custom domains | Cloudflare for SaaS from the console | you map them with `celld kv put` |20+| The client's address | `cf-connecting-ip` | the header your proxy sets, `CLIENT_IP_HEADER` |21+| The lease door's limit | rate limit bindings | token buckets on the node, the same figures |22+| Fuel and zaps | LNURL over outbound fetch | the same |23+| Logs | `wrangler tail` | the node's stdout, [telemetry](https://celld.dev/docs/telemetry) |24+25+What does not carry over: the console's custom domain tab says the feature is not enabled, because there is no Cloudflare for SaaS to register hostnames with; the operator maps them by hand instead. The fuel prices in the config were set from Cloudflare's rates; set your own from what your machines cost, or leave `LIGHTNING_ADDRESS` empty and run without top-ups, allowances still apply.26+27+## What you need28+29+- A Linux box, x86-64 or ARM64. Two for celld's default durability posture, which acknowledges a write once a second node holds it; one node works with `CELLD_DURABILITY=bucket`, where every write waits for the bucket instead. Apple Silicon works for development.30+- An S3-compatible bucket with conditional writes. celld acquires each cell with `If-None-Match: *` and updates it with `If-Match`, and is not correct on a store without them. celld qualifies Amazon S3, Tigris, Google Cloud Storage and Azure Blob Storage for production, and says MinIO's community edition passes its storage test without being qualified. Backblaze B2, Hetzner Object Storage and DigitalOcean Spaces lack the conditional writes. `celld diagnose` tests a bucket before you trust it. The steps below use MinIO, the simplest thing to run on the same box; use `RELEASE.2025-09-07T16-13-09Z` or later, the release before it answers the conditional create wrongly.31+- A domain, with the apex and a wildcard pointed at your proxy.32+- A reverse proxy for TLS: Caddy is shown, nginx works the same way.33+- Node 22 and the repo's `node_modules`, which carry the esbuild that `celld deploy` needs.34+35+## The bucket36+37+MinIO on the box, as one binary, with a bucket for the fleet:38+39+```40+MINIO_ROOT_USER=bindws MINIO_ROOT_PASSWORD=<secret> minio server /var/lib/minio --address 127.0.0.1:900041+mc alias set local http://127.0.0.1:9000 bindws <secret>42+mc mb local/bindws-fleet43+```44+45+Then tell celld where it is, and let it probe the conditional writes:46+47+```48+export AWS_ACCESS_KEY_ID=bindws AWS_SECRET_ACCESS_KEY=<secret> AWS_REGION=us-east-149+export CELLD_BUCKET=s3://bindws-fleet S3_ENDPOINT=http://127.0.0.1:900050+celld diagnose51+```52+53+The line to look for is `ok bucket conditional write: create, reject-create, update, reject-stale`. Give the credentials to one fleet only; celld's [security notes](https://celld.dev/docs/security) say why. A managed bucket at a qualified provider takes the same variables with its endpoint and region.54+55+## Configure56+57+`wrangler.celld.jsonc` is the Worker's config for celld: the same code, bindings, migrations and prices as `wrangler.jsonc`, and only the keys celld accepts, since it stops on any other. `node scripts/check-celld.mjs`, part of `npm run typecheck`, fails when the two drift. Edit:58+59+| Setting | Meaning |60+|---|---|61+| `DOMAIN` | your domain, such as `relays.example` |62+| `CLIENT_IP_HEADER` | the header your proxy puts the client's address in: `x-forwarded-for` for Caddy, `x-real-ip` for nginx. Empty means no address is known, and per-address limits and blocks stand down. Set it only behind a proxy that overwrites the header on every request, or a client can name its own address |63+| `LIGHTNING_ADDRESS`, `SERVICE_PUBKEY` | as on Cloudflare ([Service key](11-hosting-bindws.md#service-key)) |64+| `FREE_*`, `SATS_PER_*` | your allowances and prices |65+66+`ZONE_ID` and `CNAME_TARGET` stay empty. A node can also override any var at start with `CELLD_VAR_<NAME>=value` or a `CELLD_VARS_FILE`, which is where a value you would rather not commit goes.67+68+## Install celld, deploy, run a node69+70+```71+curl -fsSL https://celld.dev/install.sh -o install.sh && sh install.sh # into ~/.local/bin72+npm ci73+PATH="$PWD/node_modules/.bin:$PATH" celld deploy wrangler.celld.jsonc74+```75+76+`celld deploy` bundles the Worker with esbuild and writes it to the bucket; a running node adopts the new version at its next poll, without a restart, so a deploy is the same command again. Then a node:77+78+```79+CELLD_DURABILITY=bucket celld --listen 127.0.0.1:8080 --trust-forwarded-headers80+```81+82+`--trust-forwarded-headers` lets the proxy's `X-Forwarded-Host` and `X-Forwarded-Proto` set the request's host and scheme, which the relay prints in every URL it hands out; set it only behind your proxy. A second node makes the fleet, and the default durability posture, work: give each a private address peers can reach, `--internal-listen 10.0.0.2:8081 --advertise 10.0.0.2:8081`, on a network of your own or an overlay such as WireGuard, since peer traffic has no TLS of its own. The [celld docs](https://celld.dev/docs) cover the fleet: `celld diagnose --peer`, draining, memory limits, the health path at `/.well-known/celld/health`.83+84+Run it under systemd or whatever keeps your daemons up. Idle relays hibernate in celld as on Cloudflare; a node keeps about 4 MB per resident cell.85+86+## TLS and names: the proxy87+88+Every name under the domain is a relay, so the proxy needs a certificate for any name. Two ways:89+90+**Certificates on demand.** Caddy can fetch a certificate the first time a hostname is asked for, and asks the Worker first whether the hostname is one of ours, so a stranger cannot make it fetch certificates for junk. The Worker answers that at `/.well-known/bindws/hostname?domain=<host>`, on any host, with 200 or 404 and no body: the apex, a valid name under the domain, or a custom hostname in the map. A Caddyfile:91+92+```93+{94+ on_demand_tls {95+ ask http://127.0.0.1:8080/.well-known/bindws/hostname96+ }97+}98+99+https:// {100+ tls {101+ on_demand102+ }103+ reverse_proxy 127.0.0.1:8080104+}105+```106+107+Caddy passes `Host` through and sets `X-Forwarded-For`, `X-Forwarded-Host` and `X-Forwarded-Proto` on its own, and proxies websockets without more said, so this is the whole file. `CLIENT_IP_HEADER` is `x-forwarded-for`.108+109+**One wildcard certificate.** A certificate for `*.relays.example` and the apex, through the DNS-01 challenge with your DNS provider, covers every name at once and needs no question asked. Caddy does that with a DNS plugin for the provider; certbot does it too. Custom hostnames still need their own certificates, which on-demand issuance gives and a wildcard does not; with a wildcard, add on-demand issuance for hostnames outside the domain, or skip custom domains.110+111+DNS: an `A` or `AAAA` record for the apex and for `*`, both at the proxy.112+113+With nginx, set `proxy_set_header` for `Host`, `X-Forwarded-Host`, `X-Forwarded-Proto` and `X-Real-IP`, and `Upgrade` and `Connection` for the websocket; `CLIENT_IP_HEADER` is `x-real-ip`.114+115+## Custom domains116+117+An owner who wants `relay.example.org` on their relay sends you the hostname and the relay name, and points a CNAME at your proxy. You map it:118+119+```120+celld kv put hosts relay.example.org <name>121+```122+123+`hosts` is the namespace id in `wrangler.celld.jsonc`. The Worker looks unknown hostnames up in that map, with a minute of cache per isolate, and the on-demand question above says yes for mapped hostnames, so Caddy fetches the certificate on the first visit. `celld kv delete hosts relay.example.org` takes it back; deleting the relay does not, since the object never learns about hostnames mapped this way.124+125+## Local development126+127+```128+npm ci129+PATH="$PWD/node_modules/.bin:$PATH" celld dev wrangler.celld.jsonc130+```131+132+That serves `http://127.0.0.1:9876` as the relay named `DEV_RELAY`, with state in `.celld/dev`, rebuilding on every change; `http://<name>.localhost:9876` is the relay `<name>`, as under `wrangler dev`. No bucket is needed. The black-box suite runs against it:133+134+```135+CLAIM=1 RELAY_URL=ws://127.0.0.1:9876 npm run test:conformance136+```137+138+## What is checked, and the promise139+140+Three things keep this path working rather than merely described:141+142+- CI's `celld` job installs a pinned celld, starts `celld dev` on `wrangler.celld.jsonc` and runs the conformance suite against it on every push: the protocol, and files through the R2 binding (`test/conformance/media.test.ts`).143+- `npm run typecheck` runs `scripts/check-celld.mjs`, so a binding, migration or price changed in one config and not the other fails the build.144+- `test/selfhost.test.ts` covers what the Worker does without Cloudflare's bindings: the address header, the lease door's own count, the hostname question.145+146+Cloudflare is where bind.ws runs and what gets first attention; a celld release that breaks something here may take a little while to be caught up with, and the pinned version in CI says which one is known good. The path is not dropped quietly: removing support for any host takes a notice at the top of this page and in the README first, and the commit that removes it names the last commit that had it, so anyone on that host can stay there.@@ -8,7 +8,7 @@
88 "deploy": "wrangler deploy",99 "test": "vitest run",1010 "test:conformance": "vitest run --config vitest.conformance.config.ts",11- "typecheck": "tsc --noEmit && node scripts/build-signer.mjs --check && node scripts/check-console.mjs",11+ "typecheck": "tsc --noEmit && node scripts/build-signer.mjs --check && node scripts/check-console.mjs && node scripts/check-celld.mjs",1212 "build:signer": "node scripts/build-signer.mjs"1313 },1414 "dependencies": {@@ -0,0 +1,71 @@
1+// Keeps wrangler.celld.jsonc, the config for hosting without Cloudflare2+// (docs/16-hosting-without-cloudflare.md), in step with wrangler.jsonc: the3+// same code, bindings, migrations and prices, and only the keys celld4+// accepts. Runs as part of `npm run typecheck`, so a change to one file that5+// forgets the other fails the build instead of quietly dropping the host.6+import { readFileSync } from "node:fs";7+8+// stripComments removes // and /* */ comments outside strings.9+function stripComments(text) {10+ let out = "";11+ let inString = false;12+ for (let i = 0; i < text.length; i++) {13+ const c = text[i];14+ if (inString) {15+ out += c;16+ if (c === "\\") out += text[++i];17+ else if (c === '"') inString = false;18+ } else if (c === '"') {19+ inString = true;20+ out += c;21+ } else if (c === "/" && text[i + 1] === "/") {22+ while (i < text.length && text[i] !== "\n") i++;23+ out += "\n";24+ } else if (c === "/" && text[i + 1] === "*") {25+ i = text.indexOf("*/", i + 2) + 1;26+ } else out += c;27+ }28+ return out;29+}30+const load = (path) => JSON.parse(stripComments(readFileSync(new URL("../" + path, import.meta.url), "utf8")));31+32+const cf = load("wrangler.jsonc");33+const celld = load("wrangler.celld.jsonc");34+const problems = [];35+const same = (what, a, b) => {36+ if (JSON.stringify(a) !== JSON.stringify(b)) problems.push(`${what} differs: wrangler.jsonc has ${JSON.stringify(a)}, wrangler.celld.jsonc has ${JSON.stringify(b)}`);37+};38+39+// The keys celld deploy accepts (https://celld.dev/docs/cloudflare-compat);40+// any other top-level key stops its deployment.41+const ACCEPTED = new Set(["$schema", "name", "main", "no_bundle", "compatibility_date", "compatibility_flags", "durable_objects", "migrations", "assets", "services", "triggers", "vars", "d1_databases", "kv_namespaces", "queues", "workflows", "r2_buckets"]);42+for (const k of Object.keys(celld)) if (!ACCEPTED.has(k)) problems.push(`wrangler.celld.jsonc has "${k}", which celld deploy refuses`);43+44+same("name", cf.name, celld.name);45+same("main", cf.main, celld.main);46+same("compatibility_date", cf.compatibility_date, celld.compatibility_date);47+same("compatibility_flags", cf.compatibility_flags, celld.compatibility_flags);48+same("durable_objects", cf.durable_objects, celld.durable_objects);49+same("migrations", cf.migrations, celld.migrations);50+same("r2_buckets bindings", cf.r2_buckets.map((b) => b.binding), celld.r2_buckets.map((b) => b.binding));51+same("kv_namespaces bindings", cf.kv_namespaces.map((b) => b.binding), celld.kv_namespaces.map((b) => b.binding));52+53+// Vars: the prices and allowances match; the site-specific ones may differ;54+// the Cloudflare-only ones are empty on celld; celld has the proxy header.55+const SITE = new Set(["DOMAIN", "DEV_RELAY", "LIGHTNING_ADDRESS", "SERVICE_PUBKEY", "LEASE_DAYS"]);56+const CLOUDFLARE_ONLY = new Set(["ZONE_ID", "CNAME_TARGET"]);57+const CELLD_ONLY = new Set(["CLIENT_IP_HEADER"]);58+for (const k of Object.keys(cf.vars)) {59+ if (!(k in celld.vars)) problems.push(`vars.${k} is missing from wrangler.celld.jsonc`);60+ else if (CLOUDFLARE_ONLY.has(k)) {61+ if (celld.vars[k] !== "") problems.push(`vars.${k} is Cloudflare-only and must be empty in wrangler.celld.jsonc`);62+ } else if (!SITE.has(k)) same(`vars.${k}`, cf.vars[k], celld.vars[k]);63+}64+for (const k of Object.keys(celld.vars)) if (!(k in cf.vars) && !CELLD_ONLY.has(k)) problems.push(`vars.${k} is in wrangler.celld.jsonc but not in wrangler.jsonc`);65+for (const k of CELLD_ONLY) if (!(k in celld.vars)) problems.push(`vars.${k} is missing from wrangler.celld.jsonc`);66+67+if (problems.length) {68+ console.error("wrangler.celld.jsonc is out of step with wrangler.jsonc:\n " + problems.join("\n "));69+ process.exit(1);70+}71+console.log("wrangler.celld.jsonc matches wrangler.jsonc");@@ -134,13 +134,14 @@ function save(relay: Relay, hosts: CustomHost[]) {
134134 // remembers it. KV is written only after Cloudflare accepted the hostname.135135 export async function addDomain(relay: Relay, raw: string): Promise<DomainView | string> {136136 const api = relay.hostnames;137- if (!api) return UNSUPPORTED;137+ const kv = relay.hosts;138+ if (!api || !kv) return UNSUPPORTED;138139 const c = checkHostname(raw, relay.domain);139140 if ("error" in c) return c.error;140141 const hosts = stored(relay);141142 if (hosts.some((h) => h.host === c.host)) return "duplicate: that hostname is already on this relay";142143 if (hosts.length >= MAX_CUSTOM_HOSTS) return `restricted: at most ${MAX_CUSTOM_HOSTS} custom domains per relay`;143- const taken = await relay.hosts.get(c.host);144+ const taken = await kv.get(c.host);144145 if (taken && taken !== relay.slug) return "restricted: that hostname points at another relay";145146 let state: HostnameState;146147 try {@@ -148,7 +149,7 @@ export async function addDomain(relay: Relay, raw: string): Promise<DomainView |
148149 } catch (err) {149150 return "error: " + (err instanceof Error ? err.message : String(err));150151 }151- await relay.hosts.put(c.host, relay.slug);152+ await kv.put(c.host, relay.slug);152153 const h: CustomHost = { host: c.host, id: state.id, at: Math.floor(Date.now() / 1000), status: state.status, sslStatus: state.sslStatus };153154 save(relay, [...hosts, h]);154155 return view(h, relay.cnameTarget, state);@@ -187,7 +188,8 @@ export async function removeDomain(relay: Relay, raw: string): Promise<string> {
187188 } catch (err) {188189 return "error: " + (err instanceof Error ? err.message : String(err));189190 }190- if ((await relay.hosts.get(host)) === relay.slug) await relay.hosts.delete(host);191+ const kv = relay.hosts;192+ if (kv && (await kv.get(host)) === relay.slug) await kv.delete(host);191193 save(relay, hosts.filter((x) => x.host !== host));192194 return "";193195 }@@ -206,7 +208,8 @@ export async function forgetDomains(relay: Relay): Promise<void> {
206208 console.log(`custom hostname ${h.host} not removed at Cloudflare: ${err instanceof Error ? err.message : String(err)}`);207209 }208210 try {209- if ((await relay.hosts.get(h.host)) === relay.slug) await relay.hosts.delete(h.host);211+ const kv = relay.hosts;212+ if (kv && (await kv.get(h.host)) === relay.slug) await kv.delete(h.host);210213 } catch (err) {211214 console.log(`custom hostname ${h.host} not removed from KV: ${err instanceof Error ? err.message : String(err)}`);212215 }@@ -0,0 +1,83 @@
1+// What the edge in front of the Worker provides, and what stands in when it2+// is not Cloudflare's: the client's address, the lease door's rate limit,3+// the custom hostname map, and the answer to a proxy asking whether a4+// hostname belongs here. Hosting without Cloudflare is5+// docs/16-hosting-without-cloudflare.md.6+import type { Env } from "./relay.ts";7+import { Bucket } from "./ratelimit.ts";8+import { validName } from "./names.ts";9+10+// Custom domains map to relay names through KV. Looked up once a minute per11+// hostname per isolate; a miss is cached too, so an unknown host does not12+// cost a read on every request.13+const HOST_TTL_MS = 60_000;14+const hostCache = new Map<string, { name: string | null; at: number }>();15+export async function customHost(env: Pick<Env, "HOSTS">, host: string): Promise<string | null> {16+ const hit = hostCache.get(host);17+ if (hit && Date.now() - hit.at < HOST_TTL_MS) return hit.name;18+ let name: string | null = null;19+ try {20+ name = env.HOSTS ? await env.HOSTS.get(host) : null;21+ } catch {22+ name = null;23+ }24+ if (hostCache.size > 10_000) hostCache.clear();25+ hostCache.set(host, { name, at: Date.now() });26+ return name;27+}28+29+// clientIP is the address the edge saw. On Cloudflare that is30+// cf-connecting-ip, which the edge sets and nothing a client sends can31+// forge. Elsewhere it is the header CLIENT_IP_HEADER names, written by the32+// operator's own proxy; the last address in a comma list is the one that33+// proxy appended. An empty CLIENT_IP_HEADER means no address is known, and34+// everything keyed by address stands down (relay.ts, ipLimit).35+export function clientIP(req: Request, env: Pick<Env, "CLIENT_IP_HEADER">): string {36+ const header = env.CLIENT_IP_HEADER ?? "cf-connecting-ip";37+ if (header === "") return "unknown";38+ const last = (req.headers.get(header) ?? "").split(",").pop()?.trim() ?? "";39+ return last || "unknown";40+}41+42+// The lease door's width when there is no rate limit binding: the same43+// figures as `ratelimits` in wrangler.jsonc, kept per isolate, which on44+// one celld node is the whole relay.45+export const LEASES_PER_IP_MINUTE = 5;46+export const LEASES_PER_MINUTE = 60;47+const leaseBuckets = new Map<string, Bucket>();48+let leasesAll: Bucket | null = null;49+50+// leaseAllowed spends one lease from the address's allowance and from51+// everyone's: Cloudflare's rate limit bindings where they exist, token52+// buckets in memory where they do not.53+export async function leaseAllowed(env: Pick<Env, "LEASE_LIMIT_IP" | "LEASE_LIMIT_ALL">, ip: string): Promise<boolean> {54+ if (env.LEASE_LIMIT_IP && env.LEASE_LIMIT_ALL) {55+ const [perIP, all] = await Promise.all([env.LEASE_LIMIT_IP.limit({ key: ip }), env.LEASE_LIMIT_ALL.limit({ key: "all" })]);56+ return perIP.success && all.success;57+ }58+ if (leaseBuckets.size > 10_000) leaseBuckets.clear();59+ let b = leaseBuckets.get(ip);60+ if (!b) {61+ b = new Bucket(LEASES_PER_IP_MINUTE);62+ leaseBuckets.set(ip, b);63+ }64+ leasesAll ??= new Bucket(LEASES_PER_MINUTE);65+ const mine = b.take() === "";66+ const everyone = leasesAll.take() === "";67+ return mine && everyone;68+}69+70+// hostnameKnown says whether a hostname is one of ours: the apex, a valid71+// name under it, or a custom hostname in the map. A proxy that issues72+// certificates on demand asks this before it asks a certificate authority,73+// so a stranger cannot make it request certificates for junk names.74+export async function hostnameKnown(env: Pick<Env, "DOMAIN" | "HOSTS">, raw: string): Promise<boolean> {75+ const host = raw.toLowerCase().replace(/\.$/, "");76+ const domain = env.DOMAIN.toLowerCase();77+ if (host === domain || host === "www." + domain) return true;78+ if (host.endsWith("." + domain)) {79+ const name = host.slice(0, -(domain.length + 1));80+ return validName(name) && !name.includes(".");81+ }82+ return host !== "" && (await customHost(env, host)) !== null;83+}@@ -7,29 +7,13 @@ import { FAVICON_SVG } from "./ui.ts";
77 import { verifyNIP98 } from "./manage.ts";88 import { leaseNames, leaseDays, validName } from "./names.ts";99 import { now } from "./event.ts";10+import { clientIP, customHost, hostnameKnown, leaseAllowed } from "./edge.ts";1011 12+// The entry module exports the handler and the object only: workerd13+// refuses any other kind of export here.1114 export { Relay };1215 export { NAME_RE, RESERVED, validName } from "./names.ts";1316 14-// Custom domains map to relay names through KV. Looked up once a minute per15-// hostname per isolate; a miss is cached too, so an unknown host does not16-// cost a read on every request.17-const HOST_TTL_MS = 60_000;18-const hostCache = new Map<string, { name: string | null; at: number }>();19-async function customHost(env: Env, host: string): Promise<string | null> {20- const hit = hostCache.get(host);21- if (hit && Date.now() - hit.at < HOST_TTL_MS) return hit.name;22- let name: string | null = null;23- try {24- name = env.HOSTS ? await env.HOSTS.get(host) : null;25- } catch {26- name = null;27- }28- if (hostCache.size > 10_000) hostCache.clear();29- hostCache.set(host, { name, at: Date.now() });30- return name;31-}32-3317 // lease hands out a temporary relay at a memorable name: open to everyone3418 // for a while, wiped after unless claimed. Anyone may ask, no key needed;3519 // a NIP-98 signature reserves the claim for that key. For scripts, agents@@ -39,9 +23,7 @@ async function lease(req: Request, env: Env, apex: URL): Promise<Response> {
3923 const json = (b: unknown, status = 200) => new Response(JSON.stringify(b), { status, headers: cors });4024 // Each lease is an object with storage, so the door is narrow: a few per4125 // address per minute, and a ceiling for everyone together.42- const ip = req.headers.get("cf-connecting-ip") ?? "unknown";43- const [perIP, all] = await Promise.all([env.LEASE_LIMIT_IP.limit({ key: ip }), env.LEASE_LIMIT_ALL.limit({ key: "all" })]);44- if (!perIP.success || !all.success) return json({ error: "rate-limited: too many leases, try again in a minute" }, 429);26+ if (!(await leaseAllowed(env, clientIP(req, env)))) return json({ error: "rate-limited: too many leases, try again in a minute" }, 429);4527 const body = await req.text();4628 let holder = "";4729 const authz = req.headers.get("authorization");@@ -77,6 +59,12 @@ export default {
7759 const url = new URL(req.url);7860 const host = url.hostname.toLowerCase();7961 const domain = env.DOMAIN.toLowerCase();62+ // A proxy issuing certificates on demand asks whether a hostname is63+ // ours before it asks the certificate authority (docs/16). Any host,64+ // no body: 200 or 404.65+ if (url.pathname === "/.well-known/bindws/hostname" && req.method === "GET") {66+ return new Response(null, { status: (await hostnameKnown(env, url.searchParams.get("domain") ?? "")) ? 200 : 404 });67+ }8068 let name: string | null;8169 if (host === domain || host === "www." + domain || host === domain + ".localhost") name = null; // apex (<domain>.localhost in wrangler dev)8270 else if (host.endsWith("." + domain)) name = host.slice(0, -(domain.length + 1));@@ -97,9 +85,9 @@ export default {
9785 const stub = env.RELAY.getByName(name);9886 const headers = new Headers(req.headers);9987 headers.set("x-relay-name", name);100- // The client's address, as Cloudflare saw it. Set here, never read from88+ // The client's address, as the edge saw it. Set here, never read from10189 // the client's own headers, so nothing a caller sends can spoof it.102- headers.set("x-relay-ip", req.headers.get("cf-connecting-ip") ?? "unknown");90+ headers.set("x-relay-ip", clientIP(req, env));10391 return stub.fetch(new Request(req, { headers }));10492 },10593 } satisfies ExportedHandler<Env>;@@ -46,11 +46,16 @@ export interface Env {
4646 DOMAIN: string;4747 DEV_RELAY: string;4848 LEASE_DAYS?: string; // how long a temporary relay lives; 14 by default49- LEASE_LIMIT_IP: RateLimit; // leases per address per minute50- LEASE_LIMIT_ALL: RateLimit; // leases per minute, everyone together49+ // Cloudflare's rate limit bindings; without them (celld) edge.ts keeps50+ // token buckets in memory instead.51+ LEASE_LIMIT_IP?: RateLimit; // leases per address per minute52+ LEASE_LIMIT_ALL?: RateLimit; // leases per minute, everyone together53+ // The header the client's address arrives in; cf-connecting-ip if unset,54+ // no address at all if empty (edge.ts, clientIP).55+ CLIENT_IP_HEADER?: string;5156 // Custom domains (see domains.ts): hostname to relay name, and the5257 // Cloudflare for SaaS credentials. Without the token the feature is off.53- HOSTS: KVNamespace;58+ HOSTS?: KVNamespace;5459 CF_API_TOKEN?: string;5560 ZONE_ID?: string;5661 CNAME_TARGET?: string; // what owners point their CNAME at; customers.<DOMAIN> if unset@@ -171,7 +176,7 @@ export class Relay extends DurableObject<Env> {
171176 this.fuel = new Fuel(ctx.storage.sql, fuelConfig(env as unknown as Record<string, unknown>));172177 this.identity = new Identity(ctx.storage);173178 this.audit = new Audit(ctx.storage.sql, () => this.slug);174- this.hostnames = env.CF_API_TOKEN && env.ZONE_ID ? new Hostnames(env.CF_API_TOKEN, env.ZONE_ID, (u, i) => this.fetcher(u, i)) : null;179+ this.hostnames = env.CF_API_TOKEN && env.ZONE_ID && env.HOSTS ? new Hostnames(env.CF_API_TOKEN, env.ZONE_ID, (u, i) => this.fetcher(u, i)) : null;175180 ctx.blockConcurrencyWhile(async () => {176181 this.store.init();177182 this.settings.load();@@ -196,8 +201,9 @@ export class Relay extends DurableObject<Env> {
196201 return this.env.DOMAIN;197202 }198203 199- // hosts is the custom domain map the worker routes by.200- get hosts(): KVNamespace {204+ // hosts is the custom domain map the worker routes by; absent on a host205+ // without the namespace, where hostnames is null too.206+ get hosts(): KVNamespace | undefined {201207 return this.env.HOSTS;202208 }203209 @@ -0,0 +1,59 @@
1+// Files, from outside: a Blossom upload, the download with and without a2+// range, the listing and the delete. The one door that touches the media3+// bucket, so a host whose bucket is not R2 (docs/16) is checked here.4+import { describe, it, expect } from "vitest";5+import { sha256 } from "@noble/hashes/sha2.js";6+import { HTTP_URL, newEvent, newKey, now, pub, rand } from "./helpers.ts";7+8+const hex = (b: Uint8Array) => Array.from(b, (x) => x.toString(16).padStart(2, "0")).join("");9+10+// A Blossom token: kind 24242 with the verb and the hash, base64 in the header.11+function token(sk: Uint8Array, verb: "upload" | "get" | "list" | "delete", sha?: string): string {12+ const tags = [["t", verb], ...(sha ? [["x", sha]] : []), ["expiration", String(now() + 300)]];13+ return "Nostr " + Buffer.from(JSON.stringify(newEvent(sk, 24242, verb, tags))).toString("base64");14+}15+16+describe("Blossom media", () => {17+ it("uploads a file, serves it whole and by range, lists it, and deletes it", async () => {18+ const sk = newKey();19+ const body = new TextEncoder().encode(`a small file ${rand()} with enough bytes for a range`);20+ const sha = hex(sha256(body));21+22+ const put = await fetch(`${HTTP_URL}/upload`, { method: "PUT", headers: { authorization: token(sk, "upload", sha), "content-type": "text/plain" }, body });23+ expect(put.status, await put.clone().text()).toBe(200);24+ const desc: any = await put.json();25+ expect([desc.sha256, desc.size, desc.type]).toEqual([sha, body.length, "text/plain"]);26+ expect(desc.url).toMatch(new RegExp(`/${sha}\\.txt$`));27+28+ // The whole file, with its type and hash-derived etag.29+ const get = await fetch(`${HTTP_URL}/${sha}.txt`);30+ expect(get.status).toBe(200);31+ expect(get.headers.get("content-type")).toBe("text/plain");32+ expect(new Uint8Array(await get.arrayBuffer())).toEqual(body);33+34+ // A byte range: the second word.35+ const part = await fetch(`${HTTP_URL}/${sha}`, { headers: { range: "bytes=2-6" } });36+ expect(part.status).toBe(206);37+ expect(part.headers.get("content-range")).toBe(`bytes 2-6/${body.length}`);38+ expect(await part.text()).toBe("small");39+40+ // HEAD says it is there without a body.41+ const head = await fetch(`${HTTP_URL}/${sha}`, { method: "HEAD" });42+ expect(head.status).toBe(200);43+ expect(await head.text()).toBe("");44+45+ // The uploader's listing carries it.46+ const list = await fetch(`${HTTP_URL}/list/${pub(sk)}`, { headers: { authorization: token(sk, "list") } });47+ expect(list.status).toBe(200);48+ expect((await list.json<any>()).map((d: any) => d.sha256)).toContain(sha);49+50+ // The uploader deletes it; the same bytes can come back afterwards.51+ const del = await fetch(`${HTTP_URL}/${sha}`, { method: "DELETE", headers: { authorization: token(sk, "delete", sha) } });52+ expect(del.status).toBe(204);53+ expect((await fetch(`${HTTP_URL}/${sha}`)).status).toBe(404);54+ const again = await fetch(`${HTTP_URL}/upload`, { method: "PUT", headers: { authorization: token(sk, "upload", sha), "content-type": "text/plain" }, body });55+ expect(again.status).toBe(200);56+ expect((await fetch(`${HTTP_URL}/${sha}`)).status).toBe(200);57+ await fetch(`${HTTP_URL}/${sha}`, { method: "DELETE", headers: { authorization: token(sk, "delete", sha) } });58+ });59+});@@ -0,0 +1,62 @@
1+// A host without Cloudflare's edge (docs/16-hosting-without-cloudflare.md):2+// the client's address comes from the proxy's header or not at all, the3+// lease door keeps its own count, and a proxy can ask which hostnames are4+// ours before it fetches certificates for them.5+import { SELF, env } from "cloudflare:test";6+import { describe, it, expect } from "vitest";7+import { clientIP, hostnameKnown, leaseAllowed, LEASES_PER_IP_MINUTE } from "../src/edge.ts";8+9+const req = (headers: Record<string, string>) => new Request("http://x/", { headers });10+11+describe("the client's address", () => {12+ it("is cf-connecting-ip by default, the named header's last entry elsewhere, and unknown when the header is empty", () => {13+ expect(clientIP(req({ "cf-connecting-ip": "1.2.3.4", "x-forwarded-for": "9.9.9.9" }), {})).toBe("1.2.3.4");14+ expect(clientIP(req({}), {})).toBe("unknown");15+ expect(clientIP(req({ "x-forwarded-for": "9.9.9.9, 10.0.0.7" }), { CLIENT_IP_HEADER: "x-forwarded-for" })).toBe("10.0.0.7");16+ expect(clientIP(req({ "x-real-ip": "2001:db8::1" }), { CLIENT_IP_HEADER: "x-real-ip" })).toBe("2001:db8::1");17+ expect(clientIP(req({ "x-forwarded-for": "" }), { CLIENT_IP_HEADER: "x-forwarded-for" })).toBe("unknown");18+ // Empty: nothing a client sends counts as an address.19+ expect(clientIP(req({ "cf-connecting-ip": "1.2.3.4", "x-forwarded-for": "9.9.9.9" }), { CLIENT_IP_HEADER: "" })).toBe("unknown");20+ });21+});22+23+describe("the lease door without rate limit bindings", () => {24+ it("allows a few per address per minute and then refuses, per address", async () => {25+ const none = {};26+ for (let i = 0; i < LEASES_PER_IP_MINUTE; i++) expect(await leaseAllowed(none, "203.0.113.1")).toBe(true);27+ expect(await leaseAllowed(none, "203.0.113.1")).toBe(false);28+ expect(await leaseAllowed(none, "203.0.113.2")).toBe(true);29+ });30+31+ it("uses the bindings when both exist", async () => {32+ const calls: string[] = [];33+ const binding = (ok: boolean): RateLimit => ({ limit: async ({ key }) => (calls.push(key), { success: ok }) });34+ expect(await leaseAllowed({ LEASE_LIMIT_IP: binding(true), LEASE_LIMIT_ALL: binding(true) }, "203.0.113.9")).toBe(true);35+ expect(await leaseAllowed({ LEASE_LIMIT_IP: binding(true), LEASE_LIMIT_ALL: binding(false) }, "203.0.113.9")).toBe(false);36+ expect(calls).toEqual(["203.0.113.9", "all", "203.0.113.9", "all"]);37+ });38+});39+40+describe("which hostnames are ours", () => {41+ it("knows the apex, valid names under it and mapped custom hostnames, and nothing else", async () => {42+ const e = { DOMAIN: "bind.ws", HOSTS: env.HOSTS };43+ await env.HOSTS.put("relay.example.org", "kitchen");44+ expect(await hostnameKnown(e, "bind.ws")).toBe(true);45+ expect(await hostnameKnown(e, "www.bind.ws")).toBe(true);46+ expect(await hostnameKnown(e, "Kitchen.bind.ws.")).toBe(true);47+ expect(await hostnameKnown(e, "relay.example.org")).toBe(true);48+ expect(await hostnameKnown(e, "a.b.bind.ws")).toBe(false);49+ expect(await hostnameKnown(e, "-bad.bind.ws")).toBe(false);50+ expect(await hostnameKnown(e, "other.example.org")).toBe(false);51+ expect(await hostnameKnown(e, "")).toBe(false);52+ });53+54+ it("answers a proxy's question on any host with a status and no body", async () => {55+ const ask = async (domain: string) => (await SELF.fetch(`http://bind.ws/.well-known/bindws/hostname?domain=${encodeURIComponent(domain)}`)).status;56+ expect(await ask("kitchen.bind.ws")).toBe(200);57+ expect(await ask("bind.ws")).toBe(200);58+ expect(await ask("nope.example.org")).toBe(404);59+ const r = await SELF.fetch("http://anything.example/.well-known/bindws/hostname?domain=kitchen.bind.ws");60+ expect([r.status, await r.text()]).toEqual([200, ""]);61+ });62+});@@ -0,0 +1,43 @@
1+{2+ // The same Worker on celld (https://celld.dev), for hosting without3+ // Cloudflare: `celld dev wrangler.celld.jsonc` locally, `celld deploy4+ // wrangler.celld.jsonc --bucket ...` to a fleet. celld accepts a subset of5+ // the Wrangler keys and stops on any other, so this file has no routes,6+ // rate limits, observability or environments. `node scripts/check-celld.mjs`7+ // keeps it in step with wrangler.jsonc. See docs/16-hosting-without-cloudflare.md.8+ "name": "bindws-relay",9+ "main": "src/index.ts",10+ "compatibility_date": "2026-08-01",11+ "compatibility_flags": ["nodejs_compat"],12+ "vars": {13+ "DOMAIN": "bind.ws",14+ "DEV_RELAY": "dev",15+ // The header a trusted proxy puts the client's address in: x-forwarded-for16+ // for Caddy, x-real-ip for nginx. Empty means no address is known, so17+ // per-address limits are off. Set it only behind a proxy that overwrites18+ // the header on every request.19+ "CLIENT_IP_HEADER": "",20+ // Cloudflare for SaaS is not available here; custom hostnames are the21+ // proxy's job (see the doc). These stay empty.22+ "ZONE_ID": "",23+ "CNAME_TARGET": "",24+ "LIGHTNING_ADDRESS": "",25+ "SERVICE_PUBKEY": "",26+ "FREE_EVENTS_MB": "100",27+ "FREE_MEDIA_MB": "1024",28+ "FREE_ACTIVE_HOURS": "100",29+ "FREE_ROWS_WRITTEN": "250000",30+ "SATS_PER_GB_MONTH_EVENTS": "400",31+ "SATS_PER_GB_MONTH_MEDIA": "30",32+ "SATS_PER_ACTIVE_HOUR": "11",33+ "SATS_PER_MILLION_ROWS": "2000"34+ },35+ "durable_objects": {36+ "bindings": [{ "name": "RELAY", "class_name": "Relay" }]37+ },38+ // Media lives in the fleet bucket under r2/bindws-media/.39+ "r2_buckets": [{ "binding": "MEDIA", "bucket_name": "bindws-media" }],40+ // Custom hostname to relay name; filled by hand with `celld kv put HOSTS`.41+ "kv_namespaces": [{ "binding": "HOSTS", "id": "hosts" }],42+ "migrations": [{ "tag": "v1", "new_sqlite_classes": ["Relay"] }]43+}