clone

a self-hosted sync node for lofi apps: jazz sync over iroh, ticket-gated, with no static ip or cloud needed

chore: open-source readiness — license, CI, contributing, docs

Dami committed 49e51fe
Matching the lofi repo's conventions: MIT LICENSE (lofi-node
contributors), ci.yml gate (fmt/lint/typecheck + cargo-built addon +
full suite + compile smoke), PR template, CONTRIBUTING with the hard
boundaries (vendoring contract, version pins, no silent degradation,
secrets policy, watch-API ban). deno.json gains license/description,
fmt/lint with native/iroh-js excluded (vendored files stay
byte-identical — fmt had reformatted index.d.ts; restored), @std/assert
via import map, and a native build task. README rewritten as the
first-class self-host story for lofi apps; docs/hosting-lofi-apps.md
captures the validated hosting walkthrough including the alpha.53
browser-gate caveat (parity with lofi's managed dev server).
commit
49e51feffee3cac21abd7792d2e8cfbd86a1e0f3
parent
c2cfafb747e91b16f9683f699c7e230583edf424
author
Dami <dami.etoile@gmail.com>
18 files changed, +410 -248
A.github/pull_request_template.md15+++
A.github/workflows/ci.yml35++++++
ACONTRIBUTING.md41+++++++
ALICENSE21++++
MREADME.md157+++++++++++++-------------
Mdeno.json16++-
Mdeno.lock44+++-----
Adocs/hosting-lofi-apps.md66+++++++++++
Mdocs/port-iroh-js.md178++++++++++++++----------------
Mnative/iroh-js/UPSTREAM.md52++++-----
Msrc/jazz.ts4+-
Msrc/native/addon.ts3+-
Msrc/node.ts13+-
Mtests/codec_test.ts2+-
Mtests/convergence_test.ts5+-
Mtests/mesh_test.ts2+-
Mtests/pair_test.ts2+-
Mtests/tunnel_iroh_test.ts2+-
A .github/pull_request_template.md +15 -0
A .github/workflows/ci.yml +35 -0
A CONTRIBUTING.md +41 -0
@@ -0,0 +1,41 @@
1+# Contributing to lofi-node2+3+## Prerequisites4+5+- Deno 2.9+ (the only global runtime requirement for the TS side)6+- Rust stable (only when touching `native/iroh-js/`; `deno task native` builds the addon)7+8+## The gate9+10+```sh11+deno task check && deno task test12+```13+14+`check` runs fmt --check, lint, and typecheck. `test` runs the full fixture suite; iroh/jazz15+integration tests auto-skip (with a visible marker) when the native addon isn't built, so the fast16+path never needs Rust.17+18+## Branches & PRs19+20+Prefix branches by intent (`fix/`, `feat/`, `docs/`, `chore/`), keep a PR to one concern, and state21+the evidence for behavior claims (test name or command output) in the description.22+23+## Hard boundaries24+25+- **Vendoring contract**: everything under `native/iroh-js/` except `src/lofi_ext.rs` is upstream26+ code. Never edit vendored files beyond `// lofi-node:` marked lines; follow the update procedure27+ in `native/iroh-js/UPSTREAM.md`. `deno fmt`/`lint` exclude the directory — keep it that way.28+- **Version pins are decisions**: `jazz-tools`/`jazz-napi` are pinned to the exact alpha the lofi29+ framework pins. Bumps are coordinated changes with the full suite as evidence, never drive-by.30+- **No silent degradation**: a missing capability (native addon, platform) must surface as a typed31+ reason in `status()`, never as quietly reduced behavior.32+- **Secrets**: nothing under `lofi-node-data/`, `data/`, or generated `config.json` files may be33+ committed; test secrets are obviously-fake fixed strings.34+- **Never call the upstream watch APIs** from the TS layer — at iroh-ffi v1.1.0 they abort the35+ process under Deno (see UPSTREAM.md known issues).36+37+## Publishing38+39+Not yet published: the JSR packaging story for a repo that ships native prebuilts is an open40+decision (JSR payload size vs. artifact downloads). Until that lands, consumers use the repo41+directly or `deno task compile`.
A LICENSE +21 -0
M README.md +78 -79
@@ -1,97 +1,96 @@
11 # @nzip/lofi-node22 3-A self-hostable sync node for [lofi](../lofi) apps: a Jazz 2.0 sync server4-(jazz-napi `JazzServer`) plus iroh node-to-node transport5-([db-iroh-ffi](../db-iroh-ffi) via `Deno.dlopen`). Browsers keep speaking6-Jazz's WebSocket protocol — `JAZZ_SERVER_URL` points at a node and lofi's7-`adapter: "jazz"` contract (DX-SYNC-01) is untouched. iroh lives entirely in8-the daemon: two nodes pair by ticket and replicate over a WS-over-iroh tunnel,9-dialed by key, no static IPs.10-11-**Status: private prove-out.** The native layer is a trimmed vendor of12-upstream [n0-computer/iroh-ffi](https://github.com/n0-computer/iroh-ffi)'s13-`iroh-js` napi crate, built in-repo (`cargo build --release` in14-`native/iroh-js/`; see `native/iroh-js/UPSTREAM.md` for provenance and the15-tag-bump procedure). Override with `LOFI_NODE_IROH=<path>`.16-17-## Layout18-19-| Piece | Where |20-|---|---|21-| One-constructor library | `src/node.ts` (`createSyncNode`) |22-| WS+HTTP-over-iroh tunnel | `src/tunnel.ts` (HELLO/TEXT/BIN/CLOSE frames, 1 conn : 1 ws or 1 http request) |23-| Vendored iroh-js napi crate | `native/iroh-js/` (upstream + `lofi_ext.rs` Buffer framing) |24-| Addon loader + typed surface | `src/native/addon.ts`, `src/native/loader.ts` |25-| Adapter (IrohNode/IrohConn) | `src/iroh/node.ts` |26-| Ticket shape check (tickets are upstream `EndpointTicket` strings) | `src/ticket.ts` |27-| Jazz wrapper (pinned `2.0.0-alpha.53`) | `src/jazz.ts` |28-| CLI: init / start / pair / status | `cli.ts` |29-| In-process test mesh (no iroh) | `testing/mod.ts` |30-31-## Usage3+The first-class way to self-host the sync backend for4+[lofi](https://github.com/FelineStateMachine/lofi) apps: one daemon embedding a Jazz 2.0 sync server5+plus [iroh](https://iroh.computer) node-to-node transport. Browsers keep speaking Jazz's protocol —6+`JAZZ_SERVER_URL` points at your node and lofi's `adapter: "jazz"` contract is untouched. Two nodes7+pair by ticket and replicate over iroh: dialed by key, hole-punched, no static IPs, no cloud8+dependency.9+10+- Hosting walkthrough: [docs/hosting-lofi-apps.md](docs/hosting-lofi-apps.md)11+- Native layer provenance: [native/iroh-js/UPSTREAM.md](native/iroh-js/UPSTREAM.md)12+- License: [MIT](LICENSE)13+14+## Quick start3215 3316 ```sh34-deno run -A cli.ts init # data dir, app id, secrets, iroh key35-deno run -A cli.ts start # prints Jazz URL + pairing ticket36-deno run -A cli.ts pair LFN1.… # persist peer election; restart to apply17+deno task compile # one self-contained binary (dist/lofi-node)18+dist/lofi-node init # data dir, app id, secrets, node key19+dist/lofi-node start # Jazz URL + pairing ticket20+dist/lofi-node pair endpoint… # elect another node as upstream3721 ```3822 39-Library:23+Or embed it:4024 4125 ```ts4226 import { createSyncNode } from "@nzip/lofi-node";27+4328 const node = await createSyncNode({44- appId, backendSecret, adminSecret,29+ appId,30+ backendSecret,31+ adminSecret,4532 dataDir: "./data",46- upstream: { peer: "LFN1.…" }, // or { url: "wss://cloud.jazz.tools/…" } or "none"33+ upstream: { peer: "endpoint…" }, // or { url: "wss://…" } or "none"4734 });48-node.url; // -> JAZZ_SERVER_URL49-node.ticket(); // -> share with the peer35+node.url; // -> JAZZ_SERVER_URL for the lofi app36+node.ticket(); // -> share with the peer node37+await node.pair(otherTicket); // re-elect upstream at runtime; port unchanged38+node.status(); // jazz health, upstream, mesh + live conn stats (direct/relay, rtt)5039 ```5140 52-## Design decisions53-54-- **napi async, not a flat C ABI.** The vendored iroh-js crate exposes real55- Promises; `acceptNext()` resolves `null` on `endpoint.close()`, so the56- accept loop exits cleanly. The one extension (`lofi_ext.rs`) adds57- Buffer-based length-prefixed framing — upstream's `Array<number>` stream58- I/O measured 7.1 MiB/s at gate 0. Never call the upstream watch APIs: at59- v1.1.0 they panic outside a tokio context and abort the process.60-- **The tunnel terminates WS on both ends**, so the dialer forwards the upgrade61- path/subprotocol in a HELLO frame and the acceptor replays them against its62- local JazzServer.63-- **No silent degradation** (lofi's boot-gate ethos): a missing/unsupported64- dylib leaves the Jazz server up but reports `mesh: unavailable — <reason>`,65- and pairing throws `MeshUnavailableError`.66-- **Version invariant:** this package pins the exact `jazz-napi` alpha that the67- consuming lofi app pins (`2.0.0-alpha.53`). Wire compat across alphas is not68- guaranteed; bump in lockstep.69-70-## Native layer71-72-Executed per [docs/port-iroh-js.md](docs/port-iroh-js.md): `native/iroh-js/`73-vendors upstream at v1.1.0 (endpoint/key/net/path/relay/ticket/watch modules;74-services dropped), plus `lofi_ext.rs`. db-iroh-ffi is no longer a dependency.75-Live tunnel connection stats surface in `SyncNodeStatus.mesh.connections`.76-77-Prebuilts (`native/iroh-js/prebuilt/<triple>/`): macOS arm64 (built on a78-mac), Linux x86_64 + aarch64 (Nix host via `nix develop -c79-./scripts/cross-build.sh`). **Windows is open**: napi-build's `*-gnu` path80-needs a libnode.dll import lib (upstream ships msvc, delay-loaded) — needs81-cargo-xwin or a Windows CI runner.82-83-## Tests41+## What you get42+43+- **A real Jazz sync server** (jazz-napi `JazzServer`, SQLite-backed, `/health` endpoint,44+ local-first auth) that any lofi app can use by URL.45+- **Node-to-node replication over iroh**: pairing tickets are upstream `EndpointTicket` strings;46+ sync and catalogue traffic (WS + HTTP) tunnel over QUIC, so two homes converge without port47+ forwarding. Administering either node reaches the root through the tunnel.48+- **Observability**: `status().mesh.connections` reports live tunnel connections with rtt and path49+ counts (direct vs relay).50+- **No silent degradation**: if the native layer can't load, the Jazz server still runs (LAN-only)51+ and `status()` says exactly why pairing is off.52+- **One binary**: `deno task compile` embeds the prebuilt native matrix and extracts it to a53+ version-keyed OS cache at first run.54+55+## Where lofi-node fits56+57+- The data plane is Jazz 2.0 **alpha**, pinned exactly (`jazz-tools@2.0.0-alpha.53`); a node must58+ run the same alpha as the app it serves — treat version bumps as coordinated. Early-stage59+ software.60+- Platforms: macOS arm64, Linux x86_64/aarch64 (prebuilt in-repo; or `deno task native` with a Rust61+ toolchain). **Windows is a documented gap**: napi-build's `*-gnu` linking needs a libnode.dll62+ import library (upstream ships msvc, delay-loaded); a Windows artifact needs cargo-xwin or a63+ Windows CI runner. Until then Windows runs LAN-only with a typed `mesh: unavailable` reason.64+- Schema deploys to a self-hosted node use the jazz-tools schema-project flow — see the hosting65+ walkthrough, including the current alpha caveat on lofi's browser convergence gate (fails66+ identically against lofi's own managed dev server; lofi-node has behavior parity).67+68+## Architecture69+70+| Piece | Where |71+| --------------------------------------- | --------------------------------------------- |72+| One-constructor library | `src/node.ts` (`createSyncNode`) |73+| WS+HTTP-over-iroh tunnel | `src/tunnel.ts` (1 iroh conn : 1 ws / 1 http) |74+| Vendored iroh-js napi crate (iroh 1.x) | `native/iroh-js/` (+ `lofi_ext.rs` framing) |75+| Addon loader (dev shim / cache extract) | `src/native/addon.ts`, `src/native/loader.ts` |76+| Adapter over the addon | `src/iroh/node.ts` (`IrohNode`/`IrohConn`) |77+| Jazz wrapper (pinned alpha) | `src/jazz.ts` |78+| CLI: init / start / pair / status | `cli.ts` |79+| In-process test mesh (no iroh) | `testing/mod.ts` |80+81+The native layer is a trimmed, provenance-tracked vendor of upstream82+[n0-computer/iroh-ffi](https://github.com/n0-computer/iroh-ffi)'s `iroh-js` napi crate —83+module-level vendoring, byte-identical files, every local edit marked `// lofi-node:`, one extension84+module, tag bumps by `diff -r`. Design history: [docs/port-iroh-js.md](docs/port-iroh-js.md).85+86+## Developing lofi-node8487 8588 ```sh86-deno task test # codec units always; iroh tunnel + jazz boot integration87- # auto-skip when the dylib / napi build is unavailable89+deno task native # build the native addon (Rust; once, or per vendored change)90+deno task check # fmt --check + lint + typecheck — the PR gate91+deno task test # full suite; iroh/jazz integration auto-skips without the addon8892 ```8993 90-## Roadmap91-92-- Convergence test: lofi's two-client Playwright fixtures against two paired93- nodes (the real spike exit-criterion).94-- Runtime `pair()` without restart (needs JazzServer restart orchestration).95-- `deno compile` packaging with embedded dylibs (doorbearer's96- extract-to-versioned-cache loader).97-- Away-from-home story: `upstream: "cloud"` chaining behind one flag.94+Cross-compiling the prebuilt matrix runs on any x86_64 Linux host with Nix:95+`nix develop -c ./scripts/cross-build.sh` in `native/iroh-js/`. See96+[CONTRIBUTING.md](CONTRIBUTING.md) for boundaries (vendoring rules, secrets policy, version pins).
M deno.json +13 -3
M deno.lock +15 -29
@@ -190,19 +190,12 @@
190190 "@opentelemetry/semantic-conventions"191191 ]192192 },193- "@opentelemetry/core@2.9.0_@opentelemetry+api@1.9.1": {194- "integrity": "sha512-m2nckMT80NnmjTYSPjJQObBJ+8dgkoajEOUbznL8AHZ3T3yHRk2P7gI1PhEBc1+lOnrYE9UWrWHqJDsmqjmNbw==",195- "dependencies": [196- "@opentelemetry/api",197- "@opentelemetry/semantic-conventions"198- ]199- },200193 "@opentelemetry/exporter-logs-otlp-http@0.216.0_@opentelemetry+api@1.9.1": {201194 "integrity": "sha512-8SUzQY/aExKkz6Ab3vOf6gu690Xk4wHH90dGwXinejQzazn5HCIRR7yPVU/2fEuiZ73R92MU4qI3djHfYP7NJg==",202195 "dependencies": [203196 "@opentelemetry/api",204197 "@opentelemetry/api-logs",205- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",198+ "@opentelemetry/core",206199 "@opentelemetry/otlp-exporter-base",207200 "@opentelemetry/otlp-transformer",208201 "@opentelemetry/sdk-logs"
@@ -212,10 +205,10 @@
212205 "integrity": "sha512-DhWjvj0PUPFwFnhOEivpum8sJzj6FTuyx88zff+oHVLUhfd6cLyw4AIai/F4j0PZqYZBFuMT/OTMUd9wdXnBEQ==",213206 "dependencies": [214207 "@opentelemetry/api",215- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",208+ "@opentelemetry/core",216209 "@opentelemetry/otlp-exporter-base",217210 "@opentelemetry/otlp-transformer",218- "@opentelemetry/resources@2.7.1_@opentelemetry+api@1.9.1",211+ "@opentelemetry/resources",219212 "@opentelemetry/sdk-trace-base"220213 ]221214 },
@@ -223,7 +216,7 @@
223216 "integrity": "sha512-sSnvb5f+FYa4mfYxj03rmmUh+aDwo3jok62dgIWUDw8ZCUPzEbgtv/YhZyKUSlKNNey7Uc5xmJgmtTLLIV6UDQ==",224217 "dependencies": [225218 "@opentelemetry/api",226- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",219+ "@opentelemetry/core",227220 "@opentelemetry/otlp-transformer"228221 ]229222 },
@@ -232,8 +225,8 @@
232225 "dependencies": [233226 "@opentelemetry/api",234227 "@opentelemetry/api-logs",235- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",236- "@opentelemetry/resources@2.7.1_@opentelemetry+api@1.9.1",228+ "@opentelemetry/core",229+ "@opentelemetry/resources",237230 "@opentelemetry/sdk-logs",238231 "@opentelemetry/sdk-metrics",239232 "@opentelemetry/sdk-trace-base",
@@ -244,15 +237,7 @@
244237 "integrity": "sha512-DeT6KKolmC4e/dRQvMQ/RwlnzhaqeiFOXY5ngoOPJ07GgVVKxZOg9EcrNZb5aTzUn+iCrJldAgOfQm1O/QfPAQ==",245238 "dependencies": [246239 "@opentelemetry/api",247- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",248- "@opentelemetry/semantic-conventions"249- ]250- },251- "@opentelemetry/resources@2.9.0_@opentelemetry+api@1.9.1": {252- "integrity": "sha512-jyA5MBLQ+Dkl3+JsZkUoUvL7yHvU64kLsvpXKarWm6347Sl1t1bXFTFykUePNpT5WH5pm9a2Qtt03iIYQhZ1Fg==",253- "dependencies": [254- "@opentelemetry/api",255- "@opentelemetry/core@2.9.0_@opentelemetry+api@1.9.1",240+ "@opentelemetry/core",256241 "@opentelemetry/semantic-conventions"257242 ]258243 },
@@ -261,8 +246,8 @@
261246 "dependencies": [262247 "@opentelemetry/api",263248 "@opentelemetry/api-logs",264- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",265- "@opentelemetry/resources@2.7.1_@opentelemetry+api@1.9.1",249+ "@opentelemetry/core",250+ "@opentelemetry/resources",266251 "@opentelemetry/semantic-conventions"267252 ]268253 },
@@ -270,16 +255,16 @@
270255 "integrity": "sha512-MpDJdkiFDs3Pm1RHO3KByuZbuBdJEXEAkiC0+yJdsZGVCdf1RpHR6n+LHDcS7ffmfrt5kVCzJSCfm4z2C7v0uQ==",271256 "dependencies": [272257 "@opentelemetry/api",273- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",274- "@opentelemetry/resources@2.7.1_@opentelemetry+api@1.9.1"258+ "@opentelemetry/core",259+ "@opentelemetry/resources"275260 ]276261 },277262 "@opentelemetry/sdk-trace-base@2.7.1_@opentelemetry+api@1.9.1": {278263 "integrity": "sha512-NAYIlsF8MPUsKqJMiDQJTMPOmlbawC1Iz/omMLygZ1C9am8fTKYjTaI+OZM+WTY3t3Glo0wnOg/6/pac6RGPPw==",279264 "dependencies": [280265 "@opentelemetry/api",281- "@opentelemetry/core@2.7.1_@opentelemetry+api@1.9.1",282- "@opentelemetry/resources@2.7.1_@opentelemetry+api@1.9.1",266+ "@opentelemetry/core",267+ "@opentelemetry/resources",283268 "@opentelemetry/semantic-conventions"284269 ]285270 },
@@ -386,7 +371,7 @@
386371 "@opentelemetry/api",387372 "@opentelemetry/exporter-logs-otlp-http",388373 "@opentelemetry/exporter-trace-otlp-http",389- "@opentelemetry/resources@2.9.0_@opentelemetry+api@1.9.1",374+ "@opentelemetry/resources",390375 "@opentelemetry/sdk-logs",391376 "@opentelemetry/sdk-trace-base",392377 "@scure/bip39",
@@ -452,6 +437,7 @@
452437 },453438 "workspace": {454439 "dependencies": [440+ "jsr:@std/assert@1",455441 "npm:jazz-napi@2.0.0-alpha.53",456442 "npm:jazz-tools@2.0.0-alpha.53"457443 ]
A docs/hosting-lofi-apps.md +66 -0
@@ -0,0 +1,66 @@
1+# Hosting a lofi app on your own sync node2+3+lofi-node is the first-class way to self-host the sync backend for a4+[lofi](https://github.com/FelineStateMachine/lofi) app: the browser keeps speaking Jazz's protocol5+(`adapter: "jazz"` untouched); only the server URL changes. This walkthrough was validated end to6+end against lofi's reference app (`apps/reference`) with real Chromium clients.7+8+## 1. Host a node9+10+```sh11+lofi-node init --dir ./data --port 4802 # prints the app id (a UUID)12+lofi-node start --dir ./data # Jazz URL + pairing ticket13+```14+15+`data/config.json` holds the app id and the admin secret you'll need next.16+17+## 2. Deploy the app's schema to the node18+19+A lofi app's schema must be published to the sync server it uses (the managed dev server does this20+automatically; a self-hosted node needs it once — and on every schema change). From the app21+directory, use the same schema-project deploy the lofi dev flow uses:22+23+```ts24+// deploy-schema.ts (run: deno run -A deploy-schema.ts)25+import { deploy } from "./node_modules/.deno/jazz-tools@2.0.0-alpha.53/node_modules/jazz-tools/dist/dev/catalogue-project.js";26+await deploy({27+ serverUrl: "http://127.0.0.1:4802",28+ appId: "<app id from config.json>",29+ adminSecret: "<admin secret from config.json>",30+ schemaDir: "./src",31+});32+```33+34+> The schemaDir-project deploy is not yet a public jazz-tools export — a `lofi-node deploy` CLI verb35+> wrapping this is a natural follow-up once jazz-tools exposes it (or lofi's provision command grows36+> a self-host mode).37+38+## 3. Build the app against the node39+40+```sh41+JAZZ_APP_ID=<app id> JAZZ_SERVER_URL=http://127.0.0.1:4802 deno task build42+deno task preview43+```44+45+`JAZZ_SERVER_URL` must be http(s) — lofi's preflight rejects ws URLs; the browser client derives the46+WebSocket endpoint itself (`/apps/<uuid>/ws`). For LAN/production use an https reverse proxy in47+front of the node's port; installed PWAs require a secure origin.48+49+## Validation status (2026-07-18, jazz-tools 2.0.0-alpha.53)50+51+- Reference app **builds, boots, and serves** against a lofi-node backend; two real Chromium clients52+ (lofi's `createTwoClientFixture`) load the app, hydrate the store, go offline, and edit53+ independently.54+- lofi's opt-in browser convergence gate (`tests/convergence_e2e_test.ts`) currently fails at the55+ convergence stage with `CatalogueWriteDenied` from the server when the second (cloned identity)56+ client reconnects — **identically against lofi-node and against lofi's own managed dev server**57+ (controlled experiment, same error, same catalogue object id). lofi-node has exact behavior parity58+ with the first-party local server; the denial is a jazz-tools alpha.53 local-server behavior,59+ upstream of this project. Headless two-client convergence (jazz-tools `createDb` clients, same60+ account, concurrent offline edits) passes against lofi-node — see `tests/convergence_test.ts`.61+62+## Pairing two homes63+64+On the second node: `lofi-node pair <ticket from the first>` (or `SyncNode.pair(ticket)` at65+runtime). Catalogue reads/writes and sync then flow to the peer over iroh — deploying the schema to66+either node lands on the root through the tunnel's HTTP proxy.
M docs/port-iroh-js.md +84 -94
@@ -2,138 +2,128 @@
22 33 ## Why44 5-lofi-node currently binds iroh through `../db-iroh-ffi` — a hand-written flat6-C ABI built for the Doorbearer game. It works (all fixtures green), but it is7-the wrong long-term base for a published package:8-9-- **Provenance**: db-iroh-ffi's surface is game-shaped (docs determinism10- hinge, encrypted lineage store, gossip barriers) and lofi-node uses none of11- it — only connections + framed messages. A reviewer of @nzip/lofi-node12- cannot tell what the FFI is, where it came from, or why it exists.13-- **Upstream tracking**: n0-computer/iroh-ffi is the official, actively14- hardened binding surface for iroh 1.x (v1.1.0, July 2026). A hand-rolled15- ABI diverges forever; a port hardens as upstream hardens, tag by tag.16-- **Portability**: db-iroh-ffi is a sibling-checkout git dependency; lofi-node17- should build and vendor its own native layer.18-19-Upstream ships two binding crates: a uniffi crate (Swift/Kotlin/Python) and20-`iroh-js/` — a **napi-rs** crate (napi 3, async) whose generator emits21-`index.js` + `index.d.ts` and per-platform prebuilds. iroh-js is the porting22-base because:5+lofi-node currently binds iroh through `../db-iroh-ffi` — a hand-written flat C ABI built for the6+Doorbearer game. It works (all fixtures green), but it is the wrong long-term base for a published7+package:8+9+- **Provenance**: db-iroh-ffi's surface is game-shaped (docs determinism hinge, encrypted lineage10+ store, gossip barriers) and lofi-node uses none of it — only connections + framed messages. A11+ reviewer of @nzip/lofi-node cannot tell what the FFI is, where it came from, or why it exists.12+- **Upstream tracking**: n0-computer/iroh-ffi is the official, actively hardened binding surface for13+ iroh 1.x (v1.1.0, July 2026). A hand-rolled ABI diverges forever; a port hardens as upstream14+ hardens, tag by tag.15+- **Portability**: db-iroh-ffi is a sibling-checkout git dependency; lofi-node should build and16+ vendor its own native layer.17+18+Upstream ships two binding crates: a uniffi crate (Swift/Kotlin/Python) and `iroh-js/` — a19+**napi-rs** crate (napi 3, async) whose generator emits `index.js` + `index.d.ts` and per-platform20+prebuilds. iroh-js is the porting base because:2321 2422 - Deno loads napi addons natively — jazz-napi already proves it in this stack.2523 - napi async fns return real Promises: no parked FFI-pool threads.26-- `Endpoint.acceptNext()` resolves `null` on `close()` — erases db-iroh-ffi's27- known accept-cancellation gap by design.28-- Upstream already exposes what we hand-rolled or missed: `EndpointTicket`29- (replaces the custom `LFN1.` codec, interoperable with the iroh ecosystem),30- `ConnectionStats` / `PathSnapshot` / watch APIs (direct-vs-relay metrics for31- `status()`), builder presets + `secretKey(bytes)` for persisted identity.24+- `Endpoint.acceptNext()` resolves `null` on `close()` — erases db-iroh-ffi's known25+ accept-cancellation gap by design.26+- Upstream already exposes what we hand-rolled or missed: `EndpointTicket` (replaces the custom27+ `LFN1.` codec, interoperable with the iroh ecosystem), `ConnectionStats` / `PathSnapshot` / watch28+ APIs (direct-vs-relay metrics for `status()`), builder presets + `secretKey(bytes)` for persisted29+ identity.3230 3331 ## Scope discipline3432 35-Take only what lofi-node uses: connections. **No** docs / blobs / gossip /36-encrypted store, **no** services, **no** android glue. If a future feature37-needs gossip-presence or blob-backups, vendor those upstream modules then,38-same procedure.33+Take only what lofi-node uses: connections. **No** docs / blobs / gossip / encrypted store, **no**34+services, **no** android glue. If a future feature needs gossip-presence or blob-backups, vendor35+those upstream modules then, same procedure.3936 4037 ## Vendoring rules (the provenance contract)4138 42-1. **Trim at module level, never inside a file.** Vendored files stay43- byte-identical to upstream wherever possible so `diff -r` against a new44- upstream tag is the whole update procedure. Only permitted edits:45- `lib.rs`/`Cargo.toml` module + dependency lists, and compile fixes forced46- by a dropped module — every such line marked `// lofi-node: <reason>`.39+1. **Trim at module level, never inside a file.** Vendored files stay byte-identical to upstream40+ wherever possible so `diff -r` against a new upstream tag is the whole update procedure. Only41+ permitted edits: `lib.rs`/`Cargo.toml` module + dependency lists, and compile fixes forced by a42+ dropped module — every such line marked `// lofi-node: <reason>`.4743 2. **Extensions live only in `src/lofi_ext.rs`.** Upstream never collides.48-3. **`native/iroh-js/UPSTREAM.md`** records: upstream repo/tag/commit,49- vendored file list, every marked delta, dropped modules and why, and the50- tag-bump procedure (fetch tag → diff → re-apply markers → run fixtures).44+3. **`native/iroh-js/UPSTREAM.md`** records: upstream repo/tag/commit, vendored file list, every45+ marked delta, dropped modules and why, and the tag-bump procedure (fetch tag → diff → re-apply46+ markers → run fixtures).5147 4. iroh-ffi is MIT/Apache-2.0 dual-licensed; vendor both licenses alongside.5248 5349 ## Module selection5450 55-| Upstream file | Verdict | Reason |56-|---|---|---|57-| `endpoint.rs` | vendor | Endpoint/Builder, connect/acceptNext/close, watch APIs, streams |58-| `key.rs` | vendor | SecretKey (`fromBytes` ← persisted `iroh.key`), EndpointId |59-| `net.rs` | vendor | EndpointAddr, EndpointTicket |60-| `relay.rs` | vendor | RelayMap/Mode/Config — required by builder presets |61-| `error.rs` | vendor | error mapping |62-| `path.rs` | vendor only if endpoint.rs requires it | PathSnapshot powers `status()`; nice-to-have |63-| `services.rs`, `android_init.rs`, `npm/*` | drop | unused / upstream's packaging |51+| Upstream file | Verdict | Reason |52+| ----------------------------------------- | -------------------------------------- | --------------------------------------------------------------- |53+| `endpoint.rs` | vendor | Endpoint/Builder, connect/acceptNext/close, watch APIs, streams |54+| `key.rs` | vendor | SecretKey (`fromBytes` ← persisted `iroh.key`), EndpointId |55+| `net.rs` | vendor | EndpointAddr, EndpointTicket |56+| `relay.rs` | vendor | RelayMap/Mode/Config — required by builder presets |57+| `error.rs` | vendor | error mapping |58+| `path.rs` | vendor only if endpoint.rs requires it | PathSnapshot powers `status()`; nice-to-have |59+| `services.rs`, `android_init.rs`, `npm/*` | drop | unused / upstream's packaging |6460 65-Dependency diet follows: `iroh`, `iroh-base`, `iroh-relay` (no default66-features), `napi`/`napi-derive`/`napi-build`, `anyhow`. Drop `iroh-services`;67-`iroh-metrics` only if endpoint.rs demands it.61+Dependency diet follows: `iroh`, `iroh-base`, `iroh-relay` (no default features),62+`napi`/`napi-derive`/`napi-build`, `anyhow`. Drop `iroh-services`; `iroh-metrics` only if63+endpoint.rs demands it.6864 6965 ## The one extension: Buffer framed I/O7066 7167 Upstream streams move bytes as `Array<number>` (`writeAll(buf:72-Array<number>)`, `readExact(...): Promise<Array<number>>`) — per-byte boxing73-plus two awaited hops per length-prefixed message is a perf trap for sync74-frames. `lofi_ext.rs` adds:68+Array<number>)`,69+`readExact(...): Promise<Array<number>>`) — per-byte boxing plus two awaited hops per70+length-prefixed message is a perf trap for sync frames. `lofi_ext.rs` adds:7571 7672 ```rust7773 #[napi] pub async fn write_frame(stream: &SendStream, data: Buffer) -> Result<()>7874 #[napi] pub async fn read_frame(stream: &RecvStream, max: u32) -> Result<Option<Buffer>>7975 ```8076 81-u32-BE length prefix + payload, one await per message, `None` on clean stream82-end. `MAX_FRAME` sized for Jazz sync payloads (16 MiB to start — measured,83-not inherited from the game's 64 MiB). If upstream later grows Buffer I/O,84-the extension deletes.77+u32-BE length prefix + payload, one await per message, `None` on clean stream end. `MAX_FRAME` sized78+for Jazz sync payloads (16 MiB to start — measured, not inherited from the game's 64 MiB). If79+upstream later grows Buffer I/O, the extension deletes.8580 8681 ## TS adapter mapping (contract-preserving)8782 88-`IrohNode`/`IrohConn` keep their current signatures — `tunnel.ts`, `node.ts`,89-and every fixture are the regression spec.83+`IrohNode`/`IrohConn` keep their current signatures — `tunnel.ts`, `node.ts`, and every fixture are84+the regression spec.9085 91-| Current (db-iroh-ffi) | Ported (iroh-js) |92-|---|---|93-| `openIrohLib()` dlopen + ABI probe | `createRequire` of the `.node` addon; version probe = addon export |94-| `IrohNode.open(lib, secret)` | `Endpoint.builder()` + `applyN0()` + `secretKey(bytes)` + bind + `setAlpns([LOFI_ALPN])` |95-| `node.id()` / `node.addr()` | `endpoint.id()` / `endpoint.addr()` |96-| `node.addAddr(bytes)` | dropped — `connect(EndpointAddr)` takes the addr directly |97-| `node.connect(addrBytes)` | `endpoint.connect(ticket.endpointAddr(), LOFI_ALPN)` → `openBi()` |98-| `node.accept()` (uncancellable) | `acceptNext()` → `accept()` → `connect()` → `acceptBi()`; **null on close = clean loop exit** |99-| `conn.sendMsg` / `conn.recvMsg` | `writeFrame(bi.send, buf)` / `readFrame(bi.recv, MAX_FRAME)` |100-| ticket `LFN1.` + postcard addr | `EndpointTicket` string (breaking, pre-publish, no migration) |101-| — | NEW: `conn.stats()` / `conn.paths()` surfaced into `SyncNodeStatus.mesh` |86+| Current (db-iroh-ffi) | Ported (iroh-js) |87+| ---------------------------------- | --------------------------------------------------------------------------------------------- |88+| `openIrohLib()` dlopen + ABI probe | `createRequire` of the `.node` addon; version probe = addon export |89+| `IrohNode.open(lib, secret)` | `Endpoint.builder()` + `applyN0()` + `secretKey(bytes)` + bind + `setAlpns([LOFI_ALPN])` |90+| `node.id()` / `node.addr()` | `endpoint.id()` / `endpoint.addr()` |91+| `node.addAddr(bytes)` | dropped — `connect(EndpointAddr)` takes the addr directly |92+| `node.connect(addrBytes)` | `endpoint.connect(ticket.endpointAddr(), LOFI_ALPN)` → `openBi()` |93+| `node.accept()` (uncancellable) | `acceptNext()` → `accept()` → `connect()` → `acceptBi()`; **null on close = clean loop exit** |94+| `conn.sendMsg` / `conn.recvMsg` | `writeFrame(bi.send, buf)` / `readFrame(bi.recv, MAX_FRAME)` |95+| ticket `LFN1.` + postcard addr | `EndpointTicket` string (breaking, pre-publish, no migration) |96+| — | NEW: `conn.stats()` / `conn.paths()` surfaced into `SyncNodeStatus.mesh` |10297 103-The `mesh: unavailable` boot-gate contract is unchanged: addon load failure →104-typed reason, Jazz stays up, pairing throws.98+The `mesh: unavailable` boot-gate contract is unchanged: addon load failure → typed reason, Jazz99+stays up, pairing throws.105100 106101 ## Build & packaging107102 108-- `native/iroh-js/`: cargo crate; `napi build --release` emits the `.node`109- plus regenerated `index.d.ts`/`index.js` (committed — they are the typed110- contract, reviewable in diffs).111-- Dev resolution order (as today): `LOFI_NODE_IROH` env → explicit option →112- `native/iroh-js` build output → `prebuilt/<triple>/`.113-- Cross-compilation: the recipe already proven in db-iroh-ffi — repo-local114- Nix shell + cargo-zigbuild (Linux x86_64/aarch64, Windows x86_64), macOS on115- a mac, `ring` TLS.103+- `native/iroh-js/`: cargo crate; `napi build --release` emits the `.node` plus regenerated104+ `index.d.ts`/`index.js` (committed — they are the typed contract, reviewable in diffs).105+- Dev resolution order (as today): `LOFI_NODE_IROH` env → explicit option → `native/iroh-js` build106+ output → `prebuilt/<triple>/`.107+- Cross-compilation: the recipe already proven in db-iroh-ffi — repo-local Nix shell +108+ cargo-zigbuild (Linux x86_64/aarch64, Windows x86_64), macOS on a mac, `ring` TLS.116109 117110 ## Phases118111 119-1. **Gate 0 — spike (blocks everything else):** build upstream iroh-js120- UNMODIFIED, load under Deno. Smoke: endpoint bind, ticket round-trip,121- loopback connect/accept, bi-stream echo, `close()` resolving a pending122- `acceptNext()` as null, watch-callback fires (threadsafe fns under Deno).112+1. **Gate 0 — spike (blocks everything else):** build upstream iroh-js UNMODIFIED, load under Deno.113+ Smoke: endpoint bind, ticket round-trip, loopback connect/accept, bi-stream echo, `close()`114+ resolving a pending `acceptNext()` as null, watch-callback fires (threadsafe fns under Deno).123115 Measure Array<number> throughput to size the extension's necessity.124-2. Vendor the trimmed crate into `native/iroh-js/`, drop modules per table,125- write `UPSTREAM.md`; same smoke passes against the trimmed build.126-3. `lofi_ext.rs` framed I/O + TS adapter behind the existing127- `IrohNode`/`IrohConn` contract; full fixture suite green (tunnel echo,128- pair, convergence ×3) with db-iroh-ffi untouched as fallback.129-4. Switch tickets to `EndpointTicket`; surface stats/paths in `status()`;130- delete the flat-C loader and the db-iroh-ffi dependency; update README.116+2. Vendor the trimmed crate into `native/iroh-js/`, drop modules per table, write `UPSTREAM.md`;117+ same smoke passes against the trimmed build.118+3. `lofi_ext.rs` framed I/O + TS adapter behind the existing `IrohNode`/`IrohConn` contract; full119+ fixture suite green (tunnel echo, pair, convergence ×3) with db-iroh-ffi untouched as fallback.120+4. Switch tickets to `EndpointTicket`; surface stats/paths in `status()`; delete the flat-C loader121+ and the db-iroh-ffi dependency; update README.131122 5. Prebuilt cross-compile artifacts + resolution order finalized.132123 133124 ## What this dissolves134125 135-- The "reconcile db-iroh-ffi before publishing" gate — lofi-node no longer136- depends on it; db-iroh-ffi remains Doorbearer's, unreconciled.137-- The accept-cancellation upstream ask, the nonblocking-FFI thread pool, the138- custom ticket codec, and the ABI-version handshake (napi addon exports139- replace it).126+- The "reconcile db-iroh-ffi before publishing" gate — lofi-node no longer depends on it;127+ db-iroh-ffi remains Doorbearer's, unreconciled.128+- The accept-cancellation upstream ask, the nonblocking-FFI thread pool, the custom ticket codec,129+ and the ABI-version handshake (napi addon exports replace it).
M native/iroh-js/UPSTREAM.md +25 -27
@@ -7,19 +7,19 @@
77 88 ## Vendored files (byte-identical to upstream unless a `// lofi-node:` marker says otherwise)99 10-| File | Status |11-|---|---|12-| `src/endpoint.rs` | 2 marked edits: `pub(crate)` on the SendStream/RecvStream tuple fields (needed by `lofi_ext`) |13-| `src/key.rs` | identical |14-| `src/net.rs` | identical |15-| `src/path.rs` | identical |16-| `src/relay.rs` | identical |17-| `src/ticket.rs` | identical |18-| `src/watch.rs` | identical (but see Known issues) |19-| `src/lib.rs` | marked edits: services module/re-export dropped; `lofi_ext` module registered |20-| `Cargo.toml` | marked edit: `iroh-services` dependency dropped |21-| `build.rs` | identical |22-| `index.d.ts` | upstream-generated, vendored as reference; services classes listed there are NOT exported by this build |10+| File | Status |11+| ----------------- | ------------------------------------------------------------------------------------------------------- |12+| `src/endpoint.rs` | 2 marked edits: `pub(crate)` on the SendStream/RecvStream tuple fields (needed by `lofi_ext`) |13+| `src/key.rs` | identical |14+| `src/net.rs` | identical |15+| `src/path.rs` | identical |16+| `src/relay.rs` | identical |17+| `src/ticket.rs` | identical |18+| `src/watch.rs` | identical (but see Known issues) |19+| `src/lib.rs` | marked edits: services module/re-export dropped; `lofi_ext` module registered |20+| `Cargo.toml` | marked edit: `iroh-services` dependency dropped |21+| `build.rs` | identical |22+| `index.d.ts` | upstream-generated, vendored as reference; services classes listed there are NOT exported by this build |2323 2424 ## Dropped (not vendored)2525
@@ -29,24 +29,22 @@
2929 3030 ## Not upstream3131 32-- `src/lofi_ext.rs` — Buffer-based length-prefixed framing (`writeFrame`,33- `readFrame`, `maxFrame`). Justified at gate 0: upstream `Array<number>`34- stream I/O measured 7.1 MiB/s loopback round-trip. Deletes if upstream35- grows Buffer stream I/O.32+- `src/lofi_ext.rs` — Buffer-based length-prefixed framing (`writeFrame`, `readFrame`, `maxFrame`).33+ Justified at gate 0: upstream `Array<number>` stream I/O measured 7.1 MiB/s loopback round-trip.34+ Deletes if upstream grows Buffer stream I/O.3635 3736 ## Known issues (upstream, observed under Deno at v1.1.0)3837 39-- **watch APIs abort the process**: `watchAddr`/`watchPaths`/… call40- `tokio::spawn` from the calling JS thread (`watch.rs:35`, "no reactor41- running") and the panic cannot unwind through napi → hard abort. The TS42- layer must never call watch APIs; poll instead. Candidate upstream report.38+- **watch APIs abort the process**: `watchAddr`/`watchPaths`/… call `tokio::spawn` from the calling39+ JS thread (`watch.rs:35`, "no reactor running") and the panic cannot unwind through napi → hard40+ abort. The TS layer must never call watch APIs; poll instead. Candidate upstream report.4341 4442 ## Update procedure (tag bump)4543 4644 1. `git clone --depth 1 --branch <newtag> https://github.com/n0-computer/iroh-ffi`47-2. `diff -r <clone>/iroh-js/src native/iroh-js/src` — every diff hunk must be48- either upstream's change (take it) or a `// lofi-node:` marker (keep it).49-3. Re-check: does upstream now have Buffer stream I/O (delete `lofi_ext`)?50- Did the watch tokio-context bug get fixed (unblock watch APIs)?51-4. Update tag/commit here; `cargo build --release`; run the full Deno fixture52- suite (`deno task test`) — it is the regression gate.45+2. `diff -r <clone>/iroh-js/src native/iroh-js/src` — every diff hunk must be either upstream's46+ change (take it) or a `// lofi-node:` marker (keep it).47+3. Re-check: does upstream now have Buffer stream I/O (delete `lofi_ext`)? Did the watch48+ tokio-context bug get fixed (unblock watch APIs)?49+4. Update tag/commit here; `cargo build --release`; run the full Deno fixture suite50+ (`deno task test`) — it is the regression gate.
M src/jazz.ts +2 -2
M src/native/addon.ts +2 -1
M src/node.ts +6 -7
M tests/codec_test.ts +1 -1
M tests/convergence_test.ts +3 -2
M tests/mesh_test.ts +1 -1
M tests/pair_test.ts +1 -1
M tests/tunnel_iroh_test.ts +1 -1