clone

a self-hosted sync node for lofi apps: jazz sync over iroh, ticket-gated, with no static ip or cloud needed

feat/connection-observability
lofi-node / docs / port-iroh-js.md
port-iroh-js.md 8.5K rendered
1# Plan: replace db-iroh-ffi with a port of upstream iroh-ffi (iroh-js)23## Why45lofi-node currently binds iroh through `../db-iroh-ffi` — a hand-written flat C ABI built for the6Doorbearer game. It works (all fixtures green), but it is the wrong long-term base for a published7package:89- **Provenance**: db-iroh-ffi's surface is game-shaped (docs determinism hinge, encrypted lineage10 store, gossip barriers) and lofi-node uses none of it — only connections + framed messages. A11 reviewer of @nzip/lofi-node cannot tell what the FFI is, where it came from, or why it exists.12- **Upstream tracking**: n0-computer/iroh-ffi is the official, actively hardened binding surface for13 iroh 1.x (v1.1.0, July 2026). A hand-rolled ABI diverges forever; a port hardens as upstream14 hardens, tag by tag.15- **Portability**: db-iroh-ffi is a sibling-checkout git dependency; lofi-node should build and16 vendor its own native layer.1718Upstream ships two binding crates: a uniffi crate (Swift/Kotlin/Python) and `iroh-js/` — a19**napi-rs** crate (napi 3, async) whose generator emits `index.js` + `index.d.ts` and per-platform20prebuilds. iroh-js is the porting base because:2122- Deno loads napi addons natively — jazz-napi already proves it in this stack.23- napi async fns return real Promises: no parked FFI-pool threads.24- `Endpoint.acceptNext()` resolves `null` on `close()` — erases db-iroh-ffi's known25 accept-cancellation gap by design.26- Upstream already exposes what we hand-rolled or missed: `EndpointTicket` (replaces the custom27 `LFN1.` codec, interoperable with the iroh ecosystem), `ConnectionStats` / `PathSnapshot` / watch28 APIs (direct-vs-relay metrics for `status()`), builder presets + `secretKey(bytes)` for persisted29 identity.3031## Scope discipline3233Take only what lofi-node uses: connections. **No** docs / blobs / gossip / encrypted store, **no**34services, **no** android glue. If a future feature needs gossip-presence or blob-backups, vendor35those upstream modules then, same procedure.3637## Vendoring rules (the provenance contract)38391. **Trim at module level, never inside a file.** Vendored files stay byte-identical to upstream40 wherever possible so `diff -r` against a new upstream tag is the whole update procedure. Only41 permitted edits: `lib.rs`/`Cargo.toml` module + dependency lists, and compile fixes forced by a42 dropped module — every such line marked `// lofi-node: <reason>`.432. **Extensions live only in `src/lofi_ext.rs`.** Upstream never collides.443. **`native/iroh-js/UPSTREAM.md`** records: upstream repo/tag/commit, vendored file list, every45 marked delta, dropped modules and why, and the tag-bump procedure (fetch tag → diff → re-apply46 markers → run fixtures).474. iroh-ffi is MIT/Apache-2.0 dual-licensed; vendor both licenses alongside.4849## Module selection5051| Upstream file | Verdict | Reason |52| ----------------------------------------- | -------------------------------------- | --------------------------------------------------------------- |53| `endpoint.rs` | vendor | Endpoint/Builder, connect/acceptNext/close, watch APIs, streams |54| `key.rs` | vendor | SecretKey (`fromBytes` ← persisted `iroh.key`), EndpointId |55| `net.rs` | vendor | EndpointAddr, EndpointTicket |56| `relay.rs` | vendor | RelayMap/Mode/Config — required by builder presets |57| `error.rs` | vendor | error mapping |58| `path.rs` | vendor only if endpoint.rs requires it | PathSnapshot powers `status()`; nice-to-have |59| `services.rs`, `android_init.rs`, `npm/*` | drop | unused / upstream's packaging |6061Dependency diet follows: `iroh`, `iroh-base`, `iroh-relay` (no default features),62`napi`/`napi-derive`/`napi-build`, `anyhow`. Drop `iroh-services`; `iroh-metrics` only if63endpoint.rs demands it.6465## The one extension: Buffer framed I/O6667Upstream streams move bytes as `Array<number>` (`writeAll(buf:68Array<number>)`,69`readExact(...): Promise<Array<number>>`) — per-byte boxing plus two awaited hops per70length-prefixed message is a perf trap for sync frames. `lofi_ext.rs` adds:7172```rust73#[napi] pub async fn write_frame(stream: &SendStream, data: Buffer) -> Result<()>74#[napi] pub async fn read_frame(stream: &RecvStream, max: u32) -> Result<Option<Buffer>>75```7677u32-BE length prefix + payload, one await per message, `None` on clean stream end. `MAX_FRAME` sized78for Jazz sync payloads (16 MiB to start — measured, not inherited from the game's 64 MiB). If79upstream later grows Buffer I/O, the extension deletes.8081## TS adapter mapping (contract-preserving)8283`IrohNode`/`IrohConn` keep their current signatures — `tunnel.ts`, `node.ts`, and every fixture are84the regression spec.8586| Current (db-iroh-ffi) | Ported (iroh-js) |87| ---------------------------------- | --------------------------------------------------------------------------------------------- |88| `openIrohLib()` dlopen + ABI probe | `createRequire` of the `.node` addon; version probe = addon export |89| `IrohNode.open(lib, secret)` | `Endpoint.builder()` + `applyN0()` + `secretKey(bytes)` + bind + `setAlpns([LOFI_ALPN])` |90| `node.id()` / `node.addr()` | `endpoint.id()` / `endpoint.addr()` |91| `node.addAddr(bytes)` | dropped — `connect(EndpointAddr)` takes the addr directly |92| `node.connect(addrBytes)` | `endpoint.connect(ticket.endpointAddr(), LOFI_ALPN)` → `openBi()` |93| `node.accept()` (uncancellable) | `acceptNext()` → `accept()` → `connect()` → `acceptBi()`; **null on close = clean loop exit** |94| `conn.sendMsg` / `conn.recvMsg` | `writeFrame(bi.send, buf)` / `readFrame(bi.recv, MAX_FRAME)` |95| ticket `LFN1.` + postcard addr | `EndpointTicket` string (breaking, pre-publish, no migration) |96| — | NEW: `conn.stats()` / `conn.paths()` surfaced into `SyncNodeStatus.mesh` |9798The `mesh: unavailable` boot-gate contract is unchanged: addon load failure → typed reason, Jazz99stays up, pairing throws.100101## Build & packaging102103- `native/iroh-js/`: cargo crate; `napi build --release` emits the `.node` plus regenerated104 `index.d.ts`/`index.js` (committed — they are the typed contract, reviewable in diffs).105- Dev resolution order (as today): `LOFI_NODE_IROH` env → explicit option → `native/iroh-js` build106 output → `prebuilt/<triple>/`.107- Cross-compilation: the recipe already proven in db-iroh-ffi — repo-local Nix shell +108 cargo-zigbuild (Linux x86_64/aarch64, Windows x86_64), macOS on a mac, `ring` TLS.109110## Phases1111121. **Gate 0 — spike (blocks everything else):** build upstream iroh-js UNMODIFIED, load under Deno.113 Smoke: endpoint bind, ticket round-trip, loopback connect/accept, bi-stream echo, `close()`114 resolving a pending `acceptNext()` as null, watch-callback fires (threadsafe fns under Deno).115 Measure Array<number> throughput to size the extension's necessity.1162. Vendor the trimmed crate into `native/iroh-js/`, drop modules per table, write `UPSTREAM.md`;117 same smoke passes against the trimmed build.1183. `lofi_ext.rs` framed I/O + TS adapter behind the existing `IrohNode`/`IrohConn` contract; full119 fixture suite green (tunnel echo, pair, convergence ×3) with db-iroh-ffi untouched as fallback.1204. Switch tickets to `EndpointTicket`; surface stats/paths in `status()`; delete the flat-C loader121 and the db-iroh-ffi dependency; update README.1225. Prebuilt cross-compile artifacts + resolution order finalized.123124## What this dissolves125126- The "reconcile db-iroh-ffi before publishing" gate — lofi-node no longer depends on it;127 db-iroh-ffi remains Doorbearer's, unreconciled.128- The accept-cancellation upstream ask, the nonblocking-FFI thread pool, the custom ticket codec,129 and the ABI-version handshake (napi addon exports replace it).