Replaces the db-iroh-ffi prove-out binding with a trimmed, provenance- tracked vendor of n0-computer/iroh-ffi's napi-rs crate: connections modules only, one Buffer-framing extension, UPSTREAM.md contract, gate-0 Deno smoke before any porting.
- commit
- 9fc652251c80dbb05053dd9a9639d94dc49afed8
- parent
- 765ec197acb4203cd9efae6037bc4e9fda8b91e1
- author
- Dami <dami.etoile@gmail.com>
| M | README.md | 24 | ++--- |
| A | docs/ | 139 | ++++++++++++++++++++++++++++++ |
@@ -62,21 +62,15 @@ node.ticket(); // -> share with the peer
6262 consuming lofi app pins (`2.0.0-alpha.53`). Wire compat across alphas is not6363 guaranteed; bump in lockstep.6464 65-## db-iroh-ffi reconciliation list (before publishing)66-67-Found while consuming the crate generically (tracked for the de-game-ification68-pass in the db-iroh-ffi repo):69-70-1. **`db_accept` cancellation** — nothing wakes a parked accept; `db_node_close`71- removes the handle but `incoming.pop()` never resolves. A daemon can only72- tear down by process exit. Wants `db_node_wake`/accept-with-deadline.73-2. **Env var / naming** — `DOORBEARER_IROH` → neutral (we use `LOFI_NODE_IROH`);74- crate/symbol prefix `db_` reads as "Doorbearer"; fine to keep, but README75- framing ("the game") should go generic.76-3. **Prebuilt distribution** — versioned release artifacts with checksums77- (in-repo prebuilts are fine for submodules, not for JSR consumers).78-4. **`MAX_FRAME`/`CHUNK` tuning** are sized to game payloads (64 MiB cap is79- plenty for Jazz sync frames; just document the contract).65+## Native layer roadmap66+67+db-iroh-ffi is the prove-out binding, not the destination. The plan68+([docs/port-iroh-js.md](docs/port-iroh-js.md)) replaces it with a trimmed69+vendor of upstream **n0-computer/iroh-ffi's `iroh-js`** (napi-rs) built in70+this repo: connections surface only, one Buffer-framing extension module,71+`UPSTREAM.md` provenance, upstream tag tracking. That dissolves the old72+db-iroh-ffi reconciliation list (accept cancellation comes free from73+`acceptNext() → null` on close; tickets become upstream `EndpointTicket`s).8074 8175 ## Tests8276 @@ -0,0 +1,139 @@
1+# Plan: replace db-iroh-ffi with a port of upstream iroh-ffi (iroh-js)2+3+## Why4+5+lofi-node currently binds iroh through `../db-iroh-ffi` — a hand-written flat6+C ABI built for the Doorbearer game. It works (all fixtures green), but it is7+the wrong long-term base for a published package:8+9+- **Provenance**: db-iroh-ffi's surface is game-shaped (docs determinism10+ hinge, encrypted lineage store, gossip barriers) and lofi-node uses none of11+ it — only connections + framed messages. A reviewer of @nzip/lofi-node12+ cannot tell what the FFI is, where it came from, or why it exists.13+- **Upstream tracking**: n0-computer/iroh-ffi is the official, actively14+ hardened binding surface for iroh 1.x (v1.1.0, July 2026). A hand-rolled15+ ABI diverges forever; a port hardens as upstream hardens, tag by tag.16+- **Portability**: db-iroh-ffi is a sibling-checkout git dependency; lofi-node17+ should build and vendor its own native layer.18+19+Upstream ships two binding crates: a uniffi crate (Swift/Kotlin/Python) and20+`iroh-js/` — a **napi-rs** crate (napi 3, async) whose generator emits21+`index.js` + `index.d.ts` and per-platform prebuilds. iroh-js is the porting22+base because:23+24+- Deno loads napi addons natively — jazz-napi already proves it in this stack.25+- napi async fns return real Promises: no parked FFI-pool threads.26+- `Endpoint.acceptNext()` resolves `null` on `close()` — erases db-iroh-ffi's27+ known accept-cancellation gap by design.28+- Upstream already exposes what we hand-rolled or missed: `EndpointTicket`29+ (replaces the custom `LFN1.` codec, interoperable with the iroh ecosystem),30+ `ConnectionStats` / `PathSnapshot` / watch APIs (direct-vs-relay metrics for31+ `status()`), builder presets + `secretKey(bytes)` for persisted identity.32+33+## Scope discipline34+35+Take only what lofi-node uses: connections. **No** docs / blobs / gossip /36+encrypted store, **no** services, **no** android glue. If a future feature37+needs gossip-presence or blob-backups, vendor those upstream modules then,38+same procedure.39+40+## Vendoring rules (the provenance contract)41+42+1. **Trim at module level, never inside a file.** Vendored files stay43+ byte-identical to upstream wherever possible so `diff -r` against a new44+ upstream tag is the whole update procedure. Only permitted edits:45+ `lib.rs`/`Cargo.toml` module + dependency lists, and compile fixes forced46+ by a dropped module — every such line marked `// lofi-node: <reason>`.47+2. **Extensions live only in `src/lofi_ext.rs`.** Upstream never collides.48+3. **`native/iroh-js/UPSTREAM.md`** records: upstream repo/tag/commit,49+ vendored file list, every marked delta, dropped modules and why, and the50+ tag-bump procedure (fetch tag → diff → re-apply markers → run fixtures).51+4. iroh-ffi is MIT/Apache-2.0 dual-licensed; vendor both licenses alongside.52+53+## Module selection54+55+| Upstream file | Verdict | Reason |56+|---|---|---|57+| `endpoint.rs` | vendor | Endpoint/Builder, connect/acceptNext/close, watch APIs, streams |58+| `key.rs` | vendor | SecretKey (`fromBytes` ← persisted `iroh.key`), EndpointId |59+| `net.rs` | vendor | EndpointAddr, EndpointTicket |60+| `relay.rs` | vendor | RelayMap/Mode/Config — required by builder presets |61+| `error.rs` | vendor | error mapping |62+| `path.rs` | vendor only if endpoint.rs requires it | PathSnapshot powers `status()`; nice-to-have |63+| `services.rs`, `android_init.rs`, `npm/*` | drop | unused / upstream's packaging |64+65+Dependency diet follows: `iroh`, `iroh-base`, `iroh-relay` (no default66+features), `napi`/`napi-derive`/`napi-build`, `anyhow`. Drop `iroh-services`;67+`iroh-metrics` only if endpoint.rs demands it.68+69+## The one extension: Buffer framed I/O70+71+Upstream streams move bytes as `Array<number>` (`writeAll(buf:72+Array<number>)`, `readExact(...): Promise<Array<number>>`) — per-byte boxing73+plus two awaited hops per length-prefixed message is a perf trap for sync74+frames. `lofi_ext.rs` adds:75+76+```rust77+#[napi] pub async fn write_frame(stream: &SendStream, data: Buffer) -> Result<()>78+#[napi] pub async fn read_frame(stream: &RecvStream, max: u32) -> Result<Option<Buffer>>79+```80+81+u32-BE length prefix + payload, one await per message, `None` on clean stream82+end. `MAX_FRAME` sized for Jazz sync payloads (16 MiB to start — measured,83+not inherited from the game's 64 MiB). If upstream later grows Buffer I/O,84+the extension deletes.85+86+## TS adapter mapping (contract-preserving)87+88+`IrohNode`/`IrohConn` keep their current signatures — `tunnel.ts`, `node.ts`,89+and every fixture are the regression spec.90+91+| Current (db-iroh-ffi) | Ported (iroh-js) |92+|---|---|93+| `openIrohLib()` dlopen + ABI probe | `createRequire` of the `.node` addon; version probe = addon export |94+| `IrohNode.open(lib, secret)` | `Endpoint.builder()` + `applyN0()` + `secretKey(bytes)` + bind + `setAlpns([LOFI_ALPN])` |95+| `node.id()` / `node.addr()` | `endpoint.id()` / `endpoint.addr()` |96+| `node.addAddr(bytes)` | dropped — `connect(EndpointAddr)` takes the addr directly |97+| `node.connect(addrBytes)` | `endpoint.connect(ticket.endpointAddr(), LOFI_ALPN)` → `openBi()` |98+| `node.accept()` (uncancellable) | `acceptNext()` → `accept()` → `connect()` → `acceptBi()`; **null on close = clean loop exit** |99+| `conn.sendMsg` / `conn.recvMsg` | `writeFrame(bi.send, buf)` / `readFrame(bi.recv, MAX_FRAME)` |100+| ticket `LFN1.` + postcard addr | `EndpointTicket` string (breaking, pre-publish, no migration) |101+| — | NEW: `conn.stats()` / `conn.paths()` surfaced into `SyncNodeStatus.mesh` |102+103+The `mesh: unavailable` boot-gate contract is unchanged: addon load failure →104+typed reason, Jazz stays up, pairing throws.105+106+## Build & packaging107+108+- `native/iroh-js/`: cargo crate; `napi build --release` emits the `.node`109+ plus regenerated `index.d.ts`/`index.js` (committed — they are the typed110+ contract, reviewable in diffs).111+- Dev resolution order (as today): `LOFI_NODE_IROH` env → explicit option →112+ `native/iroh-js` build output → `prebuilt/<triple>/`.113+- Cross-compilation: the recipe already proven in db-iroh-ffi — repo-local114+ Nix shell + cargo-zigbuild (Linux x86_64/aarch64, Windows x86_64), macOS on115+ a mac, `ring` TLS.116+117+## Phases118+119+1. **Gate 0 — spike (blocks everything else):** build upstream iroh-js120+ UNMODIFIED, load under Deno. Smoke: endpoint bind, ticket round-trip,121+ loopback connect/accept, bi-stream echo, `close()` resolving a pending122+ `acceptNext()` as null, watch-callback fires (threadsafe fns under Deno).123+ Measure Array<number> throughput to size the extension's necessity.124+2. Vendor the trimmed crate into `native/iroh-js/`, drop modules per table,125+ write `UPSTREAM.md`; same smoke passes against the trimmed build.126+3. `lofi_ext.rs` framed I/O + TS adapter behind the existing127+ `IrohNode`/`IrohConn` contract; full fixture suite green (tunnel echo,128+ pair, convergence ×3) with db-iroh-ffi untouched as fallback.129+4. Switch tickets to `EndpointTicket`; surface stats/paths in `status()`;130+ delete the flat-C loader and the db-iroh-ffi dependency; update README.131+5. Prebuilt cross-compile artifacts + resolution order finalized.132+133+## What this dissolves134+135+- The "reconcile db-iroh-ffi before publishing" gate — lofi-node no longer136+ depends on it; db-iroh-ffi remains Doorbearer's, unreconciled.137+- The accept-cancellation upstream ask, the nonblocking-FFI thread pool, the138+ custom ticket codec, and the ABI-version handshake (napi addon exports139+ replace it).