clone

a deno framework for local-first web apps that open instantly, work offline and sync when users choose

feat/141-effect-library
lofi / docs / testing.md
testing.md 10.2K rendered
1# Testing a lofi app23The generated template separates fast deterministic tests from opt-in browser scenarios that need a4running app, Chromium, and—for convergence—managed sync. Framework release checks also exercise5recoverable accounts and access policies; generated applications reuse those package-owned seams.67```mermaid8flowchart TB9 Change["Application change"] --> Fast["deno task test<br/>deterministic suite"]10 Fast --> Build["build + preview<br/>production shell"]11 Build --> Manual["reload and offline checks"]12 Build --> Browser["two-client convergence<br/>opt-in Playwright"]13 Manual --> Device["physical iOS and Android"]14 Browser --> Device15 Device --> Ship["Ready to ship"]16```1718## Run the default suite1920```sh21deno task test22```2324The default suite covers application tests without launching a browser. Framework runtime contracts25run once in the `@nzip/lofi` package suite instead of being copied into every application. Keep26domain logic and permission-shape checks in this fast path when possible.2728## Simulation-test sync with scenarios2930The `@nzip/lofi/testing` scenario surface turns sync simulation testing into plain test31declarations: named peers make concurrent and offline edits through your app's own schema, and the32test asserts they converge.3334```ts35import { assertNoRow, converge, scenario } from "@nzip/lofi/testing";36import { app, permissions } from "../src/lofi/schema.ts";3738scenario("offline rename versus remote delete", { app, permissions }, async ({ alice, bob }) => {39 const doc = await alice.db.documents.insert({ title: "Untitled" });40 await converge(alice, bob);4142 await alice.offline();43 await alice.db.documents.update(doc.id, { title: "Draft" });44 await bob.db.documents.remove(doc.id);45 await alice.online();46 await alice.settle();4748 await alice.restart();49 await converge(alice, bob);50 await assertNoRow(alice, app.documents, doc.id);51 await assertNoRow(bob, app.documents, doc.id);52});53```5455Each `scenario` call registers one test that boots a real local sync server, deploys the app's56schema and permissions, creates the peers as real synced clients on a memory driver, and tears57everything down afterwards. What converges in a scenario is what converges under production sync;58nothing is mocked. The local server needs FFI, so run scenario tests explicitly with `deno test -A`59rather than inside the default fast suite.6061Each peer offers:6263- `alice.db.<table>.insert / update / remove / all / get` — the app's own tables, applied to the64 peer's local view immediately, online or offline;65- `alice.offline()` and `alice.online()` — an offline window: edits keep applying locally and upload66 on reconnect;67- `alice.settle()` — wait until every write the peer issued has been durably accepted or adjudicated68 by the sync node;69- `alice.restart()` — close and reboot the peer with the same identity, dropping session-local state70 and re-syncing from the server;71- `addPeer("carol")` (on the scenario context) — boot a fresh reader, useful for asserting the72 canonical state a new device would see.7374`converge(alice, bob)` settles the peers, then polls until every peer reads identical rows in every75table, and fails with a per-table, per-peer diff at its deadline. `assertRow`, `assertNoRow`, and76`assertRowCount` check one peer's local view. Two merge behaviors are built into the comparison:7778- counter-merged columns are compared by row presence, not value, because a live replica that79 watched a counter's history reads a different total than a fresh boot of the same account;80- a live replica that edited a row another peer concurrently deleted keeps showing its own write for81 the rest of its session, even though the delete wins for every other view. Restart that peer to82 observe the adjudicated outcome, as the example above does.8384Scenarios use real sync with scripted offline windows: op sequences are exactly reproducible, but85network timing is not, so assertions poll within deadlines rather than assuming schedules.86Transport-level fault injection (latency, dropped or reordered messages) and a virtual clock are not87part of this surface.8889### Fuzz scenarios9091`scenario.fuzz` generates a seeded sequence of inserts, updates, removes, and offline windows across92both peers, executes it, restarts the peers, and asserts that they and a fresh reader converge on93identical state:9495```ts96scenario.fuzz("fuzzed edits converge", { app, permissions, seed: 7, steps: 40 });97```9899Omit `seed` to explore: each run picks a random seed, and every failure reports the seed and the100full op trace. Replay a failure exactly by passing the reported seed (or setting `LOFI_FUZZ_SEED`).101The generator drives tables whose required columns it can fill with scalar values and skips the102rest; writes that the sync node legitimately rejects, such as updating a row the other peer already103deleted, count as outcomes rather than failures — the invariant under test is convergence.104105## Test the production build manually106107```sh108deno task build109deno task preview110```111112Then verify:1131141. Add data and reload.1152. Confirm the status reports local durability.1163. Disable the network and continue reading and writing.1174. Reload while offline to exercise the production service worker.1185. Restore the network and confirm the local data remains.119120The service worker is intentionally disabled during `deno task dev`; use a production build when121testing offline shell startup.122123## Run the two-client convergence example124125First configure managed sync and start the app:126127```sh128deno task jazz:provision129deno task dev130```131132In another terminal, point the included browser example at the printed URL:133134```sh135LOFI_E2E_BASE_URL=http://127.0.0.1:4321/ \136 deno test -A tests/convergence_e2e_test.ts137```138139```mermaid140sequenceDiagram141 participant A as Client A142 participant B as Client B143 participant S as Managed sync144145 A->>A: Go offline and edit146 B->>B: Go offline and edit147 A->>S: Reconnect and upload148 B->>S: Reconnect and upload149 S-->>A: Both edits150 S-->>B: Both edits151 A->>A: Assert convergence152 B->>B: Assert convergence153```154155Both browser contexts use one test identity. Without `LOFI_E2E_BASE_URL`, the example skips so the156default suite remains fast.157158Two more opt-in browser gates ship with the starter:159160- `tests/backup_migration_e2e_test.ts` — the account journey in one real browser: rows written161 local-only must survive electing backup and sync (the runtime copies them into the managed162 namespace during the reload, settling at local durability, so the sync server does not need to be163 reachable), and the phrase-reveal guard must accept only its enrolled passkey — after the164 authenticator loses its credentials, the reveal fails closed. Serve on `localhost` (a passkey165 RP-ID cannot bind to a bare IP).166- `tests/auth_e2e_test.ts` — the device-credential enroll → authenticate round-trip on a stable167 origin listed in `credentialOrigins`.168169If Chromium is missing:170171```sh172deno run -A npm:playwright@1.61.1 install chromium173```174175## Adapt the example to your UI176177The example is a `scenario.browser` declaration: the same scenario controls as the headless surface,178driving two real browser clients against your served app. Update these pieces in179`tests/convergence_e2e_test.ts`:180181- `ready` — a DOM condition proving the local store has hydrated;182- `snapshot` — a value-free view of the peer's state (counts and booleans, never user data),183 compared when settling and captured in failure artifacts;184- the body — page-mediated edits inside the offline window, then proof that both peers eventually185 render all expected results.186187Use stable accessible roles, labels, and application-owned `data-*` attributes. Avoid arbitrary188sleeps; readiness helpers retry observable conditions until their timeout.189190## Failure artifacts191192Browser fixtures can save artifacts under `test-results/`. Snapshot callbacks should contain counts,193booleans, state names, or sanitized identifiers—not task text, environment values, recovery phrases,194or other user data.195196## Framework recovery and permission evidence197198The framework repository's `deno task check` includes the access-security suite. It uses Jazz's199permission-test app for owner, recipient, unrelated-user, revoke, fixed-role, membership, removal,200and self-leave rejection cases, plus a real local Jazz server for offline grant/revoke and201membership reconciliation.202203`deno task test:golden` extends the existing generated-app runner with a two-profile journey. A204Chromium virtual authenticator creates and exports a resident credential, a fresh browser process205imports it, and the real WebAuthn assertion restores the same Jazz principal and synced rows. The206runner pins the exported credential as the assertion's allow-list because headless Chromium cannot207show its resident-credential account chooser. It also proves recovery-phrase fallback when the208credentials API is unavailable and proves that stopped sync retains the managed local replica209through a production offline reload.210211This is automated virtual-authenticator evidence, not physical iOS or Android evidence. It does not212prove passkey-provider portability, mobile installed-PWA lifecycle behavior, or transport213convergence without the local Jazz server used by the journey.214215## Physical-device checks216217Browser automation does not prove installed-PWA storage and lifecycle behavior on iOS or Android.218Before shipping, use a stable HTTPS origin and exercise installation, termination, device restart,219offline cold start, foreground recovery, and account recovery on every supported mobile surface.220221The WebAuthn PRF extension is the one credential path no virtual authenticator models, so PRF is222feature-detected in the runtime and must be validated on real hardware. The manual pass, per223supported platform/passkey-provider pair:2242251. `getAuthCapability()` reports `prf: "available"` (or `"not-reported"` with a working derive).2262. Enroll a device credential on the pinned production RP-ID and derive a PRF secret twice with the227 same salt — both ceremonies must yield the same 32 bytes.2283. Encrypt with the derived at-rest key, restart the browser/app, derive again, and decrypt.2294. Confirm a different salt yields a different secret and cannot decrypt the first blob.2305. On providers that sync passkeys, repeat the derive on a second device of the same ecosystem and231 record whether PRF results roam — do not assume they do.