- commit
- b49c7377f07ed5a667cd778d37e9342ff5be4792
- parent
- 5f993f10bd076415c31da73fc90525e7f0453467
- author
- Dami <dami.etoile@gmail.com>
| M | AGENTS.md | 215 | +++++++++++++++++++++++------- |
| M | README.md | 21 | ++- |
@@ -1,50 +1,171 @@
1-# Sigil agent guide2-3-Sigil is a Tauri v2 remote desktop app with FIDO2 key authentication over Iroh P2P. The app is built and functional — spikes are done, the core flow works end to end.4-5-## Current focus6-7-The project is past the spike/validation phase. ffmpeg subprocess is now the primary capture+encode path (xcap+openh264 is the fallback), achieving ~28 fps with NVENC (up from ~1–2 fps). The bottleneck has moved to network/relay throughput and client-side decode.8-9-Active work areas:10-11-- **Codec configurability** — H.264, H.265, and AV1 are all supported. Users can select codec + encoder backend (nvenc, vaapi, qsv, amf, videotoolbox, software) in the info panel.12-- **Multi-codec WebCodecs decode** — frontend handles H.264 (AVCC), H.265 (hvcC), and AV1 (OBU) for hardware-accelerated decode.13-- **Connection tracking** — fixed with `conn.closed()` future + `tokio::select!` for reliable disconnect detection; connection counter now properly decrements.14-- **Wire protocol** — header expanded to 14 bytes: `width(4) + height(4) + size(4) + keyframe(1) + codec(1)`. Codec byte: 0=h264, 1=h265, 2=av1.15-- **Client-side stats** — info panel shows codec received, dropped frames, total frames received when connected as client. Encoder config and host performance sections are hidden when connected.16-- **Remaining performance work** — network/relay throughput, client-side decode efficiency.17-18-## Key files19-20-- `src-tauri/src/commands.rs` — all backend logic (FIDO2 key derivation, Iroh networking, screen capture, H.264 encoding, input injection)21-- `src/index.html` — the entire frontend (single-file UI)22-23-## Tech notes24-25-- **Rust edition 2024** — requires Rust 1.85+.26-- **Iroh 1.0 API** — uses the latest Iroh endpoint, protocol, and ticket APIs. Older Iroh 0.x code will not compile.27-- Always run `source ~/.cargo/env` before any `cargo` command.28-- On Linux with NVIDIA, set `WEBKIT_DISABLE_DMABUF_RENDERER=1` before running or building.1+# goq agent guide2+3+goq is a one-host, one-user game-streaming appliance. The public project and4+website are **goq.sh**. **Sigil** is the pure Rust Linux host command and5+daemon. **Portal** is the installed Tauri client that opens a session to a6+Sigil host.7+8+## Product boundary9+10+- Sigil runs on a dedicated, physically headless bare-metal Linux machine.11+- Gamescope owns the private Steam/game display; capture must not require an12+ interactive desktop or XDG ScreenCast portal.13+- The host is a pure Rust daemon. It must never depend on Tauri, WebKit, or a14+ host-facing UI.15+- Portal is client-only. It must not bundle host capture, encoding, daemon,16+ identity, configuration, or Linux input-injection assets.17+- Iroh owns native connectivity, endpoint identity, encryption, direct-path18+ discovery, and relay fallback.19+- Exactly one media session and one matching client are active at a time.20+- Controller is the primary interaction model. Keyboard and relative mouse are21+ secondary inputs.22+23+## Current architecture24+25+- `crates/sigil-protocol` owns bounded, versioned handshakes, capabilities,26+ media/audio headers, input messages, protocol limits, and ALPN constants.27+- `crates/sigil-host` builds the `sigil` daemon and `sigil-probe` diagnostic.28+ - `main.rs` owns the CLI, endpoint/router setup, and capture probes.29+ - `server.rs` owns session admission and separate media, input, and audio30+ protocols.31+ - `source.rs` owns test-pattern and Gamescope PipeWire capture plus AMD GstVA32+ H.264 encoding.33+ - `input.rs` owns strict Linux `uinput` pointer, keyboard, and Xbox-style34+ virtual gamepad devices.35+ - `audio.rs` owns bounded PipeWire/Opus capture.36+ - `identity.rs` and `config.rs` own fail-closed persistent host state.37+- `src-tauri` is Portal's native boundary.38+ - `commands/network.rs` owns Iroh connections, bounded media-object receive,39+ binary Tauri channels, input transport, and diagnostics.40+ - `commands/auth.rs` owns FIDO2 `hmac-secret` derivation.41+ - `commands/state.rs` owns launch options and process-wide bounded state.42+- `src/main.js` and the focused `src/*.mjs` modules own WebCodecs decode,43+ AudioWorklet playback, controller-first navigation, input capture, A/V44+ synchronization, window geometry, and client diagnostics.45+- `website/` is the static goq.sh site. A merge to `main` publishes it through46+ Cloudflare Pages after the static-site gate passes.47+48+## Proven baseline49+50+- Sigil is independent of Tauri/WebKit and passes the pure-host dependency51+ gate.52+- Gamescope's exact PipeWire node is resolved from strict configured53+ properties. AMD GstVA H.264 sustains the fixed 1280x800/60 first target on54+ the GPD Pocket 4 test host with bounded post-encode delivery.55+- Media v2 uses one bounded, prioritized QUIC object stream per encoded frame.56+ Stale dependent objects are cancelled; discontinuity keyframes provide a57+ latest-frame barrier. Media v1 remains a compatibility fallback.58+- Portal crosses Rust-to-webview video and audio through bounded binary Tauri59+ channels. It reports transport, frontend, decoder, presentation, and audio60+ queue/drop timing separately.61+- Input uses an Iroh connection independent of media backpressure. The host62+ advertises only operational capabilities and neutralizes every held pointer,63+ keyboard, and gamepad state when a session ends.64+- Relative pointer movement, scroll, keyboard, virtual gamepad reports, Opus65+ audio, reconnects, and second-client rejection have focused and loopback66+ coverage. Hardware acceptance that remains incomplete is listed below.67+- Portal's window scales to the incoming stream while preserving aspect ratio;68+ larger client screens do not stretch the host image.69+70+## Security boundary71+72+- The normal inherited FIDO flow discovers the host identity; it is not yet73+ host-side client authorization. A peer that already knows the node ID and74+ ALPN can currently attempt a connection.75+- `--dev-connect` is a test-only routing bypass, never an authorization claim.76+ Debug builds and the explicit `demo-direct-node` feature may accept it and77+ must display the development warning. Ordinary release builds must reject it.78+- Production authorization remains tracked in issue #3. The intended UX is a79+ one-time, short-lived, peer-bound capability enrollment; normal startup must80+ remain **PIN -> tap -> play**.81+- View, pointer/keyboard, and gamepad permissions must be independently82+ grantable. Replay, expiry, peer mismatch, cross-host use, and capability83+ escalation must fail closed.84+- Treat node IDs and connection metadata as operationally sensitive until that85+ authorization boundary lands.86+87+## Release and installation boundary88+89+- Sigil is machine setup, not a desktop download. The public bootstrap is90+ `curl -fsSL https://goq.sh/install-sigil | bash`.91+- `website/install-sigil` intentionally fails closed until a reviewed Minisign92+ publisher key and immutable signed release tag are configured.93+- Product Sigil packages must come from clean committed source, contain only94+ the allowlisted runtime payload, carry complete checksums/provenance, and have95+ a detached Minisign signature. Installation must preserve identity and host96+ configuration and must not silently start, restart, or enable the service.97+- Portal is a compiled desktop download, never a shell install. The first98+ public target is macOS arm64 and requires Developer ID signing, hardened99+ runtime, notarization, stapling, and strict Gatekeeper verification. Do not100+ advertise an unavailable platform/architecture as a download.101+- The Minisign secret, Apple certificate, notarization credentials, host102+ identity, and FIDO-derived secrets must never enter the repository, release103+ archive, logs, command line, or general website deployment environment.104+105+## Latency and correctness invariants106+107+- Bound every capture, encode, media, frontend, decode, audio, and input queue.108+- Prefer dropping or cancelling stale video over increasing latency.109+- Keep input transport independent from media and audio backpressure.110+- Never send uncompressed video through Tauri IPC or Iroh.111+- Never reintroduce base64 media events; use binary Tauri channels.112+- A discontinuity must resume from a keyframe carrying codec configuration.113+- A slow or resetting receiver must not accumulate a playable history.114+- Keep H.264 at 1280x800/60 as the known-safe target until adaptive work is115+ measured end to end. Do not select codecs by compression ratio alone.116+- Preserve native pointer coordinates independently from encoded resolution or117+ client window size.118+- Session teardown must release all held input transitions and emit a neutral119+ gamepad state even after an error or disconnect.120+121+## Remaining public-alpha gates122+123+- Issue #3: one-time capability enrollment and controller-usable Portal import.124+- Issue #4: configure the offline Minisign trust root, publish the signed Sigil125+ asset set, and prove clean install plus upgrade/rollback from the public126+ command.127+- Issue #5: publish the signed/notarized macOS arm64 Portal DMG, digest, and128+ manifest.129+- Issue #6: prove physically headless cold boot, physical client controller130+ gameplay, mouse buttons consumed by Gamescope/an actual game, sustained A/V131+ and resource percentiles without latency growth, difficult-NAT relay132+ diagnostics, and reconnect/second-client rejection using the exact release133+ candidates.134+- Issues #7-#10 are streaming hardening: full Iroh/MoQ semantics, adaptive135+ bitrate, motion-sensitive resolution, and automatic low-latency codec136+ selection. The current independent media objects are an incremental transport137+ step, not a claim that the full MoQ milestone is complete.29138 30139 ## Working rules31140 32-- Use OpenSpec for capability planning before coding.33-- Treat `openspec/` artifacts as the source of truth for requirements and scope.34-- Do not build a browser-only version; the core architecture is native Tauri + Rust backend.35-- Prefer small, verifiable Rust spikes over broad scaffolding when exploring new capabilities.36-37-## Validated assumptions38-39-The highest-risk assumptions have all been validated via spikes:40-41-1. ~~Native YubiKey flows usable from a Tauri backend.~~ ✅ Validated (spikes 002, 003 — ctap-hid-fido2 communicates with Google Titan v2)42-2. ~~Safe storage or derivation of Iroh identity/addressing material from a YubiKey.~~ ✅ Validated (spike 004 — Titan hmac-secret → Iroh SecretKey → working endpoint, 6.5ms RTT)43-3. ~~Iroh session setup and authenticated protocol binding.~~ ✅ Validated (spikes 001, 007 — Iroh endpoints, ALPN routing, and key-derived identity all work)44-4. ~~Remote desktop stream/input latency and OS permissions.~~ ✅ Validated (spikes 005, 006 — frame streaming and input injection over Iroh both functional)45-46-## OpenSpec workflow47-48-- Create/modify proposal, design, tasks, and delta specs under `openspec/changes/<change>/`.49-- Validate with `openspec validate --all --strict` before reporting specs as done.50-- Do not implement active changes until approved.141+- Use Rust edition 2024. Sigil and Portal require Rust 1.91 or newer and the142+ repository pins Rust 1.95 in `rust-toolchain.toml`; `sigil-protocol` retains a143+ 1.85 minimum. Source `~/.cargo/env` before invoking Cargo directly.144+- Run `./scripts/verify-demo-build.sh` for the complete repository gate. It145+ covers Rust format/tests/clippy, the Linux cross-build when available,146+ frontend syntax/tests, ShellCheck, package tests, loopback transport, and147+ release-profile containment of `--dev-connect`.148+- Run `./scripts/verify-website.sh` for every website or public installer149+ change. Exercise interactive website changes in a real browser.150+- Treat `docs/fresh-bazzite-host.md`, `docs/public-release-delivery.md`, and151+ GitHub issues #3-#10 as the current acceptance sources. Distinguish local152+ tests from hardware proof and from still-pending operator acceptance.153+- Use the strict live capture gate on Bazzite when capture changes:154+155+ ```bash156+ sigil capture probe \157+ --source gamescope-pipewire \158+ --config ~/.config/sigil-spark/host.toml \159+ --frames 300 \160+ --expect-size 1280x800 \161+ --minimum-fps 55162+ ```163+164+- On Linux with NVIDIA, set `WEBKIT_DISABLE_DMABUF_RENDERER=1` for Portal.165+- Preserve `/Users/dami/Developer/sigil` untouched; it is the inherited source,166+ not this working repository.167+- Preserve unrelated worktree changes. Never sweep generated credentials,168+ evidence, local Claude/Codex configuration, `.env` files, or test-host state169+ into commits.170+- Do not weaken a fail-closed production path to make a development proof pass.171+ Keep explicit development flags visibly labeled and build-time contained.@@ -60,20 +60,25 @@ Steam and games.
6060 rollback, and package-owned user assets that follow the active release.6161 6262 The software JPEG client decode fallback and legacy wire compatibility remain.63-Media still uses reliable QUIC rather than drop-aware Iroh/MoQ. A dedicated64-persistent PipeWire sink, bounded Opus datagrams, WebCodecs decode, and65-AudioWorklet playback are implemented; longer-run audio/video synchronization66-measurement remains. Client authorization still needs67-short-lived capability tickets; the debug direct-node bypass is routing, not68-authentication.63+Media v2 now uses bounded independent QUIC object streams with prioritized64+keyframes, dependent-object cancellation, and discontinuity recovery; full MoQ65+group/control semantics remain promoted hardening work. A dedicated persistent66+PipeWire sink, bounded Opus datagrams, WebCodecs decode, and AudioWorklet67+playback are implemented; longer-run audio/video synchronization measurement68+remains. Client authorization still needs short-lived capability enrollment;69+the debug direct-node bypass is routing, not authentication.6970 7071 ## Immediate milestones7172 7273 1. **Done:** split Sigil into a pure Rust host binary; keep Portal client-only.73-2. **Hardware-proven:** capture the Gamescope PipeWire node without an XDG portal or physical display; cold-boot service proof remains.74+2. **Hardware-proven on the attached-display GPD host:** capture the exact75+ Gamescope PipeWire node without an XDG portal. Physically headless cold boot76+ and service startup remain public-alpha acceptance gates.7477 3. **Hardware-proven:** encode H.264 at 1280x800/60 with bounded buffers and no B-frames.7578 4. **Done:** replace base64 WebCodecs delivery with a bounded binary Tauri channel.76-5. Replace FIFO frame delivery with drop-aware iroh/MoQ media delivery.79+5. **Incremental object transport done:** bounded independent QUIC media80+ objects cancel stale dependent frames and recover at discontinuity81+ keyframes. Full Iroh/MoQ group/control semantics remain tracked in issue #7.7782 6. **Host hardware-proven; physical client controller pending:** the virtual7883 Xbox-style controller negotiated over Iroh and produced the expected7984 button, stick, trigger, D-pad, and neutral-release events on the Bazzite