- commit
- 82b3f084dd10718430190a7d6d642ce6882853ef
- parent
- 5c71338ea3d392741563c2317a9c9df6cf8b7a24
- author
- dami etoile <dami.etoile@gmail.com>
| M | AGENTS.md | 11 | ++++++++++- |
| M | README.md | 2 | +- |
| M | docs/ | 31 | +++++++++++++++++++++++++++--- |
@@ -4,7 +4,16 @@ Keyhome is a Tauri v2 remote desktop app with FIDO2 key authentication over Iroh
44 55 ## Current focus66 7-The project is past the spike/validation phase. The main work now is **performance optimization** — improving frame rate (FPS), reducing latency, and tuning the H.264 encoding pipeline.7+The project is past the spike/validation phase. ffmpeg subprocess is now the primary capture+encode path (xcap+openh264 is the fallback), achieving ~28 fps with NVENC (up from ~1–2 fps). The bottleneck has moved to network/relay throughput and client-side decode.8+9+Active work areas:10+11+- **Codec configurability** — H.264, H.265, and AV1 are all supported. Users can select codec + encoder backend (nvenc, vaapi, qsv, amf, videotoolbox, software) in the info panel.12+- **Multi-codec WebCodecs decode** — frontend handles H.264 (AVCC), H.265 (hvcC), and AV1 (OBU) for hardware-accelerated decode.13+- **Connection tracking** — fixed with `conn.closed()` future + `tokio::select!` for reliable disconnect detection; connection counter now properly decrements.14+- **Wire protocol** — header expanded to 14 bytes: `width(4) + height(4) + size(4) + keyframe(1) + codec(1)`. Codec byte: 0=h264, 1=h265, 2=av1.15+- **Client-side stats** — info panel shows codec received, dropped frames, total frames received when connected as client. Encoder config and host performance sections are hidden when connected.16+- **Remaining performance work** — network/relay throughput, client-side decode efficiency.817 918 ## Key files1019 @@ -77,7 +77,7 @@ Seven spikes were completed as viability evidence. They live in `spikes/`.
7777 7878 ## Requirements7979 80-- **OS:** Linux or macOS80+- **OS:** Linux, macOS, or Windows8181 - **Hardware:** A FIDO2 security key (Google Titan, YubiKey)8282 - **System dependencies:** See [docs/SETUP.md](docs/SETUP.md)8383 @@ -52,6 +52,20 @@ Grant the following permissions in **System Settings > Privacy & Security**:
5252 - **Accessibility** — required for input injection (mouse/keyboard control)5353 - **Screen Recording** — required for screen capture5454 55+### Windows56+57+Install the Tauri CLI (v2):58+59+```bash60+cargo install tauri-cli --version "^2"61+62+# ffmpeg — screen capture (gdigrab) + hardware encoding (NVENC/QSV/AMF)63+choco install ffmpeg64+# or: scoop install ffmpeg65+```66+67+ffmpeg auto-detects the best available encoder in this order: NVENC → QSV → AMF → libx264 (software).68+5569 ### Clone5670 5771 ```bash@@ -101,13 +115,24 @@ Click **Connect** and tap your Titan key. The app derives the same host endpoint
101115 ## Architecture102116 103117 - **Transport**: Iroh 1.0 (peer-to-peer, relay-assisted)104-- **Screen capture + encoding**: ffmpeg subprocess (x11grab + NVENC/VAAPI on Linux, avfoundation + VideoToolbox on macOS)118+- **Screen capture + encoding**: ffmpeg subprocess (x11grab on Linux, avfoundation on macOS, gdigrab on Windows)119+- **Encoder backends**: NVENC, VAAPI, QSV, AMF, VideoToolbox, or software (auto-detected or user-selected)120+- **Video codecs**: H.264, H.265, AV1 (configurable in the info panel)121+- **Client-side decode**: WebCodecs — H.264 (AVCC), H.265 (hvcC), AV1 (OBU), hardware-accelerated105122 - **Fallback**: xcap capture + openh264 software encoding (if ffmpeg not installed)106123 - **FIDO2**: ctap-hid-fido2 (CTAP 2.0/2.1 over HID)107-- **Frame encoding**: H.264 (ffmpeg/NVENC primary, openh264 fallback for client-side decode)108124 - **Input injection**: enigo (cross-platform mouse/keyboard)109-- **Protocol**: BiStream with frame header `[width:u32][height:u32][h264_len:u32][keyframe:u8][h264_data]`110125 - **Identity**: Iroh SecretKey derived from Titan hmac-secret extension; no address copy/paste126+- **Protocol**: BiStream with a 14-byte frame header:127+128+```129+[width:u32][height:u32][size:u32][keyframe:u8][codec:u8][frame_data]130+```131+132+Codec byte: `0` = H.264, `1` = H.265, `2` = AV1.133+134+- **Connection tracking**: Uses `conn.closed()` future + `tokio::select!` for reliable disconnect detection; connection counter decrements on disconnect.135+- **Client-side stats**: Info panel shows codec received, dropped frames, and total frames received when connected as a client. Encoder config and host performance sections are hidden when connected.111136 112137 ## Spikes (evidence)113138