cat / goq
clone

turns a linux box into a private steam streaming appliance: plug in, tap your key, play from another computer

test: preserve Bazzite hardware acceptance (#36)

FelineStateMachine committed 8010a8e
Add the fail-closed host runner, exact workflow provenance, sanitized capture evidence, and aggregate reconnect/recovery results for commit 7920c5d21434. Keep the fixed performance target enforced while reporting native-resolution cadence honestly.
commit
8010a8e144ea59a036da2e4a0aac11b508bf5e4a
parent
7920c5d214344f859e0cd5987433a3b303eb85f3
author
FelineStateMachine <dami.etoile@gmail.com>
7 files changed, +796 -0
Adocs/hardware-uat/7920c5d21434/REPORT.md25+
Adocs/hardware-uat/7920c5d21434/fixed-capture.log10+
Adocs/hardware-uat/7920c5d21434/native-capture.log10+
Adocs/hardware-uat/7920c5d21434/summary.env47++
Adocs/hardware-uat/7920c5d21434/uat-provenance.txt9+
Adocs/hardware-uat/README.md45++
Ascripts/run-bazzite-hardware-uat.sh650++++++++++++++++++++++++++++++
A docs/hardware-uat/7920c5d21434/REPORT.md +25 -0
A docs/hardware-uat/7920c5d21434/fixed-capture.log +10 -0
A docs/hardware-uat/7920c5d21434/native-capture.log +10 -0
A docs/hardware-uat/7920c5d21434/summary.env +47 -0
@@ -0,0 +1,47 @@
1+hardware_uat=pass2+candidate_kind=unsigned-development-hardware-uat3+source_commit=7920c5d214344f859e0cd5987433a3b303eb85f34+workflow_run_id=299199201095+fixed_size=1280x8006+fixed_minimum_fps=557+fixed_observed_fps=58.7018+native_size=2560x16009+native_performance_gate=observational10+native_observed_fps=29.75911+native_55fps_status=below12+persistent_authenticated_identity=pass13+invitation_redemption=pass14+invitation_replay=rejected15+maximum_required_glibc=2.1716+session_id_scope=daemon-invocation17+fixed_daemon_unique_sessions=2018+native_daemon_unique_sessions=2019+total_sessions=4020+fixed-iroh-moq_cycles=1021+fixed-iroh-moq_unique_sessions=1022+fixed-iroh-moq_sequence_gaps=023+fixed-iroh-moq_recovery_p50_micros=2860424+fixed-iroh-moq_recovery_p95_micros=3424725+fixed-iroh-moq_recovery_max_micros=3424726+fixed-grouped-v3_cycles=1027+fixed-grouped-v3_unique_sessions=1028+fixed-grouped-v3_sequence_gaps=029+fixed-grouped-v3_recovery_p50_micros=6316930+fixed-grouped-v3_recovery_p95_micros=7072431+fixed-grouped-v3_recovery_max_micros=7072432+native-iroh-moq_cycles=1033+native-iroh-moq_unique_sessions=1034+native-iroh-moq_sequence_gaps=035+native-iroh-moq_recovery_p50_micros=6312036+native-iroh-moq_recovery_p95_micros=6398337+native-iroh-moq_recovery_max_micros=6398338+native-grouped-v3_cycles=1039+native-grouped-v3_unique_sessions=1040+native-grouped-v3_sequence_gaps=041+native-grouped-v3_recovery_p50_micros=5791742+native-grouped-v3_recovery_p95_micros=6629543+native-grouped-v3_recovery_max_micros=6629544+original_service_restored=pass45+original_config_preserved=pass46+original_identity_preserved=pass47+original_release_preserved=pass
A docs/hardware-uat/7920c5d21434/uat-provenance.txt +9 -0
A docs/hardware-uat/README.md +45 -0
@@ -0,0 +1,45 @@
1+# Bazzite hardware acceptance2+3+The repository's hardware acceptance is a two-stage, commit-pinned gate:4+5+1. `.github/workflows/sigil-hardware-uat.yml` builds and inspects an unsigned6+ development candidate for one clean commit already on `main`.7+2. `scripts/run-bazzite-hardware-uat.sh` verifies that artifact and exercises it8+ against a live Gamescope PipeWire session on a Bazzite host.9+10+The runner is intentionally disruptive for the duration of the test. It stops11+the installed `sigil-host.service`, runs isolated fixed and native candidates,12+then restores the original service. Invocation-specific systemd rollback timers13+and a host-wide lock prevent an interrupted or concurrent run from stranding the14+host. A run can report `hardware_uat=pass` only after verifying the original15+service, config, identity, and release link.16+17+The enforced performance target is 1280x800 at at least 55 encoded frames per18+second. Native resolution is a functional compatibility leg: its actual cadence19+is recorded without claiming that every native mode meets the fixed performance20+target. Both legs require 300 captured frames with zero post-encode drops, ten21+authenticated Iroh/MoQ sessions, ten grouped-v3 sessions, bounded forced-IDR22+recovery, input acknowledgement, direct-path confirmation, and zero sequence23+gaps.24+25+Each evidence directory is named after the first 12 characters of its exact26+source commit. It contains the aggregate result, both bounded capture logs, and27+the GitHub Actions provenance used to obtain the candidate. Probe session IDs28+are deliberately excluded from committed evidence.29+30+To run the gate, download the artifact produced by the pinned workflow into a31+new owner-only directory matching:32+33+```text34+$HOME/.local/state/goq-hardware-uat.<12-char-commit>.<6-char-suffix>35+```36+37+Place the archive, its checksum, `release-manifest.json`, and38+`uat-provenance.txt` in an owner-only `incoming/` subdirectory on the Bazzite39+host, copy the runner to that host, then execute there:40+41+```sh42+./run-bazzite-hardware-uat.sh UAT_ROOT FULL_COMMIT WORKFLOW_RUN_ID43+```44+45+The runner rejects reused output directories and concurrent hardware runs.
A scripts/run-bazzite-hardware-uat.sh +650 -0
@@ -0,0 +1,650 @@
1+#!/usr/bin/env bash2+3+set -euo pipefail4+5+if [[ $# -ne 3 ]]; then6+ echo "usage: $0 UAT_ROOT UAT_COMMIT WORKFLOW_RUN_ID" >&27+ exit 648+fi9+10+uat_root="$1"11+uat_commit="$2"12+workflow_run_id="$3"13+short_commit="${uat_commit:0:12}"14+15+[[ "$uat_commit" =~ ^[0-9a-f]{40}$ ]] || {16+ echo "invalid UAT commit" >&217+ exit 6418+}19+[[ "$workflow_run_id" =~ ^[0-9]+$ ]] || {20+ echo "invalid workflow run ID" >&221+ exit 6422+}23+case "$uat_root" in24+ "$HOME"/.local/state/goq-hardware-uat."$short_commit".??????) ;;25+ *) echo "unsafe UAT root: $uat_root" >&2; exit 64 ;;26+esac27+for trusted_path in "$HOME/.local" "$HOME/.local/state" "$uat_root" "$uat_root/incoming"; do28+ [[ -d "$trusted_path" && ! -L "$trusted_path" ]] || {29+ echo "UAT path is missing or symlinked: $trusted_path" >&230+ exit 6431+ }32+done33+[[ "$(stat -c '%a:%U' "$uat_root")" == "700:$(id -un)" ]] || {34+ echo "UAT root is not owner-only" >&235+ exit 6436+}37+38+umask 07739+XDG_RUNTIME_DIR="/run/user/$(id -u)"40+export XDG_RUNTIME_DIR41+42+lock_file="$HOME/.local/state/goq-hardware-uat.lock"43+[[ ! -L "$lock_file" ]] || {44+ echo "unsafe UAT lock path: $lock_file" >&245+ exit 6446+}47+exec {uat_lock_fd}>"$lock_file"48+if ! flock -n "$uat_lock_fd"; then49+ echo "another hardware UAT owns the host-wide lock" >&250+ exit 7551+fi52+53+root_suffix="${uat_root##*.}"54+[[ "$root_suffix" =~ ^[[:alnum:]]{6}$ ]] || {55+ echo "invalid UAT root suffix" >&256+ exit 6457+}58+invocation_id="$short_commit-$workflow_run_id-$root_suffix"59+60+incoming="$uat_root/incoming"61+archive="$incoming/sigil-hardware-uat-$short_commit-bazzite-x86_64.tar.gz"62+payload_root="$uat_root/candidate"63+private="$uat_root/private"64+evidence="$uat_root/evidence"65+raw="$private/raw"66+for new_path in "$payload_root" "$private" "$evidence"; do67+ [[ ! -e "$new_path" && ! -L "$new_path" ]] || {68+ echo "UAT output path already exists: $new_path" >&269+ exit 6470+ }71+done72+install -d -m 0700 "$payload_root" "$private" "$evidence" "$raw"73+for new_path in "$payload_root" "$private" "$evidence" "$raw"; do74+ [[ -d "$new_path" && ! -L "$new_path" ]]75+ [[ "$(stat -c '%a:%U' "$new_path")" == "700:$(id -un)" ]]76+done77+78+real_config="$HOME/.config/sigil-spark/host.toml"79+real_sigil="$HOME/.local/libexec/sigil-spark/current/sigil"80+real_identity="$(sed -n 's/^identity_path = "\(.*\)"$/\1/p' "$real_config")"81+[[ -n "$real_identity" && -f "$real_identity" ]]82+[[ -x "$real_sigil" ]]83+original_ready() {84+ local invocation pid85+ systemctl --user is-active --quiet sigil-host.service || return 186+ pid="$(systemctl --user show -p MainPID --value sigil-host.service)"87+ invocation="$(systemctl --user show -p InvocationID --value sigil-host.service)"88+ [[ "$pid" =~ ^[0-9]+$ && "$pid" -gt 1 ]] || return 189+ [[ "$invocation" =~ ^[0-9a-f]{32}$ ]] || return 190+ cmp -s "$real_sigil" "/proc/$pid/exe" || return 191+ journalctl --user _SYSTEMD_INVOCATION_ID="$invocation" \92+ _SYSTEMD_USER_UNIT=sigil-host.service _PID="$pid" --no-pager -o cat \93+ | grep -Fx status=ready >/dev/null94+}95+original_ready96+original_enabled="$(systemctl --user is-enabled sigil-host.service)"97+original_main_pid="$(systemctl --user show -p MainPID --value sigil-host.service)"98+[[ "$original_main_pid" =~ ^[0-9]+$ && "$original_main_pid" -gt 1 ]]99+sha256sum "$real_config" >"$private/original-config.sha256"100+readlink "$HOME/.local/libexec/sigil-spark/current" >"$private/original-current-link"101+stat -c '%i:%s:%Y:%a:%U:%G' "$real_identity" >"$private/original-identity.stat"102+sha256sum "$real_identity" >"$private/original-identity.sha256"103+104+[[ -f "$archive" && -f "$archive.sha256" ]]105+[[ -f "$incoming/release-manifest.json" && -f "$incoming/uat-provenance.txt" ]]106+(107+ cd "$incoming"108+ sha256sum -c "$(basename -- "$archive.sha256")"109+)110+grep -Fxq "source_commit=$uat_commit" "$incoming/uat-provenance.txt"111+grep -Fxq "workflow_run_id=$workflow_run_id" "$incoming/uat-provenance.txt"112+113+tar -tzf "$archive" >"$private/archive-members.txt"114+if grep -Eq '(^/|(^|/)\.\.(/|$))' "$private/archive-members.txt"; then115+ echo "archive contains an unsafe member" >&2116+ exit 1117+fi118+tar --no-same-owner -xzf "$archive" -C "$payload_root"119+(120+ cd "$payload_root/payload"121+ sha256sum -c PACKAGE-SHA256SUMS122+)123+(124+ cd "$payload_root/payload/release"125+ sha256sum -c SHA256SUMS126+)127+128+sigil="$payload_root/payload/release/sigil"129+probe="$payload_root/payload/release/sigil-probe"130+manifest="$payload_root/payload/release/release-manifest.json"131+cmp "$manifest" "$incoming/release-manifest.json"132+jq -e --arg commit "$uat_commit" '133+ .git_commit == $commit134+ and .git_dirty == false135+ and .binary_provenance == "self-built-clean-head"136+ and .binary_provenance_verified == true137+ and .release_kind == "development"138+ and .release_tag == "development"139+ and .target == "x86_64-unknown-linux-gnu.2.17"140+ and .features == ["default", "in-process-gstreamer"]141+ and .demo_direct_node == false142+' "$manifest" >/dev/null143+[[ -x "$sigil" && -x "$probe" ]]144+145+readelf -d "$sigil" >"$private/sigil.dynamic.txt"146+if grep -Eq '\((RPATH|RUNPATH)\)' "$private/sigil.dynamic.txt"; then147+ echo "candidate contains an unexpected runtime library path" >&2148+ exit 1149+fi150+for soname in \151+ libgstreamer-1.0.so \152+ libgstapp-1.0.so \153+ libgstbase-1.0.so \154+ libgstvideo-1.0.so \155+ libgobject-2.0.so \156+ libglib-2.0.so157+do158+ grep -Fq "Shared library: [$soname" "$private/sigil.dynamic.txt"159+done160+readelf --version-info "$sigil" >"$private/sigil.versions.txt"161+max_glibc="$({162+ sed -n 's/.*Name: GLIBC_\([0-9][0-9.]*\).*/\1/p' "$private/sigil.versions.txt"163+ echo 0164+} | sort -V | tail -n 1)"165+[[ "$(printf '%s\n' "$max_glibc" 2.17 | sort -V | tail -n 1)" == 2.17 ]] || {166+ echo "candidate requires GLIBC_$max_glibc, above 2.17" >&2167+ exit 1168+}169+ldd "$sigil" >"$private/sigil.ldd"170+ldd "$probe" >"$private/sigil-probe.ldd"171+if grep -q 'not found' "$private/sigil.ldd"; then172+ echo "candidate host has an unresolved dynamic library" >&2173+ exit 1174+fi175+if grep -q 'not found' "$private/sigil-probe.ldd"; then176+ echo "candidate probe has an unresolved dynamic library" >&2177+ exit 1178+fi179+timeout --signal=TERM --kill-after=2s 5s "$sigil" --version180+timeout --signal=TERM --kill-after=2s 5s "$probe" --version181+182+host_identity="$private/host.key"183+probe_identity="$private/probe.key"184+state="$private/state"185+fixed_config="$private/host-1280x800.toml"186+native_config="$private/host-native.toml"187+invitation="$private/probe.goq-invite"188+189+"$sigil" identity init --output "$host_identity" >"$private/host-identity.log"190+"$sigil" identity init --output "$probe_identity" >"$private/probe-identity.log"191+host_node_id="$(sed -n 's/^node_id=//p' "$private/host-identity.log")"192+probe_node_id="$(sed -n 's/^node_id=//p' "$private/probe-identity.log")"193+[[ -n "$host_node_id" && -n "$probe_node_id" ]]194+195+pw_dump="$(command -v pw-dump)"196+gst_launch="$(command -v gst-launch-1.0)"197+gst_inspect="$(command -v gst-inspect-1.0)"198+ffmpeg="$(command -v ffmpeg)"199+render_node=/dev/dri/renderD128200+va_encoder=vah264enc201+[[ -S "$XDG_RUNTIME_DIR/pipewire-0" && -r "$render_node" && -w "$render_node" ]]202+"$gst_inspect" "$va_encoder" >/dev/null203+204+cat >"$fixed_config" <<EOF205+identity_path = "$host_identity"206+state_path = "$state"207+source = "gamescope-pipewire"208+width = 1280209+height = 800210+framerate = 60211+codec = "h264"212+input_mode = "log"213+ffmpeg_path = "$ffmpeg"214+215+[gamescope_pipewire]216+node_name = "gamescope"217+media_class = "Video/Source"218+xwayland_display = ":0"219+pw_dump_path = "$pw_dump"220+gst_launch_path = "$gst_launch"221+gst_inspect_path = "$gst_inspect"222+encoder_backend = "in-process-gstreamer"223+vaapi_encoder = "$va_encoder"224+vaapi_render_node = "$render_node"225+rate_control = "cbr"226+bitrate_kbps = 12000227+EOF228+229+cat >"$native_config" <<EOF230+identity_path = "$host_identity"231+state_path = "$state"232+source = "gamescope-pipewire"233+framerate = 60234+codec = "h264"235+input_mode = "log"236+ffmpeg_path = "$ffmpeg"237+238+[gamescope_pipewire]239+node_name = "gamescope"240+media_class = "Video/Source"241+xwayland_display = ":0"242+pw_dump_path = "$pw_dump"243+gst_launch_path = "$gst_launch"244+gst_inspect_path = "$gst_inspect"245+encoder_backend = "in-process-gstreamer"246+vaapi_encoder = "$va_encoder"247+vaapi_render_node = "$render_node"248+rate_control = "cbr"249+bitrate_kbps = 12000250+EOF251+chmod 0600 "$fixed_config" "$native_config"252+253+"$sigil" config check --config "$fixed_config" >"$private/fixed-config-check.log"254+"$sigil" config check --config "$native_config" >"$private/native-config-check.log"255+grep -Fxq config=ok "$private/fixed-config-check.log"256+grep -Fxq capture_preflight=ok "$private/fixed-config-check.log"257+grep -Fxq encoded_mode=1280x800@60 "$private/fixed-config-check.log"258+grep -Fxq config=ok "$private/native-config-check.log"259+grep -Fxq capture_preflight=ok "$private/native-config-check.log"260+native_size="$(sed -n 's/^encoded_mode=\([0-9][0-9]*x[0-9][0-9]*\)@60$/\1/p' "$private/native-config-check.log")"261+[[ -n "$native_size" ]]262+[[ "$native_size" != 1280x800 ]] || {263+ echo "native UAT resolved to the fixed 1280x800 mode" >&2264+ exit 1265+}266+267+"$sigil" invitation create \268+ --config "$fixed_config" \269+ --peer "$probe_node_id" \270+ --expires-in-seconds 900 \271+ --pointer-keyboard \272+ --output "$invitation" \273+ >"$private/invitation-create.log"274+[[ "$(stat -c %a "$invitation")" == 600 ]]275+276+restore_required=false277+rollback_units=()278+rollback_helper="$private/rollback-original.sh"279+fixed_candidate_unit="goq-uat-fixed-candidate-$invocation_id"280+native_candidate_unit="goq-uat-native-candidate-$invocation_id"281+candidate_units=("$fixed_candidate_unit" "$native_candidate_unit")282+active_candidate_unit=""283+284+cat >"$rollback_helper" <<EOF285+#!/usr/bin/env bash286+set -u287+XDG_RUNTIME_DIR='$XDG_RUNTIME_DIR'288+export XDG_RUNTIME_DIR289+real_sigil='$real_sigil'290+real_config='$real_config'291+for candidate_unit in '$fixed_candidate_unit' '$native_candidate_unit'; do292+ systemctl --user stop "\$candidate_unit.service" >/dev/null 2>&1 || true293+ systemctl --user reset-failed "\$candidate_unit.service" >/dev/null 2>&1 || true294+done295+original_ready() {296+ local invocation pid297+ systemctl --user is-active --quiet sigil-host.service || return 1298+ pid="\$(systemctl --user show -p MainPID --value sigil-host.service)"299+ invocation="\$(systemctl --user show -p InvocationID --value sigil-host.service)"300+ [[ "\$pid" =~ ^[0-9]+\$ && "\$pid" -gt 1 ]] || return 1301+ [[ "\$invocation" =~ ^[0-9a-f]{32}\$ ]] || return 1302+ cmp -s "\$real_sigil" "/proc/\$pid/exe" || return 1303+ journalctl --user _SYSTEMD_INVOCATION_ID="\$invocation" \304+ _SYSTEMD_USER_UNIT=sigil-host.service _PID="\$pid" --no-pager -o cat \305+ | grep -Fx status=ready >/dev/null306+}307+for _ in \$(seq 1 60); do308+ systemctl --user reset-failed sigil-host.service >/dev/null 2>&1 || true309+ systemctl --user start sigil-host.service >/dev/null 2>&1 || true310+ sleep 5311+ if systemctl --user is-active --quiet sigil-host.service && original_ready; then312+ exit 0313+ fi314+done315+exit 1316+EOF317+chmod 0700 "$rollback_helper"318+319+stop_candidate() {320+ local candidate_unit321+ for candidate_unit in "${candidate_units[@]}"; do322+ systemctl --user stop "$candidate_unit.service" >/dev/null 2>&1 || true323+ systemctl --user reset-failed "$candidate_unit.service" >/dev/null 2>&1 || true324+ done325+ active_candidate_unit=""326+}327+328+disarm_rollbacks() {329+ local unit330+ for unit in "${rollback_units[@]}"; do331+ systemctl --user stop "$unit.timer" "$unit.service" 2>/dev/null || true332+ done333+}334+335+restore_original() {336+ stop_candidate337+ if $restore_required || ! systemctl --user is-active --quiet sigil-host.service; then338+ systemctl --user reset-failed sigil-host.service || true339+ systemctl --user start sigil-host.service || return 1340+ for _ in $(seq 1 60); do341+ if systemctl --user is-active --quiet sigil-host.service && original_ready; then342+ restore_required=false343+ disarm_rollbacks344+ return 0345+ fi346+ sleep 0.5347+ done348+ return 1349+ fi350+ disarm_rollbacks351+}352+353+cleanup() {354+ local status=$?355+ trap - EXIT356+ trap '' INT TERM HUP357+ if ! restore_original; then358+ echo "CRITICAL: original Sigil service did not return ready; rollback timer remains armed" >&2359+ status=1360+ fi361+ exit "$status"362+}363+trap cleanup EXIT364+trap 'exit 130' INT365+trap 'exit 143' TERM366+trap 'exit 129' HUP367+368+arm_rollback() {369+ local unit="$1"370+ systemd-run --user --unit="$unit" --on-active=20m "$rollback_helper" >/dev/null371+ systemctl --user is-active --quiet "$unit.timer"372+ rollback_units+=("$unit")373+}374+375+start_candidate() {376+ local mode="$1"377+ local config="$2"378+ host_log="$raw/$mode-host.log"379+ case "$mode" in380+ fixed) active_candidate_unit="$fixed_candidate_unit" ;;381+ native) active_candidate_unit="$native_candidate_unit" ;;382+ *) echo "unknown candidate mode: $mode" >&2; return 1 ;;383+ esac384+ : >"$host_log"385+ systemd-run --user --quiet --collect \386+ --unit="$active_candidate_unit" \387+ --service-type=exec \388+ --property="StandardOutput=append:$host_log" \389+ --property="StandardError=append:$host_log" \390+ --property=TimeoutStopSec=10s \391+ --setenv=RUST_LOG=info,sigil::server=debug \392+ "$sigil" serve --config "$config" --max-runtime-seconds 1080393+ for _ in $(seq 1 300); do394+ grep -Fxq status=ready "$host_log" 2>/dev/null && break395+ systemctl --user is-active --quiet "$active_candidate_unit.service"396+ sleep 0.1397+ done398+ grep -Fxq status=ready "$host_log"399+ grep -Fxq "node_id=$host_node_id" "$host_log"400+}401+402+validate_probe() {403+ local log="$1"404+ local transport="$2"405+ local size="$3"406+ local request_id="$4"407+ grep -Fxq probe=ok "$log"408+ grep -Fxq "dimensions=$size" "$log"409+ grep -Fxq "transport=$transport" "$log"410+ grep -Fxq sequence_gaps=0 "$log"411+ grep -Fxq first_configured_idr=ok "$log"412+ grep -Fxq frame_sequence=monotonic "$log"413+ grep -Fxq recovery_barrier=configured-idr "$log"414+ grep -Fxq keyframe_recovery=ok "$log"415+ grep -Fxq "keyframe_request_id=$request_id" "$log"416+ grep -Fxq path_mode=direct "$log"417+ awk -F= '$1 == "keyframe_recovery_micros" && $2 ~ /^[0-9]+$/ && $2 <= 2000000 { ok=1 } END { exit !ok }' "$log"418+ awk -F= '$1 == "input_ack_micros" && $2 ~ /^[0-9]+$/ { ok=1 } END { exit !ok }' "$log"419+ if [[ "$transport" == iroh-moq ]]; then420+ grep -Fxq control_alpn=sigil/control/1 "$log"421+ grep -Fxq moq_group_capacity=1 "$log"422+ grep -Fxq moq_unrecovered_group_gaps=0 "$log"423+ grep -Fxq moq_historical_suffix_frames=0 "$log"424+ fi425+}426+427+wait_for_count() {428+ local log="$1"429+ local text="$2"430+ local expected="$3"431+ local deadline=$((SECONDS + 20))432+ local count433+ while ((SECONDS < deadline)); do434+ count="$(grep -Fc -- "$text" "$log" 2>/dev/null || true)"435+ [[ "$count" -ge "$expected" ]] && return 0436+ [[ -n "$active_candidate_unit" ]] \437+ && systemctl --user is-active --quiet "$active_candidate_unit.service" \438+ || return 1439+ sleep 0.1440+ done441+ return 1442+}443+444+run_probe_cycle() {445+ local mode="$1"446+ local transport="$2"447+ local size="$3"448+ local request_id="$4"449+ local invitation_path="${5:-}"450+ local log="$raw/$mode-$transport-$request_id.log"451+ local command=(452+ "$probe"453+ --node-id "$host_node_id"454+ --identity "$probe_identity"455+ --frames 30456+ --timeout-seconds 20457+ --expect-size "$size"458+ --keyframe-smoke459+ --keyframe-request-id "$request_id"460+ )461+ [[ "$transport" == grouped-v3 ]] && command+=(--media-v3)462+ [[ -n "$invitation_path" ]] && command+=(--invitation "$invitation_path")463+ timeout --signal=TERM --kill-after=5s 45s "${command[@]}" >"$log" 2>&1464+ validate_probe "$log" "$transport" "$size" "$request_id"465+}466+467+validate_host_recovery() {468+ local log="$1"469+ local plain470+ plain="$private/$(basename -- "$log").plain"471+ wait_for_count "$log" 'MoQ control client released' 10472+ wait_for_count "$log" 'media v3 client released' 10473+ wait_for_count "$log" 'input client released' 20474+ sed -E $'s/\x1B\\[[0-9;]*[[:alpha:]]//g' "$log" >"$plain"475+ [[ "$(grep -Fc 'MoQ control client released' "$plain")" -eq 10 ]]476+ [[ "$(grep -Fc 'media v3 client released' "$plain")" -eq 10 ]]477+ [[ "$(grep -Fc 'input client released' "$plain")" -eq 20 ]]478+ [[ "$(grep -Fc 'forced-IDR recovery acknowledged' "$plain")" -eq 20 ]]479+ [[ "$(grep -Fc 'forced_idr_disposition=Requested' "$plain")" -eq 20 ]]480+ [[ "$(grep -Fc 'coalesced=false' "$plain")" -ge 20 ]]481+ ! grep -Eq 'forced-IDR request failed|forced-IDR recovery was not acknowledged|forced-IDR acknowledgement task failed|forced-IDR acknowledgement task ended without a result' "$plain"482+}483+484+fixed_timer="goq-uat-fixed-$invocation_id"485+arm_rollback "$fixed_timer"486+restore_required=true487+systemctl --user stop sigil-host.service488+if systemctl --user is-active --quiet sigil-host.service; then489+ echo "original Sigil service did not stop" >&2490+ exit 1491+fi492+[[ "$(systemctl --user show -p MainPID --value sigil-host.service)" -eq 0 ]]493+494+timeout --signal=TERM --kill-after=5s 60s \495+ "$sigil" capture probe --source gamescope-pipewire \496+ --config "$fixed_config" --frames 300 --expect-size 1280x800 --minimum-fps 55 \497+ >"$evidence/fixed-capture.log"498+grep -Fxq probe=ok "$evidence/fixed-capture.log"499+grep -Fxq dropped_after_encode_before_probe_consumer=0 "$evidence/fixed-capture.log"500+501+start_candidate fixed "$fixed_config"502+run_probe_cycle fixed iroh-moq 1280x800 1001 "$invitation"503+wait_for_count "$raw/fixed-host.log" 'MoQ control client released' 1504+wait_for_count "$raw/fixed-host.log" 'input client released' 1505+enrollment="$("$sigil" enrollment show --config "$fixed_config")"506+grep -Fxq enrollment=active <<<"$enrollment"507+grep -Fxq "peer_node_id=$probe_node_id" <<<"$enrollment"508+grep -Fxq grants=view,pointer-keyboard <<<"$enrollment"509+if timeout --signal=TERM --kill-after=5s 20s \510+ "$probe" --node-id "$host_node_id" --identity "$probe_identity" \511+ --invitation "$invitation" --frames 4 >"$raw/invitation-replay.log" 2>&1; then512+ echo "redeemed invitation replay was accepted" >&2513+ exit 1514+fi515+grep -Fq 'host rejected control stream: Portal peer is not authorized' \516+ "$raw/invitation-replay.log"517+mv "$invitation" "$private/redeemed-probe.goq-invite"518+for cycle in $(seq 2 10); do519+ run_probe_cycle fixed iroh-moq 1280x800 "$((1000 + cycle))"520+done521+for cycle in $(seq 1 10); do522+ run_probe_cycle fixed grouped-v3 1280x800 "$((1100 + cycle))"523+done524+validate_host_recovery "$raw/fixed-host.log"525+stop_candidate526+527+native_timer="goq-uat-native-$invocation_id"528+arm_rollback "$native_timer"529+systemctl --user stop "$fixed_timer.timer" "$fixed_timer.service" 2>/dev/null || true530+timeout --signal=TERM --kill-after=5s 90s \531+ "$sigil" capture probe --source gamescope-pipewire \532+ --config "$native_config" --frames 300 --expect-size "$native_size" \533+ >"$evidence/native-capture.log"534+grep -Fxq probe=ok "$evidence/native-capture.log"535+grep -Fxq dropped_after_encode_before_probe_consumer=0 "$evidence/native-capture.log"536+537+start_candidate native "$native_config"538+for cycle in $(seq 1 10); do539+ run_probe_cycle native iroh-moq "$native_size" "$((2000 + cycle))"540+done541+for cycle in $(seq 1 10); do542+ run_probe_cycle native grouped-v3 "$native_size" "$((2100 + cycle))"543+done544+validate_host_recovery "$raw/native-host.log"545+546+install -d -m 0700 "$evidence/probes"547+for log in \548+ "$raw"/fixed-iroh-moq-*.log \549+ "$raw"/fixed-grouped-v3-*.log \550+ "$raw"/native-iroh-moq-*.log \551+ "$raw"/native-grouped-v3-*.log552+do553+ grep -vE '^session_id=' "$log" >"$evidence/probes/$(basename -- "$log")"554+done555+556+summarize_group() {557+ local prefix="$1"558+ local values="$private/$prefix.recovery-values"559+ local files=("$evidence/probes/$prefix"-*.log)560+ local raw_files=("$raw/$prefix"-*.log)561+ local count unique_sessions p50_index p95_index p50 p95 maximum gaps562+ [[ "${#files[@]}" -eq 10 ]]563+ [[ "${#raw_files[@]}" -eq 10 ]]564+ awk -F= '$1 == "keyframe_recovery_micros" { print $2 }' "${files[@]}" | sort -n >"$values"565+ count="$(wc -l <"$values" | tr -d ' ')"566+ [[ "$count" -eq 10 ]]567+ unique_sessions="$(awk -F= '$1 == "session_id" { print $2 }' "${raw_files[@]}" | sort -u | wc -l | tr -d ' ')"568+ [[ "$unique_sessions" -eq 10 ]]569+ p50_index=$(((count * 50 + 99) / 100))570+ p95_index=$(((count * 95 + 99) / 100))571+ p50="$(sed -n "${p50_index}p" "$values")"572+ p95="$(sed -n "${p95_index}p" "$values")"573+ maximum="$(tail -n 1 "$values")"574+ gaps="$(awk -F= '$1 == "sequence_gaps" { sum += $2 } END { print sum + 0 }' "${files[@]}")"575+ [[ "$gaps" -eq 0 && "$p95" -le 2000000 ]]576+ printf '%s_cycles=%s\n' "$prefix" "$count"577+ printf '%s_unique_sessions=%s\n' "$prefix" "$unique_sessions"578+ printf '%s_sequence_gaps=%s\n' "$prefix" "$gaps"579+ printf '%s_recovery_p50_micros=%s\n' "$prefix" "$p50"580+ printf '%s_recovery_p95_micros=%s\n' "$prefix" "$p95"581+ printf '%s_recovery_max_micros=%s\n' "$prefix" "$maximum"582+}583+584+all_raw_probes=(585+ "$raw"/fixed-iroh-moq-*.log586+ "$raw"/fixed-grouped-v3-*.log587+ "$raw"/native-iroh-moq-*.log588+ "$raw"/native-grouped-v3-*.log589+)590+[[ "${#all_raw_probes[@]}" -eq 40 ]]591+fixed_unique_sessions="$(awk -F= '$1 == "session_id" { print $2 }' \592+ "$raw"/fixed-iroh-moq-*.log "$raw"/fixed-grouped-v3-*.log \593+ | sort -u | wc -l | tr -d ' ')"594+native_unique_sessions="$(awk -F= '$1 == "session_id" { print $2 }' \595+ "$raw"/native-iroh-moq-*.log "$raw"/native-grouped-v3-*.log \596+ | sort -u | wc -l | tr -d ' ')"597+[[ "$fixed_unique_sessions" -eq 20 && "$native_unique_sessions" -eq 20 ]]598+total_sessions="${#all_raw_probes[@]}"599+fixed_observed_fps="$(sed -n 's/^observed_fps=//p' "$evidence/fixed-capture.log")"600+native_observed_fps="$(sed -n 's/^observed_fps=//p' "$evidence/native-capture.log")"601+[[ -n "$fixed_observed_fps" && -n "$native_observed_fps" ]]602+if awk -v fps="$native_observed_fps" 'BEGIN { exit !(fps >= 55) }'; then603+ native_55fps_status=met604+else605+ native_55fps_status=below606+fi607+608+{609+ echo candidate_kind=unsigned-development-hardware-uat610+ echo "source_commit=$uat_commit"611+ echo "workflow_run_id=$workflow_run_id"612+ echo fixed_size=1280x800613+ echo fixed_minimum_fps=55614+ echo "fixed_observed_fps=$fixed_observed_fps"615+ echo "native_size=$native_size"616+ echo native_performance_gate=observational617+ echo "native_observed_fps=$native_observed_fps"618+ echo "native_55fps_status=$native_55fps_status"619+ echo persistent_authenticated_identity=pass620+ echo invitation_redemption=pass621+ echo invitation_replay=rejected622+ echo "maximum_required_glibc=$max_glibc"623+ echo session_id_scope=daemon-invocation624+ echo "fixed_daemon_unique_sessions=$fixed_unique_sessions"625+ echo "native_daemon_unique_sessions=$native_unique_sessions"626+ echo "total_sessions=$total_sessions"627+ summarize_group fixed-iroh-moq628+ summarize_group fixed-grouped-v3629+ summarize_group native-iroh-moq630+ summarize_group native-grouped-v3631+} >"$private/summary.pending"632+633+restore_original634+[[ "$(systemctl --user is-active sigil-host.service)" == active ]]635+[[ "$(systemctl --user is-enabled sigil-host.service)" == "$original_enabled" ]]636+sha256sum -c "$private/original-config.sha256"637+sha256sum -c "$private/original-identity.sha256"638+[[ "$(readlink "$HOME/.local/libexec/sigil-spark/current")" == "$(cat "$private/original-current-link")" ]]639+[[ "$(stat -c '%i:%s:%Y:%a:%U:%G' "$real_identity")" == "$(cat "$private/original-identity.stat")" ]]640+641+{642+ echo hardware_uat=pass643+ cat "$private/summary.pending"644+ echo original_service_restored=pass645+ echo original_config_preserved=pass646+ echo original_identity_preserved=pass647+ echo original_release_preserved=pass648+} >"$evidence/summary.env"649+chmod 0600 "$evidence/summary.env" "$evidence"/probes/*.log "$evidence"/*-capture.log650+cat "$evidence/summary.env"