* feat: define grouped media v3 protocol Add bounded MoQ-style object metadata, delivery deadlines, and explicit keyframe control requests while retaining v1/v2 ALPN compatibility. * feat: add Portal media v3 recovery Prefer grouped media objects, validate wire GOP ordering, coalesce explicit keyframe requests, and reset WebCodecs immediately when delivery or decode state is poisoned. * feat: serve bounded grouped media v3 Add authenticated v3 routing, GOP/object accounting, expiring prioritized streams, stable cancellation, and coalesced keyframe recovery without restarting the Gamescope encoder. * fix: bind discontinuities to v3 group starts Reject recovery-boundary flags on nonzero objects so every discontinuity resumes from object zero with codec configuration. * test: prove grouped media recovery Make grouped v3 the transport-probe default, retain explicit v2/v1 compatibility modes, and exercise keyframe requests through real loopback sessions. * docs: describe grouped media hardening Document v3 semantics, compatibility scope, and the remaining in-process force-key-unit and hardware proof required before claiming full MoQ support. * fix: harden grouped media recovery Remove stale Portal decoder state, recognize grouped-v3 diagnostics, ignore superseded object outcomes after discontinuity recovery, and guard replacement decoders from stale callbacks. * fix: stop recovery keyframe feedback loops Treat an arriving discontinuity keyframe as the requested recovery boundary, resetting decode state without requesting its replacement. * fix: require explicit ALPN downgrade signals Fall back from media v3 only when QUIC reports TLS no_application_protocol, keeping transient failures and interference from silently selecting legacy transport. * fix: harden media v3 host recovery Keep late Join idempotent for static GOP replay, ignore stale send failures from superseded groups, and pace rejected control bursts at the read boundary. * test: correlate media recovery requests Bind grouped-v3 smoke proof to a unique accepted host request and discard late superseded probe outcomes after recovery barriers. * fix: normalize colored loopback evidence
- commit
- 3d4d6d095f5b2505360300741b753e3d588f29fa
- parent
- 16ad85a3f229853863395ee9bc0817d0c4904eb8
- author
- FelineStateMachine <dami.etoile@gmail.com>
| M | AGENTS.md | 16 | +- |
| M | README.md | 16 | +- |
| M | crates/ | 740 | +++++++++++++++- |
| M | crates/ | 14 | +- |
| M | crates/ | 1453 | ++++++++++++++++++++++++++++-- |
| M | crates/ | 13 | + |
| A | crates/ | 783 | ++++++++++++++++ |
| M | portal/ | 10 | + |
| A | portal/ | 123 | +++ |
| A | portal/ | 164 | +++ |
| M | portal/ | 108 | ++- |
| M | scripts/ | 106 | ++- |
| M | src-tauri/ | 840 | ++++++++++++++++-- |
| M | src-tauri/ | 7 | + |
| M | src-tauri/ | 1 | + |
@@ -52,9 +52,12 @@ Sigil host.
5252 - Gamescope's exact PipeWire node is resolved from strict configured5353 properties. AMD GstVA H.264 sustains the fixed 1280x800/60 first target on5454 the GPD Pocket 4 test host with bounded post-encode delivery.55-- Media v2 uses one bounded, prioritized QUIC object stream per encoded frame.56- Stale dependent objects are cancelled; discontinuity keyframes provide a57- latest-frame barrier. Media v1 remains a compatibility fallback.55+- Media v3 uses one bounded, prioritized QUIC object stream per encoded frame56+ with explicit GOP/object identity, publisher priority, delivery deadlines,57+ and a persistent keyframe-request leg. Stale dependent objects are cancelled;58+ discontinuity keyframes provide a latest-frame barrier. Media v2 and v159+ remain compatibility fallbacks. This is a custom MoQ-style protocol, not a60+ claim of IETF MoQ Transport compatibility.5861 - Portal crosses Rust-to-webview video and audio through bounded binary Tauri5962 channels. It reports transport, frontend, decoder, presentation, and audio6063 queue/drop timing separately.@@ -136,8 +139,11 @@ Sigil host.
136139 candidates.137140 - Issues #7-#10 are streaming hardening: full Iroh/MoQ semantics, adaptive138141 bitrate, motion-sensitive resolution, and automatic low-latency codec139- selection. The current independent media objects are an incremental transport140- step, not a claim that the full MoQ milestone is complete.142+ selection. Media v3 adds grouped object identity, expiry, cancellation, and143+ explicit recovery, but the external `gst-launch` encoder can only honor a144+ keyframe request at its next natural configured IDR. An in-process GStreamer145+ control path plus hardware loss/relay proof remain before claiming the full146+ MoQ milestone complete.141147 142148 ## Working rules143149 @@ -62,9 +62,10 @@ Steam and games.
6262 rollback, and package-owned user assets that follow the active release.6363 6464 The software JPEG client decode fallback and legacy wire compatibility remain.65-Media v2 now uses bounded independent QUIC object streams with prioritized66-keyframes, dependent-object cancellation, and discontinuity recovery; full MoQ67-group/control semantics remain promoted hardening work. A dedicated persistent65+Media v3 adds bounded QUIC objects with explicit GOP/object identity, publisher66+priority, delivery expiry, dependent-object cancellation, and client-requested67+discontinuity recovery. It is deliberately MoQ-style rather than an IETF MoQ68+Transport compatibility claim; v2 and v1 remain fallbacks. A dedicated persistent6869 PipeWire sink, bounded Opus datagrams, WebCodecs decode, and AudioWorklet6970 playback are implemented; longer-run audio/video synchronization measurement7071 remains. The debug direct-node bypass is routing, not authentication.@@ -77,9 +78,12 @@ remains. The debug direct-node bypass is routing, not authentication.
7778 and service startup remain public-alpha acceptance gates.7879 3. **Hardware-proven:** encode H.264 at 1280x800/60 with bounded buffers and no B-frames.7980 4. **Done:** replace base64 WebCodecs delivery with a bounded binary Tauri channel.80-5. **Incremental object transport done:** bounded independent QUIC media81- objects cancel stale dependent frames and recover at discontinuity82- keyframes. Full Iroh/MoQ group/control semantics remain tracked in issue #7.81+5. **Grouped object transport implemented; hardware proof pending:** bounded82+ QUIC media objects carry GOP/object identity and delivery deadlines, cancel83+ stale dependent frames, and recover through explicit keyframe requests.84+ The external encoder currently responds at its next natural configured IDR;85+ in-process force-key-unit control and induced-loss/relay proof remain in86+ issue #7.8387 6. **Host hardware-proven; physical client controller pending:** the virtual8488 Xbox-style controller negotiated over Iroh and produced the expected8589 button, stick, trigger, D-pad, and neutral-release events on the Bazzite@@ -8,9 +8,11 @@ use clap::Parser;
88 use iroh::{Endpoint, EndpointAddr, EndpointId, SecretKey, endpoint::presets};99 use sigil_protocol::{1010 Capability, ClientHello, FrameFlags, GAMEPAD_AXIS_MAX, GAMEPAD_AXIS_MIN, GAMEPAD_TRIGGER_MAX,11- GamepadState, INPUT_ALPN_V1, InputEvent, MEDIA_ALPN_V1, MEDIA_ALPN_V2, MediaFrame,11+ GamepadState, INPUT_ALPN_V1, InputEvent, KeyframeRequestReasonV3, MAX_MEDIA_GROUP_BYTES_V3,12+ MEDIA_ALPN_V1, MEDIA_ALPN_V2, MEDIA_ALPN_V3, MediaControlRequestV3, MediaFrame, MediaObjectV3,1213 PointerSurfaceDimensions, ProtocolError, read_host_hello, read_input_ack, read_media_frame,13- read_media_object, write_client_hello, write_input_event,14+ read_media_object, read_media_object_v3, write_client_hello, write_input_event,15+ write_media_control_request_v3,1416 };1517 1618 const MEDIA_OBJECT_CAPACITY: usize = 4;@@ -30,10 +32,21 @@ struct Args {
3032 minimum_fps: Option<f64>,3133 3234 expect_size: (u16, u16),33- /// Exercise the reliable ordered v1 media stream instead of independent34- /// v2 media objects. Intended only for compatibility validation.35- 35+ /// Exercise independent v2 media objects instead of grouped v3 media36+ /// objects. Intended only for compatibility validation.37+ 38+ media_v2: bool,39+ /// Exercise the reliable ordered v1 media stream instead of grouped v340+ /// media objects. Intended only for compatibility validation.41+ 3642 media_v1: bool,43+ /// Request a configured recovery keyframe after three accepted v3 frames,44+ /// then prove no delta history is delivered before the discontinuity.45+ 46+ keyframe_smoke: bool,47+ /// Correlation identifier for `--keyframe-smoke` host evidence.48+ 49+ keyframe_request_id: u64,3750 /// Require gamepad negotiation and emit one bounded non-neutral snapshot3851 /// followed by neutral. Intended for evtest-backed uinput proof.3952 @@ -46,6 +59,7 @@ struct Args {
4659 4760 4861 enum MediaTransport {62+ GroupedV3,4963 IndependentV2,5064 ReliableV1,5165 }@@ -53,6 +67,7 @@ enum MediaTransport {
5367 impl MediaTransport {5468 fn alpn(self) -> &'static [u8] {5569 match self {70+ Self::GroupedV3 => MEDIA_ALPN_V3,5671 Self::IndependentV2 => MEDIA_ALPN_V2,5772 Self::ReliableV1 => MEDIA_ALPN_V1,5873 }@@ -60,12 +75,16 @@ impl MediaTransport {
6075 6176 fn label(self) -> &'static str {6277 match self {78+ Self::GroupedV3 => "grouped-v3",6379 Self::IndependentV2 => "independent-v2",6480 Self::ReliableV1 => "reliable-v1",6581 }6682 }6783 }6884 85+const KEYFRAME_SMOKE_REQUEST_AFTER_FRAMES: u32 = 3;86+const KEYFRAME_SMOKE_MINIMUM_FRAMES: u32 = KEYFRAME_SMOKE_REQUEST_AFTER_FRAMES + 1;87+6988 7089 enum MediaObjectOutcome {7190 Frame { index: u64, frame: MediaFrame },@@ -227,6 +246,187 @@ impl Drop for MediaObjectReceiver {
227246 }228247 }229248 249+250+enum MediaObjectOutcomeV3 {251+ Object {252+ accept_index: u64,253+ object: MediaObjectV3,254+ },255+ Dropped {256+ accept_index: u64,257+ },258+ Malformed {259+ accept_index: u64,260+ error: ProtocolError,261+ },262+}263+264+impl MediaObjectOutcomeV3 {265+ fn accept_index(&self) -> u64 {266+ match self {267+ Self::Object { accept_index, .. }268+ | Self::Dropped { accept_index }269+ | Self::Malformed { accept_index, .. } => *accept_index,270+ }271+ }272+273+ fn is_fast_forward_barrier(&self) -> bool {274+ let Self::Object { object, .. } = self else {275+ return false;276+ };277+ object.header.object_id == 0278+ && object.header.flags.contains(FrameFlags::KEYFRAME)279+ && object.header.flags.contains(FrameFlags::CODEC_CONFIG)280+ && object.header.flags.contains(FrameFlags::DISCONTINUITY)281+ }282+}283+284+285+struct MediaObjectReorderV3 {286+ next_accept_index: u64,287+ completed: BTreeMap<u64, MediaObjectOutcomeV3>,288+}289+290+impl MediaObjectReorderV3 {291+ fn new(first_accept_index: u64) -> Self {292+ Self {293+ next_accept_index: first_accept_index,294+ completed: BTreeMap::new(),295+ }296+ }297+298+ fn pending_len(&self) -> usize {299+ self.completed.len()300+ }301+302+ fn push(&mut self, outcome: MediaObjectOutcomeV3) -> Result<Option<MediaObjectOutcomeV3>> {303+ if matches!(outcome, MediaObjectOutcomeV3::Malformed { .. }) {304+ return Ok(Some(outcome));305+ }306+ let accept_index = outcome.accept_index();307+ if accept_index < self.next_accept_index {308+ // A discontinuity barrier may advance beyond older in-flight309+ // reads. Their eventual timeout/reset outcomes belong to the310+ // superseded GOP and must not poison the recovered sequence.311+ return Ok(None);312+ }313+ let fast_forward = outcome.is_fast_forward_barrier();314+ ensure!(315+ self.completed.insert(accept_index, outcome).is_none(),316+ "media v3 accept index {accept_index} completed more than once"317+ );318+ if fast_forward {319+ self.completed.retain(|index, _| *index >= accept_index);320+ self.next_accept_index = accept_index;321+ }322+ self.take_next()323+ }324+325+ fn take_next(&mut self) -> Result<Option<MediaObjectOutcomeV3>> {326+ let Some(outcome) = self.completed.remove(&self.next_accept_index) else {327+ return Ok(None);328+ };329+ self.next_accept_index = self330+ .next_accept_index331+ .checked_add(1)332+ .context("media v3 accept index overflowed")?;333+ Ok(Some(outcome))334+ }335+}336+337+struct MediaObjectReceiverV3 {338+ connection: iroh::endpoint::Connection,339+ reads: tokio::task::JoinSet<MediaObjectOutcomeV3>,340+ reorder: MediaObjectReorderV3,341+ next_accept_index: u64,342+ accepting: bool,343+ read_timeout: Duration,344+}345+346+impl MediaObjectReceiverV3 {347+ fn new(connection: iroh::endpoint::Connection, read_timeout: Duration) -> Self {348+ Self {349+ connection,350+ reads: tokio::task::JoinSet::new(),351+ reorder: MediaObjectReorderV3::new(0),352+ next_accept_index: 0,353+ accepting: true,354+ read_timeout,355+ }356+ }357+358+ async fn next(&mut self) -> Result<Option<MediaObjectOutcomeV3>> {359+ loop {360+ if let Some(completed) = self.reorder.take_next()? {361+ return Ok(Some(completed));362+ }363+ if !self.accepting && self.reads.is_empty() {364+ ensure!(365+ self.reorder.pending_len() == 0,366+ "media v3 connection closed with an incomplete object order"367+ );368+ return Ok(None);369+ }370+371+ tokio::select! {372+ accepted = self.connection.accept_uni(),373+ if self.accepting374+ && self.reads.len() + self.reorder.pending_len()375+ < MEDIA_OBJECT_CAPACITY => {376+ match accepted {377+ Ok(mut stream) => {378+ let accept_index = self.next_accept_index;379+ self.next_accept_index = self380+ .next_accept_index381+ .checked_add(1)382+ .context("media v3 accept index overflowed")?;383+ let read_timeout = self.read_timeout;384+ self.reads.spawn(async move {385+ match tokio::time::timeout(386+ read_timeout,387+ read_media_object_v3(&mut stream),388+ )389+ .await390+ {391+ Ok(Ok(object)) => MediaObjectOutcomeV3::Object {392+ accept_index,393+ object,394+ },395+ Ok(Err(ProtocolError::Io(_))) | Err(_) => {396+ MediaObjectOutcomeV3::Dropped { accept_index }397+ }398+ Ok(Err(error)) => MediaObjectOutcomeV3::Malformed {399+ accept_index,400+ error,401+ },402+ }403+ });404+ }405+ Err(_) => self.accepting = false,406+ }407+ }408+ completed = self.reads.join_next(), if !self.reads.is_empty() => {409+ match completed.expect("guarded by non-empty media v3 object task set") {410+ Ok(outcome) => {411+ if let Some(completed) = self.reorder.push(outcome)? {412+ return Ok(Some(completed));413+ }414+ }415+ Err(error) if error.is_cancelled() => continue,416+ Err(error) => return Err(error).context("media v3 object read task failed"),417+ }418+ }419+ }420+ }421+ }422+}423+424+impl Drop for MediaObjectReceiverV3 {425+ fn drop(&mut self) {426+ self.reads.abort_all();427+ }428+}429+230430 231431 enum MediaObjectDecision {232432 Deliver,@@ -292,6 +492,139 @@ impl MediaObjectSequence {
292492 }293493 }294494 495+496+enum MediaObjectDecisionV3 {497+ Deliver { discontinuity: bool },498+ DropLate,499+ DropUntilKeyframe,500+}501+502+503+struct MediaObjectSequenceV3 {504+ group_id: Option<u64>,505+ last_object_id: Option<u32>,506+ last_sequence: Option<u64>,507+ group_payload_bytes: usize,508+ waiting_for_keyframe: bool,509+}510+511+impl MediaObjectSequenceV3 {512+ fn new() -> Self {513+ Self {514+ waiting_for_keyframe: true,515+ ..Self::default()516+ }517+ }518+519+ fn request_recovery(&mut self) {520+ self.waiting_for_keyframe = true;521+ }522+523+ fn note_drop(&mut self) -> bool {524+ let entered = !self.waiting_for_keyframe;525+ self.waiting_for_keyframe = true;526+ entered527+ }528+529+ fn classify(&mut self, object: &MediaObjectV3) -> MediaObjectDecisionV3 {530+ let header = &object.header;531+ if self532+ .group_id533+ .is_some_and(|group_id| header.group_id < group_id)534+ || self535+ .last_sequence536+ .is_some_and(|sequence| header.sequence <= sequence)537+ {538+ return MediaObjectDecisionV3::DropLate;539+ }540+541+ let new_group = self.group_id != Some(header.group_id);542+ let configured_group_start = header.object_id == 0543+ && header.flags.contains(FrameFlags::KEYFRAME)544+ && header.flags.contains(FrameFlags::CODEC_CONFIG);545+ if new_group && !configured_group_start {546+ self.waiting_for_keyframe = true;547+ return MediaObjectDecisionV3::DropUntilKeyframe;548+ }549+ if !new_group && self.waiting_for_keyframe {550+ return MediaObjectDecisionV3::DropUntilKeyframe;551+ }552+553+ let sequence_contiguous = self554+ .last_sequence555+ .is_none_or(|sequence| sequence.checked_add(1) == Some(header.sequence));556+ let object_contiguous = new_group557+ || self558+ .last_object_id559+ .is_some_and(|object_id| object_id.checked_add(1) == Some(header.object_id));560+ let Some(next_group_bytes) = (if new_group {561+ Some(object.payload.len())562+ } else {563+ self.group_payload_bytes.checked_add(object.payload.len())564+ }) else {565+ self.waiting_for_keyframe = true;566+ return MediaObjectDecisionV3::DropUntilKeyframe;567+ };568+ let recovery_requires_discontinuity = new_group569+ && self.group_id.is_some()570+ && (self.waiting_for_keyframe || !sequence_contiguous)571+ && !header.flags.contains(FrameFlags::DISCONTINUITY);572+ if (!sequence_contiguous && !new_group)573+ || !object_contiguous574+ || next_group_bytes > MAX_MEDIA_GROUP_BYTES_V3575+ || recovery_requires_discontinuity576+ {577+ self.waiting_for_keyframe = true;578+ return MediaObjectDecisionV3::DropUntilKeyframe;579+ }580+581+ let discontinuity = header.flags.contains(FrameFlags::DISCONTINUITY)582+ || self.waiting_for_keyframe583+ || !sequence_contiguous;584+ self.group_id = Some(header.group_id);585+ self.last_object_id = Some(header.object_id);586+ self.last_sequence = Some(header.sequence);587+ self.group_payload_bytes = next_group_bytes;588+ self.waiting_for_keyframe = false;589+ MediaObjectDecisionV3::Deliver { discontinuity }590+ }591+}592+593+struct AcceptedMedia {594+ flags: FrameFlags,595+ width: u16,596+ height: u16,597+ sequence: u64,598+ payload_len: usize,599+ v3_group_id: Option<u64>,600+}601+602+impl From<MediaFrame> for AcceptedMedia {603+ fn from(frame: MediaFrame) -> Self {604+ Self {605+ flags: frame.header.flags,606+ width: frame.header.width,607+ height: frame.header.height,608+ sequence: frame.header.sequence,609+ payload_len: frame.payload.len(),610+ v3_group_id: None,611+ }612+ }613+}614+615+impl From<MediaObjectV3> for AcceptedMedia {616+ fn from(object: MediaObjectV3) -> Self {617+ Self {618+ flags: object.header.flags,619+ width: object.header.width,620+ height: object.header.height,621+ sequence: object.header.sequence,622+ payload_len: object.payload.len(),623+ v3_group_id: Some(object.header.group_id),624+ }625+ }626+}627+295628 296629 async fn main() -> Result<()> {297630 let args = Args::parse();@@ -299,6 +632,14 @@ async fn main() -> Result<()> {
299632 args.timeout_seconds > 0,300633 "--timeout-seconds must be greater than zero"301634 );635+ ensure!(636+ !args.keyframe_smoke || (!args.media_v1 && !args.media_v2),637+ "--keyframe-smoke requires grouped v3 media"638+ );639+ ensure!(640+ !args.keyframe_smoke || args.frames >= KEYFRAME_SMOKE_MINIMUM_FRAMES,641+ "--keyframe-smoke requires at least {KEYFRAME_SMOKE_MINIMUM_FRAMES} accepted frames"642+ );302643 303644 let secret = SecretKey::generate();304645 let mut nonce = [0_u8; 16];@@ -312,8 +653,10 @@ async fn main() -> Result<()> {
312653 let address = EndpointAddr::new(args.node_id);313654 let media_transport = if args.media_v1 {314655 MediaTransport::ReliableV1315- } else {656+ } else if args.media_v2 {316657 MediaTransport::IndependentV2658+ } else {659+ MediaTransport::GroupedV3317660 };318661 319662 let media_connection = endpoint@@ -336,13 +679,21 @@ async fn main() -> Result<()> {
336679 let session_id = media_negotiation.session_id;337680 let mut media_send = Some(media_send);338681 let mut media_recv = Some(media_recv);339- if media_transport == MediaTransport::IndependentV2 {340- media_send341- .take()342- .expect("media handshake stream is present")343- .finish()344- .context("finishing media v2 handshake request")?;345- drop(media_recv.take());682+ match media_transport {683+ MediaTransport::GroupedV3 => {684+ // Sigil finishes its response half after HostHello. Keep our send685+ // half alive as the bounded v3 keyframe-control stream.686+ drop(media_recv.take());687+ }688+ MediaTransport::IndependentV2 => {689+ media_send690+ .take()691+ .expect("media handshake stream is present")692+ .finish()693+ .context("finishing media v2 handshake request")?;694+ drop(media_recv.take());695+ }696+ MediaTransport::ReliableV1 => {}346697 }347698 348699 let input_connection = endpoint@@ -464,20 +815,35 @@ async fn main() -> Result<()> {
464815 )465816 });466817 let mut object_sequence = MediaObjectSequence::new();818+ let mut object_receiver_v3 = (media_transport == MediaTransport::GroupedV3).then(|| {819+ MediaObjectReceiverV3::new(820+ media_connection.clone(),821+ Duration::from_secs(args.timeout_seconds),822+ )823+ });824+ let mut object_sequence_v3 = MediaObjectSequenceV3::new();825+ let mut keyframe_request_sent = false;826+ let mut keyframe_recovery_verified = false;827+ let mut keyframe_request_group_id = None;828+ let mut keyframe_request_last_sequence = None;467829 468830 while received < args.frames {469- let frame = match media_transport {470- MediaTransport::ReliableV1 => tokio::time::timeout(471- Duration::from_secs(args.timeout_seconds),472- read_media_frame(473- media_recv474- .as_mut()475- .expect("v1 media receive stream is present"),476- ),477- )478- .await479- .context("timed out waiting for media frame")??480- .context("host closed the media stream")?,831+ let (frame, recovery_frame): (AcceptedMedia, bool) = match media_transport {832+ MediaTransport::ReliableV1 => (833+ tokio::time::timeout(834+ Duration::from_secs(args.timeout_seconds),835+ read_media_frame(836+ media_recv837+ .as_mut()838+ .expect("v1 media receive stream is present"),839+ ),840+ )841+ .await842+ .context("timed out waiting for media frame")??843+ .context("host closed the media stream")?844+ .into(),845+ false,846+ ),481847 MediaTransport::IndependentV2 => loop {482848 let outcome = tokio::time::timeout(483849 Duration::from_secs(args.timeout_seconds)@@ -503,7 +869,7 @@ async fn main() -> Result<()> {
503869 }504870 MediaObjectOutcome::Frame { index, frame } => {505871 match object_sequence.classify(index, &frame) {506- MediaObjectDecision::Deliver => break frame,872+ MediaObjectDecision::Deliver => break (frame.into(), false),507873 MediaObjectDecision::DropLate => {508874 media_objects_late = media_objects_late.saturating_add(1);509875 }@@ -514,6 +880,78 @@ async fn main() -> Result<()> {
514880 }515881 }516882 },883+ MediaTransport::GroupedV3 => loop {884+ let outcome = tokio::time::timeout(885+ Duration::from_secs(args.timeout_seconds)886+ .saturating_add(Duration::from_secs(1)),887+ object_receiver_v3888+ .as_mut()889+ .expect("v3 media object receiver is present")890+ .next(),891+ )892+ .await893+ .context("timed out waiting for media v3 object")??894+ .context("host closed the media v3 object connection")?;895+ match outcome {896+ MediaObjectOutcomeV3::Dropped { .. } => {897+ if object_sequence_v3.note_drop() {898+ media_objects_dropped = media_objects_dropped.saturating_add(1);899+ } else {900+ media_objects_late = media_objects_late.saturating_add(1);901+ }902+ }903+ MediaObjectOutcomeV3::Malformed {904+ accept_index,905+ error,906+ } => {907+ bail!("media v3 object {accept_index} is malformed: {error}");908+ }909+ MediaObjectOutcomeV3::Object { object, .. } => {910+ match object_sequence_v3.classify(&object) {911+ MediaObjectDecisionV3::Deliver { discontinuity } => {912+ let recovery = keyframe_request_sent && !keyframe_recovery_verified;913+ if recovery {914+ ensure!(915+ discontinuity916+ && object.header.object_id == 0917+ && object.header.flags.contains(FrameFlags::KEYFRAME)918+ && object919+ .header920+ .flags921+ .contains(FrameFlags::CODEC_CONFIG)922+ && object923+ .header924+ .flags925+ .contains(FrameFlags::DISCONTINUITY),926+ "keyframe request recovery did not begin at a configured discontinuity object zero"927+ );928+ ensure!(929+ keyframe_request_group_id930+ .is_some_and(931+ |group_id| object.header.group_id > group_id932+ ),933+ "keyframe request recovery did not advance the media group"934+ );935+ ensure!(936+ keyframe_request_last_sequence.is_some_and(|sequence| {937+ object.header.sequence > sequence938+ }),939+ "keyframe request recovery did not advance media sequence"940+ );941+ keyframe_recovery_verified = true;942+ }943+ break (object.into(), recovery);944+ }945+ MediaObjectDecisionV3::DropLate => {946+ media_objects_late = media_objects_late.saturating_add(1);947+ }948+ MediaObjectDecisionV3::DropUntilKeyframe => {949+ media_objects_dropped = media_objects_dropped.saturating_add(1);950+ }951+ }952+ }953+ }954+ },517955 };518956 519957 let accepted_at = Instant::now();@@ -522,13 +960,13 @@ async fn main() -> Result<()> {
522960 523961 if received == 0 {524962 ensure!(525- frame.header.flags.contains(FrameFlags::KEYFRAME)526- && frame.header.flags.contains(FrameFlags::CODEC_CONFIG),963+ frame.flags.contains(FrameFlags::KEYFRAME)964+ && frame.flags.contains(FrameFlags::CODEC_CONFIG),527965 "first media frame is not a decodable keyframe with codec configuration"528966 );529967 }530968 531- let frame_dimensions = (frame.header.width, frame.header.height);969+ let frame_dimensions = (frame.width, frame.height);532970 if let Some(expected) = dimensions {533971 ensure!(534972 frame_dimensions == expected,@@ -537,13 +975,41 @@ async fn main() -> Result<()> {
537975 } else {538976 dimensions = Some(frame_dimensions);539977 }540- if let Some(previous) = last_sequence {541- gaps += sequence_gap(previous, frame.header.sequence)?;978+ if let Some(previous) = last_sequence979+ && !recovery_frame980+ {981+ gaps += sequence_gap(previous, frame.sequence)?;542982 }543- last_sequence = Some(frame.header.sequence);544- keyframes += u32::from(frame.header.flags.contains(FrameFlags::KEYFRAME));545- bytes = bytes.saturating_add(frame.payload.len() as u64);983+ last_sequence = Some(frame.sequence);984+ keyframes += u32::from(frame.flags.contains(FrameFlags::KEYFRAME));985+ bytes = bytes.saturating_add(frame.payload_len as u64);546986 received += 1;987+988+ if args.keyframe_smoke989+ && !keyframe_request_sent990+ && received == KEYFRAME_SMOKE_REQUEST_AFTER_FRAMES991+ {992+ let request = MediaControlRequestV3::request_keyframe(993+ args.keyframe_request_id,994+ Some(frame.sequence),995+ KeyframeRequestReasonV3::DecoderReset,996+ );997+ tokio::time::timeout(998+ Duration::from_secs(1),999+ write_media_control_request_v3(1000+ media_send1001+ .as_mut()1002+ .expect("v3 media control stream is present"),1003+ &request,1004+ ),1005+ )1006+ .await1007+ .context("timed out writing v3 keyframe request")??;1008+ keyframe_request_group_id = frame.v3_group_id;1009+ keyframe_request_last_sequence = Some(frame.sequence);1010+ keyframe_request_sent = true;1011+ object_sequence_v3.request_recovery();1012+ }5471013 }5481014 5491015 let Some((width, height)) = dimensions else {@@ -557,6 +1023,10 @@ async fn main() -> Result<()> {
5571023 );5581024 ensure!(keyframes > 0, "probe received no H.264 keyframe");5591025 ensure!(gaps == 0, "probe observed {gaps} media sequence gaps");1026+ ensure!(1027+ !args.keyframe_smoke || keyframe_recovery_verified,1028+ "probe did not verify requested keyframe recovery"1029+ );5601030 let accepted_span = first_accepted_at5611031 .zip(last_accepted_at)5621032 .map_or(Duration::ZERO, |(first, last)| {@@ -597,6 +1067,19 @@ async fn main() -> Result<()> {
5971067 println!("media_objects_dropped={media_objects_dropped}");5981068 println!("media_objects_late={media_objects_late}");5991069 println!("transport={}", media_transport.label());1070+ println!(1071+ "keyframe_recovery={}",1072+ if args.keyframe_smoke {1073+ "ok"1074+ } else {1075+ "not-requested"1076+ }1077+ );1078+ if args.keyframe_smoke {1079+ println!("keyframe_request_id={}", args.keyframe_request_id);1080+ } else {1081+ println!("keyframe_request_id=not-requested");1082+ }6001083 match accepted_fps {6011084 Some(fps) => println!("accepted_fps={fps:.3}"),6021085 None => println!("accepted_fps=unknown"),@@ -827,6 +1310,43 @@ mod tests {
8271310 }8281311 }8291312 1313+ fn media_object_v3(1314+ group_id: u64,1315+ object_id: u32,1316+ sequence: u64,1317+ flags: FrameFlags,1318+ ) -> MediaObjectV3 {1319+ let payload = vec![0x65, 0x88, 0x84];1320+ let header = sigil_protocol::MediaObjectHeaderV3::h264(1321+ 1_280,1322+ 800,1323+ payload.len(),1324+ if object_id == 0 { 0 } else { 128 },1325+ flags,1326+ object_id,1327+ group_id,1328+ sequence,1329+ sequence * 1_000,1330+ i64::try_from(sequence * 1_000).unwrap(),1331+ 100,1332+ )1333+ .unwrap();1334+ MediaObjectV3::new(header, payload).unwrap()1335+ }1336+1337+ fn media_outcome_v3(1338+ accept_index: u64,1339+ group_id: u64,1340+ object_id: u32,1341+ sequence: u64,1342+ flags: FrameFlags,1343+ ) -> MediaObjectOutcomeV3 {1344+ MediaObjectOutcomeV3::Object {1345+ accept_index,1346+ object: media_object_v3(group_id, object_id, sequence, flags),1347+ }1348+ }1349+8301350 8311351 fn media_object_reorder_restores_accept_order_before_sequence_checks() {8321352 let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);@@ -883,6 +1403,55 @@ mod tests {
8831403 );8841404 }8851405 1406+ 1407+ fn media_v3_discontinuity_object_zero_fast_forwards_accept_order() {1408+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);1409+ let barrier = keyframe.union(FrameFlags::DISCONTINUITY);1410+ let mut reorder = MediaObjectReorderV3::new(0);1411+ let mut sequence = MediaObjectSequenceV3::new();1412+1413+ assert!(1414+ reorder1415+ .push(media_outcome_v3(1, 10, 1, 11, FrameFlags::NONE))1416+ .unwrap()1417+ .is_none()1418+ );1419+ let recovered = reorder1420+ .push(media_outcome_v3(2, 20, 0, 20, barrier))1421+ .unwrap()1422+ .unwrap();1423+ assert_eq!(recovered.accept_index(), 2);1424+ let MediaObjectOutcomeV3::Object { object, .. } = recovered else {1425+ panic!("recovery barrier must remain an object");1426+ };1427+ assert_eq!(1428+ sequence.classify(&object),1429+ MediaObjectDecisionV3::Deliver {1430+ discontinuity: true1431+ }1432+ );1433+ assert_eq!(reorder.pending_len(), 0);1434+ assert!(1435+ reorder1436+ .push(MediaObjectOutcomeV3::Dropped { accept_index: 0 })1437+ .unwrap()1438+ .is_none()1439+ );1440+ let next = reorder1441+ .push(media_outcome_v3(3, 20, 1, 21, FrameFlags::NONE))1442+ .unwrap()1443+ .unwrap();1444+ let MediaObjectOutcomeV3::Object { object, .. } = next else {1445+ panic!("recovered-group delta must remain an object");1446+ };1447+ assert_eq!(1448+ sequence.classify(&object),1449+ MediaObjectDecisionV3::Deliver {1450+ discontinuity: false1451+ }1452+ );1453+ }1454+8861455 8871456 fn minimum_fps_parser_and_first_to_last_evaluation_are_bounded() {8881457 assert_eq!(parse_minimum_fps("55").unwrap(), 55.0);@@ -970,4 +1539,107 @@ mod tests {
9701539 MediaObjectDecision::Deliver9711540 );9721541 }1542+1543+ 1544+ fn media_v3_enforces_wire_group_object_and_sequence_continuity() {1545+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);1546+ let barrier = keyframe.union(FrameFlags::DISCONTINUITY);1547+ let mut sequence = MediaObjectSequenceV3::new();1548+1549+ assert_eq!(1550+ sequence.classify(&media_object_v3(10, 0, 10, keyframe)),1551+ MediaObjectDecisionV3::Deliver {1552+ discontinuity: true1553+ }1554+ );1555+ assert_eq!(1556+ sequence.classify(&media_object_v3(10, 1, 11, FrameFlags::NONE)),1557+ MediaObjectDecisionV3::Deliver {1558+ discontinuity: false1559+ }1560+ );1561+ assert_eq!(1562+ sequence.classify(&media_object_v3(10, 3, 13, FrameFlags::NONE)),1563+ MediaObjectDecisionV3::DropUntilKeyframe1564+ );1565+ assert_eq!(1566+ sequence.classify(&media_object_v3(10, 2, 12, FrameFlags::NONE)),1567+ MediaObjectDecisionV3::DropUntilKeyframe1568+ );1569+ assert_eq!(1570+ sequence.classify(&media_object_v3(20, 0, 20, barrier)),1571+ MediaObjectDecisionV3::Deliver {1572+ discontinuity: true1573+ }1574+ );1575+ assert_eq!(1576+ sequence.classify(&media_object_v3(10, 2, 12, FrameFlags::NONE)),1577+ MediaObjectDecisionV3::DropLate1578+ );1579+ }1580+1581+ 1582+ fn explicit_v3_recovery_never_delivers_same_group_delta_history() {1583+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);1584+ let barrier = keyframe.union(FrameFlags::DISCONTINUITY);1585+ let mut sequence = MediaObjectSequenceV3::new();1586+1587+ assert!(matches!(1588+ sequence.classify(&media_object_v3(10, 0, 10, keyframe)),1589+ MediaObjectDecisionV3::Deliver { .. }1590+ ));1591+ assert!(matches!(1592+ sequence.classify(&media_object_v3(10, 1, 11, FrameFlags::NONE)),1593+ MediaObjectDecisionV3::Deliver { .. }1594+ ));1595+ sequence.request_recovery();1596+ assert_eq!(1597+ sequence.classify(&media_object_v3(10, 2, 12, FrameFlags::NONE)),1598+ MediaObjectDecisionV3::DropUntilKeyframe1599+ );1600+ assert_eq!(1601+ sequence.classify(&media_object_v3(20, 0, 20, keyframe)),1602+ MediaObjectDecisionV3::DropUntilKeyframe,1603+ "recovery after skipped history must be explicitly discontinuous"1604+ );1605+ assert_eq!(1606+ sequence.classify(&media_object_v3(30, 0, 30, barrier)),1607+ MediaObjectDecisionV3::Deliver {1608+ discontinuity: true1609+ }1610+ );1611+ }1612+1613+ 1614+ fn media_v3_group_payload_is_bounded_by_the_shared_protocol_limit() {1615+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);1616+ let mut sequence = MediaObjectSequenceV3::new();1617+ assert!(matches!(1618+ sequence.classify(&media_object_v3(1, 0, 1, keyframe)),1619+ MediaObjectDecisionV3::Deliver { .. }1620+ ));1621+ sequence.group_payload_bytes = MAX_MEDIA_GROUP_BYTES_V3;1622+ assert_eq!(1623+ sequence.classify(&media_object_v3(1, 1, 2, FrameFlags::NONE)),1624+ MediaObjectDecisionV3::DropUntilKeyframe1625+ );1626+ }1627+1628+ 1629+ fn media_compatibility_flags_are_mutually_exclusive() {1630+ let node_id = SecretKey::from_bytes(&[0x2a; 32]).public().to_string();1631+ let default = Args::try_parse_from(["sigil-probe", "--node-id", &node_id]).unwrap();1632+ assert!(!default.media_v1);1633+ assert!(!default.media_v2);1634+ assert!(1635+ Args::try_parse_from([1636+ "sigil-probe",1637+ "--node-id",1638+ &node_id,1639+ "--media-v1",1640+ "--media-v2",1641+ ])1642+ .is_err()1643+ );1644+ }9731645 }@@ -24,7 +24,7 @@ use iroh::protocol::Router;
2424 use iroh::{Endpoint, EndpointId};2525 use sigil_protocol::{2626 AUDIO_ALPN_V1, INPUT_ALPN_V1, InvitationGrants, MAX_INVITATION_TTL_SECS, MEDIA_ALPN_V1,27- MEDIA_ALPN_V2, SignedInvitation,27+ MEDIA_ALPN_V2, MEDIA_ALPN_V3, SignedInvitation,2828 };2929 use tracing::{info, warn};3030 use tracing_subscriber::EnvFilter;@@ -35,7 +35,9 @@ use crate::authorization::{
3535 use crate::config::{HostConfig, InputMode, VideoSource};3636 use crate::cursor::PointerPositionTracker;3737 use crate::input::InputBackend;38-use crate::server::{AudioHandler, InputHandler, MediaHandler, MediaV2Handler, SessionRegistry};38+use crate::server::{39+ AudioHandler, InputHandler, MediaHandler, MediaV2Handler, MediaV3Handler, SessionRegistry,40+};3941 4042 const CONNECTION_IDLE_TIMEOUT: Duration = Duration::from_secs(5);4143 const CONNECTION_KEEP_ALIVE_INTERVAL: Duration = Duration::from_secs(1);@@ -637,6 +639,14 @@ async fn serve_command(args: ServeArgs) -> Result<()> {
637639 638640 let sessions = Arc::new(SessionRegistry::default());639641 let router = Router::builder(endpoint)642+ .accept(643+ MEDIA_ALPN_V3,644+ MediaV3Handler {645+ config: config.clone(),646+ sessions: Arc::clone(&sessions),647+ authorization: authorization.clone(),648+ },649+ )640650 .accept(641651 MEDIA_ALPN_V2,642652 MediaV2Handler {@@ -10,9 +10,12 @@ use iroh::endpoint::{Connection, SendStream};
1010 use iroh::protocol::ProtocolHandler;1111 use sigil_protocol::{1212 AUDIO_HEADER_LEN, AudioFlags, AudioPacket, AudioPacketHeader, Capability, ClientHello,13- FrameFlags, HostHello, InputAck, InvitationGrants, MAX_AUDIO_PAYLOAD_LEN, MediaFrame,14- MediaFrameHeader, read_client_hello, read_input_event, write_host_hello, write_input_ack,15- write_media_frame,13+ FrameFlags, HostHello, InputAck, InvitationGrants, KeyframeRequestReasonV3,14+ MAX_AUDIO_PAYLOAD_LEN, MAX_MEDIA_GROUP_BYTES_V3, MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS,15+ MAX_MEDIA_OBJECT_ID_V3, MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS, MediaControlRequestV3,16+ MediaFrame, MediaFrameHeader, MediaObjectHeaderV3, MediaObjectV3, read_client_hello,17+ read_input_event, read_media_control_request_v3, write_host_hello, write_input_ack,18+ write_media_frame, write_media_object_v3,1619 };1720 use tracing::{debug, error, info, warn};1821 @@ -36,6 +39,17 @@ const MEDIA_V2_IN_FLIGHT_CAPACITY: usize = 4;
3639 const MEDIA_V2_KEYFRAME_PRIORITY: i32 = 10;3740 const MEDIA_V2_DELTA_PRIORITY: i32 = 0;3841 const MEDIA_V2_RESET_CODE: u32 = 0x5356;42+const MEDIA_V3_IN_FLIGHT_CAPACITY: usize = 4;43+const MEDIA_V3_KEYFRAME_PUBLISHER_PRIORITY: u8 = 0;44+const MEDIA_V3_DELTA_PUBLISHER_PRIORITY: u8 = 128;45+const MEDIA_V3_KEYFRAME_TRANSPORT_PRIORITY: i32 = 10;46+const MEDIA_V3_DELTA_TRANSPORT_PRIORITY: i32 = 0;47+// Wire-visible application reset code. Keep stable across compatible v3 releases.48+const MEDIA_V3_RESET_CODE: u32 = 0x5357;49+const MEDIA_V3_DELIVERY_FRAME_PERIODS: u64 = 4;50+const MEDIA_V3_MAX_CONTROL_REQUESTS_PER_SECOND: u32 = 10;51+const MEDIA_V3_CONTROL_REQUEST_INTERVAL: Duration =52+ Duration::from_millis(1_000 / MEDIA_V3_MAX_CONTROL_REQUESTS_PER_SECOND as u64);3953 const INPUT_ACK_TIMEOUT: Duration = Duration::from_secs(1);4054 const REJECTION_DRAIN_TIMEOUT: Duration = Duration::from_secs(1);4155 const MAX_PENDING_HANDSHAKES: usize = 4;@@ -575,6 +589,357 @@ impl MediaV2Scheduler {
575589 }576590 }577591 592+593+struct MediaV3ObjectPosition {594+ group_id: u64,595+ object_id: u32,596+ discontinuity: bool,597+}598+599+600+enum MediaV3GroupDecision {601+ Send(MediaV3ObjectPosition),602+ SkipUntilKeyframe,603+ EnterResync,604+}605+606+607+struct MediaV3GroupCursor {608+ group_id: Option<u64>,609+ last_sequence: Option<u64>,610+ last_object_id: Option<u32>,611+ payload_bytes: usize,612+ waiting_for_keyframe: bool,613+ discontinuity_pending: bool,614+}615+616+impl Default for MediaV3GroupCursor {617+ fn default() -> Self {618+ Self {619+ group_id: None,620+ last_sequence: None,621+ last_object_id: None,622+ payload_bytes: 0,623+ waiting_for_keyframe: true,624+ discontinuity_pending: false,625+ }626+ }627+}628+629+impl MediaV3GroupCursor {630+ fn classify(&mut self, frame: &EncodedFrame) -> MediaV3GroupDecision {631+ let independently_decodable = frame.keyframe && frame.codec_config;632+ if independently_decodable {633+ if frame.data.len() > MAX_MEDIA_GROUP_BYTES_V3 {634+ self.enter_resync();635+ return MediaV3GroupDecision::EnterResync;636+ }637+ let sequence_discontinuity = self638+ .last_sequence639+ .is_some_and(|last| last.checked_add(1) != Some(frame.sequence));640+ let position = MediaV3ObjectPosition {641+ group_id: frame.sequence,642+ object_id: 0,643+ discontinuity: self.discontinuity_pending || sequence_discontinuity,644+ };645+ self.group_id = Some(frame.sequence);646+ self.last_sequence = Some(frame.sequence);647+ self.last_object_id = Some(0);648+ self.payload_bytes = frame.data.len();649+ self.waiting_for_keyframe = false;650+ self.discontinuity_pending = false;651+ return MediaV3GroupDecision::Send(position);652+ }653+654+ if self.waiting_for_keyframe {655+ return MediaV3GroupDecision::SkipUntilKeyframe;656+ }657+ if frame.keyframe || frame.codec_config {658+ self.enter_resync();659+ return MediaV3GroupDecision::EnterResync;660+ }661+ let Some(group_id) = self.group_id else {662+ self.enter_resync();663+ return MediaV3GroupDecision::EnterResync;664+ };665+ let contiguous =666+ self.last_sequence.and_then(|last| last.checked_add(1)) == Some(frame.sequence);667+ let object_id = self.last_object_id.and_then(|last| last.checked_add(1));668+ let payload_bytes = self.payload_bytes.checked_add(frame.data.len());669+ let (Some(object_id), Some(payload_bytes)) = (object_id, payload_bytes) else {670+ self.enter_resync();671+ return MediaV3GroupDecision::EnterResync;672+ };673+ if !contiguous674+ || object_id > MAX_MEDIA_OBJECT_ID_V3675+ || payload_bytes > MAX_MEDIA_GROUP_BYTES_V3676+ {677+ self.enter_resync();678+ return MediaV3GroupDecision::EnterResync;679+ }680+681+ self.last_sequence = Some(frame.sequence);682+ self.last_object_id = Some(object_id);683+ self.payload_bytes = payload_bytes;684+ MediaV3GroupDecision::Send(MediaV3ObjectPosition {685+ group_id,686+ object_id,687+ discontinuity: false,688+ })689+ }690+691+ fn request_keyframe(&mut self) {692+ self.enter_resync();693+ }694+695+ fn enter_resync(&mut self) {696+ self.group_id = None;697+ self.last_object_id = None;698+ self.payload_bytes = 0;699+ self.waiting_for_keyframe = true;700+ self.discontinuity_pending = true;701+ }702+}703+704+705+enum MediaV3ScheduleDecision {706+ Send {707+ discontinuity: bool,708+ cancel_sequences: Vec<u64>,709+ },710+ SkipUntilKeyframe,711+ EnterResync {712+ cancel_sequences: Vec<u64>,713+ },714+}715+716+717+struct MediaV3Scheduler {718+ in_flight: Vec<u64>,719+ last_scheduled_sequence: Option<u64>,720+ waiting_for_keyframe: bool,721+ discontinuity_pending: bool,722+}723+724+impl Default for MediaV3Scheduler {725+ fn default() -> Self {726+ Self {727+ in_flight: Vec::with_capacity(MEDIA_V3_IN_FLIGHT_CAPACITY),728+ last_scheduled_sequence: None,729+ waiting_for_keyframe: true,730+ discontinuity_pending: false,731+ }732+ }733+}734+735+impl MediaV3Scheduler {736+ fn schedule(737+ &mut self,738+ sequence: u64,739+ independently_decodable: bool,740+ ) -> MediaV3ScheduleDecision {741+ if self742+ .last_scheduled_sequence743+ .is_some_and(|last| sequence <= last)744+ {745+ return MediaV3ScheduleDecision::SkipUntilKeyframe;746+ }747+748+ let sequence_discontinuity = self749+ .last_scheduled_sequence750+ .is_some_and(|last| last.checked_add(1) != Some(sequence));751+ if independently_decodable {752+ let cancel_sequences = std::mem::take(&mut self.in_flight);753+ let discontinuity = self.discontinuity_pending754+ || sequence_discontinuity755+ || !cancel_sequences.is_empty();756+ self.in_flight.push(sequence);757+ self.last_scheduled_sequence = Some(sequence);758+ self.waiting_for_keyframe = false;759+ self.discontinuity_pending = false;760+ return MediaV3ScheduleDecision::Send {761+ discontinuity,762+ cancel_sequences,763+ };764+ }765+766+ if self.waiting_for_keyframe {767+ return MediaV3ScheduleDecision::SkipUntilKeyframe;768+ }769+ if sequence_discontinuity || self.in_flight.len() == MEDIA_V3_IN_FLIGHT_CAPACITY {770+ return MediaV3ScheduleDecision::EnterResync {771+ cancel_sequences: self.enter_resync(),772+ };773+ }774+775+ self.in_flight.push(sequence);776+ self.last_scheduled_sequence = Some(sequence);777+ MediaV3ScheduleDecision::Send {778+ discontinuity: false,779+ cancel_sequences: Vec::new(),780+ }781+ }782+783+ fn complete(&mut self, sequence: u64) {784+ if let Some(index) = self.in_flight.iter().position(|value| *value == sequence) {785+ self.in_flight.swap_remove(index);786+ }787+ }788+789+ fn fail(&mut self, sequence: u64) -> Option<Vec<u64>> {790+ let index = self.in_flight.iter().position(|value| *value == sequence)?;791+ self.in_flight.swap_remove(index);792+ Some(self.enter_resync())793+ }794+795+ fn fail_all(&mut self) -> Vec<u64> {796+ self.enter_resync()797+ }798+799+ fn request_keyframe(&mut self) -> Vec<u64> {800+ if self.waiting_for_keyframe {801+ self.discontinuity_pending = true;802+ return Vec::new();803+ }804+ self.enter_resync()805+ }806+807+ fn enter_resync(&mut self) -> Vec<u64> {808+ self.waiting_for_keyframe = true;809+ self.discontinuity_pending = true;810+ std::mem::take(&mut self.in_flight)811+ }812+}813+814+fn apply_media_v3_keyframe_request(815+ scheduler: &mut MediaV3Scheduler,816+ group_cursor: &mut MediaV3GroupCursor,817+ replay_cursor: &mut MediaReplayCursor,818+ through_sequence: Option<u64>,819+ reason: KeyframeRequestReasonV3,820+) -> (bool, Vec<u64>) {821+ // Every v3 session already begins by replaying the bounded current GOP822+ // from object zero. A Join arriving after that replay was scheduled must823+ // not cancel the only decodable image on a damage-driven static source.824+ if reason == KeyframeRequestReasonV3::Join || scheduler.waiting_for_keyframe {825+ return (false, Vec::new());826+ }827+ let cancel_sequences = scheduler.request_keyframe();828+ group_cursor.request_keyframe();829+ replay_cursor.enter_resync_through(through_sequence);830+ (true, cancel_sequences)831+}832+833+fn apply_media_v3_send_failure(834+ scheduler: &mut MediaV3Scheduler,835+ group_cursor: &mut MediaV3GroupCursor,836+ replay_cursor: &mut MediaReplayCursor,837+ sequence: u64,838+ through_sequence: Option<u64>,839+) -> Option<Vec<u64>> {840+ let cancel_sequences = scheduler.fail(sequence)?;841+ group_cursor.request_keyframe();842+ replay_cursor.enter_resync_through(through_sequence);843+ Some(cancel_sequences)844+}845+846+847+enum MediaV3ControlDecision {848+ Accept,849+ Pace(Duration),850+}851+852+853+struct MediaV3ControlGate {854+ last_request_id: Option<u64>,855+ last_accepted_at: Option<Instant>,856+}857+858+impl MediaV3ControlGate {859+ fn accept(860+ &mut self,861+ request: MediaControlRequestV3,862+ now: Instant,863+ ) -> Result<MediaV3ControlDecision> {864+ ensure!(865+ self.last_request_id866+ .is_none_or(|last| request.request_id > last),867+ "v3 media control request IDs must be strictly increasing"868+ );869+ self.last_request_id = Some(request.request_id);870+ if let Some(last) = self.last_accepted_at {871+ let elapsed = now.saturating_duration_since(last);872+ if elapsed < MEDIA_V3_CONTROL_REQUEST_INTERVAL {873+ return Ok(MediaV3ControlDecision::Pace(874+ MEDIA_V3_CONTROL_REQUEST_INTERVAL - elapsed,875+ ));876+ }877+ }878+ self.last_accepted_at = Some(now);879+ Ok(MediaV3ControlDecision::Accept)880+ }881+}882+883+async fn forward_media_v3_control_requests<R>(884+ mut reader: R,885+ sender: tokio::sync::watch::Sender<Option<MediaControlRequestV3>>,886+) -> Result<()>887+where888+ R: tokio::io::AsyncRead + Unpin,889+{890+ let mut gate = MediaV3ControlGate::default();891+ while let Some(request) = read_media_control_request_v3(&mut reader).await? {892+ match gate.accept(request, Instant::now())? {893+ MediaV3ControlDecision::Accept => {}894+ MediaV3ControlDecision::Pace(retry_after) => {895+ // Stop reading during the rejection interval so QUIC flow896+ // control, rather than this task, absorbs an abusive burst.897+ // This also bounds rejection logging to the configured rate.898+ debug!(899+ request_id = request.request_id,900+ retry_after_ms = retry_after.as_millis(),901+ "paced rate-limited v3 keyframe request"902+ );903+ tokio::time::sleep(retry_after).await;904+ continue;905+ }906+ }907+ sender.send_replace(Some(request));908+ if sender.is_closed() {909+ return Ok(());910+ }911+ }912+ Ok(())913+}914+915+struct ResetOnDropSendStreamV3(Option<SendStream>);916+917+impl ResetOnDropSendStreamV3 {918+ fn new(stream: SendStream) -> Self {919+ Self(Some(stream))920+ }921+922+ fn stream(&self) -> &SendStream {923+ self.0.as_ref().expect("send stream guard is armed")924+ }925+926+ fn stream_mut(&mut self) -> &mut SendStream {927+ self.0.as_mut().expect("send stream guard is armed")928+ }929+930+ fn disarm(&mut self) {931+ self.0.take();932+ }933+}934+935+impl Drop for ResetOnDropSendStreamV3 {936+ fn drop(&mut self) {937+ if let Some(stream) = self.0.as_mut() {938+ let _ = stream.reset(MEDIA_V3_RESET_CODE.into());939+ }940+ }941+}942+578943 struct ResetOnDropSendStream(Option<SendStream>);579944 580945 impl ResetOnDropSendStream {@@ -662,6 +1027,17 @@ impl MediaReplayCursor {
6621027 self.waiting_for_keyframe = false;6631028 self.discontinuity_pending = false;6641029 }1030+1031+ fn enter_resync_through(&mut self, through_sequence: Option<u64>) {1032+ if let Some(through_sequence) = through_sequence {1033+ self.last_sequence = Some(1034+ self.last_sequence1035+ .map_or(through_sequence, |last| last.max(through_sequence)),1036+ );1037+ }1038+ self.waiting_for_keyframe = true;1039+ self.discontinuity_pending = true;1040+ }6651041 }6661042 6671043 fn maximum_media_replay_age(framerate: u32) -> Duration {@@ -713,6 +1089,13 @@ pub struct MediaV2Handler {
7131089 pub authorization: AuthorizationPolicy,7141090 }7151091 1092+1093+pub struct MediaV3Handler {1094+ pub config: HostConfig,1095+ pub sessions: Arc<SessionRegistry>,1096+ pub authorization: AuthorizationPolicy,1097+}1098+7161099 impl ProtocolHandler for MediaHandler {7171100 async fn accept(&self, connection: Connection) -> Result<(), iroh::protocol::AcceptError> {7181101 let remote = connection.remote_id();@@ -747,6 +1130,23 @@ impl ProtocolHandler for MediaV2Handler {
7471130 }7481131 }7491132 1133+impl ProtocolHandler for MediaV3Handler {1134+ async fn accept(&self, connection: Connection) -> Result<(), iroh::protocol::AcceptError> {1135+ let remote = connection.remote_id();1136+ if let Err(error) = serve_media_v3(1137+ connection,1138+ self.config.clone(),1139+ &self.sessions,1140+ &self.authorization,1141+ )1142+ .await1143+ {1144+ warn!(%remote, %error, "media v3 connection ended");1145+ }1146+ Ok(())1147+ }1148+}1149+7501150 7511151 pub struct InputHandler {7521152 pub backend: InputBackend,@@ -1056,29 +1456,127 @@ async fn serve_media_v2(
10561456 session_result10571457 }10581458 1059-async fn run_media_v2_session(1060- connection: &Connection,1061- config: &HostConfig,1062- current_gop_receiver: &mut tokio::sync::watch::Receiver<Option<EncodedGop>>,1063- remote: EndpointId,1459+async fn serve_media_v3(1460+ connection: Connection,1461+ config: HostConfig,1462+ sessions: &Arc<SessionRegistry>,1463+ authorization: &AuthorizationPolicy,10641464 ) -> Result<()> {1065- let maximum_replay_age = maximum_media_replay_age(config.framerate);1066- let mut replay_cursor = MediaReplayCursor::default();1067- let mut scheduler = MediaV2Scheduler::default();1068- let mut send_tasks = tokio::task::JoinSet::new();1465+ let remote = connection.remote_id();1466+ let handshake_permit = sessions1467+ .pending_handshakes1468+ .try_acquire()1469+ .context("too many pending handshakes")?;1470+ let (mut send, mut recv) = tokio::time::timeout(HANDSHAKE_TIMEOUT, connection.accept_bi())1471+ .await1472+ .context("timed out accepting media v3 handshake stream")?1473+ .context("accepting media v3 handshake stream")?;1474+ let hello = receive_hello(&mut recv, Capability::VideoH264).await?;1475+ drop(handshake_permit);1476+ debug!(%remote, agent = %hello.agent, "media v3 hello received");10691477 1070- let result = loop {1071- tokio::select! {1072- closed = connection.closed() => {1073- debug!(%remote, ?closed, "media v2 connection closed");1074- break Ok(());1075- }1076- task = send_tasks.join_next(), if !send_tasks.is_empty() => {1077- match task.expect("guarded by non-empty send task set") {1078- Ok((sequence, Ok(()))) => scheduler.complete(sequence),1079- Ok((sequence, Err(error))) => {1080- warn!(sequence, %error, "media v2 object send failed; waiting for keyframe");1081- if !scheduler.fail(sequence).is_empty() {1478+ let grants = match authorization.authorize_or_redeem(1479+ remote,1480+ hello.invitation.as_deref(),1481+ unix_timestamp_now()?,1482+ ) {1483+ Ok(grants) => grants,1484+ Err(error) => {1485+ send_rejection(&mut send, "Portal peer is not authorized").await?;1486+ return Err(error.context("authorizing media v3 peer"));1487+ }1488+ };1489+ ensure!(1490+ grants.contains(InvitationGrants::VIEW),1491+ "authorized media peer lacks view permission"1492+ );1493+1494+ let lease = match sessions.claim(remote, hello.nonce, grants) {1495+ Ok(lease) => lease,1496+ Err(error) => {1497+ send_rejection(&mut send, "host already has an active client").await?;1498+ return Err(error);1499+ }1500+ };1501+1502+ let source = match config.source {1503+ VideoSource::TestPattern => Ok(spawn_test_pattern(config.clone(), lease.session_clock)),1504+ VideoSource::GamescopePipewire => {1505+ let primary = spawn_gamescope_pipewire_after_static_preflight(1506+ config.clone(),1507+ lease.session_clock,1508+ )1509+ .await?;1510+ select_gamescope_startup_source(config.clone(), lease.session_clock, primary).await1511+ }1512+ };1513+ let EncodedSource {1514+ frames: frame_receiver,1515+ current_gop: mut current_gop_receiver,1516+ task: source_task,1517+ pointer_surface_dimensions,1518+ } = match source {1519+ Ok(source) => source,1520+ Err(error) => {1521+ send_rejection(&mut send, "video source is unavailable").await?;1522+ return Err(error);1523+ }1524+ };1525+ let source_task = SourceTaskGuard::new(source_task);1526+1527+ let mut media_hello = HostHello::accepted(1528+ lease.session_id,1529+ negotiated_capabilities(&hello, MEDIA_CAPABILITIES),1530+ );1531+ if let Some(dimensions) = pointer_surface_dimensions {1532+ media_hello = media_hello.with_pointer_surface_dimensions(dimensions);1533+ }1534+ write_host_hello(&mut send, &media_hello).await?;1535+ send.finish()1536+ .context("finishing media v3 handshake response")?;1537+ drop(send);1538+ info!(%remote, session_id = lease.session_id, "media v3 client accepted");1539+1540+ let session_result = run_media_v3_session(1541+ &connection,1542+ &config,1543+ &mut current_gop_receiver,1544+ recv,1545+ remote,1546+ )1547+ .await;1548+1549+ drop(current_gop_receiver);1550+ drop(frame_receiver);1551+ source_task.wait_or_abort(SOURCE_REAP_GRACE_TIMEOUT).await;1552+ drop(lease);1553+ info!(%remote, "media v3 client released");1554+ session_result1555+}1556+1557+async fn run_media_v2_session(1558+ connection: &Connection,1559+ config: &HostConfig,1560+ current_gop_receiver: &mut tokio::sync::watch::Receiver<Option<EncodedGop>>,1561+ remote: EndpointId,1562+) -> Result<()> {1563+ let maximum_replay_age = maximum_media_replay_age(config.framerate);1564+ let mut replay_cursor = MediaReplayCursor::default();1565+ let mut scheduler = MediaV2Scheduler::default();1566+ let mut send_tasks = tokio::task::JoinSet::new();1567+1568+ let result = loop {1569+ tokio::select! {1570+ closed = connection.closed() => {1571+ debug!(%remote, ?closed, "media v2 connection closed");1572+ break Ok(());1573+ }1574+ task = send_tasks.join_next(), if !send_tasks.is_empty() => {1575+ match task.expect("guarded by non-empty send task set") {1576+ Ok((sequence, Ok(()))) => scheduler.complete(sequence),1577+ Ok((sequence, Err(error))) => {1578+ warn!(sequence, %error, "media v2 object send failed; waiting for keyframe");1579+ if !scheduler.fail(sequence).is_empty() {10821580 send_tasks.abort_all();10831581 }10841582 }@@ -1154,42 +1652,390 @@ async fn run_media_v2_session(
11541652 // scheduling reorder the QUIC object sequence even though11551653 // each object carries a monotonic media sequence number.11561654 let stream = match tokio::time::timeout(1157- MEDIA_WRITE_TIMEOUT,1655+ MEDIA_WRITE_TIMEOUT,1656+ connection.open_uni(),1657+ )1658+ .await1659+ {1660+ Ok(Ok(stream)) => stream,1661+ Ok(Err(error)) => {1662+ warn!(sequence, %error, "opening media v2 object stream failed");1663+ if !scheduler.fail(sequence).is_empty() {1664+ send_tasks.abort_all();1665+ }1666+ continue;1667+ }1668+ Err(_) => {1669+ warn!(sequence, "opening media v2 object stream timed out");1670+ if !scheduler.fail(sequence).is_empty() {1671+ send_tasks.abort_all();1672+ }1673+ continue;1674+ }1675+ };1676+ let stream = ResetOnDropSendStream::new(stream);1677+ if let Err(error) = stream1678+ .stream()1679+ .set_priority(media_v2_priority(keyframe))1680+ {1681+ warn!(sequence, %error, "setting media v2 object priority failed");1682+ if !scheduler.fail(sequence).is_empty() {1683+ send_tasks.abort_all();1684+ }1685+ continue;1686+ }1687+ send_tasks.spawn(async move {1688+ (1689+ sequence,1690+ send_media_v2_object(stream, media_frame).await,1691+ )1692+ });1693+ replay_cursor.commit_sent(&frame);1694+ }1695+ }1696+ }1697+ };1698+1699+ send_tasks.abort_all();1700+ while send_tasks.join_next().await.is_some() {}1701+ result1702+}1703+1704+fn media_frame_for_encoded(1705+ config: &HostConfig,1706+ frame: &EncodedFrame,1707+ discontinuity: bool,1708+) -> Result<MediaFrame> {1709+ let mut flags = FrameFlags::NONE;1710+ if frame.keyframe {1711+ flags = flags.union(FrameFlags::KEYFRAME);1712+ }1713+ if frame.codec_config {1714+ flags = flags.union(FrameFlags::CODEC_CONFIG);1715+ }1716+ if discontinuity {1717+ flags = flags.union(FrameFlags::DISCONTINUITY);1718+ }1719+ let width = u16::try_from(config.width).context("width exceeds protocol")?;1720+ let height = u16::try_from(config.height).context("height exceeds protocol")?;1721+ let header = MediaFrameHeader::h264(1722+ width,1723+ height,1724+ frame.data.len(),1725+ frame.sequence,1726+ frame.capture_timestamp_micros,1727+ frame.presentation_timestamp_micros,1728+ flags,1729+ )?;1730+ MediaFrame::new(header, frame.data.as_ref().to_vec()).map_err(Into::into)1731+}1732+1733+async fn send_media_v2_object(mut stream: ResetOnDropSendStream, frame: MediaFrame) -> Result<()> {1734+ tokio::time::timeout(1735+ MEDIA_WRITE_TIMEOUT,1736+ write_media_frame(stream.stream_mut(), &frame),1737+ )1738+ .await1739+ .context("timed out writing media v2 object")??;1740+ stream1741+ .stream_mut()1742+ .finish()1743+ .context("finishing media v2 object stream")?;1744+ match tokio::time::timeout(MEDIA_V2_PEER_ACK_TIMEOUT, stream.stream().stopped())1745+ .await1746+ .context("timed out waiting for media v2 object acknowledgement")?1747+ .context("waiting for media v2 object acknowledgement")?1748+ {1749+ None => {1750+ stream.disarm();1751+ Ok(())1752+ }1753+ Some(code) => bail!("peer stopped media v2 object stream with code {code}"),1754+ }1755+}1756+1757+fn media_v2_priority(keyframe: bool) -> i32 {1758+ if keyframe {1759+ MEDIA_V2_KEYFRAME_PRIORITY1760+ } else {1761+ MEDIA_V2_DELTA_PRIORITY1762+ }1763+}1764+1765+async fn run_media_v3_session(1766+ connection: &Connection,1767+ config: &HostConfig,1768+ current_gop_receiver: &mut tokio::sync::watch::Receiver<Option<EncodedGop>>,1769+ control_recv: iroh::endpoint::RecvStream,1770+ remote: EndpointId,1771+) -> Result<()> {1772+ let maximum_replay_age = maximum_media_replay_age(config.framerate);1773+ let delivery_timeout_ms = media_v3_delivery_timeout_ms(config.framerate);1774+ let mut replay_cursor = MediaReplayCursor::default();1775+ let mut group_cursor = MediaV3GroupCursor::default();1776+ let mut scheduler = MediaV3Scheduler::default();1777+ let mut send_tasks = tokio::task::JoinSet::new();1778+ let (control_sender, mut control_requests) = tokio::sync::watch::channel(None);1779+ let mut control_task = tokio::spawn(forward_media_v3_control_requests(1780+ control_recv,1781+ control_sender,1782+ ));1783+ let mut control_task_finished = false;1784+ let mut control_receiver_open = true;1785+1786+ let result = loop {1787+ tokio::select! {1788+ biased;1789+ control_result = &mut control_task, if !control_task_finished => {1790+ control_task_finished = true;1791+ match control_result {1792+ Ok(Ok(())) => {1793+ // Keep polling the watch receiver after clean EOF so a1794+ // final request published immediately before the1795+ // sender closed cannot be lost.1796+ debug!(%remote, "media v3 control stream closed");1797+ }1798+ Ok(Err(error)) => {1799+ break Err(error).context("reading media v3 control stream");1800+ }1801+ Err(error) => {1802+ break Err(error).context("media v3 control task failed");1803+ }1804+ }1805+ }1806+ changed = control_requests.changed(), if control_receiver_open => {1807+ if changed.is_err() {1808+ control_receiver_open = false;1809+ continue;1810+ }1811+ let request = *control_requests.borrow_and_update();1812+ let Some(request) = request else {1813+ continue;1814+ };1815+ let through_sequence = current_gop_receiver1816+ .borrow()1817+ .as_ref()1818+ .and_then(|gop| gop.frames.last())1819+ .map(|frame| frame.sequence);1820+ let (transitioned, cancel_sequences) = apply_media_v3_keyframe_request(1821+ &mut scheduler,1822+ &mut group_cursor,1823+ &mut replay_cursor,1824+ through_sequence,1825+ request.reason,1826+ );1827+ if !cancel_sequences.is_empty() {1828+ send_tasks.abort_all();1829+ }1830+ debug!(1831+ %remote,1832+ request_id = request.request_id,1833+ ?request.reason,1834+ advisory_last_sequence = ?request.last_sequence,1835+ coalesced = !transitioned,1836+ ?cancel_sequences,1837+ "accepted media v3 keyframe request"1838+ );1839+ }1840+ closed = connection.closed() => {1841+ debug!(%remote, ?closed, "media v3 connection closed");1842+ break Ok(());1843+ }1844+ task = send_tasks.join_next(), if !send_tasks.is_empty() => {1845+ match task.expect("guarded by non-empty send task set") {1846+ Ok((sequence, Ok(()))) => scheduler.complete(sequence),1847+ Ok((sequence, Err(error))) => {1848+ let through_sequence = current_gop_receiver1849+ .borrow()1850+ .as_ref()1851+ .and_then(|gop| gop.frames.last())1852+ .map(|frame| frame.sequence);1853+ if let Some(cancel_sequences) = apply_media_v3_send_failure(1854+ &mut scheduler,1855+ &mut group_cursor,1856+ &mut replay_cursor,1857+ sequence,1858+ through_sequence,1859+ ) {1860+ warn!(1861+ sequence,1862+ %error,1863+ "media v3 object send failed; waiting for keyframe"1864+ );1865+ if !cancel_sequences.is_empty() {1866+ send_tasks.abort_all();1867+ }1868+ } else {1869+ debug!(1870+ sequence,1871+ %error,1872+ "ignored stale media v3 object failure from a superseded group"1873+ );1874+ }1875+ }1876+ Err(error) if error.is_cancelled() => {}1877+ Err(error) => {1878+ warn!(%error, "media v3 object task failed; waiting for keyframe");1879+ let cancel_sequences = scheduler.fail_all();1880+ group_cursor.request_keyframe();1881+ let through_sequence = current_gop_receiver1882+ .borrow()1883+ .as_ref()1884+ .and_then(|gop| gop.frames.last())1885+ .map(|frame| frame.sequence);1886+ replay_cursor.enter_resync_through(through_sequence);1887+ if !cancel_sequences.is_empty() {1888+ send_tasks.abort_all();1889+ }1890+ }1891+ }1892+ }1893+ changed = current_gop_receiver.changed() => {1894+ if let Err(error) = changed {1895+ break Err(error).context("encoded source stopped");1896+ }1897+ let Some(current_gop) = current_gop_receiver.borrow_and_update().clone() else {1898+ continue;1899+ };1900+ let initial_replay_started_at =1901+ replay_cursor.last_sequence.is_none().then(Instant::now);1902+ let replay_through_sequence = current_gop1903+ .frames1904+ .last()1905+ .map(|frame| frame.sequence)1906+ .context("current GOP snapshot is empty")?;1907+1908+ for frame in new_current_gop_frames(current_gop, replay_cursor.last_sequence) {1909+ let replay_discontinuity = match replay_cursor.classify(1910+ &frame,1911+ replay_through_sequence,1912+ initial_replay_started_at,1913+ Instant::now(),1914+ maximum_replay_age,1915+ ) {1916+ MediaReplayDecision::Send { discontinuity } => discontinuity,1917+ MediaReplayDecision::SkipUntilKeyframe => {1918+ replay_cursor.enter_resync_through(Some(replay_through_sequence));1919+ break;1920+ }1921+ MediaReplayDecision::DiscardStaleSuffix { .. } => {1922+ group_cursor.request_keyframe();1923+ let cancel_sequences = scheduler.fail_all();1924+ if !cancel_sequences.is_empty() {1925+ send_tasks.abort_all();1926+ }1927+ break;1928+ }1929+ };1930+ let position = match group_cursor.classify(&frame) {1931+ MediaV3GroupDecision::Send(position) => position,1932+ MediaV3GroupDecision::SkipUntilKeyframe => {1933+ replay_cursor.enter_resync_through(Some(replay_through_sequence));1934+ break;1935+ }1936+ MediaV3GroupDecision::EnterResync => {1937+ replay_cursor.enter_resync_through(Some(replay_through_sequence));1938+ let cancel_sequences = scheduler.fail_all();1939+ if !cancel_sequences.is_empty() {1940+ send_tasks.abort_all();1941+ }1942+ break;1943+ }1944+ };1945+ let independently_decodable = position.object_id == 0;1946+ let (scheduler_discontinuity, cancel_sequences) =1947+ match scheduler.schedule(frame.sequence, independently_decodable) {1948+ MediaV3ScheduleDecision::Send {1949+ discontinuity,1950+ cancel_sequences,1951+ } => (discontinuity, cancel_sequences),1952+ MediaV3ScheduleDecision::SkipUntilKeyframe => continue,1953+ MediaV3ScheduleDecision::EnterResync { cancel_sequences } => {1954+ group_cursor.request_keyframe();1955+ replay_cursor.enter_resync_through(Some(replay_through_sequence));1956+ if !cancel_sequences.is_empty() {1957+ send_tasks.abort_all();1958+ }1959+ break;1960+ }1961+ };1962+ if !cancel_sequences.is_empty() {1963+ debug!(1964+ sequence = frame.sequence,1965+ group_id = position.group_id,1966+ ?cancel_sequences,1967+ "configured keyframe superseding media v3 objects"1968+ );1969+ send_tasks.abort_all();1970+ }1971+1972+ let media_object = media_v3_object_for_encoded(1973+ config,1974+ &frame,1975+ position,1976+ replay_discontinuity1977+ || position.discontinuity1978+ || scheduler_discontinuity,1979+ delivery_timeout_ms,1980+ )?;1981+ let sequence = frame.sequence;1982+ let stream = match tokio::time::timeout(1983+ Duration::from_millis(u64::from(delivery_timeout_ms)),11581984 connection.open_uni(),11591985 )11601986 .await11611987 {11621988 Ok(Ok(stream)) => stream,11631989 Ok(Err(error)) => {1164- warn!(sequence, %error, "opening media v2 object stream failed");1165- if !scheduler.fail(sequence).is_empty() {1990+ warn!(sequence, %error, "opening media v3 object stream failed");1991+ if let Some(cancel_sequences) = apply_media_v3_send_failure(1992+ &mut scheduler,1993+ &mut group_cursor,1994+ &mut replay_cursor,1995+ sequence,1996+ Some(replay_through_sequence),1997+ ) && !cancel_sequences.is_empty()1998+ {11661999 send_tasks.abort_all();11672000 }1168- continue;2001+ break;11692002 }11702003 Err(_) => {1171- warn!(sequence, "opening media v2 object stream timed out");1172- if !scheduler.fail(sequence).is_empty() {2004+ warn!(sequence, "opening media v3 object stream timed out");2005+ if let Some(cancel_sequences) = apply_media_v3_send_failure(2006+ &mut scheduler,2007+ &mut group_cursor,2008+ &mut replay_cursor,2009+ sequence,2010+ Some(replay_through_sequence),2011+ ) && !cancel_sequences.is_empty()2012+ {11732013 send_tasks.abort_all();11742014 }1175- continue;2015+ break;11762016 }11772017 };1178- let stream = ResetOnDropSendStream::new(stream);1179- if let Err(error) = stream1180- .stream()1181- .set_priority(media_v2_priority(keyframe))1182- {1183- warn!(sequence, %error, "setting media v2 object priority failed");1184- if !scheduler.fail(sequence).is_empty() {2018+ let stream = ResetOnDropSendStreamV3::new(stream);2019+ let transport_priority =2020+ media_v3_transport_priority(media_object.header.publisher_priority)?;2021+ if let Err(error) = stream.stream().set_priority(transport_priority) {2022+ warn!(sequence, %error, "setting media v3 object priority failed");2023+ if let Some(cancel_sequences) = apply_media_v3_send_failure(2024+ &mut scheduler,2025+ &mut group_cursor,2026+ &mut replay_cursor,2027+ sequence,2028+ Some(replay_through_sequence),2029+ ) && !cancel_sequences.is_empty()2030+ {11852031 send_tasks.abort_all();11862032 }1187- continue;2033+ break;11882034 }11892035 send_tasks.spawn(async move {11902036 (11912037 sequence,1192- send_media_v2_object(stream, media_frame).await,2038+ send_media_v3_object(stream, media_object).await,11932039 )11942040 });11952041 replay_cursor.commit_sent(&frame);@@ -1200,14 +2046,31 @@ async fn run_media_v2_session(
12002046 12012047 send_tasks.abort_all();12022048 while send_tasks.join_next().await.is_some() {}2049+ if !control_task_finished {2050+ control_task.abort();2051+ let _ = control_task.await;2052+ }12032053 result12042054 }12052055 1206-fn media_frame_for_encoded(2056+fn media_v3_delivery_timeout_ms(framerate: u32) -> u32 {2057+ debug_assert!(framerate > 0);2058+ let timeout_ms = 1_000_u642059+ .saturating_mul(MEDIA_V3_DELIVERY_FRAME_PERIODS)2060+ .div_ceil(u64::from(framerate.max(1)));2061+ u32::try_from(timeout_ms).unwrap_or(u32::MAX).clamp(2062+ MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS,2063+ MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS,2064+ )2065+}2066+2067+fn media_v3_object_for_encoded(12072068 config: &HostConfig,12082069 frame: &EncodedFrame,2070+ position: MediaV3ObjectPosition,12092071 discontinuity: bool,1210-) -> Result<MediaFrame> {2072+ delivery_timeout_ms: u32,2073+) -> Result<MediaObjectV3> {12112074 let mut flags = FrameFlags::NONE;12122075 if frame.keyframe {12132076 flags = flags.union(FrameFlags::KEYFRAME);@@ -1218,49 +2081,65 @@ fn media_frame_for_encoded(
12182081 if discontinuity {12192082 flags = flags.union(FrameFlags::DISCONTINUITY);12202083 }1221- let width = u16::try_from(config.width).context("width exceeds protocol")?;1222- let height = u16::try_from(config.height).context("height exceeds protocol")?;1223- let header = MediaFrameHeader::h264(1224- width,1225- height,2084+ let publisher_priority = if position.object_id == 0 {2085+ MEDIA_V3_KEYFRAME_PUBLISHER_PRIORITY2086+ } else {2087+ MEDIA_V3_DELTA_PUBLISHER_PRIORITY2088+ };2089+ let header = MediaObjectHeaderV3::h264(2090+ u16::try_from(config.width).context("width exceeds protocol")?,2091+ u16::try_from(config.height).context("height exceeds protocol")?,12262092 frame.data.len(),2093+ publisher_priority,2094+ flags,2095+ position.object_id,2096+ position.group_id,12272097 frame.sequence,12282098 frame.capture_timestamp_micros,12292099 frame.presentation_timestamp_micros,1230- flags,2100+ delivery_timeout_ms,12312101 )?;1232- MediaFrame::new(header, frame.data.as_ref().to_vec()).map_err(Into::into)2102+ MediaObjectV3::new(header, frame.data.as_ref().to_vec()).map_err(Into::into)12332103 }12342104 1235-async fn send_media_v2_object(mut stream: ResetOnDropSendStream, frame: MediaFrame) -> Result<()> {1236- tokio::time::timeout(1237- MEDIA_WRITE_TIMEOUT,1238- write_media_frame(stream.stream_mut(), &frame),1239- )1240- .await1241- .context("timed out writing media v2 object")??;1242- stream1243- .stream_mut()1244- .finish()1245- .context("finishing media v2 object stream")?;1246- match tokio::time::timeout(MEDIA_V2_PEER_ACK_TIMEOUT, stream.stream().stopped())1247- .await1248- .context("timed out waiting for media v2 object acknowledgement")?1249- .context("waiting for media v2 object acknowledgement")?1250- {1251- None => {1252- stream.disarm();1253- Ok(())2105+async fn send_media_v3_object(2106+ mut stream: ResetOnDropSendStreamV3,2107+ object: MediaObjectV3,2108+) -> Result<()> {2109+ let delivery_timeout = Duration::from_millis(u64::from(object.header.delivery_timeout_ms));2110+ tokio::time::timeout(delivery_timeout, async {2111+ write_media_object_v3(stream.stream_mut(), &object)2112+ .await2113+ .context("writing media v3 object")?;2114+ stream2115+ .stream_mut()2116+ .finish()2117+ .context("finishing media v3 object stream")?;2118+ match stream2119+ .stream()2120+ .stopped()2121+ .await2122+ .context("waiting for media v3 object acknowledgement")?2123+ {2124+ None => {2125+ stream.disarm();2126+ Ok(())2127+ }2128+ Some(code) => bail!("peer stopped media v3 object stream with code {code}"),12542129 }1255- Some(code) => bail!("peer stopped media v2 object stream with code {code}"),1256- }2130+ })2131+ .await2132+ .context("media v3 object exceeded its delivery timeout")?12572133 }12582134 1259-fn media_v2_priority(keyframe: bool) -> i32 {1260- if keyframe {1261- MEDIA_V2_KEYFRAME_PRIORITY1262- } else {1263- MEDIA_V2_DELTA_PRIORITY2135+fn media_v3_transport_priority(publisher_priority: u8) -> Result<i32> {2136+ // V3 follows MoQ's lower-is-higher publisher priority, while Iroh/QUIC2137+ // uses larger integers for more important streams. Keep the inversion2138+ // explicit rather than leaking either convention across the boundary.2139+ match publisher_priority {2140+ MEDIA_V3_KEYFRAME_PUBLISHER_PRIORITY => Ok(MEDIA_V3_KEYFRAME_TRANSPORT_PRIORITY),2141+ MEDIA_V3_DELTA_PUBLISHER_PRIORITY => Ok(MEDIA_V3_DELTA_TRANSPORT_PRIORITY),2142+ _ => bail!("unsupported media v3 publisher priority {publisher_priority}"),12642143 }12652144 }12662145 @@ -1815,6 +2694,430 @@ mod tests {
18152694 assert!(media_v2_priority(true) > media_v2_priority(false));18162695 }18172696 2697+ fn media_v3_encoded_frame(2698+ sequence: u64,2699+ keyframe: bool,2700+ codec_config: bool,2701+ payload_len: usize,2702+ ) -> EncodedFrame {2703+ EncodedFrame {2704+ sequence,2705+ capture_timestamp_micros: sequence,2706+ presentation_timestamp_micros: sequence as i64,2707+ observed_at: Instant::now(),2708+ keyframe,2709+ codec_config,2710+ data: Arc::from(vec![sequence as u8; payload_len]),2711+ }2712+ }2713+2714+ 2715+ fn media_v3_groups_begin_at_configured_idr_and_assign_contiguous_objects() {2716+ let mut cursor = MediaV3GroupCursor::default();2717+ assert_eq!(2718+ cursor.classify(&media_v3_encoded_frame(10, true, true, 4)),2719+ MediaV3GroupDecision::Send(MediaV3ObjectPosition {2720+ group_id: 10,2721+ object_id: 0,2722+ discontinuity: false,2723+ })2724+ );2725+ assert_eq!(2726+ cursor.classify(&media_v3_encoded_frame(11, false, false, 3)),2727+ MediaV3GroupDecision::Send(MediaV3ObjectPosition {2728+ group_id: 10,2729+ object_id: 1,2730+ discontinuity: false,2731+ })2732+ );2733+2734+ cursor.request_keyframe();2735+ assert_eq!(2736+ cursor.classify(&media_v3_encoded_frame(12, false, false, 3)),2737+ MediaV3GroupDecision::SkipUntilKeyframe2738+ );2739+ assert_eq!(2740+ cursor.classify(&media_v3_encoded_frame(20, true, true, 4)),2741+ MediaV3GroupDecision::Send(MediaV3ObjectPosition {2742+ group_id: 20,2743+ object_id: 0,2744+ discontinuity: true,2745+ })2746+ );2747+ }2748+2749+ 2750+ fn media_v3_group_rejects_noncontiguous_or_unconfigured_frames() {2751+ let mut cursor = MediaV3GroupCursor::default();2752+ assert_eq!(2753+ cursor.classify(&media_v3_encoded_frame(10, true, false, 1)),2754+ MediaV3GroupDecision::SkipUntilKeyframe2755+ );2756+ assert!(matches!(2757+ cursor.classify(&media_v3_encoded_frame(20, true, true, 1)),2758+ MediaV3GroupDecision::Send(_)2759+ ));2760+ assert_eq!(2761+ cursor.classify(&media_v3_encoded_frame(22, false, false, 1)),2762+ MediaV3GroupDecision::EnterResync2763+ );2764+ assert!(cursor.waiting_for_keyframe);2765+ }2766+2767+ 2768+ fn media_v3_group_accepts_exact_limits_and_rejects_overflow() {2769+ let mut object_cursor = MediaV3GroupCursor::default();2770+ assert!(matches!(2771+ object_cursor.classify(&media_v3_encoded_frame(0, true, true, 1)),2772+ MediaV3GroupDecision::Send(_)2773+ ));2774+ for sequence in 1..=u64::from(MAX_MEDIA_OBJECT_ID_V3) {2775+ assert!(matches!(2776+ object_cursor.classify(&media_v3_encoded_frame(sequence, false, false, 1)),2777+ MediaV3GroupDecision::Send(MediaV3ObjectPosition { object_id, .. })2778+ if u64::from(object_id) == sequence2779+ ));2780+ }2781+ assert_eq!(2782+ object_cursor.classify(&media_v3_encoded_frame(2783+ u64::from(MAX_MEDIA_OBJECT_ID_V3) + 1,2784+ false,2785+ false,2786+ 1,2787+ )),2788+ MediaV3GroupDecision::EnterResync2789+ );2790+2791+ let mut byte_cursor = MediaV3GroupCursor::default();2792+ assert!(matches!(2793+ byte_cursor.classify(&media_v3_encoded_frame(10, true, true, 1)),2794+ MediaV3GroupDecision::Send(_)2795+ ));2796+ byte_cursor.payload_bytes = MAX_MEDIA_GROUP_BYTES_V3 - 1;2797+ assert!(matches!(2798+ byte_cursor.classify(&media_v3_encoded_frame(11, false, false, 1)),2799+ MediaV3GroupDecision::Send(_)2800+ ));2801+ assert_eq!(byte_cursor.payload_bytes, MAX_MEDIA_GROUP_BYTES_V3);2802+ assert_eq!(2803+ byte_cursor.classify(&media_v3_encoded_frame(12, false, false, 1)),2804+ MediaV3GroupDecision::EnterResync2805+ );2806+ }2807+2808+ 2809+ fn media_v3_keyframe_requests_cancel_once_and_recover_discontinuously() {2810+ let mut scheduler = MediaV3Scheduler::default();2811+ assert!(matches!(2812+ scheduler.schedule(10, true),2813+ MediaV3ScheduleDecision::Send { .. }2814+ ));2815+ assert!(matches!(2816+ scheduler.schedule(11, false),2817+ MediaV3ScheduleDecision::Send { .. }2818+ ));2819+ assert_eq!(scheduler.request_keyframe(), vec![10, 11]);2820+ assert!(scheduler.request_keyframe().is_empty());2821+ assert_eq!(2822+ scheduler.schedule(12, false),2823+ MediaV3ScheduleDecision::SkipUntilKeyframe2824+ );2825+ assert_eq!(2826+ scheduler.schedule(20, true),2827+ MediaV3ScheduleDecision::Send {2828+ discontinuity: true,2829+ cancel_sequences: vec![],2830+ }2831+ );2832+ }2833+2834+ 2835+ fn media_v3_join_request_preserves_initial_current_gop_replay() {2836+ let mut scheduler = MediaV3Scheduler::default();2837+ let mut group_cursor = MediaV3GroupCursor::default();2838+ let mut replay_cursor = MediaReplayCursor::default();2839+2840+ let (transitioned, cancel_sequences) = apply_media_v3_keyframe_request(2841+ &mut scheduler,2842+ &mut group_cursor,2843+ &mut replay_cursor,2844+ Some(42),2845+ KeyframeRequestReasonV3::Join,2846+ );2847+2848+ assert!(!transitioned);2849+ assert!(cancel_sequences.is_empty());2850+ assert_eq!(replay_cursor.last_sequence, None);2851+ assert!(replay_cursor.waiting_for_keyframe);2852+ assert!(!replay_cursor.discontinuity_pending);2853+ assert!(group_cursor.waiting_for_keyframe);2854+ assert!(!group_cursor.discontinuity_pending);2855+ assert!(!scheduler.discontinuity_pending);2856+2857+ assert!(matches!(2858+ group_cursor.classify(&media_v3_encoded_frame(10, true, true, 1)),2859+ MediaV3GroupDecision::Send(MediaV3ObjectPosition {2860+ discontinuity: false,2861+ ..2862+ })2863+ ));2864+ assert!(matches!(2865+ scheduler.schedule(10, true),2866+ MediaV3ScheduleDecision::Send {2867+ discontinuity: false,2868+ ..2869+ }2870+ ));2871+ }2872+2873+ 2874+ fn media_v3_late_join_preserves_active_static_current_gop() {2875+ let mut scheduler = MediaV3Scheduler::default();2876+ let mut group_cursor = MediaV3GroupCursor::default();2877+ let mut replay_cursor = MediaReplayCursor::default();2878+ let keyframe = media_v3_encoded_frame(10, true, true, 1);2879+ let delta = media_v3_encoded_frame(11, false, false, 1);2880+2881+ assert!(matches!(2882+ group_cursor.classify(&keyframe),2883+ MediaV3GroupDecision::Send(_)2884+ ));2885+ assert!(matches!(2886+ scheduler.schedule(keyframe.sequence, true),2887+ MediaV3ScheduleDecision::Send { .. }2888+ ));2889+ replay_cursor.commit_sent(&keyframe);2890+ assert!(matches!(2891+ group_cursor.classify(&delta),2892+ MediaV3GroupDecision::Send(_)2893+ ));2894+ assert!(matches!(2895+ scheduler.schedule(delta.sequence, false),2896+ MediaV3ScheduleDecision::Send { .. }2897+ ));2898+ replay_cursor.commit_sent(&delta);2899+2900+ let (transitioned, cancel_sequences) = apply_media_v3_keyframe_request(2901+ &mut scheduler,2902+ &mut group_cursor,2903+ &mut replay_cursor,2904+ Some(delta.sequence),2905+ KeyframeRequestReasonV3::Join,2906+ );2907+2908+ assert!(!transitioned);2909+ assert!(cancel_sequences.is_empty());2910+ assert_eq!(scheduler.in_flight, vec![10, 11]);2911+ assert!(!scheduler.waiting_for_keyframe);2912+ assert!(!scheduler.discontinuity_pending);2913+ assert_eq!(group_cursor.group_id, Some(10));2914+ assert_eq!(group_cursor.last_sequence, Some(11));2915+ assert!(!group_cursor.waiting_for_keyframe);2916+ assert_eq!(replay_cursor.last_sequence, Some(11));2917+ assert!(!replay_cursor.waiting_for_keyframe);2918+ }2919+2920+ 2921+ fn media_v3_stale_old_group_failure_preserves_replacement_group() {2922+ let mut scheduler = MediaV3Scheduler::default();2923+ let mut group_cursor = MediaV3GroupCursor::default();2924+ let mut replay_cursor = MediaReplayCursor::default();2925+ let old_keyframe = media_v3_encoded_frame(60, true, true, 1);2926+ let old_delta = media_v3_encoded_frame(61, false, false, 1);2927+ let replacement = media_v3_encoded_frame(70, true, true, 1);2928+2929+ for frame in [&old_keyframe, &old_delta] {2930+ assert!(matches!(2931+ group_cursor.classify(frame),2932+ MediaV3GroupDecision::Send(_)2933+ ));2934+ assert!(matches!(2935+ scheduler.schedule(frame.sequence, frame.keyframe && frame.codec_config),2936+ MediaV3ScheduleDecision::Send { .. }2937+ ));2938+ replay_cursor.commit_sent(frame);2939+ }2940+ assert!(matches!(2941+ group_cursor.classify(&replacement),2942+ MediaV3GroupDecision::Send(_)2943+ ));2944+ assert_eq!(2945+ scheduler.schedule(replacement.sequence, true),2946+ MediaV3ScheduleDecision::Send {2947+ discontinuity: true,2948+ cancel_sequences: vec![60, 61],2949+ }2950+ );2951+ replay_cursor.commit_sent(&replacement);2952+2953+ assert_eq!(2954+ apply_media_v3_send_failure(2955+ &mut scheduler,2956+ &mut group_cursor,2957+ &mut replay_cursor,2958+ old_keyframe.sequence,2959+ Some(replacement.sequence),2960+ ),2961+ None2962+ );2963+ assert_eq!(scheduler.in_flight, vec![70]);2964+ assert!(!scheduler.waiting_for_keyframe);2965+ assert_eq!(group_cursor.group_id, Some(70));2966+ assert!(!group_cursor.waiting_for_keyframe);2967+ assert_eq!(replay_cursor.last_sequence, Some(70));2968+ assert!(!replay_cursor.waiting_for_keyframe);2969+2970+ let next_delta = media_v3_encoded_frame(71, false, false, 1);2971+ assert!(matches!(2972+ group_cursor.classify(&next_delta),2973+ MediaV3GroupDecision::Send(MediaV3ObjectPosition {2974+ group_id: 70,2975+ object_id: 1,2976+ ..2977+ })2978+ ));2979+ assert!(matches!(2980+ scheduler.schedule(next_delta.sequence, false),2981+ MediaV3ScheduleDecision::Send {2982+ discontinuity: false,2983+ ..2984+ }2985+ ));2986+ }2987+2988+ 2989+ fn media_v3_control_gate_is_monotonic_and_accepts_at_most_ten_per_second() {2990+ let started = Instant::now();2991+ let request = |request_id| {2992+ MediaControlRequestV3::request_keyframe(2993+ request_id,2994+ None,2995+ sigil_protocol::KeyframeRequestReasonV3::TransportGap,2996+ )2997+ };2998+ let mut gate = MediaV3ControlGate::default();2999+ assert_eq!(3000+ gate.accept(request(1), started).unwrap(),3001+ MediaV3ControlDecision::Accept3002+ );3003+ assert_eq!(3004+ gate.accept(3005+ request(2),3006+ started + MEDIA_V3_CONTROL_REQUEST_INTERVAL - Duration::from_nanos(1),3007+ )3008+ .unwrap(),3009+ MediaV3ControlDecision::Pace(Duration::from_nanos(1))3010+ );3011+ assert_eq!(3012+ gate.accept(request(3), started + MEDIA_V3_CONTROL_REQUEST_INTERVAL)3013+ .unwrap(),3014+ MediaV3ControlDecision::Accept3015+ );3016+ assert!(3017+ gate.accept(request(3), started + Duration::from_secs(1))3018+ .is_err()3019+ );3020+ }3021+3022+ 3023+ async fn media_v3_rejected_control_requests_apply_read_side_pacing() {3024+ use tokio::io::AsyncWriteExt as _;3025+3026+ let requests = [1, 2, 3].map(|request_id| {3027+ MediaControlRequestV3::request_keyframe(3028+ request_id,3029+ None,3030+ KeyframeRequestReasonV3::TransportGap,3031+ )3032+ });3033+ let (mut writer, reader) = tokio::io::duplex(128);3034+ for request in &requests {3035+ sigil_protocol::write_media_control_request_v3(&mut writer, request)3036+ .await3037+ .unwrap();3038+ }3039+ writer.shutdown().await.unwrap();3040+ let (sender, mut receiver) = tokio::sync::watch::channel(None);3041+3042+ let started = Instant::now();3043+ forward_media_v3_control_requests(reader, sender)3044+ .await3045+ .unwrap();3046+3047+ assert!(started.elapsed() >= MEDIA_V3_CONTROL_REQUEST_INTERVAL);3048+ receiver.changed().await.unwrap();3049+ assert_eq!(*receiver.borrow_and_update(), Some(requests[2]));3050+ }3051+3052+ 3053+ async fn media_v3_control_eof_is_clean_and_malformed_input_is_terminal() {3054+ use tokio::io::AsyncWriteExt as _;3055+3056+ let (writer, reader) = tokio::io::duplex(64);3057+ let (sender, mut receiver) = tokio::sync::watch::channel(None);3058+ drop(writer);3059+ forward_media_v3_control_requests(reader, sender)3060+ .await3061+ .unwrap();3062+ assert!(receiver.changed().await.is_err());3063+3064+ let final_request = MediaControlRequestV3::request_keyframe(3065+ 1,3066+ Some(9),3067+ sigil_protocol::KeyframeRequestReasonV3::DecoderReset,3068+ );3069+ let (mut writer, reader) = tokio::io::duplex(64);3070+ let (sender, mut receiver) = tokio::sync::watch::channel(None);3071+ sigil_protocol::write_media_control_request_v3(&mut writer, &final_request)3072+ .await3073+ .unwrap();3074+ writer.shutdown().await.unwrap();3075+ forward_media_v3_control_requests(reader, sender)3076+ .await3077+ .unwrap();3078+ receiver.changed().await.unwrap();3079+ assert_eq!(*receiver.borrow_and_update(), Some(final_request));3080+3081+ let (mut writer, reader) = tokio::io::duplex(64);3082+ let (sender, _receiver) = tokio::sync::watch::channel(None);3083+ writer3084+ .write_all(&[0; sigil_protocol::MEDIA_CONTROL_REQUEST_V3_LEN])3085+ .await3086+ .unwrap();3087+ writer.shutdown().await.unwrap();3088+ assert!(3089+ forward_media_v3_control_requests(reader, sender)3090+ .await3091+ .is_err()3092+ );3093+ }3094+3095+ 3096+ fn media_v3_deadline_ceil_clamps_four_frame_periods() {3097+ assert_eq!(media_v3_delivery_timeout_ms(60), 67);3098+ assert_eq!(media_v3_delivery_timeout_ms(240), 17);3099+ assert_eq!(3100+ media_v3_delivery_timeout_ms(1_000),3101+ MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS3102+ );3103+ assert_eq!(3104+ media_v3_delivery_timeout_ms(1),3105+ MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS3106+ );3107+ }3108+3109+ 3110+ fn media_v3_publisher_priority_maps_to_inverse_transport_priority() {3111+ let keyframe_transport =3112+ media_v3_transport_priority(MEDIA_V3_KEYFRAME_PUBLISHER_PRIORITY).unwrap();3113+ let delta_transport =3114+ media_v3_transport_priority(MEDIA_V3_DELTA_PUBLISHER_PRIORITY).unwrap();3115+ assert_eq!(keyframe_transport, MEDIA_V3_KEYFRAME_TRANSPORT_PRIORITY);3116+ assert_eq!(delta_transport, MEDIA_V3_DELTA_TRANSPORT_PRIORITY);3117+ assert!(keyframe_transport > delta_transport);3118+ assert!(media_v3_transport_priority(1).is_err());3119+ }3120+18183121 fn endpoint(byte: u8) -> EndpointId {18193122 iroh::SecretKey::from_bytes(&[byte; 32]).public()18203123 }@@ -12,6 +12,7 @@ mod handshake;
1212 mod input;1313 mod invitation;1414 mod media;15+mod media_v3;1516 1617 pub use audio::{AUDIO_HEADER_LEN, AudioCodec, AudioFlags, AudioPacket, AudioPacketHeader};1718 pub use error::{ProtocolError, Result};@@ -34,6 +35,13 @@ pub use media::{
3435 FrameFlags, MEDIA_HEADER_LEN, MediaCodec, MediaFrame, MediaFrameHeader, read_media_frame,3536 read_media_object, write_media_frame,3637 };38+pub use media_v3::{39+ KeyframeRequestReasonV3, MAX_MEDIA_GROUP_BYTES_V3, MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS,40+ MAX_MEDIA_OBJECT_ID_V3, MEDIA_CONTROL_REQUEST_V3_LEN, MEDIA_OBJECT_V3_HEADER_LEN,41+ MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS, MediaControlRequestTypeV3, MediaControlRequestV3,42+ MediaObjectHeaderV3, MediaObjectV3, read_media_control_request_v3, read_media_object_v3,43+ write_media_control_request_v3, write_media_object_v3,44+};3745 3846 /// Protocol version encoded in v1 messages.3947 pub const PROTOCOL_VERSION: u16 = 1;@@ -42,6 +50,10 @@ pub const PROTOCOL_VERSION: u16 = 1;
4250 pub const MEDIA_ALPN_V1: &[u8] = b"sigil/media/1";4351 /// ALPN for one encoded video object per host-opened unidirectional stream.4452 pub const MEDIA_ALPN_V2: &[u8] = b"sigil/media/2";53+/// ALPN for custom MoQ-style grouped media objects and keyframe control.54+///55+/// This is a Sigil protocol and is not IETF MoQ Transport compatible.56+pub const MEDIA_ALPN_V3: &[u8] = b"sigil/media/3";4557 /// ALPN for the v1 latency-independent input stream.4658 pub const INPUT_ALPN_V1: &[u8] = b"sigil/input/1";4759 /// ALPN for the v1 session-control stream.@@ -81,6 +93,7 @@ mod tests {
8193 fn alpn_values_are_deployment_golden_vectors() {8294 assert_eq!(MEDIA_ALPN_V1, b"sigil/media/1");8395 assert_eq!(MEDIA_ALPN_V2, b"sigil/media/2");96+ assert_eq!(MEDIA_ALPN_V3, b"sigil/media/3");8497 assert_eq!(INPUT_ALPN_V1, b"sigil/input/1");8598 assert_eq!(CONTROL_ALPN_V1, b"sigil/control/1");8699 assert_eq!(AUDIO_ALPN_V1, b"sigil/audio/1");@@ -0,0 +1,783 @@
1+//! Custom MoQ-style media objects used by `sigil/media/3`.2+//!3+//! This wire format borrows MoQ's group, object, publisher-priority, and4+//! delivery-timeout semantics. It is intentionally not an implementation of5+//! the IETF MoQ Transport protocol.6+7+use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};8+9+use crate::{10+ FrameFlags, MAX_MEDIA_PAYLOAD_LEN, MAX_VIDEO_DIMENSION, MAX_VIDEO_PIXELS, MediaCodec,11+ ProtocolError, Result,12+};13+14+const MEDIA_OBJECT_V3_MAGIC: [u8; 4] = *b"SGO3";15+const MEDIA_CONTROL_V3_MAGIC: [u8; 4] = *b"SGC3";16+const MEDIA_V3_VERSION: u16 = 3;17+18+/// Size of a v3 media-object header in bytes.19+pub const MEDIA_OBJECT_V3_HEADER_LEN: usize = 64;20+const MEDIA_OBJECT_V3_HEADER_LEN_U16: u16 = MEDIA_OBJECT_V3_HEADER_LEN as u16;21+/// Size of a v3 media-control request in bytes.22+pub const MEDIA_CONTROL_REQUEST_V3_LEN: usize = 28;23+const MEDIA_CONTROL_REQUEST_V3_LEN_U16: u16 = MEDIA_CONTROL_REQUEST_V3_LEN as u16;24+25+/// Lowest delivery timeout a v3 peer may place on an object.26+pub const MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS: u32 = 16;27+/// Highest delivery timeout a v3 peer may place on an object.28+pub const MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS: u32 = 1_000;29+/// Highest object identifier in a v3 group.30+pub const MAX_MEDIA_OBJECT_ID_V3: u32 = 255;31+/// Maximum aggregate encoded payload retained or accepted for one v3 group.32+pub const MAX_MEDIA_GROUP_BYTES_V3: usize = 32 * 1024 * 1024;33+34+/// Fixed-width header for one independently streamed v3 encoded access unit.35+///36+/// Integer fields use network byte order. A lower `publisher_priority` value37+/// denotes a higher priority. `group_id` identifies one encoded GOP and38+/// `object_id` identifies the access unit within that GOP.39+40+pub struct MediaObjectHeaderV3 {41+ pub codec: MediaCodec,42+ pub publisher_priority: u8,43+ pub flags: FrameFlags,44+ pub width: u16,45+ pub height: u16,46+ pub payload_len: u32,47+ pub object_id: u32,48+ pub group_id: u64,49+ pub sequence: u64,50+ pub capture_timestamp_us: u64,51+ pub pts_us: i64,52+ pub delivery_timeout_ms: u32,53+}54+55+impl MediaObjectHeaderV3 {56+ 57+ pub fn h264(58+ width: u16,59+ height: u16,60+ payload_len: usize,61+ publisher_priority: u8,62+ flags: FrameFlags,63+ object_id: u32,64+ group_id: u64,65+ sequence: u64,66+ capture_timestamp_us: u64,67+ pts_us: i64,68+ delivery_timeout_ms: u32,69+ ) -> Result<Self> {70+ let payload_len =71+ u32::try_from(payload_len).map_err(|_| ProtocolError::InvalidMediaPayloadLength {72+ actual: payload_len,73+ maximum: MAX_MEDIA_PAYLOAD_LEN,74+ })?;75+ let header = Self {76+ codec: MediaCodec::H264,77+ publisher_priority,78+ flags,79+ width,80+ height,81+ payload_len,82+ object_id,83+ group_id,84+ sequence,85+ capture_timestamp_us,86+ pts_us,87+ delivery_timeout_ms,88+ };89+ header.validate()?;90+ Ok(header)91+ }92+93+ pub fn validate(&self) -> Result<()> {94+ let pixels = u32::from(self.width) * u32::from(self.height);95+ if self.width == 096+ || self.height == 097+ || self.width > MAX_VIDEO_DIMENSION98+ || self.height > MAX_VIDEO_DIMENSION99+ || pixels > MAX_VIDEO_PIXELS100+ {101+ return Err(ProtocolError::InvalidDimensions {102+ width: self.width,103+ height: self.height,104+ });105+ }106+107+ let payload_len = self.payload_len as usize;108+ if payload_len == 0 || payload_len > MAX_MEDIA_PAYLOAD_LEN {109+ return Err(ProtocolError::InvalidMediaPayloadLength {110+ actual: payload_len,111+ maximum: MAX_MEDIA_PAYLOAD_LEN,112+ });113+ }114+ if FrameFlags::from_bits(self.flags.bits()).is_none() {115+ return Err(ProtocolError::UnsupportedFlags(self.flags.bits()));116+ }117+ if self.object_id > MAX_MEDIA_OBJECT_ID_V3 {118+ return Err(ProtocolError::InvalidMessage {119+ message_type: "v3 media object header",120+ reason: "object id must be 0..=255",121+ });122+ }123+ if !(MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS..=MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS)124+ .contains(&self.delivery_timeout_ms)125+ {126+ return Err(ProtocolError::InvalidMessage {127+ message_type: "v3 media object header",128+ reason: "delivery timeout must be 16..=1000 milliseconds",129+ });130+ }131+132+ let keyframe = self.flags.contains(FrameFlags::KEYFRAME);133+ let codec_config = self.flags.contains(FrameFlags::CODEC_CONFIG);134+ if self.object_id == 0 && !(keyframe && codec_config) {135+ return Err(ProtocolError::InvalidMessage {136+ message_type: "v3 media object header",137+ reason: "object zero must be a keyframe carrying codec configuration",138+ });139+ }140+ if self.object_id != 0141+ && (keyframe || codec_config || self.flags.contains(FrameFlags::DISCONTINUITY))142+ {143+ return Err(ProtocolError::InvalidMessage {144+ message_type: "v3 media object header",145+ reason: "nonzero objects must not carry recovery-boundary flags",146+ });147+ }148+ Ok(())149+ }150+151+ pub fn encode(&self) -> Result<[u8; MEDIA_OBJECT_V3_HEADER_LEN]> {152+ self.validate()?;153+ let mut wire = [0_u8; MEDIA_OBJECT_V3_HEADER_LEN];154+ wire[0..4].copy_from_slice(&MEDIA_OBJECT_V3_MAGIC);155+ wire[4..6].copy_from_slice(&MEDIA_V3_VERSION.to_be_bytes());156+ wire[6..8].copy_from_slice(&MEDIA_OBJECT_V3_HEADER_LEN_U16.to_be_bytes());157+ wire[8] = self.codec as u8;158+ wire[9] = self.publisher_priority;159+ wire[10] = self.flags.bits();160+ // Byte 11 is reserved and must remain zero.161+ wire[12..14].copy_from_slice(&self.width.to_be_bytes());162+ wire[14..16].copy_from_slice(&self.height.to_be_bytes());163+ wire[16..20].copy_from_slice(&self.payload_len.to_be_bytes());164+ wire[20..24].copy_from_slice(&self.object_id.to_be_bytes());165+ wire[24..32].copy_from_slice(&self.group_id.to_be_bytes());166+ wire[32..40].copy_from_slice(&self.sequence.to_be_bytes());167+ wire[40..48].copy_from_slice(&self.capture_timestamp_us.to_be_bytes());168+ wire[48..56].copy_from_slice(&self.pts_us.to_be_bytes());169+ wire[56..60].copy_from_slice(&self.delivery_timeout_ms.to_be_bytes());170+ // Bytes 60..64 are reserved and must remain zero.171+ Ok(wire)172+ }173+174+ pub fn decode(wire: &[u8; MEDIA_OBJECT_V3_HEADER_LEN]) -> Result<Self> {175+ let magic: [u8; 4] = wire[0..4].try_into().expect("fixed slice length");176+ if magic != MEDIA_OBJECT_V3_MAGIC {177+ return Err(ProtocolError::InvalidMediaMagic(magic));178+ }179+ let version = u16::from_be_bytes(wire[4..6].try_into().expect("fixed slice length"));180+ if version != MEDIA_V3_VERSION {181+ return Err(ProtocolError::UnsupportedVersion {182+ expected: MEDIA_V3_VERSION,183+ actual: version,184+ });185+ }186+ let header_len = u16::from_be_bytes(wire[6..8].try_into().expect("fixed slice length"));187+ if header_len != MEDIA_OBJECT_V3_HEADER_LEN_U16 {188+ return Err(ProtocolError::InvalidMessage {189+ message_type: "v3 media object header",190+ reason: "header length must be 64 bytes",191+ });192+ }193+ if wire[11] != 0 || wire[60..64].iter().any(|byte| *byte != 0) {194+ return Err(ProtocolError::InvalidMessage {195+ message_type: "v3 media object header",196+ reason: "reserved fields must be zero",197+ });198+ }199+ let flags =200+ FrameFlags::from_bits(wire[10]).ok_or(ProtocolError::UnsupportedFlags(wire[10]))?;201+ let header = Self {202+ codec: MediaCodec::try_from(wire[8])?,203+ publisher_priority: wire[9],204+ flags,205+ width: u16::from_be_bytes(wire[12..14].try_into().expect("fixed slice length")),206+ height: u16::from_be_bytes(wire[14..16].try_into().expect("fixed slice length")),207+ payload_len: u32::from_be_bytes(wire[16..20].try_into().expect("fixed slice length")),208+ object_id: u32::from_be_bytes(wire[20..24].try_into().expect("fixed slice length")),209+ group_id: u64::from_be_bytes(wire[24..32].try_into().expect("fixed slice length")),210+ sequence: u64::from_be_bytes(wire[32..40].try_into().expect("fixed slice length")),211+ capture_timestamp_us: u64::from_be_bytes(212+ wire[40..48].try_into().expect("fixed slice length"),213+ ),214+ pts_us: i64::from_be_bytes(wire[48..56].try_into().expect("fixed slice length")),215+ delivery_timeout_ms: u32::from_be_bytes(216+ wire[56..60].try_into().expect("fixed slice length"),217+ ),218+ };219+ header.validate()?;220+ Ok(header)221+ }222+}223+224+/// One complete v3 media object.225+226+pub struct MediaObjectV3 {227+ pub header: MediaObjectHeaderV3,228+ pub payload: Vec<u8>,229+}230+231+impl MediaObjectV3 {232+ pub fn new(header: MediaObjectHeaderV3, payload: Vec<u8>) -> Result<Self> {233+ header.validate()?;234+ if payload.len() != header.payload_len as usize {235+ return Err(ProtocolError::InvalidMessage {236+ message_type: "v3 media object",237+ reason: "payload does not match declared length",238+ });239+ }240+ Ok(Self { header, payload })241+ }242+}243+244+/// Read exactly one complete v3 media object followed by a clean stream EOF.245+pub async fn read_media_object_v3<R>(reader: &mut R) -> Result<MediaObjectV3>246+where247+ R: AsyncRead + Unpin,248+{249+ let mut wire = [0_u8; MEDIA_OBJECT_V3_HEADER_LEN];250+ reader.read_exact(&mut wire).await?;251+ let header = MediaObjectHeaderV3::decode(&wire)?;252+253+ // Header validation, including the peer-controlled payload bound, occurs254+ // before this allocation.255+ let mut payload = vec![0_u8; header.payload_len as usize];256+ reader.read_exact(&mut payload).await?;257+ let mut trailing = [0_u8; 1];258+ if reader.read(&mut trailing).await? != 0 {259+ return Err(ProtocolError::InvalidMessage {260+ message_type: "v3 media object",261+ reason: "trailing bytes after the media object",262+ });263+ }264+ Ok(MediaObjectV3 { header, payload })265+}266+267+/// Validate and write one complete v3 media object.268+pub async fn write_media_object_v3<W>(writer: &mut W, object: &MediaObjectV3) -> Result<()>269+where270+ W: AsyncWrite + Unpin,271+{272+ object.header.validate()?;273+ if object.payload.len() != object.header.payload_len as usize {274+ return Err(ProtocolError::InvalidMessage {275+ message_type: "v3 media object",276+ reason: "payload does not match declared length",277+ });278+ }279+ writer.write_all(&object.header.encode()?).await?;280+ writer.write_all(&object.payload).await?;281+ writer.flush().await?;282+ Ok(())283+}284+285+/// V3 media-control request type.286+287+288+pub enum MediaControlRequestTypeV3 {289+ RequestKeyframe = 1,290+}291+292+impl TryFrom<u8> for MediaControlRequestTypeV3 {293+ type Error = ProtocolError;294+295+ fn try_from(value: u8) -> Result<Self> {296+ match value {297+ 1 => Ok(Self::RequestKeyframe),298+ _ => Err(ProtocolError::InvalidMessage {299+ message_type: "v3 media control request",300+ reason: "unknown request type",301+ }),302+ }303+ }304+}305+306+/// Why Portal is asking Sigil for a new independently decodable frame.307+308+309+pub enum KeyframeRequestReasonV3 {310+ Join = 1,311+ TransportGap = 2,312+ DeliveryTimeout = 3,313+ DecoderReset = 4,314+ FrontendBackpressure = 5,315+}316+317+impl TryFrom<u8> for KeyframeRequestReasonV3 {318+ type Error = ProtocolError;319+320+ fn try_from(value: u8) -> Result<Self> {321+ match value {322+ 1 => Ok(Self::Join),323+ 2 => Ok(Self::TransportGap),324+ 3 => Ok(Self::DeliveryTimeout),325+ 4 => Ok(Self::DecoderReset),326+ 5 => Ok(Self::FrontendBackpressure),327+ _ => Err(ProtocolError::InvalidMessage {328+ message_type: "v3 media control request",329+ reason: "unknown keyframe request reason",330+ }),331+ }332+ }333+}334+335+/// Fixed-width client-to-host keyframe request for the v3 control stream.336+337+pub struct MediaControlRequestV3 {338+ pub request_type: MediaControlRequestTypeV3,339+ pub reason: KeyframeRequestReasonV3,340+ pub request_id: u64,341+ /// Last globally sequenced object accepted by the client, or `None` when342+ /// no object has been accepted yet.343+ pub last_sequence: Option<u64>,344+}345+346+impl MediaControlRequestV3 {347+ pub const fn request_keyframe(348+ request_id: u64,349+ last_sequence: Option<u64>,350+ reason: KeyframeRequestReasonV3,351+ ) -> Self {352+ Self {353+ request_type: MediaControlRequestTypeV3::RequestKeyframe,354+ reason,355+ request_id,356+ last_sequence,357+ }358+ }359+360+ pub fn validate(&self) -> Result<()> {361+ if self.request_type != MediaControlRequestTypeV3::RequestKeyframe {362+ return Err(ProtocolError::InvalidMessage {363+ message_type: "v3 media control request",364+ reason: "unknown request type",365+ });366+ }367+ if self.last_sequence == Some(u64::MAX) {368+ return Err(ProtocolError::InvalidMessage {369+ message_type: "v3 media control request",370+ reason: "the all-ones last sequence is reserved for none",371+ });372+ }373+ Ok(())374+ }375+376+ pub fn encode(&self) -> Result<[u8; MEDIA_CONTROL_REQUEST_V3_LEN]> {377+ self.validate()?;378+ let mut wire = [0_u8; MEDIA_CONTROL_REQUEST_V3_LEN];379+ wire[0..4].copy_from_slice(&MEDIA_CONTROL_V3_MAGIC);380+ wire[4..6].copy_from_slice(&MEDIA_V3_VERSION.to_be_bytes());381+ wire[6..8].copy_from_slice(&MEDIA_CONTROL_REQUEST_V3_LEN_U16.to_be_bytes());382+ wire[8] = self.request_type as u8;383+ wire[9] = self.reason as u8;384+ // Bytes 10..12 are reserved and must remain zero.385+ wire[12..20].copy_from_slice(&self.request_id.to_be_bytes());386+ wire[20..28].copy_from_slice(&self.last_sequence.unwrap_or(u64::MAX).to_be_bytes());387+ Ok(wire)388+ }389+390+ pub fn decode(wire: &[u8; MEDIA_CONTROL_REQUEST_V3_LEN]) -> Result<Self> {391+ let magic: [u8; 4] = wire[0..4].try_into().expect("fixed slice length");392+ if magic != MEDIA_CONTROL_V3_MAGIC {393+ return Err(ProtocolError::InvalidMessage {394+ message_type: "v3 media control request",395+ reason: "invalid control magic",396+ });397+ }398+ let version = u16::from_be_bytes(wire[4..6].try_into().expect("fixed slice length"));399+ if version != MEDIA_V3_VERSION {400+ return Err(ProtocolError::UnsupportedVersion {401+ expected: MEDIA_V3_VERSION,402+ actual: version,403+ });404+ }405+ let request_len = u16::from_be_bytes(wire[6..8].try_into().expect("fixed slice length"));406+ if request_len != MEDIA_CONTROL_REQUEST_V3_LEN_U16 {407+ return Err(ProtocolError::InvalidMessage {408+ message_type: "v3 media control request",409+ reason: "request length must be 28 bytes",410+ });411+ }412+ if wire[10..12].iter().any(|byte| *byte != 0) {413+ return Err(ProtocolError::InvalidMessage {414+ message_type: "v3 media control request",415+ reason: "reserved fields must be zero",416+ });417+ }418+ let last_sequence =419+ u64::from_be_bytes(wire[20..28].try_into().expect("fixed slice length"));420+ let request = Self {421+ request_type: MediaControlRequestTypeV3::try_from(wire[8])?,422+ reason: KeyframeRequestReasonV3::try_from(wire[9])?,423+ request_id: u64::from_be_bytes(wire[12..20].try_into().expect("fixed slice length")),424+ last_sequence: (last_sequence != u64::MAX).then_some(last_sequence),425+ };426+ request.validate()?;427+ Ok(request)428+ }429+}430+431+/// Read one fixed-width v3 media-control request. Clean EOF returns `None`.432+pub async fn read_media_control_request_v3<R>(433+ reader: &mut R,434+) -> Result<Option<MediaControlRequestV3>>435+where436+ R: AsyncRead + Unpin,437+{438+ let mut wire = [0_u8; MEDIA_CONTROL_REQUEST_V3_LEN];439+ let read = reader.read(&mut wire[..1]).await?;440+ if read == 0 {441+ return Ok(None);442+ }443+ reader.read_exact(&mut wire[1..]).await?;444+ MediaControlRequestV3::decode(&wire).map(Some)445+}446+447+/// Validate and write one fixed-width v3 media-control request.448+pub async fn write_media_control_request_v3<W>(449+ writer: &mut W,450+ request: &MediaControlRequestV3,451+) -> Result<()>452+where453+ W: AsyncWrite + Unpin,454+{455+ writer.write_all(&request.encode()?).await?;456+ writer.flush().await?;457+ Ok(())458+}459+460+461+mod tests {462+ use std::io::ErrorKind;463+464+ use tokio::io::{AsyncWriteExt, duplex};465+466+ use super::*;467+468+ fn keyframe_header() -> MediaObjectHeaderV3 {469+ MediaObjectHeaderV3::h264(470+ 1280,471+ 800,472+ 4,473+ 32,474+ FrameFlags::KEYFRAME475+ .union(FrameFlags::CODEC_CONFIG)476+ .union(FrameFlags::DISCONTINUITY),477+ 0,478+ 0x0102_0304_0506_0708,479+ 0x1112_1314_1516_1718,480+ 0x2122_2324_2526_2728,481+ -2,482+ 100,483+ )484+ .unwrap()485+ }486+487+ 488+ fn media_object_v3_header_golden_vector() {489+ assert_eq!(490+ keyframe_header().encode().unwrap(),491+ [492+ 0x53, 0x47, 0x4f, 0x33, // SGO3493+ 0x00, 0x03, 0x00, 0x40, // version, header length494+ 0x01, 0x20, 0x07, 0x00, // H.264, priority, flags, reserved495+ 0x05, 0x00, 0x03, 0x20, // 1280x800496+ 0x00, 0x00, 0x00, 0x04, // payload length497+ 0x00, 0x00, 0x00, 0x00, // object id498+ 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, // group id499+ 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, // sequence500+ 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27, 0x28, // capture timestamp501+ 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xfe, // PTS502+ 0x00, 0x00, 0x00, 0x64, // delivery timeout503+ 0x00, 0x00, 0x00, 0x00, // reserved504+ ]505+ );506+ }507+508+ 509+ fn media_object_v3_header_round_trips() {510+ let expected = keyframe_header();511+ assert_eq!(512+ MediaObjectHeaderV3::decode(&expected.encode().unwrap()).unwrap(),513+ expected514+ );515+ }516+517+ 518+ fn media_object_v3_malformed_headers_fail_closed() {519+ let valid = keyframe_header().encode().unwrap();520+ let cases: &[(usize, u8)] = &[521+ (0, 0), // magic522+ (5, 2), // version523+ (7, 63), // header length524+ (8, 9), // codec525+ (10, 128), // flags526+ (11, 1), // reserved527+ (12, 0), // zero width528+ (59, 15), // delivery timeout below minimum529+ (60, 1), // reserved530+ ];531+ for &(offset, value) in cases {532+ let mut malformed = valid;533+ malformed[offset] = value;534+ assert!(535+ MediaObjectHeaderV3::decode(&malformed).is_err(),536+ "offset {offset}"537+ );538+ }539+ }540+541+ 542+ fn media_object_v3_enforces_group_boundaries_and_limits() {543+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);544+ assert!(545+ MediaObjectHeaderV3::h264(546+ 1280,547+ 800,548+ 1,549+ 0,550+ FrameFlags::NONE,551+ 255,552+ 1,553+ 1,554+ 1,555+ 1,556+ MIN_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS,557+ )558+ .is_ok()559+ );560+ assert!(561+ MediaObjectHeaderV3::h264(1280, 800, 1, 0, FrameFlags::NONE, 0, 1, 1, 1, 1, 16)562+ .is_err()563+ );564+ assert!(565+ MediaObjectHeaderV3::h264(1280, 800, 1, 0, FrameFlags::KEYFRAME, 0, 1, 1, 1, 1, 16)566+ .is_err()567+ );568+ assert!(MediaObjectHeaderV3::h264(1280, 800, 1, 0, keyframe, 256, 1, 1, 1, 1, 16).is_err());569+ assert!(570+ MediaObjectHeaderV3::h264(1280, 800, 1, 0, FrameFlags::NONE, 1, 1, 2, 2, 2, 16).is_ok()571+ );572+ assert!(573+ MediaObjectHeaderV3::h264(574+ 1280,575+ 800,576+ 1,577+ 0,578+ FrameFlags::DISCONTINUITY,579+ 1,580+ 1,581+ 2,582+ 2,583+ 2,584+ 16,585+ )586+ .is_err()587+ );588+ assert!(589+ MediaObjectHeaderV3::h264(1280, 800, 1, 0, FrameFlags::NONE, 1, 1, 2, 2, 2, 15)590+ .is_err()591+ );592+ assert!(593+ MediaObjectHeaderV3::h264(1280, 800, 1, 0, FrameFlags::NONE, 1, 1, 2, 2, 2, 1_001)594+ .is_err()595+ );596+ assert!(597+ MediaObjectHeaderV3::h264(598+ 1280,599+ 800,600+ 1,601+ u8::MAX,602+ FrameFlags::NONE,603+ 1,604+ 1,605+ 2,606+ 2,607+ 2,608+ MAX_MEDIA_OBJECT_DELIVERY_TIMEOUT_MS,609+ )610+ .is_ok()611+ );612+ assert!(MediaObjectHeaderV3::h264(1280, 800, 0, 0, keyframe, 0, 1, 1, 1, 1, 16).is_err());613+ assert!(614+ MediaObjectHeaderV3::h264(615+ 1280,616+ 800,617+ MAX_MEDIA_PAYLOAD_LEN + 1,618+ 0,619+ keyframe,620+ 0,621+ 1,622+ 1,623+ 1,624+ 1,625+ 16626+ )627+ .is_err()628+ );629+ }630+631+ 632+ async fn media_object_v3_round_trips_with_exact_eof() {633+ let expected = MediaObjectV3::new(keyframe_header(), vec![0, 0, 0, 1]).unwrap();634+ let (mut sender, mut receiver) = duplex(256);635+ write_media_object_v3(&mut sender, &expected).await.unwrap();636+ sender.shutdown().await.unwrap();637+ assert_eq!(read_media_object_v3(&mut receiver).await.unwrap(), expected);638+ }639+640+ 641+ async fn media_object_v3_rejects_empty_truncated_and_trailing_streams() {642+ let (mut empty_sender, mut empty_receiver) = duplex(16);643+ empty_sender.shutdown().await.unwrap();644+ assert!(matches!(645+ read_media_object_v3(&mut empty_receiver).await,646+ Err(ProtocolError::Io(ref error)) if error.kind() == ErrorKind::UnexpectedEof647+ ));648+649+ let mut header = keyframe_header();650+ header.payload_len = 2;651+ let (mut short_sender, mut short_receiver) = duplex(128);652+ short_sender653+ .write_all(&header.encode().unwrap())654+ .await655+ .unwrap();656+ short_sender.write_all(&[1]).await.unwrap();657+ short_sender.shutdown().await.unwrap();658+ assert!(matches!(659+ read_media_object_v3(&mut short_receiver).await,660+ Err(ProtocolError::Io(ref error)) if error.kind() == ErrorKind::UnexpectedEof661+ ));662+663+ let object = MediaObjectV3::new(keyframe_header(), vec![0, 0, 0, 1]).unwrap();664+ let (mut sender, mut receiver) = duplex(256);665+ write_media_object_v3(&mut sender, &object).await.unwrap();666+ sender.write_all(&[0xff]).await.unwrap();667+ sender.shutdown().await.unwrap();668+ assert!(matches!(669+ read_media_object_v3(&mut receiver).await,670+ Err(ProtocolError::InvalidMessage {671+ message_type: "v3 media object",672+ ..673+ })674+ ));675+ }676+677+ 678+ async fn media_object_v3_rejects_oversized_payload_before_body_read() {679+ let mut wire = keyframe_header().encode().unwrap();680+ wire[16..20].copy_from_slice(&((MAX_MEDIA_PAYLOAD_LEN as u32) + 1).to_be_bytes());681+ let (mut sender, mut receiver) = duplex(128);682+ sender.write_all(&wire).await.unwrap();683+ assert!(matches!(684+ read_media_object_v3(&mut receiver).await,685+ Err(ProtocolError::InvalidMediaPayloadLength { .. })686+ ));687+ }688+689+ fn control_request(last_sequence: Option<u64>) -> MediaControlRequestV3 {690+ MediaControlRequestV3::request_keyframe(691+ 0x0102_0304_0506_0708,692+ last_sequence,693+ KeyframeRequestReasonV3::FrontendBackpressure,694+ )695+ }696+697+ 698+ fn media_control_request_v3_golden_vector() {699+ assert_eq!(700+ control_request(Some(0x1112_1314_1516_1718))701+ .encode()702+ .unwrap(),703+ [704+ 0x53, 0x47, 0x43, 0x33, // SGC3705+ 0x00, 0x03, 0x00, 0x1c, // version, request length706+ 0x01, 0x05, 0x00, 0x00, // type, reason, reserved707+ 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, // request id708+ 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18, // last sequence709+ ]710+ );711+ }712+713+ 714+ fn media_control_request_v3_round_trips_and_preserves_none() {715+ for expected in [control_request(Some(42)), control_request(None)] {716+ assert_eq!(717+ MediaControlRequestV3::decode(&expected.encode().unwrap()).unwrap(),718+ expected719+ );720+ }721+ assert_eq!(&control_request(None).encode().unwrap()[20..28], &[0xff; 8]);722+ assert!(723+ MediaControlRequestV3::request_keyframe(724+ 1,725+ Some(u64::MAX),726+ KeyframeRequestReasonV3::Join,727+ )728+ .encode()729+ .is_err()730+ );731+ }732+733+ 734+ fn media_control_request_v3_rejects_malformed_and_unknown_fields() {735+ let valid = control_request(Some(1)).encode().unwrap();736+ let cases: &[(usize, u8)] = &[737+ (0, 0), // magic738+ (5, 2), // version739+ (7, 27), // request length740+ (8, 2), // unknown type741+ (9, 6), // unknown reason742+ (10, 1), // reserved743+ ];744+ for &(offset, value) in cases {745+ let mut malformed = valid;746+ malformed[offset] = value;747+ assert!(748+ MediaControlRequestV3::decode(&malformed).is_err(),749+ "offset {offset}"750+ );751+ }752+ }753+754+ 755+ async fn media_control_request_v3_reads_fixed_messages_and_clean_eof() {756+ let expected = control_request(Some(9));757+ let (mut sender, mut receiver) = duplex(128);758+ write_media_control_request_v3(&mut sender, &expected)759+ .await760+ .unwrap();761+ sender.shutdown().await.unwrap();762+ assert_eq!(763+ read_media_control_request_v3(&mut receiver).await.unwrap(),764+ Some(expected)765+ );766+ assert_eq!(767+ read_media_control_request_v3(&mut receiver).await.unwrap(),768+ None769+ );770+ }771+772+ 773+ async fn media_control_request_v3_rejects_truncation() {774+ let wire = control_request(Some(9)).encode().unwrap();775+ let (mut sender, mut receiver) = duplex(128);776+ sender.write_all(&wire[..27]).await.unwrap();777+ sender.shutdown().await.unwrap();778+ assert!(matches!(779+ read_media_control_request_v3(&mut receiver).await,780+ Err(ProtocolError::Io(ref error)) if error.kind() == ErrorKind::UnexpectedEof781+ ));782+ }783+}@@ -1,4 +1,5 @@
11 import assert from 'node:assert/strict';2+import { readFile } from 'node:fs/promises';23 import test from 'node:test';34 45 import {@@ -6,6 +7,15 @@ import {
67 disconnectRejectedRustConnection,78 } from './connection-attempt.mjs';89 10+const main = await readFile(new URL('./main.js', import.meta.url), 'utf8');11+12+test('connection diagnostics recognize grouped media v3', () => {13+ assert.match(14+ main,15+ /streamTransportMode = \[\s*'grouped-v3',\s*'independent-v2',\s*'reliable-v1',\s*'reliable-v0',\s*\]/,16+ );17+});18+919 test('only an explicitly connected result owns committed Rust state', () => {1020 assert.equal(committedRustConnection({ connected: true }), true);1121 assert.equal(committedRustConnection({ connected: false }), false);@@ -0,0 +1,123 @@
1+export const DECODER_RECOVERY_REASONS = Object.freeze({2+ TRANSPORT_GAP: 'transport-gap',3+ DISCONTINUITY: 'discontinuity',4+ DELIVERY_TIMEOUT: 'delivery-timeout',5+ DECODER_RESET: 'decoder-reset',6+ DECODER_ERROR: 'decoder-error',7+ FRONTEND_BACKPRESSURE: 'frontend-backpressure',8+});9+10+const VALID_REASONS = new Set(Object.values(DECODER_RECOVERY_REASONS));11+12+function validReason(reason) {13+ if (!VALID_REASONS.has(reason)) {14+ throw new TypeError(`unsupported decoder recovery reason: ${reason}`);15+ }16+ return reason;17+}18+19+/**20+ * Owns the small state machine between partial media delivery and WebCodecs.21+ *22+ * Entering recovery issues at most one keyframe request for that episode.23+ * Receiving a keyframe does not itself end the episode: the caller must first24+ * configure the decoder and enqueue the keyframe successfully. This keeps a25+ * synchronous decode failure from admitting dependent delta frames.26+ */27+export class DecoderRecoveryState {28+ #recovering;29+ #requestIssued = false;30+ #reason = null;31+ #onKeyframeRequest;32+33+ constructor({ initiallyRecovering = true, onKeyframeRequest = () => {} } = {}) {34+ if (typeof initiallyRecovering !== 'boolean') {35+ throw new TypeError('initiallyRecovering must be a boolean');36+ }37+ if (typeof onKeyframeRequest !== 'function') {38+ throw new TypeError('onKeyframeRequest must be a function');39+ }40+ this.#recovering = initiallyRecovering;41+ this.#onKeyframeRequest = onKeyframeRequest;42+ }43+44+ get recovering() {45+ return this.#recovering;46+ }47+48+ get reason() {49+ return this.#reason;50+ }51+52+ get requestIssued() {53+ return this.#requestIssued;54+ }55+56+ /** Enter recovery, coalescing with an already active recovery episode. */57+ enter(reason, { requestKeyframe = true } = {}) {58+ validReason(reason);59+ if (typeof requestKeyframe !== 'boolean') {60+ throw new TypeError('requestKeyframe must be a boolean');61+ }62+ const entered = !this.#recovering;63+ if (entered) {64+ this.#recovering = true;65+ this.#requestIssued = false;66+ this.#reason = reason;67+ } else if (this.#reason === null) {68+ this.#reason = reason;69+ }70+71+ let requested = false;72+ if (requestKeyframe && !this.#requestIssued) {73+ // Mark the request before invoking user code. Even if that code throws,74+ // the same recovery episode must not create an unbounded retry loop.75+ this.#requestIssued = true;76+ requested = true;77+ this.#onKeyframeRequest(reason);78+ }79+ return { entered, requested };80+ }81+82+ /**83+ * Start an explicitly new recovery episode, even if the previous episode84+ * has not recovered. This is the deliberate retry boundary after an85+ * external reset; ordinary repeated loss signals should use enter().86+ */87+ restart(reason) {88+ validReason(reason);89+ this.#recovering = true;90+ this.#requestIssued = false;91+ this.#reason = reason;92+ const result = this.enter(reason);93+ return { ...result, entered: true };94+ }95+96+ /** Reset session-local state without requesting from a not-yet-live host. */97+ reset({ initiallyRecovering = true } = {}) {98+ if (typeof initiallyRecovering !== 'boolean') {99+ throw new TypeError('initiallyRecovering must be a boolean');100+ }101+ this.#recovering = initiallyRecovering;102+ this.#requestIssued = false;103+ this.#reason = null;104+ }105+106+ shouldDropFrame({ keyframe }) {107+ if (typeof keyframe !== 'boolean') throw new TypeError('keyframe must be a boolean');108+ return this.#recovering && !keyframe;109+ }110+111+ /**112+ * Confirm the result of configuring/enqueuing a candidate recovery113+ * keyframe. Failure intentionally stays in the same coalesced episode.114+ */115+ confirmKeyframeEnqueued(succeeded) {116+ if (typeof succeeded !== 'boolean') throw new TypeError('succeeded must be a boolean');117+ if (!this.#recovering || !succeeded) return false;118+ this.#recovering = false;119+ this.#requestIssued = false;120+ this.#reason = null;121+ return true;122+ }123+}@@ -0,0 +1,164 @@
1+import assert from 'node:assert/strict';2+import { readFile } from 'node:fs/promises';3+import test from 'node:test';4+5+import {6+ DECODER_RECOVERY_REASONS,7+ DecoderRecoveryState,8+} from './decoder-recovery.mjs';9+10+const main = await readFile(new URL('./main.js', import.meta.url), 'utf8');11+12+test('connect wiring uses the recovery state instead of removed legacy state', () => {13+ assert.doesNotMatch(main, /waitingForDecoderKeyframe/);14+ assert.match(main, /decoderRecovery\.reset\(\);/);15+});16+17+test('decoder callbacks cannot mutate a replacement decoder session', () => {18+ assert.match(19+ main,20+ /output: \(frame\) => \{\s*if \(videoDecoder !== decoder\) \{\s*frame\.close\(\);\s*return;/,21+ );22+ assert.match(main, /error: \(e\) => \{\s*if \(videoDecoder !== decoder\) return;/);23+});24+25+test('a delivered discontinuity keyframe resets without requesting its replacement', () => {26+ const requests = [];27+ const recovery = new DecoderRecoveryState({28+ initiallyRecovering: false,29+ onKeyframeRequest: (reason) => requests.push(reason),30+ });31+32+ assert.deepEqual(33+ recovery.enter(DECODER_RECOVERY_REASONS.DISCONTINUITY, { requestKeyframe: false }),34+ { entered: true, requested: false },35+ );36+ assert.equal(recovery.shouldDropFrame({ keyframe: true }), false);37+ assert.equal(recovery.confirmKeyframeEnqueued(true), true);38+ assert.deepEqual(requests, []);39+ assert.match(40+ main,41+ /enterDecoderRecovery\(DECODER_RECOVERY_REASONS\.DISCONTINUITY, \{\s*requestKeyframe: !keyframe,/,42+ );43+});44+45+test('drops deltas until a recovery keyframe is successfully enqueued', () => {46+ const recovery = new DecoderRecoveryState();47+48+ assert.equal(recovery.recovering, true);49+ assert.equal(recovery.shouldDropFrame({ keyframe: false }), true);50+ assert.equal(recovery.shouldDropFrame({ keyframe: true }), false);51+ assert.equal(recovery.confirmKeyframeEnqueued(false), false);52+ assert.equal(recovery.recovering, true);53+ assert.equal(recovery.shouldDropFrame({ keyframe: false }), true);54+ assert.equal(recovery.confirmKeyframeEnqueued(true), true);55+ assert.equal(recovery.recovering, false);56+ assert.equal(recovery.shouldDropFrame({ keyframe: false }), false);57+});58+59+test('coalesces keyframe requests throughout one recovery episode', () => {60+ const requests = [];61+ const recovery = new DecoderRecoveryState({62+ initiallyRecovering: false,63+ onKeyframeRequest: (reason) => requests.push(reason),64+ });65+66+ assert.deepEqual(recovery.enter(DECODER_RECOVERY_REASONS.TRANSPORT_GAP), {67+ entered: true,68+ requested: true,69+ });70+ assert.deepEqual(recovery.enter(DECODER_RECOVERY_REASONS.DELIVERY_TIMEOUT), {71+ entered: false,72+ requested: false,73+ });74+ assert.deepEqual(recovery.enter(DECODER_RECOVERY_REASONS.FRONTEND_BACKPRESSURE), {75+ entered: false,76+ requested: false,77+ });78+ assert.deepEqual(requests, [DECODER_RECOVERY_REASONS.TRANSPORT_GAP]);79+80+ recovery.confirmKeyframeEnqueued(true);81+ recovery.enter(DECODER_RECOVERY_REASONS.FRONTEND_BACKPRESSURE);82+ assert.deepEqual(requests, [83+ DECODER_RECOVERY_REASONS.TRANSPORT_GAP,84+ DECODER_RECOVERY_REASONS.FRONTEND_BACKPRESSURE,85+ ]);86+});87+88+test('every supported failure reason notifies synchronously when entering recovery', () => {89+ for (const reason of Object.values(DECODER_RECOVERY_REASONS)) {90+ const requests = [];91+ const recovery = new DecoderRecoveryState({92+ initiallyRecovering: false,93+ onKeyframeRequest: (requestedReason) => requests.push(requestedReason),94+ });95+96+ const result = recovery.enter(reason);97+ assert.deepEqual(result, { entered: true, requested: true });98+ assert.deepEqual(requests, [reason]);99+ assert.equal(recovery.recovering, true);100+ }101+});102+103+test('an initially waiting decoder requests immediately on its first loss signal', () => {104+ const requests = [];105+ const recovery = new DecoderRecoveryState({106+ onKeyframeRequest: (reason) => requests.push(reason),107+ });108+109+ assert.deepEqual(recovery.enter(DECODER_RECOVERY_REASONS.DISCONTINUITY), {110+ entered: false,111+ requested: true,112+ });113+ assert.equal(recovery.reason, DECODER_RECOVERY_REASONS.DISCONTINUITY);114+ assert.deepEqual(requests, [DECODER_RECOVERY_REASONS.DISCONTINUITY]);115+});116+117+test('synchronous keyframe failure stays coalesced until an explicit restart', () => {118+ const requests = [];119+ const recovery = new DecoderRecoveryState({120+ initiallyRecovering: false,121+ onKeyframeRequest: (reason) => requests.push(reason),122+ });123+124+ recovery.enter(DECODER_RECOVERY_REASONS.DECODER_ERROR);125+ recovery.confirmKeyframeEnqueued(false);126+ recovery.enter(DECODER_RECOVERY_REASONS.DECODER_ERROR);127+ assert.deepEqual(requests, [DECODER_RECOVERY_REASONS.DECODER_ERROR]);128+129+ assert.deepEqual(recovery.restart(DECODER_RECOVERY_REASONS.DECODER_RESET), {130+ entered: true,131+ requested: true,132+ });133+ assert.deepEqual(requests, [134+ DECODER_RECOVERY_REASONS.DECODER_ERROR,135+ DECODER_RECOVERY_REASONS.DECODER_RESET,136+ ]);137+});138+139+test('session reset clears request coalescing without notifying an inactive host', () => {140+ const requests = [];141+ const recovery = new DecoderRecoveryState({142+ initiallyRecovering: false,143+ onKeyframeRequest: (reason) => requests.push(reason),144+ });145+146+ recovery.enter(DECODER_RECOVERY_REASONS.TRANSPORT_GAP);147+ recovery.reset();148+ assert.equal(recovery.recovering, true);149+ assert.equal(recovery.reason, null);150+ assert.equal(recovery.requestIssued, false);151+ assert.deepEqual(requests, [DECODER_RECOVERY_REASONS.TRANSPORT_GAP]);152+});153+154+test('rejects malformed reasons and frame decisions', () => {155+ const recovery = new DecoderRecoveryState();156+ assert.throws(() => recovery.enter('loss'), /unsupported decoder recovery reason/);157+ assert.throws(158+ () => recovery.enter(DECODER_RECOVERY_REASONS.DISCONTINUITY, { requestKeyframe: 'no' }),159+ /requestKeyframe must be a boolean/,160+ );161+ assert.throws(() => recovery.restart(null), /unsupported decoder recovery reason/);162+ assert.throws(() => recovery.shouldDropFrame({ keyframe: 1 }), /keyframe must be a boolean/);163+ assert.throws(() => recovery.confirmKeyframeEnqueued('yes'), /succeeded must be a boolean/);164+});@@ -88,6 +88,10 @@ import {
8888 fitStreamSurface,8989 } from './window-geometry.mjs';9090 import { buildVideoDecoderConfig } from './video-decoder-config.mjs';91+import {92+ DECODER_RECOVERY_REASONS,93+ DecoderRecoveryState,94+} from './decoder-recovery.mjs';9195 import {9296 formatInvitationExpiry,9397 grantLabel,@@ -826,6 +830,7 @@ async function connectHost() {
826830 control: result.control_available === true,827831 };828832 streamTransportMode = [833+ 'grouped-v3',829834 'independent-v2',830835 'reliable-v1',831836 'reliable-v0',@@ -881,7 +886,6 @@ async function connectHost() {
881886 await teardownAudioPipeline(false);882887 setAudioState('unavailable', result.audio_error || audioSupport.error || 'host audio unavailable');883888 }884- waitingForDecoderKeyframe = true;885889 updatePanelVisibility();886890 document.getElementById('node-id-text').textContent = result.host_node_id.substring(0, 16) + '...';887891 document.getElementById('frame-canvas').style.display = 'block';@@ -1079,7 +1083,42 @@ let streamPathMode = 'unknown';
10791083 let streamTransportMode = 'unknown';10801084 let streamRttMs = null;10811085 const MAX_DECODE_QUEUE_SIZE = 2;1082-let waitingForDecoderKeyframe = true;1086+1087+function requestDecoderKeyframe(reason) {1088+ if (!Number.isSafeInteger(activeFrameGeneration) || activeFrameGeneration < 1) return;1089+ void invoke('iroh_client_request_keyframe', {1090+ generation: activeFrameGeneration,1091+ reason,1092+ }).catch((error) => {1093+ console.warn(`keyframe request failed (${reason}):`, error);1094+ });1095+}1096+1097+const decoderRecovery = new DecoderRecoveryState({1098+ onKeyframeRequest: requestDecoderKeyframe,1099+});1100+1101+function enterDecoderRecovery(reason, { restart = false, requestKeyframe = true } = {}) {1102+ const result = restart1103+ ? decoderRecovery.restart(reason)1104+ : decoderRecovery.enter(reason, { requestKeyframe });1105+ // Closing WebCodecs and clearing the latest-frame presenter is part of the1106+ // recovery boundary. Do it immediately rather than allowing already queued1107+ // frames from a poisoned GOP to continue reaching the canvas.1108+ teardownDecoderPipeline();1109+ return result;1110+}1111+1112+function finishVideoChunkEnqueue(keyframe, succeeded) {1113+ if (succeeded) {1114+ if (keyframe) decoderRecovery.confirmKeyframeEnqueued(true);1115+ return;1116+ }1117+ // A synchronous decode rejection is a decoder-error episode if playback1118+ // was previously healthy. If recovery is already active, enter() coalesces1119+ // with its existing request and leaves the keyframe gate closed.1120+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.DECODER_ERROR);1121+}10831122 10841123 function resetAvSyncTelemetry(session = activeAudioSession, { recoverRing = false } = {}) {10851124 avSkew.reset();@@ -1166,6 +1205,7 @@ function teardownDecoderPipeline() {
11661205 11671206 function resetStreamTelemetry() {11681207 teardownDecoderPipeline();1208+ decoderRecovery.reset();11691209 droppedFrames = 0;11701210 transportDroppedFrames = 0;11711211 transportObjectDroppedFrames = null;@@ -1199,7 +1239,6 @@ function resetStreamTelemetry() {
11991239 streamPathMode = 'unknown';12001240 streamTransportMode = 'unknown';12011241 streamRttMs = null;1202- waitingForDecoderKeyframe = true;12031242 }12041243 12051244 function enqueueVideoChunk(chunk, receivedAt, codecLabel, mediaPtsMicros = null) {@@ -1226,8 +1265,12 @@ function enqueueVideoChunk(chunk, receivedAt, codecLabel, mediaPtsMicros = null)
12261265 function initWebCodecsDecoder(width, height, desc, codecStr) {12271266 teardownDecoderPipeline();12281267 console.log('WebCodecs configure:', codecStr, 'w:', width, 'h:', height, 'desc:', desc.byteLength, 'bytes');1229- videoDecoder = new VideoDecoder({1268+ const decoder = new VideoDecoder({12301269 output: (frame) => {1270+ if (videoDecoder !== decoder) {1271+ frame.close();1272+ return;1273+ }12311274 const now = performance.now();12321275 decoderOutputFrames++;12331276 decoderOutputRate.record(now);@@ -1238,19 +1281,26 @@ function initWebCodecsDecoder(width, height, desc, codecStr) {
12381281 framePresenter.enqueue(frame, timing);12391282 },12401283 error: (e) => {1284+ if (videoDecoder !== decoder) return;12411285 console.error('VideoDecoder error:', e);1242- waitingForDecoderKeyframe = true;1243- teardownDecoderPipeline();1286+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.DECODER_ERROR);12441287 }12451288 });1246- videoDecoder.configure(buildVideoDecoderConfig({1247- codec: codecStr,1248- width,1249- height,1250- description: desc,1251- }));1252- decoderConfigured = true;1253- waitingForDecoderKeyframe = false;1289+ videoDecoder = decoder;1290+ try {1291+ decoder.configure(buildVideoDecoderConfig({1292+ codec: codecStr,1293+ width,1294+ height,1295+ description: desc,1296+ }));1297+ decoderConfigured = true;1298+ return true;1299+ } catch (error) {1300+ console.warn('WebCodecs configure failed:', error);1301+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.DECODER_ERROR);1302+ return false;1303+ }12541304 }12551305 12561306 function formatCadence(summary) {@@ -1447,8 +1497,7 @@ function processFramePayload(payload, binaryData = null) {
14471497 if (codec && codec !== activeCodec) {14481498 activeCodec = codec;14491499 decoderConfigured = false;1450- waitingForDecoderKeyframe = true;1451- teardownDecoderPipeline();1500+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.DECODER_RESET);14521501 console.log('codec changed:', activeCodec);14531502 }14541503 @@ -1459,14 +1508,22 @@ function processFramePayload(payload, binaryData = null) {
14591508 // base64 branch only supports an older in-process sender during migration.14601509 const raw = binaryData ?? Uint8Array.from(atob(data), c => c.charCodeAt(0));14611510 const decoderBacklogged = videoDecoder?.decodeQueueSize >= MAX_DECODE_QUEUE_SIZE;1462- if (discontinuity || decoderBacklogged) waitingForDecoderKeyframe = true;1463- if (!keyframe && waitingForDecoderKeyframe) {1511+ if (discontinuity) {1512+ // A delivered discontinuity keyframe is the recovery boundary we were1513+ // waiting for, not evidence that another keyframe is missing.1514+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.DISCONTINUITY, {1515+ requestKeyframe: !keyframe,1516+ });1517+ } else if (decoderBacklogged) {1518+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.FRONTEND_BACKPRESSURE);1519+ }1520+ if (decoderRecovery.shouldDropFrame({ keyframe })) {1521+ if (!decoderRecovery.requestIssued) {1522+ enterDecoderRecovery(DECODER_RECOVERY_REASONS.DECODER_RESET);1523+ }14641524 droppedFrames++;14651525 return;14661526 }1467- if (keyframe && waitingForDecoderKeyframe && videoDecoder) {1468- teardownDecoderPipeline();1469- }14701527 const mediaPtsMicros = exactMediaTimestampMicros(ptsMicros);14711528 const chunkTimestamp = mediaPtsMicros !== null14721529 ? mediaPtsMicros@@ -1494,7 +1551,8 @@ function processFramePayload(payload, binaryData = null) {
14941551 timestamp: chunkTimestamp,14951552 data: chunkData,14961553 });1497- enqueueVideoChunk(chunk, receivedAt, 'av1', mediaPtsMicros);1554+ const enqueued = enqueueVideoChunk(chunk, receivedAt, 'av1', mediaPtsMicros);1555+ finishVideoChunkEnqueue(keyframe, enqueued);14981556 }14991557 15001558 } else if (activeCodec === 'h265') {@@ -1523,7 +1581,8 @@ function processFramePayload(payload, binaryData = null) {
15231581 timestamp: chunkTimestamp,15241582 data: nalsToLengthPrefixed(sliceNals),15251583 });1526- enqueueVideoChunk(chunk, receivedAt, 'hevc', mediaPtsMicros);1584+ const enqueued = enqueueVideoChunk(chunk, receivedAt, 'hevc', mediaPtsMicros);1585+ finishVideoChunkEnqueue(keyframe, enqueued);15271586 }15281587 15291588 } else {@@ -1551,7 +1610,8 @@ function processFramePayload(payload, binaryData = null) {
15511610 timestamp: chunkTimestamp,15521611 data: nalsToLengthPrefixed(sliceNals),15531612 });1554- enqueueVideoChunk(chunk, receivedAt, 'h264', mediaPtsMicros);1613+ const enqueued = enqueueVideoChunk(chunk, receivedAt, 'h264', mediaPtsMicros);1614+ finishVideoChunkEnqueue(keyframe, enqueued);15551615 }15561616 }15571617 } else {@@ -13,9 +13,11 @@ probe_timeout_seconds=15
1313 command_timeout_seconds=451414 readiness_timeout_seconds=301515 host_runtime_seconds=12016-media_transport="independent-v2"17-media_accept_log="media v2 client accepted"18-media_release_log="media v2 client released"16+media_transport="grouped-v3"17+media_accept_log="media v3 client accepted"18+media_release_log="media v3 client released"19+keyframe_recovery="ok"20+media_v2=false1921 media_v1=false2022 2123 usage() {@@ -32,8 +34,10 @@ Options:
3234 --reconnect-cycles COUNT Fresh sessions after the primary (default: 3)3335 --reconnect-frames COUNT Frames per reconnect session (default: 30)3436 --probe-timeout-seconds SEC Probe per-operation timeout (default: 15)37+ --media-v2 Exercise independent media v2 compatibility38+ instead of grouped media v33539 --media-v1 Exercise ordered media v1 compatibility36- instead of independent media v240+ instead of grouped media v33741 --help Show this help3842 EOF3943 }@@ -82,12 +86,23 @@ while [[ $# -gt 0 ]]; do
8286 shift 28387 ;;8488 --media-v1)89+ [[ "$media_v2" == false ]] || die "--media-v1 and --media-v2 cannot be combined"8590 media_transport="reliable-v1"8691 media_accept_log="media client accepted"8792 media_release_log="media client released"93+ keyframe_recovery="not-requested"8894 media_v1=true8995 shift9096 ;;97+ --media-v2)98+ [[ "$media_v1" == false ]] || die "--media-v1 and --media-v2 cannot be combined"99+ media_transport="independent-v2"100+ media_accept_log="media v2 client accepted"101+ media_release_log="media v2 client released"102+ keyframe_recovery="not-requested"103+ media_v2=true104+ shift105+ ;;91106 --help|-h)92107 usage93108 exit 0@@ -111,6 +126,12 @@ require_positive_integer "--primary-frames" "$primary_frames"
111126 require_positive_integer "--reconnect-cycles" "$reconnect_cycles"112127 require_positive_integer "--reconnect-frames" "$reconnect_frames"113128 require_positive_integer "--probe-timeout-seconds" "$probe_timeout_seconds"129+if [[ "$media_v1" == false && "$media_v2" == false ]]; then130+ [[ "$primary_frames" -ge 4 ]] \131+ || die "--primary-frames must be at least 4 for grouped v3 keyframe recovery"132+ [[ "$reconnect_frames" -ge 4 ]] \133+ || die "--reconnect-frames must be at least 4 for grouped v3 keyframe recovery"134+fi114135 115136 # A fresh Iroh endpoint performs relay discovery and path establishment on116137 # every reconnect. Keep the default three-cycle proof fast, but scale the host@@ -254,6 +275,43 @@ wait_for_log_count() {
254275 return 1255276 }256277 278+host_accepted_keyframe_request() {279+ local log_path="$1"280+ local request_id="$2"281+ sed $'s/\033\\[[0-9;]*m//g' "$log_path" 2>/dev/null \282+ | awk -v request_id="$request_id" '283+ index($0, "accepted media v3 keyframe request") \284+ && $0 ~ ("request_id=" request_id "([^[:digit:]]|$)") \285+ && index($0, "coalesced=false") { found=1 }286+ END { exit !found }287+ '288+}289+290+wait_for_keyframe_request() {291+ local log_path="$1"292+ local request_id="$2"293+ local watched_pid="$3"294+ local label="$4"295+ local deadline=$((SECONDS + readiness_timeout_seconds))296+297+ while (( SECONDS < deadline )); do298+ if host_accepted_keyframe_request "$log_path" "$request_id"; then299+ return 0300+ fi301+ if ! kill -0 "$watched_pid" 2>/dev/null; then302+ printf '%s exited before accepting keyframe request %s. Log follows:\n' \303+ "$label" "$request_id" >&2304+ sed -n '1,240p' "$log_path" >&2 || true305+ return 1306+ fi307+ sleep 0.1308+ done309+ printf 'timed out waiting for %s to accept keyframe request %s. Log follows:\n' \310+ "$label" "$request_id" >&2311+ sed -n '1,240p' "$log_path" >&2 || true312+ return 1313+}314+257315 start_watchdog() {258316 local pid="$1"259317 local seconds="$2"@@ -307,11 +365,17 @@ run_bounded() {
307365 validate_probe_log() {308366 local log_path="$1"309367 local expected_frames="$2"368+ local expected_request_id="$3"369+ if [[ "$media_v1" == true || "$media_v2" == true ]]; then370+ expected_request_id="not-requested"371+ fi310372 grep -Fxq 'probe=ok' "$log_path" || return 1311373 grep -Fxq "frames=${expected_frames}" "$log_path" || return 1312374 grep -Fxq 'dimensions=1280x800' "$log_path" || return 1313375 grep -Fxq 'sequence_gaps=0' "$log_path" || return 1314376 grep -Fxq "transport=${media_transport}" "$log_path" || return 1377+ grep -Fxq "keyframe_recovery=${keyframe_recovery}" "$log_path" || return 1378+ grep -Fxq "keyframe_request_id=${expected_request_id}" "$log_path" || return 1315379 awk -F= '$1 == "media_objects_dropped" && $2 ~ /^[0-9]+$/ { found=1 } END { exit !found }' \316380 "$log_path" || return 1317381 awk -F= '$1 == "media_objects_late" && $2 ~ /^[0-9]+$/ { found=1 } END { exit !found }' \@@ -325,11 +389,23 @@ validate_probe_log() {
325389 run_probe() {326390 if [[ "$media_v1" == true ]]; then327391 "$probe_bin" --media-v1 "$@"392+ elif [[ "$media_v2" == true ]]; then393+ "$probe_bin" --media-v2 "$@"328394 else329395 "$probe_bin" "$@"330396 fi331397 }332398 399+run_proof_probe() {400+ local request_id="$1"401+ shift402+ if [[ "$media_v1" == false && "$media_v2" == false ]]; then403+ run_probe --keyframe-smoke --keyframe-request-id "$request_id" "$@"404+ else405+ run_probe "$@"406+ fi407+}408+333409 cd "$workspace_dir"334410 cargo build --locked --profile "$cargo_profile" -p sigil-host --bins335411 @@ -354,7 +430,7 @@ secondary_log="$tmp_root/secondary.log"
354430 node_id="$(sed -n 's/^node_id=//p' "$identity_log" | tail -n 1)"355431 [[ -n "$node_id" ]] || die "identity initialization did not print a node ID"356432 357-RUST_LOG=info "$host_bin" serve \433+RUST_LOG='info,sigil::server=debug' "$host_bin" serve \358434 --identity "$identity_path" \359435 --source test-pattern \360436 --state-path "$runtime_path" \@@ -370,7 +446,8 @@ wait_for_log_line "$host_log" 'status=ready' "$host_pid" 'sigil' || die "host di
370446 host_node_id="$(sed -n 's/^node_id=//p' "$host_log" | tail -n 1)"371447 [[ "$host_node_id" == "$node_id" ]] || die "ready host node ID does not match its identity"372448 373-run_probe \449+run_proof_probe \450+ 1 \374451 --node-id "$node_id" \375452 --frames "$primary_frames" \376453 --timeout-seconds "$probe_timeout_seconds" \@@ -413,10 +490,14 @@ primary_watchdog_pid=""
413490 sed -n '1,240p' "$primary_log" >&2 || true414491 die "primary probe exited with status $primary_status"415492 }416-validate_probe_log "$primary_log" "$primary_frames" || {493+validate_probe_log "$primary_log" "$primary_frames" 1 || {417494 sed -n '1,240p' "$primary_log" >&2 || true418495 die "primary probe evidence is incomplete"419496 }497+if [[ "$media_v1" == false && "$media_v2" == false ]]; then498+ wait_for_keyframe_request "$host_log" 1 "$host_pid" 'sigil' \499+ || die "primary keyframe request was not accepted by the host"500+fi420501 421502 wait_for_log_count "$host_log" "$media_release_log" 1 "$host_pid" 'sigil' \422503 || die "primary session was not released"@@ -426,16 +507,21 @@ wait_for_log_count "$host_log" 'input client released' 1 "$host_pid" 'sigil' \
426507 cycle=1427508 while [[ "$cycle" -le "$reconnect_cycles" ]]; do428509 reconnect_log="$tmp_root/reconnect-${cycle}.log"510+ reconnect_request_id=$((cycle + 1))429511 if ! run_bounded "$command_timeout_seconds" "$reconnect_log" \430- run_probe --node-id "$node_id" --frames "$reconnect_frames" \512+ run_proof_probe "$reconnect_request_id" --node-id "$node_id" --frames "$reconnect_frames" \431513 --timeout-seconds "$probe_timeout_seconds" --expect-size 1280x800; then432514 sed -n '1,240p' "$reconnect_log" >&2 || true433515 die "reconnect cycle $cycle failed"434516 fi435- validate_probe_log "$reconnect_log" "$reconnect_frames" || {517+ validate_probe_log "$reconnect_log" "$reconnect_frames" "$reconnect_request_id" || {436518 sed -n '1,240p' "$reconnect_log" >&2 || true437519 die "reconnect cycle $cycle evidence is incomplete"438520 }521+ if [[ "$media_v1" == false && "$media_v2" == false ]]; then522+ wait_for_keyframe_request "$host_log" "$reconnect_request_id" "$host_pid" 'sigil' \523+ || die "reconnect cycle $cycle keyframe request was not accepted by the host"524+ fi439525 wait_for_log_count "$host_log" "$media_release_log" "$((cycle + 1))" "$host_pid" 'sigil' \440526 || die "reconnect cycle $cycle was not released"441527 wait_for_log_count "$host_log" 'input client released' "$((cycle + 1))" "$host_pid" 'sigil' \@@ -467,7 +553,7 @@ printf 'probe_binary=%s\n' "$probe_bin"
467553 printf 'probe_sha256=%s\n' "$(sha256_file "$probe_bin")"468554 printf 'node_id=%s\n' "$node_id"469555 printf 'primary_frames=%s\n' "$primary_frames"470-grep -E '^(transport|keyframes|sequence_gaps|media_objects_dropped|media_objects_late|input_ack_micros|path_mode|path_rtt_ms)=' "$primary_log"556+grep -E '^(transport|keyframe_recovery|keyframe_request_id|keyframes|sequence_gaps|media_objects_dropped|media_objects_late|input_ack_micros|path_mode|path_rtt_ms)=' "$primary_log"471557 printf 'active_client_rejection=ok\n'472558 printf 'reconnect_cycles=%s\n' "$reconnect_cycles"473559 printf 'cleanup=ok\n'@@ -10,11 +10,13 @@ use openh264::{formats::YUVSource, nal_units};
1010 use serde::Serialize;1111 use sigil_protocol::{1212 AUDIO_ALPN_V1, AudioFlags, AudioPacket, AudioPacketHeader, Capability, ClientHello, FrameFlags,13- INPUT_ALPN_V1, InputEvent, InvitationGrants, MAX_MEDIA_PAYLOAD_LEN, MAX_VIDEO_DIMENSION,14- MAX_VIDEO_PIXELS, MEDIA_ALPN_V1, MEDIA_ALPN_V2, MediaCodec, MediaFrame, PointerPosition,13+ INPUT_ALPN_V1, InputEvent, InvitationGrants, KeyframeRequestReasonV3, MAX_MEDIA_GROUP_BYTES_V3,14+ MAX_MEDIA_PAYLOAD_LEN, MAX_VIDEO_DIMENSION, MAX_VIDEO_PIXELS, MEDIA_ALPN_V1, MEDIA_ALPN_V2,15+ MEDIA_ALPN_V3, MediaCodec, MediaControlRequestV3, MediaFrame, MediaObjectV3, PointerPosition,1516 PointerSurfaceDimensions, ProtocolError, RELATIVE_POINTER_DELTA_MAX,1617 RELATIVE_POINTER_DELTA_MIN, read_host_hello, read_input_ack, read_media_frame,17- read_media_object, write_client_hello, write_input_event,18+ read_media_object, read_media_object_v3, write_client_hello, write_input_event,19+ write_media_control_request_v3,1820 };1921 use std::collections::{BTreeMap, VecDeque};2022 use std::io::Cursor;@@ -656,6 +658,7 @@ enum MediaTransport {
656658 LegacyV0,657659 ReliableStreamV1,658660 IndependentObjectsV2,661+ GroupedObjectsV3,659662 }660663 661664 impl MediaTransport {@@ -664,6 +667,7 @@ impl MediaTransport {
664667 Self::LegacyV0 => "reliable-v0",665668 Self::ReliableStreamV1 => "reliable-v1",666669 Self::IndependentObjectsV2 => "independent-v2",670+ Self::GroupedObjectsV3 => "grouped-v3",667671 }668672 }669673 }@@ -845,6 +849,189 @@ impl Drop for MediaObjectReceiver {
845849 }846850 }847851 852+853+enum MediaObjectReadOutcomeV3 {854+ Object {855+ accept_index: u64,856+ object: MediaObjectV3,857+ },858+ Dropped {859+ accept_index: u64,860+ reason: KeyframeRequestReasonV3,861+ },862+ Malformed(String),863+}864+865+impl MediaObjectReadOutcomeV3 {866+ fn accept_index(&self) -> Option<u64> {867+ match self {868+ Self::Object { accept_index, .. } | Self::Dropped { accept_index, .. } => {869+ Some(*accept_index)870+ }871+ Self::Malformed(_) => None,872+ }873+ }874+875+ fn is_fast_forward_barrier(&self) -> bool {876+ let Self::Object { object, .. } = self else {877+ return false;878+ };879+ object.header.object_id == 0880+ && object.header.flags.contains(FrameFlags::KEYFRAME)881+ && object.header.flags.contains(FrameFlags::CODEC_CONFIG)882+ && object.header.flags.contains(FrameFlags::DISCONTINUITY)883+ }884+}885+886+887+struct MediaObjectReorderV3 {888+ next_accept_index: u64,889+ completed: BTreeMap<u64, MediaObjectReadOutcomeV3>,890+}891+892+impl MediaObjectReorderV3 {893+ fn new(first_accept_index: u64) -> Self {894+ Self {895+ next_accept_index: first_accept_index,896+ completed: BTreeMap::new(),897+ }898+ }899+900+ fn pending_len(&self) -> usize {901+ self.completed.len()902+ }903+904+ fn push(905+ &mut self,906+ outcome: MediaObjectReadOutcomeV3,907+ ) -> Result<Option<MediaObjectReadOutcomeV3>, String> {908+ let Some(accept_index) = outcome.accept_index() else {909+ return Ok(Some(outcome));910+ };911+ if accept_index < self.next_accept_index {912+ // A discontinuity barrier may advance beyond older in-flight913+ // reads. Their eventual timeout/reset outcomes belong to the914+ // superseded GOP and must not poison the recovered sequence.915+ return Ok(None);916+ }917+ if self.completed.insert(accept_index, outcome).is_some() {918+ return Err(format!(919+ "Duplicate media v3 accept index {accept_index} completed"920+ ));921+ }922+ if self923+ .completed924+ .get(&accept_index)925+ .is_some_and(MediaObjectReadOutcomeV3::is_fast_forward_barrier)926+ {927+ self.completed.retain(|index, _| *index >= accept_index);928+ self.next_accept_index = accept_index;929+ }930+ self.take_next()931+ }932+933+ fn take_next(&mut self) -> Result<Option<MediaObjectReadOutcomeV3>, String> {934+ let Some(outcome) = self.completed.remove(&self.next_accept_index) else {935+ return Ok(None);936+ };937+ self.next_accept_index = self938+ .next_accept_index939+ .checked_add(1)940+ .ok_or_else(|| "Media v3 accept index overflowed".to_string())?;941+ Ok(Some(outcome))942+ }943+}944+945+struct MediaObjectReceiverV3 {946+ connection: iroh::endpoint::Connection,947+ reads: tokio::task::JoinSet<MediaObjectReadOutcomeV3>,948+ reorder: MediaObjectReorderV3,949+ next_accept_index: u64,950+ connection_closed: bool,951+}952+953+impl MediaObjectReceiverV3 {954+ fn new(connection: iroh::endpoint::Connection) -> Self {955+ Self {956+ connection,957+ reads: tokio::task::JoinSet::new(),958+ reorder: MediaObjectReorderV3::new(1),959+ next_accept_index: 0,960+ connection_closed: false,961+ }962+ }963+964+ async fn next(&mut self) -> Result<Option<MediaObjectReadOutcomeV3>, String> {965+ loop {966+ if let Some(completed) = self.reorder.take_next()? {967+ return Ok(Some(completed));968+ }969+ if self.connection_closed && self.reads.is_empty() {970+ if self.reorder.pending_len() != 0 {971+ return Err("Media v3 connection closed with incomplete object order".into());972+ }973+ return Ok(None);974+ }975+976+ tokio::select! {977+ biased;978+ completed = self.reads.join_next(), if !self.reads.is_empty() => {979+ let completed = completed980+ .ok_or_else(|| "Media v3 object reader ended unexpectedly".to_string())?981+ .map_err(|error| format!("Media v3 object reader task failed: {error}"))?;982+ if let Some(completed) = self.reorder.push(completed)? {983+ return Ok(Some(completed));984+ }985+ }986+ accepted = self.connection.accept_uni(), if !self.connection_closed987+ && self.reads.len() + self.reorder.pending_len()988+ < CLIENT_MEDIA_OBJECT_CAPACITY => {989+ let mut stream = match accepted {990+ Ok(stream) => stream,991+ Err(_) => {992+ self.connection_closed = true;993+ continue;994+ }995+ };996+ self.next_accept_index = self.next_accept_index.checked_add(1)997+ .ok_or_else(|| "Media v3 accept index overflowed".to_string())?;998+ let accept_index = self.next_accept_index;999+ self.reads.spawn(async move {1000+ match tokio::time::timeout(1001+ CLIENT_MEDIA_OBJECT_READ_TIMEOUT,1002+ read_media_object_v3(&mut stream),1003+ )1004+ .await1005+ {1006+ Err(_) => MediaObjectReadOutcomeV3::Dropped {1007+ accept_index,1008+ reason: KeyframeRequestReasonV3::DeliveryTimeout,1009+ },1010+ Ok(Err(ProtocolError::Io(_))) => MediaObjectReadOutcomeV3::Dropped {1011+ accept_index,1012+ reason: KeyframeRequestReasonV3::TransportGap,1013+ },1014+ Ok(Err(error)) => MediaObjectReadOutcomeV3::Malformed(format!(1015+ "Invalid media v3 object: {error}"1016+ )),1017+ Ok(Ok(object)) => MediaObjectReadOutcomeV3::Object {1018+ accept_index,1019+ object,1020+ },1021+ }1022+ });1023+ }1024+ }1025+ }1026+ }1027+}1028+1029+impl Drop for MediaObjectReceiverV3 {1030+ fn drop(&mut self) {1031+ self.reads.abort_all();1032+ }1033+}1034+8481035 8491036 enum MediaObjectSequenceDecision {8501037 Deliver { discontinuity: bool },@@ -904,6 +1091,93 @@ impl MediaObjectSequence {
9041091 }9051092 }9061093 1094+1095+enum MediaObjectSequenceDecisionV3 {1096+ Deliver { discontinuity: bool },1097+ DropLate,1098+ DropUntilKeyframe,1099+}1100+1101+1102+struct MediaObjectSequenceV3 {1103+ group_id: Option<u64>,1104+ last_object_id: Option<u32>,1105+ last_sequence: Option<u64>,1106+ group_payload_bytes: usize,1107+ waiting_for_keyframe: bool,1108+}1109+1110+impl MediaObjectSequenceV3 {1111+ fn new() -> Self {1112+ Self {1113+ waiting_for_keyframe: true,1114+ ..Self::default()1115+ }1116+ }1117+1118+ fn note_dropped_object(&mut self) -> bool {1119+ let entered = !self.waiting_for_keyframe;1120+ self.waiting_for_keyframe = true;1121+ entered1122+ }1123+1124+ fn classify(&mut self, object: &MediaObjectV3) -> MediaObjectSequenceDecisionV3 {1125+ let header = &object.header;1126+ if self1127+ .group_id1128+ .is_some_and(|group_id| header.group_id < group_id)1129+ || self1130+ .last_sequence1131+ .is_some_and(|sequence| header.sequence <= sequence)1132+ {1133+ return MediaObjectSequenceDecisionV3::DropLate;1134+ }1135+1136+ let new_group = self.group_id != Some(header.group_id);1137+ let recovery_keyframe = header.object_id == 01138+ && header.flags.contains(FrameFlags::KEYFRAME)1139+ && header.flags.contains(FrameFlags::CODEC_CONFIG);1140+ if new_group && !recovery_keyframe {1141+ self.waiting_for_keyframe = true;1142+ return MediaObjectSequenceDecisionV3::DropUntilKeyframe;1143+ }1144+ if !new_group && self.waiting_for_keyframe {1145+ return MediaObjectSequenceDecisionV3::DropUntilKeyframe;1146+ }1147+1148+ let sequence_contiguous = self1149+ .last_sequence1150+ .is_none_or(|sequence| sequence.checked_add(1) == Some(header.sequence));1151+ let object_contiguous = new_group1152+ || self1153+ .last_object_id1154+ .is_some_and(|object_id| object_id.checked_add(1) == Some(header.object_id));1155+ let next_group_bytes = if new_group {1156+ object.payload.len()1157+ } else {1158+ self.group_payload_bytes1159+ .saturating_add(object.payload.len())1160+ };1161+ if (!sequence_contiguous && !new_group)1162+ || !object_contiguous1163+ || next_group_bytes > MAX_MEDIA_GROUP_BYTES_V31164+ {1165+ self.waiting_for_keyframe = true;1166+ return MediaObjectSequenceDecisionV3::DropUntilKeyframe;1167+ }1168+1169+ let discontinuity = header.flags.contains(FrameFlags::DISCONTINUITY)1170+ || self.waiting_for_keyframe1171+ || !sequence_contiguous;1172+ self.group_id = Some(header.group_id);1173+ self.last_object_id = Some(header.object_id);1174+ self.last_sequence = Some(header.sequence);1175+ self.group_payload_bytes = next_group_bytes;1176+ self.waiting_for_keyframe = false;1177+ MediaObjectSequenceDecisionV3::Deliver { discontinuity }1178+ }1179+}1180+9071181 9081182 struct InputAvailability {9091183 relative_pointer: bool,@@ -1091,6 +1365,27 @@ async fn open_negotiated_input_stream(
10911365 Ok((send, recv, negotiation))10921366 }10931367 1368+fn connection_error_is_unsupported_alpn(error: &iroh::endpoint::ConnectionError) -> bool {1369+ matches!(1370+ error,1371+ iroh::endpoint::ConnectionError::ConnectionClosed(close)1372+ if close.error_code == iroh::endpoint::TransportErrorCode::crypto(0x78)1373+ )1374+}1375+1376+fn connect_error_is_unsupported_alpn(error: &iroh::endpoint::ConnectError) -> bool {1377+ match error {1378+ iroh::endpoint::ConnectError::Connecting {1379+ source: iroh::endpoint::ConnectingError::ConnectionError { source, .. },1380+ ..1381+ }1382+ | iroh::endpoint::ConnectError::Connection { source, .. } => {1383+ connection_error_is_unsupported_alpn(source)1384+ }1385+ _ => false,1386+ }1387+}1388+10941389 async fn open_negotiated_media_stream(10951390 endpoint: &Endpoint,10961391 address: &iroh::EndpointAddr,@@ -1100,68 +1395,159 @@ async fn open_negotiated_media_stream(
11001395 (11011396 iroh::endpoint::Connection,11021397 iroh::endpoint::RecvStream,1398+ Option<iroh::endpoint::SendStream>,11031399 NegotiatedV1Stream,11041400 MediaTransport,11051401 ),11061402 String,11071403 > {1108- match endpoint.connect(address.clone(), MEDIA_ALPN_V2).await {1404+ match endpoint.connect(address.clone(), MEDIA_ALPN_V3).await {11091405 Ok(connection) => {11101406 let (mut send, mut recv) = connection11111407 .open_bi()11121408 .await1113- .map_err(|error| format!("Failed to open media v2 handshake: {error}"))?;1409+ .map_err(|error| format!("Failed to open media v3 handshake: {error}"))?;11141410 let negotiation = negotiate_v1(11151411 &mut send,11161412 &mut recv,11171413 nonce,11181414 vec![Capability::VideoH264],11191415 Some(Capability::VideoH264),1120- "media v2",1416+ "media v3",11211417 invitation,11221418 )11231419 .await?;1124- send.finish()1125- .map_err(|error| format!("Failed to finish media v2 handshake: {error}"))?;11261420 Ok((11271421 connection,11281422 recv,1423+ Some(send),11291424 negotiation,1130- MediaTransport::IndependentObjectsV2,1425+ MediaTransport::GroupedObjectsV3,11311426 ))11321427 }1133- Err(v2_error) => {1134- let connection = endpoint1135- .connect(address.clone(), MEDIA_ALPN_V1)1136- .await1137- .map_err(|v1_error| {1138- format!(1139- "Failed to connect media v2 ({v2_error}); v1 compatibility connection also failed ({v1_error})"1428+ Err(v3_error) if connect_error_is_unsupported_alpn(&v3_error) => {1429+ match endpoint.connect(address.clone(), MEDIA_ALPN_V2).await {1430+ Ok(connection) => {1431+ let (mut send, mut recv) = connection1432+ .open_bi()1433+ .await1434+ .map_err(|error| format!("Failed to open media v2 handshake: {error}"))?;1435+ let negotiation = negotiate_v1(1436+ &mut send,1437+ &mut recv,1438+ nonce,1439+ vec![Capability::VideoH264],1440+ Some(Capability::VideoH264),1441+ "media v2",1442+ invitation,11401443 )1141- })?;1142- let (mut send, mut recv) = connection1143- .open_bi()1144- .await1145- .map_err(|error| format!("Failed to open media v1 stream: {error}"))?;1146- let negotiation = negotiate_v1(1147- &mut send,1148- &mut recv,1149- nonce,1150- vec![Capability::VideoH264],1151- Some(Capability::VideoH264),1152- "media v1",1153- invitation,1154- )1155- .await?;1156- send.finish()1157- .map_err(|error| format!("Failed to finish media v1 handshake: {error}"))?;1158- Ok((1159- connection,1160- recv,1161- negotiation,1162- MediaTransport::ReliableStreamV1,1163- ))1444+ .await?;1445+ send.finish()1446+ .map_err(|error| format!("Failed to finish media v2 handshake: {error}"))?;1447+ Ok((1448+ connection,1449+ recv,1450+ None,1451+ negotiation,1452+ MediaTransport::IndependentObjectsV2,1453+ ))1454+ }1455+ Err(v2_error) if connect_error_is_unsupported_alpn(&v2_error) => {1456+ let connection = endpoint1457+ .connect(address.clone(), MEDIA_ALPN_V1)1458+ .await1459+ .map_err(|v1_error| {1460+ format!(1461+ "Failed to connect media v3 ({v3_error}); v2 compatibility connection failed ({v2_error}); v1 compatibility connection also failed ({v1_error})"1462+ )1463+ })?;1464+ let (mut send, mut recv) = connection1465+ .open_bi()1466+ .await1467+ .map_err(|error| format!("Failed to open media v1 stream: {error}"))?;1468+ let negotiation = negotiate_v1(1469+ &mut send,1470+ &mut recv,1471+ nonce,1472+ vec![Capability::VideoH264],1473+ Some(Capability::VideoH264),1474+ "media v1",1475+ invitation,1476+ )1477+ .await?;1478+ send.finish()1479+ .map_err(|error| format!("Failed to finish media v1 handshake: {error}"))?;1480+ Ok((1481+ connection,1482+ recv,1483+ None,1484+ negotiation,1485+ MediaTransport::ReliableStreamV1,1486+ ))1487+ }1488+ Err(v2_error) => Err(format!(1489+ "Media v2 compatibility connection failed without an explicit unsupported-ALPN signal; refusing an unsafe downgrade to v1: {v2_error}"1490+ )),1491+ }1492+ }1493+ Err(v3_error) => Err(format!(1494+ "Media v3 connection failed without an explicit unsupported-ALPN signal; refusing an unsafe compatibility downgrade: {v3_error}"1495+ )),1496+ }1497+}1498+1499+async fn run_media_control_writer_v3(1500+ mut stream: iroh::endpoint::SendStream,1501+ mut requests: tokio::sync::mpsc::Receiver<(KeyframeRequestReasonV3, Option<u64>)>,1502+) {1503+ let mut request_id = 0_u64;1504+ while let Some((reason, last_sequence)) = requests.recv().await {1505+ let Some(next_request_id) = request_id.checked_add(1) else {1506+ eprintln!("[client] media v3 keyframe request id overflowed");1507+ break;1508+ };1509+ request_id = next_request_id;1510+ let request = MediaControlRequestV3::request_keyframe(request_id, last_sequence, reason);1511+ match tokio::time::timeout(1512+ Duration::from_secs(1),1513+ write_media_control_request_v3(&mut stream, &request),1514+ )1515+ .await1516+ {1517+ Ok(Ok(())) => {}1518+ Ok(Err(error)) => {1519+ eprintln!("[client] media v3 keyframe request failed: {error}");1520+ break;1521+ }1522+ Err(_) => {1523+ eprintln!("[client] media v3 keyframe request timed out");1524+ break;1525+ }1526+ }1527+ }1528+ let _ = stream.finish();1529+}1530+1531+fn parse_keyframe_request_reason(reason: &str) -> Result<KeyframeRequestReasonV3, String> {1532+ match reason {1533+ "join" => Ok(KeyframeRequestReasonV3::Join),1534+ "transport-gap" => Ok(KeyframeRequestReasonV3::TransportGap),1535+ "delivery-timeout" => Ok(KeyframeRequestReasonV3::DeliveryTimeout),1536+ "discontinuity" | "decoder-reset" | "decoder-error" => {1537+ Ok(KeyframeRequestReasonV3::DecoderReset)11641538 }1539+ "frontend-backpressure" => Ok(KeyframeRequestReasonV3::FrontendBackpressure),1540+ _ => Err(format!("Unsupported keyframe request reason: {reason}")),1541+ }1542+}1543+1544+fn try_queue_media_keyframe_request(1545+ sender: Option<&tokio::sync::mpsc::Sender<(KeyframeRequestReasonV3, Option<u64>)>>,1546+ reason: KeyframeRequestReasonV3,1547+ last_sequence: Option<u64>,1548+) {1549+ if let Some(sender) = sender {1550+ let _ = sender.try_send((reason, last_sequence));11651551 }11661552 }11671553 @@ -1649,48 +2035,53 @@ pub async fn iroh_client_connect(
16492035 let use_v1 = true;16502036 let input_alpn = if use_v1 { INPUT_ALPN_V1 } else { INPUT_ALPN };16512037 1652- let (frame_conn, mut frame_recv, media_negotiation, media_transport) = if use_v1 {1653- let first_attempt =1654- open_negotiated_media_stream(&endpoint, &addr, handshake_nonce, invitation.as_deref())1655- .await;1656- let (connection, recv, negotiation, transport) = match first_attempt {1657- Ok(result) => result,1658- Err(invitation_error)1659- if invitation.is_some()1660- && invitation_error.contains("Portal peer is not authorized") =>1661- {1662- // Recover only the narrow crash window where Sigil durably1663- // consumed the invitation but Portal did not durably clear it.1664- // The replay itself remains rejected; a second, ticket-free1665- // connection can succeed only as the already-enrolled Iroh1666- // peer authenticated by the exact invited host.1667- open_negotiated_media_stream(&endpoint, &addr, handshake_nonce, None)2038+ let (frame_conn, mut frame_recv, media_control_stream, media_negotiation, media_transport) =2039+ if use_v1 {2040+ let first_attempt = open_negotiated_media_stream(2041+ &endpoint,2042+ &addr,2043+ handshake_nonce,2044+ invitation.as_deref(),2045+ )2046+ .await;2047+ let (connection, recv, control, negotiation, transport) = match first_attempt {2048+ Ok(result) => result,2049+ Err(invitation_error)2050+ if invitation.is_some()2051+ && invitation_error.contains("Portal peer is not authorized") =>2052+ {2053+ // Recover only the narrow crash window where Sigil durably2054+ // consumed the invitation but Portal did not durably clear it.2055+ // The replay itself remains rejected; a second, ticket-free2056+ // connection can succeed only as the already-enrolled Iroh2057+ // peer authenticated by the exact invited host.2058+ open_negotiated_media_stream(&endpoint, &addr, handshake_nonce, None)16682059 .await16692060 .map_err(|retry_error| {16702061 format!(16712062 "{invitation_error}; ticket-free enrollment recovery also failed: {retry_error}"16722063 )16732064 })?1674- }1675- Err(error) => return Err(error),2065+ }2066+ Err(error) => return Err(error),2067+ };2068+ (connection, recv, control, Some(negotiation), transport)2069+ } else {2070+ let connection = endpoint2071+ .connect(addr.clone(), FRAME_ALPN)2072+ .await2073+ .map_err(|e| format!("Failed to connect frame stream: {e}"))?;2074+ let (mut send, recv) = connection2075+ .open_bi()2076+ .await2077+ .map_err(|e| format!("Failed to open frame stream: {e}"))?;2078+ send.write_all(&[1u8])2079+ .await2080+ .map_err(|e| format!("Failed to send start: {e}"))?;2081+ send.finish()2082+ .map_err(|e| format!("Failed to finish frame start stream: {e}"))?;2083+ (connection, recv, None, None, MediaTransport::LegacyV0)16762084 };1677- (connection, recv, Some(negotiation), transport)1678- } else {1679- let connection = endpoint1680- .connect(addr.clone(), FRAME_ALPN)1681- .await1682- .map_err(|e| format!("Failed to connect frame stream: {e}"))?;1683- let (mut send, recv) = connection1684- .open_bi()1685- .await1686- .map_err(|e| format!("Failed to open frame stream: {e}"))?;1687- send.write_all(&[1u8])1688- .await1689- .map_err(|e| format!("Failed to send start: {e}"))?;1690- send.finish()1691- .map_err(|e| format!("Failed to finish frame start stream: {e}"))?;1692- (connection, recv, None, MediaTransport::LegacyV0)1693- };16942085 let frame_connection_for_stats = frame_conn.clone();16952086 let media_session_id = media_negotiation16962087 .as_ref()@@ -1869,6 +2260,16 @@ pub async fn iroh_client_connect(
18692260 *ce = Some(endpoint.clone());18702261 }18712262 *state.media_connection.lock().await = Some((media_generation, frame_conn.clone()));2263+ let media_control_requests = if let Some(control_stream) = media_control_stream {2264+ let (control_tx, control_rx) = tokio::sync::mpsc::channel(1);2265+ *state.media_control.lock().await = Some((media_generation, control_tx.clone()));2266+ tokio::spawn(run_media_control_writer_v3(control_stream, control_rx));2267+ let _ = control_tx.try_send((KeyframeRequestReasonV3::Join, None));2268+ Some(control_tx)2269+ } else {2270+ *state.media_control.lock().await = None;2271+ None2272+ };18722273 let frame_events_in_flight = Arc::new(AtomicUsize::new(0));18732274 *state.frame_delivery.lock().await =18742275 Some((media_generation, Arc::clone(&frame_events_in_flight)));@@ -1986,6 +2387,9 @@ pub async fn iroh_client_connect(
19862387 let mut media_objects = (media_transport == MediaTransport::IndependentObjectsV2)19872388 .then(|| MediaObjectReceiver::new(frame_connection_for_stats.clone()));19882389 let mut media_object_sequence = MediaObjectSequence::new();2390+ let mut media_objects_v3 = (media_transport == MediaTransport::GroupedObjectsV3)2391+ .then(|| MediaObjectReceiverV3::new(frame_connection_for_stats.clone()));2392+ let mut media_object_sequence_v3 = MediaObjectSequenceV3::new();19892393 19902394 let mut decoder = if use_webcodecs {19912395 None@@ -2052,6 +2456,86 @@ pub async fn iroh_client_connect(
20522456 pts_micros,20532457 discontinuity,20542458 ) = match media_transport {2459+ MediaTransport::GroupedObjectsV3 => {2460+ let receiver = media_objects_v32461+ .as_mut()2462+ .expect("media v3 receiver must exist for media v3 transport");2463+ loop {2464+ let outcome = match receiver.next().await {2465+ Ok(Some(outcome)) => outcome,2466+ Ok(None) => {2467+ emit_frame_error(&app, media_generation, "Connection closed");2468+ break 'frames;2469+ }2470+ Err(error) => {2471+ emit_frame_error(&app, media_generation, error);2472+ break 'frames;2473+ }2474+ };2475+ match outcome {2476+ MediaObjectReadOutcomeV3::Dropped { reason, .. } => {2477+ let begins_resync = media_object_sequence_v3.note_dropped_object();2478+ let mut metrics = lock_client_media_metrics(&metrics);2479+ metrics.observe_transport_object_drop(!begins_resync);2480+ if begins_resync {2481+ frontend_waiting_for_keyframe = true;2482+ metrics.begin_frontend_resync(metrics_started.elapsed());2483+ try_queue_media_keyframe_request(2484+ media_control_requests.as_ref(),2485+ reason,2486+ media_object_sequence_v3.last_sequence,2487+ );2488+ }2489+ }2490+ MediaObjectReadOutcomeV3::Malformed(error) => {2491+ emit_frame_error(&app, media_generation, error);2492+ break 'frames;2493+ }2494+ MediaObjectReadOutcomeV3::Object { object, .. } => {2495+ let was_waiting = media_object_sequence_v3.waiting_for_keyframe;2496+ let discontinuity = match media_object_sequence_v3.classify(&object)2497+ {2498+ MediaObjectSequenceDecisionV3::Deliver { discontinuity } => {2499+ discontinuity2500+ }2501+ MediaObjectSequenceDecisionV3::DropLate => {2502+ lock_client_media_metrics(&metrics)2503+ .observe_transport_object_drop(true);2504+ continue;2505+ }2506+ MediaObjectSequenceDecisionV3::DropUntilKeyframe => {2507+ frontend_waiting_for_keyframe = true;2508+ let mut metrics = lock_client_media_metrics(&metrics);2509+ metrics.observe_transport_object_drop(false);2510+ metrics.begin_frontend_resync(metrics_started.elapsed());2511+ if !was_waiting {2512+ try_queue_media_keyframe_request(2513+ media_control_requests.as_ref(),2514+ KeyframeRequestReasonV3::TransportGap,2515+ media_object_sequence_v3.last_sequence,2516+ );2517+ }2518+ continue;2519+ }2520+ };2521+ let codec = match object.header.codec {2522+ MediaCodec::H264 => "h264".to_string(),2523+ };2524+ break (2525+ u32::from(object.header.width),2526+ u32::from(object.header.height),2527+ object.payload,2528+ object.header.flags.contains(FrameFlags::KEYFRAME),2529+ codec,2530+ Some(object.header.sequence),2531+ Some(object.header.capture_timestamp_us),2532+ Some(object.header.pts_us),2533+ discontinuity,2534+ );2535+ }2536+ }2537+ }2538+ }20552539 MediaTransport::IndependentObjectsV2 => {20562540 let receiver = media_objects20572541 .as_mut()@@ -2235,10 +2719,18 @@ pub async fn iroh_client_connect(
22352719 continue;22362720 }22372721 if !try_reserve_frame_channel_slot(&frame_events_in_flight) {2722+ let begins_resync = !frontend_waiting_for_keyframe;22382723 frontend_waiting_for_keyframe = true;22392724 let mut metrics = lock_client_media_metrics(&metrics);22402725 metrics.observe_frontend_queue_drop();22412726 metrics.begin_frontend_resync(metrics_started.elapsed());2727+ if begins_resync {2728+ try_queue_media_keyframe_request(2729+ media_control_requests.as_ref(),2730+ KeyframeRequestReasonV3::FrontendBackpressure,2731+ sequence,2732+ );2733+ }22422734 continue;22432735 }22442736 lock_client_media_metrics(&metrics)@@ -2450,6 +2942,43 @@ mod tests {
24502942 }24512943 }24522944 2945+ fn media_object_v3(2946+ group_id: u64,2947+ object_id: u32,2948+ sequence: u64,2949+ flags: FrameFlags,2950+ ) -> MediaObjectV3 {2951+ let payload = vec![sequence as u8];2952+ let header = sigil_protocol::MediaObjectHeaderV3::h264(2953+ 1280,2954+ 800,2955+ payload.len(),2956+ if object_id == 0 { 0 } else { 128 },2957+ flags,2958+ object_id,2959+ group_id,2960+ sequence,2961+ sequence * 1_000,2962+ sequence as i64 * 1_000,2963+ 100,2964+ )2965+ .unwrap();2966+ MediaObjectV3::new(header, payload).unwrap()2967+ }2968+2969+ fn media_object_outcome_v3(2970+ accept_index: u64,2971+ group_id: u64,2972+ object_id: u32,2973+ sequence: u64,2974+ flags: FrameFlags,2975+ ) -> MediaObjectReadOutcomeV3 {2976+ MediaObjectReadOutcomeV3::Object {2977+ accept_index,2978+ object: media_object_v3(group_id, object_id, sequence, flags),2979+ }2980+ }2981+24532982 fn completed_object_index(outcome: &MediaObjectReadOutcome) -> Option<u64> {24542983 outcome.object_index()24552984 }@@ -2603,6 +3132,127 @@ mod tests {
26033132 );26043133 }26053134 3135+ 3136+ fn media_v3_groups_use_wire_object_identity_and_recover_on_new_group_zero() {3137+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);3138+ let mut sequence = MediaObjectSequenceV3::new();3139+3140+ assert_eq!(3141+ sequence.classify(&media_object_v3(10, 0, 10, keyframe)),3142+ MediaObjectSequenceDecisionV3::Deliver {3143+ discontinuity: true3144+ }3145+ );3146+ assert_eq!(3147+ sequence.classify(&media_object_v3(10, 1, 11, FrameFlags::NONE)),3148+ MediaObjectSequenceDecisionV3::Deliver {3149+ discontinuity: false3150+ }3151+ );3152+ assert_eq!(3153+ sequence.classify(&media_object_v3(10, 3, 13, FrameFlags::NONE)),3154+ MediaObjectSequenceDecisionV3::DropUntilKeyframe3155+ );3156+ assert_eq!(3157+ sequence.classify(&media_object_v3(10, 2, 12, FrameFlags::NONE)),3158+ MediaObjectSequenceDecisionV3::DropUntilKeyframe3159+ );3160+ assert_eq!(3161+ sequence.classify(&media_object_v3(3162+ 20,3163+ 0,3164+ 20,3165+ keyframe.union(FrameFlags::DISCONTINUITY),3166+ )),3167+ MediaObjectSequenceDecisionV3::Deliver {3168+ discontinuity: true3169+ }3170+ );3171+ assert_eq!(3172+ sequence.classify(&media_object_v3(10, 2, 12, FrameFlags::NONE)),3173+ MediaObjectSequenceDecisionV3::DropLate3174+ );3175+ }3176+3177+ 3178+ fn late_dropped_completion_after_v3_barrier_cannot_poison_recovered_group() {3179+ let barrier = FrameFlags::KEYFRAME3180+ .union(FrameFlags::CODEC_CONFIG)3181+ .union(FrameFlags::DISCONTINUITY);3182+ let mut reorder = MediaObjectReorderV3::new(1);3183+ let mut sequence = MediaObjectSequenceV3::new();3184+3185+ let recovered = reorder3186+ .push(media_object_outcome_v3(5, 20, 0, 20, barrier))3187+ .unwrap()3188+ .unwrap();3189+ let MediaObjectReadOutcomeV3::Object { object, .. } = recovered else {3190+ panic!("recovery barrier must remain an object");3191+ };3192+ assert_eq!(3193+ sequence.classify(&object),3194+ MediaObjectSequenceDecisionV3::Deliver {3195+ discontinuity: true3196+ }3197+ );3198+3199+ assert!(3200+ reorder3201+ .push(MediaObjectReadOutcomeV3::Dropped {3202+ accept_index: 1,3203+ reason: KeyframeRequestReasonV3::DeliveryTimeout,3204+ })3205+ .unwrap()3206+ .is_none()3207+ );3208+3209+ let delta = reorder3210+ .push(media_object_outcome_v3(6, 20, 1, 21, FrameFlags::NONE))3211+ .unwrap()3212+ .unwrap();3213+ let MediaObjectReadOutcomeV3::Object { object, .. } = delta else {3214+ panic!("new group delta must remain an object");3215+ };3216+ assert_eq!(3217+ sequence.classify(&object),3218+ MediaObjectSequenceDecisionV3::Deliver {3219+ discontinuity: false3220+ }3221+ );3222+ }3223+3224+ 3225+ fn media_v3_receiver_rejects_group_payload_growth_beyond_the_shared_cap() {3226+ let keyframe = FrameFlags::KEYFRAME.union(FrameFlags::CODEC_CONFIG);3227+ let mut sequence = MediaObjectSequenceV3::new();3228+ assert!(matches!(3229+ sequence.classify(&media_object_v3(1, 0, 1, keyframe)),3230+ MediaObjectSequenceDecisionV3::Deliver { .. }3231+ ));3232+ sequence.group_payload_bytes = MAX_MEDIA_GROUP_BYTES_V3;3233+ assert_eq!(3234+ sequence.classify(&media_object_v3(1, 1, 2, FrameFlags::NONE)),3235+ MediaObjectSequenceDecisionV3::DropUntilKeyframe3236+ );3237+ }3238+3239+ 3240+ fn keyframe_request_reason_mapping_is_strict_and_coalesces_decoder_reasons() {3241+ assert_eq!(3242+ parse_keyframe_request_reason("transport-gap").unwrap(),3243+ KeyframeRequestReasonV3::TransportGap3244+ );3245+ assert_eq!(3246+ parse_keyframe_request_reason("discontinuity").unwrap(),3247+ KeyframeRequestReasonV3::DecoderReset3248+ );3249+ assert_eq!(3250+ parse_keyframe_request_reason("decoder-error").unwrap(),3251+ KeyframeRequestReasonV3::DecoderReset3252+ );3253+ assert!(parse_keyframe_request_reason("please").is_err());3254+ }3255+26063256 26073257 fn binary_frame_envelope_has_exact_stable_layout() {26083258 let payload = [0, 0, 0, 1, 0x65];@@ -3389,6 +4039,24 @@ mod tests {
33894039 );33904040 }33914041 4042+ 4043+ fn compatibility_downgrade_requires_tls_no_application_protocol() {4044+ let unsupported = iroh::endpoint::ConnectionError::ConnectionClosed(4045+ iroh::endpoint::TransportError::new(4046+ iroh::endpoint::TransportErrorCode::crypto(0x78),4047+ "no application protocol".to_string(),4048+ )4049+ .into(),4050+ );4051+ assert!(connection_error_is_unsupported_alpn(&unsupported));4052+ assert!(!connection_error_is_unsupported_alpn(4053+ &iroh::endpoint::ConnectionError::TimedOut4054+ ));4055+ assert!(!connection_error_is_unsupported_alpn(4056+ &iroh::endpoint::ConnectionError::Reset4057+ ));4058+ }4059+33924060 33934061 fn local_invitation_grants_bound_input_offers_before_the_host_intersection() {33944062 let view_only = input_capability_offers(InvitationGrants::VIEW);@@ -3494,6 +4162,29 @@ pub async fn iroh_client_send_input(
34944162 }34954163 }34964164 4165+4166+pub async fn iroh_client_request_keyframe(4167+ state: State<'_, AppState>,4168+ generation: u64,4169+ reason: String,4170+) -> Result<bool, String> {4171+ let reason = parse_keyframe_request_reason(&reason)?;4172+ let control = state.media_control.lock().await;4173+ let Some((current_generation, sender)) = control.as_ref() else {4174+ return Ok(false);4175+ };4176+ if *current_generation != generation {4177+ return Ok(false);4178+ }4179+ match sender.try_send((reason, None)) {4180+ Ok(()) => Ok(true),4181+ Err(tokio::sync::mpsc::error::TrySendError::Full(_)) => Ok(false),4182+ Err(tokio::sync::mpsc::error::TrySendError::Closed(_)) => {4183+ Err("Media control channel closed".to_string())4184+ }4185+ }4186+}4187+34974188 34984189 pub async fn iroh_client_ack_frame(34994190 state: State<'_, AppState>,@@ -3590,6 +4281,7 @@ pub async fn iroh_client_disconnect(state: State<'_, AppState>) -> Result<bool,
35904281 let _connection_serial = state.client_connection_serial.lock().await;35914282 next_audio_generation(&state.audio_connection_generation)?;35924283 lock_audio_deliveries(&state.audio_deliveries).clear();4284+ *state.media_control.lock().await = None;35934285 // Do not rely on endpoint shutdown alone to retire the session. The frame35944286 // reader and diagnostics task both own connection clones, and a surviving35954287 // media connection keeps the host's encoder and one-client lease alive.@@ -1,4 +1,5 @@
11 use serde::Serialize;2+use sigil_protocol::KeyframeRequestReasonV3;23 use std::collections::BTreeSet;34 use std::ffi::OsString;45 use std::sync::atomic::{AtomicBool, AtomicU64, AtomicUsize, Ordering};@@ -333,6 +334,7 @@ pub struct AppState {
333334 pub input_send: TokioMutex<Option<tokio::sync::mpsc::Sender<sigil_protocol::InputEvent>>>,334335 pub client_endpoint: TokioMutex<Option<iroh::Endpoint>>,335336 pub media_connection: TokioMutex<Option<(u64, iroh::endpoint::Connection)>>,337+ pub media_control: TokioMutex<Option<(u64, MediaControlRequestSender)>>,336338 pub frame_delivery: TokioMutex<Option<(u64, Arc<AtomicUsize>)>>,337339 pub client_media_generation: Arc<AtomicU64>,338340 pub audio_deliveries: Arc<StdMutex<AudioDeliveryState>>,@@ -344,6 +346,9 @@ pub struct AppState {
344346 pub dev_connect_node_id: Option<iroh::PublicKey>,345347 }346348 349+pub type MediaControlRequestSender =350+ tokio::sync::mpsc::Sender<(KeyframeRequestReasonV3, Option<u64>)>;351+347352 impl Default for AppState {348353 fn default() -> Self {349354 Self::new(LaunchOptions::default())@@ -365,6 +370,7 @@ impl AppState {
365370 input_send: TokioMutex::new(None),366371 client_endpoint: TokioMutex::new(None),367372 media_connection: TokioMutex::new(None),373+ media_control: TokioMutex::new(None),368374 frame_delivery: TokioMutex::new(None),369375 client_media_generation: Arc::new(AtomicU64::new(0)),370376 audio_deliveries: Arc::new(StdMutex::new(AudioDeliveryState::default())),@@ -643,6 +649,7 @@ mod tests {
643649 644650 assert!(state.client_endpoint.try_lock().unwrap().is_none());645651 assert!(state.media_connection.try_lock().unwrap().is_none());652+ assert!(state.media_control.try_lock().unwrap().is_none());646653 assert!(state.audio_connection.try_lock().unwrap().is_none());647654 }648655 @@ -95,6 +95,7 @@ fn main() {
9595 commands::network::iroh_client_connect,9696 commands::network::iroh_client_disconnect,9797 commands::network::iroh_client_ack_frame,98+ commands::network::iroh_client_request_keyframe,9899 commands::network::iroh_client_ack_audio,99100 commands::network::iroh_client_stop_audio,100101 commands::network::iroh_client_send_input,