Page History

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box

== Initialisation - register for callbacks ==

opt Video source attached
rialtoServer       ->  GStreamer_server:     g_signal_connect(video_decoder, getVideoUnderflowSignalName_soc(), video_underflow_cb, user_data);
GStreamer_server   --> rialtoServer:         video_handler_id
end

opt Audio source attached
rialtoServer       ->  GStreamer_server:     g_signal_connect(audio_decoder, getAudioUnderflowSignalName_soc(), audio_underflow_cb, user_data);
GStreamer_server   --> rialtoServer:         audio_handler_id
end


== Termination - unregister for callbacks ==

opt Video source removed
rialtoServer       ->  GStreamer_server:     g_signal_handler_disconnect(video_decoder, video_handler_id);
GStreamer_server   --> rialtoServer:
end

opt Audio source removed
rialtoServer       -> GStreamer_server:     g_signal_handler_disconnect(audio_decoder, audio_handler_id);
GStreamer_server   --> rialtoServer:
end


== Underflow ==

opt Data starvation in server AV pipeline
GStreamer_server   -/  rialtoServer:         video_underflow_cb() or audio_underflow_cb()
rialtoServer       ->/  rialtoServerrialtoClient:            Set pipeline state to paused
rialtoServer  notifyBufferUnderflow(source_id)
rialtoClient       -/  rialtoClientGStreamer_client:          notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PAUSED)
rialtoClient       -/  GStreamer_client:  notifyBufferUnderflow(source_id)
GStreamer_client   -/  Starboard:            emit notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PAUSED)
rialtoServer       -/  rialtoClient:         notifyNetworkState(pipeline_session, NETWORK_STATE_STALLED)
rialtoClient       -/  GStreamer_client:     notifyNetworkState(pipeline_session, NETWORK_STATE_STALLED)

note over Starboard, GStreamer_client
Starboard does not have any support for underflow
so the event can be ignored for this video_underflow_cb() or audio_underflow_cb()

note over Starboard, GStreamer_client
Starboard does not have any support for underflow
so the event can be ignored for this integration.
end note

note across
There will be one orere more pending need data requests at this point which if serviced will allow playback to resume
end note

end

== Recovery ==

opt rialtoServer detects that any need media data requests pending at point of underflow are now serviced and pushed to GStreamer_server || EOS signalled for any underflowed sources
note across
It is likely that underflow is due to one source becoming starved whilst data is buffered for other sources, so waiting until pending request(s) are serviced should allow playback to resume.
There are also some YT conformance tests that delay signalling EOS for an underflowed source whilst the other continues to stream hence the EOS condition to allow streaming to resume for the valid source.
end note

rialtoServer       ->  rialtoServer:         Set pipeline state to playing
rialtoServer       -/  rialtoClient:         notifyNetworkState(pipeline_session, NETWORK_STATE_BUFFERED)
rialtoClient       -/  GStreamer_client:     notifyNetworkState(pipeline_session, NETWORK_STATE_BUFFERED)
rialtoServer       -/  rialtoClient:         notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PLAYING)
rialtoClient       -/  GStreamer_client:     notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PLAYING)

end

@enduml