@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box


Cobalt             ->  Starboard:@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box


Cobalt             ->  Starboard:        SbPlayerSetPlaybackRate(player, rate)
note over GStreamer_client
For now we assume max 1 pipeline session in
GStreamer_client and therefore store its
Rialto handle in a local variable. This
will need to be fixed to support dual
playback.
end note

opt rate == 0
Starboard          ->  GStreamer_client: Set pipeline state PAUSED
GStreamer_client   ->  rialtoClient:     pause(pipeline_session)
else rate != 0
Starboard        SbPlayerSetPlaybackRate(player,  ->  rate)
note over GStreamer_client: Set
For now we assume max 1 pipeline statesession PLAYINGin
GStreamer_client and therefore ->store its
Rialto rialtoClient:handle in a local  play(pipeline_session)
end
rialtoClient       --> GStreamer_client: status
GStreamer_client   --> Starboard:        status
variable. This
will need to be fixed to support dual
playback.
end note

opt rate == 0
Starboard          ->  GStreamer_client: Set pipeline playbackstate ratePAUSED
GStreamer_client   ->  rialtoClient:     setPlaybackRate(rate)
rialtoClientpause(pipeline_session)
else rate != 0
Starboard          -->  GStreamer_client: status Set pipeline state PLAYING
GStreamer_client   --> Starboard rialtoClient:        status

Starboard   play(pipeline_session)
end
rialtoClient       --> Cobalt:           status



opt Pause->play successful
rialtoClient       -/  GStreamer_client: status
GStreamer_client: notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PLAYING)
else Play->pause successful
rialtoClient   --> Starboard:     -/  GStreamer_client: notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PAUSED)
else Play<->pause state change failed
rialtoClient  status

Starboard          -/>  GStreamer_client: Set notifyPlaybackState(pipeline_session, PLAYBACK_STATE_FAILURE)
end

@enduml 

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box

== Initialisation - register for callbacks ==

opt Video source attached
rialtoServer       ->  GStreamer_server:     g_signal_connect(video_decoder, getVideoUnderflowSignalName_soc(), video_underflow_cb, user_data);
GStreamer_server   --> rialtoServer:         video_handler_id
end

opt Audio source attached
rialtoServer       ->  GStreamer_server:     g_signal_connect(audio_decoder, getAudioUnderflowSignalName_soc(), audio_underflow_cb, user_data);
GStreamer_server   --> rialtoServer:         audio_handler_id
end


== Termination - unregister for callbacks ==

opt Video source removed
rialtoServer       ->  GStreamer_server:     g_signal_handler_disconnect(video_decoder, video_handler_id);
GStreamer_server   --> rialtoServer:
end

opt Audio source removed
rialtoServer       -> GStreamer_server:     g_signal_handler_disconnect(audio_decoder, audio_handler_id);
GStreamer_server   --> rialtoServer:
end


== Underflow ==

opt Data starvation in server AV pipeline
GStreamer_server   -/  rialtoServer:         video_underflow_cb() or audio_underflow_cb()
rialtoServer       ->  rialtoServer:         Set pipeline state to paused
rialtoServer       -/  rialtoClient:         notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PAUSED)
rialtoClient       -/  GStreamer_client:     notifyPlaybackState(pipeline_session, PLAYBACK_STATE_PAUSED)
rialtoServer       -/  rialtoClient:         notifyNetworkState(pipeline_session, NETWORK_STATE_STALLED)
rialtoClient       -/  GStreamer_client:     notifyNetworkState(pipeline_session, NETWORK_STATE_STALLED)

note over Starboard, GStreamer_client
Starboard does not have any support for underflow
so the event can be ignored for this integration.
end note

note across
There will be one ore more pending need data requests at this point which if serviced will allow playback to resume
end note

end

== Recovery ==

opt rialtoServer detects that any need media data requests pending at point of underflow are now serviced and pushed to GStreamer_server || EOS signalled for any underflowed sources
note across
It is likely that underflow is due to one source becoming starved whilst data is buffered for other sources, so waiting until pending request(s) are serviced should allow playback to resume.
There are also some YT conformance tests that delay signalling EOS for an underflowed source whilst the other continues to stream hence the EOS condition to allow streaming to resume for the valid source.
end note

rialtoServer       ->  rialtoServer:         Set pipeline state to playing
rialtoServer       -/  rialtoClient:         notifyNetworkState(pipeline_session, NETWORK_STATE_BUFFEREDnote across
underflow_enabled: Underflow is enabled when we're in playing state and source is attached.
underflow_cancelled: Underflow may be cancelled when haveData is called between notification from GStreamer and Underflow task handling.
end note
opt underflow_enabled && !underflow_cancelled
rialtoServer       -/  rialtoClient:         notifyBufferUnderflow(source_id)
rialtoClient       -/  GStreamer_client:     notifyBufferUnderflow(source_id)
GStreamer_client   -/  Starboard:            emit video_underflow_cb() or audio_underflow_cb()

note over Starboard, GStreamer_client
Starboard does not have any support for underflow
so the event can be ignored for this integration.
end note

end

note across
There will be one re more pending need data requests at this point which if serviced will allow playback to resume
end note

end

@enduml