@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box


Cobalt            ->  Starboard:        SbIsMediaSupported(video_codec, audio_codec, key_system)
note right: key_system ignored


Starboard         ->  rialtoClient:     createMediaPipelineCapabilities()
rialtoClient      ->  rialtoServer:     createMediaPipelineCapabilities()
rialtoServer      ->  GStreamer_server: Get decoder capabilities
GStreamer_server  ->  rialtoServer:     Decoder caps
rialtoServer      ->  rialtoServer:     Convert caps to list of supported MIME\ntypes: 'supportedMimeTypes'
note right
Rialto has pre-defined list of its known MIME types.
The list of supported MIME types is the subset of
MIME types that are in the pre-defined  list AND
are supported by the underlying decoders.
end note
rialtoServer      ->  rialtoClient:     media_pipeline_caps
rialtoClient      ->  Starboard:        media_pipeline_caps

Starboard         ->  Starboard:        Convert video_codec to MIME type
Starboard         ->  rialtoClient:     media_pipeline_caps.isMimeTypeSupported(mimeType)
rialtoClient      ->  rialtoServer:     media_pipeline_caps.isMimeTypeSupported(mimeType)
rialtoServer      ->  rialtoServer:     Check mimeType against supportedMimeTypes
rialtoServer      ->  rialtoClient:     result
rialtoClient      ->  Starboard:        result

opt video_codec supported
Starboard         ->  Starboard:        Convert audio_codec to MIME type
Starboard         ->  rialtoClient:     media_pipeline_caps.isMimeTypeSupported(mimeType)
rialtoClient      ->  rialtoServer:     media_pipeline_caps.isMimeTypeSupported(mimeType)
rialtoServer      ->  rialtoServer:     Check mimeType against supportedMimeTypes
rialtoServer      ->  rialtoClient:     result
rialtoClient      ->  Starboard:        result

opt audio_codec supported
Starboard         --> Cobalt:           true
else
Starboard         --> Cobalt:           false
end

else
Starboard         ->  Cobalt:           false
end



== Get Supported Mime Types ==
note across: An alternative implementation could use the getsupporteMimeTypes() API

Starboard         ->  rialtoClient:     media_pipeline_caps.getSupportedMimeTypes(sourceType)
note right: sourceType indicates audio or video
rialtoClient      ->  rialtoServer:     media_pipeline_caps.getSupportedMimeTypes(sourceType)
rialtoServer      ->  rialtoServer:     Extract relevant MIME types from\nsupportedMimeTypes based on sourceType
rialtoServer      ->  rialtoClient:     mimeTypes[]
rialtoClient      ->  Starboard:        mimeTypes[]

@enduml

@startuml


autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box


Cobalt            ->  Starboard:        SetBounds(z, x, y, w, h)
Starboard         ->  GStreamer_client: set video sink geometry (pipeline, x, y, w, h)
GStreamer_client  ->  rialtoClient:     setVideoWindow(pipeline_session, x, y, w, h)
rialtoClient      ->  rialtoServer:     setVideoWindow(pipeline_session, x, y, w, h)
rialtoServer      ->  GStreamer_server: set video sink geometry (pipeline, x, y, w, h)
GStreamer_server  --> rialtoServer:     status
rialtoServer      --> rialtoClient:     status
rialtoClient      --> GStreamer_client: status
GStreamer_client  --> Starboard:
Starboard         --> Cobalt:

@enduml

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box

== During pipeline creation ==

rialtoServer      ->  GStreamer_server: Subscribe for QOS event

== During playback ==

opt Frames dropped or element changed processing strategy

note across

This event is generate whenever GStreamer generates a QoS event which is currently for these reasons:

 - The element dropped a buffer because of QoS reasons.

 - An element changed its processing strategy because of QoS reasons. This could include a decoder that

   decided to drop every B frame to increase its processing speed or an effect element switching to a lower

   quality algorithm.

end note

GStreamer_server  -/  rialtoServer:     GstBusMessage(qos_event)
rialtoServer      ->  rialtoServer:     Extract required data from event
rialtoServer      -/  rialtoClient:     notifyQos(pipeline_session, source_id, qos_info)
rialtoClient      -/  GStreamer_client: notifyQos(pipeline_session, source_id, qos_info)
GStreamer_client  ->  GStreamer_client: Create GST_EVENT_QOS message from qos_info
GStreamer_client  -/  Starboard:        GST_MESSAGE_QOS(qos_event)

note right
Only number of processed & dropped frames/
samples is sent by Rialto so the QOS event
sent by GStreamer_client will have less
valid properties set than the original server
side event.
end note

note over Cobalt, Starboard
QOS events are not propogated up to
Cobalt but can be queried anytime
by calling SbPlayerInfo2()
end note

end

@enduml

@startuml
autonumber

box "Container" #LightGreen
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box

GStreamer_client  ->  rialtoClient:     setVolume(pipeline_session, volume)
rialtoClient      ->  rialtoServer:     setVolume(pipeline_session, volume)
rialtoServer      ->  GStreamer_server: gst_stream_volume_set_volume(pipeline, GST_STREAM_VOLUME_FORMAT_LINEAR, volume)
GStreamer_server  --> rialtoServer:     status
rialtoServer      --> rialtoClient:     status
rialtoClient      --> GStreamer_client: status
@enduml

@startuml
autonumber

box "Container" #LightGreen
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box

GStreamer_client  ->  rialtoClient:     getVolume(pipeline_session)
rialtoClient      ->  rialtoServer:     getVolume(pipeline_session)
rialtoServer      ->  GStreamer_server: gst_stream_volume_get_volume(pipeline, GST_STREAM_VOLUME_FORMAT_LINEAR)
GStreamer_server  --> rialtoServer:     volume
rialtoServer      --> rialtoClient:     volume
rialtoClient      --> GStreamer_client: volume
@enduml

@startuml
autonumber

box "Container" #LightGreen
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box

GStreamer_client  ->  rialtoClient:     setMute(pipeline_session, mute)
rialtoClient      ->  rialtoServer:     setMute(pipeline_session, mute)
rialtoServer      ->  GStreamer_server: gst_stream_volume_set_mute(pipeline, mute)
GStreamer_server  --> rialtoServer:     status
rialtoServer      --> rialtoClient:     status
rialtoClient      --> GStreamer_client: status
@enduml