@startuml
autonumber
box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box
box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box
== Create Media Pipeline ==
Cobalt -> Starboard: SbPlayerCreate(media_keys_handle, video_info, audio_info)
Starboard -> GStreamer_client: Create Gst pipeline with Rialto sinks
activate GStreamer_client
noteopt right of GStreamer_client
As the Gst pipeline is brought up the creation of
the first rialto sink for a pipeline triggers the
createMediaPipeline() call. Creation of additional
rialto sinks increments a ref count to the media
pipeline. To identify which pipeline a sink relates
to it will pass a unique ID for that pipeline which
is TBD but could be pointer to the pipeline root
element or the gstreamer pipeline bus handle.
For now videoRequirements can be set to 2160x3840
until resource management comes into scope.
end note
GStreamer_client -> rialtoClient: createMediaPipeline(client, videoRequirements)
rialtoClient -> rialtoServer: createMediaPipeline(client, videoRequirements)
rialtoServer -> rialtoServer: Check resource permissions that were granted to app and currently\nallocated pipelines allows creation of this new pipeline
note left: Currently this check is simply num_current_pipeline_sessions < max_playback_sessions
opt Permission check passed
rialtoServermaxWidth
Starboard -> GStreamer_client: Set PROP_MAX_VIDEO_WIDTH
end
opt maxHeight
Starboard -> GStreamer_client: Set PROP_MAX_VIDEO_HEIGHT
end
note right of GStreamer_client
As the Gst pipeline is brought up the creation of
the first rialto sink for a pipeline triggers the
createMediaPipeline() call. Creation of additional
rialto sinks increments a ref count to the media
pipeline. To identify which pipeline a sink relates
to it will pass the parent bin object pointer to
the pipeline root element.
end note
note right of GStreamer_client
If max width and max height are not set by the client
the mediaPipeline shall be created with videoRequirements
set to 2160x3840.
If max width and max height are set, then rialto shall
assume the playback is secondary video.
end note
GStreamer_client -> rialtoClient: createMediaPipeline(client, videoRequirements)
rialtoClient -> rialtoServer: Store video requirementscreateMediaPipeline(client, videoRequirements)
rialtoServer --> rialtoClientrialtoServer: pipeline_session
rialtoClientCheck resource permissions that were granted --> GStreamer_client: pipeline_session
GStreamer_client -> rialtoClient: load(pipeline_session, MEDIA_TYPE_MSE, ~"", ~"").
rialtoClientto app and currently\nallocated pipelines allows creation of this new pipeline
note left: Currently this check is simply num_current_pipeline_sessions < max_playback_sessions
opt Permission check passed
rialtoServer -> rialtoServer: load(pipeline_session, MEDIA_TYPE_MSE, ~"", ~"")Store video requirements
rialtoServer --> GStreamer_serverrialtoClient: Create rialto server MSE pipeline
GStreamer_server_session
rialtoClient --> rialtoServer:
rialtoServer -/GStreamer_client: pipeline_session
GStreamer_client -> rialtoClient: notifyNetworkStateload(pipeline_session, NETWORKMEDIA_STATE_BUFFERING)TYPE_MSE, ~"", ~"").
rialtoClient -/> GStreamer_client: notifyNetworkStaterialtoServer: load(pipeline_session, NETWORKMEDIA_STATE_BUFFERINGTYPE_MSE, ~"", ~"")
rialtoServer --> rialtoClientGStreamer_server: Create rialto server MSE
rialtoClient pipeline
GStreamer_server --> GStreamer_clientrialtoServer:
opt video_codecvideoRequirements != None
GStreamer_client -> rialtoClient: attachSource(pipeline_session, video_source)
rialtoClient2160x3840 && westerossink
rialtoServer -> GStreamer_server: Set secondary video
GStreamer_server --> rialtoServer:
end
rialtoServer ->/ rialtoServerrialtoClient: attachSourcenotifyNetworkState(pipeline_session, videoNETWORK_STATE_sourceBUFFERING)
rialtoServerrialtoClient ->/ GStreamer_serverclient: Add video sink to pipeline
GStreamer_server --> rialtoServer:notifyNetworkState(pipeline_session, NETWORK_STATE_BUFFERING)
rialtoServer --> rialtoClient:
rialtoClient --> GStreamer_client:
end
opt audiovideo_codec != None
GStreamer_client -> rialtoClient: attachSource(pipeline_session, audiovideo_source)
rialtoClient -> rialtoServer: attachSource(pipeline_session, audiovideo_source)
rialtoServer -> GStreamer_server: Add audiovideo sink to pipeline
GStreamer_server --> rialtoServer:
rialtoServer --> rialtoClient:
rialtoClient --> GStreamer_client:
end
rialtoClientopt audio_codec != None
GStreamer_client -> rialtoServerrialtoClient: allSourcesAttachedattachSource(pipeline_session, audio_source)
rialtoServerrialtoClient -> GStreamer_serverrialtoServer: Setup and add app source
attachSource(pipeline_session, audio_source)
rialtoServer -> GStreamer_server: Add audio sink to pipeline
GStreamer_server --> rialtoServer:
rialtoServer --> rialtoClient:
deactivate GStreamer_client
GStreamer_clientrialtoClient --> Starboard
StarboardGStreamer_client:
end
rialtoClient --> Cobalt:
== Notify that Pipeline & network session is currently IDLE ==
rialtoServer: allSourcesAttached(pipeline_session)
rialtoServer -/> rialtoClientGStreamer_server: Setup and notifyNetworkState(NETWORK_STATE_IDLE)
note right
Note that notification may occur before createMediaPipeline() call returns
end note
rialtoClientadd app source
GStreamer_server --> rialtoServer:
rialtoServer -/ --> rialtoClient:
deactivate GStreamer_client: notifyNetworkState(NETWORK_STATE_IDLE)
rialtoServerGStreamer_client --> Starboard
Starboard -/ rialtoClient: notifyPlaybackState(PLAYBACK_STATE_IDLE)
note right
Note--> Cobalt:
== Notify that notificationPipeline may& occurnetwork beforesession createMediaPipeline()is call returns
end note
rialtoClientcurrently IDLE ==
rialtoServer -/ GStreamer_clientrialtoClient: notifyPlaybackState(PLAYBACK notifyNetworkState(NETWORK_STATE_IDLE)
else Permission check failed
rialtoServer --> rialtoClient: ERROR_DENIEDnote right
Note that notification may occur before createMediaPipeline() call returns
end note
rialtoClient -->/ GStreamer_client: ERROR_DENIED
end
@enduml | 
