Page History

Table of Contents

MediaPipeline

Create

Note
This sequence is aspirational and does not reflect the current implementation which uses a timer to create the pipeline in one call when it is assumed that all sources have been attached.

PlantUML Macro

format	SVG
title	Create Player

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box


== Create Media Pipeline ==

Cobalt            ->  Starboard:        SbPlayerCreate(media_keys_handle, video_info, audio_info)
Starboard         ->  GStreamer_client: Create Gst pipeline with Rialto sinks  
 
activate GStreamer_client
 
noteopt right of GStreamer_client
As the Gst pipeline is brought up the creation of
the first rialto sink for a pipeline triggers the
createMediaPipeline() call. Creation of additional
rialto sinks increments a ref count to the media
pipeline. To identify which pipeline a sink relates
to it will pass a unique ID for that pipeline which
is TBD but could be pointer to the pipeline root
element or the gstreamer pipeline bus handle.

For now videoRequirements can be set to 2160x3840
until resource management comes into scope.
end note

GStreamer_client  ->  rialtoClient:     createMediaPipeline(client, videoRequirements)
rialtoClient      ->  rialtoServer:     createMediaPipeline(client, videoRequirements)
rialtoServer      ->  rialtoServer:     Check resource permissions that were granted to app and currently\nallocated pipelines allows creation of this new pipeline
note left: Currently this check is simply num_current_pipeline_sessions < max_playback_sessions

opt Permission check passed
rialtoServer      maxWidth 
Starboard         ->  GStreamer_client: Set PROP_MAX_VIDEO_WIDTH
end
opt maxHeight 
Starboard         ->  GStreamer_client: Set PROP_MAX_VIDEO_HEIGHT
end

note right of GStreamer_client
As the Gst pipeline is brought up the creation of
the first rialto sink for a pipeline triggers the
createMediaPipeline() call. Creation of additional
rialto sinks increments a ref count to the media
pipeline. To identify which pipeline a sink relates
to it will pass the parent bin object pointer to 
the pipeline root element.
end note    
 
note right of GStreamer_client  
If max width and max height are not set by the client
the mediaPipeline shall be created with videoRequirements 
set to 2160x3840.

If max width and max height are set, then rialto shall
assume the playback is secondary video.
end note   

GStreamer_client  ->  rialtoClient:     createMediaPipeline(client, videoRequirements)
rialtoClient      ->  rialtoServer:     createMediaPipeline(client, videoRequirements)
rialtoServer      ->  rialtoServer:     Store video requirements
rialtoServer      --> rialtoClient:     pipeline_session
rialtoClient      --> GStreamer_client: pipeline_session

GStreamer_clientCheck resource permissions that were granted to app and currently\nallocated pipelines allows creation of this new pipeline
note left: Currently this check is simply num_current_pipeline_sessions < max_playback_sessions

opt Permission check passed
rialtoServer      ->  rialtoClientrialtoServer:     load(pipeline_session, MEDIA_TYPE_MSE, ~"", ~""). 
rialtoClientStore video requirements

rialtoServer      -->  rialtoServerrialtoClient:     load(pipeline_session, MEDIA_TYPE_MSE, ~"", ~"")
rialtoServer      
rialtoClient      -->  GStreamer_serverclient: Create rialto server MSE pipeline
pipeline_session

GStreamer_serverclient  --> rialtoServer:
rialtoServer      -/  rialtoClient:     notifyNetworkStateload(pipeline_session, NETWORKMEDIA_STATE_BUFFERING)
rialtoClient   TYPE_MSE, ~"", ~""). 
rialtoClient      -/>  GStreamer_clientrialtoServer:  notifyNetworkState   load(pipeline_session, NETWORKMEDIA_STATE_BUFFERINGTYPE_MSE, ~"", ~"")
rialtoServer      --> rialtoClient GStreamer_server: Create rialto server MSE pipeline
rialtoClientGStreamer_server      --> GStreamer_clientrialtoServer: 

opt video_codecvideoRequirements != None
GStreamer_client2160x3840 && westerossink
rialtoServer  	  ->  rialtoClient:     attachSource(pipeline_session, video_source)
rialtoClient GStreamer_server: Set secondary video
GStreamer_server  --> rialtoServer:
end   
rialtoServer      ->/  rialtoServerrialtoClient:     attachSourcenotifyNetworkState(pipeline_session, videoNETWORK_STATE_sourceBUFFERING)
rialtoServerrialtoClient      ->/  GStreamer_serverclient: Add video sink to pipeline
GStreamer_server  --> rialtoServer:notifyNetworkState(pipeline_session, NETWORK_STATE_BUFFERING)
rialtoServer      --> rialtoClient:     
rialtoClient      --> GStreamer_client:
end 

opt audiovideo_codec != None
GStreamer_client  ->  rialtoClient:     attachSource(pipeline_session, audiovideo_source)
rialtoClient      ->  rialtoServer:     attachSource(pipeline_session, audiovideo_source)
rialtoServer      ->  GStreamer_server: Add audiovideo sink to pipeline
GStreamer_server  --> rialtoServer:
rialtoServer      --> rialtoClient:
rialtoClient      --> GStreamer_client:
end

deactivateopt GStreamer_client
audio_codec != None
GStreamer_client  --> Starboard
Starboard rialtoClient:        --> Cobalt:



== Notify that Pipeline & network session is currently IDLE ==


rialtoServer attachSource(pipeline_session, audio_source)
rialtoClient      ->  rialtoServer:     attachSource(pipeline_session, audio_source)
rialtoServer      -/>  rialtoClientGStreamer_server: Add audio sink  notifyNetworkState(NETWORK_STATE_IDLE)
note right
Note that notification may occur before createMediaPipeline() call returns
end noteto pipeline
GStreamer_server  --> rialtoServer:
rialtoServer      --> rialtoClient:
rialtoClient       --/ > GStreamer_client: notifyNetworkState(NETWORK_STATE_IDLE)
end

rialtoServerrialtoClient       -/> rialtoServer: rialtoClient:     notifyPlaybackStateallSourcesAttached(PLAYBACKpipeline_STATE_IDLEsession)
noterialtoServer right
Note that notification may occur before-> createMediaPipeline() call returns
end note
rialtoClient       -/  GStreamer_client: notifyPlaybackState(PLAYBACK_STATE_IDLE)

else Permission check failed

 GStreamer_server:  Setup and add app source
GStreamer_server  --> rialtoServer:
rialtoServer      --> rialtoClient:

deactivate GStreamer_client

GStreamer_client  -->  ERROR_DENIED
rialtoClientStarboard
Starboard         --> GStreamer_client: ERROR_DENIEDCobalt:

end

@enduml

Destroy

PlantUML Macro

format	SVG
title	Destroy Player

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box


Cobalt== Notify that Pipeline & network session is currently IDLE ==

rialtoServer       -/  rialtoClient:     ->  Starboard:        SbPlayerDestroy(player_session)
StarboardnotifyNetworkState(NETWORK_STATE_IDLE)
note right
Note that notification may occur before createMediaPipeline() call returns
end note
rialtoClient         ->/  GStreamer_client: Destroy Rialto Client Gst pipeline
activate GStreamer_client
note right of GStreamer_client
As each rialto sink is freed it will decrement the
ref count to its media player (via the TBD unique
pipeline ID). When the ref count reaches zero the
destructor is triggered.
end note
GStreamer_client  -> notifyNetworkState(NETWORK_STATE_IDLE)

rialtoServer       -/  rialtoClient:     notifyPlaybackState(PLAYBACK_STATE_IDLE)
note right
Note that notification may occur before createMediaPipeline() call returns
end note
rialtoClient       -/  GStreamer_client: notifyPlaybackState(PLAYBACK_STATE_IDLE)

else Permission check failed

rialtoServer      --> rialtoClient:     ~IMediaPipeline()
deactivate GStreamer_clientERROR_DENIED
rialtoClient      --> GStreamer_client: ERROR_DENIED

end
@enduml

Destroy

PlantUML Macro

format	SVG
title	Destroy Player

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant GStreamer_server
end box


Cobalt       rialtoServer:     ~IMediaPipeline()
rialtoServer      ->  GStreamer_server: Destroy rialto server pipeline
GStreamer_server  --> rialtoServer:
rialtoServer      --> rialtoClient Starboard:        
rialtoClientSbPlayerDestroy(player_session)
Starboard         -->  GStreamer_client: 
GStreamer_clientDestroy  --> Starboard
Starboard         --> Cobalt:

@enduml

MediaKeys

Check Supported Key Systems - indicative use of Rialto

PlantUML Macro

format	SVG
title	Check supported key systems

@startuml

autonumber

box "Container" #LightGreen
participant Client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


alt Client should check if the required key system is supported
Client Rialto Client Gst pipeline
activate GStreamer_client
note right of GStreamer_client
As each rialto sink is freed it will decrement the
ref count to its media player (via the TBD unique
pipeline ID). When the ref count reaches zero the
destructor is triggered.
end note
GStreamer_client  ->  rialtoClient:     ~IMediaPipeline()
deactivate GStreamer_client
rialtoClient      ->  rialtoClientrialtoServer:     supportsKeySystem~IMediaPipeline(key_system)
rialtoClientrialtoServer ->  rialtoServer:  supportsKeySystem(key_system)
rialtoServer ->  OcdmGStreamer_server: Destroy rialto server pipeline
GStreamer_server  --> rialtoServer:
rialtoServer   opencdm_is_type_supported(key_system, "")
Ocdm     --> rialtoClient:     
rialtoClient  --> rialtoServer:  result
rialtoServer --> rialtoClientGStreamer_client:  result
rialtoClient
GStreamer_client  --> Client:Starboard
Starboard         --> Cobalt:

@enduml

MediaKeys

Check Supported Key Systems - indicative use of Rialto

PlantUML Macro

format	SVG
title	Check supported key systems

@startuml

autonumber

box "Container" #LightGreen
participant Client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


alt Client should check if the required key system is supported
Client       result
else Or find the supported key systems then choose its preferred
Client       ->  rialtoClient:  getSupportedKeySystemssupportsKeySystem(key_system)
rialtoClient ->  rialtoServer:  getSupportedKeySystems()
loop For each key system supported by RialtosupportsKeySystem(key_system)
rialtoServer ->  Ocdm:          opencdm_is_type_supported(key_system, "")
Ocdm         --> rialtoServer:  result
opt If key_system supported
rialtoServer -->  rialtoServerrialtoClient:  Add key_system to array of supported keys systems
end
end
rialtoServer -->result
rialtoClient --> Client:        result
else Or find the supported key systems then choose its preferred
Client       ->  rialtoClient:  key_systemsgetSupportedKeySystems()
rialtoClient -->  ClientrialtoServer:  getSupportedKeySystems()
loop For each key system supported by Rialto
rialtoServer -> key_systems
end


 Ocdm:          opencdm_is_type_supported(key_system, "")
Ocdm         --> rialtoServer:  result
opt If key_system supported
rialtoServer ->  rialtoServer:  Add key_system to array of supported keys systems
end
end
rialtoServer --> rialtoClient:  key_systems
rialtoClient --> Client:        key_systems
end


@enduml

Get DRM Version - Netflix/native Rialto

PlantUML Macro

format	SVG
title	Get DRM Version - Netflix/native Rialto

@startuml

autonumber

box "Container" #LightGreen
participant Netflix
participant DPI
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Netflix      ->  DPI
box "Container" #LightGreen
participant Netflix
participant DPI
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Netflix      ->  DPI:           getDrmVersion()
DPI          ->  rialtoClient:  getSupportedKeySystemVersion("com.netflix.playready")
rialtoClient ->  rialtoServer:  getSupportedKeySystemVersion(key_system)
rialtoServer ->  Ocdm:          opencdm_create_system(key_system);
note right: This temporary media keys does not count towards\nany quota allocated by Rialto Server Manager
Ocdm         --> rialtoServer:  media_keys
rialtoServer ->  Ocdm:          opencdm_system_get_version(media_keys)
Ocdm         --> rialtoServer:  version
rialtoServer ->  Ocdm:           getDrmVersion(opencdm_destruct_system(media_keys)
DPIOcdm          -->  rialtoClient:  getSupportedKeySystemVersion("com.netflix.playready")
rialtoClient ->  rialtoServer:  getSupportedKeySystemVersion(key_system)rialtoServer:  status
opt Key system version retrieved successfully
rialtoServer --> rialtoClient:  rv=true, version
else Error querying supported key system version
rialtoServer -->  OcdmrialtoClient:  rv=false, version=""
end
rialtoClient       opencdm_create_system(key_system);
note right: This temporary media keys does not count towards\nany quota allocated by Rialto Server Manager
Ocdm --> DPI:           rv, version
DPI        --> rialtoServer:  media_keys
rialtoServer  -->  OcdmNetflix:       rv, version

@enduml

Create - Cobalt/OCDM

PlantUML Macro

format	SVG
title	Create MediaKeys - Cobalt/OCDM

@startuml

autonumber

box  opencdm_system_get_version(media_keys)
Ocdm         --> rialtoServer:  version
rialtoServer ->  Ocdm:          opencdm_destruct_system(media_keys)
Ocdm         --> rialtoServer:  status
opt Key system version retrieved successfully
rialtoServer --> rialtoClient:  rv=true, version
else Error querying supported key system version
rialtoServer --> rialtoClient:  rv=false, version=""
end
rialtoClient --> DPI"Container" #LightGreen
participant Cobalt
participant Starboard
participant ocdmProxy
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Cobalt       ->  Starboard:     SbDrmCreateSystem(key_system)
Starboard    ->  ocdmProxy:     opencdm_create_system(key_system)
ocdmProxy    ->  rialtoClient:  createMediaKeys(key_system)
rialtoClient ->  rialtoServer:  createMediaKeys(key_system)
rialtoServer ->  Ocdm:           rv, version
DPIopencdm_create_system(key_system)
Ocdm          --> NetflixrialtoServer:  handle
rialtoServer ->  rialtoServer:  Store rv, version

@enduml

Create - Cobalt/OCDM

PlantUML Macro

format	SVG
title	Create MediaKeys - Cobalt/OCDM

@startuml

autonumber

box "Container" #LightGreen
participant Cobalt
participant Starboard
participant ocdmProxy
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Cobalt       ->  Starboard:     SbDrmCreateSystem(key_system)
Starboard    -> Media Keys instance handle
note right
This stored handle is only needed for deleting any
MediaKeys objects that the client fails to destroy
when moved out of the Running state.
end note
rialtoServer --> rialtoClient:  media_keys_handle
rialtoClient --> ocdmProxy:     opencdmmedia_create_system(key_system)keys_handle
ocdmProxy    -->  rialtoClientStarboard:  createMediaKeys(key_system)
rialtoClient ->  rialtoServer:  createMediaKeys(key_system)
rialtoServerhandle
Starboard    -->  OcdmCobalt:        handle

@enduml

Create - Netflix/native Rialto

PlantUML Macro

format	SVG
title	Create - Netflix/non-OCDM

@startuml

autonumber

box "Container" #LightGreen
participant Netflix
participant  opencdm_create_system(key_system)
Ocdm         --> rialtoServer:  handle
rialtoServer DPI
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box

Netflix      ->  rialtoServerDPI:  Store Media Keys instance handle
note right
This stored handle is only needed for deleting any
MediaKeys objects that the client fails to destroy
when moved out of the Running state.
end note
rialtoServer --> rialtoClient:  media_keys_handle
rialtoClient --> ocdmProxy:     media_keys_handle
ocdmProxyinitialize(drmSystemCallback, provisioningEngine)
DPI          ->  rialtoClient:  createMediaKeys("com.netflix.playready")
rialtoClient ->  rialtoServer:  createMediaKeys(key_system)
rialtoServer ->  Ocdm:          opencdm_create_system(key_system)
Ocdm         --> StarboardrialtoServer:  handle
rialtoServer ->  handle
StarboardrialtoServer:  Store Media --> Cobalt:        handle

@enduml

Create - Netflix/native Rialto

PlantUML Macro

format	SVG
title	Create - Netflix/non-OCDM

@startuml

autonumber

box "Container" #LightGreen
participant Netflix
participant DPI
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box

Netflix      -> Keys instance handle
note right
This stored handle is only needed for deleting any
MediaKeys objects that the client fails to destroy
when moved out of the Running state.
end note
rialtoServer --> rialtoClient:  media_keys_handle
rialtoClient --> DPI:           initialize(drmSystemCallback, provisioningEngine)media_keys_handle
DPI          -->  rialtoClientNetflix:  createMediaKeys("com.netflix.playready")
rialtoClient ->  rialtoServer:   status

@enduml

Destroy - Cobalt/OCDM

PlantUML Macro

format	SVG
title	Destroy - Cobalt/OCDM

@startuml

autonumber


box "Container" #LightGreen
participant Cobalt
participant Starboard
participant ocdmProxy
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Cobalt       createMediaKeys(key_system)
rialtoServer ->  Ocdm:          opencdm_create_system(key_system)
Ocdm         --> rialtoServer:  handle
rialtoServer ->  rialtoServerStarboard:  Store Media Keys instance SbDrmDestroySystem(handle)
noteStarboard right
This stored handle is-> only neededocdmProxy: for deleting any
MediaKeys objects that the client fails to destroy
when moved out of the Running state.
end note
rialtoServer --> rialtoClient:   opencdm_destruct_system(handle)
ocdmProxy    ->  rialtoClient:  ~IMediaKeys()
rialtoClient ->  rialtoServer:  destroyMediaKeys(media_keys_handle)
rialtoClientrialtoServer --> DPI Ocdm:           mediaopencdm_keysdestruct_system(handle)
DPI Ocdm         --> Netflix:       status

@enduml

Destroy - Cobalt/OCDM

PlantUML Macro

format	SVG
title	Destroy - Cobalt/OCDM

@startuml

autonumber


box "Container" #LightGreen
participant Cobalt
participant Starboard
participant ocdmProxy
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Cobalt     --> rialtoServer:  status
rialtoServer ->  rialtoServer:  Remove stored MediaKeys handle
rialtoServer --> rialtoClient:  status
rialtoClient --> ocdmProxy:     status
ocdmProxy    -->  Starboard:     SbDrmDestroySystem(handle)status
Starboard    -->  ocdmProxy:     opencdm_destruct_system(handle)
ocdmProxy    -> Cobalt:        status

@enduml

Destroy - Netflix/native Rialto

PlantUML Macro

format	SVG
title	Destroy - Netflix/native Rialto

@startuml

autonumber


box "Container" #LightGreen
participant Netflix
participant DPI
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Netflix      rialtoClient:  ~IMediaKeys()
rialtoClient ->  rialtoServer:  destroyMediaKeys(media_keys_handle)
rialtoServer ->  OcdmDPI:          opencdm_destruct_system(handle teardown()
OcdmDPI          -->  rialtoServerrialtoClient:  status~IMediaKeys()
rialtoServerrialtoClient ->  rialtoServer:  Remove stored MediaKeys handledestroyMediaKeys(media_keys_handle)
rialtoServer --> rialtoClient:  status
rialtoClient --> ocdmProxy Ocdm:     status
ocdmProxy    --> Starboard: opencdm_destruct_system(handle)
Ocdm     status
Starboard    --> CobaltrialtoServer:        status

@enduml

Destroy - Netflix/native Rialto

PlantUML Macro

format	SVG
title	Destroy - Netflix/native Rialto

@startuml

autonumber


box "Container" #LightGreen
participant Netflix
participant DPI
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box


Netflix      ->    status
rialtoServer ->  rialtoServer:  Remove stored MediaKeys handle
rialtoServer --> rialtoClient:  status
rialtoClient --> DPI:           teardown()status
DPI          --> Netflix:     rialtoClient:  ~IMediaKeys()
rialtoClient ->  rialtoServer:  destroyMediaKeys(media_keys_handle)
rialtoServer ->  Ocdm:          opencdm_destruct_system(handle)
Ocdm  status

note over Netflix: To prevent leaks ~IDrmSystem() should call\n~IMediaKeys() if teardown() wasn't called

@enduml

Check, if server certificate is supported

PlantUML Macro

format	SVG
title	isServerCertificateSupported

@startuml
autonumber

box "Container" #LightGreen
participant Client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant Ocdm
end box

Client       -->  rialtoServerrialtoClient:  status
rialtoServerisServerCertificateSupported(key_system)
rialtoClient ->  rialtoServer:  Remove stored MediaKeys handle
rialtoServer --> rialtoClient:  status
rialtoClient --> DPIisServerCertificateSupported(key_system)
rialtoServer ->  Ocdm:           status
DPIopencdm_system_supports_server_certificate(key_system)
Ocdm          --> NetflixrialtoServer:  result
rialtoServer --> rialtoClient:   statusresult

note over Netflix: To prevent leaks ~IDrmSystem() should call\n~IMediaKeys() if teardown() wasn't called
rialtoClient --> Client:        result
@enduml

IPC

Initialise Rialto Client

...

RDK Resources

[RDK Preferred]

Code Management Facility

RDK Forums

[RDK Conferences]

RDK Support

Archives

Versions Compared

Old Version 2

New Version Current

Key

MediaPipeline

Create

Destroy

Destroy

MediaKeys

Check Supported Key Systems - indicative use of Rialto

MediaKeys

Check Supported Key Systems - indicative use of Rialto

Get DRM Version - Netflix/native Rialto

Create - Cobalt/OCDM

Create - Cobalt/OCDM

Create - Netflix/native Rialto

Create - Netflix/native Rialto

Destroy - Cobalt/OCDM

Destroy - Cobalt/OCDM

Destroy - Netflix/native Rialto

Destroy - Netflix/native Rialto

Check, if server certificate is supported

IPC

Initialise Rialto Client

RDK Resources

[*RDK Preferred*]

Code Management Facility

RDK Forums

[RDK Conferences]

RDK Support

Archives

Page History

Versions Compared

Old Version 2

New Version Current

Key

MediaPipeline

Create

Destroy

Destroy

MediaKeys

Check Supported Key Systems - indicative use of Rialto

MediaKeys

Check Supported Key Systems - indicative use of Rialto

Get DRM Version - Netflix/native Rialto

Create - Cobalt/OCDM

Create - Cobalt/OCDM

Create - Netflix/native Rialto

Create - Netflix/native Rialto

Destroy - Cobalt/OCDM

Destroy - Cobalt/OCDM

Destroy - Netflix/native Rialto

Destroy - Netflix/native Rialto

Check, if server certificate is supported

IPC

Initialise Rialto Client

[RDK Preferred]