Introduction

Dynamic audio stream switching is currently a Netflix specific requirement to allow a new audio track to be selected whilst streaming is on progress with minimal disruption to the users viewing experience (no more than a short audio dropout). The new audio track may use a different codec to the previous one. The current implementation relies on platform specific implementations of the performAudioTrackCodecChannelSwitch_soc() API but it would be desirable in future to implement a platform agnostic solution for this.

Design

Netflix + rialto-gstreamer

@startuml

autonumber

box "Container" #LightGreen
participant Netflix
participant "DPI\n(IElementaryStreamAudioPlayer)" as DPI
participant GStreamer_client
participant rialtoClient
end box

box "Platform" #LightBlue
participant rialtoServer
participant rdk_gstreamer_utils
end box


note across
disable()/flush()/enable()/firstSampleAvailable() call sequence is guaranteed by Netflix but is only
used to detect that an audio stream switch has been initiated in DPI implementation.
end note

Netflix            ->  DPI:              disable()
DPI                --> Netflix:
Netflix            ->  DPI:              flush()
DPI                ->  DPI:              Set audio_switch flag
DPI                --> Netflix:
Netflix            ->  DPI:              enable()
DPI                --> Netflix:
Netflix            ->  DPI:              firstSampleAvailable()
DPI                --> Netflix:

DPI                --/ Netflix:          getNextMediaSample()
Netflix            ->  DPI:              write(sample)
note right: ISampleWriter detail skipped for clarity

opt audio_switch flag set

DPI                ->  DPI:              clear audio_switch flag
DPI                ->  GStreamer_client: performAudioTrackCodecChannelSwitch(audioAttr, caps, ...)
note right: This will be a custom Rialto implementation of the API
GStreamer_client   ->  rialtoClient:     flush(current_audio_src_id)
rialtoClient       ->  rialtoServer:     flush(current_audio_src_id)
rialtoServer       ->  rialtoServer:     flush
note right: Flush procedure described in Flush diagram
rialtoServer       --> rialtoClient:
rialtoClient       --> GStreamer_client:

GStreamer_client   ->  GStreamer_client: Convert audioAttr & caps to rialto::mediaSource
GStreamer_client   ->  rialtoClient:     switchSource(mediaSource)
rialtoClient       ->  rialtoServer:     switchSource(mediaSource)

opt audio caps changed
rialtoServer       ->  rdk_gstreamer_utils: performAudioTrackCodecChannelSwitch(audioAttr, caps, ...)
note right: This will call the underlying _soc\nimplementation for the platform
rdk_gstreamer_utils   --> rialtoServer:
end
rialtoServer       --> rialtoClient:
rialtoClient       --> GStreamer_client:
GStreamer_client   --> DPI:
DPI                --> Netflix:

end


@enduml

Netflix direct Rialto C++ integration

The removeSource() & remove Source implementations are as shown in the above diagram, the only difference is how they are triggered from DPI.

@startuml

autonumber

box "Container" #LightGreen
participant Netflix
participant "DPI\n(IElementaryStreamAudioPlayer)" as DPI
participant rialtoClient
end box


Netflix            ->  DPI:              disable()
DPI                --> Netflix:
Netflix            ->  DPI:              flush()
DPI                ->  rialtoClient:     removeSource(current_audio_src_id)
rialtoClient       --> DPI:
DPI                --> Netflix:

Netflix            ->  DPI:              enable()
DPI                --> Netflix:
Netflix            ->  DPI:              firstSampleAvailable()
DPI                --> Netflix:

opt First media sample after firstSampleAvailable() called

DPI                --/ Netflix:          getNextMediaSample()
Netflix            ->  DPI:              write(sample)
note right: ISampleWriter detail skipped for clarity


DPI                ->  DPI:              Convert sample info to rialto::mediaSource
DPI                ->  rialtoClient:     attachSource(mediaSource)
rialtoClient       --> DPI:
DPI                --> Netflix:

end

@enduml