GStreamer Training

Gstreamer Overview

IP TV-Gstreamer Overview

GStreamer Abstraction Layer & Plugin

AAMP has two modes of operation - gstreamer player (when JS/Native API's are used by the application) and gstreamer element (when application manages playbin as in the case of html5 video playback)

AAMP as Gstreamer player

In this mode of operation, AAMP manages the gstreamer pipeline. AAMP creates one playbin for each stream ( Audio or Video) and adds these playbins to pipeline. AAMP manages the pipeline state as well as injects data to the playbins through gstreamer appsrc element. AAMP sets caps of appsrc src pads based on the format of injected data. Buffers and events are sent to gstreamer using these appsrc objects.

AAMP as Gstreamer element

In this mode of operation, AAMP is a gstreamer element. Usually, application manages the lifecycle of the playbin which has aamp as one element. There is a dummy element aampsrc to handle protocol "aamp://" and "aamps://" which is replaced to http internally. aamp gstreamer element manages state of AAMP engine as well as gets media data and injects it to next element. aamp gstreamer element exposes dynamic pads which have caps corresponding to the format of injected data on that srcpad. 

...

For HLS, AAMP uses software demux which is a part of AAMP code by default. If codec info is not in the manifest, AAMP does not demux the TS fragments, which causes h/w demux gstreamer element of the platform to be used by playbin to demux the TS fragments. In summary, AAMP outputs elementary streams on using s/w demux, else output mpeg TS buffers which results in hardware demux to be used.

HLS on RPI

HLS on

...

RPI (UVE and S/W Demux)

AAMP manages the pipeline. For audio and video there are separate playbins. AAMP provides audio/video elementary stream to the playbins using appsrc. Playbin connects platform decoder and sink elements using the capability of the  appsrc src pads.

...

Image Removed

HLS on

...

RPI (UVE, H/W Demux)

AAMP manages the pipeline. AAMP provides  TS  to the playbins using appsrc. Playbin connects platform ts-demux, decoder and sink elements.

Pipeline graph of HLS playback :

Image RemovedImage Added

DASH Widevine on

...

RPI(UVE)

AAMP manages the pipeline. For audio and video there are separate playbins. AAMP provides audio/video fragmented mp4 buffers to the playbins using appsrc. Playbin connects qtdemux, platform decoder and sink elements using the capability of the src pads.

...

If content is encrypted with Widevine or Playready DRM, aamp decryptor elements )aampplayreadydecryptor or aampwidevinedecryptor) are automatically plugged in after qtdemux by playbin. These elements decrypt samples using opencdm interface.

Pipeline graph of DASH stream:

Image RemovedImage Added

HTML5 Video Tag based playback on Broadcom Device

Here Webkit manages the pipeline. aampsrc is a dummy gstreamer source just to handle aamp:// protocol. aamp element fetches manifets/playlists and fragments, demuxes TS fragment in case of HLS, injects buffers to the downstream element using srcpads. RDK WPE uses Westeros sink gstreamer element as video sink.

Pipeline graph

Image Removed

Progressive Support in Aamp

...

In order to support progressive playback, aamp by default uses souphttpsrc as the source element instead of appsrc which is used for adaptive streams.Aamp decides from the url extension that the stream is progressive.Once this is confirmed, aamp would set the url to uri property of playbin.Aamp would then take its natural course and manage the pipeline.

Pipeline Graph (used by default)

Image Removed

Aamp can however choose appsrc over souphttpsrc by toggling a runtime configuration switch termed as "useAppSrcForProgressivePlayback". This switch gets enabled when the keyword "appSrcForProgressivePlayback" is added in aamp.cfg of the box.

When "appSrcForProgressivePlayback" is enabled in aamp.cfg, aamp will choose appsrc as the source element and complete its pipeline.The pipeline graph then, will look like below.

Pipeline Graph

Image Removed

UVE playback on Intel STB

Playback is similar to Broadcom platform, but uses Intel ismd gstreamer plugins to decode/render. For DASH normal playback,  audio and video are re-stamped since start-time  of segment events are not properly supported in Intel platform.

Pipeline graph

Image Removed

HTML5 Video Tag based playback on Intel STB

Playback is similar to Broadcom platform, but uses Intel ismd gstreamer plugins to decode/render.

Pipeline graph

Image Removed

Gstreamer SOC element properties used by AAMP

AAMP Gstreamer player uses custom properties of SOC elements to achieve requirements.

Broadcom SOC element properties used by AAMP

...

Element

...

Property

...

Description

...

Intel gstreamer element properties used by AAMP

...

Element

...

Property

...

Description

...