1. Intro

The job of the BundleGenerator is to produce from a SoC-agnostic oci Application Container Image a so called "bundle" or also refered to as "bundle-image". This bundle-image contains those same Application binaries (*) but now also a container run config (see dobby / oci runtime-spec config).

That container run config has sections which will be generic, same across all STB platforms (eg same security config/policy) but also has sections that are STB platform specific part because it must add in there instructions to mount bind into the AppContainer, the named SoC specific .so Graphics libraries providing (and symlinked to) libGlESv2.so.2 libEGL.so.1 and their associated full dependency tree, also associated required /dev devices need to be in container config file or container will fail to execute.

Result is a run config that is specific for a STB platform and as consequence the associated bundle tgz, is specific for a STB platform because despite binary compatible rootfs, inside it contains this run config.json which is platform specific.

Moreover depending on what is inside the firmware image & version of the STB platform it might be compatible or not. (eg RDK6 vs RDK5)

To indicate the compatibility of a bundle with set or series of firmware versions a "dacBundleFirmwareCompatibilityKey" string is introduced and will be further explained below. End result is a bundle that is compatible with series of firmware versions for a specific STB platform.

In order to generate bundles of all applications for the various platforms we need to provide the bundlegenerator Cloud Component with set of STB template files, 2 files delivered as one .tgz, here after referred to as stbBundleTemplate.tgz,  that are specific for those STB platforms and guaranteed to work on specific set/series of its firmware versions.

These template files need to be delivered by CPE platform teams (responsible for accepting vendor layer & generating firmware image) to DAC cloud team in certain format/location and have specific naming convention in order for bundle generator to associate and generate bundles for the right STB platformName with a compatible firmware version set. 

The sections below will describe what is inside this stbBundleTemplate.tgz and what is the naming convention of it. 

Before going there it is good to understand what bundle generator expects as input parameters and provides as output.

Below if concrete format of a JIT-BG RabbitMQ message towards bundlegenerator to generate a bundle (taken from description in JIT-BG Interactions#6)JIT-BGtoBundleGenviaRabbitMQ  and code https://bitbucket.upc.biz/projects/SPARK/repos/bundle-generator-service/browse/service/config_dev.json )

# copied from https://github.com/LibertyGlobal/bundle-generator-service/blob/main/service/config_dev.json#L22

   "templates_archive_name": "{platformName}_{firmwareVersion}_dac_configs.tgz", 	# Name of specific Container config template it will using when generating the bundle. Will be generated for specific platform with specific dacBundleFirmwareCompatibilityKey
     #question : what if that specific filename is not present ?

   "message": {
        "uuid": "as_is:id",
        "platform": "as_is:platformName",
        "image_url": "as_is:ociImageUrl",			# location of oci image
        "app_metadata": "literal:",                 # input string will be empty for now, then parameter will not be taken into account (now app_metdata.json is included in DAC oci images itself, in / dir)
        "lib_match_mode": "literal:normal",			# TO DO : STEFAN TO EXPLAIN WHAT THIS DOES
        "output_filename": "format_string:{appId}-{appVersion}-{platformName}-{firmwareVersion}",  	# name pattern of bundle file it needs to generate. Note that with {firmwareVersion} we mean same value as {dacBundleFirmwareCompatibilityKey}
         "searchpath": "format_string:{bundle_store_dir}/{id}",  									# path dir location of where it will look for the dac config "templates_archive_name", Is then cloud component copying  defined above
        "outputdir": "or:encrypt|{nginx_store_dir}|{bundle_store_dir}",  							
        "outputdir": "or:encrypt|{nginx_store_dir}/{appId}/{appVersion}/{platformName}/{firmwareVersion}|{bundle_store_dir}/{appId}/{appVersion}/{platformName}/{firmwareVersion}",
				# read as if encrypt flag is true we use bundle_store_dir with creation of following subdirs {appId}/{appVersion}/{platformName}/{firmwareVersion} if Flase we use nginx_store_dir with same subdirs
        "createmountpoints": "bool:true"             //is always set to true in Liberty case?
    }
# Documentation of expected bundlegen parameters you can see here https://github.com/rdkcentral/BundleGen/blob/ca703a0e71b9c302f3579afe623475d085b0beb1/bundlegen/rabbitmq/README.md

(*) = same Application Binaries : Depending on how you configure the bundlegenerator it could also remove some binaries (and replace them with bind mounts from associated files on rootfs) to optimize size but this has big disadvantage that you will introduce much stricter dependencies of the bundles on specific firmware version, move away from self-sufficient & portabilty goal and in practice lose your independence of STB host rootfs/firmware version, need to recheck apps upon each firmware upgrade, we will not do that, is against scaling of binary compatible DAC & Appstore.


2. VA bundle template - stbBundleTemplate.tgz - {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz


The CPE team is responsible for generating the stbBundleTemplate.tgz for the relevant platforms and naming them correctly according to naming convention explained in this section

And CPE team must deliver the stbBundleTemplate.tgz to DAC cloud and based on the naming convention the Cloud bundle generation service will associate the incoming bundle generation request with the right associated stbBundleTemplate.tgz to create the right bundle

stbBundleTemplate.tgz contains 2 files inside which are described in the following subsection

The naming convention of the stbBundleTemplate.tgz is {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz

see this naming convention, mapping and concrete example in code block below

STBtemplate 
 The naming pattern is
├── {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz  
│   ├── {platformName}.json
│   └── {platformName}_libs.json

During the DAC cloud development we did not define in {dacBundleFirmwareCompatibilityKey} yet, there you will see it as {firmwareVersion} in their documentation/configs but they mean the same, so value will be same.
├── {platformName}-{firmwareVersion}_dac_configs.tgz  
│   ├── {platformName}.json
│   └── {platformName}_libs.json

Concrete examples

├── 2008C-STB_0.1.1-38e90b375c587cd2a378f7c04e1016f9b33ad2f3-dbg_dac_configs.tgz
│   ├── 2008C-STB.json
│   └── 2008C-STB_libs.json
├── VIP7002W_0.1.1-53c2928ebfa3011a18f572e2d8e7a56e03cbed02-dbg_dac_configs.tgz
│   ├── VIP7002W.json
│   └── VIP7002W_libs.json

Below you can find the definition and further naming convention of {platformName} and {dacBundleFirmwareCompatibilityKey}

3. platformName

Since bundles are CPE model/platform specific we need to define the names of these models/platforms in DAC cloud platform/ASMS and use the appropriate platformName when we want to get such bundle.

The platformNames for the VA are in file names of templates.tgz, see are https://github.com/stagingrdkm/lgpub/tree/master/dac/templates/oe_3.1

The platformName of DAC cloud does not necessary correspond one to one to RDK model name in the firmware of particular model.  Hence you might need to use mapping table. we created such mapping table in config of thunder lisa service

rdk6.1:  https://code.rdkcentral.com/r/plugins/gitiles/components/generic/rdk-oe/meta-cmf-video/+/refs/heads/rdk-next/recipes-extended/rdkservices/rdkservice-lisa-dac-config.inc

rdk-e: https://github.com/rdkcentral/meta-rdk-video/blob/develop/recipes-extended/entservices/include/entservices-lisa-dac-config.inc#L3

The STB can based on this config, retrieve platformName to be used towards DAC cloud by retrieving the "dacBundlePlatformNameOverride" value from LISA#lisa.dac.confg

4. dacBundleFirmwareCompatibilityKey

To indicate the compatibility of a bundle with set or series of firmware versions of a given platform a dacBundleFirmwareCompatibilityKey key/value is introduced.

When requesting to generate or download a bundle this identifier must be provided as input parameter in order to generate or return the right bundle.

The value is a concatenation of 3 strings, defined here below as  "stringA-stringB-stringC" with 

  • stringA standing for  RDK DAC_CONTRACT_VERSION , exact definition of that is still be determined together with RDK but foreseeing here room for major.minor.revision numbering. Example value of stringA = "0.1.1" 
  • stringB is the string that will be the same for set of firmware versions for which this bundle will be compatible with. Also if we change the container run time config (eg different policy) for the bundle, that string will change.  Technically it is hash of the 2 files in STB template : sha1(platform_config.json + platform_libs.json)  The file platform_libs.json is generated from firmware image and contains version set of the relevant library dependency tree. This is expected to remain the same for set of subsequent firmware revisions. When it does change from a particular firmware version, this will result in new hash and updated template files needs to be made by CPE team and provided to DAC cloud team. STBs upgrading to this new firmware version will also need to remove already stored app bundles and download/replace them with the new bundles associated with the updated firmware hash in order to guarantee that Apps will work. Example value of stringB =  "406e1982d1a9ce6359e5a11bb08502b63d51ae47"
  • stringC stands for dbg, rel or prod software version, Example value of stringC = "dbg"   

concatenated the above examples results in value of dacPlatformFWCompatibleBundleVersion =  "0.1.0-406e1982d1a9ce6359e5a11bb08502b63d51ae47-dbg"

The STB can retrieve the "dacBundleFirmwareCompatibilityKey" configured in its firmware version from LISA. See use getMetadata {"type":"application/LISA", "id":"lisa.dac.confg", "version":"0"} in LISA#lisa.dac.confg

and default list https://github.com/rdkcentral/meta-rdk-video/blob/develop/recipes-extended/entservices/include/entservices-lisa-dac-config.inc

4.1.1. Flow of generation/updating dacBundleFirmwareCompatibilityKey 

Generation of new  dacPlatformFWCompatibleBundleVersion and associated stbBundleTemplate.tgz ( {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz )  has to be done manually by CPE platform team (vendor layer depedent), you can use script https://github.com/stagingrdkm/lgpub/blob/master/dac/templates/oe_3.1/create_tarballs.sh on stbDacContainerConfig and stbDacLibs files, latter file is generated together the firmware image creation. 

CPE team needs to inform  and distributed that there is new stbBundleTemplate.tgz version to DAC cloud team for them to add to bundlegenerator service. For now this creation and distribution is manual job by CPE team for each platform. Later on we can look at automating this with jenkins.

4.1.2. retrieve dacBundleFirmwareCompatibilityKey

ResidentApp/STB can retrieve the "dacBundleFirmwareCompatibilityKey" configured in its firmware version using LISA API getMetadata {"type":"application/LISA", "id":"lisa.dac.confg", "version":"0"} see info in LISA#lisa.dac.confg

(on condition it is configured in LISA as part of its LISA thunder config).

4.2. git location of STB templates

At the moment (in this startup phase) the templates are stored in https://github.com/stagingrdkm/lgpub/tree/master/dac/templates

( in future we might move to other git location, stagingrdkm is not ideal long term place for this)


5. Process of Updating & Releasing VA bundle templates

1) With every SoC driver update and Major RDK upgrade you need to carefully analyse full dependency tree in vendor layer behind libraries providing libgles2.so & libEGL.so, libessos.so both full .so dependency tree and required drivers (/dev/ ) because these can have changed (slightly) and thus require new VA bundle template update to be able to generate correct runtime config. If one dependent library or driver is missing in container run config file, the loading/running of the Appcontainer will fail.

Each platform VA platform team needs to create following files & template pattern and push to https://github.com/stagingrdkm/lgpub/tree/master/dac/templates  use oe_4.0 for RDK-E (as it is kirkstone).

3 files, of which tgz is versioned tgz of 2 files naming pattern of bundle template is and it consists out of these 2 json files
├── {platformName}.json
├── {platformName}_libs.json
├── {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz   // this tgz is 2 above files tgz together with version naming (dacBundleFirmwareCompatiblityKey) in file name applied.

concrete example from https://github.com/stagingrdkm/lgpub/tree/master/dac/templates/oe_4.0

├──ah212.json
├──ah212_1.0.0-15cd0290c4412eb17d1888767806b69d25d66eb8-dbg_dac_configs.tgz
├──ah212_libs.json
The {platformName}_libs.json file you can get as build artifact file when building the target platform firmware / software image image. 

There is a bbclass for this here: https://code.rdkcentral.com/r/plugins/gitiles/rdk/components/generic/rdk-oe/meta-cmf/+/refs/heads/rdk-next/classes/generate_libs_json.bbclass   TO DO on RDK-E meta-cmf layer is no longer included by default, NEED TO FIX so it's available by default. 

The bbclass uses readelf to generate this information. You can enable this generation by adding the following to your image target recipe: inherit generate_libs_json. The result can be found in file $MACHINE_libs.json inside your tmp/deploy dir.

The bundlegenerator requires this file to find the platform specific dynamic library dependency tree behind  the libraries in "[Gfx]" section of the {platformName}.json template, typically libEGL.so libGLESv2.so libessos.so , see link to concrete example of such gfx section 

see also information in https://github.com/rdkcentral/BundleGen/blob/6b0a728686e5b581875cb03bec2b44888e79b540/docs/TemplateLibs.md#generating-a-libs-json-file  Whenever this platform specific Graphics dependency tree is changing (change with new driver release), new "dacBundleFirmwareCompatibilityKey" need to be created with the updated  {platformName}_libs.json


The {platformName}.json you need to create as platform team yourself and that requires careful analysis of platform specific drivers dependencies that are needed to be mounted into the container in order for application within that container to be able to do any openGLES2 / EGL graphics. You should look at examples of other platforms. use tools to check deps gfx drivers, and add them in that {platformName}.json which is template for container config, test with local bundle generator of simple wayland-egl-test examples that it works.

More information on {platformName}.json format is here  https://github.com/rdkcentral/BundleGen/blob/6b0a728686e5b581875cb03bec2b44888e79b540/docs/Templates.md

2) If it's working fine you need to generate new official dacbundleFirmwareCompatibilitykey and create {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz see https://github.com/stagingrdkm/lgpub/blob/master/dac/templates/oe_3.1/create_tarballs.sh and do Pull Request with that tarball and files to https://github.com/stagingrdkm/lgpub/tree/master/dac/templates

3) Then ask DAC Cloud team (RDK-M DAC cloud support team (dac-info@consult.red)) to upload that new template ({platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz to DAC cloud bundle generator so it can autogenerates bundles with this container config for boxes requesting it for this dacBundleFirmwareCompatibilityKey. Or if you have access to RDK DAC cloud Admin UI ( https://ui.rdkm.consult-red.net ) you can upload ({platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz yourself via UI, see "Platforms" section on left, press "+Add" button,  "Upload Configuration" button. see screenshot. You can tag that version in admin UI with Name field, typically we use {platformName} with optionally suffix that points to specific version/reason of change (eg driver or RDK7 version update of platform firmware)

4) You need to change to LISA config to point to new dacBundleFirmwareCompatibilityKey for that particular platform, see https://github.com/rdkcentral/meta-rdk-video/blob/develop/recipes-extended/entservices/include/entservices-lisa-dac-config.inc#L3
so that resident UI will use that new dacBundleFirmwareCompatibilityKey to install (first uninstall) new DAC apps. Do Pull Request for that so everyone will have that. Then you can be sure these will be bundles installed with the right dacBundleFirmwareCompatibilityKey version, that is compatible with firmware of the device it is running on.

# set correct mapping of machine name in build to "dacBundlePlatformNameOverride" and configure the appropriate default, out of the box, "dacBundleFirmwareCompatibilityKey" for this PlaformName/model.
# Note that this empty for few Platforms in example below. That is not good. It won't work out of the box yet with UI for those boxes because UI will not be able to know for what "dacBundleFirmwareCompatibilityKey" version to ask the bundle for. These still need to be added by platform team.

from within  https://github.com/rdkcentral/meta-rdk-video/blob/develop/recipes-extended/entservices/include/entservices-lisa-dac-config.inc#L3
machine_config_map_kirkstone = {
        'raspberrypi-': ('rpi3', ''),
        'raspberrypi4': ('rpi4', '1.0.0-b34e9a38a2675d4cd02cf89f7fc72874a4c99eb0-dbg'),
        'brcm972180hbc': ('7218c', ''),
        'brcm972126ott': ('72126ott', '1.0.0-6a60471939a0ea09e5568c2f6fbd672c9406eb0a-dbg'),
        'brcm972127ott': ('72127ott', ''),
        'ah212': ('ah212', '1.0.0-15cd0290c4412eb17d1888767806b69d25d66eb8-dbg'),
        'at301': ('ah212', '1.0.0-15cd0290c4412eb17d1888767806b69d25d66eb8-dbg'),
        'hp44h': ('ah212', '1.0.0-15cd0290c4412eb17d1888767806b69d25d66eb8-dbg'),
        'ap222': ('ap222', ''),
        'mediabox': ('rtd1319', '1.0.0-f9dbb0a5f4b6814f298740ddd148b3b56e3eafed-dbg'),
        'blade': ('rtd1319', '1.0.0-f9dbb0a5f4b6814f298740ddd148b3b56e3eafed-dbg'),
        'm393': ('m393', '')








  • No labels

DISCLAIMER: Please note that the use of the RDK Wiki is subject to its Privacy Policy & Terms of Use. In addition, this Wiki may be accessed by all RDK licensees and their contractors.