Intro

The job of the BundleGenerator is to produce from a SoC-agnostic oci Application Container Image a so called "bundle" or also refered to as "bundle-image". This bundle-image contains those same Application binaries (*) but now also a container run config (see dobby / oci runtime-spec config).

That container run config contains a part which will be generic, same across all STB platforms (eg same security config/policy) but also contains a part that is STB platform specific part because it needs to mount bind SoC specific Gfx libs in that container.

Result is a run config and bundle that is specific for a STB platform. Moreover depending on what is inside the firmware image & version of the STB platform it might be compatible or not. (eg RDK6 vs RDK5)

To indicate the compatibility of a bundle with set or series of firmware versions a "dacBundleFirmwareCompatibilityKey" string is introduced and will be further explained below. End result is a bundle that is compatible with series of firmware versions for a specific STB platform.

In order to generate bundles of all applications for the various platforms we need to provide the bundlegenerator Cloud Component with set of STB template files, 2 files delivered as one .tgz, here after referred to as stbBundleTemplate.tgz,  that are specific for those STB platforms and guaranteed to work on specific set/series of its firmware versions.

These template files need to be delivered by CPE team to DAC cloud team in certain format/location and have specific naming convention in order for bundle generator to associate and generate bundles for the right STB platformName with a compatible firmware version set. 

The sections below will describe what is inside this stbBundleTemplate.tgz and what is the naming convention of it. 

Before going there it is good to understand what bundle generator expects as input parameters and provides as output.

Below if concrete format of a JIT-BG RabbitMQ message towards bundlegenerator to generate a bundle (taken from description in JIT-BG Interactions#6)JIT-BGtoBundleGenviaRabbitMQ  and code https://bitbucket.upc.biz/projects/SPARK/repos/bundle-generator-service/browse/service/config_dev.json )

# copied from https://github.com/LibertyGlobal/bundle-generator-service/blob/main/service/config_dev.json#L22

   "templates_archive_name": "{platformName}_{firmwareVersion}_dac_configs.tgz", 	# Name of specific Container config template it will using when generating the bundle. Will be generated for specific platform with specific dacBundleFirmwareCompatibilityKey
     #question : what if that specific filename is not present ?

   "message": {
        "uuid": "as_is:id",
        "platform": "as_is:platformName",
        "image_url": "as_is:ociImageUrl",			# location of oci image
        "app_metadata": "literal:",                 # input string will be empty for now, then parameter will not be taken into account (now app_metdata.json is included in DAC oci images itself, in / dir)
        "lib_match_mode": "literal:normal",			# TO DO : STEFAN TO EXPLAIN WHAT THIS DOES
        "output_filename": "format_string:{appId}-{appVersion}-{platformName}-{firmwareVersion}",  	# name pattern of bundle file it needs to generate. Note that with {firmwareVersion} we mean same value as {dacBundleFirmwareCompatibilityKey}
         "searchpath": "format_string:{bundle_store_dir}/{id}",  									# path dir location of where it will look for the dac config "templates_archive_name", Is then cloud component copying  defined above
        "outputdir": "or:encrypt|{nginx_store_dir}|{bundle_store_dir}",  							
        "outputdir": "or:encrypt|{nginx_store_dir}/{appId}/{appVersion}/{platformName}/{firmwareVersion}|{bundle_store_dir}/{appId}/{appVersion}/{platformName}/{firmwareVersion}",
				# read as if encrypt flag is true we use bundle_store_dir with creation of following subdirs {appId}/{appVersion}/{platformName}/{firmwareVersion} if Flase we use nginx_store_dir with same subdirs
        "createmountpoints": "bool:true"             //is always set to true in Liberty case?
    }
# Documentation of expected bundlegen parameters you can see here https://github.com/rdkcentral/BundleGen/blob/ca703a0e71b9c302f3579afe623475d085b0beb1/bundlegen/rabbitmq/README.md

(*) = same Application Binaries : Depending on how you configure the bundlegenerator it could also remove some binaries (and replace them with bind mounts from associated files on rootfs) to optimize size but this has big disadvantage that you will introduce much stricter dependencies of the bundles on specific firmware version, move away from self-sufficient & portabilty goal and in practice lose your independence of STB host rootfs/firmware version, need to recheck apps upon each firmware upgrade, we will not do that, is against scaling of binary compatible DAC & Appstore.

stbBundleTemplate.tgz

The CPE team is responsible for generating the stbBundleTemplate.tgz for the relevant platforms and naming them correctly according to naming convention explained in this section

And CPE team must deliver the stbBundleTemplate.tgz to DAC cloud and based on the naming convention the Cloud bundle generation service will associate the incoming bundle generation request with the right associated stbBundleTemplate.tgz to create the right bundle

stbBundleTemplate.tgz contains 2 files inside which are described in the following subsection

The naming convention of the stbBundleTemplate.tgz is {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz

see this naming convention, mapping and concrete example in code block below

 The naming pattern is
├── {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz  
│   ├── {platformName}.json
│   └── {platformName}_libs.json

During the DAC cloud development we did not define in {dacBundleFirmwareCompatibilityKey} yet, there you will see it as {firmwareVersion} in their documentation/configs but they mean the same, so value will be same.
├── {platformName}-{firmwareVersion}_dac_configs.tgz  
│   ├── {platformName}.json
│   └── {platformName}_libs.json

Concrete examples

├── 2008C-STB_0.1.1-38e90b375c587cd2a378f7c04e1016f9b33ad2f3-dbg_dac_configs.tgz
│   ├── 2008C-STB.json
│   └── 2008C-STB_libs.json
├── VIP7002W_0.1.1-53c2928ebfa3011a18f572e2d8e7a56e03cbed02-dbg_dac_configs.tgz
│   ├── VIP7002W.json
│   └── VIP7002W_libs.json

Below you can find the definition and further naming convention of {platformName} and {dacBundleFirmwareCompatibilityKey}

platformName

dacBundleFirmwareCompatibilityKey

To indicate the compatibility of a bundle with set or series of firmware versions of a given platform a dacBundleFirmwareCompatibilityKey key/value is introduced.

When requesting to generate or download a bundle this identifier must be provided as input parameter in order to generate or return the right bundle.

The value is a concatenation of 3 strings, defined here below as  "stringA-stringB-stringC" with 

• stringA standing for  RDK DAC_CONTRACT_VERSION , exact definition of that is still be determined together with RDK but foreseeing here room for major.minor.revision numbering. Example value of stringA = "0.1.1" 
• stringB is the string that will be the same for set of firmware versions for which this bundle will be compatible with. Also if we change the container run time config (eg different policy) for the bundle, that string will change.  Technically it is hash of the 2 files in STB template : sha1(platform_config.json + platform_libs.json)  The file platform_libs.json is generated from firmware image and contains version set of the relevant library dependency tree. This is expected to remain the same for set of subsequent firmware revisions. When it does change from a particular firmware version, this will result in new hash and updated template files needs to be made by CPE team and provided to DAC cloud team. STBs upgrading to this new firmware version will also need to remove already stored app bundles and download/replace them with the new bundles associated with the updated firmware hash in order to guarantee that Apps will work. Example value of stringB =  "406e1982d1a9ce6359e5a11bb08502b63d51ae47"
• stringC stands for dbg, rel or prod software version, Example value of stringC = "dbg"   

concatenated the above examples results in value of dacPlatformFWCompatibleBundleVersion =  "0.1.0-406e1982d1a9ce6359e5a11bb08502b63d51ae47-dbg"

Flow of generation/updating dacBundleFirmwareCompatibilityKey 

Generation of this dacPlatformFWCompatibleBundleVersion is being done manually by CPE team using hash script https://github.com/stagingrdkm/lgpriv/blob/1deb05c8a5a1c3d3e78a00f8fc47f8e2844b8cf8/lg_platform_templates/oe_3.1/calc_hashes.sh on stbDacContainerConfig and stbDacLibs files, latter file is generated together the firmware image creation. 

CPE team needs to then create associated stbBundleTemplate.tgz ( {platformName}-{dacBundleFirmwareCompatibilityKey}_dac_configs.tgz )  and distributed to DAC cloud team for them to add to bundlegenerator service. For now this creation and distribution is manual job by CPE team for each platform. Later on we can look at automating this with jenkins.

retrieve dacBundleFirmwareCompatibilityKey

ResidentApp can get to know the right "dacBundleFirmwareCompatibilityKey" for the firmware version and platform it is running on via LISA  (on condition it is configured in LISA as part of its LISA thunder config).

TODO explain how to retrieve

git location of STB templates

At the moment (in this startup phase) the templates are stored in https://github.com/stagingrdkm/lgpub/tree/master/dac/templates

( in future we might move to other git location, stagingrdkm is not ideal long term place for this)