RDK Resources

[*RDK Preferred*]

Code Management Facility

Code Releases

RDK Forums

[RDK Conferences]

RDK Support

Archives

Papers & Presentations Archive

In the News!

Skip to end of metadata
Go to start of metadata

Introduction

This manual describes the rdk-generic-mediaclient-image build procedure for Yocto 3.1 builds. This will upgrade the default build system used in RDK Video, from morty(2.2) to dunfell(3.1). In addition to upgrades to opensource packages, Yocto 3.1 also contain Long term Support release feature. The current version lacks the variety of features available in the dunfell. The upgraded build tools will be used to generate rdk-generic-mc-image for Raspberrypi reference platform. 

Yocto 3.1 Upgradation support the following:

  • Yocto BSP layer for meta-raspberrypi.
  • OpenEmbedded and Yocto Dunfell.
  • Linux kernel 4.19.
  • Version upgrades for bitbake, gstreamer and other oe/wpe recipes.
  • Playing a given stream using gst-launch command.
  • rmfApp Support.
  • Controlling apps using appmanager.

Each component in RDK is a standalone repository with its own individual build tools producing a library or set of binaries. When we upgrade the OE layers to the newer versions, we need to make necessary changes in the RDK layers which use these components, to avoid build failures.

Host Setup

The OpenEmbedded build system should be able to run on Ubuntu 18.04 distribution/other compatible linux distribution with the following versions for Git, tar, and Python.

  • Git 1.8.3.1 or greater
  • tar 1.27 or greater
  • Python 3.4.0 or greater
  • Coreutils(E.g realpath)

Note: You should also have about 50 Gbytes of free disk space for building images.

The essential packages you need for a supported Ubuntu or Debian distribution are shown in the following command:

$ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib \
  build-essential chrpath socat cpio python python3 python3-pip python3-pexpect \
  xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev \
  pylint3 xterm bmap-tools

Repo Setup

In order to use Yocto build system, the repo tool must be properly installed on the machine.

To install Repo make sure you have a /bin directory in your home directory and that it is included in your path

Repo Setup Steps
$ mkdir ~/bin
$ PATH=~/bin:$PATH
Download the repo tool and ensure that it is executable
$ curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
$ chmod a+x ~/bin/repo

Note: it is also recommended to put credentials in ~/.netrc when interacting with repo.

A sample ~/.netrc file is illustrated below

.netrc
machine code.rdkcentral.com
login YOUR_USERNAME
password YOUR_PASSWORD

Executing System

To tackle the build issues from moving one version to another, we have decided to split the problem into different subtasks. So instead of building a complete image at one go, we have planned for the generation of following images:

  • core-minimal-image for Raspberrypi
  • wpe-westeros-image
  • rdk-generic-image for hybrid and mediaclient
  • rdk-generic-wpe-image for hybrid and mediaclient

These images need to be tested against different configurations in cmf, to ensure that nothing has broken the existing system.

In order to provide a smooth transition without disturbing the existing branch, we have created a branch called `yocto-dunfell-upgrade` for non-OE layers and another branch called `rdk/yocto-dunfell-upgrade` for OE layers.

Right now we are in the process of resolving build/parse errors. The main source for errors can be classified into 4 categories.

--> Incompatible license with RDK.
--> Trying to apply older patches into newer version of recipes.
--> Recipe version changes due to dependency.
--> If there is a license upgrade to a new recipe which is not compatible with RDK terms then we need to stick to the older version.

Going forward no more RDK specific BSP layer(meta-cmf-raspberrypi for instance) will be available and RDK specific changes will be brought under dynamic layer in SoC/OEM layer. Hence the important recipes and append files will be kept in the SoC/OEM layer in future.

Some of the build errors and its root cause is described in the below table. We will keep updating the table as we encounter more errors.


Oops, it seems that you need to place a table or a macro generating a table within the Table Filter macro.

The table is being loaded. Please wait for a bit ...

recipelayererror/warningroot causeaction
N/AN/ALayer should set LAYERSERIES_COMPAT in its conf/layer.conf file to list the core layer names it is compatible with.Newer version of bitbake throws warning if we don’t set LAYERSERIES_COMPAT in its conf/layer.conf in the respective layers.

Set LAYERSERIES_COMPAT_<layer name> = "dunfell" in all layers.

E.g LAYERSERIES_COMPAT_rdk = "dunfell" for meta-rdk layer.

N/Ameta-rdk-extbb.data_smart.ExpansionError: Failure expanding variable AVAILABLE_LICENSES[:=], expression was ${@' '.join(available_licenses(d))} which triggered exception FileNotFoundError: [Errno 2] No such file or directory.There is no directory called licenses in the corresponding meta layer.Need to create a licenses directory under the meta-layer and add RDK license file to it
N/A

meta-rdk-restricted

meta-rdk-video

meta-rdk-ext

meta-rdk-containers

meta-rdk

Error due to the usage of base_contains in multiple recipes.base_contains is deprecated.use bb.utils.contains instead of deprecated base_contains in the respective recipes.
N/AN/AError due to the usage of some of the functions in pythonnative.bbclass in multiple recipes.pythonnative.bbclass is not available in newer version.Add meta-python2 layer to get support for legacy pyhton code(E.g pythonnative.bbclass)

openssl.inc

openssl-0.9.inc

image_container_generator.bbclass

rdk.conf

meta-rdk

meta-rdk-ext

Error due to the usage of base_conditional in multiple recipes.base_conditional is deprecated.use oe.utils.conditional instead of deprecated base_conditional in the respective recipes.
uclibc.incmeta-rdk-extError due to the usage of oe_filter_out in multiple recipes.oe_filter_out is deprecated.use oe.utils.str_filter_out instead of deprecated oe_filter_out in the respective recipes.
gssdp_0.14.10.bbmeta-rdk-extCould not include required file recipes-connectivity/gupnp/gssdp.inc in gssdp_0.14.10.bbThe required file is not present in the location.Replace "require" directive to "include" to specify a loose dependency.

rdk-generic-broadband-dev-image.bb

rdk-generic-hybrid-dev-image.bb

rdk-generic-mediaclient-dev-image.bb

meta-rdkErrors due to required files not being present in yocto 3.1(dunfell) oe layers.Those file are not available in newer version of Yocto.Replace "require" directive to "include" to specify a loose dependency.

rmfhalheaders_git.bb


meta-rdk-videoError while parsing LIC_FILES_CHKSUM in rmfhalheadersThe recipe uses "files://" instead "file://" in LIC_FILES_CHKSUM fieldUpdate the LIC_FILES_CHKSUM field accordingly.
busybox_%.bbappendmeta-rdk-extError in busybox as busybox_1.31.1.bb:do_patch: failed with exit code '1'Build error  in do_package task due to some of the older patches availableThe following patches are removed from busybox recipe
for dunfell build
1. udhcp.patch
2. 0001-networking-add-ip-neigh-command.patch
3. ip6_neigh_show_
Crash.patch

glib-networking_2.54.1.bb

rdk.conf

meta-rdk-ext

meta-rdk

Glib-networking require gnutls >= 3.4.6 gnutls 3.3.x is the last branch which supports.nettle 2.7.1. So it is not possible to increase the version,Used glib-networking-2.54.1 instead of updating gnutls.
gstreamer1.0-plugins-bad.incmeta-rdk-extError while inheriting bluetooth.bbclassbluetooth.bbclass is not available in newer version of yocto

inherit bluetooth on a conditional basis as shown below:

inherit gettext ${@bb.utils.contains('DISTRO_CODENAME', 'dunfell', '', 'bluetooth', d)}

glib-2.0_2.62%.bbappendmeta-rdk-extError reported for glib-2.0-native-1_2.62.4-r0 patchUnable to apply the 2nd hunk due to version changes. rebased RDK patch to glib 2.62.4
openssl_1.1.0g.bbmeta-rdk-extError while fetching openssl_1.1.0g Recipe version 1.1.0 is moved to a different location. SRC_URI updated with the new changes.
systemd_216.bbmeta-rdk-extError while parsing systemd_216 legacy recipe  Circular dependency problem with systemd_216.

Skip this recipe by specifying the COMPATIBLE_HOST in bb file as shown below:

COMPATIBLE_HOST_dunfell = "null"


netsrvmgr_git.bbappend

meta-cmf-videoError due to invalid version of netsrvmgr.SRCREV expects a valid revision on dunfell framework.SRCREV for netsrvmgr set to AUTOREV
iptables_%.bbappendmeta-rdk-extError while installing iptablessysint provides it's own iptables systemd service file which gives a conflict on do_rootfs()

Remove systemd service in the bbappend file to avoid conflict as shown below:

do_install_append_client() {
rm ${D}${systemd_system_unitdir}/${PN}.service
}

SYSTEMD_SERVICE_${PN}_remove_client = "${PN}.service"

dropbear_%.bbappend


meta-rdk-extPatch error while building dropbear-2019.78Older version of patches were giving build error.Rebase patches to build on dunfell.
openssl_1.0.2o.bbmeta-rdk-extCircular dependency problems with openssl 1.0.2o for dunfell builds.Higher versions of the recipes are available for newer builds.

Skip this recipe by specifying the COMPATIBLE_HOST in bb file as shown below:

COMPATIBLE_HOST_dunfell = "null"

setup-environmentmeta-rdkdunfell codename not reflected in DISTRO_CODENAME.Bitbake version is not updated for dunfell branch.

Update DISTRO_CODENAME based on the bitbake version in setip-environment script

E.g:-

1.44.*|1.46.*) _DISTRO_CODENAME="dunfell"

RDKmeta-cmfExpansionError: Failure expanding variable AVAILABLE_LICENSES[:=], expression was ${@' '.join(available_licenses(d))} which triggered exception FileNotFoundError: [Errno 2] No such file or directory:RDK licence file not present in the preferred location.Add RDK licence file to the licenses directory in the corresponding layer.
qtbase-native_5.1.1.bbappendmeta-cmf-qt5qtbase-native: fix build error due to gcc 9.xqtbase native pkg throws build error due to gcc 9.2.

The following patch file is available in the repo to resolve the issue:

0001-RDK-27410-qtbase-native-build-error-due-to-gcc-versi.patch

We need to conditionally enable this for dunfell builds. 

logrotate_%.bbappendmeta-rdk-extSysint trying to install logrotate.service, logrotate.timer but it is already installed by logrotate.as sysint provides logrotate's systemd service, actual recipe's service gives conflict

As sysint installs RDK specific logrotate's systemd service file, we can remove them as shown below:


SYSTEMD_SERVICE_${PN}_remove = " \
${BPN}.service \
${BPN}.timer \
"

do_install_append() {
rm ${D}${systemd_system_unitdir}/${BPN}.service
rm ${D}${systemd_system_unitdir}/${BPN}.timer
}

dnsmasq_%.bbappendmeta-rdk-ext Postinstall scriptlets of ['systemd'] failed.As systemd in dunfell provides resolv.conf, no need to install again from dnsmasq.Removed resolv.conf from do_install for dunfell

qtbase_5.1.1.bbappend


meta-cmf-qt5Declaration conflict for qopengl's GLdouble because of this qtbase throws build error.GLdouble is typedefed as GLFloat in qopengl.h this leads to a conflict with the earlier definition of GLdouble as double.We can typedef GLdouble to double.
msgpack-c_2.1.1.bbmeta-rdk-extError about 'copying an object of non-trivial type' in msgpack-c.Typecasting is missing while using memcpy().

Insteadof using 

std::memcpy(&o, &v, sizeof(v)), we can typecast the object as shown below

std::memcpy(static_cast<void*>(&o), &v, sizeof(v))

wpe-webkit.incmeta-rdk-extdunfell build throws do_package_qa error as libWPEWebKit.so requires libTTSClient.soUnable to satisfy runtime dependency libTTSClient.soAdd tts package in the RDEPENDS for wpe-webkit 
ledmgr-extended-noop_git.bbmeta-rdk-videofatal error: sysMgr.h: No such file or directorysysMgr.h file not avalable in the sysroot directory.Add iarmmgrs in the DEPENDS of meta-rdk-video/recipes-extended/ledmgr/ledmgr-extended-noop_git.bb
aamp_git.bbmeta-rdk-videoirMgr.h not foundirMgr.h file not avalable in the sysroot directory.

Add iarmmgrs in the DEPENDS of

meta-cmf-video/recipes-extended/aamp/aamp_git.bbappend

wrp-c_1.0.bbmeta-rdk-ext
error: 'strncpy' output truncated before terminating nul 
copying 23 bytes from a string of the same length.
Use of strncpy for coping.Replace strncpy with memcpy.
ledmgr_git.bbmeta-rdk-videoyocto dunfell framework throws do_package_qa() filedeps error while building the packages.required runtime dependencies are missing.

Add devicesettings in RDEPENDS of 

meta-rdk-video/recipes-extended/ledmgr/ledmgr_git.bb

tr69hostif_git.bbmeta-rdk-videoyocto dunfell framework throws do_package_qa() filedeps error while building the packages.required runtime dependencies are missing.

Add devicesettings in RDEPENDS of 

meta-rdk-video/recipes-thirdparty/tr69/tr69hostif_git.bb

bluetooth-core      N/Afatal error: bluetooth/audio/a2dp-codecs.h: No such file or directorya2dp codec macros are changed in newer bluez version (5.54) from oe-core dunfell branch.

We can add definitions for MIN_BITPOOL and MAX_BITPOOL in btrCore_avMedia.c as shown below

#ifndef MIN_BITPOOL
#define MIN_BITPOOL SBC_MIN_BITPOOL
#endif
#ifndef MAX_BITPOOL
#define MAX_BITPOOL SBC_MAX_BITPOOL
#endif

bluetooth_mgr      N/Aundefined reference error for bluetooth-mgrbluetooth-mgr main.c calls sd_notify() API but not linked libsystemd.so which is causing undefined reference errorUpdate AM_LDFLAGS with -lsystemd to resolve undefined reference error.
bluetooth_mgr      N/Amemcpy issue for MediaElementListInfo.Improper structure size was mentioned in the memcpy.

Replace sizeof(BTRMGR_IARMMediaElementListInfo_t) with sizeof(BTRMGR_MediaElementListInfo_t) in the following function in btmgr_iarm_external_interface.c

memcpy (mediaElementListInfo, &mediaElementList.m_mediaTrackListInfo, sizeof(BTRMGR_IARMMediaElementListInfo_t)) 

bluetooth-core      N/A

Error due to incorrect to use of AdapterInfo struct.

appropriate structure is not mentioned in sizeof().

Replace sizeof(stBTDeviceInfo) in the following function with sizeof(stBTAdapterInfo) in btCore_dbus_bluez5.c

memset(&lstBTAdapterInfo, 0, sizeof(stBTDeviceInfo))

fog      N/AError when HLSPlaylistsDownload.ccp in fogcli calls 'SHA1' API-lcrypto not included in the MakefileUpdate fogcli_LDFLAGS in Makefile.am with -lcrypto.
sectionfilter-hal-noop_git.bbmeta-rdk-videoError due to missing dependencies in sectionfilter-haldunfell framework expects all possible dependencies to be mentioned in DEPENDS.Update sectionfilter-hal-noop_git.bb DEPENDS with jansson.
sys-utils_git.bbmeta-rdk-videopackage_qa error while building sys-utils_git.bbdunfell framework expects to mention all runtime deps to avoid package_qa error.Add devicesettings in the RDEPENDS field of sys-utils_git.bb
iarmmgrs_git.bbappendmeta-cmf-videopackage_qa error while building iarmmgrs_git.bbhybrid specific runtime deps overriding basic runtime deps which causes package_qa error.

Use append instead of simple assignment for the runtime dependency for iarmmgrs as shown below:

RDEPENDS_${PN}_append_hybrid = " virtual/mfrlib"

cairo_%.bbappendmeta-rdk-extConflict due to patches available in other layers while patching cairo.meta-wpe also maintains cairo patches that makes conflict while apply.

apply patches conditionally based on meta-wpe as shown below:

SRC_URI_append = " \
${@'file://cairo-egl-device-create-for-egl-surface.patch' if 'wpe-layer' not in d.getVar('BBFILE_COLLECTIONS').split() else ''} \
${@'file://0008-add-noaa-compositor.patch' if 'wpe-layer' not in d.getVar('BBFILE_COLLECTIONS').split() else ''} \
"

xupnp      N/A
error: too many arguments to function 'gupnp_context_new'.
Based on the gupnp version the we need to call gupnp_context_new and gupnp_root_device_new functions. 

Fix is available in the following commit:

REFPLTV-473

rmfgeneric_git.bbmeta-rdk-videormfgeneric throws do_package() error says dwarfsrcfiles exit with error code for librbi.a.This issue comes only when RBI is enabled (--enable-rbi) for hybrid.Issue canbe resolved by adding INHIBIT_PACKAGE_DEBUG_SPLIT_hybrid = "1" in rf=mfgeneric_git.bb

procps_%.bbappend

wpe-webkit.inc


meta-rdk-extBuild failure due to missing dependencies.dunfell build expects all possible dependencies in place.

Fix is available in the following commit:

39396 : REFPLTV-473

tr69agent      N/Aerror: storage size of 'ctx' isn't knownThe build is failing due to openssl 1.0 and 1.1 compatibility issues.

Fix is available in the following commit:

39371 : REFPLTV-473

tr69agent_git.bb

tr69hostif_git.bb

meta-rdk-video

configure: error: Package requirements (libproc >= 3.2.8) were not met:

No package 'libproc' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables PROCPS_CFLAGS
and PROCPS_LIBS to avoid the need to call pkg-config.

--enable-latest-procps OECONF option was only enabled in morty builds.

--enable-latest-procps OECONF option can be added to dunfell builds as well by editing the respective recipes as shown below:

EXTRA_OECONF_append_dunfell = " --enable-latest-procps"

tr69N/A--enable-morty name conflict while building tr69hostif from dunfell framework.--enable-morty name conflicts by enabling on dunfell builds.

Fix is available in the following commit:

39342 : REFPLTV-412

tr69hostifN/Aaclocal: error: couldn't open directory 'cfg': No such file or directorylatest autotools version expects m4 macro directory.We can fix this issue by replacing AC_CONFIG_MACRO_DIR([cfg]) with AC_CONFIG_MACRO_DIR([m4]) in configure.ac for tr69hostif.
bluetooth-coreN/A

building bluetooth-core on gcc 9.x throws below errors

-Werror=stringop-overflow=

-Werror=stringop-truncation

-Werror=format-truncation=

on below files

src/btrCore.c

src/bt-iface/btrCore_dbus_bluez.c

gcc 9.x enables more string operations flags that throw errors on compilation.

Fix is available in the following commit:

39257 : REFPLTV-412

bluetooth-coreN/ABuild failure due to incorrect structure definition.appropriate structure is not mentioned in sizeof() while memset-ting the same.

Replace sizeof(stBTDeviceInfo) with sizeof(stBTAdapterInfo) in the following function in btrCore_dbus_bluez5.c

memset(&lstBTAdapterInfo, 0, sizeof(stBTDeviceInfo))

power-state-monitorN/ABuild failure due to directfb header file inclusion while building power-state-monitor from dunfell framework.directfb depends is no more from recipe so, no need of directfb headers / API in the src.Remove #include <direct/list.h> from powerStateMonitorMain.c
dibbler_%.bbappendmeta-rdk-extDibbler issues: patch doesn't apply, couldn't install script file which resulted in packaging errors.patch already incorporated into src.

We can remove the following patch from the bbappend file for dunfell builds:

0001-DELIA-19972-Dibbler-client-crash-in-checkDecline.patch

mediaframeworkN/Aaclocal: error: couldn't open directory 'cfg': No such file or directorylatest autotools version expects m4 macro directory.We can fix this issue by replacing AC_CONFIG_MACRO_DIR([cfg]) with AC_CONFIG_MACRO_DIR([m4]) in configure.ac for mediaframework.

Currently, all the changes are being pushed in these branches. Once it is merged, you will be able to follow the build instructions without any error.

Yocto Build Steps

To build, follow below instructions

Build Steps
$ mkdir <workspace dir>
$ cd <workspace dir>
$ repo init -u https://code.rdkcentral.com/r/manifests -b yocto-dunfell-upgrade -m rdkv-asp-nosrc.xml
$ repo sync -j4 --no-clone-bundle
$ MACHINE=raspberrypi-rdk-mc source meta-cmf-raspberrypi/setup-environment
$ bitbake rdk-generic-mediaclient-image

 

Flashing Procedure

Following command can be used to flash the RPI image to sd card using linux machine . bmap tool should be available in linux

Flash command
bzip2 -d <path to ImageName.wic.bz2>
sudo -E bmaptool copy --nobmap <path to ImageName.wic> <path to SD card space>

Example:

$ bzip2 -d rdk-generic-mediaclient-image-raspberrypi-rdk-mc.wic.bz2
$ sudo -E bmaptool copy --nobmap rdk-generic-mediaclient-image-raspberrypi-rdk-mc.wic /dev/sdc

Known Issues and Limitations

  1. Certain services like dsmgr.service and netsrvmgr.service are crashing and as a result of this sysint is triggering a reboot. To avoid this issue we need to disable the above-mentioned services after a remote login using systemctl command.
  2. Audio is not working with gst-launch command
  3. rmfApp prompt is not getting displayed after playing video using launch/play command. So we need to hard close the video using Ctrl + C combination.
  4. appmanager application is exiting automatically even if it is running in background.
  5. pxscene browser is not launching.




  • No labels