Introduction

This manual describes  Yocto 3.1 (dunfell) build procedure for the Raspberry Pi Reference Platform. This is an upgrade from the default RDK build system, Yocto 2.2 (morty). In addition to upgrades to opensource packages, Yocto 3.1 also contains Long term Support release feature. The current RDK build system lacks the variety of features available in the dunfell. The upgraded build tools will be used to generate different images for the target.

Yocto 3.1 Upgradation support the following:

Each component in RDK is a standalone repository with its own individual build tools producing a library or set of binaries. When we upgrade the OE layers to the newer versions, we need to make necessary changes in the RDK Yocto meta layers which use these components, to avoid build failures.

Host Setup

How to Build#SettinguptheHostEnvironment

Executing System

Migration Efforts

To tackle the build issues from moving one version to another, we have decided to split the problem into different subtasks. So instead of building a complete image at one go, we have planned for the generation of the following images:

These images need to be tested against different configurations in cmf, to ensure that nothing has broken the existing system.

In order to provide a smooth transition without disturbing the existing branch, we have created a branch called `yocto-dunfell-upgrade` for non-OE layers and another branch called `rdk/yocto-dunfell-upgrade` for OE layers.

Right now we are in the process of resolving build/parse errors. The main source for errors can be classified into 4 categories.

--> Incompatible license with RDK.
--> Trying to apply older patches into newer version of recipes.
--> Recipe version changes due to dependency.
--> If there is a license upgrade to a new recipe which is not compatible with RDK terms then we need to stick to the older version.

Some of the build errors and its root cause is described in the below table. 


recipelayererror/warningroot causeaction
N/AN/ALayer should set LAYERSERIES_COMPAT in its conf/layer.conf file to list the core layer names it is compatible with.Newer version of bitbake throws warning if we don’t set LAYERSERIES_COMPAT in its conf/layer.conf in the respective layers.

Set LAYERSERIES_COMPAT_<layer name> = "dunfell" in all layers.

E.g LAYERSERIES_COMPAT_rdk = "dunfell" for meta-rdk layer.

N/Ameta-rdk-extbb.data_smart.ExpansionError: Failure expanding variable AVAILABLE_LICENSES[:=], expression was ${@' '.join(available_licenses(d))} which triggered exception FileNotFoundError: [Errno 2] No such file or directory.There is no directory called licenses in the corresponding meta layer.Need to create a licenses directory under the meta-layer and add RDK license file to it
N/A

meta-rdk-restricted

meta-rdk-video

meta-rdk-ext

meta-rdk-containers

meta-rdk

Error due to the usage of base_contains in multiple recipes.base_contains is deprecated.use bb.utils.contains instead of deprecated base_contains in the respective recipes.
N/AN/AError due to the usage of some of the functions in pythonnative.bbclass in multiple recipes.pythonnative.bbclass is not available in newer version.Add meta-python2 layer to get support for legacy pyhton code(E.g pythonnative.bbclass)

openssl.inc

openssl-0.9.inc

image_container_generator.bbclass

rdk.conf

meta-rdk

meta-rdk-ext

Error due to the usage of base_conditional in multiple recipes.base_conditional is deprecated.use oe.utils.conditional instead of deprecated base_conditional in the respective recipes.
uclibc.incmeta-rdk-extError due to the usage of oe_filter_out in multiple recipes.oe_filter_out is deprecated.use oe.utils.str_filter_out instead of deprecated oe_filter_out in the respective recipes.
gssdp_0.14.10.bbmeta-rdk-extCould not include required file recipes-connectivity/gupnp/gssdp.inc in gssdp_0.14.10.bbThe required file is not present in the location.Replace "require" directive to "include" to specify a loose dependency.

rdk-generic-broadband-dev-image.bb

rdk-generic-hybrid-dev-image.bb

rdk-generic-mediaclient-dev-image.bb

meta-rdkErrors due to required files not being present in yocto 3.1(dunfell) oe layers.Those file are not available in newer version of Yocto.Replace "require" directive to "include" to specify a loose dependency.

rmfhalheaders_git.bb


meta-rdk-videoError while parsing LIC_FILES_CHKSUM in rmfhalheadersThe recipe uses "files://" instead "file://" in LIC_FILES_CHKSUM fieldUpdate the LIC_FILES_CHKSUM field accordingly.
busybox_%.bbappendmeta-rdk-extError in busybox as busybox_1.31.1.bb:do_patch: failed with exit code '1'Build error  in do_package task due to some of the older patches availableThe following patches are removed from busybox recipe
for dunfell build
1. udhcp.patch
2. 0001-networking-add-ip-neigh-command.patch
3. ip6_neigh_show_
Crash.patch

glib-networking_2.54.1.bb

rdk.conf

meta-rdk-ext

meta-rdk

Glib-networking require gnutls >= 3.4.6 gnutls 3.3.x is the last branch which supports.nettle 2.7.1. So it is not possible to increase the version,Used glib-networking-2.54.1 instead of updating gnutls.
gstreamer1.0-plugins-bad.incmeta-rdk-extError while inheriting bluetooth.bbclassbluetooth.bbclass is not available in newer version of yocto

inherit bluetooth on a conditional basis as shown below:

inherit gettext ${@bb.utils.contains('DISTRO_CODENAME', 'dunfell', '', 'bluetooth', d)}

glib-2.0_2.62%.bbappendmeta-rdk-extError reported for glib-2.0-native-1_2.62.4-r0 patchUnable to apply the 2nd hunk due to version changes. rebased RDK patch to glib 2.62.4
openssl_1.1.0g.bbmeta-rdk-extError while fetching openssl_1.1.0g Recipe version 1.1.0 is moved to a different location. SRC_URI updated with the new changes.
systemd_216.bbmeta-rdk-extError while parsing systemd_216 legacy recipe  Circular dependency problem with systemd_216.

Skip this recipe by specifying the COMPATIBLE_HOST in bb file as shown below:

COMPATIBLE_HOST_dunfell = "null"


netsrvmgr_git.bbappend

meta-cmf-videoError due to invalid version of netsrvmgr.SRCREV expects a valid revision on dunfell framework.SRCREV for netsrvmgr set to AUTOREV
iptables_%.bbappendmeta-rdk-extError while installing iptablessysint provides it's own iptables systemd service file which gives a conflict on do_rootfs()

Remove systemd service in the bbappend file to avoid conflict as shown below:

do_install_append_client() {
rm ${D}${systemd_system_unitdir}/${PN}.service
}

SYSTEMD_SERVICE_${PN}_remove_client = "${PN}.service"

dropbear_%.bbappend


meta-rdk-extPatch error while building dropbear-2019.78Older version of patches were giving build error.Rebase patches to build on dunfell.
openssl_1.0.2o.bbmeta-rdk-extCircular dependency problems with openssl 1.0.2o for dunfell builds.Higher versions of the recipes are available for newer builds.

Skip this recipe by specifying the COMPATIBLE_HOST in bb file as shown below:

COMPATIBLE_HOST_dunfell = "null"

setup-environmentmeta-rdkdunfell codename not reflected in DISTRO_CODENAME.Bitbake version is not updated for dunfell branch.

Update DISTRO_CODENAME based on the bitbake version in setip-environment script

E.g:-

1.44.*|1.46.*) _DISTRO_CODENAME="dunfell"

RDKmeta-cmfExpansionError: Failure expanding variable AVAILABLE_LICENSES[:=], expression was ${@' '.join(available_licenses(d))} which triggered exception FileNotFoundError: [Errno 2] No such file or directory:RDK licence file not present in the preferred location.Add RDK licence file to the licenses directory in the corresponding layer.
qtbase-native_5.1.1.bbappendmeta-cmf-qt5qtbase-native: fix build error due to gcc 9.xqtbase native pkg throws build error due to gcc 9.2.

The following patch file is available in the repo to resolve the issue:

0001-RDK-27410-qtbase-native-build-error-due-to-gcc-versi.patch

We need to conditionally enable this for dunfell builds. 

logrotate_%.bbappendmeta-rdk-extSysint trying to install logrotate.service, logrotate.timer but it is already installed by logrotate.as sysint provides logrotate's systemd service, actual recipe's service gives conflict

As sysint installs RDK specific logrotate's systemd service file, we can remove them as shown below:


SYSTEMD_SERVICE_${PN}_remove = " \
${BPN}.service \
${BPN}.timer \
"

do_install_append() {
rm ${D}${systemd_system_unitdir}/${BPN}.service
rm ${D}${systemd_system_unitdir}/${BPN}.timer
}

dnsmasq_%.bbappendmeta-rdk-ext Postinstall scriptlets of ['systemd'] failed.As systemd in dunfell provides resolv.conf, no need to install again from dnsmasq.Removed resolv.conf from do_install for dunfell

qtbase_5.1.1.bbappend


meta-cmf-qt5Declaration conflict for qopengl's GLdouble because of this qtbase throws build error.GLdouble is typedefed as GLFloat in qopengl.h this leads to a conflict with the earlier definition of GLdouble as double.We can typedef GLdouble to double.
msgpack-c_2.1.1.bbmeta-rdk-extError about 'copying an object of non-trivial type' in msgpack-c.Typecasting is missing while using memcpy().

Insteadof using 

std::memcpy(&o, &v, sizeof(v)), we can typecast the object as shown below

std::memcpy(static_cast<void*>(&o), &v, sizeof(v))

wpe-webkit.incmeta-rdk-extdunfell build throws do_package_qa error as libWPEWebKit.so requires libTTSClient.soUnable to satisfy runtime dependency libTTSClient.soAdd tts package in the RDEPENDS for wpe-webkit 
ledmgr-extended-noop_git.bbmeta-rdk-videofatal error: sysMgr.h: No such file or directorysysMgr.h file not avalable in the sysroot directory.Add iarmmgrs in the DEPENDS of meta-rdk-video/recipes-extended/ledmgr/ledmgr-extended-noop_git.bb
aamp_git.bbmeta-rdk-videoirMgr.h not foundirMgr.h file not avalable in the sysroot directory.

Add iarmmgrs in the DEPENDS of

meta-cmf-video/recipes-extended/aamp/aamp_git.bbappend

wrp-c_1.0.bbmeta-rdk-ext
error: 'strncpy' output truncated before terminating nul 
copying 23 bytes from a string of the same length.
Use of strncpy for coping.Replace strncpy with memcpy.
ledmgr_git.bbmeta-rdk-videoyocto dunfell framework throws do_package_qa() filedeps error while building the packages.required runtime dependencies are missing.

Add devicesettings in RDEPENDS of 

meta-rdk-video/recipes-extended/ledmgr/ledmgr_git.bb

tr69hostif_git.bbmeta-rdk-videoyocto dunfell framework throws do_package_qa() filedeps error while building the packages.required runtime dependencies are missing.

Add devicesettings in RDEPENDS of 

meta-rdk-video/recipes-thirdparty/tr69/tr69hostif_git.bb

bluetooth-core      N/Afatal error: bluetooth/audio/a2dp-codecs.h: No such file or directorya2dp codec macros are changed in newer bluez version (5.54) from oe-core dunfell branch.

We can add definitions for MIN_BITPOOL and MAX_BITPOOL in btrCore_avMedia.c as shown below

#ifndef MIN_BITPOOL
#define MIN_BITPOOL SBC_MIN_BITPOOL
#endif
#ifndef MAX_BITPOOL
#define MAX_BITPOOL SBC_MAX_BITPOOL
#endif

bluetooth_mgr      N/Aundefined reference error for bluetooth-mgrbluetooth-mgr main.c calls sd_notify() API but not linked libsystemd.so which is causing undefined reference errorUpdate AM_LDFLAGS with -lsystemd to resolve undefined reference error.
bluetooth_mgr      N/Amemcpy issue for MediaElementListInfo.Improper structure size was mentioned in the memcpy.

Replace sizeof(BTRMGR_IARMMediaElementListInfo_t) with sizeof(BTRMGR_MediaElementListInfo_t) in the following function in btmgr_iarm_external_interface.c

memcpy (mediaElementListInfo, &mediaElementList.m_mediaTrackListInfo, sizeof(BTRMGR_IARMMediaElementListInfo_t)) 

bluetooth-core      N/A

Error due to incorrect to use of AdapterInfo struct.

appropriate structure is not mentioned in sizeof().

Replace sizeof(stBTDeviceInfo) in the following function with sizeof(stBTAdapterInfo) in btCore_dbus_bluez5.c

memset(&lstBTAdapterInfo, 0, sizeof(stBTDeviceInfo))

fog      N/AError when HLSPlaylistsDownload.ccp in fogcli calls 'SHA1' API-lcrypto not included in the MakefileUpdate fogcli_LDFLAGS in Makefile.am with -lcrypto.
sectionfilter-hal-noop_git.bbmeta-rdk-videoError due to missing dependencies in sectionfilter-haldunfell framework expects all possible dependencies to be mentioned in DEPENDS.Update sectionfilter-hal-noop_git.bb DEPENDS with jansson.
sys-utils_git.bbmeta-rdk-videopackage_qa error while building sys-utils_git.bbdunfell framework expects to mention all runtime deps to avoid package_qa error.Add devicesettings in the RDEPENDS field of sys-utils_git.bb
iarmmgrs_git.bbappendmeta-cmf-videopackage_qa error while building iarmmgrs_git.bbhybrid specific runtime deps overriding basic runtime deps which causes package_qa error.

Use append instead of simple assignment for the runtime dependency for iarmmgrs as shown below:

RDEPENDS_${PN}_append_hybrid = " virtual/mfrlib"

cairo_%.bbappendmeta-rdk-extConflict due to patches available in other layers while patching cairo.meta-wpe also maintains cairo patches that makes conflict while apply.

apply patches conditionally based on meta-wpe as shown below:

SRC_URI_append = " \
${@'file://cairo-egl-device-create-for-egl-surface.patch' if 'wpe-layer' not in d.getVar('BBFILE_COLLECTIONS').split() else ''} \
${@'file://0008-add-noaa-compositor.patch' if 'wpe-layer' not in d.getVar('BBFILE_COLLECTIONS').split() else ''} \
"

xupnp      N/A
error: too many arguments to function 'gupnp_context_new'.
Based on the gupnp version the we need to call gupnp_context_new and gupnp_root_device_new functions. 

Fix is available in the following commit:

REFPLTV-473

rmfgeneric_git.bbmeta-rdk-videormfgeneric throws do_package() error says dwarfsrcfiles exit with error code for librbi.a.This issue comes only when RBI is enabled (--enable-rbi) for hybrid.Issue canbe resolved by adding INHIBIT_PACKAGE_DEBUG_SPLIT_hybrid = "1" in rf=mfgeneric_git.bb

procps_%.bbappend

wpe-webkit.inc


meta-rdk-extBuild failure due to missing dependencies.dunfell build expects all possible dependencies in place.

Fix is available in the following commit:

39396 : REFPLTV-473

tr69agent      N/Aerror: storage size of 'ctx' isn't knownThe build is failing due to openssl 1.0 and 1.1 compatibility issues.

Fix is available in the following commit:

39371 : REFPLTV-473

tr69agent_git.bb

tr69hostif_git.bb

meta-rdk-video

configure: error: Package requirements (libproc >= 3.2.8) were not met:

No package 'libproc' found

Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.

Alternatively, you may set the environment variables PROCPS_CFLAGS
and PROCPS_LIBS to avoid the need to call pkg-config.

--enable-latest-procps OECONF option was only enabled in morty builds.

--enable-latest-procps OECONF option can be added to dunfell builds as well by editing the respective recipes as shown below:

EXTRA_OECONF_append_dunfell = " --enable-latest-procps"

tr69N/A--enable-morty name conflict while building tr69hostif from dunfell framework.--enable-morty name conflicts by enabling on dunfell builds.

Fix is available in the following commit:

39342 : REFPLTV-412

tr69hostifN/Aaclocal: error: couldn't open directory 'cfg': No such file or directorylatest autotools version expects m4 macro directory.We can fix this issue by replacing AC_CONFIG_MACRO_DIR([cfg]) with AC_CONFIG_MACRO_DIR([m4]) in configure.ac for tr69hostif.
bluetooth-coreN/A

building bluetooth-core on gcc 9.x throws below errors

-Werror=stringop-overflow=

-Werror=stringop-truncation

-Werror=format-truncation=

on below files

src/btrCore.c

src/bt-iface/btrCore_dbus_bluez.c

gcc 9.x enables more string operations flags that throw errors on compilation.

Fix is available in the following commit:

39257 : REFPLTV-412

bluetooth-coreN/ABuild failure due to incorrect structure definition.appropriate structure is not mentioned in sizeof() while memset-ting the same.

Replace sizeof(stBTDeviceInfo) with sizeof(stBTAdapterInfo) in the following function in btrCore_dbus_bluez5.c

memset(&lstBTAdapterInfo, 0, sizeof(stBTDeviceInfo))

power-state-monitorN/ABuild failure due to directfb header file inclusion while building power-state-monitor from dunfell framework.directfb depends is no more from recipe so, no need of directfb headers / API in the src.Remove #include <direct/list.h> from powerStateMonitorMain.c
dibbler_%.bbappendmeta-rdk-extDibbler issues: patch doesn't apply, couldn't install script file which resulted in packaging errors.patch already incorporated into src.

We can remove the following patch from the bbappend file for dunfell builds:

0001-DELIA-19972-Dibbler-client-crash-in-checkDecline.patch

mediaframeworkN/Aaclocal: error: couldn't open directory 'cfg': No such file or directorylatest autotools version expects m4 macro directory.We can fix this issue by replacing AC_CONFIG_MACRO_DIR([cfg]) with AC_CONFIG_MACRO_DIR([m4]) in configure.ac for mediaframework.
rmfstreamer_git.bbappendmeta-cmf-raspberrypierror: 'rdkStorage_getTSBCapacity' was not declared in this scopestorage manager is removed from DEPENDS variable for all builds.

Instead of removing storage manager dependency from all builds just remove it from morty builds alone as shown below:

DEPENDS_remove_morty = "storagemanager"

wifi-client-hal-raspberrypi_0.1.bbmeta-cmf-raspberrypiwifi-client-hal-raspberrypi-yocto-dunfell-upgrade+gitAUTOINC+5339f06be3-r0 do_package_qa: QA Issue: /usr/lib/libwifihal.so.0.0.0 contained in package wifi-client-hal-raspberrypi requires libwpa_client.so, but no providers found in RDEPENDS_wifi-client-hal-raspberrypi? [file-rdeps]Missing runtime dependencies in the recipe file.

Add the runtime dependencyin wifi-client-hal-raspberrypi_0.1.bbas shown below:

RDEPENDS_${PN} += “wpa-supplicant”

qtbase_5.1.%.bbappendmeta-cmf-raspberrypiNothing PROVIDES 'kms'As RDK uses qtbase 5.1.1, changes affecting from higher version to be removed.

Create a bbappend file in meta-raspberrypi/recipes-qt/qt5/qtbase_5.1.%.bbappend with the following content

PACKAGECONFIG_remove = "kms"

xupnp_git.bbmeta-rdkxupnp configure error due to safec library version changes.Unable to find the library because of the safec library version changes.

Fix is available in the following commits:

40178 : REFPLTV-474

40921 : REFPLTV-474

wpeframework_git.bbmeta-rdk-extError while building wpeframework becaue of stime() usage in SystemInfo.cpp.As the GLIBC moved to 2.31 from 2.30 the stime() was removed in 2.31 and it throws error while building wpeframework.

Replace the stime() with clocksesttime() as shown in the below changeset:

40759 : REFPLTV-474

wpe-webkit_%.bbappendmeta-cmf-raspberrypi

rdk-generic-mediaclient-image.bb:do_rootfs error missing runtime dependency of the following components needed by wpe-webkit:

gstreamer1.0-plugins-bad-faad

gstreamer1.0-plugins-bad-opengl

gstreamer1.0-plugins-ugly-mpg123

Failed because of missing runtime dependency.

Fix is available in the following changeset:

40191 : REFPLTV-474

packagegroup-rdk-media-common.bbappendmeta-cmf-raspberrypiRun time issue: westeros-sink not available in the imagewesteros-sink is not installed by default in the rdk-generic-mediaclient-image dunfell builds.

Fix is available in the following changeset:

41481 : REFPLTV-474

appmanager_git.bbappendmeta-cmf-raspberrypiRun time issue: appmanager was not running when we invoke it using /lib/rdk/runAppManager.shThe LD_PRELOAD vlaue in the appmanager script uses libopenmaxil.so but we need to specify libwesteros_gl.so for dunfell builds.

A do_install_append in the recipe file with the following content will fix this issue.

do_install_append() {
if ${@bb.utils.contains('MACHINE_FEATURES', 'vc4graphics', 'true', 'false', d)}; then
sed -i 's#libopenmaxil.so#libwesteros_gl.so.0#g' ${D}/${base_libdir}/rdk/runAppManager.sh
fi
sed -i 's/libwayland-egl.so:/libwayland-egl.so.1:/g' ${D}/${base_libdir}/rdk/runAppManager.sh
}

The corresponding change set is:

40338 : REFPLTV-474

rpi-config_%.bbappendmeta-cmf-raspberrypiRun time issue: audio and dsmgr service crash. sysint will trigger a reboot if certain components (dsmgr service in this case) crashes a particular no.of times. So we may see multiple reboots if this issue is not addressed.

vc4graphics in dtoverlay is automatically set in the BSP layer. But the order of "dtparam=audio=on" matters without which it won't be able to use bcm2835_alsa from /proc/asound/cards.

Observation: If we set "dtparam=audio=on" just after specifying "dtoverlay=vc4-kms-v3d" then /proc/asound/cards will list only vc4-hdmi. This in turn crashes dsmgr service.

Set "dtparam=audio=on" before setting "dtoverlay=vc4-kms-v3d" 

The following change set can be used to implement the same:

40631 : REFPLTV-474

rdkmediaplayer.bbmeta-rdk-videoRun time issue: rdkmediaplayer crashO2 flags in gcc 9.2 causes runtime issues.

The following flags can be set to resolve the issue:

# O2 flags in gcc 9.2 causes runtime issues
CXXFLAGS_append_dunfell = " -fno-reorder-blocks -fno-align-loops"

# O2 flags in gcc 9.2 causes runtime issues for arm architecture
CXXFLAGS_append_dunfell_arm = " -fno-cse-follow-jumps"

Currently, all the changes are being pushed in these branches. Once it is merged, you will be able to follow the build instructions without any error.