<work in progress>

This Document explains how users can create a build for Raspberrypi (RPI) / Reference platform, bring it up in target device and then create and run a sample lightning app .

Basic Skills

Though not mandatory, the below skills will help the user to understand RDK, RDK build, and to try out RDK better :

• Familiar with Linux based platforms
• Familiar with Yocto
• Familiar with RDK basics
• Knowledge in using Raspberry Pi
• Experience in setting up boards

Relevant RDK links

Prerequisites

Host Setup

Install the following packages for setting up your host VM

The instructions provided below are meant to be executed via the command line on an Ubuntu machine

# essential packages installation
# super user mode is required

# major essential packages
sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib g++-multilib build-essential chrpath socat bison curl
# supportive packages
sudo apt-get install libfile-slurp-perl libncurses-dev autoconf flex doxygen libtool automake libpcre3-dev zlib1g-dev libbz2-dev subversion minicom putty libssl-dev rpm python-pexpect python-svn python-argparse vim tofrodos meld dos2unix cmake uuid-dev ruby transfig libglib2.0-dev xutils-dev lynx-cur gperf autopoint python-dulwich python-dev openjdk-7-jre

# essential packages installation
# super user mode is required

# major essential packages
sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib g++-multilib build-essential chrpath socat bison curl cpio python3 python3-pip python3-pexpect xz-utils debianutils iputils-ping python3-git python3-jinja2 libegl1-mesa libsdl1.2-dev pylint3 xterm

Configure bash as default command interpreter for shell scripts

sudo dpkg-reconfigure dash

Select “No”
To choose bash, when the prompt asks if you want to use dash as the default system shell - select “No”

Configure Git

Upgrade your Git version to 1.8.x or higher

On Ubuntu 16.04 LTS, if you are unable to upgrade your git version using apt-get, then follow the below steps in order to upgrade 

sudo apt-get install software-properties-common
sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
sudo apt-get install git

Once git is installed, configure your name and email using the below commands

# review your existing configuration
git config --list --show-origin

# configure user name and email address
git config --global user.name "John Doe"
git config --global user.email johndoe@example.com

# configure git cookies. Needed for Gerrit to only contact the LDAP backend once.
git config --global http.cookieFile /tmp/gitcookie.txt
git config --global http.saveCookies true

Configure repo

In order to use Yocto build system, first you need to make sure that repo is properly installed on the machine:

# create a bin directory
mkdir ~/bin
export PATH=~/bin:$PATH

# Download the repo tool and ensure that it is executable
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
chmod a+x ~/bin/repo

 Credential configuration

Note: it is also recommended to put credentials in .netrc when interacting with the repo.

A sample .netrc file is illustrated below

Build Setup

repo init -u https://code.rdkcentral.com/r/manifests -b dunfell -m rdkv-nosrc.xml
repo sync -j `nproc` --no-clone-bundle --no-tags

1. Insert an SD card in the SD card port of the USB SD card reader (or Laptop).

     Prefer to use 32gb sd card and there should be minimum 12gb free space available in the device .

2. Verify that the SD card has been detected by executing either of the commands listed below

$lsblk
$sudo fdisk –l 

 3. Type the following command to ensure that the partitions, if present, on the SD card are not mounted

$mount

4. Repeat the below command to unmount all the mounted partition present on the SD card.

$unmount <partition-mountpoint>

5. Execute the following command to flash the image on the SD card

sudo dd if=<path to ImageName.rpi-sdimg> of=<path to SD card space> bs=4M
Example:
$ sudo dd if=rdk-generic-broadband-image-raspberrypi-rdk-broadband.rpi-sdimg of=/dev/sdb bs=4M

6. Increase size of “ext3” File system based partition of SDCard. The increased partition is required to copy media files for streaming. Any tools/utilities can be used to alter partition size. The explanations captured below are using “Gparted” a linux-based tool to alter partition size.  

Follow instruction given below:- 

• Install ‘Gparted’ using below instruction:- 

$sudo apt-get install gparted

Once Gparted installed successfully. Open ‘Gparted’ with root privileges:-

“Gparted” graphical window shall appear.

• Select “ext3” File System based partition of SDCard to alter partition size:-

• Extend the ext3 partition to maximum available space in the 32gb sd card .

• Click on “apply all operations” to modify the size:-

7. Repeat the below command to unmount all the mounted partition present on the SD card.

$unmount <partition-mountpoint> 

8. Remove the SD card and insert it to the Raspberry Pi SD card slot 

9. Similarly flash the mediaclient image on another SD Card and insert it to another Raspberry Pi SD Card slot 

10. Bring up the device

What is Wpeframework?

WPEFramework provides a unified web-based interface with a consistent navigation model. In this model, plugins (custom or generic) are controlled and queried, through the WPEFramework application.

The main responsibilities of WPEFramework application are:

• Modular loading and unloading of plugins.
• Plugin process localization. In or out-of-process communicating with the framework over a lightweight RPC communication channel.
• Runtime enabling/disabling of tracing information within the plugins and the WPEFramework application.
• Light-weight implementation of the HTTP [RFC2616] specification.
• Light-weight implementation of the WebSocket [RFC6455] specification.

Each instance of a plugin in the WPE id identified by a name. This name is referred to as Callsign of the plugin. The callsign must be unique in the context of all configured plugins.

What is RDKServices?

RDK services are a set of JSON-RPC services that provide access to the RDK core components on set-top devices. RDK services are implemented as Thunder plugins for the Thunder framework. The Thunder framework is responsible for, among other things, managing plugins and handling client requests. RDK services are invoked over HTTP or Web Sockets using their JSON-RPC services-based interface. This makes RDK services accessible to any client that can process JSON, such as Lightning JavaScript applications or HTML5 web applications. For native applications, you can also invoke RDK services directly using C/C++ native code.

Explain layer structures in RDK-V build

ToDo

../meta-cmf-video/recipes-extended/rdkservices/rdkservices_git.bbappend
../meta-rdk-broadcom-generic-rdk/meta-brcm-refboard/meta-rdk-video/recipes-extended/rdkservices/rdkservices_git.bbappend
../meta-rdk-bsp-emulator/recipes-extended/rdkservices/rdkservices_git.bbappend
../meta-rdk-ext/recipes-extended/rdkservices/rdkservices_%.bbappend
../meta-rdk-oem-comm-bcm-accel/meta-vip7802/recipes-extended/rdkservices/rdkservices_git.bbappend
../meta-rdk-oem-comm-bcm-accel/meta-vip5202w/recipes-extended/rdkservices/rdkservices_git.bbappend
../meta-rdk-soc-broadcom/meta-brcm/recipes-extended/wpe-framework/rdkservices_git.bbappend
../meta-rdk-soc-broadcom/meta-brcm/recipes-extended/wpe-framework/rdkservices-comcast_git.bbappend
../meta-rdk-video/recipes-extended/rdkservices/rdkservices_git.bb

Steps involved in implementing new RDK services Plug-In

RDK components implemented as Thunder plugins are called as RDKServices. it is  developed based on the Thunder (WPE) Framework. Services each other or a particular service can be COMRPC ( for (communication between plugins) or RPC (for external communication). it has a web-based controller UI.

Reference : https://github.com/rdkcentral/rdkservices/pull/960 

 In RDK services -plugins workspace:

 Cloned from https://github.com/rdkcentral/rdkservices

	$ git clone https://github.com/rdkcentral/rdkservices

Inside PluginName directory

1.  <PluginName>.json : This file contains the plugin's information like schema, information and interface json file.
   1. PluginTemplate.json
   2. DeviceInfo.json
2.  CmakeLists.txt:  CMAKE based configuration file which contains a set of directives and instructions describing the project's source files and targets. This is used to compile the Plug-in code to generate the plugin library(Shared library by default; “.so”). External dependencies can be included/linked to the target with the help of CMakeLists.txt configurations.
3. Module.h: This header file includes the support for JSON request, response, logging etc...
4. Module.cpp: This file is used to declare the module name for the Plug-in. This file contains the plugin's information like schema, information and interface json file (defined earlier).
5. <PluginName>.config: This file is used to set configurations of the Plug-in . Ex:- set (autostart true) - Used to make the Plug-in to start automatically along with wpeframework daemon
6. <PluginName>.h :Declare the plugin class in this which should contains all the structures, variables and methods which are needed for plugin implementation. The interface header auto-generated earlier will be used here,
7. <PluginName>.cpp: This class does contains all the definitions for the methods declared in the Plugin.h and those definitions should be defined inside the below namespace.
8. Cmake / (directory) :
     

PluginTemplate/
  ├── CMakeLists.txt
  ├── PluginTemplate.config
  ├── PluginTemplate.cpp
  ├── PluginTemplate.h
  ├── PluginTemplate.json
  ├── Module.cpp
  ├── Module.h
  ├── README.md
  ├── cmake
  |   ├── FindDS.cmake
  │   └── FindIARMBus.cmake
  └── doc
   └── PluginTemplate.md

<PluginName>.json

 This file contains the plugin's information like schema, information and interface json file.

Syntax :

   {
    {
    "$schema": "plugin.schema.json",
    "info": {
    "title": "Plugin Name Plugin",
    "callsign": "PluginName",
    "locator": "libWPEFrameworkPluginName.so",
    "status": "production",
    "description": "The PluginName plugin allows retrieving of various plugin-related information.",
    "version": "1.0"
     },
     "interface": {
    "$ref": "{interfacedir}/PluginName.json#"
     }
   }

eg: PluginTemplate.json

 {
     "locator":"libWPEFrameworkPluginTemplate.so",
     "classname":"PluginTemplate",
     "precondition":[
      "Platform"
     ],
     "callsign":"org.rdk.PluginTemplate",
     "autostart":false
 } 

<PluginName>.config

.config files are files used to configure the parameters and initial settings for some computer programs.

set (autostart false)                       #we are setting autostart condition disable 
set (preconditions Platform) 
set (callsign "org.rdk.PluginTemplate") 	#The callsign name was given to an instance of a plugin. 

#One plugin can be instantiated multiple times. but each instance, the instance-name "callsign" must be unique. here we using org.rdk.PluginTemplate. 

<PluginName>.h

Declare the plugin class in this which should contain all the structures, variables, and methods which are needed for plugin implementation. 

 namespace WPEFramework {
  namespace Plugin {
   
   class PluginName : public PluginHost::IPlugin, public PluginHost::IWeb, public PluginHost::JSONRPC {
   public:
     PluginName()
      : _skipURL(0)
      , _service(nullptr)
      , _subSystem(nullptr)
     {
      RegisterAll();
     }
  
     virtual ~PluginName()
     {
      UnregisterAll();
     }
   }
   ---------------------------------------
   ---------------------------------------
  }
}

eg: PluginTemplate.h

for more information refer PluginTemplate.h 

<PluginName>.cpp

This class does contain all the definitions for the methods declared in the PluginTemplate.h and those definitions should be defined inside the below namespace. 

The plugin should register using service registration MACRO as declared below :

  namespace WPEFramework {
   namespace Plugin {  
    SERVICE_REGISTRATION(Plugin, 1, 0);
    ---------------------------------------
    ---------------------------------------
    ---------------------------------------
   }
  }

 To initialize and deinitialize or activate or deactivate handler for the plugin services :

const string PluginTemplate::Initialize(PluginHost::IShell* /* service */)
	{
	//shared pointer initialized
	//initialize external library
        LOGINFO();
        return (string());
        }
void PluginTemplate::Deinitialize(PluginHost::IShell* /* service */)
        {
 	//shared pointer deinitialized
	//deinitialize external library
        LOGINFO();
        }

eg:

namespace WPEFramework {
    namespace Plugin {  
         SERVICE_REGISTRATION(Plugin, 1, 0);
                
          //registration
          //All the methods declared in Plugin.h should be registered here
                
          //initialize and deinitialize the handlers for the plug-in service
                
          //All the methods declared in Plugin.h should be defined here
                              
    }
} 

CMakeLists.txt

Using the CMake utility this file contains the task needed to be done to make a plug-in. Also contains packages, libraries needed to compile, its path, and other plugin-in configuration option.

This file contains a set of directives and instructions describing the project's source files and targets (executable, library, or both).

set(PLUGIN_NAME PluginTemplate)                          # to set a environment variable set(<variable> <value>)
set(MODULE_NAME ${NAMESPACE}${PLUGIN_NAME})
find_package(${NAMESPACE}Plugins REQUIRED)               # to Finds and loads settings from an external project.

#Adds a library target called <name> to be built from the source files listed in the command invocation. The <name> corresponds to the logical target name and must be globally unique within a project.
add_library(${MODULE_NAME} SHARED
        PluginTemplate.cpp
        Module.cpp
        ../helpers/utils.cpp)

Code flow

• Enable or disable the plug-in flag in the recipe file.
• Add this flag into the main CMakeLists.txt file present in the rdkservice.
• It will invoke CMakeLists file present in the <plugin name>/ (eg: PluginTemplate/CMakeLists.txt).
• When this file started to execute it finds dependencies, packages. it compiled and generate .so file.

To add plugin in rdkservices CMakeLists.txt

In rdkservices directory open CmakeLists.txt :

	$ vi CmakeLists.txt

add these lines ( by default its disabled ) :

    if(PLUGIN_PLUGINTEMPLATE)
    	add_subdirectory(PluginTemplate)
    endif()

 it will invoke your CMakeLists.txt file present in your plugin directory.

Compilation and Install

To include plugintemplate plugin in build sequence, Open rdkservices recipe file and add below line. By default; its configured to be disabled while building rdkservices.

            $ vi meta-rdk-video/recipes-extended/rdkservices/rdkservices_git.bb

PACKAGECONFIG[plugintemplate]      = " -DPLUGIN_PLUGINTEMPLATE=OFF,-DPLUGIN_PLUGINTEMPLATE=ON, "  

To include the plugin in rdkservises build; add the same in packageconfig in rdkservices recipe:

              PACKAGECONFIG += " plugintemplate"

to compile and install in build directory :

	$ bitbake -c compile -f rdkservices

once build complete copy .json, .so file into raspberrypi.

Copy the Plugin.json (eg: PluginTemplate.json) file to “/etc/WPEFramework/plugins” in raspberrypi

          Copy the plugin library (libWPEFrameworkPluginTemplate.so) to “/usr/lib/wpeframework/plugins”

so that the controller plugin identify it and list it in the WebUI ( controller UI ).

Controller UI

Controller UI is a web UI that can be launched from a host machine's (machine under the same network where RPi resides) browser. This UI can be loaded with the RPi box's IP address with Thunder's port number configured (here). RDKServices uses 9998 as port.

		URL: http://<IP address of the Target device>:9998

Defalut page of Controller UI shall be loaded on web-browser and that will be of Controller tab. Controller tab allows all available plugins to be enabled or disabled.

PluginTemplate JSON RPC command

Each RDK Service can be validated through JSON RPC Commands through HTTP. It has a request and response in JSN format.

Note: the argument is case sensitive.

"callsign":"org.rdk.PluginTemplate"

OUT OF PROCESS Plugin

Here the plugin is developed as out of process, which runs as a separate thread from WPEFramework. Services each other or a particular service can be COMRPC ( for communication between plugins) or JSONRPC (for external communication). it has a web-based controller UI.

Inside PluginName directory

OutOfProcessPlugin/
  ├── CMakeLists.txt
  ├── OutOfProcessPlugin.config
  ├── OutOfProcessPlugin.cpp
  ├── OutOfProcessPlugin.h

  ├── OutOfProcessPluginJsonRpc.cpp
  ├── OutOfProcessPlugin.json
  ├── Module.cpp
  ├── Module.h
  ├── OutOfProcessPlugin.md

<PluginName>.json

 This file contains the plugin's information like schema, information and interface json file. Here the outofprocess will be true, which indicates that the plugin run as a seperate process.

 eg: OutOfProcessPlugin.json

<PluginName>.config

.config files are files used to configure the parameters and initial settings for some computer programs.

Here outofprocess is set to true, to make plugin as out of process plugin.

<PluginName>.h

Declare the plugin class in this which should contain all the structures, variables, and methods which are needed for plugin implementation. 

<PluginName>.cpp

This class does contain all the definitions for the methods declared in the Plugin.h and those definitions should be defined inside the below namespace. 

The plugin should register using service registration MACRO as declared below :

To initialize and deinitialize or activate or deactivate handler for the plugin services :

Process handler plugin services to receive request and sent responses based on the services :

eg: refer OutOfProcessPlugin.cpp

<PluginNameJsonRpc>.cpp>

The PluginNameJsonRpc file contains the registration for methods and properties which are declared in PluginName.h

CMakeLists.txt

Using the CMake utility this file contains the task needed to be done to make a plug-in. Also contains packages, libraries needed to compile, its path, and other plugin-in configuration option.

This file contains a set of directives and instructions describing the project's source files and targets (executable, library, or both).

The Code flow, Compilation and Install steps are similar to the PluginTemplate.

The last step,

Copy the Plugin.json (eg: OutOfProcessPlugin .json) file to “/etc/WPEFramework/plugins” in raspberrypi

          Copy the plugin library (libWPEFrameworkOutOfProcessPlugin.so) to “/usr/lib/wpeframework/plugins”

so that the controller plugin identify it and list it in the WebUI ( controller UI ).

Controller UI

• Inside PluginName directory

Controller UI is a web UI that can be launched from a host machine's (machine under the same network where RPi resides) browser.

URL: http://<IP address of the Target device>:9998

OutOfProcessPlugin JSON RPC command

Each RDK Service can be validated through JSON RPC Commands through HTTP. It has a request and response in JSON format.

Note: the argument is case sensitive.

"callsign":"OutOfProcessPlugin"

 how the interfaces COMRPC and JSONRPC used and its details

Few references:
https://wiki.rdkcentral.com/display/RDK/RDK-4+Architecture
https://wiki.rdkcentral.com/display/CONF/Tech+Summit+2019+-+Day+3?preview=/83561063/83562791/ThunderTechSummit2019%20-%20edits%20JPR3.pdf
https://etwiki.sys.comcast.net/pages/viewpage.action?pageId=880685927&preview=%2F880685927%2F914198299%2FV2.1_+RDK-V+Plugin+HelloWorld.docx
https://etwiki.sys.comcast.net/display/CPE/Thunder+Plugin+Development+and+Code+Generation

ThunderJS is used to make easy to make API calls to Thunder (WPEframework) over a Websocket connection. ThunderJS can also be used to listen to (and act upon) notifications broadcasted by Thunder. ThunderJS is an isomorphic library, which means it can be used in a browser environment as well as a NodeJS environment.

Lightning is a Javascript TV app development framework based on NodeJS environment. So ThunderJS can be easily integrated to the Lightning apps.

Adding ThunderJS dependency to package.json

ThunderJS dependencies can be added manually to the lightning projects by adding ‘"ThunderJS": "github:rdkcentral/ThunderJS",’ to the package.json under ‘devDependencies’ and then run ‘npm install’.

OR

ThunderJS can be installed into your project via NPM command. Then the package.json will be updated with the thunder dependency "npm install github:rdkcentral/ThunderJS"

Snippet of package.json is given below .To use the ES6 syntax, we need add the Babel dependency also.

Import ThunderJS dependency in Lightning script

Next you can import the ThunderJS dependency into your own script like given below.

import ThunderJS from 'ThunderJS',

Initializing the library

The library can be initialised by passing the IP, port and other parameters mentioned below

const config = {  
 host: '192.168.1.100', // defaults to localhost,  
 port: 2020, // defaults to 80  endpoint: '/api', // defaults to '/jsonrpc'  
 protocol: 'wss://', // defaults to 'ws://'  
 subprotocols: 'notification', // WebSocket sub-protocols, defaults to 'notification'
}
 const thunderJS = ThunderJS(config)

Example:

Making API Calls and read results

The library supports 2 ways of making API calls, depending on your coding style preferences.

Option 1 - Argument based

const plugin = 'DeviceInfo'

const method = 'systeminfo'

const params = {

  foo: 'bar'

}

thunderJS.call(plugin, method, params)

const params = {

  foo: 'bar'

}

thunderJS.DeviceInfo.systeminfo(params)

The result can be processed in two ways also

thunderJS.DeviceInfo.systeminfo()

  .then(result => {

    console.log('Success', result)

  }).catch(err => {

    console.error('Error', err)

  })

thunderJS.DeviceInfo.systeminfo((err, result) => {

  if(err) {

    console.error('Error', err)

  }

  else {

    console.log('Success!', result)

  }

})

Example for Argument based call for Bluetooth plugin is given below where the result is processed in callback based method

Bluetooth plugin pair method - https://github.com/rdkcentral/rdkservices/blob/sprint/2107/Bluetooth/doc/BluetoothPlugin.md#pair-method

Corresponding implementation in ThunderJS

   this.thunderJS.call('org.rdk.Bluetooth', 'pair', { deviceID: deviceIDval },

      (err, result) => {

        if (err) {

          Log.info('\n Bluetooth Pair error' + JSON.stringify(err))

        } else {

          Log.info('Pairing success' + JSON.stringify(result))

        }

      }

    )

Notifications

Thunder (WPEframework) broadcasts notifications when events ocur in the system. However it will only broadcast those events that the client has subscribed to.

ThunderJS makes it easy to subscribe to specific events, and execute a callback-function upon every notification of each event.

Example for notification event subscriptions is given below.

Bluetooth onDiscovered event -  https://github.com/rdkcentral/rdkservices/blob/sprint/2107/Bluetooth/doc/BluetoothPlugin.md#ondiscovereddevice-event

Corresponding implementation in the Lightning App