Overview

This page presents an brief overview about webPA 2.0 (xmidt) components required for a reference webPA cluster setup and explains how to setup the cluster & establish an end-to-end connection with CPE devices.

WebPA is a secure web protocol messaging system for bi-directional communication between cloud server and RDK devices. It was built from the ground up specifically with security and performance as priorities. It is currently used by millions of devices and services and will continue to expand in scale into the future.

WebPA 2.0 commonly known as Xmidt (pronounced "transmit") is a combination of a server cluster and client that provide a highly available data path to devices deployed all over the world.

• One active use case for WebPA within the RDK community focuses on pre-emptive troubleshooting, by automatically associating customer call-in logs regarding RDK devices, with the near-real-time connectivity data (gathered via WebPA) about all RDK devices.
• As a result, RDK-based multichannel video providers (MVPDs) can start seeing correlations between the two — which counts as early-stage momentum around what can be accomplished, when machine data gets correlated with call-in data.
• Unlike TR-69, which polls wide-and-deep across a device landscape, on a less frequent basis, WebPA can precision-poll for the most useful data, much more quickly. That’s mainly because it’s lightweight, and because the load can be redistributed into all the apps needing to access the data.
• Specifically, WebPA is a lightweight connectivity socket, with lower-level protocol connectivity between devices, and the cloud — much like a websocket. It works by each client registering with the cloud, so that it maintains a socket connection to the cloud. On the cloud side, APIs enable polls through which other cloud components could receive events, or translate them in to a TR-181 device data model for further analysis.
• And, because WebPA is an “always on” connection (websocket), asynchronous notifications for a change in device data are a straightforward exercise, where the application “listens” for webhook events.
• As for security, WebPA provides a simple REST interface that is accessible from anywhere on the open Internet, with tokenized/encrypted security built-in. In fact, the entire WebPA ecosystem will be open-sourced, including Talaria, Petasos, and Scytale.
• Ultimately, TR69 was solving for a different set of problems than what MVPD/broadband providers need, RDK community members said. Not needed were the proprietary and “northbound” ACS (Automatic Configuration Server) interfaces, for instance, managed by the telecom vendor eco-system.

Setting up a single node webPA cluster

System Requirement

Dependencies

1) Supervisor

Supervisor is a client/server system that allows its users to monitor and control a number of processes on UNIX-like operating systems.

Unlike other system initialization services, it is not meant to be run as a substitute for init. Instead it is meant to be used to control processes related to a project or a customer, and is meant to start like any other program at boot time.

How to install

a) Enable Extra Packages for Enterprise Linux (EPEL)
	$ wget http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
	$ rpm -Uvh epel-release-6-8.noarch.rpm
b) Install Python meld3
	$ yum install python-meld3
c) Install supervisor
	$ yum install supervisor

2) ZooKeeper

ZooKeeper is a high-performance coordination service for distributed applications. It exposes common services - such as naming, configuration management, synchronization, and group services - in a simple interface so you don't have to write them from scratch. You can use it off-the-shelf to implement consensus, group management, leader election, and presence protocols. And you can build on it for your own, specific needs.

How to install

$ yum install zookeeper

Adding a startup script

# cat /etc/init.d/zookeeper 
#!bin/bash
#
# zookeeper Startup Script
#
# chkconfig: 345 90 14
# description: Zookeeper Application Startup Script

# Source function library
. /etc/rc.d/init.d/functions

#-------------------------------------------------------------------------------

start() {
	echo -n $"Starting Zookeeper: "
	/usr/lib/zookeeper/bin/zkServer.sh start
	RETVAL=$?
	echo
	[ $RETVAL -eq 0 ] && echo "[ OK ]"
}

stop() {
	echo -n $"Stopping Zookeeper: "
	/usr/lib/zookeeper/bin/zkServer.sh stop
	RETVAL=$?
	echo
	[ $RETVAL -eq 0 ] && echo "[ OK ]"
}

restart() {
	stop
	start
}

case "$1" in
  start)
	start
	;;
  stop) 
	stop
	;;
  restart|force-reload|reload)
	restart
	;;
  status)
	/usr/lib/zookeeper/bin/zkServer.sh status
	RETVAL=$?
	;;
  *)
	echo $"Usage: $0 {start|stop|status|restart|reload|force-reload}"
	exit 1
esac

exit $RETVAL

Enable the service at bootup

Launch system-config-services from a console and enable the zookeeper service from the services list.

WebPA components

Below is the list of components needed for a Xmidt (webPA 2.0) cluster setup. For a single node reference setup, few of the services are not mandatory hence not used.

Install released version of packages from GitHub

a) Import the GPG Key (Required once, common for all the packages)
	$ rpm --import https://github.com/Comcast/tr1d1um/releases/download/0.0.1-65/RPM-GPG-KEY-comcast-webpa
b) Install the packages
	$ rpm -Uvh https://github.com/Comcast/scytale/releases/download/0.1.1-83/scytale-0.1.1-83.el6.x86_64.rpm
	$ rpm -Uvh https://github.com/Comcast/talaria/releases/download/0.1.1-153/talaria-0.1.1-153.el6.x86_64.rpm
	$ rpm -Uvh https://github.com/Comcast/tr1d1um/releases/download/0.1.1-228/tr1d1um-0.1.1-228.el6.x86_64.rpm

Build from Source

If pre-built packages are already installed as explained in previous section & we want to use the same, skip to configuration section

Dependency

Install golang

Required for compiling server components written in go language.

$ sudo yum install golang

install glide

Glide is a package manager for Go that is conceptually similar to package managers for other languages. Glide provides the following functionality:

• Records dependency information in a glide.yaml file. This includes a name, version or version range, version control information for private repos or when the type cannot be detected, and more.
• Tracks the specific revision each package is locked to in a glide.lock file. This enables reproducibly fetching the dependency tree.
• Utilizes vendor/ directories, known as the Vendor Experiment, so that different projects can have differing versions of the same dependencies.

$ wget -c https://github.com/Masterminds/glide/releases/download/v0.13.1/glide-v0.13.1-linux-amd64.tar.gz
$ tar -xzf glide-v0.13.1-linux-amd64.tar.gz -C /opt
$ echo "export PATH=$PATH:/opt/linux-amd64/" >> $HOME/.bash_profile

Downloading the packages

1. create a directory in $HOME say webpa_modules
$ mkdir $HOME/webpa_modules && cd $HOME/webpa_modules

2. Checkout the components from GitHub repository.
$ git clone https://github.com/Comcast/talaria.git
$ git clone https://github.com/Comcast/scytale.git
$ git clone https://github.com/Comcast/tr1d1um.git

Building the components

1. Set the GOPATH & change to the source directory, e.g.
	$ export GOPATH=$HOME/webpa_modules/petasos/
	$ cd $HOME/webpa_modules/petasos/src/petasos

2. Resolve package dependencies using glide.
	$ glide install --strip-vendor

3. Build the component from source
	$ go build petasos

4. Create the package
	$ mkdir $HOME/rpmbuild
	$ ./build_rpm.sh --no-sign

5. Install the locally built webPA component package
e.g. $ cd /root/rpmbuild/RPMS/x86_64/
     $ rpm -Uvh petasos-0.1.1-87.el6.x86_64.rpm

Configuring the server components