Source code repository

Xconf consist of two web applications - Xconf dataservice and Xconf admin. Xconf DataService is the app that the STBs talk to. Xconf Admin allows humans to enter all the information necessary for Xconf to provide the correct information to STBs. This repo contains the source code for both the applications -  https://github.com/rdkcentral/xconfserver 

Software Requirements (Reference setup)

Installation of dependencies

1. Install Java JDK

Java JDK version should be 8.  Get the supported version from Oracle or use the OpenJDK packages.

Steps to install Open JDK

• sudo apt-get update

• sudo apt-get install openjdk-8-jdk

Check your installation using the command : java  -version

2. Install Maven (Version 3.6.0)

Maven version should be 3 +. 

To install maven follow these steps:

• sudo apt update

• sudo apt install maven

• Check the installation using : mvn -version

3. Download xconf server code

We can download the latest xconfserver code from https://github.com/rdkcentral/xconfserver. The latest version of the code is available in main branch.

• Create a folder : mkdir xconf

• Step into the folder cd xconf/

• git clone https://github.com/rdkcentral/xconfserver.git -b main

4. Install and setup Cassandra(Version 3.11.9)

Install Cassandra

To install Cassandra , follow the below steps

• Download the tarball file for the version 3.11.9 : 

  wget -c https://archive.apache.org/dist/cassandra/3.11.9/apache-cassandra-3.11.9-bin.tar.gz  
  

• Unpack the tarball :  
  tar -xvf  apache-cassandra-3.11.9-bin.tar.gz
  
  

• Step into apache-cassandra-3.11.9 folder :

  cd apache-cassandra-3.11.9
  
  

• To start Cassandra, run the following command

  sudo bin/cassandra
  
  

• To verify that Cassandra is up and running, enter the following command :

  bin/nodetool status

Configure Cassandra

Note :  For the next step, make sure that python is installed. Because cqlsh is python based command line tool. If python is not installed , use this command : sudo apt install python2.7

• schema.cql file is available in 'xconf-angular-admin/src/test/resources/schema.cql'. We can use this cql file to create a corresponding schema . Open another terminal , step into apache-cassandra-3.11.9 folder and  run the following command

          bin/cqlsh -f  { path-to-the-schem.cql file}

         eg : bin/cqlsh  -f ~/xconf/xconfserver/xconf-angular-admin/src/test/resources/schema.cql

• To check if tables are created successfully, we can use cqlsh 
• To start cqlsh, step into cassandra folder and enter the command:
  

   bin/cqlsh 

• It gives cassandra cqlsh prompt as output. To check if all the tables are present enter the following commands in cqlsh prompt :

          USE "demo"; 

          DESCRIBE KEYSPACE;

• To exit from cqlsh prompt, we can use the command : quit

Configuration and Service startup

1. Start Cassandra Service

• To start an Xconf application, start the Cassandra server by executing the following commands:  
  

          cd apache-cassandra-3.11.9
          sudo bin/cassandra

• Status of xconf server can be verified by using  the command

          bin/nodetool status

2. Configure and Start Application Services

Build and run steps mentioned below is based on these steps - https://github.com/rdkcentral/xconfserver#run-application.

a. Build Project

• Go to the xconf-server folder and run the following command to download all dependencies.

          cd ~/xconf/xconfserver

• Run the following command from the xconfserver folder 

          mvn clean install

b. Configure Angular Admin UI

• For first time application deployment, create a "service.properties"  file under the path xconfserver/xconf-angular-admin/src/main/resources/service.properties with the following contents. The sample service.properties file will be available in xconf-angular-admin/src/test/resources/service.properties, the below content is taken from there and modified with change in cassandra port.

• Go to xconf-angular-admin folder

         cd ~/xconf/xconfserver/xconf-angular-admin

• Run the following command from xconf-angular-admin folder
  

      mvn jetty:run -DappConfig=${path-to-service-properties} -f pom.xml

• For first time run only, we need to specify the path to service.properties . For the subsequent runs execute the below command in the folder xconfserver/xconf-angular-admin:          mvn jetty:run

To run the admin UI launch it as http://<XCONF-SERVER-IP>:19093/admin/  in any browser. This will redirect to the login page.

If the user wants both read and write permissions ,then enter username and password for the login as admin and admin respectively

If the user wants only read permissions ,then enter username and password for the login as user and user respectively.

c. Configure DataService

• For first time application deployment, create a "service.properties"  file under the path xconfserver/xconf-dataservice/src/main/resources/service.properties with the following contents. The sample service.properties file will be available in xconf-dataservice/src/test/resources/service.properties, the below content is taken from there and modified with change in cassandra port.
  

  cassandra.keyspaceName=demo cassandra.contactPoints=127.0.0.1 cassandra.username= cassandra.password= cassandra.port=9042 cassandra.authKey= dataaccess.cache.tickDuration=60000 dataaccess.cache.retryCountUntilFullRefresh=10 dataaccess.cache.changedKeysTimeWindowSize=900000 dataaccess.cache.reloadCacheEntries=false dataaccess.cache.reloadCacheEntriesTimeout=1 dataaccess.cache.reloadCacheEntriesTimeUnit=DAYS dataaccess.cache.numberOfEntriesToProcessSequentially=10000 dataaccess.cache.keysetChunkSizeForMassCacheLoad=500 dataaccess.cache.changedKeysCfName=XconfChangedKeys4

  
  

• Step into xconf-dataservice folder
   

      cd ~xconf/xconfserver/xconf-dataservice

• Run the following command from xconf-angular-admin folder
  

      mvn jetty:run -DappConfig=${path-to-service-properties} -f pom.xml
  
  

• For first time run only, we need to specify the path to service.properties . For the subsequent runs execute the below command in the folder xconfserver/xconf-dataservice:

         mvn jetty:run      
  

To launch the application go to http://<XCONF-SERVER-IP>:19092/queries/environments .  To verify, add an entry in the environments tab of the xconf admin application and check whether the same is updated here in data service.

• Data Service Endpoints

  
  The endpoints available in data service is listed and described here - https://github.com/rdkcentral/xconfserver#endpoints

NOTE: To run the Admin UI and data service applications in background start jetty server as follows: nohup mvn jetty:run &

3. Expected build issues

1. The below exceptions are observed during the mvn clean install . This is an exception from unit test , but the tests will run successfully. The execution may stall for some time but the build will be successful and the application can be launched successfully. So it can be ignores as well.

Solution : This exception can be resolved by copying the .so file to the path /usr/lib.

Configuration and Validation of services

Admin UI Common Configuration

Below steps will affect all the features in xconf system and should be configured after initial setup. Go to the steps given in site navigation step and press on create button to create new entries.

Define Environments

Define Models

Define MAC List (This will be used to target certain list of MACs against a particular feature configuration)

Define IP list

Feature Validation(RFC)

Configuration

RDK Feature control configuration can be added by adding below 2 sections

1. Define the Feature
2. Define the Feature Rule

• Define the feature

A new feature can be defined via RFC-> Feature -> Create. 'Feature Name' should be unique and understandable, 'Config data' should be key value pairs.

• Define the Feature Rule

Feature rule is to map devices to a particular feature. A new feature rule can be created via RFC->feature rule -> Create

Verification

1. Verification of feature and feature rule via test page.

After creating the feature and feature rule, go to RFC->Test page and give a parameter that will match the one of the rules that you have created. Then matched rule and the json response will be displayed like below.

    2.  Verification via curl command

The curl command mocks the request being sent from an STB like below and sample response is also given. It can be given as a curl command or as a get request via postman or browser

eg : curl 'http://<XCONF_IP>:19092/featureControl/getSettings?estbMacAddress= B8:27:EB:94:71:82’ Here the feature mapped to this particular mac address will be obtained

Sample Response:

{

    "featureControl": {

        "features": [

            {

                "name": "EmulatorFeature",

                "effectiveImmediate": true,

                "enable": false,

                "configData": {

                    "ENABLE_AAMP": "false"

                },

                "featureInstance": "AAMP-mock feature"

            }

        ]

    }

}

Client end verification (RPI)

Verification and setup from RPI

Feature Validation (LogUpload)

Configuration

1. Create upload repository via DCM->Upload repository -> Create.   Here we can add where to configure the log upload, ie the upload url and its protocol.

2. Create rule via DCM->Formulas->Create. 

3.  Once you save the created formula  , then a ‘define Settings’ tab with ‘Create Device Settings’, ‘Create log settings’, Create VOD settings’ will be available

4.  Click on 'Create Device Settings' tab and edit the  Device Settings.

5.  Edit the Log upload Setting (Create schedule & Add the upload repository created beforehand).

Verification

1. Verification of log upload settings test page.

After creating the feature and feature rule, go to DCM->Test page and give a parameter that will match the one of the formulas that you have created. Then matched rule and the settings will be displayed like below

2. Verification via curl command

The curl command mocks the request being sent from an STB like below and sample response is also given. It can be given as a curl command or as a get request via postman or browser

eg : curl 'http://<XCONF_IP>: 19092/loguploader/getSettings?estbMacAddress=B8:27:EB:94:71:82’. 

Sample response : 

{

    "urn:settings:GroupName": "RPI_Device_Settings",

    "urn:settings:CheckOnReboot": true,

    "urn:settings:CheckSchedule:cron": "10 00 * * *",

    "urn:settings:CheckSchedule:DurationMinutes": 0,

    "urn:settings:LogUploadSettings:Message": null,

    "urn:settings:LogUploadSettings:Name": "Log upload settings test",

    "urn:settings:LogUploadSettings:NumberOfDays": 5,

    "urn:settings:LogUploadSettings:UploadRepositoryName": "Upload repository test",

    "urn:settings:LogUploadSettings:RepositoryURL": "http://35.155.171.121/xconf/logupload.php",

    "urn:settings:LogUploadSettings:UploadOnReboot": false,

    "urn:settings:LogUploadSettings:UploadImmediately": false,

    "urn:settings:LogUploadSettings:upload": true,

    "urn:settings:LogUploadSettings:UploadSchedule:cron": "10 00 * * *",

    "urn:settings:LogUploadSettings:UploadSchedule:levelone:cron": null,

    "urn:settings:LogUploadSettings:UploadSchedule:leveltwo:cron": null,

    "urn:settings:LogUploadSettings:UploadSchedule:levelthree:cron": null,

    "urn:settings:LogUploadSettings:UploadSchedule:DurationMinutes": 0,

    "urn:settings:VODSettings:Name": null,

    "urn:settings:VODSettings:LocationsURL": null,

    "urn:settings:VODSettings:SRMIPList": null

}

Client end verification (RPI)

Feature Validation (Telemetry)

1. Telemetry configuration can be done by adding a permanent profile which contains below objects
   1. Upload repository
   2. Profile options (Header, content, frequency etc.)
2. Creating a targeting rule which is basically mapping the profile to a set of MAC/IP/Device etc.

Configuration

1. Create a permanent profile Telemetry - > Permanent Profiles -> Create

     2. Once you save the permanent profile, you will get a message overlay ‘Profile added to the pending changes’. Then go to Changes -> select the profile you create -> Click on “Approve selected changes”.
Then the permanent profile will be listed under Telemetry - > Permanent profiles

     3. Create targeting rule via Telemetry -> Targeting rules. Targeting rules is to map the profiles with rules.

Verification

1. Verification of telemetry test page.

After creating the permanent profile and targeting rules, go to Telemetry->Test page and give a parameter that will match the one of the rule that you have created. Then matched rule will be displayed like below.

      2.  Verification via curl command

The curl command mocks the request being sent from an STB like below and sample response is also given. It can be given as a curl command or as a get request via postman or browser. The same url used for logupload verification is used here, the response will have telemetry settings data like below (urn:settings:TelemetryProfile)

eg : curl 'http://<XCONF_IP>: 19092/loguploader/getSettings?estbMacAddress=B8:27:EB:94:71:82’

Sample Response :

{

    "urn:settings:GroupName": "RPI_Device_Settings",

    "urn:settings:CheckOnReboot": true,

    "urn:settings:CheckSchedule:cron": "10 00 * * *",

    "urn:settings:CheckSchedule:DurationMinutes": 0,

    "urn:settings:LogUploadSettings:Message": null,

    "urn:settings:LogUploadSettings:Name": "Log upload settings test",

    "urn:settings:LogUploadSettings:NumberOfDays": 5,

    "urn:settings:LogUploadSettings:UploadRepositoryName": "Upload repository test",

    "urn:settings:LogUploadSettings:RepositoryURL": "http://35.155.171.121/xconf/logupload.php",

    "urn:settings:LogUploadSettings:UploadOnReboot": false,

    "urn:settings:LogUploadSettings:UploadImmediately": false,

    "urn:settings:LogUploadSettings:upload": true,

    "urn:settings:LogUploadSettings:UploadSchedule:cron": "10 00 * * *",

    "urn:settings:LogUploadSettings:UploadSchedule:levelone:cron": null,

    "urn:settings:LogUploadSettings:UploadSchedule:leveltwo:cron": null,

    "urn:settings:LogUploadSettings:UploadSchedule:levelthree:cron": null,

    "urn:settings:LogUploadSettings:UploadSchedule:DurationMinutes": 0,

    "urn:settings:VODSettings:Name": null,

    "urn:settings:VODSettings:LocationsURL": null,

    "urn:settings:VODSettings:SRMIPList": null,

    "urn:settings:TelemetryProfile": {

        "id": "b160a5a8-c753-4e71-b267-992965a4437b",

        "telemetryProfile": [

            {

                "header": "Firewall",

                "content": "starting firewall service",

                "type": "FirewallDebug.txt",

                "pollingFrequency": "1"

            }

        ],

        "schedule": "5",

        "expires": 0,

        "telemetryProfile:name": "Permanent profile test",

        "uploadRepository:URL": "http://35.155.171.121/xconf/logupload.php",

        "uploadRepository:uploadProtocol": "HTTP"

    }

}

Client end verification (RPI)

Feature Validation (Firmware update)

Configuration

1.  Firmware config can be created via Firmware - > Firmware Configs -> Create. Here we can define the file name and version of the image/firmware that need to be downloaded to the CPE device. The models that we defined in Common → Models section will be available here, We can select the needed models by clicking on it.

2.  To create a firmware template, go to Firmware → Firmware templates. Enter the ID name. Select priority from the 'Priority' drop down menu. Add conditions

3.  Firmware rule can be create via Firmware -> Firmware rules -> Create. On clicking on Create button, a list of templates will be presented. We can select the required template (There will be default templates like ENV_MODEL_RULE, IP_RULE, MAC_RULE etc. and also the custom templates created from Firmware->Firmware template -> Create).

Build Condition should be added in the Build Conditions tab. To add firmware config , go to the 'Actions' tab and select the firmware config from 'Firmware config' drop down list.

4. From where to download the image can be specified via Firmware -> Download location filter-> Edit, where we can specify the location from where we can download the firmware. Enter the FQDN and Full http location for the firmware download server . For TFTP , we can enter the IP address and percentage. For where TFTP applies, a device will get back one of the locations below based on the percent listed for the location

Import and Export Feature

We can import and export all the configuration data from the UI itself.  This feature can be primarily used for transferring the configuration data from one xconfserver setup to the other one. The export and import

data need to be done separately for the Application - stb, xhome, rdkcloud and sky for all the pages except those in Common tab.

To export data from a page, Click on ‘Export All’ button in that page.  The configuration data will be downloaded as a json file.

To import data :

1. Open the dropdown menu next to ‘Create’ button.
2. From the drop down menu, click on ‘Import’.
3. A new page will be displayed with option to browse the location of the json file to be imported. Select the file that need to be imported.
4. All the data from the file will be listed in the page.
5. Click on ‘Import All’ to import all data