Table of Contents

...

Architecture and working principle

Why we need telemetry upload system

In RDK Telemetry, the process of ingestion refers to the collection of telemetry data from various sources within the RDK ecosystem, such as applications, subsystems, and network interfaces. This data is then brought into a central system for processing and analysis. The transformation aspect involves converting this raw telemetry data into a format that is useful for analysis. This may include structuring, filtering, and enriching the data to make it more meaningful for specific uses, like performance monitoring or issue diagnosis. The transformed data can then be used for various purposes, including troubleshooting, network optimization, and improving user experiences. This process ensures that the vast amounts of data generated by RDK devices are effectively utilized to enhance service delivery and operational efficiency.

This reference setup is a simpler solution to upload and view telemetry data from RDK devices based on the Spring boot  based Microservice – Elasticsearch DB – Kibana architecture which are used in data processing, data storage and data visualization, respectively.

Architecture and a brief overview of the work flow

• Telemetry is configured in the RDKB /RDKV devices with on xconf(Single profile telemetry) based telemetry or telemetry2 multiprofile based telemetry.
• The upload URL will be configured as the Telemetry-collector microservice URL , So data will be uploaded to the microservice from the boxes.
• Telemetry microservice is a springboot based microservice that will receive the telemetry data, parses it and sends the processed data to Elastic Search
• Elastic search DB is  where the data is stored, It  is deployed along with Kibana.
• Kibana is a data visualization tool and  user can analyze and filter out the data stored in Elasticsearch DB , with Kibana
• Kibana Admin user will create dashboards based on the requirements from the end user.

Advantages of the reference telemetry setup

...

3.Performance Analysis: By tracking various metrics and KPIs, telemetry dashboards enable detailed performance analysis. This can lead to insights into trends, patterns, and potential areas for improvement.

How to setup the telemetry upload system

Elastic Search -  Kibana Setup

Server Prerequisites

VM :  Ubuntu 20.04 +

You can either deploy the EK setup in the same VM or use another VM. For reference setup, we have used two separate  ubuntu server instances

Given below is the VM Configuration for the RDKM reference setup,  the configs needs to be modified as per the requirement

 VM Configuration

• RAM : 16GB
• Disk space :  300 GB
• Ubuntu : 20.04.6 LTS 

Install Java

1. Run the following commands from the command line

$ sudo apt update
$ sudo apt install openjdk-8-jdk

      2. Check Java installation by running the below command

$ java -version

      3. The output should look something like this:

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

Install NGINX

1. Install Nginx by running the following command from command line:

sudo apt-get install nginx

     2. If prompted, type y and hit Enter for the process to finish.

Install ElasticSearch

Elastic repositories enable access to all the open-source software in the ELK stack. To add them, start by importing the GPG key.

1 . Enter the following into terminal window to import the PGP key for Elastic:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -

2.The system should respond with OK,

3. Next, install the apt-transport-https package:

sudo apt-get install apt-transport-https

4. Add the Elastic repository to your system’s repository list:

echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee –a /etc/apt/sources.list.d/elastic-7.x.list

5. Prior to installing Elasticsearch, update the repositories by entering:

sudo apt-get update

6. Install Elasticsearch with the following command:

sudo apt-get install elasticsearch

7. Elasticsearch uses a configuration file to control how it behaves. Edit the file.

sudo nano /etc/elasticsearch/elasticsearch.yml

8. You should see a configuration file with several different entries and descriptions. Scroll down to find the following entries:

#network.host: 192.168.0.1
#http.port: 9200

9.Uncomment the lines by deleting the hash (#) sign at the beginning of both lines and replace 192.168.0.1 with localhost.

network.host: localhost

http.port: 9200

10 . Just below, find the Discovery section. We are adding one more line, as we are configuring a single node cluster:

discovery.type: single-node

11. By default, JVM heap size is set at 1GB. It is recommended setting it to no more than half the size of your total memory. Open the following file for editing:

sudo nano /etc/elasticsearch/jvm.options

12. Find the lines starting with -Xms and -Xmx  and set the value. In reference setup we set it as 4gb

13. Start the Elasticsearch service by running a systemctl command: 

It may take some time for the system to start the service. There will be no output if successful.

sudo systemctl start elasticsearch.service

14.  To enable Elasticsearch to start on boot:

sudo systemctl enable elasticsearch.service

15.  Use the curl command to test your configuration. Enter the following:

curl -X GET "localhost:9200"

16. The name of your system should display, and elasticsearch for the cluster name. This indicates that Elasticsearch is functional and is listening on port 9200.

Install Kibana

1.  Run the following command to install Kibana:

sudo apt-get install kibana

2. Allow the process to finish. Once finished, it’s time to configure Kibana. Next, open the kibana.yml configuration file for editing:

sudo nano /etc/kibana/kibana.yml

3.  Delete the # sign at the beginning of the following lines to activate them:

#server.port: 5601

#server.host: "your-hostname"

#elasticsearch.hosts: ["http://localhost:9200"]

4. The above-mentioned lines should look as follows:

server.port: 5601

server.host: "localhost"

elasticsearch.hosts: ["http://localhost:9200"]

5. Save the file

6.  Start the Kibana service:

sudo systemctl start kibana

7. Next, configure Kibana to launch at boot if you want

sudo systemctl enable kibana

8. If the UFW firewall is enabled on your Ubuntu system, you need to allow traffic on port 5601 to access the Kibana dashboard. In a terminal window, run the following command:

sudo ufw allow 5601/tcp

9. To access Kibana, open a web browser and browse to the following address:

http://<serverIP>:5601

10 .If you receive a “Kibana server not ready yet” error, Wait for some time ,check if the Elasticsearch and Kibana services are active. 

Configure Kibana for the dashboard

Create required index for RDKB and RDKV as per the requirement

1. We need to create index for RDKV and RDKB as per the requirement
2. To create an index from the command line, run the below given curl command for creating RDKB telemetry index in elastic search DB

curl -X PUT "localhost:9200/rdkb-telemetry" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}


'

     To create RDKV index in elastic search DB, run the following command

curl -X PUT "localhost:9200/rdkv-telemetry" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}
'

     3. To create an index from the  Kibana 

•   Click on menu
  
  Image Added

• Go to DevTools under 'Management tools'
  
  Image Added

• Go to 'Console' tab

Image Added

• Paste the below code and click on the execute button to create index for RDKB

PUT /rdkb-telemetry
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}

• Paste the below code and click on the execute button to create index for RDKB

PUT /rdkv-telemetry
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}

Image Added

Configuring the index for use in the dashboard

1. Go to the menu Stack management →Index patterns. Click on 'Create index pattern'

Image Added

   2.  If you want to setup RDKB telemetry dashboard, add name as rdkb-telemetry, Select 'Time' from the drop down menu, You will get a message like this - 'Your index pattern matches 1 source.' Click on 'Create index pattern'

If you want to setup RDKV telemetry dashboard, add name as rdkv-telemetry, Select 'Time' from the drop down menu, You will get a message like this - 'Your index pattern matches 1 source.' Click on 'Create index pattern'.

Image Added

Telemetry collector microservice application

Server Prerequisites

VM :  Ubuntu 20.04 +

Given below is the VM Configuration for the RDKM reference setup, the configs needs to be modified as per the requirement

...

• RAM : 16GB
• Disk space :  128 GB
• Ubuntu : 20.04.6 LTS )

Install Java

1. Run the following commands from the command line

...

     4. Logs can be found in the file -  apache-tomcat<>/logs/catalina.out

Elastic Search -  Kibana Setup

Server Prerequisites

VM :  Ubuntu 20.04 +

You can either deploy the EK setup in the same VM or use another VM. For reference setup, we have used two separate  ubuntu server instances

Given below is the VM Configuration for the RDKM reference setup,  the configs needs to be modified as per the requirement

 VM Configuration

Telemetry upload APIS

The APis support upload of both telemetry (Single profile) and telemetry(Multi Profile) which are in different JSON schema/format

RDKB telemetry upload API : 

http://<Server IP/FQDN>:8080/telemetry-collector/rdkb-collector

RDKV telemetry upload API

http://<Server IP/FQDN>:8080/telemetry-collector/rdkv-collector

Reference setup details

Server details

We have two servers which are used for microservice deployment and Elasticsearch-Kibana setup

Telemetry collector VM - 

IP - 52.0.158.162

FQDN :  telemetrycollector.rdkcentral.com

RAM : 16GB

Disk space :  128 GB

Ubuntu : 20.04.6 LTS

Telemetry Dashboard VM - 

IP- 52.71.72.93

FQDN :  telemetrydashboard.rdkcentral.com

RAM : 16GB

Disk space : 300 GB +

Ubuntu 

...

: 20.04.6

...

LTS

Install Java

1. Run the following commands from the command line

$ sudo apt update
$ sudo apt install openjdk-8-jdk

      2. Check Java installation by running the below command

$ java -version

      3. The output should look something like this:

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

Telemetry collector

These are the telemetry upload URLs configured for RDKB and RDKV

RDKB : https://telemetrycollector.rdkcentral.com/telemetry-collector/rdkb-collector

RDKV: https://telemetrycollector.rdkcentral.com/telemetry-collector/rdkv-collector

Telemetry dashboard

We have currently setup RDKB and RDKV dashboards in the Kibana instance. Given below are the details

RDKB dashboards

Device Info   - https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/d7462060-ab98-11ee-9d36-e37c9e3d744b?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

Device Performance - https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/714270f0-a933-11ee-9d36-e37c9e3d744b?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

Error Markers - https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/8953c6a0-a954-11ee-9d36-e37c9e3d744b?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

RDKV dashboards

https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/30d28b50-95c3-11ee-818a-3bf22ddefffd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

Reference

• https://www.elastic.co/guide/en/kibana/7.17/index.html
• https://phoenixnap.com/kb/how-to-install-elk-stack-on-ubuntu

...