...

Architecture and working principle

Why we need telemetry upload system

In RDK Telemetry, the process of ingestion refers to the collection of telemetry data from various sources within the RDK ecosystem, such as applications, subsystems, and network interfaces. This data is then brought into a central system for processing and analysis. The transformation aspect involves converting this raw telemetry data into a format that is useful for analysis. This may include structuring, filtering, and enriching the data to make it more meaningful for specific uses, like performance monitoring or issue diagnosis. The transformed data can then be used for various purposes, including troubleshooting, network optimization, and improving user experiences. This process ensures that the vast amounts of data generated by RDK devices are effectively utilized to enhance service delivery and operational efficiency.

This reference setup is a simpler solution to upload and view telemetry data from RDK devices based on the Spring boot  based Microservice – Elasticsearch DB – Kibana architecture which are used in data processing, data storage and data visualization, respectively.

Architecture and a brief overview of the work flow

• Telemetry is configured in the RDKB /RDKV devices with on xconf(Single profile telemetry) based telemetry or telemetry2 multiprofile based telemetry.
• The upload URL will be configured as the Telemetry-collector microservice URL , So data will be uploaded to the microservice from the boxes.
• Telemetry microservice is a springboot based microservice that will receive the telemetry data, parses it and sends the processed data to Elastic Search
• Elastic search DB is  where the data is stored, It  is deployed along with Kibana.
• Kibana is a data visualization tool and  user can analyze and filter out the data stored in Elasticsearch DB , with Kibana
• Kibana Admin user will create dashboards based on the requirements from the end user.

Advantages of the reference telemetry setup

...

3.Performance Analysis: By tracking various metrics and KPIs, telemetry dashboards enable detailed performance analysis. This can lead to insights into trends, patterns, and potential areas for improvement.

How to setup the telemetry upload

...

Server Prerequisites

VM :  Ubuntu 20.04 +

Given below is the VM Configuration for the RDKM reference setup, the configs needs to be modified as per the requirement

 VM Configuration

• RAM : 16GB
• Disk space :  128 GB
• Ubuntu : 20.04.6 LTS )

Install Java

1. Run the following commands from the command line

$ sudo apt update
$ sudo apt install openjdk-8-jdk

      2. Check Java installation by running the below command

$ java -version

      3. The output should look something like this:

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

...

Install Maven

1. Install Maven using the command below:

sudo apt-get -y install maven

     2. Verify the Apache Maven version using the following command:

mvn -version

Checkout and create war

Check out the code from  : https://github.com/rdkcentral/telemetry-data-collector

...

Elasticsearch configuration in application.properties

There are  upload APIS for RDKB and RDKV devices separately in the microservice. So we need to  setup elastic search URL and indexes corresponding to rdkb and rdkv in  telemetry-collector\src\main\resources\application.properties. Sample application.properties with reference setup Elastic search given. Change the elastic search URL corresponding to your elastic search-Kibana setup

rdkv.index=rdkv-telemetry
rdkb.index=rdkb-telemetry
elasticsearch.url=http://52.71.72.93:9200/  <Change this accordingly>


server.port=8080

      2.   Go to telemetry-collector and run the below maven command

mvn clean install

     3. War file- telemetry-collector.war will be created in the folder - telemetry-collector\target 

Install Tomcat 9

1. Download Tomcat 9 from the Apache Tomcat Archive with the below given step

wget -c https://downloads.apache.org/tomcat/tomcat-9/v9.0.85/bin/apache-tomcat-9.0.85.tar.gz

       2. Extract the Tomcat archive to the directory in which you want to deploy the Automatics Tools

sudo tar -xvf apache-tomcat-9.0.85.tar.gz -C /opt/automatics/

 Deployment of the war  

1. Copy the telemetry-collector.war file to the apache-tomcat/webapps folder.
2. Navigate to apache-tomcat/bin directory and start the server by executing command.

./startup.sh

     3. Sample output is given below

Image Removed

system

Elastic

...

Search -  Kibana Setup

Server Prerequisites

VM :  Ubuntu 20.04 +

You can either deploy the EK setup in the same VM or use another VM. For reference setup, we have used two separate  ubuntu server instances

...

• RAM : 16GB
• Disk space :  300 GB
• Ubuntu : 20.04.6 LTS 

Install Java

1. Run the following commands from the command line

...

sudo systemctl start elasticsearch.service

14.  To enable Elasticsearch to start on boot:

sudo systemctl enable elasticsearch.service

15.  Use the curl command to test your configuration. Enter the following:

curl -X GET "localhost:9200"

16. The name of your system should display, and elasticsearch for the cluster name. This indicates that Elasticsearch is functional and is listening on port 9200.

Install Kibana

1.  Run the following command to install Kibana:

sudo apt-get install kibana

2. Allow the process to finish. Once finished, it’s time to configure Kibana. Next, open the kibana.yml configuration file for editing:

sudo nano /etc/kibana/kibana.yml

3.  Delete the # sign at the beginning of the following lines to activate them:

#server.port: 5601

#server.host: "your-hostname"

#elasticsearch.hosts: ["http://localhost:9200"]

4. The above-mentioned lines should look as follows:

server.port: 5601

server.host: "localhost"

elasticsearch.hosts: ["http://localhost:9200"]

5. Save the file

6.  Start the Kibana service:

sudo systemctl start kibana

7. Next, configure Kibana to launch at boot if you want

sudo systemctl enable kibana

8. If the UFW firewall is enabled on your Ubuntu system, you need to allow traffic on port 5601 to access the Kibana dashboard. In a terminal window, run the following command:

sudo ufw allow 5601/tcp

9. To access Kibana, open a web browser and browse to the following address:

http://<serverIP>:5601

10 .If you receive a “Kibana server not ready yet” error, Wait for some time ,check if the Elasticsearch and Kibana services are active. 

Configure Kibana for the dashboard

Create required index for RDKB and RDKV as per the requirement

1. We need to create index for RDKV and RDKB as per the requirement
2. To create an index from the command line, run the below given curl command for creating RDKB telemetry index in elastic search DB

curl -X PUT "localhost:9200/rdkb-telemetry" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}


'

     To create RDKV index in elastic search DB, run the following command

curl -X PUT "localhost:9200/rdkv-telemetry" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}
'

     3. To create an index from the  Kibana 

•   Click on menu
  
  Image Added

• Go to DevTools under 'Management tools'
  
  Image Added

• Go to 'Console' tab

Image Added

• Paste the below code and click on the execute button to create index for RDKB

PUT /rdkb-telemetry
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}

• Paste the below code and click on the execute button to create index for RDKB

PUT /rdkv-telemetry
{
  "mappings": {
    "properties": {
      "Time": {
        "type": "date" ,
         "format": "yyyy-MM-dd HH:mm:ss"
      }
    }
  }
}

Image Added

Configuring the index for use in the dashboard

1. Go to the menu Stack management →Index patterns. Click on 'Create index pattern'

Image Added

   2.  If you want to setup RDKB telemetry dashboard, add name as rdkb-telemetry, Select 'Time' from the drop down menu, You will get a message like this - 'Your index pattern matches 1 source.' Click on 'Create index pattern'

If you want to setup RDKV telemetry dashboard, add name as rdkv-telemetry, Select 'Time' from the drop down menu, You will get a message like this - 'Your index pattern matches 1 source.' Click on 'Create index pattern'.

Image Added

Telemetry collector microservice application

Server Prerequisites

VM :  Ubuntu 20.04 +

Given below is the VM Configuration for the RDKM reference setup, the configs needs to be modified as per the requirement

 VM Configuration

• RAM : 16GB
• Disk space :  128 GB
• Ubuntu : 20.04.6 LTS )

Install Java

1. Run the following commands from the command line

$ sudo apt update
$ sudo apt install openjdk-8-jdk

      2. Check Java installation by running the below command

$ java -version

      3. The output should look something like this:

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-8u252-b09-1ubuntu1-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

War creation

Install Maven

1. Install Maven using the command below:

sudo apt-get -y install maven

     2. Verify the Apache Maven version using the following command:

mvn -version

Checkout and create war

Check out the code from  : https://github.com/rdkcentral/telemetry-data-collector

Elasticsearch configuration in application.properties

There are  upload APIS for RDKB and RDKV devices separately in the microservice. So we need to  setup elastic search URL and indexes corresponding to rdkb and rdkv in  telemetry-collector\src\main\resources\application.properties. Sample application.properties with reference setup Elastic search given. Change the elastic search URL corresponding to your elastic search-Kibana setup

rdkv.index=rdkv-telemetry
rdkb.index=rdkb-telemetry
elasticsearch.url=http://52.71.72.93:9200/  <Change this accordingly>


server.port=8080

      2.   Go to telemetry-collector and run the below maven command

mvn clean install

     3. War file- telemetry-collector.war will be created in the folder - telemetry-collector\target 

Install Tomcat 9

1. Download Tomcat 9 from the Apache Tomcat Archive with the below given step

wget -c https://downloads.apache.org/tomcat/tomcat-9/v9.0.85/bin/apache-tomcat-9.0.85.tar.gz

       2. Extract the Tomcat archive to the directory in which you want to deploy the Automatics Tools

sudo tar -xvf apache-tomcat-9.0.85.tar.gz -C /opt/automatics/

 Deployment of the war  

1. Copy the telemetry-collector.war file to the apache-tomcat/webapps folder.
2. Navigate to apache-tomcat/bin directory and start the server by executing command.

./startup.sh

     3. Sample output is given below

Image Added

     4. Logs can be found in the file -  apache-tomcat<>/logs/catalina.out

Telemetry upload APIS

The APis support upload of both telemetry (Single profile) and telemetry(Multi Profile) which are in different JSON schema/format

RDKB telemetry upload API : 

http://<Server IP/FQDN>:8080/telemetry-collector/rdkb-collector

RDKV telemetry upload API

http://<Server IP/FQDN>:8080/telemetry-collector/rdkv-collector

Reference setup details

Server details

We have two servers which are used for microservice deployment and Elasticsearch-Kibana setup

Telemetry collector VM - 

IP - 52.0.158.162

FQDN :  telemetrycollector.rdkcentral.com

RAM : 16GB

Disk space :  128 GB

Ubuntu : 20.04.6 LTS

Telemetry Dashboard VM - 

IP- 52.71.72.93

FQDN :  telemetrydashboard.rdkcentral.com

RAM : 16GB

Disk space : 300 GB +

Ubuntu : 20.04.6 LTS

Telemetry collector

These are the telemetry upload URLs configured for RDKB and RDKV

RDKB : https://telemetrycollector.rdkcentral.com/telemetry-collector/rdkb-collector

RDKV: https://telemetrycollector.rdkcentral.com/telemetry-collector/rdkv-collector

Telemetry dashboard

We have currently setup RDKB and RDKV dashboards in the Kibana instance. Given below are the details

RDKB dashboards

Device Info   - https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/d7462060-ab98-11ee-9d36-e37c9e3d744b?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

Device Performance - https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/714270f0-a933-11ee-9d36-e37c9e3d744b?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

Error Markers - https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/8953c6a0-a954-11ee-9d36-e37c9e3d744b?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

RDKV dashboards

https://telemetrydashboard.rdkcentral.com:5601/app/dashboards#/view/30d28b50-95c3-11ee-818a-3bf22ddefffd?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-2w,to:now))

Reference

• https://www.elastic.co/guide/en/kibana/7.17/index.html
• https://phoenixnap.com/kb/how-to-install-elk-stack-on-ubuntu