Telemetry(1.0) in RPI User manual Broadband - 2019 M9

Telemetry Introduction:

Telemetry is the automatic recording and transmission of data from remote or inaccessible sources to an IT system in a different location for monitoring and analysis.

RDKB Telemetry Components:

1. Xconf Server
2. Rpi Board with RDK Broadband image
3. Tftp Server

RDKB Telemetry Architecture:

Rpi board will connect to Xconf Server to get the device configuration information, telemetry profile information and log upload repository (ip address of tftp server). It will search for the markers (contents mentioned in the permanent profile) in the mentioned log file. If it is found in the log file, it will process those information and create a text file and upload that file in the tftp server. This way all the critical data related to device health will be available on timely basis in the tftp server for further analysis.

XConf Server Configuration procedure:

1. Login to Xconf Server and go to DCM menu item and click on “Formulas” sub menu

2. Click on Create button

3. i) Provide “Name”, “Description”, and build conditions and save the formula.
ii) Verify Percentage and Priority field(normally it displays the sequence number of formula, can be changed to make the formula higher/lower priority) is auto field up with their respective values
iii) Go to “Build condition” section and click on the first empty field to get all the available field displayed there
iv) Select estbMacAddress and provide the MAC address of Rpi board

4. Verify the created formula (ex. RPI_formula22Aug) displayed in the formula list.

5. Click on Edit formula for the created formula (ex. RPI_formula_22Aug).

6. Edit formula page opens.

7. Click on “Create Device Settings” and fill up the fields as displayed in the screen below and click on save button:

8. Click on DCM menu then “Upload repository” submenu and click on “Create” button

9. Provide inputs as mentioned below screen and save the upload repository.

10. Click on Formula and go back to the Edit formula page of the created formula (ex. RPI_formula_22Aug) and click on “Create Log Upload settings” button. Upload repository drop down menu field will show all the upload repositories including the created one (ex. Chennai_22Aug_repo)

11. Provide inputs as mentioned in the screen below:

12. Save the “Edit Formula” page

13. Verify the Formula page for the created formula (ex. RPI_formula_22Aug) where View Settings column displays the “Device” and “LogUpload”

14. Go to Telemetry menu and click on “Permanent Profile”

15. Click on “Create” button to create permanent profile with different markers.

Telemetry profile entries - 

First field is “Header” - This is the name for the particular logs collection. This is a custom name and can be decided based on the use case or error condition or any other scenario.
Second field is “Content” - This is the "log message string" that appears in the RDK logs.
Third field is “type” - This is the name of the log file where the above log message string needs to be searched.
Fourth field is “Skip frequency” - This is a skip frequency. It can have values like 0, 1, 2 etc.
                     0 - This particular log message is never skipped.
                     1 - Skip every alternate occurrence of this log message.
                     2 - Skip 2 consecutive log message occurrences and then collect 1 occurrence and then again skip 2 occurrences.

Based on the R-Pi board supported markers available,we have selected few of them as mentioned below:

16. Save Permanent profile (ex. Perm_Prof_22Aug) and it will display in the Permanent profiles page

17. Click on “Telemetry” and then click on “Targeting Rules”

18. Click on “Create” button, provide Rule name and select different rule parameters and map the Permanent profile with this rule by selecting “Bound profile” drop down menu. Save the Targeting rule.

19. Verify the created Targeting rule displayed on the Targeting rule page

RPi device side configuration and testing process:

Below steps needed after flashing for the first time:

1. Flash the Telemetry image and load it on RPi board
2. Configure the CLOUDURL parameter and Log Server/Http Server parameter by providing IP address of XConf server in the two mentioned files /etc/include.properties and /etc/dcm.properties respectively.

For ex. In include.properties file:

CLOUDURL=http://35.155.171.121:9092/xconf/swu/stb?eStbMac=

In dcm.properties file:

DCM_LOG_SERVER_URL=http://35.155.171.121:9092/loguploader/getSettings
DCM_HTTP_SERVER_URL=http://35.155.171.121/xconf/telemetry_upload.php
DCM_LA_SERVER_URL=http://35.155.171.121/xconf/logupload.php

3. After board boot up, disable log rotation by using the command: 

systemctl disable rdkbLogMonitor

4. Verify the image version: cat /version.txt
5. Perform reboot: reboot -f

Important Note:

Disabling log monitor is required because log rotation feature will wipe out logs when threshold reached, due to which telemetry feature can't able to identify the event count.

Telemetry functionality  requires rdk logs.

To bring both log rotation and telemetry feature at same time, user needs to increase the threshold size of max size which is used by log rotation feature  . Please follow Log Rotation Support in RPI User manual - Broadband - 2019 M8 to change threshold size of log rotation.

Below steps are to check telemetry log whenever box boots up ( after first time boot ) :

1. Verify tftp server is running and XConf DCM and Telemetry Permanent profile is configured properly (see Telemetry User manual for XConf side configuration for Telemetry)
2. After reboot, verify Telemetry log: cat /rdklogs/logs/telemetry.log
3. Verify that rtl_json.txt and zipped log files for all other logs displayed in tftp server
4. Verify that rtl_json file and zipped log files for all other logs displayed in AWS(http) server(http://35.155.171.121/xconf/upload)

Future Enhancement:

1. We are not considering the cron job time setting for log collection and upload.
2. We are not considering the polling frequency parameter in Telemetry profile entries section of Permanent profile page under Telemetry menu. As of now polling frequency value is considered as 0.

        These things are under future development.