1. Required equipment's

• Raspberry pi device

• Ethernet cable

• SD card

• Standard USB keyboard

• TV/Monitor with HDMI input

2. Telemetry

Telemetry is the automatic recording and transmission of data from remote or inaccessible sources to an IT system in a different location for monitoring and analysis.

3. Telemetry Components

• Xconf Server
• Rpi Board with RDK Broadband image
• Tftp Server
  
  

4. Steps to build code 

• repo init -u https://code.rdkcentral.com/r/manifests -b rdk-next -m rdkb-extsrc.xml
• repo sync -j4 --no-clone-bundle
• source meta-cmf-raspberrypi/setup-environment
• choose option 8: meta-cmf-raspberrypi/conf/machine/raspberrypi-rdk-broadband.conf
• bitbake rdk-generic-broadband-image

5. Steps to flash image

• The image will be available under build-raspberrypi-rdk-broadband/tmp/deploy/images/raspberrypi-rdk-broadband
• Copy the image “rdkb-generic-broadband-image_default_<xx>.rootfs.rpi-sdimg” to your server.
• To flash the image
      dmesg
      mount
      umount <partition-mountpoint>
      sudo dd if=<rpi sdimg> of=/dev/sdb bs=1M

• Remove the SD card and insert it to the Raspberry Pi SD card slot
• Bring up the device

6. Xconf server

• Refer XConf Server for XCONF Server setup details.
• Server is already configured and hosted in the AWS VM : 35.155.171.121
• Xconf server page :  http://35.155.171.121:9093/admin/ux/#/
• Common configurations in UI

6.1.1. Define Environments   

                 URL : http://35.155.171.121:9093/admin/ux/#/environment/all
                 Flow : Xconf-server -> Common -> Environments

6.1.2. Define Models 

                            URL : http://35.155.171.121:9093/admin/ux/#/model/all
                            Flow : Xconf-server -> Common -> Models

6.1.3. Define MAC List

                         URL : http://35.155.171.121:9093/admin/ux/#/namespacedlist/MAC_LIST
                         Flow : Xconf-server -> Common -> MAC Lists -> Select the Id -> Add your MAC                            

7. Configuring Telemetry on Xconf server

7.1. Log upload

• 7.1.1. Change Application to stb in top right corner of the window

       

• 7.1.2. Defining the formula

               Sample 

• As soon as the formula is saved , displays the device settings and Log upload settings options
     

• 7.1.3. Device settings

  Name	RDKB_telemetry
  CheckOnReboot	true
  Are active	true
  Expression	2 1 2 1 1

    

  7.1.4. Defining the Upload repositories

  URL	http://35.155.171.121:9093/admin/ux/#/uploadrepository
  Flow	Xconf-server ->DCM -> Upload repositories
  Name	RDKB_telemetry
  Description	RDKB_telemetry
  URL	tftp    : 35.155.171.121 HTTP :  http://35.155.171.121/xconf/logupload.php

  

  7.1.5. Log upload settings

  Name	<Enter the name given during Log upload formula creation>
  Upload On Reboot	true
  Number of Days	1
  Are settings active	true
  Upload Repository	RDKB_telemetry
  Cron Expression	2 1 1 1 1

  

  7.1.6. Test Page
  

  URL	http://35.155.171.121:9093/admin/ux/#/formulas/all
  Path	Xconf-server-> DCM -> Test Page

7.2. Telemetry

7.2.1. Defining Telemetry profile

                Sample :
               

7.2.2. Create Targeting rule 

1. Give the Unique rule name
2. Select the rule. Here , used estbMacAddress as the rule
3. Select the Bound profile (Telemetry Permanent Profile Name)

         Sample

7.2.3. Test Page

1. To test whether the configured profile details are retrieved properly from xconf-server
2. Give the rule specified in “Targeting Rule” section i.e., estbMacAddress here   

       Sample:

8. Configuring Telemetry on Raspberry pi

• Once the board is up , go to /etc/dcm.properties and add the below details

• Restart the dcm-log service using “systemctl restart dcm-log”
• To check the status of the service “systemctl status dcm-log”
• By triggering the service , rpi starts uploading the logs to the xconf-server
• The log upload and Telemetry markers can be uploaded via tftp and http
• The profile can be verified using curl, conf file , through logs and in server as below

1.  CURL

• Syntax : curl -i 'http://<ip>:9092/loguploader/getSettings?estbMacAddress=<MAC>’

2. Configuration file

• Once the profile details are successfully retrieved , the conf file will be created under /tmp which has the server configurations 
• The telemetry markers are compared with the corresponding log files and if the match is found then the markers are uploaded to the http://35.155.171.121/xconf/upload/

• Sample 1: tftp root@RaspberryPi-Gateway:~# cat /tmp/DCMSettings.conf urn:settings:GroupName=RDKB_telemetry urn:settings:CheckOnReboot=true urn:settings:CheckSchedule:cron=2 1 2 1 1 urn:settings:CheckSchedule:DurationMinutes=0 urn:settings:LogUploadSettings:Message=null urn:settings:LogUploadSettings:Name=RDKB_telemetry urn:settings:LogUploadSettings:NumberOfDays=0 urn:settings:LogUploadSettings:UploadRepositoryName=RDKB_telemetry urn:settings:LogUploadSettings:RepositoryURL=tftp://35.155.171.121 urn:settings:LogUploadSettings:UploadOnReboot=true urn:settings:LogUploadSettings:UploadImmediately=false urn:settings:LogUploadSettings:upload=true urn:settings:LogUploadSettings:UploadSchedule:cron=2 1 1 1 1 urn:settings:LogUploadSettings:UploadSchedule:levelone:cron=null urn:settings:LogUploadSettings:UploadSchedule:leveltwo:cron=null urn:settings:LogUploadSettings:UploadSchedule:levelthree:cron=null urn:settings:LogUploadSettings:UploadSchedule:DurationMinutes=0 urn:settings:VODSettings:Name=null urn:settings:VODSettings:LocationsURL=null urn:settings:VODSettings:SRMIPList=null "urn:settings:TelemetryProfile":{"id":"72ac2ca9-b248-4a5e-b120-27ff37382564","telemetryProfile":[{"header" : "Firewall","content" : "staring firewall service","type" : "FirewallDebug.txt","pollingFrequency":"1"}],"schedule":"3","expires":0,"telemetryProfile:name":"RDKB_tele,"uploadRepository:URL":"35.155.171.121","uploadRepository:uploadProtocol":"TFTP"} Sample 2:  http root@RaspberryPi-Gateway:~# cat /tmp/DCMSettings.conf urn:settings:GroupName=RDKB_telemetry urn:settings:CheckOnReboot=true urn:settings:CheckSchedule:cron=2 1 2 1 1 urn:settings:CheckSchedule:DurationMinutes=0 urn:settings:LogUploadSettings:Message=null urn:settings:LogUploadSettings:Name=RDKB_telemetry urn:settings:LogUploadSettings:NumberOfDays=0 urn:settings:LogUploadSettings:UploadRepositoryName=RDKB_telemetry urn:settings:LogUploadSettings:RepositoryURL=http://35.155.171.121/xconf/logupload.php urn:settings:LogUploadSettings:UploadOnReboot=true urn:settings:LogUploadSettings:UploadImmediately=false urn:settings:LogUploadSettings:upload=true urn:settings:LogUploadSettings:UploadSchedule:cron=2 1 1 1 1 urn:settings:LogUploadSettings:UploadSchedule:levelone:cron=null urn:settings:LogUploadSettings:UploadSchedule:leveltwo:cron=null urn:settings:LogUploadSettings:UploadSchedule:levelthree:cron=null urn:settings:LogUploadSettings:UploadSchedule:DurationMinutes=0 urn:settings:VODSettings:Name=null urn:settings:VODSettings:LocationsURL=null urn:settings:VODSettings:SRMIPList=null "urn:settings:TelemetryProfile":{"id":"72ac2ca9-b248-4a5e-b120-27ff37382564","telemetryProfile":[{"header" : "Firewall","content" : "starting firewall service","type" : "FirewallDebug.txt","pollingFrequency":"1"}],"schedule":"3","expires":0,"telemetryProfile:name":"RDKB_tele","uploadRepository:URL":"http://35.155.171.121/xconf/telemetry_upload.php","uploadRepository:uploadProtocol":"HTTP"}

 3. Logs

• Log file to monitor and analyze the files which are uploaded to server
• /rdklogs/logs/dcmscript.log
  TFTP 
  
  
  HTTP
  
  
  
• /rdklogs/logs/telemetry.log
  TFTP
  
  
  HTTP
  
  
  

4. Xconf-Server

• URL : http://35.155.171.121/xconf/upload/

• The tar file (of all logs) and JSON file of telemetry will be uploaded
• Sample 
  TFTP 
  

           HTTP   
          

9. Configuring more entries to a profile

• Go to Telemetry -> Permanent Profile
• Under “Telemetry Profile entries” add your new required markers
  

• Here, added 2 new markers for wifihealth.txt file

                1. WIFI_MAC_2 with the profile name as “Wifihealth_1” and polling frequency as 1
                2. WIFI_BYTESSENTCLIENTS with the profile name as “Wifihealth_2” and polling frequency as 1

• Save
• In Rpi , make sure that you stopped the rdkbLogMontior service , in order to stop the log rotation process (systemctl stop rdkbLogMonitor)
• Now restart the dcm-log service using systemctl restart dcm-log
• With successful response , tar file and JSON file gets uploaded to the xconf-server
• Open the upload JSON file in the browser

• We could see the profile entries with the number of times the specified marker has appeared in the file.

  Sample: {"searchResult":[{"Firewall":"1"},{"Wifihealth_1":"2"},{"Wifihealth_2":"2"},{"Profile":"RDKB"},{"mac":"B8:27:EB:22:16:36"},{"erouterIpv4":"192.168.30.125"},{"erouterIpv6":"null"},{"PartnerId":"RDKM"},{"AccountId":"Unknown"},{"Version":"rdkb-generic-broadband-image_default_20200224150713"},{"Time":"2020-02-26 12:23:51"}]}

  
  

  NOTE : JSON file holds only the number of times the particular marker has appeared in the respective log file. To see the complete log file, need to download the tar file uploaded and extract the files to see the complete logs or value of the parameters

10. Observations :

• Pushing script from XConf

Currently pushing the script from xconf to the target device is not possible due to the security issue , as it may lead to downloading the licensed file to the vulnerable device. The pushing may also lead to removal of file from rootfs , so this is not advised to do for now. Whereas , we can initiate the download from device through tftp/curl command either from the script or from console.

• telemetry target type

 Target type is provided to handle dynamic parameters change . We have 4 target types .

         1.  # 0 if as part of normal execution

         2.  # 1 if initiated due to an XCONF update

        3.  # 2 if forced execution before log upload

        4.  # 3 if modify the cron schedule 

During boot-up or service start , target type update will be initiated .

On each initiation , Whenever it finds the markers in the log files , it provides the search results with marker details , when it doesn't find , it just provides the basic RDKB profile details(without markers). It all depends on the markers availability in the log files.

Currently Type 1 and 2 are supported .

• Empty JSON message

On having the Schedule type in DCM either as "ActNow" or "CronExpression" , with the expression in schedule under telemetry profile . JSON messages are seen with generic RDKB profile details on every 5 min / 10 mins (Expression : /5 * * * *  or */10 * * * * ) . Once in a day uploads JSON with markers to the server .

This is seen due to the schedule expression . As a workaround , we can give the schedule in numbers (Example : 2 / 5 / 10)

The ticket is in place to implement the cron expression support in RPi. Once it is in place , the empty JSON messages will not be seen .