RDK Remote debugger - Diagnostic Debug Data Collection Tool

Intro

As a Field Triage Engineer and Developer responsible for debugging field issues, having the ability to collect diagnostic/ debug data from the device without requiring SSH access would be beneficial. The Triage Engineer can use WebPA communication to initiate the data reporting process and retrieve the report from the S3 log service

Architecture

• Please refere https://github.com/rdkcentral/remote_debugger/tree/develop

Use case diagram

• rdbg

Samples

• Upload scripts location

  • /lib/rdk/uploadRRDLogs.sh
    /rdklogger/uploadRDKBLogs.sh

• Report generation location

  • /tmp/rrd

• Log files

  • /rdklogs/logs/remote-debugger.log.0 

• static json file location

  • Path in CPE
       /etc/rrd/remote_debugger.json
    Path in code base
       https://github.com/rdkcentral/remote_debugger/blob/develop/remote_debugger.json

• Datamodel command

  • dmcli eRT setv Device.DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.RDKRemoteDebugger.IssueType string "Device.Uptime"

rdm agent download url path

• /tmp/.xconfssrdownloadurl

Removed existing rdm in rdkb builds

• meta-cmf/recipes-core/packagegroups/packagegroup-rdk-ccsp-broadband.bbappend # Comment out or remove the line: RDEPENDS_packagegroup-rdk-ccsp-broadband_append = " rdm"

Distro

• rrd to be enabled

RFC

• DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.RDKRemoteDebugger.Enable

Enable service from yocto

• remote-debugger.service 

Image install

• Install remote debugger & rdm-agent in monolithic image

Code base

• remote debugger

  • Source repo

    • https://github.com/rdkcentral/remote_debugger/tree/develop

  • meta layer

    • https://github.com/rdkcentral/meta-rdk/blob/develop/recipes-extended/remotedebugger/remotedebugger.bb

• rdm agent

  • Source repo

    • https://github.com/rdkcentral/rdm-agent/tree/develop

  • meta layer

    • https://github.com/rdkcentral/meta-rdk/blob/develop/recipes-support/rdmagent/rdmagent.bb

  • Dependencies

    • https://github.com/rdkcentral/common_utilities
    • https://code.rdkcentral.com/r/plugins/gitiles/rdk/components/generic/rdk-oe/meta-rdk/+/refs/heads/rdk-next/recipes-common/utils/commonutilities_git.bb
    • to be moved to common area - https://github.com/rdkcentral/meta-rdk-video/blob/develop/recipes-common/utils/commonutilities_git.bb

  • Additional changes

    • rdm_downloadutils.c 
      
      #ifdef IARMBUS_SUPPORT 
      #include "rdmMgr.h" 
      #else 
      #define RDM_PKG_INVALID_INPUT 11
      #endif 

Customizations to support cash report uploading

• Note: This is only for testing in BPI R4.For production you need to follow the actual signing process.
• https://github.com/cpokuru/remote_debugger/commit/d2f717039216809a7e137b0a0ec94e783248c1ba

Dependencies removal

• remove utopia dependencies from remote debugger service file

Roles and responsibilities of ISP

• Develop your own triaging tool on top of webpa/usp/tr69

Use case for production

1. static profile using remote debugger (Works) 
2. dynamic profile using remote debugger and rdm

Test logs

• ~# dmcli eRT setv Device.DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.RDKRemoteDebugger.IssueType string "Device.Uptime"
  CR component name is: eRT.com.cisco.spvtg.ccsp.CR
  subsystem_prefix eRT.
  Execution succeed.
  
  root@Filogic-GW:~# cat /rdklogs/logs/remote-debugger.log.0 
  220428-17:44:16.829555 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6844] [isRRDEnabled:112]:RFC is enabled, starting remote-debugger
  220428-17:44:16.889345 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [RRDEventThreadFunc:44]:Waiting for for TR69/RBUS Events... 
  260122-19:38:52.063885 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6889] [_remoteDebuggerEventHandler:322]: Received event for RRD_SET_ISSUE_EVENT Device.DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.RDKRemoteDebugger.IssueType 
  260122-19:38:52.063962 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6889] [pushIssueTypesToMsgQueue:395]: SUCCESS: Message sending Done, ID=0 MSG=Device.Uptime Size=13 Type=1 AppendMode=0! 
  260122-19:38:52.064019 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [getIssueInfo:191]:  Received Main Node= Device, SubNode= Uptime
  260122-19:38:52.064049 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [processIssueTypeInStaticProfile:296]: Checking Static Profile... 
  260122-19:38:52.067965 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [processIssueTypeInStaticProfile:312]: Issue Data Node: Device and Sub-Node: Uptime found in Static JSON File /etc/rrd/remote_debugger.json... 
  260122-19:38:52.068239 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:350]: No MACRO found, proceeding with commands:"uptime" 
  260122-19:38:52.068315 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:376]: Executing following commands using systemd-run:
   ""uptime""
  260122-19:38:52.069350 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:390]: Starting remote_debugger_Device.Uptime service success...
  260122-19:38:52.168889 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:396]: Using journalctl to log command output...
  260122-19:38:52.169811 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:404]: journalctl remote_debugger_Device.Uptime service success...
  260122-19:38:52.176470 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:414]: Sleeping with timeout 10 sec before stopping service...
  260122-19:39:02.176595 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [executeCommands:418]: Stopping remote_debugger_Device.Uptime service...
  260122-19:39:02.189241 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [uploadDebugoutput:46]: Starting Upload Debug output Script: /lib/rdk/uploadRRDLogs.sh... 
  2026-01-22T19:39:02.339Z: /lib/rdk/uploadRRDLogs.sh: Executing uploadRRDLogs.sh script to upload Debug info of ISSUETYPE=DEVICE_UPTIME
  2026-01-22T19:39:02.345Z: /lib/rdk/uploadRRDLogs.sh: Using upstream RRD URL: http://10.2.166.120:8080/upload
  2026-01-22T19:39:02.350Z: /lib/rdk/uploadRRDLogs.sh: Checking /tmp/rrd/Device-DebugReport-2026-01-22-19-38-52 size and contents
  2026-01-22T19:39:02.357Z: /lib/rdk/uploadRRDLogs.sh: Creating 020100B65218_DEVICE_UPTIME_2026-01-22-19-39-02PM_RRD_DEBUG_LOGS.tgz tarfile from Debug Commands output
  2026-01-22T19:39:02.367Z: /lib/rdk/uploadRRDLogs.sh: Invoking uploadRDKBLogs.sh script to upload 020100B65218_DEVICE_UPTIME_2026-01-22-19-39-02PM_RRD_DEBUG_LOGS.tgz file
  2026-01-22T19:39:02.941Z: /lib/rdk/uploadRRDLogs.sh: RRD DEVICE_UPTIME Debug Information Report upload Success
  2026-01-22T19:39:02.946Z: /lib/rdk/uploadRRDLogs.sh: Removing uploaded report 020100B65218_DEVICE_UPTIME_2026-01-22-19-39-02PM_RRD_DEBUG_LOGS.tgz
  260122-19:39:02.949497 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [checkIssueNodeInfo:552]: RRD Upload Script Execution Success...
  260122-19:39:02.949609 [mod=REMOTEDEBUGGER, lvl=INFO] [tid=6937] [RRDEventThreadFunc:44]:Waiting for for TR69/RBUS Events... 
  root@Filogic-GW:~#