Introduction

Logging and Log Upload are critical aspects when considering large-scale RDK deployment. CPEs have a space constraint to store device logs for future use so either we store them for a limited number of days in CPE or in cloud servers.

Logs play a crucial role in RDK deployments by providing data for deriving field metrics and assessing the criticality of situations, such as determining whether an issue needs immediate attention or can wait until the next day. Controlled log uploads, ranging from 100% to very small numbers (2-3%) based on device maturity, help manage the volume of data. Configurable upload intervals, typically set to once a day, and the right level of abstraction ensure a balance between available storage space and the logs generated. Additionally, rotate-backup-upload policies are implemented to maintain efficient log management. Log Upload uses XConf as the primary endpoint, allowing the configuration to enable or disable log upload for a device or group of devices. During bootup, the CPE retrieves configuration information from XConf, and based on rules defined in the XConf server, it can control whether a particular device needs to perform log uploads. This setup allows for the sampling of devices from a larger population. The configuration server is responsible for retrieving and delivering settings to any client, and it supports configurable upload methods such as HTTPS, HTTP, and TFTP.

Architecture

RDK Log Upload Mechanism

Log uploader (/usr/bin/logupload) is responsible for gathering local logs, packaging them, and uploading them to a remote URL. A client-side scheduler would control when it executes, but it could also be activated via immediate triggers from the log upload checker or other local actions.

The Uploader is responsible for sending CPE log files to a remote file store. It performs the following:

• Responds to triggers (i.e. from cron)
• Checks local Device Settings for the URL of the file store, the logs it should send, and the repository it should use
• Gets connection credentials
• Makes a connection to the remote server and sends the files

Repository

https://github.com/rdkcentral/dcm-agent

How CPEs Upload Logs and Metrics

This section describes what happens after CPE boot, how log upload is triggered (or can be triggered), which endpoint is used, how that endpoint is configured, and how upload execution happens on-device.

1) Boot-time initialization and configuration fetch

1. Device boots and core services start (dcmd, telemetry, reboot helpers, service plugins).
2. DCM/XConf configuration is fetched using device identity attributes (for example: MAC, model, firmware, environment).
3. XConf returns log-upload policy, typically including:
   • upload schedule (cron-like)
   • upload endpoint URL
   • upload protocol (HTTP/HTTPS)
   • behavior toggles (e.g., reboot upload, mTLS controls via RFC)
4. Effective values are resolved into local runtime config used by callers/uploader.

Common config sources used at runtime

• XConf/DCM settings:
  • LogUploadSettings:UploadRepository:URL
  • LogUploadSettings:UploadRepository:uploadProtocol
  • LogUploadSettings:UploadOnReboot
• Local files:
  • /etc/dcm.properties or /opt/dcm.properties (e.g., LOG_SERVER)
  • /tmp/DCMSettings.conf (resolved runtime values)

2) How log upload is triggered after boot

After boot, upload may start from any of these trigger paths:

A) Scheduled trigger (DCM scheduler)

• dcmd executes scheduled job (DCM_LOGUPLOAD_SCHED).
• Calls uploadstblogs_run() with trigger type TRIGGER_SCHEDULED.

B) Maintenance window trigger

• MaintenanceManager runs maintenance task chain.
• Upload task invokes upload binary/wrapper (platform-image dependent).

C) On-demand app/API trigger

• org.rdk.SystemServices.uploadLogs initiates immediate upload path.
• Caller resolves protocol/URL and invokes upload command path.

D) TR69hostif immediate trigger (UploadLogsNow RFC)

• RFC: Device.DeviceInfo.X_RDKCENTRAL-COM_xOpsDeviceMgmt.Logging.xOpsDMUploadLogsNow
• Invokes uploadstblogs uploadlogsnow (on-demand immediate mode).

E) Reboot upload trigger

• Boot sequence waits on marker synchronization (backup_logs, reboot info, telemetry markers).
• Then uploader runs with TRIGGER_REBOOT (subject to timeout guards).

3) Which endpoint is reached and how it is chosen

The final upload destination is selected from resolved config:

1. Primary: XConf/DCM upload repository values (URL, uploadProtocol)
2. Fallback/auxiliary: local properties (LOG_SERVER) depending on caller path
3. Security mode:
   • plain HTTP/HTTPS as configured
   • mTLS/xPKI path if enabled by RFC: Device.DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.MTLS.mTlsLogUpload.Enable

So the uploader reaches the cloud URL from DCM/XConf policy (or caller fallback), using configured protocol/security mode.

4) How it executes on-device (what runs)

Regardless of trigger source, execution pattern is:

1. Acquire concurrency lock: /tmp/.log-upload.lock
2. Collect eligible logs
3. Apply exclusions/filters
4. Create compressed archive (tar.gz)
5. Upload archive to configured endpoint
6. Record completion:
   • maintenance path may emit completion/error signaling
   • uploadlogsnow writes /tmp/logUploadStatus.txt

5) “Downloads and executes” clarification

In this flow, the device typically downloads configuration, not a new upload executable.

• Downloaded from server: policy/config (schedule, URL, protocol, flags)
• Executed locally: existing on-device upload engine (uploadstblogs/logupload path via caller)
• In short: backend tells the device where/when/how to upload; device uses its preinstalled binaries to perform upload.

6) Quick reference table (post-boot perspective)

7) Runtime monitoring

Use:

tail -f /opt/logs/dcmscript.log

For UploadLogsNow status:

cat /tmp/logUploadStatus.txt

Configuring server details

The value set for Device.DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.Telemetry.ConfigURL determines where the device will fetch its DCM/Telemetry settings. Those settings, in turn, provide the URL for log upload and other related parameters. 

This can be retrieved from xconf server as a response to the RFC query: https://xconf.rdkcentral.com/featureControl/getSettings?<parameters>

For RDK7 release in community devices, this is set from https://github.com/rdkcentral/meta-image-assembler-rdke/blob/main/conf/community-rfc-configs.ini

 mTLS support

• RDKE LogUpload functionality expects mTLS connection by default to Download & upload logs to the server.
• Support to add mTLS to RDKM XConf server is under investigation
  RDKCLOUD-74 - Getting issue details... STATUS  
  RDKECOREMW-870 - Getting issue details... STATUS
  
• Until then, logUpload functionality is broken in RDKE builds

Configuration Details

⁠To know how XConf server-side configuration and feature validation are done please refer to User guide for configuration and feature validation .

Validation details