Introduction

Logging and Log Upload are critical aspects when considering large-scale RDK deployment. CPEs have a space constraint to store device logs for future use so either we store them for a limited number of days in CPE or in cloud servers.

Logs play a crucial role in RDK deployments by providing data for deriving field metrics and assessing the criticality of situations, such as determining whether an issue needs immediate attention or can wait until the next day. Controlled log uploads, ranging from 100% to very small numbers (2-3%) based on device maturity, help manage the volume of data. Configurable upload intervals, typically set to once a day, and the right level of abstraction ensure a balance between available storage space and the logs generated. Additionally, rotate-backup-upload policies are implemented to maintain efficient log management. Log Upload uses XConf as the primary endpoint, allowing the configuration to enable or disable log upload for a device or group of devices. During bootup, the CPE retrieves configuration information from XConf, and based on rules defined in the XConf server, it can control whether a particular device needs to perform log uploads. This setup allows for the sampling of devices from a larger population. The configuration server is responsible for retrieving and delivering settings to any client, and it supports configurable upload methods such as HTTPS, HTTP, and TFTP.

Architecture

RDK Log Upload Mechanism

Log uploader scripts are responsible for gathering local logs, packaging them, and uploading them to a remote URL. A client-side scheduler would control when it executes, but it could also be activated via immediate triggers from the log upload checker or other local actions.

The Uploader is responsible for sending CPE log files to a remote file store. It performs the following:

• Responds to triggers (i.e. from cron)
• Checks local Device Settings for the URL of the file store, the logs it should send, and the repository it should use
• Gets connection credentials
• Makes a connection to the remote server and sends the files

How CPEs Upload Logs and Metrics

CPEs check with XConf for their logging and telemetry configuration through a query. Part of their query string includes several (about four) parameters of this particular CPE, such as the MAC address, model, current firmware version, and environment. The XConf server looks at these parameters and runs them through a rules engine that finds the closest targeting rule match to the CPE’s profile. Once it finds a match, it retrieves the configuration for that rule. The configuration includes things like an upload schedule, repository URL, upload protocol, path to the file server, protocol indicator, upload time smear, and more. Once the CPE gets the response, it parses it and sets the specified random schedule that delays uploading the files. At that time, the CPE packages up all of its log files, compresses the bundle and uploads the archive to the destination.

The DCM service, which can be checked using the command $systemctl status dcm-log, is responsible for log upload in CPE. This service runs as a system service and invokes /lib/rdk/StartDCM.sh on start-up. The StartDCM.sh script internally calls /lib/rdk/DCMscript.sh for log upload. Once the logs are prepared, the script /lib/rdk/uploadSTBLogs.sh takes over the actual uploading task. To monitor the log upload process, you can use the command $tail -f /opt/logs/dcmscript.og.

Configuration Details

⁠To know how XConf server-side configuration and feature validation are done please refer to XConf Server - User guide for configuration and feature validation .