RDK Maintenance Manager

Overview

Maintenance Manager is one of the WPEFramework Thunder Plugin that manages the daily maintenance activities on RDK device. 

Source Code

Terminology

Types of Maintenance

• Unsolicited Maintenance (Maintenance on Boot)
• Solicited Maintenance (Scheduled Maintenance)

Unsolicited Maintenance

Also known as Maintenance on Bootup. RDK automatically performs a Maintenance as soon as the device boots up. Unsolicited maintenance occurs on each and every bootup. Device will look for critical updates on boot and performs maintenance of critical activities without application triggering maintenance. The isRebootPending will be set to false unless there is an RFC/FWUpdate needed. Below are the tasks executed as part of unsolicited maintenance.

Solicited Maintenance

Also known as Scheduled Maintenance/ Daily Maintenance. This is executed daily based on Maintenance Start Time triggered by Application. The isRebootPending is set to true irrespective of XConf configuration. Below are the tasks executed as part of Solicited Maintenance

Note: Both Unsolicited and Solicited maintenance cannot be triggered at the same time. If a Solicited Maintenance is started while an Unsolicited Maintenance is in progress, then the Solicited Maintenance will be notified and postponed to the next day.

Functionality Observations

• Maintenance Manager notifies the upper layer components when the next daily maintenance activity is required to take place (based on the start time scheduled by XConf for the specific device)
• Maintenance Manager starts the daily maintenance upon request from the upper layer components
• Capable of running the maintenance activity in the background or aborting or skipping a daily activity upon request of the upper layers
• Capable for running the list of the activity in time order, one at a time. If multiple activities have the same start time, it will run one after the other one, not concurrently.
• A default Maintenance Start Time will be maintained by Maintenance Manager on first initialization, with initial status as MAINTENANCE_IDLE
• Maintenance Start Time will be updated by the Maintenance Plugin by calling getMaintenanceStartTime jsonPRC call refer JsonPRC section for more info
• Maintenance Manager Plugin, interacts with Network Plugin polling to get the Network status (if its connected to active internet or not). The polling happens for 4 times with a retry delay of 30 seconds.
• On an un-successful polling session, Maintenance Manager will subscribe to onInternetStatusChange event and set the Maintenance Status to MAINTENANCE_ERROR, on successful event reception, MaintenanceMaintenance will trigger Critical Tasks ()

WhoAmI

WhoAmI is used by device to determine the operating context (partner, product/experience, Regional configService) and load the right firmware so that we can activate and use the service that user signed up for. It happens only in Unsolicited Maintenance. It is not defaulted in RDKE yet (as of 25Q2). WhoAmI has dependency on components such as AuthService which has to be refactored before making it suitable for community builds usage.

Maintenance Activity/ Task

XConf Settings

The DCMScript.sh (in RDKV) or dcmd (in RDKE) will parse the /tmp/DCMSettings.conf file to get XConf Data fetched by Telemetry. Data likeurn:settings:CheckSchedule:cron is used to get start_hr and start_min values and urn:settings:TimeZoneMode is used to get the tz_mode data. Device is expected to receive the time based on the time zone confiigured in the device. This parsed data is then stored to /opt/rdk_maintenance.conf file by DCM. The data can be used to calculate the Maintenance Start Time for the next scheduled Maintenance (SOLICITED). The Maintenance Start Time is calculated using /lib/rdk/getMaintenanceStartTime.sh (in RDKV) or using CalculateStartTime() API in MaintenanceManager internally.

Architecture Overview (HLA)

Activity Diagram

High Level Sequence Diagram

Maintenance Status Notifications

• MAINTENANCE_IDLE - Initial state
• MAINTENANCE_STARTED - Sent immediately on maintenance startup either scheduled or onboot
• MAINTENANCE_ERROR - Sent after receiving error notification while executing any of the maintenance activities
• MAINTENANCE_COMPLETE - Sent after receiving *_COMPLETE notification from all critical maintenance tasks. 
• MAINTENANCE_INCOMPLETE - Sent whenever Maintenance service doesn't execute any of the tasks. MAINTENANCE_ERROR is returned even if one task returns error while others weren't executed.

Distros and RFCs used in Maintenance Manager

• Maintenance Manager feature is controlled by distro "enable_maintenance_manager"
• stopMaintenance support is controlled by RFC "Device.DeviceInfo.X_RDKCENTRAL-COM_RFC.Feature.StopMaintenance.Enable" which is enabled by default
• whoAmI feature is controlled by both distros "whoami_enabled" and "sec_manager_whoami_enabled" and a compile time flag "ENABLE_WHOAMI".

Notable features

1. Network check - Before triggering maintenance tasks, first it will check for internet connectivity in the device. If the device is connected with internet, it will proceed to execute maintenance tasks otherwise it will wait for 2 minutes and if still it has no internet connection, it will exit from maintenance and status will be MAINTENANCE_ERROR
2. Maintenance Manager will subscribe for network event if device is not able to acquire network in 2mins before exiting from maintenance activities. Once device got internet, Maintenance Manager will be notified, and it will trigger RFC and xconfImageCheck.sh
3. StopMaintenance is allowed only when the maintenance is in progress and it will set the status as MAINTENANCE_ERROR. 
4. StartMaintenance is allowed only when the maintenance is already not in progress.
5. Who Am I integration changes are done in Maintenance Manager to acquire the device Initialization context

Logs

Maintenance Manager logs can be found in wpeframework log. 

Grep string: grep -inr "MaintenanceManager.cpp" /opt/logs/maintenance.log*