Introduction

• This document  used for upgrading firmware for RPI RDK Broadband devices using XCONF server & TFTP/HTTP protocol.
• Downloaded firmware from local/public http server with support of http protocol.
• Also modified the firmware script with support of both protocols. If user configured http protocol support in xconf UI, On that time firmware will be downloaded by http protocol else firmware will be downloaded by tftp protocol(By default,tftp protocol will be selected).

Environment Setup

•   Firmware scripts and systemd service files are available in RPI build by default. Also, by default systemd firmware service file was enabled.
•   Need to configure our RPI device configuration in XCONF Server.
•   Using systemd command to re-start the firmware service .
•   RPI model name is ARMV7

Build procedure to generate R-Pi image with Firmware upgrade

The build procedure for broadband are as follows:

1. repo init -u https://code.rdkcentral.com/r/manifests -b rdk-next -m rdkb-extsrc.xml
2. repo sync -j4 --no-clone-bundle
3. source meta-cmf-raspberrypi/setup-environment (Select option raspberrypi-rdk-broadband.conf)
4. bitbake rdk-generic-broadband-image 

Flashing the image

Command to flash the image

Generated image has to be flashed to an SD card using this command in local PC:

Ex:

The SD card is inserted to the Raspberry Pi board and booted to check for containers created.

The Raspberry Pi board is connected to the PC via a USB to serial converter and the logs can be checked in console or can be connected via HDMI cable to a TV and logs will be shown in the terminal

Creating checksum file

It is necessary to flash the stable image during upgrade. So checksum file has to be created for the image to be uploaded using below command

After checksum file is created, place this file in tftp location along with actual image file which is expected by firmware upgrade script. Without which firmware upgrade wont be allowed by scripts

Creating Multi-Boot compatible SD-CARD if RDKB is Flashed for first time

If firmware upgrade is intended for multi-boot, it is necessary to have partizion size matching to  RDKB image size ( especially rootfs ). 

To make compatible, size of RDKB rootfs partition should be increased to 2 GB ( 2098 MB) from 300MB default

This resize can be achieved using gpartedui tool which should be available in host pc

Please resize the rootfs partition to 2098 MB or 2GB (/dev/sdb1) accordingly from below procedure as captured in various stages

1. Select gparted tool from menu

2. Select the rootfs partition of format ext3 for resize/move option

3. Resize the rootfs partition by giving new size image to 2098 MiB

4. After resize , position stands as below

5.After modifying size, "Apply all operations" using tick symbol in UI which displays icon as below

6. After giving "Apply icon" position stands as below

7. After resizing is completed 

8. After applying close size increased to 2 GB can be seen

 Executing System

           Using the below command to re-start the firmware service.                           

          Firmware logs generation path.

Setting up XCONF Server

Please follow the below procedure in xconf server for placing new firmware for upgrade to specific device and model ( RPI in this case ) as a reference:

• XConf - Configuring Firmware
  
  
• XConf - Configuring Firmware Download Location

 XConf Server  Configuration Set-up:

 Create Environment

Create Model

Create MAC list by entering the MAC address of RPI board

 Firmware Config

1.  Firmware config can be created via Firmware - > Firmware Configs -> Create. Enter a description for this config.  Also we can define the file name and version of the image/firmware that need to be downloaded to the CPE device. The models that we defined in Common → Models section will be available here, We can select the required models by clicking on it.

2.  To create a firmware template, go to Firmware → Firmware templates. Enter the ID name. Select priority from the 'Priority' drop down menu. Add conditions. There are some already existing templates, if you are using the existing Firmware Templates for configuration ,you can skip this step.

 Firmware Rule

3.  Firmware rule can be create via Firmware -> Firmware rules -> Rule Action -> Create. On clicking on Create button, a list of templates will be presented. We can select the required template (There will be default templates like ENV_MODEL_RULE, IP_RULE, MAC_RULE etc. and also the custom templates created from Firmware->Firmware template -> Create).

After we select the required template, 'Add firmware Rule' page will be displayed. Here the build conditions will be present from the 'template' that we added and in addition to that we can add additional Build Conditions also. To add firmware config , go to the 'Actions' tab and select the firmware config from 'Firmware config' drop down list (Select the firmware config that you have created).

4. The download location needs to be specified so that it can be returned in the response. Choose Firmware -> Download location filter-> Edit, where we can specify the location from where we can download the firmware. Enter the FQDN and Full http location for the firmware download server . Http location will be returned by default to all devices.

With selection of HTTP Protocol

Server Set-up :

1. Install the apache2 http server in local pc
2.  Add the below line in /etc/hosts (192.168.0.107(wan IP) http://www.myfirmware.com)
3.  place the rpi image and checksum files in document root directory.

HTTP Location folder

Verify that image file and its respective md5 checksum file is present in the http folder:

keerthana@keerthana-INVALID:/var/www/html$ ls
rdkb-generic-broadband-image_default_20200529140558.rootfs.rpi-sdimg
folder rdkb-generic-broadband-image_default_20200529140558.txt

XCONF side :

Override the default download Location set at Download Location Filter

There is a new option added in the Firmware Config, where we can add parameters. For example if we add parameters 'firmwareLocation' and 'firmwareDownloadProtocol'. then we will be able to override the default download location set from the 'DownLoad Location Filter' page.

Override Firmware Location with http

To create a new firmware configuration for a particular set of devices with http download location :

1. Go to Firmware → Firmware Configs → Create. Enter a description for this config.  Also we can define the file name and version of the image/firmware that need to be downloaded to the CPE device. The models that we defined in Common → Models section will be available here, We can select the required models by clicking on it. There is also an option 'Parameters'. Add the key values 'firmwareLocation' and 'firmwareDownloadProtocol' as 'http location url' and 'http' respectively

2. Create a firmware rule like the steps given in 'Configuration' and map this Firmware Config to it. Check using the steps in below 'Verification' sections and verify  if the firmwareLocation and firmwareDownloadProtocol are the same as we configured in FirmwareConfig page.

Sample curl response(refer below steps to check)  - https://xconf.rdkcentral.com:19092/xconf/swu/stb?eStbMac=AA:BB:CC:DD:AA:AA

Verification

a. Verification of Firmware test page.

After creating the Firmware configs and Firmware rules , go to Firmware->Test page and give a parameter that will match the one of the rule that you have created. Then matched rule will be displayed like below. 

b.  Verification via curl command

The curl command mocks the request being sent from an STB like below and sample response is also given. It can be given as a curl command or as a get request via postman or browser. 

eg :

$ curl 'https://<XCONF_IP>:19092/xconf/swu/stb?eStbMac=B8:27:EB:BE:D7:12

Sample Response :

Client end verification (RPI)

Add TFTP location

An HTTP location will be returned by default to all devices. To enable tftp(If you have the download location of the firmware as tftp, then only you need to setup this)as download location for a particular set of devices, we need to override it from firmware rules.

1. Go to Firmware -> Firmware Rules -> Define Properties -> Create

   2.  A page will be displayed with options to select the template. Select ‘DOWNLOAD_LOCATION_FILTER’ from the list

3. In this 'firmware rule'  page with type 'DOWNLOAD_LOCATION_FILTER', we need to specify the ‘Build Conditions’ and ‘Action’ , The build condition should be same as that we used to set rule actions(which will set rules for our intended devices). In ‘Properties’ option under ‘Action’, add ‘firmwareDownloadProtocol’ as ‘tftp’, ‘firmwareLocation’ as ‘your tftp location IPV4 address’. This property will override the default value set from ‘Download Round Robin location filter’.

4. To add IPV6 address of tftp servers, you can either specify it here in the define properties rule or else from ‘Download Round Robin Filter’ page. To add IPV6, add it as ‘ipv6FirmwareLocation’ Property in ‘define properties rule’ page (ipv6FirmwareLocation key will be there by default, you need to add the value as tftp ipv6 address). 

5. To add IPV6 address of tftp servers in ‘Download Round Robin Filter’, Go to Firmware - > Download Location Filter. Enter the tftp IPV6 locations and also the percentages. The devices will get back one of the locations based on the percentage listed for the location.

6. Response example for https://{xconf-ip}:{port}/xconf/swu/stb?eStbMac={mac}. Here the 'firmwareLocation' and  'firmwareDownloadProtocol' are overrided at 'Define Properties' firmware rule.
The 'ipv6' addresses will be one of the addresses mentioned in the 'Download Filter' page . If you don't want "ipv6FirmwareLocation", then don't setup it in 'Define Properties' or in the 'Download location filter' pages and you will get only "firmwareLocation" in the response
{

  "firmwareDownloadProtocol": "tftp",

  "firmwareFilename": "rdkb-generic-broadband-image_default_20200406103506.rootfs.rpi-sdimg",

  "firmwareLocation": "192.168.1.9",

  "firmwareVersion": "rdkb-generic-broadband-image_default_20200406103506.txt",

  "ipv6FirmwareLocation": "2601:1f18:227b:c00:767a:afd0:82bb:efa6",

  "rebootImmediately": false

}

7. Setting up IPV4 locations via ‘Download Location Round Robin Filter’ is not supported. This can be set only by the property 'firmwareLocation' from the 'Define Properties' firmware rule page.

8. Note : Just like we added tftp location and protocol here, we can also override the default value with http as well. For firmwareDownloadProtocol, add 'http' and for the 'firmwareLocation', add http location

JSON Output

Verify that XConf Server configuration displays the Json response from the mentioned URL:

Using the below url to view the JSON Format result of http protocol ,
https://xconf.rdkcentral.com:19092/xconf/swu/stb?eStbMac=<RPI MAC ADDRESS>&model=<model_name>&capabilities=RCDL&capabilities=supportsFullHttpUrl

TFTP Server Setup:

Configure tftp server and keep the image file and reapective md5 checksum file inside tftp folder. Example given below:

1.Verify the content of tftp file (use  vi /etc/xinetd.d/tftp command)

service tftp

{

protocol = udp

port = 69

socket_type = dgram

wait = yes

user = nobody

server = /usr/sbin/in.tftpd

server_args = -c -v -s /home/amrita/tftphome

disable = no

}

2. Verify the content of tftpd-hpa:

# /etc/default/tftpd-hpa

TFTP_USERNAME="tftp"
TFTP_DIRECTORY="/home/amrita/tftphome/"
TFTP_ADDRESS="0.0.0.0:69"
TFTP_OPTIONS="-s -c -1"

3. Verify the tftp folder location:

amrita@amrita-OptiPlex-9020:~/tftphome$ pwd

/home/amrita/tftphome

4. Verify that image file and its respective md5 checksum file is present in the tftp folder:

amrita@amrita-OptiPlex-9020:~/tftphome$ ls

rdkb-generic-broadband-image_default_20190812074412.rootfs.rpi-sdimg

rdkb-generic-broadband-image_default_20190812074412.txt

5. Start tftp server and xinetd:

sudo service tftpd-hpa restart

/etc/init.d/xinetd restart

RPi - Board

1. Flash the RDKB image supported for Firmware upgrade feature in the Rpi and verify the 2 partitions(ex. mmcblk0p1, mmcblk0p2) present under dev folder (use command ls /dev)
2. Device auto-reboots and verify there are 2 more additional partitions are created (ex. mmcblk0p3, mmcblk0p4).
3. Open /etc/include.properties file and verify the CLOUDURL parameter where XConf Server URL is configured
4. Verify the image version ( cat /version.txt) displays the flashed image version
5. Open /rdklogs/logs/swupdate.log to verify the communication from RPI board with XConf Server and the download status
6. Verify the tftp download is happening by noticing the change of file size using below command:cd /extblock/tftpimage/imagedwnldls -lh  (use this command frequently to verify the file size change) if tftp protocol was selected in XCONF UI
   
7. Verify the http download is happening by noticing the change of file size using below command : cd /extblock/httpimage/imagedwnldls -lh(use this command frequently to verify the file size change)if http protocol was selected in XCONF UI
   
8. Once image download completed, the Rpi board will auto reboot and come up with downloaded image
9.  Verify the version of booted image which shows the downloaded image version

TroubleShooting

1.  If output.json will not be generated in /nvram folder. Please check the network connection in RPI Device.
2. Also, check the status of swupdate service by using the below command,
   1.  systemctl status swupdate.service
3. Check the swupdate logs in below path,
   1. cat /rdklogs/logs/swupdate.log

Conclusion

Please find the summary on firmware upgrade explained above

• Build the image ( RDKB ).
  
• Ensure 8GB sd-card is available for flashing
• Flash the build image to sd-card
• If Firmware upgrade is targeted for multi boot and RDKB image is flashed for the first time follow resize procedure
• Setup xconf server ready with rules and download location for RPI device
• Place the new image in xconf server
• Boot the flashed image

Note : 1. RPI will go for reboot while booting for the first time, since two more partition needs to be created which requires reboot. Depends on the image present in xconf and RPI, further reboot will happen for upgrade. Please refer to design (RDKB RPI Firmware Upgrade with support of HTTP protocol - Design - 2020 M6) for how firmware upgrade works with bank switching

2. SD card should be zero filled formatted before starting the firmware upgrade procedure. DriveWipe software is used for this. Please refer the you-tube video link for the SD card full format process.

https://youtu.be/CZhsVBEMYuk

3. If the user wants to view swupdate.log file then follow below steps:
    a) Disable log rotation feature - This wipes out the log file if size exceeds after 1min
        For B image  command - systemctl disable rdkbLogMonitor
       
   b) Reboot the box - This will make swupdate.service restart and user can start getting swupdate.log for B image

4. Broadband(size 292MB) image takes roughly 20 min time from Cloud server(Xconf).

5. Minimum extended partition should be 1GB, to verify firmware feature.