Test Execution in Raspberry Pi device

There are generic test cases that can be executed on Raspberry Pi device loaded with RDKB stack. This helps partners to familiarize with the Automatics system and helps to validate basic features in RDKB in Raspberry Pi.

Below diagram shows the communication between different components in Automatics and the device.

A user can login to RPi device via different ways such making ssh connection or telnet connection and this method is specific to partners.  So there should be an intermediate layer between RPi device and Automatics which helps  to connect with device. This intermediate layer is the RPi Partner Provider Implementation which contains partner specific implementation of different Providers defined by Automatics. 

To connect with an RPi device, DeviceConnection Provider is used. The project rpi-provider/java-handler contains implementation of DeviceConnection Provider based on ssh connection. So it makes an ssh connection to device, execute command within device and return the response. The test scripts uses DeviceConnection Provider to execute commands within device and verify the response.

To execute tests on Raspberry Pi device, partner has to ensure below setups are done.

• Raspberry Pi device loaded with RDKB stack is available
• Automatics system (Includes Automatics Orchestration, Automatics Properties, Device Manager, Jenkins)
• Automatics Core and RDKB Utils maven artifacts deployed in self hosted artifactory repository
• Configure Raspberry Pi device in Device Manager, Automatics Orchestration and Automatics Properties
• SSH connection to Raspberry Pi device is available from Jenkins VM
• Configure the test scripts in Automatics Orchestration
• Maven Deploy generic Raspberry Pi Provider Implementation provided by RDKM or partner specific Raspberry Pi Provider Implementation to self hosted artifactory repository.
• Configure Raspberry Pi Provider Implementation artifact details in jenkins job.

Automatics Architecture

Automatics system involves Automatics Orchestration, Automatics Properties, Jenkins, Automatics Core, Test Utils, Test Scripts, Device Manager and Partner Provider implementations.

The details on how various components are linked is explained in Automatics Architecture.

Setting up Automatics for Test Execution in Raspberry Pi 

Partners can follow below steps for setting up Automatics for executing test cases in Raspberry Pi.

1. Raspberry Pi with RDKB Stack

Ensure that a Raspberry Pi device loaded with RDKB stack is available for test execution.

2. Setup Automatics

As the next step, partner can work on setting up Automatics system. This includes Automatics Orchestration, Automatics Properties, Device Manager, Jenkins and job configurations, Maven artifact deployment of Automatics Core and RDKB Test Utils.

Hardware Requirements for Automatics is available at Automatics Hardware Requirements.

After VMs with required hardware specifications are ready and available, partner can start setting up tools in the VMs as mentioned below.

• Setup and configure Device Manager - Device Manager Deployment Document
• Setup and configure Automatics Orchestration - Automatics Orchestration Setup
• Setup and configure Automatics Properties - Automatics Properties
• Setup and configure Jenkins jobs for test execution - Automatics Jenkins Setup
• Deploy maven artifact of Automatics Core in self hosted maven artifact repository. Following the steps in Automatics Core Deployment, jenkins job can be created to deploy Automatics-Core. After deployment, Click on 'Build Now' option to deploy automatics-core.
• Deploy maven artifact of RDKB Utils in self hosted maven artifact repository - RDKB Utils : Jenkins Deployment Job
• Deploy maven artifact of Raspberry Pi Partner Provider Implementation in self hosted maven artifact repository - RPi Java API Provider : Jenkins Deployment Job

3. Configure Raspberry Pi Partner Provider Artifact Details in Jenkins Test Execution Job

As part of executing RDKB tests in partner environment , partner has to update rdkb-test project pom file to point to the Raspberry Pi Provider implementation (it will vary for partners)
We have added a supporting shell script , which can be configured to the Jenkins Job, so it will add the RPi Provider dependency to the rdkb-test project pom.xml during run time. And, this can be done by configuring Build Step under Build Section

Steps to be followed.

...

in Test Execution Jenkins Job. 

• • In Test Execution Jenkins Job, under Build Section, please configure 'Shell Script' referring Jenkins Job for Test Execution#ConfigureShellScriptforPartnerJavaAPIdependency 

Jenkins Job → Build Section → Shell Script. 

...

/mnt/scripts/pre-test-config.sh -g 'com.automatics.providers' -a 'rpi-provider-impl' -v '0.0.1-SNAPSHOT'

4. Configure

...

Map Test script to device. <Shameem> 

Device in Automatics

After the setup is ready, partner has to configure the device model and device details in Automatics and its details are available at Device Configuration in Automatics.

5. Validate Automatics Setup

After the system is ready, partner can validate the system by executing test script TC-QUICK-DUMMY-ALL-MODELS from Automatics Orchestration. From the execution, partner can verify communication between below components are working fine. 

• Communication between Orchestration and Device Manager.
• Communication between Orchestration and Jenkins.
• Communication between Jenkins and Automatics Properties.

Please follow Automatics Generic Tests#ConfigureTC-QUICK-DUMMY-ALL-MODELS to configure TC-QUICK-DUMMY-ALL-MODELS in Orchestration.

After setup validation, partner can configure RDKB test scripts in Orchestration.

6. Configure RDKB Test Scripts in Orchestration

To configure RDKB test scripts to Orchestration, please follow the steps below.

• Download the script file RPi_RDKB_TEST_CASE.xls to your machine.
• Open the excel in edit mode and update the column 'RUN ON MODELS' for each test script with the Raspberry Pi device model name.
• Import the updated test script to Orchestration following details available at Automatics Orchestration ImportTestScriptandSteps

Note: In Manage Script page, ensure that only single Quick Test script is enabled for each device model. Please follow, Automatics Orchestration User Manual#EnableScript, to enable/disable test script.

7. Configure Device Login Details in Jenkins VM

Some test scripts has to execute commands within device and verify its response. The DeviceConnection Provider implementation in rpi-provider/java-handler helps to execute commands within device. It creates SSH connection to device and then executes the commands. So to create SSH connection, ssh login details are to be provided to Automatics. This can be done by configuring ssh login credentials in the server-config.xml and then keep it in Jenkins VM. The location where this config file is kept should be configured in Automatics Properties on parameter 'serverConfig.path'.

From Automatics core version  2.1.0 and device manager version 1.0.2 it is possible to configure SSH Connection login  details in devicemanger by adding sshcredentials in extraproperties  section of  POST /deviceModel/add  API. The priority is given first to get ssh connection details from serverconfig.xml file, if it is not there we can get details from device manager side.

For eg: if server-config.xml is available at /mnt/config in Jenkins VM, then in Automatics Properties parameter will be configured as serverConfig.path=/mnt/config/server-config.xml

8. Trigger Execution

After the test scripts are configured, partner can trigger test execution from Orchestration via Trigger Execution Manually page.

While triggering execution, Quick Test(Image Upgrade) should be skipped by un-selecting option 'Is Quick Test Required' in Trigger Execution Manually page. Steps to trigger execution is available at Automatics Orchestration-TriggerTestExecution.

Test Execution Results will be available at Execution Results page. Details are available at Automatics Orchestration-ExecutionResults 

...