Introduction 

CMF perform fully automated sanity testing of multiple video reference platforms on both Gerrit and Github contributions and on nightly branches.

Testing can also be performed on tip dunfell/rdk-next on demand.

CMF Sanity Tests are command line driven (e.g. gst-launch, aampcli) as well as UI driven. We have UI tests for the following UI's:

• RDK Video Accelerator UI (all platforms)
• LGI Refapp2 (brcm97218 and RPI-3)
• DTVKit DVB UI (brcm97218)

CMF Video Sanity Tests use the StormTest product to do image comparison, video and audio detection, OCR etc..

CMF currently support the following platforms and profiles for contribution and nightly branch CI.

Automated Sanity Flows

Each sanity test flow follows the same principle:

• Look for a free DUT resource (Jenkins Lock)
• Retrieve the automated build image
• Deploy the image on the DUT
• Verify the image has flashed successfully
  • check version.txt 
  • key service and process checks e.g. wpeframework, rmfstreamer etc
  • has IP
• Retrieve the DUT logs pre test
  • includes a long set of commands CMF define
  • retrieve the actual device logs (/opt/logs)
• Perform the video command line and UI testing
  • Screenshots and other useful info is captured here
• Retrieve the DUT logs post test
• Prepare a test report (for contributions this is sent to Gerrit/Github and Verified/Platform-Verified status is set accordinly)
• Power down the device and free it for the next request

Some examples of the automated flows: