Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Validation / Test Coverage Status


Category

Subcategory

Existing/New FeatureTest Readiness

Category

Subcategory

Feature/Component
Owner

Test Status

Existing Test CoverageTest StepsTest Plan

Test Scope

ETA

Discussion PointsApplicable platforms

Streaming apps

YouTube

Not available

Expand
titleRiato

Discussion points

Netflix

Available
UpgradePremium app certification test suite

Netflix

Upgrade

Premium app

published

certification test suite

Amazon Prime

Available
Upgrade

Premium app

published

certification test suite

Premium app published test suite

Framework






Firebolt

Upgrade
•Firebolt

Firebolt Native Apps Compliance

Test

Suite

•Firebolt JS test suite ( Yet to open source )

Firebolt Certification App ( FCA ) - Open sourcing in progress - Expected ETA - Aug EOM 

Ripple

NewFirebolt Certification App ( FCA ) - Open sourcing in progress - Expected ETA - Aug EOM 

Lightning

Upgrade

Exercise different lightning apps - Refer animation test apps under TDK MVS Test Apps Media Playback And Graphics Validation Test Setup

Strike test tool - https://strike.lightningjs.io/es6/#home

Ripple

Open

TBD

TBD

Lightning

NA

QT for BLE

Open

TBD

Q1
ExistingManual tests using BLE remote.

Core Thunder framework

Upgrade

RDK

Service

API

and

And Event

Test

Certification Suite

•Improve the test coverage

Testing Areas

  • Execute rdkservices plugins test
  • Perform System testing
  • Perform Bootup testing
  • Execute Performance testing
  • Perform WebApp testing

Browser - WPEWebkit

Upgrade

Browser

performance test suite•Track the browser performance test scores

Performance Suite 

rdkcentral/MVT: Media Validation Tool for verification of browser's media playback capabilities 

Screen cast

DIAL

Open

•Xcast RDK service validation

Existing

Test coverage using Xcast RDK service - RDK API And Event Certification Suite 

Manual tests for real screen validation

Q1

Security

DAC (Containerization)

Open
New
TBD
Containerization Test Framework 

Dobby

In Progress

•Dobby test tool enhancement and integration

New

Containerization Test Framework 

Q1

Rialto 

In ProgressQ1
New

•Playback tests using Rialto sink

Rialto Gstreamer Test Framework  

Packager (Lisa)

Open

TBD

TBD

NewRDK API And Event Certification Suite 

AppArmor 

NewAppArmor Test Framework

AppArmor 

Open

TBD

Q1

Graphics

Rendering

Existing
•Westeros
Westeros ESSOS test apps
integration
- Firebolt Native Apps Compliance Suite 
Graphics ( OpenGL ES )ExistingGlmark2 - Hardware Performance Suite 
GstreamerGstreamerExistingGstreamer unit tests
Gstreamer SoC SDKExistingFirebolt Native Apps Compliance Suite

DRM & Content protection

PlayReady

Existing

•Browser Playback tests using DASHIF test streams

Media Playback And Graphics Validation Suite

rdkcentral/MVT: Media Validation Tool for verification of browser's media playback 

Widevine

Existing

Media Playback And Graphics Validation Suite

Common SVP

New

Verimatrix

New

Widevine

Common SVP

Open

TBD

Verimatrix

Open

TBD

Build & Platform

Yocto version

NA
Existing

Kernel

In Progress

•Kernel unit tests – Kself tests

•Kernel performance test tool

•L1 to L3 validations

Phase 1 – LTP - Q1

Phase 2 – Q3

ExistingKernel Test Framework

Init system

ExistingBasic sanity and Performance tests

Init system

NA

Application Manager

•RDK
ExistingRDK Shell service test coverage

•Improve the application management test coverage

Q1

, RDK API And Event Certification Suite

Broadcom SDK

ExistingHAL Certification Suite

Broadcom SDK

NA

Video Player 

 IP

•Media validation test suite

 QAM

•RDKV RMF components

Existing

Media Playback And Graphics Validation Suite

rdkcentral/MVT: Media Validation Tool for verification of browser's media playback 

 QAM

ExistingNA

 DVB

ExistingDVB RDK service test coverage, RDK API And Event Certification Suite

 ATSC

ExistingNA

 TSB

NewFog test coverage (Test cases/streams for Live protected contents are not available)

 DAC Apps

ExistingContainerization Test Framework 
RDK API And Event Certification Suite 

 DVB

•DVB RDK service test coverage

•DVB component and HAL test coverage

 ATSC

Open

 TSB

•Fog test coverage

 DAC Apps

Open

Remote Control and Voice

Voice Stack

Open

•RDK Service test coverage

•Voice HAL test coverage

New


Manual tests for voice remote


RF4CE remote

Existing
RF4CE remote

BLE voice remote

Existing

External Audio support

Existing

Networking

Wi-Fi

•Wifi
Existing

•Wi-Fi service test coverage

, RDK API And Event Certification Suite

•Wi-Fi

•Wifi

HAL test coverage, HAL Certification Suite

LAN

Existing
•Test TBD
Test execution using ethernet connected DUT

Device Management

Telemetry

Open


Automatics Tests

Feature Control

Open

Automatics Tests

TBD

Crash Upload

Open

TBD

Firmware Upgrade

Open

TBD

Automatics Tests

Firmware Upgrade


Xconf firmware upgrade tests via System service RDK API And Event Certification Suite

Webpa


Automatics tests


Validation Platforms

  •  Validation support for a single build ( 32 bit ) on each platform
  •  Along with regular RDK builds for RPI HYB and RPI MC, separate Alexa enabled build for the same.
  •  UEI Tatlow, T4H Lima RCUs supported platform builds for validation.

Platform NameRemarks
1RPI HYB




Complete test suites execution for each RDK6 RC builds

2RPI MC
3Realtek SOC Reference
4Amlogic SOC Reference
5Broadcom SOC Reference
6Sercomm - Realtek Video Accelerator
7Skyworth - Amlogic Video Accelerator
8Sagemcom - Broadcom Video Accelerator
9Amlogic TVApplicable test suites execution.

Validation Process

  • Flash the devices using RDK6 RC builds provided by CMF.
  • Run the tests available in TDK, Automatics and Manual tests against each feature.
  • Attach the test reports including defect mapping in the feature ticket and also provide the tested image details in ticket.
  • Link all open RDK/SOC/OEM issues identified as part of the feature validation in the corresponding feature ticket.
  • If there are no failures then will resolve the ticket as Verified , otherwise Failed and assign back to developer.

Defect Dashboard

                            Overall defect summary                                                                                                 

Jira Charts
bordertrue
showinfortrue
serverJIRA - 2
jqllabels%20%3D%20RDK6_feature%20and%20type%3Dbug
statTypestatuses
chartTypepie
width700
isAuthenticatedtrue
serverId11deff04-0380-3a3d-a916-0849d4e573f7

                         Open Issues


Jira
serverJIRA - 2
columnIdsissuekey,summary,issuetype,created,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,due,assignee,reporter,priority,status,resolution
maximumIssues20
jqlQuerylabels = RDK6_feature and type=bug
serverId11deff04-0380-3a3d-a916-0849d4e573f7