Versions Compared

Old Version 19

changes.mady.by.user Vimal K S

Saved on

compared with

New Version Current

changes.mady.by.user Vimal K S

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Validation / Test Coverage Status


Category

Subcategory

Feature/Component

Owner

Test Status

Existing
Test CoverageTest Case / StepsTest Plan

Test Scope

Open RDK Issues

Discussion PointsApplicable platforms
/New FeatureTest Readiness

Streaming apps

YouTube

Not available
UpgradePremium app
published
certification test suite

Netflix

Available
Upgrade

Premium app certification test suite

Amazon Prime

Available
Upgrade

Premium app certification test suite

Framework






Firebolt

Partially available
Upgrade
•Firebolt

Firebolt Native Apps Compliance

Test

Suite

•Firebolt JS test suite ( Yet to open source )

Firebolt Native Apps Compliance Suite

Firebolt Certification App ( FCA ) - Open sourcing in progress - Expected ETA - Aug EOM 

Ripple

NewFirebolt Certification App ( FCA ) - Open sourcing in progress - Expected ETA - Aug EOM 

Lightning

Upgrade

Exercise different lightning apps - Refer animation test apps under TDK MVS Test Apps Media Playback And Graphics Validation Test Setup

Strike test tool - https://strike.lightningjs.io/es6/#home

QT for BLE

ExistingManual tests using BLE remote.

Rippl

Not available

Lightning

Not available

QT for BLE

Not available

Core Thunder framework

Available
Upgrade

RDK

Service

API

and Event Test SuiteRDK API

And Event Certification Suite

RDK services test coverage

Testing Areas

  • Execute rdkservices plugins test
  • Perform System testing
  • Perform Bootup testing
  • Execute Performance testing
  • Perform WebApp testing

Browser - WPEWebkit

Available
Upgrade

Browser

performance test suiteBrowser

Performance Suite 

Track the browser performance test scores

rdkcentral/MVT: Media Validation Tool for verification of browser's media playback capabilities 

Screen cast

DIAL

Not available
Existing

Test coverage using Xcast RDK service

validation

Jira
serverJIRA - 2
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId11deff04-0380-3a3d-a916-0849d4e573f7
keyREFPLTV-1591

Jira
serverJIRA - 2
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId11deff04-0380-3a3d-a916-0849d4e573f7
keyRTD131X-571

- RDK API And Event Certification Suite 

Manual tests for real screen validation

Security

DAC (Containerization)

Not available

TBD
NewContainerization Test Framework 

Dobby

In Progress

•Dobby test tool enhancement and integration

Jira
serverJIRA - 2
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId11deff04-0380-3a3d-a916-0849d4e573f7
keyRDKSEC-153

Rialto 

In Progress

•Playback tests using Rialto sink

Jira
serverJIRA - 2
columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
serverId11deff04-0380-3a3d-a916-0849d4e573f7
keyREFPLTV-1594

Expand
titleRiato

Attendees – Abhishek, Sahithya, Sravanthi, Sudhagar, Anil

  • Rialto end purpose is to run in Container mode, till now feature is integrated and tested clear stream playback with Youtube (non-container). Exact containerization mode playback will be added in next releases.
  • Rialto design approach is still in discussion with Sky team on using Rialto sink elements instead of existing Westeros sink
  • TDK playback check with mediapipeline app is fine and timeout issue fix will be pushed from platform team side
  • Other operations like play/pause, seek, trickplay - not sure how well it might work in Rialto. As Rialto is not production ready , may not work similar way as westeros sink.
    • New

    Containerization Test Framework 

    Rialto 

    		NewRialto Gstreamer Test Framework  
    Usage of Youtube specific configuration in the steps - names can be changed, will be taken up as low priority.

    Packager (Lisa)

    Not available

    TBD

    NewRDK API And Event Certification Suite 
    TBD

    AppArmor 

    Not available

    TBD

    Q1

    NewAppArmor Test Framework

    Graphics

    Rendering

    Partially available
    ExistingWesteros ESSOS test apps
    integration

    •

    •
    - Firebolt Native Apps Compliance Suite 
    Graphics ( OpenGL ES )ExistingGlmark2 - Hardware Performance Suite 
    GstreamerGstreamerExisting
    Available
    Gstreamer unit tests
    Gstreamer SoC SDK
    Not available
    ExistingFirebolt Native Apps Compliance Suite

    DRM & Content protection

    PlayReady

    Partially available

    Browser Playback tests using DASHIF test streams
    Existing

    Media Playback And Graphics Validation Suite

    rdkcentral/MVT: Media Validation Tool for verification of browser's media playback 

    Widevine

    Partially available

    Browser Playback tests using DASHIF test streams

    Existing

    Media Playback And Graphics Validation Suite

    Common SVP

    Not available
    New

    TBD


    Verimatrix

    Not available
    New

    TBD


    Build & Platform

    Yocto version

    NA
    Existing

    Kernel

    In Progress
    ExistingKernel
    test - LTP
    Test Framework

    Init system

    		ExistingBasic sanity and Performance tests
    Not available

    Application Manager

    Available
    ExistingRDK Shell service test coverage
    , RDK API And Event Certification Suite

    Improve the application management test coverage

    Q1

    Broadcom SDK

    		Existing
    NAMedia validation test suite
    HAL Certification Suite

    Video Player 

     IP

    Available

    Existing

    Media Playback And Graphics Validation Suite

    •

    rdkcentral/MVT: Media Validation Tool for verification of browser's media playback 

     QAM

    NA

    RDKV RMF components
    ExistingNA
    •

     DVB

    Partially available
    ExistingDVB RDK service test coverage
    , RDK API And Event Certification Suite

    •DVB component and HAL test coverage

     ATSC

    		ExistingNA

     ATSC

    Not available

     TSB

    Partially available
    NewFog test coverage

    •

    Jira
    serverJIRA - 2
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId11deff04-0380-3a3d-a916-0849d4e573f7
    keyREFPLTV-1595

    Jira
    serverJIRA - 2
    columnIdsissuekey,summary,issuetype,created,updated,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,updated,due,assignee,reporter,priority,status,resolution
    serverId11deff04-0380-3a3d-a916-0849d4e573f7
    keyRTD131X-578

     DAC Apps

    Not available

    (Test cases/streams for Live protected contents are not available)

     DAC Apps

    		ExistingContainerization Test Framework 
    RDK API And Event Certification Suite 

    Remote Control and Voice

    Voice Stack

    Not available

    New


    Manual tests for voice remote

    •RDK Service test coverage

    •Voice HAL test coverage


    RF4CE remote

    		Existing

    BLE voice remote

    		Existing

    External Audio support

    		Existing

    Networking

    Wi-Fi

    Available
    Existing
    •Wifi

    •Wi-Fi service test coverage

    , RDK API And Event Certification Suite

    •Wi-Fi

    •Wifi

    HAL test coverage

    , HAL Certification Suite

    RDK API And Event Certification Suite

    LAN

    Available
    ExistingTest execution using ethernet connected DUT

    Device Management

    Telemetry

    Not available

    		Automatics Tests

    TBD

    Feature Control

    Not available

    TBD

    		Automatics Tests
    Crash Upload
    Not available

    		Automatics Tests

    TBD

    Firmware Upgrade

    Not available

    TBD

    Firmware Upgrade


    		Xconf firmware upgrade tests via System service RDK API And Event Certification Suite

    Webpa


    		Automatics tests


    Validation Platforms

    •  Validation support for a single build ( 32 bit ) on each platform
    •  Along with regular RDK builds for RPI HYB and RPI MC, separate Alexa enabled build for the same.
    •  UEI Tatlow, T4H Lima RCUs supported platform builds for validation.

    		Platform NameRemarks
    1RPI HYB




    Complete test suites execution for each RDK6 RC builds

    2RPI MC
    3Realtek SOC Reference
    4Amlogic SOC Reference
    5Broadcom SOC Reference
    6Sercomm - Realtek Video Accelerator
    7Skyworth - Amlogic Video Accelerator
    8Sagemcom - Broadcom Video Accelerator
    9Amlogic TVApplicable test suites execution.

    Validation Process

    • Flash the devices using RDK6 RC builds provided by CMF.
    • Run the tests available in TDK, Automatics and Manual tests against each feature.
    • Attach the test reports including defect mapping in the feature ticket and also provide the tested image details in ticket.
    • Link all open RDK/SOC/OEM issues identified as part of the feature validation in the corresponding feature ticket.
    • If there are no failures then will resolve the ticket as Verified , otherwise Failed and assign back to developer.

    Defect Dashboard

                                Overall defect summary                                                                                                 

    Jira Charts
    bordertrue
    showinfortrue
    serverJIRA - 2
    jqllabels%20%3D%20RDK6_featurefeature%20and%20type%3Dbug
    statTypestatuses
    chartTypepie
    width700
    isAuthenticatedtrue
    serverId11deff04-0380-3a3d-a916-0849d4e573f7

    Open                          Open Issues


    Jira
    serverJIRA - 2
    columnIdsissuekey,summary,issuetype,created,duedate,assignee,reporter,priority,status,resolution
    columnskey,summary,type,created,due,assignee,reporter,priority,status,resolution
    maximumIssues20
    jqlQuerylabels = RDK6_feature and type=bug
    serverId11deff04-0380-3a3d-a916-0849d4e573f7

    ...