You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 37 Next »

Release History

DateFeature Added
31-Mar-2021

Alexa Smart Screen support added

  • Smart screen build support is available with a DISTRO feature
  • Voice chrome integration available
  • Smart screen integrated as UI Overlay
  • Control smart screen through skill mapper

AVS plugin integration with RDK services

  • Skill maps updated to make use of RDK service plugins
  • RDK Shell interfacing available for the JSON RPCs

Voice control support for new RDK UI

  • volume control functionalities routed through DisplaySettings plugin
  • Key navigations using RDK Shell
  • Supports new launch shortcuts for Bluetooth & Wi-FI settings
  • Application launch supported using Cobalt, ResidentApp and LightningApp plugins
01-Feb-2021

Added Video skill kit support & custom skill with the plugin

  • RDK Skill mapper supports custom skill invocation through plugin
  • Skill mapper module can now handle below Video Skill Kit directive
    • Speaker
    • App Launch
    • Channel Controller
    • Playback Controller
    • Keypad Controller
  • Communication with lambda service from RDK client through SQS
  • Build support for KWD
  • integration of AWS CPP SDK
30-Nov-2020

Support for AVS Thunder plugin:

  • Build support added for the AVS Plugin (Currently compatible with SDK 1.15)
  • Switch between AVS service and plugin using DISTRO alexa-plugin
  • Add support to build AVS SampleApp as library
01-Sep-2020

Fixes and skill addition:

  • Added skill mapping for Volume Up, Down, Mute, Unmute and show/hide video functionalities
16-Jul-2020

Initial commit :

  • Added meta-rdk-voice layer
  • Added skill mapper module with JSON RPC mapping and LED state control commands
  • systemd services for alexa service

Components & their availability

Below are the components/code used with the RDK Alexa solution and the origin/owner for distribution.

  • Components hosted in CMF or github can be accessed part of the release
  • Operators/vendors should contact amazon POC to get the artifacts marked as "Distributed by Amazon". 
ModuleArtifact NamePurposeLocationCurrentVersionOwned by
AVS SDK


wpe-alexa-lib.bbBuild support for AVS Device SDKCMF Hosted (meta-rdk-voice)1.15RDKM
avs-device-sdkSource code for AVS Device SDKhttps://github.com/aws/aws-sdk-cppAmazon
RDK Skill Mapperskillmapper.bbBuild support for skill mapper moduleCMF Hosted (meta-rdk-voice)
RDKM
alexa_skill_mapper

RDK component to:

  • map VSK and custom skill responses with Thunder RPC
  • Handle state notification
  • Communicate between CPE and lambda service
CMF Hosted
RDKM
AWS SDKaws-cpp-sdk.bbBuild support for AWS CPP SDK moduleCMF Hosted (meta-rdk-voice)
RDKM
aws-cpp-sdkProvides APIS to integrate AWS services such as SQS, Lambda, cognito etc. to RDK client devices.https://github.com/aws/aws-sdk-cpp1.6.53Amazon
KWD Detectorpryonlite_2.3.0.bbBuild support for Amazon KWD engine.CMF Hosted (meta-rdk-voice)
RDKM

PryonLite pre-built libraryProvides a Wake Word Engine that can be used with the Alexa plugin. This is useful for devices with far-field/microphone.Distributed by Amazon2.3.0Amazon
AVS Pluginwpeframework-plugins.bbBuild support for the Amazon voice thunder pluginHosted in rdkcentral GitHub
RDKM
thunder-services.bbBuild support for the AVS plugin with RDK servicesCMF Hosted
RDKM
AVS pluginAmazon voice thunder plugin for Alexa service, uses AVS SDK & smart screen SDK APIS as part of the implementationHosted in Metrological GitHub
Metrological
Lambda Demo codelambda-code.zipReference implementation for receiving video skill directives from Alexa and routing it to the RDK device. Uses lambda & SQS services.For Reference Only
AVS Smart Screen SDKalexa-ss-sdk_git.bbMain recipe to build Alexa smart screen SDKCMF Hosted (meta-rdk-voice)
RDKM
alexa-smart-screenSource code for the smart screen SDKhttps://github.com/alexa/alexa-smart-screen-sdk/2.0.1Amazon
avs-voice-chromeSource code to add voice chrome support over smart screen UIDistributed by Amazon2.1Amazon
APL Coreapl-core_git.bbRecipe to build Alexa Presentation LanguageCMF Hosted (meta-rdk-voice)
RDKM

APL Core LibrarySource code for rendering of screen layouts at runtime for the Alexa Presentation Languagehttps://github.com/alexa/apl-core-library1.2Amazon
WebSocket++websocketpp_0.8.1.bbRecipe to build the C++ websocket client libraryCMF Hosted (meta-rdk-voice)
RDKM

C++ websocket client/server library

https://github.com/zaphoyd/websocketpp0.8.1Open source
ASIOasio_1.10.6.bbrecipe to build C++ Network library meta-openembedded1.10.6Open source






Skill Overview

FeatureHandlerHow to?
General skillsAlexaAudio will be played for generic skills such as time, news and weather
YoutubeAlexa → CobaltCommand will be routed to Thunder

App Launch

(metrological codebase)

Alexa → UX


Command will be routed from skill mapper  To UX browser via Controller

App Launch

(Firebolt codebase)

Alexa → RDK Shell → LightningAppCommands routed from skill mapper to LightningApp or HtmlApp browser instances via RDK Shell
Google searchAlexa → UXInteractive command handled by Alexa and routed to Thunder
Player controlsAlexa → AppPlay/pause/back commands routed to app
Smart Screen UIAVS Plugin → WebKitbrowserInvoked by AVS plugin & UI launched using WebKit browser instance

Video Skill

Please refer to below page on integrating the Alexa Video Skill Kit with RDK

Integrating Alexa Video Skill Kit with RDK

Build instructions

The build instructions are mentioned by taking RaspberryPi as an example but should be almost similar for other platforms.

Build Instructions to run Alexa from AVS Plugin

1. Initialize and download the code base
 - Example for RaspberryPi:
 $ repo init -u https://code.rdkcentral.com/r/manifests -m rdkv-nosrc.xml -b thunder-next
 $ repo sync

2. Download the meta-rdk-voice layer in project root
 $ git clone https://code.rdkcentral.com/r/components/generic/rdk-oe/meta-rdk-voice

3. Enable the below DISTRO feature to build AVS plugin
 $ vi meta-rdk-voice/conf/layer.conf
 - enable the below commented line
 #DISTRO_FEATURES_append = " alexa-plugin"
4. Optionally if wanted to use MIC based implementation (Ignore if using BLE voice remote)
 $ vi meta-rdk-voice/conf/layer.conf
 - enable the below commented line
 #DISTRO_FEATURES_append = " alexa_ffv"

 - Follow the section (Enable Keyword Detector) for using amazon provided KWD

 # Please follow the steps mentioned in section #3.2 to enable USB audio kernel parameters (Not required for RaspberryPI).

5. Optionally For the external source builds (such as some Broadcom platforms) add below line in manifest/auto.conf
 - In <Manifest>.xml
   <project name="components/generic/avs/alexa_skill_mapper" revision="rdk-next"/>
 - In auto.conf (where brcmexternalsrc is enabled)
   BRCMEXTERNALSRC_pn-wpe-alexa += "components/generic/avs/alexa_skill_mapper"
   SRCPV_pn-wpe-alexa = "${BRCMEXTERNAL-SRCPV-CMF}"
 $ repo sync
6 .Build the image target
 - Example for RaspberryPi
 $ source meta-cmf-raspberrypi/setup-environment
 ### select meta-cmf-raspberrypi/conf/machine/raspberrypi-rdk-thunder-mc.conf ###
 $ bitbake rdk-generic-mediaclient-image

Build instructions to run Alexa as systemd service

1. Initialize and download the code base
 - Example for RaspberryPi:
 $ repo init -u https://code.rdkcentral.com/r/manifests -m rdkv-nosrc.xml -b thunder-next
 $ repo sync

2. Download the meta-rdk-voice layer in project root
 $ git clone https://code.rdkcentral.com/r/components/generic/rdk-oe/meta-rdk-voice

3. Optionally if wanted to use MIC based implementation (Ignore if using BLE voice remote)
 $ vi meta-rdk-voice/conf/layer.conf
 - enable the below commented line
 #DISTRO_FEATURES_append = " alexa_ffv"

 # Please follow the steps mentioned in section #3.2 to enable USB audio kernel parameters (Not required for RaspberryPI).

4. Optionally For the external source builds (such as some Broadcom platforms) add below line in manifest/auto.conf
 - In <Manifest>.xml
   <project name="components/generic/avs/alexa_skill_mapper" revision="rdk-next"/>
 - In auto.conf (where brcmexternalsrc is enabled)
   BRCMEXTERNALSRC_pn-wpe-alexa += "components/generic/avs/alexa_skill_mapper:git/VoiceToApps"
   SRCPV_pn-wpe-alexa = "${BRCMEXTERNAL-SRCPV-CMF}"
 $ repo sync
5 .Build the image target
 - Example for RaspberryPi
 $ source meta-cmf-raspberrypi/setup-environment
 ### select meta-cmf-raspberrypi/conf/machine/raspberrypi-rdk-thunder-mc.conf ###
 $ bitbake rdk-generic-mediaclient-image

Enable Keyword detector support

1. To use pryon lite KWD engine for devices with MIC input
 Note: Source code need to be obtained from amazon separately
2. Set the KWD support flag to "ON"
  $ vi meta-rdk-voice/recipes-wpe/wpeframework/include/avs.inc
    AVS_KWD_SUPPORT ?= "ON"
3. include the KWD detector recipe to build.
 - enable the commented line 
 #PREFERRED_PROVIDER_virtual/alexa-kwd-detector = "pryonlite"
4. Create a bbappend file in platform layer and provide artifact location
e.g. 
 $ vi meta-rdk-oem-<OEMNAME>/meta-<MACHINE>/recipes-avs/pryonlite/pryonlite_2.3.0.bbappend
   SRC_URI  = "file://pryon_lite_2.3.0-metrological-2019.12.23.2344.zip"


Kernel configurations to simulate FFV with an USB microphone

# If FFV profile is enabled (DISTRO_FEATURES_append = " alexa_ffv") & we want to simulate audio input using a USB mic the below kernel configuration need to be enabled for the platforms:

CONFIG_SND_HWDEP=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_USB_AUDIO=m

# After enabling the above kernel configs, recompile the kernel so that the support for USB audio will be enabled

Enable Alexa Smart Screen support

$ vi meta-rdk-voice/conf/layer.conf
 - enable the below commented DISTRO_FEATURE
  #DISTRO_FEATURES_append = " alexa_smart_screen"

Enable Voice chrome

1. Enable DISTRO options
$ vi meta-rdk-voice/conf/layer.conf
 - enable the below commented DISTRO_FEATURE
  #DISTRO_FEATURES_append = " voice-chrome"
2. Provide artifact location
 - Add an .bbappend file for alexa-ss-sdk and over-write the SRC_URI
e.g.
 $ vi meta-rdk-oem-OEMNAME/meta-<MACHINE>/recipes-avs/alexa-smart-screen/alexa-ss-sdk_git.bbappend
   SRC_URI_remove = "https://operator-artifact-url.com/location/SmartScreenSDKVoiceChrome-2.1.tar;name=voicechrome;subdir=voicechrome"
   SRC_URI += "file://SmartScreenSDKVoiceChrome-2.1.tar;name=voicechrome;subdir=voicechrome"

How to run the service

For AVS Plugin

  1.  Go to the controller web UI by typing http://<Device-IP> in a browser
  2. Select the controller option from left pane menu
  3. toggle the AVS option to activate the plugin

For systemd service (old implementation)

  1. Login to the device with "ssh root@<Device-IP>
  2. start the alexa service by invoking "systemctl start alexa.service"

Customization

Mapping to amazon developer account

You should already have a developer account, Please refer the wiki link AVS Developer Account and Skill Creation Guidelines for RDK for creating a developer account.

Modify below section to map with relevant developer account.

1. Login to the device
 $ ssh root@<RDKV-Client-IP>

2. Edit the Configuration file
 $ cd /usr/share/WPEFramework/AVS
 $ vi AlexaClientSDKConfig.json
     "deviceInfo":{
        // Unique device serial number. e.g. 123456
        "deviceSerialNumber":"<SERIAL_NO>",
        // The Client ID of the Product from developer.amazon.com
        "clientId":"<CLIENT_ID>",
        // Product ID from developer.amazon.com
        "productId":"<PRODUCT_ID>",
        // The name of the device manufacturer.
        "manufacturerName": "<MANUFACTURER_NAME>",
        // The description of the device which should contain the manufacturer name or how the device is connected.
        "description": "<DESCRIPTION"
    },

Front Panl LED Notifications

  • Customization can be done either during build time or in the device. If it is done on the device, a reboot will be required after modifications are done.
  • This feature is not integrated with RDK services currently.

Prerequisite : Alexa Client uses IOConnector plugin to set different LED notifications, so IOConnector need to be enabled for the platform

https://github.com/rdkcentral/ThunderNanoServices/tree/master/IOConnector

$ cd /home/root
$ vi Alexa_SDK/Integration/AlexaCurl.json
Set below GPIO to appropriate value
   "PinOne":"9",
   "PinTwo":"10",
   "PinThree":"11"

Example: For RaspberryPi we can use the Traffic light LED where GPIO PIN 9, 10, 11 represents the Red, Yellow & Green LEDS 

Alexa usage guide

Introduction

How To give voice Input:

  1. Using the Tech4Home BLE Remote - Press & hold the mic button → provide the voice input → release the button.
  2. Using the on-board (or USB) microphone - Wake-up the system with keyword "Alexa" → provide the voice input.

This Feature can be divided as : General skills , Custom Skills Or Video Skill,

  • in General Mode you ask general quires (ex: time , weather ,news)
  • in Custom Skill Mode you can launch applications ,navigate and other functionality.

One can switch from the "General Mode" to "Skill Mode" by using steps mentioned in the section "skill invocation enable" and you can switch back to "General Mode" following the steps in the section "skill invocation disable".

Note: By default when device is booted it will be in the general mode

First time authorization

On first time launch, we need to authorize the device with a amazon account. This steps will not be required for subsequent application launch.

We can check the authorization code using below command:

  • In case of systemd implementation:

$ systemctl status alexa

  • In case of plugin implementation

$ tail -f /opt/logs/wpeframework.log

We will be able to see a message similar to below format. Note the code that appears in console window, this will be required while authorizing with developer account.

##################################
#       NOT YET AUTHORIZED       #
##################################
#       To authorize, browse to: 'https://amazon.com/us/code' and enter the code: EXEA99
###############################################################
#       Checking for authorization (1)...       #
################################################# 
  1. Open "https://amazon.com/us/code" in a browser
  2. Login with the amazon developer account credential and provide the code that was displayed on the device console (one time only)

 

Note: If there is a prompt for OTP at this stage, Get it from linked phone number.

  1. After the authorization below log is displayed on the console
########################################
#       Alexa is currently idle!       #
########################################

At this stage, we can provide voice commands to Alexa.

Alexa Smart Screen

The Alexa Smart Screen SDK extends the AVS Device SDK to support visual display for screen-based Alexa Built-in products that complement Alexa voice responses with rich visual experiences.

The Alexa Smart Screen implementation for RDK involves below components:

  • The Alexa Smart Screen SDK
  • A GUI web app that handles presentation of Alexa visual responses
  • skill mapper which controls the overlay

Below are few screen shots which gives an idea about the smart screen integration for different scenarios:

a) Weather Information:

b) Music Playback

Example Usage

a) General Usage
ex:  1) User: what is the the time?
          Alexa: respond with current time
      2) User : what is the weather outside ?
          Alexa: it will ask for you location
          User: Provides the Location
          Alexa: respond with the climate in the current location.

          In case smart screen support enabled, the overlay will render the weather tiles.

b) Using video skill launch application

ex: 1) Launching YouTube

         User: Alexa, Launch YouTube

         Alexa: Getting Youtube from Accelerator

          Output: Youtube is launched on the device

c) Using the custom skill to Launching Metro Applications

Note: Need to mention the skill name for invocation (skill name is My Device in below examples)

ex :  1) launching RadioLine Application

          User: tell my device to launch radioline
          Alexa : respond with "radioline" by launching the application

          User : "metro" or "main UI"
          Alexa: speaks back "main UI" & navigation moves back to metro UI main screen


       2) launching Wall Street Journal Application

          User: tell my device to launch WallStreetJournal
          Alexa : respond with "WallStreetJournal" by launching the application

         User : "metro" or "main UI"
         Alexa: speaks back "main UI" & navigation moves back to metro UI main screen

Architecture & Flow diagram

AVS Integration with Video Skill & RDK Services

AVS Integration with custom skills

Roadmap

FeatureStatus
Upgrade Alexa SDKs to latest version: ADSDK (1.21), Smart Screen (2.5)Planned in April-21 Sprint
Extending support to 2 new platformsPlanned for next sprint
Context specific functionalitiesPlanned support for some apps initially
Multi room MusicOpen
  • No labels