You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 30 Next »

Release History

DateFeature Added
01-Feb-2021

Added Video skill kit support & custom skill with the plugin

  • RDK Skill mapper supports custom skill invocation through plugin
  • Skill mapper module can now handle below Video Skill Kit directive
    • Speaker
    • App Launch
    • Channel Controller
    • Playback Controller
    • Keypad Controller
  • Communication with lambda service from RDK client through SQS
  • Build support for KWD
  • integration of AWS CPP SDK
30-Nov-2020

Support for AVS Thunder plugin:

  • Build support added for the AVS Plugin (Currently compatible with SDK 1.15)
  • Switch between AVS service and plugin using DISTRO alexa-plugin
  • Add support to build AVS SampleApp as library
01-Sep-2020

Fixes and skill addition:

  • Added skill mapping for Volume Up, Down, Mute, Unmute and show/hide video functionalities
16-Jul-2020

Initial commit :

  • Added meta-rdk-voice layer
  • Added skill mapper module with JSON RPC mapping and LED state control commands
  • systemd services for alexa service

Components & their availability

Below table lists the components/code used with the RDK Alexa solution and the origin/owner for distribution. components hosted in CMF or github can be accessed part of the release but operators/vendors should contact amazon to get artifacts that are mentioned as "Distributed by Amazon". 

ModuleArtifact NamePurposeLocationOwned by
AVS SDK


wpe-alexa-lib.bbBuild support for AVS Device SDKCMF HostedRDKM
avs-device-sdkSource code for AVS Device SDKhttps://github.com/aws/aws-sdk-cppAmazon
RDK Skill Mapperskillmapper.bbBuild support for skill mapper moduleCMF HostedRDKM
alexa_skill_mapper

RDK component to:

  • map VSK and custom skill responses with Thunder RPC
  • Handle state notification
  • Communicate between CPE and lambda service
CMF HostedRDKM
AWS SDKaws-cpp-sdk.bbBuild support for AWS CPP SDK moduleCMF HostedRDKM
aws-cpp-sdkProvides APIS to integrate AWS services such as SQS, Lambda, cognito etc. to RDK client devices.https://github.com/aws/aws-sdk-cppAmazon
KWD Detectorpryonlite_2.3.0.bbBuild support for Amazon KWD engine.CMF HostedRDKM

PryonLite pre-built libraryProvides a Wake Word Engine that can be used with the Alexa plugin. This is useful for devices with far-field/microphone.Distributed by AmazonAmazon
AVS Pluginwpeframework-plugins.bbBuild support for the Amazon voice thunder pluginHosted in rdkcentral GitHubRDKM
AVS pluginAmazon voice thunder plugin for Alexa service, uses AVS SDK & smart screen SDK APIS as part of the implementationHosted in Metrological GitHubMetrological
Lambda Demo codelambda-code.zipReference implementation for receiving video skill directives from alexa and routing it to the RDK device. Uses lambda & SQS services.Distributed by AmazonAmazon

Platform status

PlatformBuild supportPush-to-talkHands Free

Wake-Word-Engine

(Supported)

Alert SoundsLEDsGeneral SkillRDK Specific Skill
RaspberryPiCompleteT4H BLE RCUUSB MICKitt.AiAvailable

Traffic Light LEDs

Integrated

Integrated

RDK Accelerator

CompleteT4H BLE RCUUSB MICKitt.AiAvailableFront Panel LEDsIntegrated

Integrated

Feature List & Custom Skill

FeatureHandlerHow to?
General skillsAlexaAudio will be played for generic skills such as time, news and weather
YoutubeAlexa → CobaltVoice command will be routed to Thunder
App LaunchAlexa → UXVoice command will be routed to Thunder
Google searchAlexa → UXInteractive command handled by Alexa and routed to Thunder
Player controlsAlexa → AppPlay/pause/back commands routed to app

Video Skill

Please refer to below page on integrating the Video Skill Kit APIs with RDK

Integrating Alexa Video Skill Kit with RDK

Build instructions

Alexa related recipes are bundled inside meta-rdk-voice/recipes-avs and contains below build targets

RecipeFunctionalityCurrent status
openblasRecipe required in case of Wake-word-engine mode is selected.CMF Hosted
wpe-alexa.bbMain recipe for downloading the AVS SDK and its compilation, linking it with RDK skill mapping module.CMF hosted

Build instructions for RaspberryPI

1. Initialize and download the code base
 - Example for RaspberryPi:
 $ repo init -u https://code.rdkcentral.com/r/manifests -m rdkv-nosrc.xml -b thunder-next
 $ repo sync

2. Download the meta-rdk-voice layer in project root
 $ git clone https://code.rdkcentral.com/r/components/generic/rdk-oe/meta-rdk-voice

3. Add to BBLAYERS
 $ vi meta-rdk/conf/bblayers.conf.sample
 BBLAYERS += "${@'${RDKROOT}/meta-rdk-voice' if os.path.isfile('${RDKROOT}/meta-rdk-voice/conf/layer.conf') else ''}"

4. Optionally if wanted to use MIC based implementation (Ignore if using BLE voice remote)
 $ vi meta-rdk-voice/conf/layer.conf
 - enable the below commented line
 #DISTRO_FEATURES_append = " alexa_ffv"

 # Please follow the steps mentioned in section #3.2 to enable USB audio kernel parameters (Not required for RaspberryPI).

5. Optionally For the external source builds (such as some Broadcom platforms) add below line in manifest/auto.conf
 - In <Manifest>.xml
   <project name="components/generic/avs/alexa_skill_mapper" revision="rdk-next"/>
 - In auto.conf (where brcmexternalsrc is enabled)
   BRCMEXTERNALSRC_pn-wpe-alexa += "components/generic/avs/alexa_skill_mapper:git/VoiceToApps"
   SRCPV_pn-wpe-alexa = "${BRCMEXTERNAL-SRCPV-CMF}"
 $ repo sync
6 .Build the image target
 - Example for RaspberryPi
 $ source meta-cmf-raspberrypi/setup-environment
 ### select meta-cmf-raspberrypi/conf/machine/raspberrypi-rdk-hybrid-thunder.conf ###
 $ bitbake rdk-generic-hybrid-thunder-image

Kernel configurations to simulate FFV with an USB microphone

Note: This configuration is not required for RaspberryPi platforms
# If FFV profile is enabled (DISTRO_FEATURES_append = " alexa_ffv") & we want to simulate audio input using a USB mic the below kernel configuration need to be enabled for non RPI platforms:

CONFIG_SND_HWDEP=m
CONFIG_SND_RAWMIDI=m
CONFIG_SND_USB_AUDIO=m

# After enabling the above kernel configs, recompile the kernel so that the support for USB audio will be enabled

Customization

Mapping to amazon developer account

Please refer the wiki link AVS Developer Account and Skill Creation Guidelines for RDK for creating a developer account.

Modify below section to map with relevant developer account.

1. Login to the device
 $ ssh root@<RDKV-Client-IP>

2. Edit the Configuration file
 $ cd /home/root
 $ vi Alexa_SDK/Integration/AlexaClientSDKConfig.json
     "deviceInfo":{
        // Unique device serial number. e.g. 123456
        "deviceSerialNumber":"<SERIAL_NO>",
        // The Client ID of the Product from developer.amazon.com
        "clientId":"<CLIENT_ID>",
        // Product ID from developer.amazon.com
        "productId":"<PRODUCT_ID>",
        // The name of the device manufacturer.
        "manufacturerName": "<MANUFACTURER_NAME>",
        // The description of the device which should contain the manufacturer name or how the device is connected.
        "description": "<DESCRIPTION"
    },

LED Support

Prerequisite : Alexa Client uses IOConnector plugin to set different LED notifications, so IOConnector need to be enabled for the platform

https://github.com/rdkcentral/ThunderNanoServices/tree/master/IOConnector

$ cd /home/root
$ vi Alexa_SDK/Integration/AlexaCurl.json
Set below GPIO to appropriate value
   "PinOne":"9",
   "PinTwo":"10",
   "PinThree":"11"

Example: For RaspberryPi we can use the Traffic light LED where GPIO PIN 9, 10, 11 represents the Red, Yellow & Green LEDS 

The above customization can be done either during build time or in the device. If it is done on the device, a reboot will be required after modifications are done.


Alexa user guide

Introduction

This Feature consists two mode "General Mode" and "Skill Mode",

  • in the General Mode you ask general quires (ex: time , weather ,news)
  • in the Skill Mode you can launch applications ,navigate and other functionality.

One can switch from the "General Mode" to "Skill Mode" by using steps mentioned in the section "skill invocation enable" and you can switch back to "General Mode" following the steps in the section "skill invocation disable".

Note: By default when device is booted it will be in the general mode


Note :click on the mic button on the RDK Tech4Home remote and give voice command and release the button

First time authorization

On first time launch, we need to authorize the device with a amazon account. This steps will not be required for subsequent application launch.

We can check the authorization code using below command:

$ systemctl status alexa

We will be able to see a message similar to below format. Note the code that appears in console window, this will be required while authorizing with developer account.

##################################
#       NOT YET AUTHORIZED       #
##################################
#       To authorize, browse to: 'https://amazon.com/us/code' and enter the code: EXEA99
###############################################################
#       Checking for authorization (1)...       #
################################################# 
  1. Open "https://amazon.com/us/code" in a browser
  2. Login with the amazon developer account credential and provide the code that was displayed on the device console (one time only)

 

Note: If there is a prompt for OTP at this stage, Get it from linked phone number.

  1. After the authorization below log is displayed on the console
########################################
#       Alexa is currently idle!       #
########################################

At this stage, we can provide voice commands to alexa.

Example Usage

a) General Usage
ex:  1) User: what is the the time?
          Alexa: respond with current time
      2) User : what is the weather outside ?
          Alexa: it will ask for you location
          User: if give the Location
          Alexa: respond with the climate in the current location.


b) Skill Invocation Enable

       User: My Device
       Alexa: respond with "how can i help you"
       User : invocation enable
       Alexa : respond with "invocation enabled"


c) Skill Invocation Disable

       (note :It is done when "skill mode" is already enabled & we want to switch back to "general mode" )

       User: invocation disable
       Alexa: respond with "invocation disabled"

d) Launching Metro Applications

ex :  1) launching RadioLine Application

          User: radioline
          Alexa : respond with "radioline" by launching the application

          User : "metro" or "main UI"
          Alexa: speaks back "main UI" & navigation moves back to metro UI main screen


       2) launching Wall Street Journal Application

          User: WallStreetJournal
          Alexa : respond with "WallStreetJournal" by launching the application

         User : "metro" or "main UI"
         Alexa: speaks back "main UI" & navigation moves back to metro UI main screen


Architecture & Flow diagram

AVS Integration with VSK

AVS Integration with custom skills

Advantages

  • Configurable to support different device profiles (FFV, Push-to-talk).
  • Most of the new skills can be dynamically added without modifying the client code.

Scope of improvement

FeatureStatus
Context specific functionalitiesIn progress
Multi room MusicOpen
Integration with other Alexa productsOpen
Smart Screen integrationOpen
  • No labels