Main tasks

Activity	Status
Add Video Skill to existing Developer account	Done
Configure Lambda function and map with VSK	Done Configured lambda function with sample code in AWS Mapped to VSK in existing developer account
Implement lambda functions against existing custom skill utterances (e.g. My Device)	Open There are total 30 voice command utterances which need to be mapped from custom skill to VSK List of Intents captured In this section
Map video skill APIs with RDK skill mapper module	Open Map existing JSON RPC commands with new response pattern from VSK Add new JSON RPC commands for additional features provided by VSK
Build support for AVS Thunder plugin	In Progress Recipe support added. Build support In progress Found some issues with SDK 1.20 Currently trying with 1.18 & 1.19 SDKs
Integration of Smart screen SDK	In Progress Dependencies for ASIO, Web Socket++ added. Recipe for alexa-ss-sdk added. Build support is in progress.
Integration of APL (Alexa Presentation Language) library	Done Build integration complete for APL core library.
Validation of the AVS plugin in video accelerator along with video skill APIs (final target)	Open

List of Custom skill intents

Category	List of Intents	Count (30)
Navigation	MoveLeftIntent, MoveRightIntent, MoveDownIntent, MoveUpIntent, EnterKeyIntent, ExitKeyIntent	6
App Launch	LaunchYouTubeIntent, CloseYouTubeIntent, VimeoIntent, BaebleMusicIntent, RadiolineIntent, WallStreetJournalIntent, AljazeeraIntent, EuroNewsIntent, FranceTwentyFourIntent, CNNNewsIntent AccuWeatherIntent, WeatherNetworkIntent, TrueOrFalseIntent, SugarBoostIntent, GuessThatCityIntent, MainUIIntent	16
Player	VolumeUpIntent, VolumeDownIntent, MuteIntent, UnMuteIntent, ShowVideoIntent, HideVideoIntent	6
Search	GoogleLaunchIntent, GoogleCloseIntent	2

AVS Plugin

AVS plugin is a Thunder plugin developed by metrological/amazon. it supports configurable audio sources and depends on the AVS client SDK.

As of now it only supports general Alexa skills (without any RDK specific skill)

Current source code is built with AVS Device SDK 1.15 & Smart Screen SDK 2.0.1

Source code

Source code is hosted in GitHub

https://github.com/WebPlatformForEmbedded/WPEPluginAVS/blob/master

Configurations Parameters

alexaclientconfig	Path to the AlexaClientSDKConfig.json
smartscreenconfig (optional)	Path to the SmartScreenSDKConfig.json This param will be used only when enablesmartscreen is true
kwdmodelspath (optional)	Path to the Keyword Detection (WWE) models which contain localeToModels.json
audiosource	This can be either portaudio (FFV) or BluetoothRemoteControl (BT)
enablesmartscreen	Optional, true or false - for Alexa Smart Screen
enablekwd	^{Optional, true or false - for WWE}

Methods

Mute

Using this method we can mute the volume coming out of AVS SDK as well we can mute the alerts.

record

Recording voice by Alexa SDK can be toggled on/off. This overrides the hot word detection i.e. if recording is off wake-word will not work.

Notification

dialoguestatechange This notification will come when any state change happens in the Alexa. e.g. when the AVS SDK goes from Idle -> Listening or from Listening -> Thinking -> Speaking

Plugin Build dependencies

CMake	Type	Purpose	Status
FindAlexaClientSDK.cmake	Required	For AVS SDK	Resolved
FindAlexaSmartScreenSDK.cmake	Optional	For Smart Screen SDK	TBA
FindGStreamer.cmake	Required	For Gstreamer	Resolved
FindLibCURL.cmake	Required	For Libcurl	Resolved
FindPortaudio.cmake	Required	For Port Audio	Resolved
FindPryonLite.cmake	Optional	For Pryon Wake-Word-Engine	TBA
FindWebsocketpp.cmake	Optional	Web socket+ For Smart Screen SDK	Resolved
FindYoga.cmake	Optional	Cross platform Layout engine Required for Smart Screen	Resolved
FindAsio.cmake	Optional	ASIO library For Smart Screen SDK	Resolved

Video catalog

With the Video Skill API, you can build skills that let customers control their entire video experience by voice.

Customers can control video devices and consume video content without invoking a specific skill. For example, the customer could do the following:

Start video playback without specifying a provider or device
Control a specific video device:
- "Alexa, turn on my video device"
- "Alexa, lower the volume on my video device"

The Video Skill API exposes below functionalities:

Searching and playing content
Playback controls - Pause, play, rewind, Volume Up/Down
Turning a video device on or off
Changing a video device's input (HDMI etc.)
Launching an app or GUI shortcut
Recording video content
Channel navigation
Video catalog ingestion

Dependency

Alexa developer account
AWS account (for lambda)
OAuth 2.0
Lambda function can be written using Node.js/Python etc.

Questions & Response

Which of the below approach are supported for device end integration
- It should be processed by a native module (skill mapper) & then instruction sent to UI
Any examples to simulate with a dummy operator and other auth fields to verify the video skill end-to-end?
- Example Link : https://github.com/amzn/sample-fire-tv-app-video-skill/tree/master/Lambda
Are Lambda functions Alexa Hosted?
- Lambda functions need to be hosted in users own space (it is not Alexa hosted)

Whether it supports open wake word engines (pryon is not available to all).
- Pryon Lite is provided on demand
How the response from Video skill APIs should be handled?
- The response path from Video skill API is different and will not come back to the AVS SDK. We need to implement communication mechanism similar to that of IOT endpoints.
What are the requirement for RDK UI (any reference UI) for interacting with smart screen (interface, API etc.)
- TBD
What are requirement for RDK UI to render the display card (text) in UI against each voice command
- TBD