Overview

LISA component is part of RDK DAC framework on STB. Its role is to retrieve, store and maintain DAC bundles and local applications' metadata. LISA is keeping track of what packages are available locally and managing the local storage, both for downloadable bundles as well as persistent storage for the applications. LISA is an abbreviation from Local Inventory & Storage Manager of DAC Apps.

...

Relation to RDK Packager

LISA is an alternative to the Packager Thunder Plugin used to retrieve ipkg packages, see https://github.com/rdkcentral/rdkservices/blob/master/Packager/doc/PackagerPlugin.md   

Comparison of LISA with RDK Packager API 

Data Types

LISA is able to handle the following data:

1. Application-specific:
   • Mandatory application metadata (type, id, version, source URL, name, category)
     • setting on installation request
     • getting
   • DAC bundles
     • downloading
     • deleting
   • Generic resources (files, e.g. icons)
     • downloading,
     • deleting
   • Auxiliary application metadata, dedicated API for:
     • setting
     • updating
     • getting
   • Persistent storage for application
     • creating
     • resetting
     • deleting
   • Storage/persistent storage usage checking
     • getting
2. Global
   • Storage usage checking
     • getting

API

Formal definition

Formal definition and Thunder plugin json schema are available in LISA API (TODO: link correction). The below overview is a short reference useful for understanding of expected use cases.

Overview

API to be exposed in JSON-RPC over WebSocket 

...

{payload} - A Structured value that holds the parameter values to be used during the invocation of the method. This member MAY be omitted.

Methods

• install 

Download the application bundle tarball from the given URL, unpack files to dedicated directory, create the db entries and persistent storage on success. Asynchronous operation, handle should be used for tracking the request. The id of an app is unique within the database. That means only one kind of (mime)type can be combined and saved with a specific app id.

...

Events

• operationStatus

Notification sent on the completion of asynchronous operation.

...

Download progress tracking

Tracking of the download progress is implemented using:

• requesting for Content-Length HTTP header before actual transfer, to check for available space and to store the size of the resource
• updating the counter of downloaded bytes on the successful transfer of each data chunk (see libcurl CURLOPT_WRITEFUNCTION)
• returning the updated value of progress parameter on invocation of getProgress API.

Block diagram

Logical modules

LISA auth module

API to be implemented by the library:

• getAuthenticationMethod(appType, id, URL); returned value should indicate one of method:
  • NONE
  • BASIC_AUTH
  • API_KEY_IN_REQUEST
  • API_KEY_IN_HEADER
  • API_KEY_IN_COOKIE
  • CLIENT_CERT
  • BEARER_TOKEN (Oauth2)
• getAPIKey(appType, id, URL, *key, bufferSize); value returned in key buffer contains API key to be included to the request; returns 0 on success, 1 on error (buffer too small), 2 on other errors
• getCredentials(appType, id, URL, *user, *password, bufferSize); returns user/apssword; returns 0 on success, 1 on error (buffer too small), 2 on other errors
• getClientCertsFile(appType, id, URL, *privKeyFileName, *privKeyPassphrase, *clientCertFileName, bufferSize); returns certificate file names; returns 0 on success, 1 on error (buffer too small), 2 on other errors
• getToken(appType, id, URL, *token, bufferSize); returns bearer token; returns 0 on success, 1 on error (buffer too small), 2 on other errors

Metadata

JSON metadata data model

sqlite schema

...

-- enable support for foreign keys, available since sqlite version 3.6.19 (2009-10-14)
PRAGMA foreign_keys = ON;

-- table for application ids and persistent storage
CREATE TABLE IF NOT EXISTS apps(
idx INTEGER PRIMARY KEY, -- unique index for the application type/id
type TEXT NOT NULL, -- MIME type 
app_id TEXT UNIQUE NOT NULL, -- unique id of app within given MIME type
data_path TEXT, -- path to app peristent storage, relative to [APPS_STORAGE]/{epoch}
created TEXT NOT NULL -- unix epoch timestamp of when the application entry was created 
);

-- table for installed applications in specific versions
CREATE TABLE IF NOT EXISTS installed_apps (
idx INTEGER PRIMARY KEY, -- unique index for the application type/id/version
app_idx INTEGER NOT NULL, -- reference to apps table, index of the application type/id
version TEXT NOT NULL, -- version of the application
name TEXT NOT NULL, -- name of the application
category TEXT, -- category
url TEXT, -- URL from which the application was downloaded
app_path TEXT, -- path to app files, realtive to [APPS]/{epoch}
created TEXT NOT NULL, -- unix epoch timestamp of when the application was installed
resources TEXT, -- json object with downloaded resources metadata
metadata TEXT, -- json object with auxiliary resources metadata
FOREIGN KEY(app_idx) REFERENCES apps(idx),
UNIQUE(app_idx, version)
);

sqlite quick startup 

1. print tables
SELECT sql FROM sqlite_master WHERE name = 'apps';
SELECT sql FROM sqlite_master WHERE name = 'installed_apps';

...

6. set meatdata for given app/version
SELECT DISTINCT installed_apps.idx,installed_apps.metadata FROM installed_apps,apps WHERE apps.type='application/vnd.rdk-app.dac.native' AND apps.app_id='com.lgi.app3' AND installed_apps.version="1.0.0";
UPDATE installed_apps SET metadata = '{[{"description","New nice application."},{"parental":"5"}]}' WHERE idx=<idx>;

Storages for data

The logical storages: [APPS] and [APPS_STORAGE] should be defined per platform, to use concrete physical storages.

Epoch

Epoch indicator {epoch} is used for easy invalidation of not supported or revoked format of apps on fundamental changes. May also be used for the antirollback (db authentication - TBD?)

Application files: [APPS]

Apps files are stored in [APPS] storage. 

...

• App's files: [APPS]/dac/images/{epoch}/{id}/{version}/
• App's auxiliary resources (e.g. icons), currently not implemented: [APPS]/dac/images/{epoch}{id}/{version}/res
• LISA's db:  [APPS]/dac/db/{epoch}/apps.db
• Temporary files storage (for downloaded files before unpacking), referred as [APPS_TMP]: 
  • [APPS_TMP] is located in [APPS]/dac/images/tmp
  • [APPS_TMP]/{id}/{version}/ - used for downloading application of given id and given version

Application persistent storage: [APPS_STORAGE]

Apps persistent storage is located in [APPS_STORAGE]. 

...

Apps' persistent storage: [APPS_STORAGE]/dac/{epoch}/{id}/

LISA operations

Initialization

Maintenance and Cleanup

Install

Uninstall

Download

LISA implementation

LISA should be implemented as a Thunder plugin.

An implementation may be based on the Packager code.

Lisa Thunder plugin configuration

{
	"$schema": "plugin.schema.json",
    "jsonrpc":"2.0",
	"info": {
		"title": "Local Inventory & Storage Manager of DAC Apps (LISA) Plugin",
        "class": "Lisa",
		"callsign": "org.rdk.dac.lisa",
		"locator": "libWPEFrameworkLisa.so",
		"status": "alpha",
		"autostart": true,
		"description": "The Local Inventory & Storage Manager of DAC Apps (LISA) plugin allows authenticated downloading and management of DAC bundles from a remote server. It also manages the persistent storage of the DAC applications.",
		"version": "1.0"
	},
	"interface": {
		"$ref": "{interfacedir}/lisa.json#"
	},
	"configuration": {
		"type": "object",
		"properties": {
			"configuration": {
				"type": "object",
				"properties": {
					"network": {
						"type": "object",
						"required": [
							"timeout",
							"default_retryIn"
						],
						"properties": {
							"timeout": {
								"type": "number",
								"description": "Single resource downloading timeout in seconds"
							},
							"default_retryIn": {
								"type": "number",
								"description": "Default value of GET retry delay in seconds on HTTP/202, when no "Retry-After" header field is returned in the response."
							}
						}
					},
					"storages": {
						"type": "object",
						"required": [
							"apps",
							"apps_storage",
							"apps_tmp"
						],
						"properties": {
							"apps": {
								"type": "string",
								"description": "Path to the persistent storage for downloaded applications top dir"
							},
							"apps_storage": {
								"type": "string",
								"description": "Path to the persistent storage for downloaded applications persistent storage top dir"
							},
							"apps_tmp": {
								"type": "string",
								"description": "Path to the temporary storage for applications being downloaded"
							}
						}
					}
				},
				"required": [
					"timeout",
					"storages"
				]
			}
		}
	}
}

{
	"$schema": "plugin.schema.json",
	"jsonrpc":"2.0",
	"info": {
		"title": "Local Inventory & Storage Manager of DAC Apps (LISA) Plugin",
        "class": "Lisa",
		"callsign": "org.rdk.dac.lisa",
		"locator": "libWPEFrameworkLisa.so",
		"status": "alpha",
		"autostart": true,
		"description": [
			"The Local Inventory & Storage Manager of DAC Apps (LISA) plugin allows authenticated downloading and management of DAC bundles from a remote server. It also manages the persistent storage of the DAC applications."
		],
		"version": "1.0"
	},
	"interface": {
		"$ref": "{interfacedir}/lisa.json"
	},
	"configuration": {
		"storages": {
			"apps": "/mnt/apps",
			"apps_storage": "/mnt/apps_storage",
			"apps_tmp": "/mnt/apps_tmp"
		},
		"network": {
			"timeout": 1800,
			"default_retryIn": 300
		}
	}
}

Development Phasing

• Phase 1: basic API implementation and libs integration:
  • initialization with basic recovery on errors (removal of all data)
  • installation
    • libcurl HTTP get with HTTPS and TLS session resumption
    • HTTP code 202 back-off
    • persistent storage creation
    • untarring downloaded package
    • no authentication
    • no full metadata support (i.e. auxmetadata and resources)
    • handling of multiple versions of the same application
    • timeout is excluded
  • uninstall
    • full wipeout
  • operationStatus
    • basic status Success/Error
  • getStorageDetails
    • no params (overall) Request payload: {}
    • single installed application Request payload: {"type":"application/vnd.rdk-app.dac.native", "id":"<string>", "version":"<string>"}
  • getList:
    • no filters
    • single app  {"type":"application/vnd.rdk-app.dac.native", "id":"<string>", "version":"<string>"}
  • getMetadata
    • single app Request payload: {"type":"application/vnd.rdk-app.dac.native", "id":"<string>", "version":"<string>"}
  • getProgress
  • auth module API interface + stub library
  • Configure logical storages on all plaforms
  • Locking support (lock, unlock, getLockInfo) to prevent app from being removed while in use. LISA API Extension (Locking mechanism)
• Phase 2: whole API
  • full initialization with cleanup and maintenance added (already done)
  • cleanup and maintenance after each operation (already done for install and uninstall)
  • installation
    • full metadata
    • configurable download timeout implementation
  • persistent storage management (full)
    • creation/deletion/reset
    • quota management and enforcing (TBD how and where)
    • usage reporting: per app, overall
  • download
  • reset
    • "storage"
    • "full"
    • "resources"
  • getStorageDetails
    • single installed application Request payload: {"type":"application/vnd.rdk-app.dac.native", "id":"<string>"}
  •  getList
    • filtering (already done)
  • setMetada (already done)
  • cancel (already done)
  • Configure quotas in AS/FSM
  • Review logical storages requirements
    • separately for all platforms
      
      • Phase 2: all supported platform
• review for all platforms: gaps
• Phase 3/later: 
  • TBD if needed: installation with authentication
    • at least API_KEY support
  • TBD: LGI authentication module
    • accessing CDNSharedSecret (check cat /mnt/secure_storage/cwmp/cwmp.xml | grep CDNSharedSecret)
  • TBD: apps.db - authentication, antirollback, encryption
  • TBD: concurrent parallel downloads

...