Launch system-config-services from a console and enable the zookeeper service from the services list.

Process flow diagram

draw.io Diagram

diagramName

Gliffy Diagram

border	true
chrome	min
name	xmidt-basic-cluster.drawio
pagePinrevision	81

WebPA Server components setup

...

If pre-built RPM packages are already installed as explained in previous section & we want to use the same, Skip to configuration section

Dependencies of Build system

...

Required for compiling server components written in go language. (Required version >=1.11

Code Block
Preferred Method: $ sudo yum install golang

install glide

Glide is a package manager for Go that is conceptually similar to package managers for other languages. Glide provides the following functionality:

- Records dependency information in a glide.yaml file. This includes a name, version or version range, version control information for private repo or when the type cannot be detected, and more.
- Tracks the specific revision each package is locked to in a glide.lock file. This enables recursively fetching the dependency tree.
- Utilizes vendor/ directories, known as the Vendor Experiment, so that different projects can have differing versions of the same dependencies.

Code Block

$ wget -c https://github.com/Masterminds/glide/releases/download/v0.13.1/glide-v0.13.1-linux-amd64.tar.gz
$ tar -xzf glide-v0.13.1-linux-amd64.tar.gz -C /opt
$ echo "export PATH=$PATH:/opt/linux-amd64/" >> $HOME/.bash_profile

Downloading the source code



[OR]
For Manual Installation:
$ wget https://dl.google.com/go/go1.11.linux-amd64.tar.gz
$ tar xzf go1.11.linux-amd64.tar.gz
$ sudo mv go /usr/local
** Add Below Lines to the profile file (.bash_profile etc.)
export GOROOT=/usr/local/go
export GOPATH=$HOME/go
export PATH=$GOPATH/bin:$GOROOT/bin:$PATH

Verify the version with below:
$ go version
go version go1.11 linux/amd64

install glide

Glide is a package manager for Go that is conceptually similar to package managers for other languages. Glide provides the following functionality:

- Records dependency information in a glide.yaml file. This includes a name, version or version range, version control information for private repo or when the type cannot be detected, and more.
- Tracks the specific revision each package is locked to in a glide.lock file. This enables recursively fetching the dependency tree.
- Utilizes vendor/ directories, known as the Vendor Experiment, so that different projects can have differing versions of the same dependencies.

Code Block

$ wget -c

Code Block

1. create a directory in $HOME say webpa_modules
$ mkdir $HOME/webpa_modules && cd $HOME/webpa_modules

2. Checkout the components from GitHub repository.
$ git clone https://github.com/Comcast/talaria.git
$ git clone https://github.com/Comcast/scytale.git
$ git clone https://github.com/Comcast/tr1d1um.git

...

Masterminds/glide/releases/download/v0.13.1/glide-v0.13.1-linux-amd64.tar.gz
$ tar -xzf glide-v0.13.1-linux-amd64.tar.gz -C /opt
$ echo "export PATH=$PATH:/opt/linux-amd64/" >> $HOME/.bash_profile

Downloading the source code

Code Block

1. Setcreate thea GOPATHdirectory &in change$HOME to the source directory, e.g.
	$ export GOPATH=$HOMEsay webpa_modules
$ mkdir $HOME/webpa_modules/petasos/
	$ && cd $HOME/webpa_modules/petasos/src/petasos

2. ResolveCheckout the packagecomponents dependenciesfrom usingGitHub gliderepository.
	$ glidegit clone https://github.com/Comcast/talaria.git
$ git clone https://github.com/Comcast/scytale.git
$ git clone https://github.com/Comcast/tr1d1um.git

Building the components

Code Block

1. cd $HOME/webpa_modules/<component-name>
  e.g. cd webpa_modules/talaria
3. Build the component from source
	$ make build

4. Create the package
	$ install --strip-vendor

3. Build the component from source
	$ go build petasos

4. Create the package
	$ mkdir $HOME/rpmbuild
	$ ./build_rpm.sh --no-sign

5. Install the locally built webPA component package
e.g. $ cd /root/rpmbuild/RPMS/x86_64/
     $ rpm -Uvh petasos-0.1.1-87.el6.x86_64.rpm

...

We can use either of the below 2 methods to generate a basic authorization string.

Anchor

	generate-auth-token
	generate-auth-token

Code Block

1.Note: UsingFor opensslnewer commandreleases to generate the base64basic encodedauth token.
[root@webpa-node1 ~]#  should be in username:password format.
1. Using openssl command to generate the base64 encoded token.
[root@webpa-node1 ~]# openssl enc -base64 <<< "user123:webpa@1234567890"
[OUTPUT] : d2VicGFAMTIzNDU2Nzg5MAodXNlcjEyMzp3ZWJwYUAxMjM0NTY3ODkwCg==
2. Using Linux coreutils tools to generate the base64 encoded token
[root@webpa-node1 ~]# echo "user123:webpa@1234567890"|base64
[OUTPUT] : d2VicGFAMTIzNDU2Nzg5MAodXNlcjEyMzp3ZWJwYUAxMjM0NTY3ODkwCg==

Talaria configuration

Edit the configuration file & modify port values if you need to run talaria service in a different port (default value is 8080).

Code Block

title	Sample configuration file [/etc/talaria/talaria.jsonyaml]

---
########################################
#   Labeling/Tracing via HTTP Headers Configuration
########################################

# The unique fully-qualified-domain-name of the server.  It is provided to
# the X-Talaria-Server header for showing what server fulfilled the request
# sent.
# (Optional)
  fqdn: <Fully Qualified Domain Name / IP>
  server: "xxx.xxx.xxx.xxx"
  env: test
  scheme: http

# Provides this build number to the X-Tr1d1um-Build header for
# showing machine version information.  The build number SHOULD
# match the scheme `version-build` but there is not a strict requirement.
# (Optional)
  build: "0.1.3-1"

# Provides the region information to the X-Tr1d1um-Region header
# for showing what region this machine is located in.  The region
# is arbitrary and optional.
# (Optional)
  region: "east"

# Provides the flavor information to the X-Tr1d1um-Flavor header
# for showing what flavor this machine is associated with.  The flavor
# is arbitrary and optional.
# (Optional)
  flavor: "mint"


  primary:
    address: ":8080"
  health:
    address: ":8180"
  pprof:
    address: ":8280"
  control:
    address: ":8203"
  metric:
    address: ":8380"
    metricsOptions:
      namespace: "xmidt"
      subsystem: "talaria"

########################################
#   Service Discovery Configuration
########################################

  # service defines the parameters needed to interact with the consul cluster
  # for service discovery.  Presently only consul is supported.  This is
  # presently only used by Prometheus to discover machines to monitor, but
  # in the not-too-distant future talaria will use this interaction to load
  # balance across all caduceus machines instead of using DNS.
  # (Optional)
  service:
    # consul configures the consul library in caduceus to use the local
    # service discovery agent
    consul:
      # client defines how to connect to the local consul agent (on the same
      # VM/container)
      client:
        # address is the address of the local consul agent
        address: "127.0.0.1:8500"
        # scheme is how the consul library should interact with the local
        # consul agent
        scheme: "http"
        # waitTime is TBD
        waitTime: "30s"

      # disableGenerateID is TBD
      disableGenerateID: true

      # registrations defines what services caduceus should register with
      # consul
      #
      #     id      - the VM/container instance name registered with consul
      #     name    - the name of service being registered
      #     tags    - a list of tags to associate with this registration
      #     address - the mechanism to reach the service (generally unique fqdn)
      #     port    - the port to reach the service at
      #     checks  - the list of checks to perform to determine if the service
      #               is available/healthy
      #         checkID                        - TBD
      #         ttl                            - how long the check is valid for
      #         deregisterCriticalServiceAfter - the duration to wait before the
      #                                          service is removed due to check
      #                                          failures
      registrations:
        -
          id: "example_talaria.xmidt.net"
          name: "talaria"
          tags:
            - "prod"
            - "mint"
            - "stage=prod"
            - "flavor=mint"
          address: "54.148.37.189"<WEBPA_SERVER_IP>
          port: 6001
          checks:
            -
              checkID: "example_talaria.xmidt.net:ttl"
              ttl: "30s"
              deregisterCriticalServiceAfter: "70s"

  log:
    file: "/var/log/talaria/talaria.log"
    level: "DEBUG"
    json: false

  device:
    manager:
      upgrader:
        handshakeTimeout: "10s"
      initialCapacity: 100000
      maxDevices: 100
      deviceMessageQueueSize: 100
      pingPeriod: "45s"
      idlePeriod: "135s"
      requestTimeout: "15s"
    outbound:
      method: "POST"
      eventEndpoints:
        default: http://127.0.0.1:6300/api/v3/notify
      requestTimeout: "125s"
      defaultScheme: "http"
      allowedSchemes:
        - "http"
        - "https"
      outboundQueueSize: 1000
      workerPoolSize: 100
      transport:
        maxIdleConns: 0
        maxIdleConnsPerHost: 100
        idleConnTimeout: "120s"
      clientTimeout: "160s"
      authKey: <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>
  inbound:
    authKey: <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>

  eventMap:
    default: http://127.0.0.1:6300/api/v3/notify

  service:
    defaultScheme: http
    fixed:
      - http://127.0.0.1:8080

...

"authHeader" : Auth token Use the auth token which was generated in previous section

"file" : Under "log" section, change the value from "stdout" to a file name if we need to redirect debug messages to a separate log file.

...

Code Block

title	Sample configuration [/etc/scytale/scytale.jsonyaml]

---

########################################
#   Labeling/Tracing via HTTP Headers Configuration
########################################

# The unique fully-qualified-domain-name of the server.  It is provided to
# the X-Scytale-Server header for showing what server fulfilled the request
# sent.
# (Optional)
server: "xxx.xxx.xxx.xxx"

# Provides this build number to the X-Tr1d1um-Build header for
# showing machine version information.  The build number SHOULD
# match the scheme `version-build` but there is not a strict requirement.
# (Optional)
build: "0.1.4-1"

# Provides the region information to the X-Tr1d1um-Region header
# for showing what region this machine is located in.  The region
# is arbitrary and optional.
# (Optional)
region: "east"

# Provides the flavor information to the X-Tr1d1um-Flavor header
# for showing what flavor this machine is associated with.  The flavor
# is arbitrary and optional.
# (Optional)
flavor: "mint"


##############################################################################
# WebPA Service configuration
##############################################################################

# For a complete view of the service config structure,
# checkout https://godoc.org/github.com/Comcast/webpa-common/server#WebPA

########################################
#   Primary Endpoint Configuration
########################################

# primary provides the configuration for the main server for this application
primary:
  address: ":7000"

########################################
#   Health Endpoint Configuration
########################################

# health defines the details needed for the health check endpoint.  The
# health check endpoint is generally used by services (like AWS Route53
# or consul) to determine if this particular machine is healthy or not.
health:
  address: ":7001"

########################################
#   Debugging/Pprof Configuration
########################################

# pprof defines the details needed for the pprof debug endpoint.
# (Optional)
pprof:
  address: ":7002"

########################################
#   Metrics Configuration
########################################

# metric defines the details needed for the prometheus metrics endpoint
# (Optional)
metric:
  address: ":7082"
  metricsOptions:
    # namespace is the namespace of the metrics provided
    # (Optional)
    namespace: "webpa"

    # subsystem is the subsystem of the metrics provided
    # (Optional)
    subsystem: "scytale"

fanout:
  fanoutTimeout: "125s"
  clientTimeout: "45s"
  endpoints:
      - "http://localhost:8080/api/v2/device"
  authorization: <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>

########################################
#   Logging Related Configuration
########################################

# log configures the logging subsystem details
log:
  # file is the name of the most recent log file.  If set to "stdout" this
  # will log to os.Stdout.
  # (Optional) defaults to os.TempDir()
  file: "/var/log/scytale/scytale.log"

  # level is the logging level to use - INFO, DEBUG, WARN, ERROR
  # (Optional) defaults to ERROR
  level: "DEBUG"

  # maxsize is the maximum log file size in MB
  # (Optional) defaults to max 100MB
  maxsize: 50

  # maxage is the maximum number of days to retain old log files
  # (Optional) defaults to ignore age limit (0)
  maxage: 30

  # maxbackups is the maximum number of old log files to retain
  # (Optional) defaults to retain all (0)
  maxbackups: 10

  # json is a flag indicating whether JSON logging output should be used.
  # (Optional) defaults to false
  json: true
aws:
  accessKey: "fake-accessKey"
  secretKey: "fake-secretKey"
  env: "fake-env"
  sns:
    awsEndpoint: http://goaws:4100
    region: "ap-east-1"
    topicArn: "arn:aws:sns:ap-east-1:999999999991:fake-env"
    urlPath: "/api/v2/aws/sns"
waitForDns: 0
authHeader: ["xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"]
start:
  duration: 1
  apiPath: http://127.0.0.1:6300/hooks
  authHeader: <xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx>

...

"authHeader" : Auth token Use the auth token which was generated in previous section.

"aws" : Add fake values as described previously.

Code Block

title	Sample configuration file [/etc/tr1d1um/tr1d1um.jsonyaml]

---

########################################
#   Labeling/Tracing via HTTP Headers Configuration
########################################

# The unique fully-qualified-domain-name of the server.  It is provided to
# the X-Tr1d1um-Server header for showing what server fulfilled the request
# sent.
# (Optional)
server: "xxx.xxx.xxx.xxx"

# Provides this build number to the X-Tr1d1um-Build header for
# showing machine version information.  The build number SHOULD
# match the scheme `version-build` but there is not a strict requirement.
# (Optional)
build: "0.1.2-1"

# Provides the region information to the X-Tr1d1um-Region header
# for showing what region this machine is located in.  The region
# is arbitrary and optional.
# (Optional)
region: "east"

# Provides the flavor information to the X-Tr1d1um-Flavor header
# for showing what flavor this machine is associated with.  The flavor
# is arbitrary and optional.
# (Optional)
flavor: "mint"


##############################################################################
# WebPA Service configuration
##############################################################################

# For a complete view of the service config structure,
# checkout https://godoc.org/github.com/Comcast/webpa-common/server#WebPA

########################################
#   Primary Endpoint Configuration
########################################

# primary provides the configuration for the main server for this application
primary:
  address: ":9003"

########################################
#   Health Endpoint Configuration
########################################

# health defines the details needed for the health check endpoint.  The
# health check endpoint is generally used by services (like AWS Route53
# or consul) to determine if this particular machine is healthy or not.
health:
  address: ":9004"

########################################
#   Debugging/Pprof Configuration
########################################

# pprof defines the details needed for the pprof debug endpoint.
# (Optional)
pprof:
  address: ":9005"

########################################
#   Metrics Configuration
########################################

# metric defines the details needed for the prometheus metrics endpoint
# (Optional)
metric:
  address: ":9082"
  metricsOptions:
    # namespace is the namespace of the metrics provided
    # (Optional)
    namespace: "webpa"

    # subsystem is the subsystem of the metrics provided
    # (Optional)
    subsystem: "tr1d1um"

########################################
#   Logging Related Configuration
########################################

# log configures the logging subsystem details
log:
  # file is the name of the most recent log file.  If set to "stdout" this
  # will log to os.Stdout.
  # (Optional) defaults to os.TempDir()
  file: "/var/log/tr1d1um/tr1d1um.log"

  # level is the logging level to use - INFO, DEBUG, WARN, ERROR
  # (Optional) defaults to ERROR
  level: "DEBUG"

  # maxsize is the maximum log file size in MB
  # (Optional) defaults to max 100MB
  maxsize: 50

  # maxage is the maximum number of days to retain old log files
  # (Optional) defaults to ignore age limit (0)
  maxage: 30

  # maxbackups is the maximum number of old log files to retain
  # (Optional) defaults to retain all (0)
  maxbackups: 10

  # json is a flag indicating whether JSON logging output should be used.
  # (Optional) defaults to false
  json: true


##############################################################################
# Webhooks Related configuration
##############################################################################

# webhooksEnabled indicates whether or not the webhooks server should be started
# It is disabled for local testing
webhooksEnabled: false

# The unique fully-qualified-domain-name of the server.  The webhooks library uses it
# to know which host to use to confirm this service is ready to receive events
# (Optional if not running webhooks)
fqdn: "tr1d1um-local-instance-123.example.com"

# start contains configuration for the logic by which Tr1d1um can
# fetch the current WebPA webhooks without having to wait for SNS
# It does so by pinging the rest of the cluter at the specified apiPath
# More detaisl at https://godoc.org/github.com/Comcast/webpa-common/webhook#StartConfig
start:
  # duration is the max amount of time allowed to wait for webhooks data to be retrieved
  duration: "20s"

  # path used to query the existing webhooks
  apiPath: http://localhost:6100/hooks

########################################
# Webhooks DNS readiness Configuration
########################################

# WaitForDns is the duration the webhooks library will wait for this server's DNS record to be
# propagated. This waiting logic is important so AWS SNS webhook confirmations are not missed
waitForDns: "30s"

#soa stands for Start of Authority and it's a type of record in a DNS
soa:
  # provider is the SOA provider used to verify DNS record readiness of this service
  provider: "example-123.awsdns-00.com:17"

########################################
# Webhooks AWS SNS Configuration
########################################

# aws provides the AWS SNS configurations the webhooks library needs
aws:
  #AWS access key
  accessKey: "fake-accessKey"

  #AWS secret key
  secretKey: "fake-secretKey"

  env: local-dev

  sns:
    # awsEndpoint is the AWS endpoint
    # this must be left out in produ
    awsEndpoint: http://goaws:4100

    #region is the AWS SNS region
    region: "us-east-1"

    # topicArn describes the SNS topic this server needs to subscribe to
    topicArn: arn:aws:sns:us-east-1:000000000000:xmidt-local-caduceus

    #urlPath is the URL path SNS will use to confirm a subscription with this server
    urlPath: "/api/v2/aws/sns"


##############################################################################
# Testing Authorization Credentials
##############################################################################

# authHeader is a list of Basic Auth credentials intended to be used for local testing purposes
# WARNING! Be sure to remove this from your production config
authHeader: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"


##############################################################################
# WRP and XMiDT Cloud configurations
##############################################################################

# targetURL is the base URL of the XMiDT cluster
targetURL: http://localhost:7000

# WRPSource is used as 'source' field for all outgoing WRP Messages
WRPSource: "dns:tr1d1um.xmidt.comcast.net"

# supportedServices is a list of endpoints we support for the WRP producing endpoints
# we will soon drop this configuration
supportedServices:
  - "config"


##############################################################################
# HTTP Transaction Configurations
##############################################################################

# clientTimeout is the timeout for the HTTP clients used to contact the XMiDT cloud
clientTimeout: "135s"

# respWaitTimeout is the max time Tr1d1um will wait for responses from the XMiDT cloud
respWaitTimeout: "129s"

# netDialerTimeout is the timeout used for the net dialer used within HTTP clients
netDialerTimeout: "5s"

# requestRetryInterval is the time between HTTP request retries against XMiDT
requestRetryInterval: "2s"

# requestMaxRetries is the max number of times an HTTP request is retried against XMiDT in
# case of ephemeral errors
requestMaxRetries: 2

...

start the service after changes are done.

Code Block
# rm –rf /tmp/parodusCmd.cmd # systemctl restart parodus

...

Code Block

language	java
title	GET Parameter

$ curl -H ''Authorization:Basic <AUTH_TOKEN>' -i http://<WEBPA-URL>/api/v2/device/mac:<DEVICE_MAC>/config?names=<PARAMETER>
e.g.
$ curl -H 'Authorization:Basic d2VicGFAMTIzNDU2Nzg5MA==' -i http://54.166.121.187:9003<WEBPA_SERVER_IP:PORT>/api/v2/device/mac:84e058575831/config?names=Device.DeviceInfo.ModelName

...

Code Block
$ curl -H "<AUTH_TOKEN>" http://<IP>:8080/api/v2/devices e.g. curl -H "Authorization: Basic d2VicGFAMTIzNDU2Nzg5MA==" http://54.166.121.187:<webpa_serverURL>:8080/api/v2/devices

Common TR181 parameters

Panel

Device.DeviceInfo.Manufacturer
Device.DeviceInfo.ManufacturerOUI
Device.DeviceInfo.ModelName
Device.DeviceInfo.SerialNumber
Device.DeviceInfo.HardwareVersion
Device.DeviceInfo.SoftwareVersion
Device.DeviceInfo.UpTime
Device.DeviceInfo.ProcessorNumberOfEntries
Device.DeviceInfo.MemoryStatus.Total
Device.DeviceInfo.MemoryStatus.Free
Device.DeviceInfo.ProcessStatus.CPUUsage
Device.DeviceInfo.ProcessStatus.ProcessNumberOfEntries

WebPA server for RDK community access

Below details can be used by RDK community members for using the reference WebPA server in configuration of any client device

Code Block
IP : 54.148.37.189 Authorization Key : “d3B1c2VyOndlYnBhQDEyMzQ1Njc4OTAK” Client device interface (talaria) Port: 8080 API interface (tr1d1um) port : 9003

RDK Resources

[RDK Preferred]

Code Management Facility

RDK Forums

[RDK Conferences]

RDK Support

Archives

In the News!

Versions Compared

Old Version 64

New Version Current

Key

Process flow diagram

WebPA Server components setup

Dependencies of Build system

install glide

Downloading the source code

install glide

Downloading the source code

Building the components

Talaria configuration

Common TR181 parameters

WebPA server for RDK community access

RDK Resources

[*RDK Preferred*]

Code Management Facility

RDK Forums

[RDK Conferences]

RDK Support

Archives

In the News!

Page History

Versions Compared

Old Version 64

New Version Current

Key

Process flow diagram

WebPA Server components setup

Dependencies of Build system

install glide

Downloading the source code

install glide

Downloading the source code

Building the components

Talaria configuration

Common TR181 parameters

WebPA server for RDK community access

[RDK Preferred]