What is Harvester ?

  • Harvester is a CCSP component, which runs on an RDKB gateway to collect data from the Wi-Fi driver in three areas: Associated Devices, Radio Traffic, and Neighboring Wireless Networks.

Repository 

Architecture

  • The Harvester Agent is a  CCSP component to handle configuration parameters to control data extraction from the device.
  • The harvester operates by polling the WiFi HAL (Hardware Access Layer) for information for each of the three reports and may store multiple polls for bulk transmission to WebPA and subsequent components.

TR-181

TR-181 Object is X_RDKCENTRAL-COM_Report. A Device Report is an Encoded Avro Schema Payload

  1. InterfaceDevicesWifi

    • The InterfaceDevicesWifi Report sends WiFi Metrics Data for the devices connected to the Gateway harvested from the WiFi HAL.
    • This report is generated by the Harvester Component

    • It supports various parameters to control the Harvesting Periodicity, Schema Identification and Default Values for the various intervals.

      TR-181

      Type

      Default value

      Range

      Description

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.Enabled

      Boolean

      False

       

      • Enables or disables InterfaceDevicesWifi data collection.
      • A value of true means that the harvester will collect data about devices connected to all the SSID's which are Enabled on the system and send them to the Cloud

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.PollingPeriod

      unsigned integer

      900

      1, 5, 10, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400

      • Stores the InterfaceDevicesWifi polling time period in seconds 
      • An unsigned integer in seconds  denotes the time the harvester will wait between querying associated device data on all SSID's which are enabled in the System. Must be less than or equal to Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.ReportingPeriod

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.ReportingPeriod

      unsigned integer

      900

      1, 5, 10, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400
      Must be greater than or equal to the PollingPeriod

      • Stores the InterfaceDevicesWifi reporting time period in seconds 
      • An unsigned integer in seconds  denotes the time the harvester will wait between uploading associated device data on all SSID's which are enabled in the System to the cloud. Must be greater than or equal to Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.PollingPeriod

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.Schema

      string

       

       

      • Stores the Schema used to encode the Associated Devices Scan Data
      • A String consisting of the Avro Schema used to Encode the InterfaceDevicesWifi data from the HAL

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.SchemaID

      string

       

       

      • Stores the UUID/HASH of the Schema used to encode the Scan data
      • UUID/HASH as a string of the Schema used to Encode the data from HAL

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.Default.PollingPeriod

      unsigned integer

      900

      1, 5, 10, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400

      • Stores the InterfaceDevicesWifi default polling time period in seconds 
      • An unsigned integer in seconds denotes the default time the harvester will wait between querying associated device data on all SSID's which are enabled in the System.

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.Default.ReportingPeriod

      unsigned integer

      900

      1, 5, 10, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400
      Must be greater than Polling Period

      • Stores the InterfaceDevicesWifi default reporting time period in seconds 
      • An unsigned interger in seconds denotes the default time the harvester will wait between uploading associated device data on all SSID's which are enabled in the System to the cloud.

      Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.Default.OverrideTTL

      unsigned integer

      300

       

      • Stores the time which an accelerated Polling or Reporting period will remain active
      • An unsigned integer in seconds denotes the time that a request to accelerate a polling interval or reporting interval will remain active. Defaults to 300 seconds if a value is not specified

  2. RadioInterfaceStatistics

    • The RadioInterfacesStatistics Report sends WiFi Radio Data.
    • This report is generated by the Harvester Component

    • It supports various parameters to control the Harvesting Periodicity, Schema Identification and Default Values for the various periods

      TR-181

      Type

      Default value

      Range

      Description

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.Enabled

      Boolean

      False


      • Enables or disables RadioInterfaceStatistics data collection.
      • A value of true means that the harvester will collect data about devices connected to all the SSID's which are Enabled on the system and send them to the Cloud

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.PollingPeriod

      unsigned integer

      900

      1, 5, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400

      • Stores the RadioInterfaceStatistics polling time period in seconds 
      • An unsigned integer in seconds  denotes the time the harvester will wait between querying associated device data on all SSID's which are enabled in the System. Must be less than or equal to Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.ReportingPeriod

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.ReportingPeriod

      unsigned integer

      900

      1, 5, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400
      Must be greater than or equal to the PollingPeriod

      • Stores the RadioInterfaceStatistics reporting time period in seconds
      • An unsigned integer in seconds  denotes the time the harvester will wait between uploading associated device data on all SSID's which are enabled in the System to the cloud. Must be greater than or equal to Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.PollingPeriod 

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.Schema

      string



      • Stores the Schema used to encode the Associated Devices Scan Data
      • A String consisting of the Avro Schema used to Encode the RadioInterfaceStatistics data from the HAL

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.SchemaID

      string



      • Stores the UUID/HASH of the Schema used to encode the Scan data
      • UUID/HASH as a string of the Schema used to Encode the data from HAL

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.Default.PollingPeriod

      unsigned integer

      900

      1, 5, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400

      • Stores the RadioInterfaceStatistics default polling time period in seconds 
      • An unsigned integer in seconds denotes the default time the harvester will wait between querying radio traffic data on all SSID's which are enabled in the System.

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.Default.ReportingPeriod

      unsigned integer

      900

      1, 5, 15, 30, 60, 300, 900, 1800, 3600, 10800, 21600, 43200, 86400
      Must be greater than Polling Period

      • Stores the RadioInterfaceStatistics default reporting time period in seconds 
      • An unsigned interger in seconds denotes the default time the harvester will wait between uploading radio traffic data on all SSID's which are enabled in the System to the cloud.

      Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.Default.OverrideTTL

      unsigned integer

      300


      • Stores the time which an accelerated Polling or Reporting period will remain active
      • An unsigned integer in seconds denotes the time that a request to accelerate a polling interval or reporting interval will remain active. Defaults to 300 seconds if a value is not specified

  3. NeighboringAP

    • The NeighboringAP Report sends data about all the neighboring access points
    • This report is generated by the Harvester Component

    • It supports various parameters to control the Harvesting Periodicity, Schema Identification and Default Values for the various periods

      TR-181

      Type

      Default value

      Range

      Description

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.Enabled

      Boolean

      False


      • Enables or disables NeighboringAP data collection.
      • A value of true means that the harvester will collect data about devices connected to all the SSID's which are Enabled on the system and send them to the Cloud

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.PollingPeriod

      unsigned integer

      21600

      300, 900, 1800, 3600, 10800, 21600, 43200, 86400

      • Stores the NeighboringAP polling time period in seconds 
      • An unsigned integer in seconds  denotes the time the harvester will wait between querying associated device data on all SSID's which are enabled in the System. Must be less than or equal to Device.X_RDKCENTRAL-COM_Report.NeighboringAP.ReportingPeriod

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.ReportingPeriod

      unsigned integer

      43200

      300, 900, 1800, 3600, 10800, 21600, 43200, 86400
      Must be greater than or equal to the PollingPeriod

      • Stores the NeighboringAP reporting time period in seconds 
      • An unsigned integer in seconds  denotes the time the harvester will wait between uploading NeighboringAP data on all SSID's which are enabled in the System to the cloud. Must be greater than or equal to Device.X_RDKCENTRAL-COM_Report.NeighboringAP.PollingPeriod

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.Schema

      string



      • Stores the Schema used to encode the Associated Devices Scan Data
      • A String consisting of the Avro Schema used to Encode the NeighboringAP data from the HAL

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.SchemaID

      string



      • Stores the UUID/HASH of the Schema used to encode the Scan data
      • UUID/HASH as a string of the Schema used to Encode the data from HAL

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.Default.PollingPeriod

      unsigned integer

      21600

      300, 900, 1800, 3600, 10800, 21600, 43200, 86400

      • Stores the NeighboringAP default polling time period in seconds 
      • An unsigned integer in seconds denotes the default time the harvester will wait between querying associated device data on all SSID's which are enabled in the System.

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.Default.ReportingPeriod

      unsigned integer

      43200

      300, 900, 1800, 3600, 10800, 21600, 43200, 86400
      Must be greater than Polling Period

      • Stores the NeighboringAP default reporting time period in seconds 
      • An unsigned interger in seconds denotes the default time the harvester will wait between uploading NeighboringAP device data on all SSID's which are enabled in the System to the cloud.

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.Default.OverrideTTL

      unsigned integer

      43200


      • Stores the time which an accelerated Polling or Reporting period will remain active
      • An unsigned integer in seconds denotes the time that a request to accelerate a polling interval or reporting interval will remain active. Defaults to 300 seconds if a value is not specified

      Device.X_RDKCENTRAL-COM_Report.NeighboringAP.LastScanData

      string



      • Returns the last scan data from previous Neighbor AP Scan

      Device.X_RDKCENTRAL- COM_Report.NeighboringAP.OnDemandScan

      string



      • Initiates a Neighbor AP Scan and immediately returns the result

SAMPLE OUTPUT AND LOGS

  • In Rpi , by default harvester process will be not running.
  • To launch the process

Go to /usr/ccsp/harvester folder
/usr/bin/harvester –DEBUG –LOGFILE /tmp/logfile.txt &

  • Sample Log 
root@RaspberryPi-Gateway:/usr/ccsp/harvester# /usr/bin/harvester -DEBUG -LOGFILE /tmp/logfile_dmcli_enabled.txt
DEBUG ENABLE ON
RDK_LOG_DEBUG, Registering Harvester component 'com.cisco.spvtg.ccsp.harvester' with CR ..
eRT.com.cisco.spvtg.ccsp.Harvester start to check eRT.com.cisco.spvtg.ccsp.psm status
eRT.com.cisco.spvtg.ccsp.psm is ready, eRT.com.cisco.spvtg.ccsp.Harvester continue
PSM module done.
Conf file /etc/debug.ini open success
rdk_dyn_log_initg_dl_socket = 8 __progname = harvester
rdk_logger_init /etc/debug.ini Already Stack Level Logging processed... not processing again.
****LOADING DM LIBRARY***************
THE LIB NAME =libHarvesterSsp.so
****LOADING DM LIBRARY***************
PLUGIN libHarvesterSsp.so LOADED SUCCESSFULLY
Inside msgBusInit : /tmp/harvester_initialized created
RDK_LOG_DEBUG, Registered Harvester component 'com.cisco.spvtg.ccsp.harvester' with CR ..
  • TR-181 output from Rpi on enabling below harvester parameters 
    dmcli eRT setv Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.Enabled bool true
    dmcli eRT setv Device.X_RDKCENTRAL-COM_Report.RadioInterfaceStatistics.Enabled bool true
    dmcli eRT setv Device.X_RDKCENTRAL-COM_Report.NeighboringAP.Enabled bool true

  • Sample debug logs from Rpi logfile.txt

  • Harvester logs are available under /rdklogs/logs/Harvesterlog.txt.<0/1>
    Sample:

    220228-10:52:18.395433 [mod=Harvester, lvl=WARN] [tid=31156] Register empty object : com.cisco.spvtg.

    220228-10:52:18.395571 [mod=Harvester, lvl=WARN] [tid=31156] Register empty object : com.cisco.spvtg.ccsp.

    220228-10:52:18.396881 [mod=Harvester, lvl=WARN] [tid=31156] DM Library module name = CCSP Base Data Model

    220228-10:52:18.398988 [mod=Harvester, lvl=WARN] [tid=31156] DM Library module name = COSA_TR181_Harvester

    220228-10:52:18.400870 [mod=Harvester, lvl=WARN] [tid=31156] Loading DM library libHarvesterSsp.so...

    220228-10:52:18.448430 [mod=Harvester, lvl=WARN] [tid=31156] Search for empty object in 'dslhCpeCheckEmptyObject'

    220228-10:52:18.452440 [mod=Harvester, lvl=INFO] [tid=31156]  rdk-logger initialzed!

    220228-10:52:18.457414 [mod=Harvester, lvl=INFO] [tid=31156] RDK_LOG_WARN, HARV : /tmp/harvester_initialized created

    220228-10:52:18.457656 [mod=Harvester, lvl=WARN] [tid=31156]   HARV : /tmp/harvester_initialized created

    220228-10:52:18.463001 [mod=Harvester, lvl=INFO] [tid=31168]  Init for parodus Success..!!

    220228-10:52:19.195747 [mod=Harvester, lvl=INFO] [tid=31161]  Ethernet WAN is enabled

    220228-10:52:19.196703 [mod=Harvester, lvl=INFO] [tid=31161] deviceMAC is b827eb221636

    220228-10:52:19.216732 [mod=Harvester, lvl=INFO] [tid=31161]  Sent message successfully to parodus

    220228-10:53:25.934303 [mod=Harvester, lvl=WARN] [tid=31159] DslhObjcoValidate: Enabled, bsUpdate = 3

    220228-10:53:25.934586 [mod=Harvester, lvl=INFO] [tid=31159] RDKB_HARVESTER_VALUE_CHANGED - WiFiInterfaceDevice Report Enabled

    220228-10:53:25.934742 [mod=Harvester, lvl=WARN] [tid=31159] Before committing:Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.

    220228-10:53:25.935239 [mod=Harvester, lvl=WARN] [tid=31159]  Harvester InterfaceDeviceWifi Report STARTED SetIDWHarvestingStatus

    220228-10:53:25.944784 [mod=Harvester, lvl=WARN] [tid=31159] After committing:Device.X_RDKCENTRAL-COM_Report.InterfaceDevicesWifi.:0

    220228-10:53:26.184683 [mod=Harvester, lvl=INFO] [tid=680]  Harvester GetWiFiApGetAssocDevicesData : SSID RPi3_RDKB-AP1 is NOT ENABLED  or ERROR retured 0

    220228-10:54:10.478550 [mod=Harvester, lvl=WARN] [tid=31159] DslhObjcoValidate: Enabled, bsUpdate = 3

    220228-10:54:10.478789 [mod=Harvester, lvl=INFO] [tid=31159] RDKB_HARVESTER_VALUE_CHANGED - WiFiNeighbourAP Report Enabled

    220228-10:54:10.478947 [mod=Harvester, lvl=WARN] [tid=31159] Before committing:Device.X_RDKCENTRAL-COM_Report.NeighboringAP.

    220228-10:54:10.484989 [mod=Harvester, lvl=WARN] [tid=31159] After committing:Device.X_RDKCENTRAL-COM_Report.NeighboringAP.:0

    220228-10:54:10.638449 [mod=Harvester, lvl=ERROR] [tid=1958]  Harvester GetRadioNeighboringAPData : Radio wifi1 is NOT ENABLED  or ERROR retured 0

    220228-10:54:10.681146 [mod=Harvester, lvl=INFO] [tid=1958]  Sent message successfully to parodus