Page History

Introduction

This page dedicated to understanding of High level design for RDK Media Streaming in R-Pi Zero.

• Supported WiFi connection.
• v4l2 Driver is used to capture data from RPI-0 camera Device.
• /dev/video0 is the RPI-0 camera device to capture data.
• Supported Soc level Gstreamer plugins to capture data from camera device.
• Supported H264 encoding format.
• Supported SD,HD,FUll HD resolution.
• Supported automatic boot-up process for RMS functionality.

Architecture

Image Removed

Design Considerations

• Gstreamer Soc Implementation for RPI-0 Camera
  

 Image Removed

• Enabled V4l2 driver in part of RPI-0 to capture data from /dev/video0  device.
• Implemented soc level gstreamer pipeline with v4l2src plugin,omxh264enc plugin and appsink plugin.
• V4l2src plugin is used to capture raw data from camera through v4l2 driver and transmitted captured raw data into omxh264enc plugin to encode raw data into h264 encoding format.After that transmitted encoded data into appsink plugin to wrote encoded data into 8080 port of mongoose server.
• Registered 8080 port in Mongoose server to listen data.
• Supported resolution SD( 640*480 , 720*576 ) ,HD( 1280*720 ) and Full HD( 1920*1080 )

• Gstreamer Implementation in RMS
  

Image Removed

Why libcamera in PipeWire?

• Standardized interface: Libcamera provides a unified and standardized interface for interacting with different types of camera devices. By integrating libcamera into pipewire, camera applications can use a single API to access and control camera hardware.
• Flexibility and modularity: Pipewire is designed to be a modular multimedia framework. 
• Improved performance: Libcamera is designed to provide better performance and lower latency than previous camera APIs for Linux-based systems.
• Cross-platform compatibility: Both libcamera and pipewire are open-source projects that can run on a variety of Linux-based platforms.
• Integrating libcamera into pipewire helps to simplify and streamline the process of working with camera devices on Linux-based systems.

Architecture PipeWire-libcamera

Image Added                         Image Added

Integration of RDKC with PipeWire-libcamera

Image Added

Below list of changes set for PipeWire-libcamera

Libcamera Recipes:
https://code.rdkcentral.com/r/plugins/gitiles/rdk/components/generic/rdk-oe/meta-cmf-raspberrypi/+/refs/heads/rdk-next/recipes-multimedia/libcamera/
https://code.rdkcentral.com/r/plugins/gitiles/rdk/components/opensource/oe/meta-openembedded/+/refs/heads/dunfell/meta-multimedia/recipes-multimedia/libcamera/

Enabling Libcamera SPA Plugin Pipewire:
https://code.rdkcentral.com/r/c/components/generic/rdk-oe/meta-cmf-camera/+/81195
https://code.rdkcentral.com/r/c/rdk/components/generic/rdk-oe/meta-cmf-raspberrypi/+/81171

PipeWire Overview and Flow in RDK-C

Image Added

• It can create audio and video server
• Apps and devices are represented by node
• Each node has the input and output port to share media between node
• The session manager is to configure nodes, ports and linking nodes
• SPA: It will link to the driver to control the driver parameter and send and receive media.
• SPA plugins: libcamera, ALSA, v4l2, JACK and etc...
• WebRTC Live Streaming- Can View from Web Application
• RMS live streaming with RTSP protocol – Can View live streaming in VLC
• CVR with KVS SDK – Can view/record data in AWS KVS service
• Session manager will control these app nodes and ports and it will link nodes along with ports for communication
• Used libcamera SPA to fetch data from the v4l2 driver

Overview of PipeWire-Libcamera Internals

     Image Added

SPA Plugin Creation (pipewire/spa/plugins)

1. Implemented each SPA plugins with a group of factories.
2. Each factory should have the below details
       * Version ( SPA_VERSION_HANDLE_FACTORY )
       * Factory Name ( SPA_NAME_API_* - eg.SPA_NAME_API_LIBCAMERA_ENUM_UDEV)
       * SPA dictionary information ( struct spa_dict )
       * Get size callback  - impl_get_size()
           - Each factory has its own impl structure for to maintain some specific details.
           - With this callback, we can get the size of this factory impl structure
       * Initialization callback – impl_init()
           - This will initialize this factory-specific activity
       * Enum interface info callback - impl_enum_interface_info()
3. Each SPA plugin's factories are associated with the spa_handle_factory_enum() function.
4. Libcamera SPA plugin’s factories(libcamera.c)

Libcamera API sequence

Image Added

PipeWire API sequence in RDK-C:

Image Added

Supported Usecases using Pipewire-Libcamera:

1. WebRTC Streaming
2. RTSP Streaming
3. Continuous Video Recording

Image Added

Libcamera Roadmap

Features planned down the lane: 

• Configuration and control algorithms using libcamera in Raspberry Pi Platform with advanced camera module.
  • 3A algorithm
  • AEC/AGC control                                                                                                                   
  • AWB control
  • AF control

Image Added                               Image Added

                                                                                                                                                                                Sony IMX477

                                                                                                                                                                               Zooming Camera.

...