Why libcamera in PipeWire?
- Standardized interface: Libcamera provides a unified and standardized interface for interacting with different types of camera devices. By integrating libcamera into pipewire, camera applications can use a single API to access and control camera hardware.
- Flexibility and modularity: Pipewire is designed to be a modular multimedia framework.
- Improved performance: Libcamera is designed to provide better performance and lower latency than previous camera APIs for Linux-based systems.
- Cross-platform compatibility: Both libcamera and pipewire are open-source projects that can run on a variety of Linux-based platforms.
- Integrating libcamera into pipewire helps to simplify and streamline the process of working with camera devices on Linux-based systems.
Architecture PipeWire-libcamera
![](/download/attachments/254404366/image2023-4-13_14-49-48.png?version=1&modificationDate=1681377589000&api=v2)
Integration of RDKC with PipeWire-libcamera
![](/download/attachments/254404366/image2023-4-13_23-7-3.png?version=1&modificationDate=1681407425000&api=v2)
Below list of changes set for PipeWire-libcamera
Libcamera Recipes:
https://code.rdkcentral.com/r/plugins/gitiles/rdk/components/generic/rdk-oe/meta-cmf-raspberrypi/+/refs/heads/rdk-next/recipes-multimedia/libcamera/
https://code.rdkcentral.com/r/plugins/gitiles/rdk/components/opensource/oe/meta-openembedded/+/refs/heads/dunfell/meta-multimedia/recipes-multimedia/libcamera/
Enabling Libcamera SPA Plugin Pipewire:
https://code.rdkcentral.com/r/c/components/generic/rdk-oe/meta-cmf-camera/+/81195
https://code.rdkcentral.com/r/c/rdk/components/generic/rdk-oe/meta-cmf-raspberrypi/+/81171
PipeWire Overview and Flow in RDK-C
![](/download/attachments/254404366/image2023-4-13_23-9-15.png?version=1&modificationDate=1681407556000&api=v2)
- It can create audio and video server
- Apps and devices are represented by node
- Each node has the input and output port to share media between node
- The session manager is to configure nodes, ports and linking nodes
- SPA: It will link to the driver to control the driver parameter and send and receive media.
- SPA plugins: libcamera, ALSA, v4l2, JACK and etc...
- WebRTC Live Streaming- Can View from Web Application
- RMS live streaming with RTSP protocol – Can View live streaming in VLC
- CVR with KVS SDK – Can view/record data in AWS KVS service
- Session manager will control these app nodes and ports and it will link nodes along with ports for communication
- Used libcamera SPA to fetch data from the v4l2 driver
Overview of PipeWire-Libcamera Internals
![](/download/attachments/254404366/image2023-4-13_15-5-11.png?version=1&modificationDate=1681378512000&api=v2)
SPA Plugin Creation (pipewire/spa/plugins)
- Implemented each SPA plugins with a group of factories.
- Each factory should have the below details
* Version ( SPA_VERSION_HANDLE_FACTORY )
* Factory Name ( SPA_NAME_API_* - eg.SPA_NAME_API_LIBCAMERA_ENUM_UDEV)
* SPA dictionary information ( struct spa_dict )
* Get size callback - impl_get_size()
- Each factory has its own impl structure for to maintain some specific details.
- With this callback, we can get the size of this factory impl structure
* Initialization callback – impl_init()
- This will initialize this factory-specific activity
* Enum interface info callback - impl_enum_interface_info() - Each SPA plugin's factories are associated with the spa_handle_factory_enum() function.
- Libcamera SPA plugin’s factories(libcamera.c)
Libcamera API sequence
![](/download/attachments/254404366/image2023-4-13_15-10-54.png?version=1&modificationDate=1681378855000&api=v2)
PipeWire API sequence in RDK-C:
![](/download/attachments/254404366/image2023-4-13_15-12-26.png?version=1&modificationDate=1681378947000&api=v2)
Supported Usecases using Pipewire-Libcamera:
- WebRTC Streaming
- RTSP Streaming
- Continuous Video Recording
![](/download/attachments/254404366/image2023-4-14_15-51-42.png?version=1&modificationDate=1681467703000&api=v2)
Libcamera Roadmap
Features planned down the lane:
- Configuration and control algorithms using libcamera in Raspberry Pi Platform with advanced camera module.
- 3A algorithm
- AEC/AGC control
- AWB control
- AF control
![](/download/thumbnails/254404366/image2023-4-14_15-4-11.png?version=1&modificationDate=1681464853000&api=v2)
Sony IMX477
Zooming Camera.