According to a report by Straits Research, the global multi-camera system market size is projected to reach 8 billion dollars by 2030. Today, there are more than 1 billion deployed cameras that capture 30 billion images per second and generate more than 2500 petabytes of video data. Smart cameras with continuous sensing capabilities are everywhere today. These cameras are powered by highly compact chipsets with powerful image-processing capabilities coupled with Artificial Intelligence (AI) and Machine Learning (ML) capabilities.

There has been a significant increase in deployment of such cameras which has led to a huge surge in visual data generated in recent years. Close to 90% of this video data is not processed or interpreted. However, this scenario is changing with advancements in technologies.

Most camera systems used in a wide variety of applications consist of a common set of hardware and software components, in addition to specialized components unique to their area of application.

Typical hardware components of a camera system

• Lens: The lens of a camera performs the function of directing light from the surroundings onto the next component in the chain. It consists of several layers of glass.
• Image Sensor. It converts light into electrical signals which can be read and processed by the electronics embedded within the camera.
• Image Signal Processor: The image signal processor (ISP) consists of several electronic components, including one or more processors, memory buffers, supporting electronics and several layers of complex software. The ISP collects the signals from the image sensor and converts them to image data.These are referred to as raw frames or raw images, suitable for the next component in this chain.
• System on Chip (SoC): A typical SoC would consist of an application processor and may also include a GPU (Graphics Processing Unit), Machine Learning Accelerator modules, System Memory, IO interfaces, peripheral interfaces and more. Embedded OS and several layers of application software run to provide a wide variety of features expected from modern day networked cameras.
• Storage: The last block shows how the generated video data is consumed. It could be saved to some on-board non-volatile memory such as SD card or flash memory. It could also be streamed to other computing devices over the local network, such as a NAS (network attached storage) device, a live stream viewing station or a recording server.

Camera Software

The main components and function of software are:

• AV Processing: After the raw video data has been captured, several processing operations may be applied to it in real-time, for example, text overlay, privacy masks, rotation/flipping and more.
• There may be custom processing steps which may also be included in specific applications, such as barcode scanning.
• AI on the Edge: This feature has been made possible due to the powerful SOCs that are able to run several of these AI algorithms directly within the camera.

Features such as object detection, face/person/animal detection, object classification, segmentation are just a few of those. Qualcomm’s AI Hub is an amazing platform which hosts many AI algorithms which could be run within modern cameras.

• Audio-Video Encoding: Encoding the audio-video data results in significant reduction in storage and network bandwidth requirement. for an identical quality of output. For videos, algorithms such as H.264, H.265, MJPEG and AV1 and for audio, encoding algorithms such as AAC and G711 are available.
• Recording: It is the most important feature of any camera software, which saves the captured, processed, and encoded data into non-volatile storage devices such as flash memory or microSD cards.
• Streaming: RTSP (Real-Time Streaming Protocol) is one of the most well-known and established media-streaming protocols. Modern streaming protocols also include HLS (HTTP Live Streaming), MPEG4-DASH (Dynamic Adaptive Streaming over HTTP), WebRTC.
• Camera Configuration and Management: In addition to all software features which directly deal with the audio and video data, networked camera software also needs to perform other functions such as system settings camera device/name, time synchronization, logs, general housekeeping, software updates etc.
• Camera Configuration: Video-capture is one of the primary functions of a camera. There are several parameters which influence this capture.
  • Video resolution: FPS (Frames Per Second), bit rate, encoders, gains, ISO are some of the key parameters
  • Data access: Access such as viewing, downloading, changing settings, etc. need to be controlled, especially with networked cameras.
  • Supervisor component: With so many software components running in the system, a supervisor component is needed which ensures that all functions are performing as expected.
  • Network configuration: Configuration such as hostname, IP Addresses/ports, ID Info (certificates/keys etc.), network security etc. need to be available to the administrator of the camera.

To make all features work collectively and perform the required functions of the camera, every software component uses well-defined APIs, (application programming interfaces). ready-to-use APIs and frameworks help reduce the development efforts of camera software.

eInfochips’ Reusable Camera Framework (RCF) provides all the core software features and is a great foundation to build unique applications on top. RCF does not limit the designer to specific hardware platforms.

The majority of RCF source code is platform-agnostic and can be quickly ported to any new SOC with minimal effort. The minimal set of RCF components which are platform specific are the ones which interface with platform specific software components and even those components are extremely easy to customize. RCF is built on a very modular, micro-service architecture, which gives complete freedom to the designer to select the specific features as needed from the framework. All the existing services are highly customizable and can be tailored to any application-specific needs.

All RCF components expose a well-defined set of APIs which enable streamlined integration of additional custom services. These have the same level of access to all other services as the native RCF services .

To know more, please reach out to marketing@einfochips.com .