In the era of the COVID19 pandemic, social distancing is the new normal.. To prevent the spread of the virus, we are avoiding crowds and crowded spaces – people are working from home instead of from offices, schools are closed and learning has moved online, people are interacting with loved ones using electronic devices instead of in-person visits, and so on.
But, we all hope that the world will soon go back to normal and that it would be safe for us to leave our homes.
We have an opportunity to focus on how things can go back to normal. Businesses, workplaces, stores, etc are determined to avoid spreading the novel coronavirus on their premises and want to monitor social distancing guidelines. But, how does one do it effectively and unobtrusively? How can artificial intelligence be helpful to enforce social distancing?
AI can be used with existing security cameras infrastructure to track if people density in offices hallways, offices corridors, stores, shops follow social distancing guidelines. If any of the rules are broken, then alerts can be sent to admins to take corrective actions immediately.
The discussed end-to-end solution to monitor social distancing can be build using IP cameras, IoT gateway, and Microsoft Azure cloud platform.
IoT Gateway based on NVIDIA Jetson Platform
The architecture of this platform consists of the following components:
- IP Cameras: Network IP cameras to send video streams over the network. These IP cameras can securely register with the cloud and can send/receive data to/from the cloud via IoT Gateway.
- IoT Gateway: Based on the NVIDIA Jetson platform, the IoT gateway provides IP camera connectivity and edge analytics to IP cameras that otherwise might not have those capabilities. Gateway provides three key components:
- Module – Deep Learning using PeopleNet: Developed by NVIDIA, PeopleNet is a highly accurate purpose-built 3 class object detection network to detect people in an image. The model detects one or more physical objects from three categories within an image and returns a box around each object, as well as a category label for each object. Three categories of objects detected by this model are – people, bags, and faces. The model is based on the NVIDIA DetectNet_v2 detector with ResNet34 as a feature extractor.
- The primary use case intended for this model is detecting people in a color (RGB) image. The model can be used to detect people from photos and videos by using appropriate video or image decoding and pre-processing. As a secondary use case, the model can also be used to detect bags and faces from images or videos. However, these additional classes are not the main intended use for this model.
The number of people in a given area can be inferred from the model and this metadata can be pushed out to the cloud.
- Azure IoT Edge Runtime– The runtime performs management and communication operations such as:
- Maintains Azure IoT Edge security standards on the gateway.
- Reports modules health to the cloud for remote monitoring.
- Manages communication between IP cameras and between the gateway and the cloud.
- Azure IoT Edge Cloud Interface– Azure cloud-based interface enables remote monitoring and management of IoT edge devices. IoT edge cloud interface provides a control panel for the deployment and management of modules in Azure IoT edge device.
- Azure IoT Hub: The IoT Hub offers built-in secure connectivity, telemetry and event ingestion, and bi-directional communication with Gateway, IP camera including device management with command and control capabilities. In addition, the IoT Hub offers an entity store that can be used to store device metadata.
- Stream Processing: Stream processing will process streams of data records coming in and evaluates rules for those streams. For stream, it is recommended that Azure Stream Analytics is used for IoT applications that require complex rule processing at scale. For simple rules processing, Azure IoT Hub Routes can be used with Azure Functions.
- Storage: Storage can be decided based on the instant use of data or later use.
- Warm path (data that is required to be available for reporting and visualization immediately from devices). Azure Cosmos DB to be used for warm path storage.
- Cold path (data that is stored long term and used for batch processing). Azure Blob Storage can be used for cold storage. For applications with time-series specific reporting, it’s better to use Azure Time Series Insights.
- UI & Reporting Tools: The user interface can be delivered on a wide array of device types, in native applications, and browsers. Various rules and actions can be defined. A user can define rules such as a crowd exceeding a set threshold. When a rule condition is met, it triggers an action defined by the builder. For example – an action to send an email or SMS alert to admin when given space is crowded.
In this solution, NVIDIA GPUs has been proposed to be used in four ways.
- Algorithm training with GPU (all cloud based)
- Inferencing (PeopleNet @ Edge)
- Video transcoding of H.264 streams
- Edge detection
Further Solutions Extension: The discussed solution can be extended to various use-case like:
- Face Mask Detection
- Temperature Detection using Thermal IP camera
References
