To manage the increasing amount of data that is generated by the devices, sensors, most of the business logic is now applied at the edge instead of the cloud to achieve low-latency and have faster response time. However, there are some challenges that need to be addressed while developing AI-enabled edge IoT devices, let’s have a look at them.
Edge is delivering three essential capabilities – local data processing, filtered data transfer to the cloud and faster decision-making. In order to deal with increasing amount of data generated by sensors, most of the business logic is now deployed at the edge instead of cloud to ensure low-latency and faster response time. Only a subset of the data generated by sensors is sent to the cloud after aggregating and filtering the data at the edge. This approach significantly saves the bandwidth and cloud storage costs for enterprises. AI has enabled new capabilities for edge computing. Since most of the decision-making is now taking advantage of artificial intelligence, the edge is becoming the perfect destination for deploying machine-learning models trained in the cloud.
AI enabled IoT solutions have deployed for different verticals including consumer electronics, medical devices, surveillance, industrial and retail automation. For example, connected home appliances are being developed with features such as voice command activation and intelligent sensor cooking. At the same time, medical devices companies are coming up intelligent monitoring solutions that monitor health vitals and combined with historical data can also generate alert of possible health deterioration in the near future. Although exciting, there are several challenges that need to be addressed while developing AI enabled edge IoT devices.
AI implementation on edge heavily depends on specialized processors that complement the CPU. Any standard CPU may not be able to improve the speed of training an AI model. To bridge the gap between the data center and edge, chip manufacturers are building a niche, purpose-built accelerator that significantly speed up model inferencing. These modern processors assist the CPU of the edge devices by taking over the complex mathematical calculations needed for running deep learning models. While these chips are not comparable to their counterparts – GPUs – running in the cloud, they do accelerate the inferencing process. This result in faster prediction, detection and classification of data ingested to the edge layer.
In 2019, chip manufacturers such as Qualcomm, NVIDIA and ARM have launched specialized chips that speed up the execution of AI-enabled applications. Qualcomm has launched IoT and AI-optimized SoC platforms, geared towards vision applications. The QCS603 and QCS605 are targeted for edge devices like security cameras, sports cameras, wearable cameras, virtual reality cameras, robotics and smart displays. Vision AI Developer Kit, manufactured by Altek and supported by eInfochips, works in conjunction with Azure IoT Edge and Microsoft Azure Machine Learning to allow makers to develop their own AI models and build their own vision AI devices. The Vision AI Development Kit is built around the Qualcomm Vision Intelligence 300 Platform and includes camera processing software, hardware-accelerated inferencing of AI models, and SDKs for machine learning and computer vision. Developers can use the kit to prototype products in applications like industrial safety, manufacturing, logistics, retail, and home and enterprise security.
Accommodating real-world problems to a machine-learning algorithm is a challenging task. Different problems will lend themselves to different algorithms. Machine learning teams are still struggling to take advantage of ML due to challenges with inflexible frameworks, lack of reproducibility, collaboration issues, and immature software tools. In order to overcome these challenges, developers are leaning towards open source AI/ML technologies that are more standardized. Open source platforms like TensorFlow, Theano, Caffe has comprehensive, flexible ecosystem of tools, libraries and community resources that lets developers easily build and deploy ML powered applications.
In addition to the ML algorithm, an important development is the emergence of machine-learning inference servers (aka inference engines and inference servers). The machine learning inference server executes the model algorithm and returns the inference output. Machine learning and deep learning models are exploiting the power of Graphics Processing Units (GPU) in the cloud to speed up training. Mainstream cloud providers such as AWS, and Microsoft Azure are offering GPU as a service. With the Vision AI Development Kit, one can easily combine the Azure Machine Learning service from Microsoft and the edge computing power of the Vision Intelligence Platform from Qualcomm Technologies. The kit contains the Microsoft Azure IoT Edge runtime and the Qualcomm Neural Processing SDK for AI which makes it easy to take models trained in the cloud and run hardware-accelerated inference at the intelligent edge. The kit runs models built using Microsoft Azure Machine Learning (AML). It also runs other Azure services like Azure Stream Analytics, Azure Functions, Azure Cognitive Services and Azure SQL Server for edge analytics and AI processing.
IoT solutions comes with lots of benefits and risks. Niche technology companies are adopting and promoting Internet of Things as a means to make your daily lives better and easier through billions of ‘smart’ IoT devices but at the same time, IT Security Professionals are considering it unnecessary and too risky due to user privacy and data security issues of the IoT Devices. According to the results of recent Altman Vilandrie & Company Survey of 397 IT executives across 19 industries, almost half of all U.S. companies that use IoT devices have been hit by a security breach in 2017. Secure and encrypted data handling is an extremely sensitive factor that needs to be accounted while deploying IoT solutions. There are out-of-the-box Security Information and Event Management (SIEM) like Splunk’s analytics-driven security operation suite that are deployed for various Industrial IoT solutions to tackle real-time security monitoring, advanced threat detection, forensics and incident management.
There are also industry specific compliance that need to be adhered while development. In healthcare organizations, where laws regulate protected health information (PHI), it is critical to understand implications of cloud adoption for privacy, security, and regulatory compliance. To eliminate this problem, Microsoft has recently launched Azure healthcare AI blueprint. The Azure Security and Compliance Blueprint – HIPAA/HITRUST Health Data and AI enables deployment of platform-as-a-solution to demonstrate how to securely capture, transmit, store, and analyze health data as per industry compliance requirements.
If you would like to know more about the Vision AI Development Kit we encourage you to view the webinar “IoT in Action – the AI Camera” hosted by Microsoft and eInfochips, an Arrow company. At the same time, if you are looking for technical support for deploying AI models leveraging Azure Machine Learning for your next-gen intelligent IoT solution, please feel free to contact us.
