Build robust, reliable, and scalable IoT solutions! Part 1: Introduction to AWS IoT Suite of Services

Introduction

In this blog post, we will introduce you to the IoT PaaS Services offered by AWS. We will investigate their purpose and understand how to leverage these IoT PaaS components in building the IoT Platform.

Please note, eInfochips (an Arrow Company) is an AWS Advanced Partner. We at eInfochips have worked on multiple IoT projects, leveraging AWS IoT PaaS components to connect devices with AWS cloud. We have assisted customers in their digital/IoT journey by implementing AWS architectural blueprints and best practices. Our expertise includes Connected Medical Devices, Connected Vending Machines, and Connected Head Mount Devices. Consider decreasing your time-to-market by using AWS IoT solutions and accelerators built by eInfochips.

AWS IoT Services

AWS IoT provides a comprehensive set of tools and services that enable customers to build and manage secure, scalable, and intelligent IoT solutions, helping organizations improve operational efficiency, increase productivity, and create new business opportunities. If your devices can connect to AWS IoT, it can seamlessly and securely connect them to other AWS cloud services. AWS IoT acts as a bridge between your IoT devices and AWS services.

Now let’s look at some of the connectivity and control services offered by AWS. These services assist in securely connecting and managing your devices from the cloud. They enable your devices to transmit messages to the cloud securely with low latency and high throughput. Additionally, these services facilitate secure and instant message transmission between your devices and other field devices.

Now let’s look at some of the data analytics services offered by AWS. These services aid in analyzing transmitted IoT data faster to extract meaningful value. They offer real-time processing as well as batch processing, generally running on the cloud side. As discussed in the previous blog post ‘Functional Building Blocks of an IoT Platform,’ these services are also capable of running on the edge (device).

With this introduction to AWS IoT Services, let’s deep-dive into these core IoT Services and investigate further to understand their purpose and how to leverage them in building an end-to-end IoT Platform.

Establish Secure Communication with IoT Assets Using AWS IoT Core

As discussed in our previous blog post titled ‘Functional Building Blocks of an IoT Platform,’ you need a service or bridge solution to connect your field devices with cloud services. This solution should be device agnostic, supporting both constrained and resourceful devices. You require a solution that connects both IoT devices and IoT Gateways, as well as industrial and non-industrial assets. AWS IoT Core offers precisely that.

AWS IoT Core is a managed cloud service that allows IP devices and IP gateways to interact easily and securely with cloud applications over the internet and other locally deployed devices in the field. It serves as a cloud entry point for most IoT applications. Once your devices connect to AWS IoT Core, they can easily and securely connect with AWS cloud services. AWS IoT Core acts as a bridge between your IoT devices and AWS services.

Connect, manage, and scale your device fleets easily and reliably without provisioning or managing servers.

• AWS IoT Core can handle a vast number of devices and messages, supporting billions of devices and trillions of messages.
• It facilitates field devices in sending messages to the cloud securely.
• AWS IoT Core efficiently processes and routes messages to various AWS endpoints, such as databases (e.g., DynamoDB), data lakes (e.g., S3 buckets), queues (e.g., SQS), and analytical tools (e.g., Kinesis data streams).
• It ensures reliable and secure routing of messages to other field IoT devices.
• AWS IoT Core provides device status notifications, indicating whether devices are online or offline, which is a crucial feature.
• You can implement custom logic to handle offline devices for extended periods, enabling continuous communication with all field devices, even when they are not connected.

Supports variety of device communication protocols

• AWS IoT Core Device Gateway serves as the entry point for IoT devices on AWS.
• AWS IoT Core Device Gateway manages all active device connections.
• AWS IoT Core Device Gateway currently supports the MQTT, MQTT over WebSockets, and HTTPS protocols. Devices can send messages to cloud using any one of these protocols. In fact, bi-directional communication is possible. Cloud applications also can send the latest configurations / commands to device.

Secure device connections and data with mutual authentication and end-to-end encryption.

• AWS IoT Core provides mutual authentication, ensuring data is never exchanged between devices and AWS IoT Core without a proven identity. Additionally, AWS IoT Core enforces encryption at all points of connection. AWS IoT Core supports various authentication methods:

HTTPS	·         The AWS method of authentication (called SigV4) ·         X.509 certificate-based authentication ·         Customer created token-based authentication (through custom authorizers)
MQTT	·         X.509 certificate-based authentication
MQTT over WSS	·         The AWS method of authentication (called SigV4) ·         Customer created token-based authentication (through custom authorizers)

Out of the box support for constrained devices

• AWS IoT Core is device agnostic, supporting even constrained LWPAN devices (e.g., LoRa devices). This is an essential differentiating factor. You can onboard LoRa devices using AWS IoT Core for LoRaWAN without the need to develop or operate a LoRaWAN Network Server (LNS). This feature simplifies the solution and significantly reduces development efforts.

Filter, transform, and act upon device data on the fly, based on your defined business rules.

AWS IoT Core has a built-in rule engine, eliminating the need to manage any infrastructure. This engine empowers you to build IoT applications that gather, process, analyze, and act on data generated by IoT devices on a global scale.

The in-built rule engine supports a variety of rule formats, including

• Rules that can be applied to data from a single device or multiple devices.
• Rules that can trigger single or multiple actions in parallel based on conditions.
• Generally, these rules are in the ‘IF-This-Then-That’ format. For example, if the room temperature is above 31 degrees (condition), then notify the user through email (action), and store this record in AWS DynamoDB (another action).

Depending on the business rules you define, AWS IoT Core Rules Engine can route messages to various AWS endpoints, including AWS IoT Analytics, AWS IoT Events, AWS Lambda, Amazon Kinesis, Amazon S3, Amazon DynamoDB, Amazon CloudWatch, Amazon Simple Notification Service (SNS), Amazon Simple Queue Service (SQS), Amazon Elasticsearch Service, AWS Step Functions, Amazon Location Service, or even external endpoints.

Track and manage your fleet of IoT devices

• If your devices have GPS hardware, you can track your fleet’s location. However, what about other devices that do not have GPS hardware? One noteworthy feature with AWS IoT Core is that it helps you track your IoT devices (approximately) even if they lack GPS hardware. AWS IoT Core supports certain location estimation technologies/algorithms (e.g., Wi-Fi solver, IP reverse lookup solver, Cellular solver). You can choose one of these techniques to estimate device location approximately. This feature is very helpful in implementing and offering GeoFence and GeoLock services.

Onboard – Manage – Monitor – Control your fleet of IoT assets using AWS IoT Device Management

As discussed in our previous blog post titled ‘Functional Building Blocks of an IoT Platform,’ many IoT deployments consist of hundreds of thousands to millions of devices. Hence, it is essential to track, monitor, and manage connected device fleets. You need to ensure your IoT devices work properly and securely after they have been deployed. Additionally, monitoring their health is crucial. Timely issue detection, remote troubleshooting, and quick remedial actions are necessary. Physically visiting every device for firmware updates is impossible, and managing millions of devices one by one is highly impractical due to time constraints. You require a reliable and automated device life cycle management solution.

As an admin / field support team / IT admin, you may want a solution:

• that helps you onboard millions of devices with zero touch.
• that assists you in deleting the devices (single or in bulk) if needed.
• which helps you migrate your devices (single or in bulk) from one region to another region, one customer to another customer.
• which helps you take remote access to your devices securely to do troubleshooting.
• which helps you reboot a device on demand / hundreds of devices as per schedule.
• which manages software and firmware updates of these millions of devices (single or in bulk).
• which helps you change the configuration of millions of devices at a time.
• which shall recommend on security vulnerabilities of these devices.
• which shall appraise you on how many devices are still running on old software.
• which shall appraise you on your device’s security posture.
• a dashboard solution with all metrics related to millions of devices life cycle management.

This is where AWS IoT Device Management Service helps you.

Bulk register, organize groups, update over the air, and easily monitor all your IoT devices.

• With AWS IoT Device Management, you can register your devices individually or in bulk with AWS IoT Core / AWS Cloud and easily manage permissions to ensure devices remain secure.
• You can also organize your devices, monitor and troubleshoot device functionality, query the state of any IoT device in your fleet, and send firmware updates over-the-air (OTA).
• AWS IoT Device Management is agnostic to device type and OS, allowing you to manage any kind of device (e.g., constrained microcontroller-based devices to sophisticated connected cars).
• AWS IoT Device Management enables you to scale and diversify your fleets, reducing the cost and effort of managing large and diverse IoT device deployments.

Secure Tunneling

• AWS IoT Device Management supports the creation of a point-to-point tunnel, which allows for a secure remote SSH session to a device installed behind a restricted firewall. This provides secure connectivity to individual devices, which you or your field support team can use to diagnose and resolve issues remotely with just a few clicks. This feature helps avoid costly field visits.

Jobs

• AWS IoT Device Management “Jobs” feature runs and monitors software updates and other remote operations such as device reboots. It lets you manage updates to a single device or to your entire fleet. You or your IT admin can control the pace of deployment (for example, deploy to 10 devices per second) and receive real-time information about the status of your jobs as they’re deployed to your devices.

Fleet Hub

• With AWS IoT Device Management, you & your IT admin can visualize your fleet’s health status and remotely perform real-time actions, such as firmware updates and device reboots, using Fleet Hub.

Continuously audit & secure your fleet of IoT assets using AWS IoT Device Defender

As discussed in our previous blog post titled “Functional Building Blocks of an IoT Platform”, you need a security solution & security recommendation system – that helps you notify automatically if there are any deviations in the device configuration / if some one tries to modify device file system, if someone tries to login to the device, if device participates in DDoS attacks. You need a solution which helps you track user activities & malicious activities associated with physical remotely deployed devices. This is where AWS IoT Device Defender Service helps you.

AWS IoT Device Defender is a fully managed service that helps you secure your fleet of IoT devices. This solution consists of two parts. One is IOT Device Defender Agent which runs in device and second one is IoT Device Defender service running in the cloud.

AWS IoT Device Defender Cloud Software takes care of following

• Continuously audit the IoT configurations on your devices against a set of predefined security best practices.
• Makes it easy to maintain and enforce IoT configurations while onboarding devices. Examples could be ensuring device identity, authenticating, and authorizing devices, and encrypting device data.
• Sends an alert if there are any gaps in your IoT configuration that might create a security risk. For example
  • If device identity certificates are shared across multiple devices, then it will send immediate alert.
  • If a device tries to connect to AWS IoT core with a revoked identity certificate, then it will send immediate alert.
• AWS IoT Device Defender also lets you continuously monitor security metrics from devices and AWS IoT Core for deviations from what you have defined as appropriate behavior for each device. If something doesn’t look right, AWS IoT Device Defender sends out an alert so you can take action to remediate the issue. For example, traffic spikes in outbound traffic might indicate that a device is participating in a DDoS attack.
• AWS IoT Device Defender can send alerts to the AWS IoT Console, Amazon CloudWatch, and Amazon SNS. If you determine that you need to take an action based on an alert, you can use AWS IoT Device Management service to take mitigating actions such as pushing security fixes / delete the device from the network gracefully / sending the device to quarantine state / reboot the device.

AWS IoT Device Defender Device Agent takes care of following

• AWS IoT Device Defender Agent continuously monitors the device logins. If someone tries to login, and if login fails multiple times, then it will send immediate alert to cloud software which will notify you. Defender will record all successful logins for audit purposes.
• AWS IoT Device Defender Agent continuously monitors the file structure of the device. If someone tries to add / edit / delete the file by logging in to device, then it will send immediate alert.
• AWS IoT Device Defender Agent continuously monitors the ports of the device. If someone tries to open / close the ports, the defender will record all such events, and immediately notify cloud software, which in turn alerts you.
• AWS IoT Device Defender Agent records all such instances where someone tries to connect USB or any other peripherals to your remotely deployed device and notify you.
• Finally, with AWS IoT Device Defender Agent, you can track user & malicious activities in the field, and offer customized security recommendation system / automated solutions.

Detect & respond to events from IoT Sensors using AWS IoT Events

In our previous blog post titled “Functional Building Blocks of an IoT Platform”, we have thoroughly discussed the importance of Edge Rule Engine, Cloud Rule Engine. Rule engines are generally decision-making engines based on templates like “IF-This-Then-That” / “IF-Then-Else”. As discussed in previous blog post, you may want to quickly find out an anomaly, and notify users (or) take automated action. Example could be switching off machine when motor temperature crosses predetermined threshold value.

You need tools which can detect events as tons of data started coming from millions of devices. You need tools to respond once an event is detected. You need tools to trigger a series of actions automatically based on events detected. AWS IoT Events is one such tool which helps you in automating these workflows.

AWS IoT Events is a fully managed IoT service. You need not manage any infrastructure. Using AWS IoT Events, it’s simple to detect events across thousands of IoT sensors sending different telemetry data, such as temperature from a freezer, humidity from respiratory equipment, and belt speed on a motor. There are many things you can do once you detect an event e.g.

• Persist to DynamoDB table.
• Send a message to IoT Analytics channel.
• Send a message to SQS Queue.
• Send a message to S3 bucket.
• Send a message to SNS.
• Trigger a lambda to notify users.

Real time IoT device data monitoring using AWS Kinesis Data Analytics Service

In our previous blog post titled “Functional Building Blocks of an IoT Platform”, we have thoroughly discussed the importance of Edge Stream Analytics (real time processing), Cloud Stream Analytics (real time processing).

Let us consider a scenario. You may want to aggregate data over short time windows continuously, detect anomalies, and take quick actions such as sending alerts. In all such scenarios, where real time processing is the key, AWS Kinesis Data Analytics Service helps you out. For example, Kinesis Analytics can calculate rolling 10-second averages of valve temperatures every 5 minutes in industrial equipment and detect when the temperature exceeds certain preset thresholds. It can then alert control systems to automatically shut off machinery, avoiding accidents. This service offers very little latency, ranging from milli seconds to few seconds.

Deriving long-term insights from IoT historical data using AWS IoT Analytics Service

In our previous blog post titled “Functional Building Blocks of an IoT Platform”, we have thoroughly discussed the importance of Edge ML (near real time, involves historical data analysis), Cloud ML (near real time, involves historical data analysis). As discussed in our previous blog post, organizations create value by implementing ML algorithms, making decisions / predictions based on IoT sensor data in near real time. Let us look at some of the common use cases and solutions where big data analytics & predictions play a key role.

• IoT for Health Care: As part of personalized patient health monitoring, you may collect the data (vitals readings) from medical devices & store it in optimized data stores. ML & data analytics solutions play a big role in trend analysis, historical data processing, and make predictions / early diagnosis accurately, and notify users. You must have heard a lot of stories about iPhone alerting users based on heartbeat inconsistency. This falls under this category.
• Industrial IoT: You may monitor critical & high priority industrial assets health continuously & store it in optimized data stores. ML algorithms does historical data processing & predict when they fail and provide early notifications which will prompt you to take necessary actions before they fail. ML plays a big role in predictive maintenance & achieving high OEE (overall equipment efficiency).

The common requirement of all these use cases is – you need a “Collect – Enrich – Process – Store – Long Term Historical Data – Analytics – Predict” platform, which can provide you with accurate insights & predictions.

AWS IoT Analytics service does the same thing. AWS IoT Analytics service is designed for analytics on data at rest / historical data analysis / predictive maintenance / making predictions. AWS IoT Analytics is a fully managed service, designed specifically for IoT and automatically captures and stores the message timestamp, so it is easy to perform time-series analytics. IoT Analytics can also enrich the data with device-specific metadata such as device type and location using the AWS IoT registry and other public data sources (ex: weather APIs data). The good thing is, AWS IoT Analytics automates each of these difficult steps that are required to analyze data from IoT devices. IoT Analytics stores the device data in IoT-optimized data store so you can run queries on large datasets.

AWS IoT Analytics service filters, transforms, and enriches IoT data before storing it in a time-series data store for analysis. You can set up the service to collect only the data (filtered) you need from your devices into channels. You can set up a new pipeline in the service to apply mathematical transforms to process the data and enrich the data with device-specific metadata such as device type and location before storing the processed data in to “Data Store”. Then, you can analyze your data by running ad hoc or scheduled queries using the built-in SQL query engine or perform more complex analytics and machine learning inference. This results in output “Data Set”.

AWS IoT Analytics makes it easy to get started with machine learning by including pre-built models for common IoT use cases. Not only pre-built ML templates, but you can also use your own custom ML models, packaged in a container, to execute on AWS IoT Analytics. Of course, AWS IoT Analytics supports even your custom logic created in Jupyter Notebook.

Enable Edge Computing using AWS Greengrass

In our previous blog post titled “Functional Building Blocks of an IoT Platform”, we have thoroughly discussed Cloud Computing & Edge Computing paradigms. In fact, we have recommended following “staged approach: data pre-processing at edge, and detailed data processing at cloud”. We have even discussed the key benefits of this staged approach.

To realize these benefits, you need a solution – which enables device to device communications, which ensures device operations continuity whether internet is there or not, which helps you in setting up the business rules / data processing jobs at the edge, which helps you in storing the data locally. AWS IoT Greengrass does exactly the same thing.

AWS IoT Greengrass seamlessly extends AWS services to devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage.

 

• With AWS IoT Greengrass, you can run AWS Lambdafunctions locally on devices, you can run predictions based on machine learning models locally on devices, you can keep device data in sync, and communicate with other devices securely – even when not connected to the internet.
• AWS IoT Greengrass can be programmed to filter device data, perform data aggregations, and only transmit necessary information back to the cloud.
• Using AWS IoT Greengrass Connectors, you can integrate device data with third-party applications, on-premises software, and AWS services out-of-the-box.
• Using AWS IoT Greengrass Connectors, you can jumpstart device onboarding with pre-built protocol adapter integrations. For example, with MobBus – TCP protocol adapter, you can start reading data from PLCs & industrial machines.
• AWS IoT Greengrass offers device authentication services locally; devices can authenticate against AWS IoT Greengrass locally – which is equivalent to authenticating against AWS Cloud.

Select suitable service from multiple data store options offered by AWS Cloud

AWS offers a range of data store options. It offers relational databases, it offers NoSQL kind of databases, it offers time-series data stores, it offers data warehouse options, it offers data lake options. You can choose the service based on your needs. The most popular being Amazon Aurora, RDS, DynamoDB, S3, S3 Glacier, TimeStream database.

• Amazon RDS: Amazon RDS supports various RDMBS such as MySQL, Postgres, SQL Server and so on. You can consider storing relational data like tenants, users, devices, device metadata etc. in this database.
• Amazon Aurora: Amazon Aurora is a relational database management system (RDBMS) built for the cloud with full MySQL and PostgreSQL compatibility. This is part of Amazon RDS family.
• Amazon DynamoDB: Amazon DynamoDB is a fully managed, serverless, key-value NoSQL database designed to run high-performance applications at any scale. DynamoDB offers built-in security, continuous backups, automated multi-Region replication, in-memory caching, and data import and export tools. In DynamoDB, tables, items and attributes are the core components that you work with. Simply put, a table is a collection of items, and each item is a collection of attributes. DynamoDB uses primary keys to uniquely identify each item in a table and secondary indexes to provide more querying flexibility. You can consider DynamoDB to store your IoT device data.
• Amazon TimeStream: Timestream is a fast, scalable, and serverless time-series database service. Unlike relational databases, the tables in Timestream are append-only, which means that no deletes or updates are permitted. You can consider this service to store your IoT devices data in time series manner.
• Amazon RedShift: Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud.
• Amazon S3: Amazon Simple Storage Service (Amazon S3) is an object storage service. It is used for storing unstructured data. You can use this service to store files, images, and videos.
• Amazon S3 Glacier: You can consider this service for your long-term data archival, cold-storage needs. This is part of Amazon S3 family.

Conclusion

In this blog we have investigated AWS IoT Suite of services in detail and discussed where to use it while building your IoT platform.

In our next blog post, we will look at AWS IoT Reference Architecture, and we will understand architecture flow from devices to AWS cloud to Custom web applications and mobile applications.