Posted on 1 Comment

Introduction to Azure IoT Hub and Related Services SDKs

In February 2021, I wrote an article exploring the differences between Azure IoT Hub Devices and Modules. It was well-received and remains one of my most-visited blog posts. Now, with more experience in device and Edge development, I’d like to revisit the topic, but expand it to include other types of Azure IoT SDKs. There have been changes to the Azure IoT Hub device landscape, and I’m better equipped to provide useful information about the various SDKs available.

Microsoft has these divided into three basic SDK types.

  • Device SDKs
  • Service SDKs
  • Management SDKs

To make this as simple as possible, I like to think of the SDKs as serving these purposes:

  • Device – this code runs on the device either as bare metal or in a container. It represents either a leaf device or an Edge gateway. If it connects to IoT to feed it data, it needs this SDK.
  • Service – this code enables me to manage my devices with updates or other forms of communication using IoT Hub. This involves administrative tasks, all of which are focused on using the IoT Hub API to interact with the devices.
  • Management is more complex, since these are SDKs designed to manage your IoT Hub or Hubs. I could use this to monitor my IoT Hub, create or delete Hub instances, and keep track of my shared access policies.

This document focuses on the SDKs and their updates to match the latest versions of Azure IoT services. Use this as a starting point for further research. If you spot any mistakes or think I’ve missed something important, please let me know in the comments. I’m always open to feedback from my readers. I’m well aware that I make mistakes, and I’m always learning.

What are Azure IoT Hub Device SDKs?

Azure IoT Hub Device SDKs provide APIs and libraries that allow developers to create applications for IoT devices and connect them to Azure IoT Hub. These SDKs support a wide range of programming languages and platforms, and offer unique sets of features tailored to the language and platform. Common use cases include sending telemetry data, receiving commands, connecting devices using various protocols, authenticating devices, handling messaging, managing device twins, and receiving notifications.

A small code example in Python

import os
from azure.iot.device import IoTHubDeviceClient
# Retrieve the connection string for the device from an environment variable
conn_str = os.getenv("IOTHUB_DEVICE_CONNECTION_STRING")
# Create an IoT Hub client
client = IoTHubDeviceClient.create_from_connection_string(conn_str)
# Send a message to IoT Hub
message = "Hello, I am a device using IoT Hub!"
client.send_message(message)
print("Sent message: {}".format(message))
# Disconnect the client
client.disconnect()

This code imports the necessary modules from the Azure IoT Hub Device SDK and creates an IoT Hub client. It then sends a message to IoT Hub and disconnects the client when finished.

To use this code, you must have the Azure IoT Hub Device SDK installed and have a connection string for a device registered with IoT Hub.

Unlike many vendor devices, the SDKs present you with the opportunity to create your own IoT devices. You may have heard of IoT Edge, so let’s address what that is now.

What is Azure IoT Edge?

Azure IoT Edge is a fully managed service that enables developers to deploy cloud intelligence directly onto IoT devices. It allows them to build, deploy, and manage applications that run on edge devices and leverage cloud-based services and data to perform tasks such as data processing, analytics, machine learning, and more.

Azure IoT Edge consists of three main components:

  1. Edge Modules: Small, lightweight pieces of code that run on edge devices and perform specific tasks. Edge modules can be written in a variety of programming languages, including C#, Java, Python, and more.
  2. IoT Edge Runtime: A lightweight container that runs on edge devices and manages the lifecycle of edge modules. The IoT Edge runtime can be installed on a wide range of operating systems and devices, including Windows, Linux, and various embedded platforms.
  3. Azure IoT Hub: A cloud-based service that acts as the central hub for communication between edge devices and the cloud. IoT Hub provides a secure and scalable platform for managing devices, sending commands and notifications to devices, and receiving telemetry data from devices.

Azure IoT Edge enables developers to create and deploy intelligent edge solutions that bring the power of the cloud to the edge of the network. These solutions can perform tasks such as data processing, analytics, machine learning, and more on edge devices, reducing the amount of data that needs to be sent to the cloud and enabling fast decision-making and action. Azure IoT Edge is especially useful for scenarios where latency or connectivity issues make it difficult to rely solely on the cloud for data processing and analysis.

A small code example

import os
from azure.iot.edge.module_client import ModuleClient
# Create a module client
client = ModuleClient.create_from_edge_environment()
# Connect the module client to IoT Hub
client.connect()
# Send a message to IoT Hub
message = "Hello from IoT Edge!"
client.send_message_to_output("channel1", message)
print("Sent message: {}".format(message))
# Disconnect the module client
client.disconnect()

This code imports the necessary modules from the Azure IoT Edge Python SDK and creates a module client. It then uses the module client to connect to IoT Hub, send a message, and disconnect.

To use this code, you must have the Azure IoT Edge Python SDK installed and an IoT Edge runtime running on your device. Additionally, an IoT Edge module must be deployed to the device with an output binding named “channel1”.

These are basic approaches to the device and modules, but I wanted to give you an idea of how easy they can be to use. Setting up IoT Edge for development involves some complexity, but the ability to run containerized cloud solutions at the edge makes it worthwhile.

What about IoT Central?

Many of the concepts covered in this article are also true or will work with IoT Central. For this article, I’m focusing on IoT Hub. I will have a future article dedicated to IoT Central and it’s SDKs and APIs.

Azure IoT Hub Device SDKs

Azure IoT Hub Device SDKs are software development kits (SDKs) that enable developers to create applications that run on IoT devices and connect them to Azure IoT Hub. These SDKs provide a set of APIs and libraries that allow devices to perform basic and complex IoT device to cloud communications and commands.

Azure IoT Hub supports device SDKs for a wide range of programming languages and platforms, including C, C#, Java, JavaScript, Python, and a few that are platform specific. Each device SDK provides a set of APIs and libraries tailored to the programming language and platform, making it easy for developers to create applications that interact with Azure IoT Hub.

Some common use cases for Azure IoT Hub Device SDKs include:

  • Sending telemetry data from devices to IoT Hub
  • Receiving commands from IoT Hub and executing them on devices
  • Connecting devices to IoT Hub using various protocols, such as MQTT, AMQP, HTTP, and more
  • Authenticating devices with IoT Hub using various authentication mechanisms, such as shared access signatures (SAS) tokens and X.509 certificates
  • Handling device-to-cloud and cloud-to-device messaging in a reliable and secure manner
  • Managing device twins, which are JSON documents that store metadata about devices in IoT Hub
  • Receiving notifications about device connections, disconnections, and other events from IoT Hub

Overview of the different device SDKs available

  • C: The C Device SDK is a lightweight, portable library that enables C-based devices to connect to Azure IoT Hub and interact with the cloud. It supports various protocols, including MQTT, AMQP, HTTP, and more, and provides APIs for tasks such as sending telemetry data, receiving commands, and authenticating with the IoT Hub.
#include <stdio.h>
#include <stdlib.h>
#include "iothub_client.h"
#include "iothub_message.h"
#include "azure_c_shared_utility/threadapi.h"
#include "azure_c_shared_utility/crt_abstractions.h"
#include "azure_c_shared_utility/platform.h"
int main(void)
{
    // Retrieve the connection string for the device from an environment variable
    char* conn_str = getenv("IOTHUB_DEVICE_CONNECTION_STRING");
    // Create an IoT Hub client
    IOTHUB_CLIENT_HANDLE client = IoTHubClient_CreateFromConnectionString(conn_str);
    if (client == NULL)
    {
        (void)printf("ERROR: iothub_client_create failed\\n");
    }
    else
    {
        // Send a message to IoT Hub
        IOTHUB_MESSAGE_HANDLE message = IoTHubMessage_CreateFromString("Hello, Azure IoT Hub!");
        if (IoTHubClient_SendEventAsync(client, message, NULL, NULL) != IOTHUB_CLIENT_OK)
        {
            (void)printf("ERROR: IoTHubClient_SendEventAsync failed\\n");
        }
        else
        {
            (void)printf("Sent message: Hello, Azure IoT Hub!\\n");
        }
        IoTHubMessage_Destroy(message);
    }
    // Disconnect the client
    IoTHubClient_Destroy(client);
    return 0;
}

This code includes the necessary header files from the Azure IoT Hub C Device SDK and uses them to create an IoT Hub client. It then sends a message to IoT Hub using the client, before disconnecting the client when complete.

To use this code, you must have the Azure IoT Hub C Device SDK installed on your machine and have a connection string for a device registered with IoT Hub.

  • C#: The C# Device SDK is a fully-featured library that enables C#-based devices to connect to Azure IoT Hub and interact with the cloud. It supports various protocols, including MQTT, AMQP, HTTP, and more, and provides APIs for tasks such as sending telemetry data, receiving commands, and authenticating with the IoT Hub. It also includes support for advanced features such as device twins and direct method invocations.
using Microsoft.Azure.Devices.Client;
using System;
using System.Text;
using System.Threading.Tasks;
namespace ConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            // Retrieve the connection string for the device from an environment variable
            string conn_str = Environment.GetEnvironmentVariable("IOTHUB_DEVICE_CONNECTION_STRING");
            // Create an IoT Hub client
            DeviceClient client = DeviceClient.CreateFromConnectionString(conn_str);
            // Send a message to IoT Hub
            string message = "Hello, Azure IoT Hub!";
            byte[] message_bytes = Encoding.UTF8.GetBytes(message);
            Message iot_hub_message = new Message(message_bytes);
            client.SendEventAsync(iot_hub_message).Wait();
            Console.WriteLine("Sent message: {0}", message);
            // Disconnect the client
            client.CloseAsync().Wait();
        }
    }
}

This code imports the necessary namespaces from the Azure IoT Hub C# Device SDK and creates an IoT Hub client. It then sends a message to IoT Hub, before disconnecting the client.

To use this code, you must install the Azure IoT Hub C# Device SDK on your machine and obtain a connection string for a device registered with IoT Hub.

  • Java: The Java Device SDK is a fully-featured library that enables Java-based devices to connect to Azure IoT Hub and interact with the cloud. It supports various protocols, including MQTT, AMQP, HTTP, and more, and provides APIs for tasks such as sending telemetry data, receiving commands, and authenticating with the IoT Hub. It also includes support for advanced features such as device twins and direct method invocations.
import com.microsoft.azure.sdk.iot.device.DeviceClient;
import com.microsoft.azure.sdk.iot.device.IotHubClientProtocol;
import com.microsoft.azure.sdk.iot.device.Message;
public class Main
{
    public static void main(String[] args) throws Exception
    {
        // Retrieve the connection string for the device from an environment variable
        String conn_str = System.getenv("IOTHUB_DEVICE_CONNECTION_STRING");
        // Create an IoT Hub client
        IotHubClientProtocol protocol = IotHubClientProtocol.AMQPS;
        DeviceClient client = new DeviceClient(conn_str, protocol);
        client.open();
        // Send a message to IoT Hub
        String message = "Hello!";
        Message iot_hub_message = new Message(message);
        client.sendEventAsync(iot_hub_message);
        System.out.println("Sent message: " + message);
        // Disconnect the client
        client.close();
    }
}

This code includes the necessary namespaces from the Azure IoT Hub Java Device SDK and uses them to create an IoT Hub client. It then uses the client to connect to IoT Hub, send a message to IoT Hub, and disconnect the client when it’s finished.

  • JavaScript: The JavaScript Device SDK is a fully-featured library that enables JavaScript-based devices to connect to Azure IoT Hub and interact with the cloud. It supports various protocols, including MQTT, AMQP, HTTP, and more, and provides APIs for tasks such as sending telemetry data, receiving commands, and authenticating with the IoT Hub. It also includes support for advanced features such as device twins and direct method invocations.
const { Client } = require('azure-iot-device');
// Retrieve the connection string for the device from an environment variable
const conn_str = process.env.IOTHUB_DEVICE_CONNECTION_STRING;
// Create an IoT Hub client
const client = Client.fromConnectionString(conn_str, Client.IotHubTransport);
// Connect the client
client.open((err) => {
    if (err) {
        console.error('Error opening the client: ' + err.message);
    } else {
        // Send a message to IoT Hub
        const message = new Message('Hello, Azure IoT Hub!');
        client.sendEvent(message, (send_err) => {
            if (send_err) {
                console.error('Error sending the message: ' + send_err.message);
            } else {
                console.log('Sent message: ' + message.getData());
            }
        });
    }
});

This code imports the necessary modules from the Azure IoT Hub JavaScript Device SDK, creating an IoT Hub client. It then connects to IoT Hub and sends a message.

  • Python: The Python Device SDK is a fully-featured library that enables Python-based devices to connect to Azure IoT Hub and interact with the cloud. It supports various protocols, including MQTT, AMQP, HTTP, and more, and provides APIs for tasks such as sending telemetry data, receiving commands, and authenticating with the IoT Hub. It also includes support for advanced features such as device twins and direct method invocations.
import os
from azure.iot.edge.module_client import ModuleClient
# Create a module client
client = ModuleClient.create_from_edge_environment()
# Connect the module client to IoT Hub
client.connect()
# Send a message to IoT Hub
message = "Hello from IoT Edge!"
client.send_message_to_output("output1", message)
print("Sent message: {}".format(message))
# Disconnect the module client
client.disconnect()

Device SDKs for specific programming languages and platforms

Azure RTOS

Azure RTOS (Real-Time Operating System) is a suite of RTOS kernels and middleware components optimized for use with small, resource-constrained devices running on Arm Cortex-M microcontrollers. It provides developers with a robust and scalable platform for building IoT solutions that require real-time performance, low power consumption, and a small footprint.

The Azure RTOS middleware is a set of software components that provide additional functionality and support for developing IoT solutions. These components include:

  • Networking libraries and protocols such as TCP/IP, HTTP, HTTPS, and MQTT, which allow devices to connect to the cloud and other devices over the network.
  • Security libraries and protocols such as SSL/TLS, X.509 certificates, and secure boot, which allow devices to authenticate and encrypt communication with the cloud and other devices.
  • A file system library that allows devices to store and retrieve data on external storage devices such as SD cards.
  • The ThreadX RTOS kernel, which provides a lightweight, scalable, and reliable platform for building real-time applications on Arm Cortex-M microcontrollers.

Azure RTOS is available for a variety of Arm Cortex-M microcontrollers, including those from STMicro.

FreeRTOS

FreeRTOS is an open-source real-time operating system (RTOS) for building small, resource-constrained devices for the Internet of Things (IoT). It is designed to be lightweight, scalable, and reliable, offering a range of features and services that are beneficial for constructing IoT solutions, including:

  • Task scheduling and management
  • Memory management
  • Timers and software timers
  • Inter-task communication and synchronization
  • Networking libraries and protocols, such as TCP/IP, HTTP, and MQTT
  • Security libraries and protocols, such as SSL/TLS, X.509 certificates, and secure boot
  • File system libraries
  • Device drivers and middleware components for various hardware peripherals

FreeRTOS is available for a wide range of microcontrollers and platforms, including those from Atmel, NXP, and STMicroelectronics. It has a large community of developers and users, and is regularly updated and improved.

Azure IoT provides libraries and tools to connect FreeRTOS-based devices to its services, such as the IoT Hub, IoT Central, and IoT Edge. This enables developers to create IoT solutions that benefit from the scalability, security, and reliability of the Azure cloud.

Bare Metal (Azure SDK for Embedded C)

The Azure SDK for Embedded C is a software development kit (SDK) that enables developers to build IoT solutions using the C programming language and Microsoft Azure cloud services. It provides libraries, tools, and documentation to easily connect C-based devices to Azure IoT services such as IoT Hub, IoT Central, and IoT Edge, and construct IoT solutions that benefit from the scalability, security, and reliability of the Azure cloud.

The SDK supports a variety of microcontrollers and platforms from manufacturers like Atmel, NXP, and STMicroelectronics. It offers support for protocols and features like MQTT, AMQP, HTTP, device twins, and direct method invocations, and provides APIs for tasks like sending telemetry data, receiving commands, and authenticating with Azure IoT services.

The Azure SDK for Embedded C is open source and constantly updated and improved by a community of developers and users. It is lightweight, portable, and easy to use, and can be integrated into various development environments and toolchains.

Connecting devices to Azure IoT Hub using the device SDKs

To connect a device to Azure IoT Hub using a device SDK, you need to:

  1. Install the device SDK on your device or development machine. The device SDK provides the necessary libraries and tools to build IoT solutions that can connect to Azure IoT Hub.
  2. Register your device with Azure IoT Hub. This step involves creating a device identity in IoT Hub and obtaining a connection string that uniquely identifies your device. The connection string includes the device’s ID, hub name, and authentication key, and is used to authenticate the device with IoT Hub when it connects.
  3. Use the device SDK to create an IoT Hub client on your device. The IoT Hub client is responsible for establishing and maintaining a connection with IoT Hub, as well as sending and receiving messages.
  4. Use the IoT Hub client to connect to IoT Hub. This typically involves providing the client with the connection string and specifying the protocol to use (such as MQTT or AMQP). The client will then establish a connection with IoT Hub and authenticate using the information in the connection string.
  5. Use the IoT Hub client to send messages to IoT Hub and receive messages from IoT Hub. The device SDK provides APIs for tasks such as sending telemetry data, receiving commands, and interacting with device twins and direct methods.

This article provided a basic overview of the device SDKs for the languages and platforms supported for Azure IoT Hub. The example code was meant to give a glimpse of how it works, but not of real world workloads. In most cases, you’ll use the local device libraries to connect to sensors and collect data to send to IoT Hub. This usually requires developing an asynchronous listener to capture the telemetry data and send it in a message to IoT Hub. This functionality is beyond the scope of this article, but I am working on a newer set of Agriculture code for one of my devices. I will post it here soon, which will provide a more complete solution to device development on a Raspberry Pi.

Azure IoT Hub Service and Management SDKs

The Azure IoT Hub Service SDKs are a set of software development kits (SDKs) that enable developers to build applications that interact with Azure IoT Hub. They provide APIs and libraries in various programming languages and platforms, allowing developers to create, manage, and interact with devices and other resources in IoT Hub. The SDKs are available for .NET, Java, Node.js, Python, and Go, and offer functions such as sending and receiving messages, creating and updating devices, managing device twins and direct methods, and monitoring device health and status.

Overview of the different service SDKs available

The Azure IoT Hub Service SDKs are a set of software development kits (SDKs) that enable developers to build applications that interact with Azure IoT Hub. They provide APIs and libraries in various programming languages and platforms, allowing developers to create, manage, and interact with devices and other resources in IoT Hub. From the choices, it’s clear that these SDKs are designed to supplement or amplify the capabilities of IoT Hub.

These are just a few of the possible uses for the Service SDK:

  • Sending and receiving messages to and from devices
  • Creating, updating, and deleting devices in IoT Hub
  • Managing device twins and direct methods
  • Monitoring the health and status of devices and other resources in IoT Hub

A couple of specific use cases I can imagine are:

  • An integration with an application or an enterprise system that is not available through the standard IoT Hub integration channels.
  • Updating device digital twins is a good use of the SDK, especially for automated processes
  • Device Registry is particularly useful when dealing with a large number of devices. Automating the process is highly recommended. I will go into more detail about this in a future blog post.

The Azure IoT Hub Service SDKs are available for a variety of programming languages and platforms, such as:

  • .NET: The Azure IoT Hub .NET Service SDK
  • Java: The Azure IoT Hub Java Service SDK
  • Node.js: The Azure IoT Hub Node.js Service SDK
  • Python: The Azure IoT Hub Python Service SDK
  • Go: The Azure IoT Hub Go Service SDK

Service SDKs for specific programming languages and platforms

  • .NET: The Azure IoT Hub .NET Service SDK

This is likely the library that most traditional Microsoft developers will gravitate toward. It’s definitely a well-developed, highly supported library. This library has been migrated to

https://github.com/Azure/azure-iot-sdk-csharp

The Azure IoT SDK for C# is a software development kit that enables developers to build IoT solutions using C# and Microsoft Azure cloud services. It provides libraries, tools, and documentation to connect C#-based devices and applications to Azure IoT services, and includes support for a variety of microcontrollers and protocols. The SDK is open source and designed to be lightweight, portable, and easy to use.

I’m not including code sample here, because they tend to be a bit larger. However, I think it’s just as important to work with the service SDKs as the device SDKs. You’ll find they make your administrative work easier.

  • Java: The Azure IoT Hub Java Service SDK

The Java Service SDK offers many of the same functions as the .Net library. It includes the following samples to complete tasks related to device management, data processing and analytics, and integration and orchestration:

  • Device management: Create, update, and delete devices in IoT Hub, manage device twins, and use direct methods.
  • Data processing and analytics: Receive telemetry data from devices in IoT Hub, process and analyze the data, and trigger alerts or create reports.
  • Integration and orchestration: Integrate IoT Hub with other systems and services, such as enterprise applications or cloud platforms, and build complex workflows and processes that span multiple devices and services.

azure-iot-sdk-java/service/iot-service-samples at main · Azure/azure-iot-sdk-java

  • JavaScript: Azure SDK for JS

The IoT Hub Service SDK for Node.js doesn’t appear to have code for the IoT Hub service, yet. However, the JavaScript is still available. It has many of the same functionality as the other SDKs. One thing to note is that many of these are in TypeScript and it appears that there are more samples available here. So if you’re looking for a functionality that doesn’t exists in the other samples, you may want to search through these code examples:

azure-sdk-for-js/sdk/iothub/arm-iothub/samples-dev at main · Azure/azure-sdk-for-js

  • Python: The Azure IoT Hub Python Service SDK

The Azure IoT Hub Python library is here:

https://github.com/Azure/azure-iot-hub-python

It has many of the same functionality as the other library. I would say that the Client library is a bit more robust than the library for IoT Hub, but most of the functionality you’ll want exists. There are also plenty of examples to get you started. And one of the things I love about Python is how much you can accomplish with so little code:

# -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See License.txt in the project root for
# license information.
# --------------------------------------------------------------------------
import os
import msrest
from azure.iot.hub import DigitalTwinClient
iothub_connection_str = os.getenv("IOTHUB_CONNECTION_STRING")
device_id = os.getenv("IOTHUB_DEVICE_ID")
try:
    # Create DigitalTwinClient
    digital_twin_client = DigitalTwinClient.from_connection_string(iothub_connection_str)
    # Get digital twin and retrieve the modelId from it
    digital_twin = digital_twin_client.get_digital_twin(device_id)
    if digital_twin:
        print(digital_twin)
        print("Model Id: " + digital_twin["$metadata"]["$model"])
    else:
        print("No digital_twin found")
except msrest.exceptions.HttpOperationError as ex:
    print("HttpOperationError error {0}".format(ex.response.text))
except Exception as exc:
    print("Unexpected error {0}".format(exc))
except KeyboardInterrupt:
    print("{} stopped".format(__file__))
finally:
    print("{} finished".format(__file__))
  • Go: Azure SDK for Go

To my knowledge, there is no official Microsoft sponsored IoT Hub or Client library for GO. At the time of this writing, the services supported are:

  • Azure Blob Storage
  • Azure Functions
  • Azure Container Registry
  • Azure Container Instance
  • Azure Kubernetes Service
  • Azure Virtual Machines
  • Azure Key Vault
  • And Azure Cognitive Service

However, the success of Go as a Cloud Native language has prompted an Azure Go SDK. There is a version that appears to have been created here:

https://github.com/amenzhinsky/iothub

There is code for Device and Service here with very few samples, but it’s probably enough to get someone started:

package main
import (
	"context"
	"fmt"
	"log"
	"os"
	"github.com/amenzhinsky/iothub/iotservice"
)
func main() {
	c, err := iotservice.NewFromConnectionString(
		os.Getenv("IOTHUB_SERVICE_CONNECTION_STRING"),
	)
	if err != nil {
		log.Fatal(err)
	}
	// subscribe to device-to-cloud events
	log.Fatal(c.SubscribeEvents(context.Background(), func(msg *iotservice.Event) error {
		fmt.Printf("%q sends %q", msg.ConnectionDeviceID, msg.Payload)
		return nil
	}))
}

Using the management SDKs to communicate with devices and manage IoT Hub resources

Here, I want to discuss some unique ways to use the Azure IoT Hub Management SDKs to work with IoT Hub. The SDKs provide APIs and libraries that enable developers to manage and operate IoT Hub, such as creating and deleting instances, setting and querying properties and quotas, managing shared access policies, monitoring usage and diagnostics, and performing maintenance tasks like repairing or backing up IoT Hub. The SDKs are available for .NET, Python, and Node.js, and can be used on Windows, macOS, and Linux.

I don’t think the average IoT developer will be too concerned with these SDKs. However, if you work in an organization with multiple IoT hubs and millions of devices, these packages could be useful. Additionally, if you want to build products that use IoT Hub as a base service, then you should definitely look into these SDKs.

  • Azure SDK for .Net IoT Management

To get down to more detail, because I think it’s important to look at the detail for this particular SDK, here’s what’s delivered in the primary IoT Hub client for management. This piece of code is what I’m reviewing:

azure-sdk-for-net/IotHubClient.cs at main · Azure/azure-sdk-for-net

The code is a part of the Azure IoT Hub .NET Management SDK, which is a set of libraries and tools for building .NET applications that manage and operate Azure IoT Hub.

The file, IotHubClient.cs, contains the implementation of the IotHubClient class, which is a client object that provides a set of methods and properties for interacting with the Azure IoT Hub service. The IotHubClient class is part of the generated code for the Azure Management Libraries for .NET, which is a set of libraries that provide a consistent, easy-to-use, and high-level API for accessing Azure cloud services.

The IotHubClient class is used to perform various tasks related to managing and operating an IoT Hub instance, such as:

  • Creating and deleting IoT Hub instances
  • Setting and querying IoT Hub properties and quotas
  • Managing IoT Hub shared access policies
  • Monitoring IoT Hub usage and diagnostics
  • Performing maintenance tasks such as repairing or backing up IoT Hub

The IotHubClient class exposes a set of methods that correspond to the different operations that can be performed on an IoT Hub instance. For example, the IotHubClient.CreateOrUpdateAsync method is used to create or update an IoT Hub instance, and the IotHubClient.GetAsync method is used to retrieve the properties of an IoT Hub instance.

  • Azure SDK for Java – Azure Resource Manager IotHub client library for Java

I believe this is a better library than the .Net version. I provides more capabilities and samples.

azure-sdk-for-java/sdk/iothub/azure-resourcemanager-iothub at main · Azure/azure-sdk-for-java

The Java library has many of the same features as other libraries. It is based on the Azure Resource Manager Java library, so if you are already familiar with it, using the IoT Hub Management Library should be straightforward.

  • Azure iotHub client library for JavaScript

This is the same library as the Service library, which makes sense. It explains why the Service library seems larger than the other service libraries. This is also built on top of the Azure Resource Management.

  • Azure SDK for Python – Azure Management IoT Hbu

This library is a bit more like the .Net library. You enter with an IoT Hub Client and access you capabilities that way. It offers most of the same capabilities of the other libraries.

Overall, these libraries offer similar capabilities. Java and JavaScript have had more development, but the .Net and Python versions are still useful. Additionally, the API provides many management capabilities. If the SDKs don’t meet your needs, you can use Azure Resource Manager to manage IoT Hub.

Azure IoT Services SDKs

Azure IoT SDKs provide a range of tools and services for building secure, connected, and location-aware IoT applications and solutions. This section will cover the basics of these products.

Azure Maps, Azure Sphere, Azure Digital Twins, Azure Object Anchors, and Azure Remote Rendering are all included in the Azure IoT resources. Each of these offers unique features and capabilities to enable the development of IoT applications and solutions.

Overview of the Azure IoT services SDKs available

Azure Maps

Azure Maps is a cloud-based mapping and location platform that provides a range of tools and services for building location-based applications and solutions. It is part of the Azure IoT resources, as it can be used to build location-aware IoT applications and solutions that incorporate geospatial data and functionality.

Some of its key features and capabilities include:

  1. Map rendering: Azure Maps offers 2D and 3D maps, street maps, satellite imagery, and terrain maps, which can be rendered and displayed in web-based, mobile, and embedded formats.
  2. Geocoding and reverse geocoding: Convert addresses and location coordinates into geographic locations, and vice versa.
  3. Routing and navigation: Calculate routes and directions between two or more locations.
  4. Location services: Geofencing, location search, and real-time traffic data for building location-based applications.

Overall, Azure Maps is a powerful and flexible platform for building location-based applications and solutions, especially for IoT scenarios.

Azure maps provides four separate SDKs to help developers:

  • REST SDK
  • Web SDK (JavaScript)
  • Android SDK (Java or Kotlin)
  • iOS (Swift)

Azure Maps Documentation

Azure Sphere

Azure Sphere is a secure and connected microcontroller platform designed for building Internet of Things (IoT) applications and solutions. It is part of the Azure IoT resources, providing a range of tools and technologies to build and deploy secure, connected IoT devices.

Key features and capabilities of Azure Sphere include:

  1. Security: Azure Sphere is designed to provide a high level of security for IoT devices. It features a custom-built Linux operating system and a hardware-based security module that provides secure boot, trusted execution, and hardware-level isolation.
  2. Connectivity: Azure Sphere supports various connectivity options, including Wi-Fi and cellular, allowing devices to connect to the internet and communicate with other devices and systems.
  3. Development tools: Azure Sphere includes a software development kit (SDK), libraries and APIs, and integration with Azure IoT Hub and other Azure services, making it easier to build and deploy IoT applications and solutions.

Overall, Azure Sphere is an excellent platform for building and deploying secure, connected IoT devices, especially in scenarios where security and connectivity are paramount. Azure Sphere comes with a complete development kit. There are two basic SDKs, a Windows and a Linux version. The only supported programming language for Sphere appears to be C, which makes sense considering it’s designed for embedded development.

Digital Twins

Azure Digital Twins is different from the device digital twin discussed in this article. It is a cloud-based platform that enables you to construct digital twin models of physical spaces, systems, and assets. A digital twin is a digital representation of a physical object or environment that reflects its real-world characteristics and behavior.

Azure Digital Twins is part of the Azure IoT resources, providing tools and capabilities to build and deploy IoT applications and solutions that incorporate digital twin models. Key features and capabilities of Azure Digital Twins include:

  1. Data modeling and integration: Create a digital twin model by defining objects and properties that represent physical objects and features in the environment, as well as the relationships between them. Integrate data from external sources, such as sensors, devices, and systems, into the digital twin model.
  2. Data storage and querying: Store and query data within the digital twin model. Use various data types and formats to represent data in the digital twin, and use SQL-like queries to retrieve and analyze the data.
  3. Behaviors and logic: Define behaviors and logic in the digital twin model, using tools such as Azure Functions and Azure Stream Analytics. This enables the digital twin to respond to events and conditions in real-time, and perform actions or calculations based on the data in the digital twin.
  4. Visualization and exploration: Use the Azure Digital Twins portal, Azure Grafana, or the Azure Digital Twins REST API to view and interact with the digital twin.

Overall, Azure Digital Twins is a powerful and flexible platform for creating and managing digital twin models, and enabling a wide range of use cases and scenarios in the IoT space.

Azure Digital Twins comes with control plane APIs, data plane APIs, and SDKs for managing instances and its elements. The SDKs available are for .NET (C#), Java, JavaScript, Python, and Go.

I plan to cover Azure Digital Twins in much greater detail in the future. I’m working on a small book on the subject for Solution Architects. I think it’s one of the most interesting technologies available today. I don’t think we’ve scratched the surface of its potential.

Object Anchors

Azure Object Anchors is a service that allows you to create, manage, and interact with digital representations of physical objects in mixed reality applications. It is included in the Azure IoT resources as it can be used to build IoT applications and solutions with mixed reality features.

Some of the key features and capabilities of Azure Object Anchors include:

  1. Object recognition: Azure Object Anchors can recognize and track physical objects in real-time using machine learning algorithms and computer vision techniques. This enables you to create digital representations of physical objects that can be displayed in mixed reality applications.
  2. Digital twin creation: Azure Object Anchors allows you to create digital twin models of physical objects, which can reflect the real-world characteristics and behavior of the objects. You can define the properties and relationships of the objects in the digital twin model, and integrate data from external sources such as sensors and devices.
  3. Mixed reality integration: Azure Object Anchors can be integrated with mixed reality platforms and tools, such as HoloLens and Unity, allowing you to build mixed reality applications that incorporate the digital twin models created with Azure Object Anchors.

Azure Object Anchors is a powerful platform for building mixed reality applications and solutions that incorporate digital representations of physical objects, particularly in IoT scenarios. Development is built around the Unity game development platform and the Hololens device, though not necessarily restricted to them. The SDKs available are a Conversion SDK for .NET designed to turn existing 3D model assets into Unity Digital Twin objects, as well as Runtime SDKs for Unity and Hololens. If you plan to include Augmented Reality into your IoT solution, Azure Object Anchors is a great option.

Remote Rendering

Azure Remote Rendering is a cloud-based service that enables high-quality 3D graphics and visualization to be rendered in real-time, using the power and scale of the cloud. It is included in the Azure IoT resources because it can be used to build IoT applications and solutions that incorporate 3D graphics and visualization, and that require real-time rendering at scale.

Key features and capabilities of Azure Remote Rendering include:

  1. 3D graphics rendering: Azure Remote Rendering offers a range of tools and technologies for rendering high-quality 3D graphics, such as support for various 3D file formats, realistic lighting and materials, and advanced rendering techniques like ray tracing.
  2. Real-time rendering: Azure Remote Rendering is designed to support real-time rendering of 3D graphics, even at high resolutions and frame rates. This is useful for applications and solutions that need to display dynamic, interactive 3D graphics in real-time.
  3. Cloud-based rendering: Azure Remote Rendering leverages the power and scale of the cloud to enable high-performance rendering of 3D graphics. This is beneficial for applications and solutions that need to support large numbers of users or devices, or that need to process large volumes of data.
  4. Integration with other Azure services: Azure Remote Rendering can be integrated with other Azure services, such as Azure IoT Hub and Azure Stream Analytics, which allows for the building of IoT applications and solutions that incorporate 3D graphics and visualization.

Overall, Azure Remote Rendering is an effective platform for building applications and solutions that incorporate 3D graphics and visualization, and that require real-time rendering at scale. It is particularly suitable for use in IoT scenarios where 3D graphics and visualization are important.

Developing for this service requires making use of the existing REST APIs. Those are:

  • Azure Mixed Reality Resource Management REST API
  • Remote Rendering REST API

There is an extensive Microsoft.Azure.RemoteRendering API library available for C#.

Spatial Anchors

Azure Spatial Anchors is a service that enables you to create and manage digital anchors in physical spaces. It can be used to build IoT applications and solutions that incorporate location-based functionality and support mixed reality scenarios. I think one of the best examples of this technology that many people are probably familiar with is the mobile game Pokemon Go. This massive multi-player platform allows gamers to track, capture, train, and battle fantasy creatures in a mixed-reality environment.

Its key features and capabilities include:

  1. Digital anchor creation: You can define the properties and relationships of the anchors in the digital model, and integrate data from external sources such as sensors and devices.
  2. Mixed reality integration: Azure Spatial Anchors can be integrated with mixed reality platforms and tools, such as HoloLens and Unity.
  3. Real-time anchor tracking: It supports real-time tracking of the digital anchors, allowing you to update the location and orientation of the anchors in the mixed reality application in real-time.
  4. Cloud-based anchor management: It provides a cloud-based platform for managing the digital anchors, which is useful for applications and solutions that need to support large numbers of users or devices, or handle large volumes of data.

Overall, Azure Spatial Anchors is a powerful platform for building mixed reality applications and solutions that incorporate location-based functionality. It is especially suitable for IoT scenarios where mixed reality and location-based functionality are important.

There are multiple Spatial Anchors SDKs:

  • SDK for Unity
  • SDK for iOS Objective-C
  • SDK for Android Java
  • SDK for Android NDK
  • SDK for HoloLens C++/WinRT
  • Azure CLI
  • Azure Powershell

There are multiple samples to work through to learn this resource. Again, this is another one of the family of resources that helps bring real world data to life in the digital world.

Time Series Insights

Azure Time Series Insights is a cloud-based platform that allows you to store, visualize, and analyze time series data in real-time. It is included in the Azure IoT resources because it can be used to build IoT applications and solutions that need to process, analyze, and visualize large volumes of time series data in real-time.

Some of its key features and capabilities include:

  1. Time series data storage: Azure Time Series Insights provides a scalable and highly available data store for storing time series data. It can handle large volumes of data and supports a variety of data types and formats.
  2. Real-time data visualization: Azure Time Series Insights offers a range of visualization tools and interfaces that allow you to view and analyze time series data in real-time. You can use the Azure Time Series Insights portal, Azure Grafana, or the Azure Time Series Insights REST API to visualize and explore the data.
  3. Query and analysis: Azure Time Series Insights provides a query language and a range of analytical functions that allow you to perform complex queries and analysis on the time series data. This can be useful for applications and solutions that need advanced analysis.
  4. Integration with other Azure services: Azure Time Series Insights can be integrated with other Azure services, such as Azure IoT Hub and Azure Stream Analytics, to build IoT applications and solutions that incorporate time series data and analysis.

Overall, Azure Time Series Insights is a powerful platform for building IoT applications and solutions that need to process, analyze, and visualize large volumes of time series data in real-time. It is particularly well-suited for scenarios where real-time data analysis and visualization are important.

Azure Time Series Insights and Azure Data Explorer are both cloud-based platforms designed to store, process, and analyze large volumes of data in real-time. However, there are key differences between the two.

Data types and formats: Azure Time Series Insights is designed for time series data, which is data collected and recorded over time at regular intervals. Azure Data Explorer is more general-purpose, and can handle structured and unstructured data.

Query language: Azure Time Series Insights uses Kusto Query Language (KQL) for time series data. Azure Data Explorer also uses KQL, but it is more flexible and can query a wider variety of data types and formats.

Visualization: Both platforms offer visualization tools and interfaces for viewing and exploring the data. Azure Time Series Insights is more specialized for time series data, and provides visualization options specifically designed for it.

Integration with other Azure services: Both platforms can integrate with Azure IoT Hub and Azure Stream Analytics, but the specific integration options and capabilities may vary.

Overall, Azure Time Series Insights is specialized for time series data, while Azure Data Explorer is more general-purpose.

Other Services

There are additional services that don’t fit into the IoT resources category, but are often used with Azure IoT Hub to fulfill the backend data processing needs of an IoT platform. These include:

  • Azure Functions
  • Azure Event Hubs
  • Azure Synapse
  • Azure Storage
  • Azure Databricks

Choosing the Right Azure IoT Hub SDK

Azure IoT Hub SDKs are available for multiple languages, platforms, protocols, and devices, and should be chosen based on the specific needs of the IoT solution. Best practices for using the SDKs include using the most recent version, following the recommended architecture, using the async programming model, and using secure communication protocols. Message routing, device twins, and device identities and keys should also be used to optimize the performance and security of the solution.

Factors to consider when choosing an Azure IoT Hub SDK

Consider several factors when selecting the right Azure IoT Hub SDK:

  1. Language support: Choose an SDK that supports the programming language you are using for your IoT solution. Azure IoT Hub SDKs are available for C, C#, Java, Node.js, and Python.
  2. Platform support: Consider the platform you are using for your IoT solution. Azure IoT Hub SDKs are available for Windows, Linux, and MacOS.
  3. Protocol support: Choose an SDK that supports the protocol you are using for communication between your IoT devices and the cloud. Azure IoT Hub supports MQTT, AMQP, and HTTP.
  4. Device support: Consider the type of device you are using and choose an SDK that supports the necessary features and capabilities. For example, if you need secure communication, make sure the SDK you choose supports secure communication protocols.
  5. Ecosystem integration: If you are building an IoT solution that integrates with other Azure services or third-party tools, choose an SDK with the necessary integration capabilities.
  6. Ease of use: Consider the learning curve and documentation associated with the SDK. Choose an SDK that is easy to use and has clear documentation to help you get started quickly.
  7. Development environment: Consider the development environment you are using, and choose an SDK that is compatible with your environment.
  8. Performance: Consider the performance needs of your IoT solution and choose an SDK that can meet those needs.
  9. Cost: Consider the cost of the SDK and whether it fits within your budget.

Best practices for using Azure IoT Hub SDKs in different scenarios

Make sure to use the most recent version of the Azure IoT Hub SDKs to take advantage of new features and improvements. Follow the recommended architecture to ensure that your solution is scalable, reliable, and secure. Use the async programming model to use non-blocking code and optimize performance. Secure communication protocols, such as TLS, should be used to protect the data transmitted between devices and the cloud. Device-to-cloud messages should be used for critical data that needs to be stored and analyzed. Cloud-to-device messages can be used to control devices or send commands. Message routing can be used to filter and route messages based on the content of the message or the device that sent it, helping to scale the IoT solution and process messages more efficiently. Device twins allow for managing the state of devices and synchronizing their state with the cloud. Finally, device identities and keys should be used securely and protected from unauthorized access.

Conclusion and Future Directions

The potential of Azure IoT Hub SDKs to drive business value

Azure IoT Hub SDKs offer several potentials to drive business value:

  1. Improve efficiency and productivity: Automate processes, gather real-time data, and enable remote monitoring and control with IoT solutions built using Azure IoT Hub.
  2. Increase customer satisfaction: Better understand and meet customer needs, leading to increased satisfaction and loyalty.
  3. Enhance decision making: Collect and analyze real-time data from IoT devices to make more informed decisions and improve operations.
  4. Improve asset management: Track and monitor the performance and maintenance of assets, leading to improved asset utilization and reduced downtime.
  5. Generate new revenue streams: Offer new products and services, create new business models, and open up new revenue streams with IoT solutions.
  6. Reduce costs: Optimize resource usage, improve maintenance schedules, and automate processes with IoT solutions.
  7. Increase competitiveness: Give businesses a competitive edge by providing them with a way to differentiate themselves and offer unique value to their customers.

Edge computing involves processing data closer to the source of the data, reducing latency, improving performance, and reducing the amount of data sent to the cloud. 5G networks are enabling faster and more reliable IoT connectivity, enabling new use cases and applications. Artificial intelligence and machine learning are being integrated into IoT solutions, making them more intelligent and autonomous, with the ability to learn and adapt. Blockchain technology is being used to improve security, traceability, and trust. The IoT is being applied in healthcare for remote monitoring, diagnostics, and treatment (IoMT). It is also being used in industrial settings, improving efficiency, maintenance, and safety (IIoT). Smart cities are being created with the IoT, enabling improved public services, transportation, and environmental sustainability. Augmented reality and virtual reality are being integrated into IoT applications, creating immersive and interactive experiences.

Closing

Azure IoT Hub Device SDKs are software development kits that enable developers to create applications that run on IoT devices and connect them to Azure IoT Hub. These SDKs provide a set of APIs and libraries that allow devices to perform basic and complex IoT device-to-cloud communications and commands, and support a wide range of programming languages and platforms. Common use cases include sending telemetry data, receiving commands, connecting devices using various protocols, authenticating devices, handling messaging, managing device twins, and receiving notifications.

Azure IoT Edge is a fully managed service that enables developers to deploy cloud intelligence directly onto IoT devices. It consists of edge modules, an IoT Edge runtime, and Azure IoT Hub. This allows developers to create and deploy intelligent edge solutions that bring the power of the cloud to the edge of the network.

This article has presented some of the major SDKs related to IoT and some of the related Azure IoT resources. It brings these resources together to show the array of options available to the Azure IoT developer. If a favorite was left out, please leave a comment. It could be a subject of a future post.

Posted on Leave a comment

Personal Post on My Continuing Journey with IoT Edge Computing

shawn deggans personal blog post

I made the biggest career change of my life recently.

I left the consulting firm I had been employed with for the past four years. It wasn’t an easy decision. I gave up a technical leadership role, I left a lot of people who I loved working with, and I gave up the security of a regular paycheck.

What was behind my decision?

Focus.

Focus was my primary reason for leaving. Two years ago I began a journey to learn and apply everything I would need to know to be a competent IoT Edge Architect. I began that journey with the hopes that my career would be heavily focused on helping organizations solve interesting problems using modern data analytics, IoT systems, and containerized machine learning on the edge.

That never really happened. I had the occasional opportunity to work with machine learning, Kubernetes, and some advanced analytics, but the bulk of interesting data work was done by other people while I focused on platform development.

I didn’t allow those IoT skills to go static though, because I did the occasional side work with partners focused on IoT, but my day job always came first. It reached the point that the day job wouldn’t allow time for anything other than the day job. I didn’t want those IoT skills to go stale, so I had to make the difficult decision. Do I stay where I am and try to be happy or do I pursue the career working with the technology I know actually moves the needle for organizations?

So here I am, completely independent. Ready to focus.

I got pretty good with infrastructure deployment and DevOps, so that’s the majority of the work the firm put me on. And they put me on a lot of it. Systems architecture and design became my everything for a while. Let me be clear that there’s absolutely nothing wrong with systems design work. It can be incredibly rewarding. It’s actually a big part of IoT Edge development. It just became clear to me that I was never going to have the opportunity to help build anything interesting on the infrastructure I was creating.

During my time with the firm, I went from a senior application developer to a cloud systems architect. It took me four years to make it happen, but it was definitely a necessary milestone for the next stage of my journey.

What is the next stage of my journey?

I’m returning my focus to IoT Edge computing.

What I want to do for my career is implement edge and IoT systems from multiple types of systems to multiple cloud and server solutions using modern communication systems, analytics, and security. I mean, for something that’s supposed to be a focus, that’s pretty broad. However, it all fits together nicely for certain, specialized use cases.

I have a lot of experience and a few certifications in Azure, so I have no plans to walk away from Azure any time soon, but I’ve had the opportunity to work with a few other products like InfluxDB, Milesight, Chirpstack, Eclipse Mosquitto, and I don’t want to limit myself to one cloud or one set of solutions. Much of my focus moving forward will be more on the IoT Edge System Design. The theory, the best practices, the understanding of why certain technologies are used over other technologies.

Basically, in order to improve my IoT Edge expertise, I need to say no to a lot of other types of work. Am I capable of helping an organization migrate all their SQL data systems from on-premises to the cloud? Yes, it’s completely within my wheelhouse. Could I build a distributed e-commerce solution using .Net applications in Docker containers for high availability using the best security Azure has to offer? Yes, also within my skillset. Will I take on any of that work? No. I won’t. And that’s the point I’ve reached in my career. I’m going to be very selective about the type of work I take on, and the type of clients who I work with.

That’s my goal. Focus. Be one of the best.

What can you expect from this blog?

It won’t change too much, but I will create more content. I do like doing technical deep dives, so you will see more of these. I also like to explore use cases. You’ll see more of that, especially in the Controlled Environment Agriculture industry. I believe this is an area that will need more help as our environment undergoes more changes in the coming years. If we want food in the future, we will need to know how to control our environments in a way that is economical and sustainable.

I will also write more about IoT architectures for multiple clouds and scenarios. sensors, endpoints, and power management. I want to look at how Claude Shannon’s Information Theory shapes the way we communicate between the cloud and devices. I will probably write far more about networking than you want to read, but it’s an area that I used to hate that I’ve now grown unreasonably in love with. Obviously, lots of discussion around Edge Computing, protocols, Fog computing, Augmented Reality, Machine Learning, MLOPs, DevOps, EdgeOps, and of course security.

That’s the technical side, but I also want to start working more with the community. DFW, Texas has quite the community of IoT organizations and engineers. I hope to connect with these individuals and organizations and capture some of ways I can help them, or they help me and record those outcomes here.

What about money?

Ah, yes. I do actually have bills to pay. So I will need to make money. Luckily, I’m in the financial position that I don’t necessarily need to earn a fulltime income immediately. I’m also fortunate enough to have work from one of my business partners that fits directly within my goals. We’re building an InfluxDB for some time series data and using Pandas to build some forecasting. I’ve also had my own LLC for a few years now, so the business side of things won’t be a hurdle.

But I do have additional plans. Next month I’m presenting a few ways that we can partner together, if you are interesting in working on an IoT Edge project. I’m opening my calendar to a small group of people for bookings through my company called, “Green Nanny LLC.” That’s a name you’ll see me mention more in the future as I build it out to its full intention.

Here are just a few of the services I’ll offer:

  • Assessments – are you ready for IoT Edge? Do you know what it is and how it applies to modern, distributed solutions? What does it look like, operationally? This helps you make better decisions and pick a direction for next steps.
  • Architecture design sessions – let’s think through the art of the possible using machine learning, IoT, and modern data analytics. What does your future system need to look like to support edge workloads?
  • Briefings – if we were to implement a project, what would it take? Do you need a proof-of-value to start or are you ready for a well-architected IoT solution?
  • Implementations- how can I help your team implement an IoT Edge solution?
  • POV – let’s see if we can create a proof of value in a very short period of time?
  • Team training – how can I help your team learn how to implement IoT Edge?
  • Well-architected review and recommendations- can we improve your existing solution? Do you need help with reliability, cost optimization, security, operational excellence, or performance?
  • Managed Service – what are some options for managing your IoT products using a managed services provider? I don’t provide this service myself, but I still have many connections who can help make this a reality.
  • Retainer – Do you just need someone with a lot of IoT Edge knowledge to bounce questions off of? Not sure where you want to start on your own IoT Edge journey, but would like a guide?

I’m excited for the future

I think and feel that I made the right decision for this move at the right time. My skills as an Azure Architect have put me in an excellent position to transition into something more specialized. The consulting work I did with the firm clarified the type of work I like to do, and the type of work that I’m good at. I see a lot of promise in my area of focus and a lot of financial opportunity for myself, my partners, and the clients who I will serve.

Posted on Leave a comment

What is Edge Analytics


When it comes to Internet of Things, much of the focus is on the “Things.” Devices are the lead guitarists and singers of the IoT world. After all, devices are what we interact with. We wear them, we use them as little handheld computers, and they make normally dumb objects smarter.

When you’re on your learning journey as a newbie to the IoT world these tend to be your real focus. You buy an Arduino or a Raspberry PI and you get to hacking. It’s not until you mature on your journey that you realize the value of the data and understand what analytics can do for your solutions.

Now you have devices gathering telemetry and you have smart devices making intelligent decisions closer to the places where it matters.

Azure IoT Edge has been the ideal tool for implementing Edge Analytics for a few years now. Azure provides an entire architecture to help gather analytics data, process the data, and turn it into intelligent code to push down to Edge compatible devices, like a Raspberry PI.

To put edge analytics into context with most data analysis and processing systems, it’s important to consider the earlier phases of IoT. Even before the popularity of cloud computing there was IoT. In fact, manufacturers have been gathering telemetry data from machines for more than forty years. It wasn’t called IoT back then, because there really wasn’t our version of the internet back then. Most telemetry was gathered using wired systems and the data was stored directly onto servers that sat on the factory floor.

As IoT evolved it moved from this client server model to a client cloud model. In both of these cases, discovering insights in the data required the data processing to be managed far from the device. Devices just stupidly sent telemetry and any intelligence was either done at the server or the cloud. Automated control systems managed working directly with the operator or machinery.

Edge analytics allows for us to shorten that cycle. By pushing intelligence down to the device, we now have the option to react to telemetry as it’s gathered from the source. This allows a machine to shut itself off without needing a connection to a server or the cloud. Working with the data directly on the device provides a few other benefits as well, like reduced data transfer to the cloud, greater security for sensitive data, and more automated behaviors from devices.

The primary purpose of collecting any type of data is to derive actionable insights from that data. Azure provides a number of tools for accomplishing these tasks, but there are a few things it’s important to understand if you want to make the most of your data and your overall IoT systems.

Standard IoT Solutions and Edge IoT Solutions

I touched on the evolution of IoT solutions earlier in this article, but I want to make it clear that you rarely jump directly into an edge solution architecture. You normally start with the foundation for an edge solution, but like any good modern development process, you take incremental steps to arrive at your solution.

An early phase of capturing data and processing that data in the cloud will look more like our old client server model. Devices send telemetry data to the cloud where it’s captured in a data storage resource like Azure Data Lake. Data engineers and machine learning engineers use the data to derive insights (typically using time-series forecasting) and produce either static code or ML models to make the data actionable. A cloud service like Event Grid is called from the code to take some type of action like sending an alert to a machine operator or firing off some type of automated system.

This is an acceptable model when you’re working with very few devices and only a handful of models.

But what happens when your have hundreds or thousands of devices and models? Azure can definitely handle the data load, but do you want to shoulder that expense? And what about delays that are inherent to all distributed systems? Can you afford for your machine operator to not receive an important maintenance alert on a machine?

As your evolve you will eventually move from the traditional IoT platform to an IoT edge solution, like Azure IoT Edge.

Azure IoT Edge Architecture

To deploy and develop edge analytics solutions you’ll need an architecture designed for edge analytics. Azure provides two primary IoT platforms for Azure IoT Edge.

Azure IoT Central is designed to be the perfect place to begin your IoT journey. And Azure IoT Hub, is the next step on your Azure IoT path when you have a more mature IoT practice and many more devices to manage.

The typical architecture for edge IoT solutions is an edge device, and edge gateway, and the cloud IoT solution. Deployment and device management is typically handled at the cloud, but because you can now perform edge device chaining, it is possible to accomplish some of these tasks from parent edge devices.

As a developer of IoT edge solutions, there are a handful tools you’ll want to understand. I won’t go into detail, but containers, Azure IoT Hub or Central Client SDKs, and Device Twins are a good place to start.

Key Points to Remember

  • Edge analytics allow you to improve security, reduce latency, and save on cloud costs
  • Edge analytics development is typically an incremental development process that evolves from traditional IoT solutions to edge solutions
  • Edge analytics requires a specific type of architecture and Azure provides his in two different platforms; Azure IoT Central for beginning your IoT journey and Azure IoT Hub for more practiced IoT development teams
Posted on Leave a comment

The Smart Factory for Brewers isn’t Just for the Big Brewers; is Your Independent Brewery Ready for IoT, AI, and the Cloud?

Wouldn’t it be great if every beer bottle that left your brewery met with your exacting quality standards, if overflowing and excessive foaming were yesterday’s problem, and not a persistent, wasteful expense?

What if you could be consistently proactive in your quality control process, and not reactive to cases of returned beer and disappointed customers?

Independent breweries like Sugar Creek Brewing faced just such a problem, and according to this Forbes article, the problem cost the brewery $30,000 a month.

Independent, hand-crafted breweries are discovering what manufacturers have understood for decades. You can measure product quality with IoT. But many breweries, like Sugar Creek Brewing, are using AI to predict quality issues before they happen allowing brewers to proactively manage the quality of their beer.

The Smart Factory for the Independent Brewery

The smart factory for breweries is here. Connecting your brewery with IoT devices capable of streaming real-time data to the cloud, analyzing that data with AI, and creating a responsive feedback loop that your employees can use to consistently improve the brewing process is no longer a tool meant only for the larger brewers with immense IT teams and budgets.

To help pinpoint the specific causes of overflow and excessive foaming during the brewing and bottling processes, you can connect your equipment with embedded flow meters and sensors that allow you to collect a high volume of accurate data on pressure, temperature, pH balance, carbonation, fill time and more.

By collecting all this data and applying the correct analytics, you can discover insights about your operations, improve efficiency, stop equipment bottlenecks, and help your employees accomplish their daily tasks.

Working around high-pressure tanks presents big risks to employees working on the production line. With the right pressure sensors on every kettle, still, brewing pump, and with object detection in place, the factory floor can become a much safer place to work, as production managers can track performance issues that could lead to serious injuries.

The fermentation process is the heart of the brewery factory. What if you could get constant feedback on this process without opening that tank?

Breweries See a Return on their Investment with AI and IoT

Not only does the Smart Factory for Breweries improve efficiency, beer quality, and employee safety, but it also helps improve your brewery’s financials.

  • Save on returns from poor quality products
  • Reduce waste of materials for bad batches
  • Fully use capital equipment
  • Protect the health and safety of employees and customers
  • Reduce energy use
  • Monitor and track batches effectively for compliance and regulations
  • Save on parts and repairs using AI predictive maintenance
  • Control inventory with RFID tracking

The cloud has allowed the cost of analytics systems to drop significantly, and many IoT platforms can right-size their solutions to fit with your consumption, giving you the option to ease into your IoT and AI journey affordably.

Connecting KPIs as a Measure of Success

As a brewer, you’ll want goals and specific metrics tied to your Smart Factory for Breweries. Some of these include the following depending on your solution approach:

  • Equipment downtime for repairs
  • Predictive cleaning schedule vs set schedule
  • Returns from retailers because of quality
  • Bbl brewed per hour
  • Bbl packaged per hour
  • Bbl lost per hour
  • Bbl to energy costs

Your Business Processes will Need to Adapt

There are daily processes undertaken in the brewery that add to waste and risk contamination. The typical small brewery will draw liquid from the tanks daily to check for quality, consistency, and the state of the fermentation process. This is the process of monitoring specific gravity. It’s not a great process when managed manually, because only checking once or twice a day isn’t likely to catch a problem quickly enough, and with each test the product is exposed to possible contamination.

To reduce the risk of contamination and increase regular monitoring, it makes sense to add a sensor to the tank to collect data. The data collected from the tank can then inform operations.

But how is this continuous monitoring done on large tanks that are usually made of copper or industrial steel? This is where it helps to understand the problem and find the right IoT device for the job. In this case, low energy Bluetooth devices floated in the liquid can communicate with gateways positioned nearby. This allows IoT to completely replace a manual process, as well as do a better job of monitoring for regular consistency.

This is why it’s important to work directly with a partner who can help match the right product with the situation. A great partner will learn your business processes and won’t be afraid to point out where your business process will need to adapt to fit with the innovation the technology brings.

A Use Case: Automate the Manual Method of Measuring Specific Gravity

Brewers are well aware of how this process works, but for this use case I’ll summarize it:

Specific gravity will indicate the stage of fermentation. Brewers use this measurement to determine the state of fermentation—is it on track, done, or perhaps stuck. Yeast can stop consuming sugar too early, and when this happens the brewer needs to take steps to save the batch. This is one of the reasons continuous monitoring is required for the modern brewery. The sooner you’re aware of a stalled batch, the sooner you can undo the damage.

This is typically measured using the manual method described above. The actual measurement is looking at the liquid’s density in comparison to water. As beer ferments and coverts sugars to alcohol and gas, the specific gravity falls and the liquid gets less dense. Eventually, there are few sugars left and the fermentation process slowly stops.

It’s clear to see why putting a machine in charge of monitoring this process consistently helps improve this task.

But what does this look like, technically? Are there tools and systems in place today to make this easy to install and monitor without risking batches of beer or installing expensive equipment?

Tilt Pro, Bluetooth, and IoT Central

If we focus on one particular use case, like measuring specific gravity, there are a number of ways to accomplish this, but the following architecture is one that I recommend. This architecture is built for a small brewery with just a few fermentation tanks. It uses off-the-shelf devices and requires minimum programming to get the solution up-and-running. But it also has room to grow into a robust platform for managing more than just fermentation monitoring.

FERMENTATION TANKS

The assumption is that you’re using industry standard fermentation tanks. These are, without a doubt, a challenge for electronics equipment. Most are made of steel or copper, which means that they basically act like a Faraday Shield capable of blocking or dampening signals.

Some IoT devices will not work in this situation. So either you need a wired solution or something clever like the Tilt Hydrometer. When I first discovered this device I thought that perhaps it was a toy for homebrewers. Surely, nothing as low in price as this device could have professional applications, right? Well, after reviewing the device and seeing it’s capabilities on paper, I’m willing to include it in this architecture.

Especially, backed up by a custom device to work with the iBeacon signal generated from the Tilt. For this architecture, I’m recommending the Tilt Pro. The standard Tilt device might be adequate, but I think it’s worth the extra money for the added battery life and sensor capabilities.

The Tilt device goes inside the tank, but it’s enclosed in a protective shell. Cleaning and care instructions are available on their website. Here’s the marketing breakdown on the device:

At 3x the volume and weight of our original Tilt hydrometer, the Tilt Pro’s larger size allows us to more than double the battery life with preinstalled Energizer® AA lithium batteries that will keep it powered for 3 to 5 years depending on use. So much time you may forget your Tilt actually runs on a battery! The extra size and weight comes with an extra bonus: improved specific gravity stability at high krausen so you will see a steadier read-out on your iPhone, Android, or Raspberry Pi. In addition, we have boosted range with a high-gain antenna that in testing increased range by 20% vs. our original Tilt. Now you can see your Tilt through thicker heads of foam and heavier stainless steel walls. To top it off we’ve included the extra decimal of precision available from the sensors and to the read-out so there are now decimal degrees Fahrenheit as well as higher resolution Celsius. Specific gravity now reads out to the 10,000th place (for example 1.0000). You can now see fermentation activity dropping less than a brewer’s point per day. In all we’ve made several adjustments we believe the craft beer professional (and serious home brewer) is sure to appreciate.

The Tilt’s cloud backend is basically writing data to Google Sheets. We definitely don’t want to write our data to Google Sheets. No offense to Google Sheets. It’s a great app for many purposes, but we want something a bit more robust. I also do not want a mobile app. I want a device dedicated to capturing the Bluetooth signal, converting the raw message from that signal to MQTT, and delivering that to a server on the Brewery’s internal network.

DATA FLOW FROM TANKS TO EDGE GATEWAY

The basic flow of data from tank to IoT gateway looks like the following:

The Tilt is a one-way leaf device and it’s tightly coupled to a nearby Raspberry Pi Zero W. By nearby, I mean as close to the tank as possible, other than inside of it.

The Pi gives us a lot of development options. We could, if we wanted to, connect the Pi directly to Azure IoT Central. This would make it a gateway device. It would still need a WiFi router or hub to connect to, but it could manage two way communication with IoT Central.

For this architecture, though, I want to couple a Pi with each Tilt. Considering that the cost of each device is minimal and the power you get when you add a more robust gateway, like the Ectron Edge Computer ECT-ECI, this allows us to remain technically small and simple, but with the option of applying more sophisticated Edge Patterns.

For instance, there might come a future date when I want to I want to deploy machine learning modules to the Pi. These modules could be specific to each tank’s beer, allowing brewers the ability to monitor for different types of beer.

THE ARCHITECTURE

The following is our more robust design.

I want to continue with the idea that this is just a proof-of-concept set with one use case. However, I’ve expanded out to manage three different tanks. Each tank as been coupled with its own Tilt device, Raspberry Pi Zero W, and all three of these connect to the Ectron Edge Computer.

Where we start to flesh out the solution, is when we collect the data in the cloud. Here we’re using Azure’s IoT Central, but we could just as easily use any other cloud IoT platform. I like IoT Central because it’s quick to get an app up-and-running, it makes device management simple, and includes all the analytics you need to prove out an IoT POC.

It’s also scalable, so building out to more devices and more use cases is easy. IoT Central is meant for smaller to mid-sized IoT projects, but it can still support multiple organizations in production scenarios.

When you start to work with more than one million devices, though, IoT Hub, AWS IoT SiteWise, or Google’s IoT Core are better choices. Though I designed this with Azure in mind, the Gateway is flexible enough to work with any cloud backend IoT service.

A couple of future use cases we could explore once this platform is built:

  • Use Machine Learning to predict fermentation stages for specific beers
  • Use time series analysis to detect anomalies in the brewing process
  • Collect data specific to certain recipes, build guidelines and intelligent monitoring around these recipes, and alert Brew Masters to tanks that drift outside the confines of the established guidelines

Who do I need to put this solution into place?

The team to build this out could be relatively small. One person with the right skills could build this entire solution, but what skills will they need?

  1. The Tilt devices are relatively complete. As an off-the-shelf solution they provide a lot of flexibility. To work with this device, you’ll need to understand how to collect Bluetooth Beacon data. Lucky for us, Tilt includes a link to work with this data from their website’s FAQ. https://kvurd.com/blog/tilt-hydrometer-ibeacon-data-format/
  2. Creating the coupled Pi Zero will pose a slightly bigger challenge. This isn’t an out-of-the-box solution, but it isn’t undiscovered country either. Basically, we want to run Containerd and the IoT Edge Modules on the device. This will allow us to deploy to the Pi as though it were an Edge Device. We could also use the IoT Central SDKs to create a basic device and send data. We have a few choices here but ideally, you want to capture the BLE message, format it to something that best fits the use case, and then wrap that in an MQTT message to send to IoT Central.
  3. The Edge Gateway is probably one of the easier pieces to implement. This article covers the process, so I won’t dive into it here: https://docs.microsoft.com/en-us/azure/iot-edge/how-to-connect-downstream-iot-edge-device?view=iotedge-2020-11&tabs=azure-portal
  4. Setting up the Edge Gateway and devices in IoT Central, is also trivial, but that might be a different person completing the set up than the device developer.

Basically, if you have someone with experience developing IoT Edge modules, lite Python programming skills, and IoT Central administration basics, you could have a solution up-and-running quickly.

Is this Secure?

If you keep the signals within the brewery on their own dedicated network and authenticate the devices through IoT Central, it should be secure. The real trick here is to make sure you provide a separate network on your brewery floor from the rest of the company’s network. For instance, many breweries are also open to the public for dining and tours. You don’t want the public to even be able to see the brewery floor network, much less access it from their devices.

In this scenario, the endpoints for IoT Central are the only exposed URLs. You can protect these with device authentication. Logging into IoT Central as an administrator would be protected by Azure Active Directory using MFA.

What about my data? Is it backed up?

IoT Central only stores about 30 days worth of data, but there are means of integrating storage solutions with your IoT Central App. This isn’t reflected in this architecture, but at a minimum I would recommend adding an Azure Storage Account to retain messages. You’ll likely want this data for future data science and trend analysis.

What if I have more than one brewery?

This solution isn’t designed for extreme robustness or resilience. I definitely put this solution in the category of proof-of-concept or minimal viable product. If you set this up, and you like your initial tests, and you feel like this is something that could expand to multiple physical locations, IoT Central can definitely handle this up to 1 million devices. If your device requirements look like they will spread beyond that number, it’s time to consider Azure’s IoT Hub solution with a more complete Azure Analytics backend. Synapse or Databricks with Azure Time Series Insights make great additions to this type of architecture.

How do I manage this environment

As long as this is kept small, it’s probably ideal for a single developer or administrator. IoT Central has a few built in roles that the administrator or App owner can assign to users so they have access to dashboards. At a minimal, I would consider these roles in IoT Central:

  • Application Administrator can manage and control every part of the application – give this role to someone in your IT operations department
  • App builder has many of the same abilities as the Admin, with the exception that they can’t make administration changes or connect to data exports
  • The Operator role is ideal for someone who needs to monitor device health and status. This is for anyone working the brewery floor

Those are basic roles. If you begin to build your solution out to multiple locations, these can be defined as Organizations. If I own a brewery in three different cities in my state, I might have my Fort Worth brewery, my Dallas brewery, and my Austin brewery. I would separate those by Organization and assign the above roles to users in those areas.

What can I do with this platform, once it’s up-and-running?

IoT Central comes with many tools needed to run operational IoT.

  • Robust Dashboards that can be customized for many different views
  • Device life-cycle management and Gateway management
  • Rules triggered based on telemetry with built in messaging to email, webhook, Azure Monitor Action Groups, Microsoft Power Automate, and Microsoft Azure Logic Apps
  • Time-series Analytics allows you to create and save queries
  • Run jobs on device groups that can send commands to devices or change device properties
  • Build and store device templates
  • Export to multiple data sources
  • Complete administration of the app, the users, and pricing

In addition to the IoT Central benefits, you can build upon the IoT Edge devices by integrating device specific logic or machine learning models.

If I have an IT team, will they need a lot of training?

Perhaps, if they aren’t familiar with IoT. However, Azure does make some of this fairly easy. It helps to work with a partner who understands how to make cloud solutions operational. Knowing and understanding the burdens and responsibilities of a lights-on operations team will help your partner understand what things the team needs to monitor for and integrate into your team’s existing playbooks and operations procedures.

I’m interested to see how IoT can help my brewery

If you aren’t already using IoT in your brewery, it’s important to understand that there is a learning curve. As I’ve outlined here, it takes some time to understand how the technology can help you and how you might need to adapt to the technology. If you have the right partner, this can be a smooth transition. There are definitely some key stages I think a partner should walk you through before you make a large financial commitment.

  1. Be careful with any solution that appears to be a one-size fits all. Your brewery is unique. Your product is unique. You didn’t become an independent, craft brewer because you wanted to mimic a certain company with the initials of AB. So AB’s amazing, multi-million dollar IT solution probably isn’t right for you.
  2. This isn’t a quick, one-and-done project. IoT is a journey best taken in measured steps. The right partner is going to understand that process and they will help guide you along the path.
  3. The right partner takes a wholistic view of you, your brewery, and cares about your bigger picture goals. Selling gadgets is great, if all you do is sell gadgets, but a solutions partner wants to make sure you have the right gadget for your needs.

Solution Visioning

Expect a couple of workshops from your solution partner. Believe it or not, these aren’t just a way to make more money off of you. These are a necessary part of the process. A partner needs to learn about you, your business, your aspirations and your challenges. These envisioning workshops are a way to achieve that.

Architectural Understanding

A good partner will want to understand your network topology, but also your enterprise architecture. Not just the technical systems, but your policies, procedures, your team structure, and even your company culture.

POCs and Pilots

Smaller, measured steps into a bigger picture solution is a great way to learn, without breaking the bank. The days of huge, multi-year projects only to deliver a technical nightmare that doesn’t meet your needs is gone. Be ready to work with a partner willing to take small steps toward bigger goals.

  • Limit your use case to one single point of value
  • Limit your scope to one particular area of the brewery
  • Time-box your efforts. Don’t make the mistake of trying to enforce an artificial deadline, but don’t get mired in complexity.
  • A POC won’t be production ready, but it will be testable with real-world scenarios
  • Take this opportunity to learn and try to avoid assumptions
  • It might require multiple POCs to arrive at a worthwhile pilot
  • It will require your time, the time of your staff, and your feedback—you can’t hand off your standard operating procedures and expect a bespoke solution to your particular needs

How do we make this production ready?

When you’re prepared to take this solution to production, there are a few factors to consider.

  1. How do we reach zero-trust security with this platform? Are our users authenticating using MFA. Does each user have the least privileges possible? Are my devices hardened against tampering? Are my Edge devices loaded with security modules?
  2. Do I have system oversight? Is there the appropriate level of logging and log monitoring in place? Are there alert groups established in Azure. Are these alert groups receiving alerts from IoT Central, devices, and any other backend analytics systems?
  3. Is data backed up securely? Is access to this data limited to integrated systems? Is access to the data monitored?

There are other considerations like building out staging layers for developers and testers. A more complete security monitoring tool like Microsoft Defender for Cloud should be a consideration. Hiring penetration testers to test your security is also a good idea.

Teaching your team how the system works

Hopefully your team has been involved from the beginning. A good IoT integration partner will talk to the brewery floor crew during development and system installation, get their feedback for alerts and dashboards, and familiarize everyone with the systems as it’s built.

Good formal training for the brewer floor will be something a partner can provide. Delivering this training in documentation, video, and one-on-one sessions is the typical method for handing off a production system. As your system grows and matures, you’ll eventually want staff trainers who have made your technical solutions part of your floor’s standard operating procedures.

What if I need continued help after the solution is built?

Let your partner supplement your time with ongoing support and monitoring. Many IoT developers and integrators either have a managed service team or they outsource that job to a trusted partner. If you don’t have the IT staff to manage and monitor your solution, your IoT partner should be able to provide one for you at an affordable rate.

Who do I include in partner meetings?

Independent brewers tend to keep the company lean. Titles like CEO, CTO, and CFO might not exist. That’s not a problem. The people who need to be at the table are decision makers, financiers, brewery operations experts, and trusted advisers.

Are you available to help us?

Yes, as a certified Azure IoT specialist and AI Engineer with twenty years of IT experience, I’m in the ideal place to help with this exact type of solution. I’m a strong advocate of cloud transformation. I’ve helped large and small organizations transform their operations using Azure and Big Data analytics platforms. I’d love the opportunity to work with a brewery interested in exploring how IoT could improve their operations and save them thousands of dollars a month.

What are the next steps?

Are you happy with your current brewery floor operations? Now that you know there are better ways to brew beer, can you continue to operate the same way moving forward? Inertia is difficult to break, but one thing that helps is a clear decision. Is a Smart Factory for Breweries something you’re ready to take on? Or can you continue on your present path?

When it comes to these type of decisions, it’s usually best to start small. This is one of the reasons I’m an advocate of the POC. With a small equipment investment, an Azure subscription that only charges based on consumption, and a hands-on consultant who can take you through every step of the process, it’s easier now than ever before to test your ideas without breaking the bank.

The cloud, IoT, and AI are tools that brewers use to improve operations, innovate, and save money. If you’re interested in seeing if this is the right path for your brewery, I’m here to help.

Posted on Leave a comment

I Read and Summarized the 3rd Edition of IoT Signals So You Don’t Have to

A report built based on survey data taken by the company named, ‘hypothesis’. This is a combined effort between Microsoft and Hypothesis to capture the most current state of IoT from the view of business leaders in certain sectors; manufacturing, energy, mobility, and smart places.

The survey was multi-national and the report includes data captured from in-depth interviews.

Things to know about IoT in 2021

The following are high-level conclusions drawn from the interviews and survey data:

  • IoT continues to drive organizations toward a more productive future
  • COVID-19 has accelerated IoT strategies and fueled business growth
  • AI, Edge Computing, and Digital Twins are essential to advance IoT strategies
  • Although IoT projects are maturing, technological complexity persists
  • Organizations are keeping a close eye on data security

Who they talked to

  • Business decision makers, developers, and IT decision makers who work at enterprise-size companies with greater than 1k employees
    • 71% were familiar with IoT
    • 95% of those familiar, have influence and decision making power on IoT strategies
      • 10% of those familiar were not in adoption of IoT
      • 90% of those familiar were in adoption of IoT

Overall Research Learnings

Big Picture

This year, IoT continues to be widely adopted. 90% of organizations surveyed are IoT adopters. IoT projects can be categorized into four stages:

  • Learn
  • Trial / Proof of Concept
  • Purchase
  • Use

Of the 90%, at least 82% have a project that reached the “use” stage.

The state of most projects overall:

  • 29% in Learn
  • 25% in Trial / POC
  • 22% in Purchase
  • 25% in Use

IoT adoption and value globally (Australia, Italy, and the US lead as primary adopters)

  • 90% of the surveyed leaders in countries fitting criteria are adopters
  • 25% have projects in use
  • Average time to reach “use” is 12 months
  • 66% plan to use IoT more in the next 2 years

IoT Adoption and Value by Industry

  • Manufacturing
    • 91% of the surveyed leaders in countries fitting criteria are adopters
    • 26% have projects in use
    • Average time to reach “use” is 13 months
    • 68% plan to use IoT more in the next 2 years
  • Energy
    • 85% of the surveyed leaders in countries fitting criteria are adopters
    • 22% have projects in use
    • Average time to reach “use” is 15 months
    • 61% plan to use IoT more in the next 2 years
  • Mobility
    • 91% of the surveyed leaders in countries fitting criteria are adopters
    • 23% have projects in use
    • Average time to reach “use” is 14 months
    • 61% plan to use IoT more in the next 2 years
  • Smart Places
    • 94% of the surveyed leaders in countries fitting criteria are adopters
    • 24% have projects in use
    • Average time to reach “use” is 13 months
    • 69% plan to use IoT more in the next 2 years

Why Adopt IoT

Overall Top 5 reasons:

  • Quality Assurance : 43%
  • Cloud Security: 42%
  • Device and Asset Security: 40%
  • Operations Optimization: 40%
  • Employee Productivity: 35%

The report includes evidence that those companies who employ IoT to improve products and service see a higher increase in overall ROI.

Manufacturing Top 5:

  • Quality and compliance
  • Industrial automation
  • Production flow monitoring
  • Production planning and scheduling
  • Supply chain and logistics

Energy (Power and Utilities) Top 5:

  • Smart grid automation
  • Grid asset maintenance
  • Remote infrastructure maintenance
  • Smart metering
  • Workplace safety

Energy (Oil and Gas)

  • Workplace safety
  • Employee safety
  • Remote infrastructure maintenance
  • Emissions monitoring and reduction
  • Asset and predictive maintenance

Mobility

  • Inventory tracking and warehousing
  • Manufacturing operation efficiency
  • Surveillance and safety
  • Remote commands
  • Fleet management

Smart places

  • Productivity enablement and workplace analysis
  • Building safety
  • Predictive maintenance
  • Regulations and compliance management
  • Space management and optimization

Benefits of IoT

Top 5 benefits organizations are reaping from IoT

  • Increases in efficiency of operations
  • Improved safety conditions
  • Allows employees to be more productive
  • Allows for better optimization of tools and equipment
  • Reduces chance for human error

Common measures of success in IoT

  • Quality
  • Security
  • Production Efficiency
  • Reliability
  • Cost efficiency

Less common measures of success

  • More informed decision making
  • Direct impact on increased revenue
  • Sustainability
  • % of project deployed using IoT

Challenges of IoT

Top 5

  • Still implementing our current solution
  • Security risk isn’t worth it
  • Want to work out existing and future challenges before adding or using IoT more
  • Too complex to implement because of technology demands
  • Too complex to implement because of business transformation needed

Top 5 reasons POCs fail

  • High cost of scaling
  • Lack of necessary technology
  • Pilots demonstrate unclear business value or ROI
  • Too many platforms to test
  • Pilot takes too long to deploy

Top 5 security concerns

  • Ensuring data privacy
  • Ensuring network-level security
  • Security endpoints for each IoT device
  • Tracking and managing each IoT device
  • Making sure all existing software is updated

The report includes a section on best practices, and notes that despite security being a big concern, very few are implementing these best practices:

Top 5 best practices

  • Assume breaches at every level of IoT project
  • Analyze dataflows for anomalies and breaches
  • Define trust boundaries between components
  • Implement least privileged access
  • Monitoring 3rd party dependencies for common vulnerabilities

IoT Implementation Strategy

Most of the companies surveyed prefer to work with outsourced resources to implement their IoT strategy. They do prefer bespoke solutions.

Those who outsource see these positive benefits:

  • Increases efficiency of operations
  • Improves safety conditions
  • Reduces changes for human error

Those who do not outsource tend to hit these challenges:

  • Too complex to implement because of business transformation needed
  • Too long to implement
  • No buy-in from senior leadership

Sustainability

Companies with more near-term zero carbon footprint goals are more motivated to implement IoT to help than those who have a longer range target.

Impact of COVID-19

When asked if C-19 was an influence in investing:

  • 44% more investment
  • 41% stay the same
  • 7% less
  • 4% too early to tell

Emerging Technologies

Those who are adopting IoT are more likely to adopt other innovative technology associated with IoT:

  • Digital Twins
  • Edge Computing
  • AI at the Edge

This is collectively known as AI Edge

AI Implementation

84% Have a strategy:

  • 31% are implementing
  • 26% developed, but not implemented
  • 26% developing

16% do not have a strategy

  • 11% want to develop
  • 5% have no plans

79% of respondents claim that AI is a core or secondary component of their overall IoT strategy

Top 5 reasons for AI in IoT Adoption:

  • Predictive maintenance
  • Prescriptive maintenance
  • User experience
  • Visual image recognition and interpretation
  • Natural language recognition and processing

Top 5 Barriers to using AI within IoT

  • Too complex to scale
  • Lack of infrastructure
  • Lack of technical knowledge
  • Implementing AI would be too complex
  • Lack of trained personnel

AI Adoption and Value by Industry

Total:

  • 84% – have an AI strategy
  • 31% – Implementing
  • 26% – developed
  • 26% – developing
  • 79% – use AI in IoT solution

Manufacturing:

  • 84% – have an AI strategy
  • 31% – Implementing
  • 23% – developed
  • 30% – developing
  • 83% – use AI in IoT solution

Energy

  • 90% – have an AI strategy
  • 26% – Implementing
  • 28% – developed
  • 36% – developing
  • 89% – use AI in IoT solution

Mobility

  • 81% – have an AI strategy
  • 36% – Implementing
  • 25% – developed
  • 20% – developing
  • 85% – use AI in IoT solution

Smart Places

  • 88% – have an AI strategy
  • 39% – Implementing
  • 28% – developed
  • 21% – developing
  • 75% – use AI in IoT solution

Edge Computing

Edge Computing Implementation Progress

79% have a strategy:

  • 29% implementing
  • 26% developed but not implemented
  • 24% developing

21% do not have a strategy:

  • 15% want to develop
  • 6% have not plans

81% Edge Computing as a core or secondary component:

  • 42% Core
  • 39% Secondary
  • 18% Considering, not yet adopted
  • 1% not considering

Top 5 Reasons to adopt Edge Computing

  • Cloud security
  • Device and asset security
  • Quality assurance
  • Securing the physical environment
  • Operations Optimization

Top 5 barriers to adoption

  • Lack of architectural guidance
  • Lack of trained personnel
  • Lack of infrastructure
  • Difficulty managing security
  • Lack of clarity on edge hardware choices

Edge Computing Adoption and Value by Industry

Total:

  • 79% – Have Edge Computing strategy
  • 29% – implementing
  • 26% – developed
  • 24% – developing
  • 81% – Use Edge Computing in IoT Solutions

Manufacturing:

  • 83% – Have Edge Computing strategy
  • 37% – implementing
  • 28% – developed
  • 18% – developing
  • 77% – Use Edge Computing in IoT Solutions

Energy:

  • 85% – Have Edge Computing strategy
  • 38% – implementing
  • 25% – developed
  • 23% – developing
  • 85% – Use Edge Computing in IoT Solutions

Mobility:

  • 85% – Have Edge Computing strategy
  • 18% – implementing
  • 30% – developed
  • 37% – developing
  • 88% – Use Edge Computing in IoT Solutions

Smart Places:

  • 85% – Have Edge Computing strategy
  • 29% – implementing
  • 26% – developed
  • 30% – developing
  • 83% – Use Edge Computing in IoT Solutions

Digital Twins

77% have a strategy:

  • 24% implementing
  • 29% developed, but not implemented
  • 24% developing

23% do not have a strategy:

  • 14% want to develop
  • 9% have no plans

81% Use and Impact of DT on IoT Solutions

  • 41% feature as core component
  • 40% feature as secondary component
  • 18% considering, but not yet adopted
  • 1% not considering

Top 5 benefits of using DT within IoT:

  • Improve overall quality
  • Increase revenue
  • Reduce operations costs
  • Enhance warranty cost and services
  • Reduce time to market for a new product

Top 5 barriers:

  • Challenges managing the value of data collected
  • Complexity of systems needed to handle digital twins
  • Integration challenges
  • Lack of trained personnel
  • Challenges modeling the environment

Digital Twins Adoption and Value by Industry

Total:

  • 77% – have a DT strategy
  • 24% – implementing
  • 29% – developed
  • 24% – developing
  • 81% – use DT in IoT solution

Manufacturing:

  • 79% – have a DT strategy
  • 31% – implementing
  • 25% – developed
  • 23% – developing
  • 86% – use DT in IoT solution

Energy:

  • 79% – have a DT strategy
  • 26% – implementing
  • 29% – developed
  • 24% – developing
  • 82% – use DT in IoT solution

Mobility:

  • 76% – have a DT strategy
  • 15% – implementing
  • 39% – developed
  • 23% – developing
  • 77% – use DT in IoT solution

Smart Places

  • 82% – have a DT strategy
  • 27% – implementing
  • 35% – developed
  • 22% – developing
  • 85% – use DT in IoT solution

By Industry

Smart Places

94% IoT Adopters

  • 27% – learn
  • 25% – POC
  • 25% – Purchase
  • 24% – Use

Top Benefits:

  • Increase the efficiency of operations
  • Improves safety conditions
  • Allows for better optimization of tools and equipment

Top 5 reasons for adoption:

  • Productivity enablement
  • Building safety
  • Predictive maintenance
  • Space management and optimization
  • Regulation and compliance management

Top 5 challenges

  • Still implementing current solution
  • Security risk isn’t worth it
  • Too complex to implement because of the need for business transformation
  • Want to work out exiting and future challenges before adding or using
  • Too complex to implement because of technology demands

Manufacturing

91% IoT Adopters

  • 27% – Learn
  • 26% – POC
  • 21% – Purchase
  • 26% – Use

Top Benefits

  • Increases the efficiency of operations
  • Increases production capacity
  • Reduces chance for human error

Top 5 Reasons for Adoption:

  • Quality and compliance
  • Industrial automation
  • Production flow monitoring
  • Production planning and scheduling
  • Supply chain and logistics

Top 5 Challenges to using IoT more

  • Still implementing current solution
  • Too complex to implement because of technology demands
  • Security risk isn’t worth it
  • Want to work out challenges before adding or using IoT more
  • Don’t have human resources to implement or manage

Mobility

91% IoT Adopters:

  • 30% – Learn
  • 26% – POC
  • 21% – Purchase
  • 23% – Use

Top benefits of IoT:

  • Increase efficiency of operations
  • Allows employees to be more productive
  • Improves safety conditions and increases production capacity

Top 5 Reasons for Adoption

  • Inventory tracking and warehousing
  • Manufacturing operations efficiency
  • Surveillance and safety
  • Remote commands
  • Fleet management

Top 5 challenges to using IoT More:

  • Want to work out challenges before adding or using IoT more
  • Too complex to implement because of technology demands
  • Still implementing our current solutions
  • Security risk isn’t worth it
  • Too complex to implement because of business transformation needed

Energy

80% IoT Adopters (Power and Utilities)

  • 28% – Learn
  • 26% – POC
  • 23% – Purchase
  • 23% – Use

Top Benefits

  • Increase the efficiency of operations
  • Increases production capacity
  • Allows employees to be more productive

Top 5 reasons for adoption:

  • Smart grid automation
  • Grid asset maintenance
  • Remote infrastructure maintenance
  • Smart metering
  • Workplace safety

Top Challenges:

  • Too complex because of technology demands
  • Security risk isn’t worth it
  • don’t have human resources to implement and manage

94% IoT Adopters (Oil & Gas)

  • 28% – Learn
  • 27% – POC
  • 24% – Purchase
  • 20% – Use

Top Benefits

  • Increase customer satisfaction
  • Improves business decision-making
  • Increases production capacity and the efficiency of operations

Top 5 reasons for adoption:

  • Workplace safety
  • Employee satisfaction
  • Remote infrastructure maintenance
  • Emissions monitoring and reduciton
  • Asset and predictive maintenance

Top challenges:

  • Lack of technical knowledge
  • Don’t know enough
  • Too complex to implement because of business transformation needed

Final Thoughts

Things worth noting:

  • IoT is not going away, in fact more money, time, and investment goes into it each year
  • Most organizations are looking to add AI, Edge Computing, and Digital Twins to their solutions
  • May organizations are outsourcing their IoT work and seeing more benefits because of this
  • Top challenges are around knowledge, skill, security, and implementation at scale

The original report

Posted on 3 Comments

Device VS Module: A Somewhat Deep Dive into the Differences Between Azure IoT Hub Devices and Modules

What’s the real difference between an IoT Hub Module and an IoT Hub Device? What is an IoT Hub module and what is an IoT Hub device according to the development team? And why would I choose one over the other?

In this exploratory article I want to attempt to answer these questions for myself and other developers. This is one of those articles to help solidify my own understanding, as well as anyone else who comes across my blog. In other words, if you read something and think it needs correction, please, please, please correct me. If you want to discuss it, I’m open to that, as well. I’m always a student and always open to learning. And it’s an exploratory article, so don’t expect it to be succinct or completely organized. I’ll do my best to keep things clear, but I might jump around a bit. Basically, this is me thinking on paper and attempting to best organize my notes on these topics into something digestible to the reader.

Let’s dive in.

The first thing to understand is that there is a plethora of IoT Hub SDKs. They split into two primary categories:

  • Devices
  • Services

There are Azure IoT Hub Device SDKS, and there are Azure IoT Hub Service SDKs. I’m more interested in the device SDKs for this exploration, but it doesn’t hurt to touch on what the service SDKs offer.

The Azure IoT service SDKs contain code to facilitate building applications that interact directly with IoT Hub to manage devices and security.

Basically, if you need help managing the operations of IoT hub using software, these SDKs are the tool for you. That’s an interesting topic and I do want to eventually develop some tools in this area, but I won’t dive any deeper into the service SDK in this article.

That leaves us with the other branch, Azure IoT Hub Device SDKs:

The Microsoft Azure IoT device SDKs contain code that facilitates building applications that connect to and are managed by Azure IoT Hub services.

This is the area that I want to explore. Once I’ve drilled down this far, I still have a few choices. There are at the time of this article’s writing seven different device SDKs.

  • Azure IoT Hub device SDK for .Net
  • Azure IoT Hub device SDK for Embedded C (ANSI C – C99)
  • Azure IoT Hub device SDK for C (ANSI C – C99)
  • Azure IoT Hub device SDK for Java
  • Azure IoT Hub device SDK for Node.js
  • Azure IoT Hub device SDK for Python
  • Azure IoT Hub device SDK for iOS

That’s a lot of different language and framework choices. For this article, I’m going to focus on Python. Python, C, and C++ appear to be the languages of choice for the majority of IoT device developers. I think Python is the more approachable for developers like me who are more generalists than specialists, but I tell you what (Hank Hill voice) that SDK for Embedded C looks like a lot of fun. Working more with MCUs (Microcontroller Units) is definitely in my future, but right now I just want to be able to conquer the simple things, like an Arduino or Raspberry PI. I’ll stick with Python for now.

I guess that’s a use case clearly established. If you are looking to develop embedded device code, code that needs to run on very little memory on a very tiny device, you want to use the SDK for Embedded C.

This would probably be a great device to develop on for developers looking to create low-power IoT devices that use an MCU:

PIC-IoT WG Development Board

I’ll clarify further that this is not meant to be a step-by-step tutorial. I will include code snippets, I might even try a few things out on my little MXCHIP IoT DevKit, but I primarily want to focus on the use cases of devices VS modules.

Back on focus, let’s examine the actual Azure IoT Hub device SDK for Python. First, I’ll look at the API, and then I’ll dive into the actual source code.

The API

azure-iot-device package

The API is simple. It has one primary package, which is ‘device’. The device package has a couple of interesting things worth looking at. First, there are synchronous and asynchronous packages. So why are there two different versions? Why do we need both an async and a sync version of certain operations in the client SDK?

I think I intuitively know the answer to this question. Sync is probably there to support older versions of Python and maybe older technologies that will use the SDK. Async is generally preferred for communication in distributed systems because you don’t want to hold up operations because one of the parties in the communication isn’t responding. Let’s see how correct I am in my thinking.

Who can I ask about this? Well, I’m working in the SDK for an open-source application. Surely, the maintainer of the OSS has guideline for its developers. And it does! I’m turning to the Azure SDK developer’s guide with the hope it can shed some light.

.NET Azure SDK Design Guidelines

Ah, look! I searched through this document for the term, ‘Async’ and look what I found.

Sync and Async ✅ DO provide both asynchronous and synchronous variants for all service methods. Many developers want to port existing applications to the Cloud. These applications are often synchronous, and the cost of rewriting them to be asynchronous is usually prohibitive. Calling asynchronous APIs from synchronous methods can only be done through a technique called sync-over-async, which can cause deadlocks. The Azure SDK provides synchronous APIs to minimize friction when porting existing application to Azure. ✅ DO ensure that the names of the asynchronous and the synchronous variants differ only by the Async suffix

Additionally, the above quote provides a link to a nine year old article that’s still relevant.

Part 1 and 2 of the article on Sync VS Async. These are older articles, but they still hold up.

Should I expose asynchronous wrappers for synchronous methods? | .NET Parallel Programming

Should I expose synchronous wrappers for asynchronous methods? | .NET Parallel Programming

There’s a whole rabbit hole here that you can go down if you aren’t familiar with the reasons why you select sync or async in your development. I like the fact that the SDK developers are taking into consideration that some developers can’t easily work with async libraries and they’ve gone to the effort to provide both.

My takeaway, when you are developing greenfield projects and you won’t use a library that depends on synchronous operations, you should favor asynchronous over synchronous.

That narrows down what I want to really focus on in the API. I want to focus on the operations that are in the ‘aio’ package. Why is it called, ‘aio’? I’m certain that stands for, Asynchronous I/O. I’m confident enough that I don’t think I need to look it up, but someone can correct me if I’m wrong.

The aio package has both modules and classes. The module appears to be patch documentation. There’s only one function in this called:

execute_patch_for_async()

The documentation reads, “This module provides hard coded patches used to modify items from the libraries. Currently we have to do like this so that we don’t use exec anywhere.”

This appears to be some sort of SDK developer tool. Maybe when we start looking at the source, we’ll get a better idea what it is.

The classes are where the interesting stuff happens. We have 3 basic async classes:

  • IoTHubDeviceClient
  • IoTHubModuleClient
  • ProvisioningDeviceClient

Yay! We’re reached a couple of items that match our search. We have a device client and a module client. We also have a provisioning device client, but I think I’ll keep my focus on the first two.

IoTHubDeviceClient VS IoTHubModuleClient

To set focus, I’m trying to discern from the code and documentation related to IoT Hub Device Client and IoT Hub Module Client when one should be selected over the other. As I dive into the code and documentation, I’ll try to uncover these answers.

One thing I haven’t even asked myself, but I am now, is it even fair to expect to find these answers in the API or the code? I don’t know. That might be a bit too philosophical. Should the code clearly communicate its use case intent just by its very existence? We can look at DNA and determine that certain DNA is human while other DNA is a fly, but that might just be because we have the finished product of the DNA to compare. But DNA is more like the completed software designed with an SDK. I guess in our case the SDK is more like RNA. It’s the little factory pumping out DNA. So no, I might not find what I’m looking for by reviewing the SDK, but it’s still worth exploring just from a knowledge gathering standpoint (hey, I warned you that this was an exploratory article).

The IoTHubDeviceClient

This class has the following:

  • A constructor
  • Methods
    • connect
    • create_from_connection_string
    • create_from_sastoken
    • create_from_symmetric_key
    • create_from_x509_certificate
    • disconnect
    • notify_blob_upload_status
    • patch_twin_reported_properties
    • receive_message
    • receive_method_request
    • receive_twin_desired_properties_patch
    • send_message
    • send_method_response
    • update_sastoken
  • Attributes
    • on_message_received

This class has a lot going on, but all of it is related to a device communicating with IoT Hub.

Constructor

How do we build this class? The constructor code looks like this:

IoTHubDeviceClient(mqtt_pipeline, http_pipeline)

So what is an mqtt_pipeline and http_pipeline? There’s not much in the API documentation in the way of help on this one, so I’m going to go ahead and dive into the source to see what I can find.

Here’s our constructor, and there’s an interesting bit of instruction in there:

def __init__(self, **kwargs):

        “””Initializer for a generic synchronous client.

        This initializer should not be called directly.

        Instead, use one of the ‘create_from_’ classmethods to instantiate

        :param mqtt_pipeline: The MQTTPipeline used for the client

        :type mqtt_pipeline: :class:`azure.iot.device.iothub.pipeline.MQTTPipeline`

        :param http_pipeline: The HTTPPipeline used for the client

        :type http_pipeline: :class:`azure.iot.device.iothub.pipeline.HTTPPipeline`

        “””

        # Depending on the subclass calling this __init__, there could be different arguments,

        # and the super() call could call a different class, due to the different MROs

        # in the class hierarchies of different clients. Thus, args here must be passed along as

        # **kwargs.

        super(GenericIoTHubClient, self).__init__(**kwargs)

        self._inbox_manager = InboxManager(inbox_type=SyncClientInbox)

        self._handler_manager = sync_handler_manager.SyncHandlerManager(self._inbox_manager)

        # Set pipeline handlers

        self._mqtt_pipeline.on_connected = CallableWeakMethod(self, “_on_connected”)

        self._mqtt_pipeline.on_disconnected = CallableWeakMethod(self, “_on_disconnected”)

        self._mqtt_pipeline.on_method_request_received = CallableWeakMethod(

            self._inbox_manager, “route_method_request”

        )

        self._mqtt_pipeline.on_twin_patch_received = CallableWeakMethod(

            self._inbox_manager, “route_twin_patch”

        )

Ok, so clearly, we don’t need to worry about this constructor, because we shouldn’t ever try to create the class directly. Instead, we should always use one of the various connection methods to create the client. This is a good coding practice. The class is useless if it isn’t connected to IoT Hub.

My takeaway, if you are creating a device using the IoTHubDeviceClient expect to tightly couple it to an IoT Hub using one of the create_from_X methods. The device can’t live without a connection.

What about disconnecting? Is that handled for us automatically, too or do we need to manage that ourselves?

The API documentation does offer some direction here by stating, “It is recommended that you make sure to call this coroutine when you are completely done with your client instance.”

Yes, we should make the actual disconnection when we’re done with the client.

Let’s explore some of the other methods. I’ll start from the top and start working my way down, skipping over the various connection methods for now.

get_storage_info_for_blob, notify_blob_upload_status

get_storage_info_for_blob(blob_name)

From the documentation, “Sends a POST request over HTTP to an IoTHub endpoint that will return information for uploading via the Azure Storage Account linked to the IoTHub your device is connected to.”

The functionality of this method is covered here:

Upload files from devices to Azure IoT Hub with Python

Understand Azure IoT Hub file upload

If we have the correct tier of IoT Hub (Free or S1 and above) and an associated blob storage account, we can upload files from the device to the storage account by using the IoT Hub as a broker.

This is ideal for uploading large media files and telemetry batches. This is ideal for use cases that involve devices that are regularly disconnected from the internet or devices that capture large files. Think of a drone capturing soil data through high-definition imaging. Those high-definition images would need to transfer from the drone to a blob storage. And you might have smaller devices that send regular telemetry, but that device might log more data than you require for real time analysis. So that data could be uploaded in batches.

This is one interesting use case that doesn’t require IoT Edge. It can be done with something that is considered just a device.

For the completion of the upload process, we can use notify_blob_upload_status to, “When the upload is complete, the device sends a POST request to the IoT Hub endpoint with information on the status of an upload to blob attempt. This is used by IoT Hub to notify listening clients.”

notify_blob_upload_status(correlation_id, is_success, status_code, status_description)

patch_twin_reported_properties, receive_twin_desired_properties_patch, and get_twin

A device also has digital twin capabilities. I like what AWS calls this better than digital twin. They call it a device shadow, but either way it’s basically the same thing. A JSON representation of the device that is in both the device’s memory and the IoT Hub.

Device twins are JSON documents that store device state information including metadata, configurations, and conditions. Azure IoT Hub maintains a device twin for each device that you connect to IoT Hub.

There are three primary parts to a device twin document:

  • tags
  • desired properties
  • reported properties.

The use case for device twins:

  • Store device specific metadata in the cloud, like its location or another important device information
  • Report current device state.
  • Synchronize the state of long running workflows between the device app and back end apps.
  • Query your device metadata, configuration, or state.

Example of a device twin:

{

    “deviceId”: “devA”,

    “etag”: “AAAAAAAAAAc=”,

    “status”: “enabled”,

    “statusReason”: “provisioned”,

    “statusUpdateTime”: “0001-01-01T00:00:00”,

    “connectionState”: “connected”,

    “lastActivityTime”: “2015-02-30T16:24:48.789Z”,

    “cloudToDeviceMessageCount”: 0,

    “authenticationType”: “sas”,

    “x509Thumbprint”: {    

        “primaryThumbprint”: null,

        “secondaryThumbprint”: null

    },

    “version”: 2,

    “tags”: {

        “$etag”: “123”,

        “deploymentLocation”: {

            “building”: “43”,

            “floor”: “1”

        }

    },

    “properties”: {

        “desired”: {

            “telemetryConfig”: {

                “sendFrequency”: “5m”

            },

            “$metadata” : {…},

            “$version”: 1

        },

        “reported”: {

            “telemetryConfig”: {

                “sendFrequency”: “5m”,

                “status”: “success”

            },

            “batteryLevel”: 55,

            “$metadata” : {…},

            “$version”: 4

        }

    }

}

There’s a lot to the device twin capabilities. I won’t deep dive on it at this point, but the fact that devices support this capability means that devices are very powerful on their own, even without being edge modules. But what’s coming clear here is that we have a very basic client server relationship. The device is the client and, in some cases, the IoT Hub is the server or the gateway to a server.

With that in mind, let’s explore what we can do with the three functions made available to us.

get_twin()

This returns a complete twin as a JSON dictionary object. It can pull this from either the IoT Hub or an Edge Hub service. I’m not sure exactly what we would do with this data at this time, because I know that we can update device twin settings from the backend. If we needed to update the device settings from the backend, we could modify the desired properties of the device twin. So what might we need to do from the device itself? And what does get_twin() do for us?

According to the documentation, receive_twin_desired_properties_patch has been deprecated in favor of the on_twin_desired_properties_patch_received property. We also have patch_twin_reported_properties which will report properties with the Azure IoT Hub or Azure IoT Edge Hub service. If the service returns an error on the patch operation, this function will raise the appropriate error.

One other thing not mentioned in the SDKs that is important to understanding the “why,” behind device digital twins in the IoT Hub query language for device and module twins, jobs, and message routing.

As an aside, this is an import thing for a developer to learn and understand for working with the Azure IoT Hub suite of tools. It’s rather simple. If you are familiar with other query languages, like SQL or KQL you’ll find it easy to work with.

Taking all of this in, it’s clear that the digital twin methods, the backend query language, and the overall digital twin data object are all about monitoring and maintaining device health.

For instance, this little query is a good example of checking the status for multiple devices:

SELECT properties.reported.telemetryConfig.status AS status,

    COUNT() AS numberOfDevices

  FROM devices

  GROUP BY properties.reported.telemetryConfig.status

This can be done from backend systems in the same way that you would write in-line SQL.

Example in C#

var query = registryManager.CreateQuery(“SELECT * FROM devices”, 100);

while (query.HasMoreResults)

{

    var page = await query.GetNextAsTwinAsync();

    foreach (var twin in page)

    {

        // do work on twin object

    }

}

I’m seeing plenty of applications for device health, monitoring, and automation jobs. Again, very powerful capabilities that doesn’t seem to make devices a second-class citizen.

The potential of what could be done from the device side doesn’t seem too limited, either.

Messaging

Here we are. The biggest and probably most important part of an IoT device’s job. Messaging.

Sending and receiving messages.

send_message(message)

Sends a message to the default events endpoint on the Azure IoT Hub or Azure IoT Edge Hub instance. If the connection to the service has not previously been opened by a call to connect, this function will open the connection before sending the event.

The Message is a separate class that’s part of the azure.iot.device package. It won’t hurt to touch on it here, since it’s the thing we’re using as a primary means of transport. The Message object appears to be shared by both devices and modules, if that’s not the case I’ll probably know better when I dive into modules.

Like the Device class, the Message class also has a constructor, a set of variables, and a few methods. It makes sense to dive into those while we’re here to get a better understanding of what the Message class gives us.

The message constructor:

Message(data, message_id=None, content_encoding=None, content_type=None, output_name=None)

Unlike the device constructor, we clearly do want to set this ourselves in order to hydrate the class with the appropriate content needed to create a message.

Variables

data – data that constitutes the payload.

custom_properties – Dictionary of custom message properties. The keys and the values of these properties will always be string.

id – A user-settable identifier for the message sued for request-reply patterns. Format: A case-sensitive string (up to 128 characters long) of ASCII 7-bit alphanumeric characters + {‘-‘, ‘:’, ‘.’, ‘+’, ‘%’, ‘_’, ‘#’, ‘*’, ‘?’, ‘!’, ‘(‘, ‘)’, ‘,’, ‘=’, ‘@’, ‘;’, ‘$’, ”’}

expiry_time_utc – Date and time of message expiration in UTC format.

correlation_id – A property in a response message that typically contains the message_id of the request, in request-reply patterns.

user_id – An ID to specify the origin of the messages.

content_encoding – Content encoding of the message data. Can be ‘utf-8’, ‘utf-16’, or ‘utf-32’

content-type – Content type property used to route messages with the message-body. Can be ‘application/json’

output_name Name of the output that the message is being sent to.

input_name Name of the input that the message was received on.

There are only two methods.

get_size()

This gets the message size.

set_as_security_message()

This is listed as a provisional API. But it should set the message as a secure message.

So to return to our device analysis, the above is what is sent when we make the call to the send_message(message) method.

There are four specific exceptions that can be handled for the send_message(message) method.

  • CredentialError
  • ConnectionFailedError
  • ConnectionDroppedError
  • ClientError

These are self-explanatory.

receive_message()

This method is deprecated. It’s recommended to use the. on_message_received property to set a handler instead.

This is also the recommended means of responding to a method request. So we won’t cover that here.

send_method_response(method_response)

This method is used to respond to a method request from Azure IoT Hub or Azure IoT Edge Hub.

.on_message_received

This is the handler function or coroutine that will be called when a message is received. It should take an argument to receive a Message object.

The above represents a cloud-to-device message. Currently, the existing Python code samples I could find referenced the deprecated method. So how do we handle incoming messages? Basically, assign a method to the property that takes a message. That would look like the following.

# this is how I would likely create the handler

def message_received_handler(message):

        print(‘do something with message’ + message.data

# Here s where we set the handler

device_client.on_message_received = message_received_handler

This handler cuts down on the number of methods in the class.

This pretty much covers the device SDK. Next, I want to dive into the Module to see what options we have there. I’ll try not to duplicate areas that are shared by both. What I’m looking for are the areas of the module that really stand out for particular use cases where the device isn’t enough.

Modules

Now it’s time to closely examine the IoTHubModuleClient Class. Much like the device, it’s described as,

An asynchronous module client that connects to an Azure IoT Hub or Azure IoT Edge instance.

And much like the device, it has a constructor, methods, and attributes.

Constructor

One difference I noticed between the device and the module is a creation method titled, create_from_edge_environment. It reads that this allows the class to be created from the IoT Edge environment.

This method can only be run from inside an IoT Edge container, or in a debugging environment configured for Edge development (e.g., Visual Studio, Visual Studio Code)

What exactly is the Edge Environment?

Ok, so maybe we need to do a little time travelling here. Go back a bit. Let’s go back to the product that superseded IoT Edge. Let’s go back to the Azure IoT Gateway SDK. That’s right, because it had a 90s cool name like, Edge. It was known as Azure IoT Gateway SDK.

I’ve forked a version as close to the original as I could find, just for learning purposes.

shawndeggans/azure-iot-gateway-sdk

It’s clear to me from the name that IoT Edge wasn’t really meant to be a device itself, but a gateway for devices to indirectly communicate with the Azure IoT backend. The original documentation includes the following image.

In this original image the Gateway running on on-premises hardware is described as a module pipeline. Also notice that this original version appears to be a one-way trip. There’s device-to-cloud data, but not necessarily cloud-to-device messages.

It’s interesting looking at the early version of IoT Edge. Especially when you take into account what is needed to build a development environment. It appears much simpler. I don’t see Docker/Moby anywhere. We seem to be missing some of agents or at least there aren’t called out in the documentation. Overall, it looks to be a nice piece of middleware for working with devices. If its original intent was to be middleware, is it fair to say that it’s still middleware today? Or is it more like middleware that can also be a device?

Let’s continue to see if we can nail down what constitutes the IoT Edge environment. If we go by the origin project, the IoT Edge environment is a “module pipeline,” that accomplishes your specific scenario. Is that still the case?

What is Azure IoT Edge

Microsoft has this defined in the Azure Docs.

Azure IoT Edge moves cloud analytics and custom business logic to devices so that your organization can focus on business insights instead of data management. Scale out your IoT solution by packaging your business logic into standard containers, then you can deploy those containers to any of your devices and monitor it all from the cloud.

That does not sound like actual devices, but more things that would be on a server or in the cloud. Analytics is basically using data and math to answer business (domain specific) question. This is shaping up to sound a lot more like an environment or an operating system than a device.

Azure IoT Edge is made up of three primary components:

  • IoT Edge Modules
  • IoT Edge Runtime
  • A Cloud-based Interface

IoT Modules are containers. Basically, these are Docker files that contain Azure services, third-party services, or our own custom applications. This brings us back to the actual SDK, which doesn’t mention anything about Docker (or Moby as the case is for the IoT Edge Environment), but judging by some of the methods (which I will dive into later), they are not necessarily meant to be the terminus of the solution, but they should support the responsibility of middleware.

What about the IoT Edge Runtime? Does it still hold up with the earlier definition of a “pipeline of modules”?

IoT Edge Runtime

The Azure IoT Edge runtime enables custom and cloud logic on IoT Edge devices. The runtime sits on the IoT Edge device and performs management and communication operations.

Hmm, this does sound a bit like an operating system or an orchestration piece. Here are some of its responsibilities:

  • Installs and updates workloads on the device.
  • Maintains Azure IoT Edge security standards on the device.
  • Ensures that IoT Edge modules are always running.
  • Reports module health to the cloud for remote monitoring.
  • Manages communication between downstream leaf devices (what I want to call Actual Devices) and an IoT Edge device (or what I would rather call a Gateway or middleware), between modules on an IoT Edge device, and between an IoT Edge device and the cloud.

I think this might have been an early point of confusion for me. The documentation continues to call IoT Edge modules running within the IoT Edge runtime a “device”. I realize it can be a device, but it’s so much more. To me, this is more a server than a client.

Here’s an image from the documentation that shows what look like robot arms (devices) sending telemetry to the Azure IoT Edge runtime, and later to the cloud using IoT Hub as a gateway.

IoT Edge cloud

This basically refers to the backend systems and the overall operation of managing a fleet of IoT devices. Azure offers several possible solutions for this backend work with IoT Hub being the queen bee of the hive and IoT Central being a very close second.

Back to IoTHubModuleClient Class

Ok, that was a bit of a rabbit hole, but I think it was necessary to put things in perspective. Microsoft approaches the IoT Edge devices as actual devices, because they clearly can be, but they can also be hubs, gateways, storage, queues, etc.. This flexibility can make it difficult for someone learning all of this to implement the correct solution when deciding how to approach solution engineering IoT architectures.

The module has many of the same connection abilities as the device, with the exception of the ability to connect within an environment.

There are also different types of methods within the module that point to this duality. Let’s see if we can pick those apart and determine if they are device-like methods, middleware-like methods, or both?

As far as the Digital Twins methods are concerned, they appear to be identical to the device Digital Twin methods. It even has the same deprecation status on the message receiving end. So I think we can just consider these universal.

invoke_method is probably the first one that looks like middleware.

Invoke a method from your client onto a device or module client, and receive the response to the method call.

invoke_method(method_params, device_id, module_id=None)

Clearly this is designed for one module to be able to call a method on either a device or another module. It’s directly addressed, so I assume this is handled by the runtime.

send_message_to_output(message, output_name)

Sends an event/message to the given module output. These are outgoing events and are meant to be “output events” If the connection to the service has not previously been opened by a call to connect, this function will open the connection before sending the event.

This is another method that only exists in the IoT Hub Module Client. The output that this refers to is the output channel on IoT Hub.

update_sastoken(sastoken)

Update the client’s SAS Token used for authentication, then reauthorizes the connection. This API can only be used if the client was initially created with a SAS Token. Note also that this API may return before the reauthorization/reconnection is completed. This means that some errors that may occur as part of the reconnection could occur in the background and will not be raised by this method.

This is only on the IoT Edge module and allows a call to the device to update the SAS Token, if it was created with a SAS Token. This is useful for managing security key rotation.

My closing opinion

Despite much of my earlier confusion around devices and modules, I think it’s very clear to me now when one should be selected over the other.

Devices

Use the Azure IoT Hub Device SDK when you want to create a device that will primarily serve as a data collection tool. This can actually be a fairly sophisticated piece of code but remember that it has a singular purpose. For instance, right now I’m creating a device for a Raspberry Pi that is collecting moisture for a plant. That’s not an IoT Edge Module. It doesn’t need to be. It just needs to run on the Pi and send telemetry to Azure IoT Hub. It doesn’t need AI, it doesn’t need event processing, no business logic, and no interaction with other devices other than IoT Hub. In my opinion, this is the ideal use case for a device and not a module.

Modules

When I want to bring some functionality of the cloud down to a device to make it faster, easier to work with in a disconnected state, or to work with AI, I will use the Azure IoT Hub Module SDK to create an Azure IoT Edge device. I still don’t like calling it a device but keeping these basic ideas in mind will help build clarity around my future architectures. I do have some plans to put IoT Edge into service using a model built from a ML project. And I want to use Azure Blob Storage on the Edge, so there’s a good use case.

Overall, I think the Azure IoT platform is wonderful. I love working with it. I always feel like I’m just scratching the surface of its capabilities, but there’s a lot to learn and some of it can be confusing for the IoT beginner. I hope my long winding exploration has been helpful. Expect future explorations like this. I know it’s a little raw, but it does allow me to get my complete thoughts down and come to working conclusions.


If you enjoyed this post, you may also want to read my recent exploration of the various Azure IoT SDKs.
Posted on Leave a comment

Basic IoT Edge Development Workflow for Simulated Devices

These are just some initial notes related to the workflow for developing simulated devices. This requires the use of the following tools:

  • VS Code
    • Azure IoT Tools
    • C#
    • Docker
  • Azure IoT Hub
  • Azure Container Registry

This is the goal. Work locally using VS Code to develop a custom module. Test and debug that module locally using the IoT Edge Simulator. View simulated telemetry in IoT Hub from local code. Publish completed component to Container Registry. Pull code from registry and run in IoT Hub.

The basic flow from system to system looks like this:

Basic flow of an IoT Edge Module from development/testing/debugging to a Docker image to a simulated device.

This won’t be a complete step-by-step. I’m primarily capturing these notes so that I can improve this workflow. However, if you want to walk through the whole process there is a lab that will take you step-by-step here:

AZ-220-Microsoft-Azure-IoT-Developer

Step 1 – Install Azure IoT EdgeHub Dev Tools

You’ll need Python on your development machine. If you don’t have pip installed, you’ll need that as well. You will need to install the IoT EdgeHub tools using this command:

pip install iotedgehubdev

More information about the project is located here:

iotedgehubdev

Step 2 – Create an Azure Container Registry

Azure Container Registry provides the storage of private Docker images for container deployment. We will need it to provide a place to securely store our docker image.

If you do not already have a container registry, create one in Azure by searching the marketplace for Container Registry. Create one and give it a meaningful name. Use a Standard SKU.

Once it is created you will need to go to settings and Access Keys in order to set it for Administrator mode. Under Admin Users click Enable.

Copy the following values, you will need them:

  • Login server
  • Username
  • Password

Log into Docker using your new server and credentials:

docker login <loginserver>

Step 3 – Create a Custom Edge Module

For VS Code there are a few commands you need to get this solution created. In the command palette, enter the command Azure IoT Edge: New and then click the presented Azure IoT Edge: New IoT Edge Solution. Give it a name and select a language (C#, Java, Python). When prompted to name the IoT Edge Module select something meaningful for the device, like RemainingLifeEstimator.

Once this is created, replace the Docker localhost with the address of the Container Registry. It follows a special format:

<acr-name>.azurecr.io/<module-name>

Once the new IoT Edge Solution has been created prepare to configure the solution for work.

  • be sure to set the env variables for your container registry username and password
  • set the default target platform by typing Azure IoT Edge: Set Default Target Platform for Edge Solution*

Step 4 – Debug in Attach Mode with IoT Edge Simulator

To me this is one of the coolest parts of this solution. You can both test this locally and see the results of it on IoT Hub. This allows you to see that the code you are writing is working as expected from both the device side and the cloud side. It’s worth all the effort to get here.

Create an IoT Edge test device by capturing a connection string from IoT Hub. Go to the IoT Edge portion of the Automatic Device Provision section on the navigation blade. Add an IoT Edge device and give it a good test name. Set it to authenticate with the Symmetric key.

Start testing by right clicking on the deployment.debug.template.json document in your solution. Select the Build and Run IoT Edge Solution in Simulator.

If you haven’t set up the iotedgehubdev first, you will be walked through the process here. Once is is set up you can run and build the project.

Once it is up and running, you can see the telemetry you’ve simulated in the terminal window. Additionally, you can use the following command in the Azure Cloud Shell for your IoT Hub:

az iot hub monitor-events --hub-name "<your-iot-hub>"

You can now debug the module using VS Code debugging tools.

Step 5 – Deploy the IoT Edge Solution

Publish the module to the Azure Container Registry. Insure that your Container Registry credentials are stored in the .env file.

Right click on the deployment.template.json and then click Build and Push IoT Edge Solution.

At the Azure portal, open your Azure Container Registry service and click your container server.

You should see your container under the repositories. From here you’ll want to grab the repository name and the tag properties. Save these values in the following format somewhere (Notepad)

<repository-name>:<tag>

Configure an IoT Edge Device to use the module by adding a new IoT Edge device in the IoT Hub. Give a device id that matches the module name. Then set modules and on the modules blade under Container Registry Settings, enter the values for the registry name, address to the login server, and the username and password.

On the Add IoT Edge Module pane, under IoT Edge Module Name, enter the name of your module. Under module settings enter the full uri and tagged name of your module’s docker image:

<container-registry-login-server>/<repository-name>:<tag>

Add that and then set the module routes. Typically you want a route to the module from other modules and a route to IoT Hub from the module. It could look something like this:

Route Settings

Create this and you now have an Edge Module deployed to an IoT Edge Device.