Question 103412 in Azure Question posted in
The official documentation to get you started can be found here.

Routing device to cloud messages from Azure IoT Hub to Azure Blob Storage

I have an IoT device sending messages to Azure IoT Hub every 5 minutes. I would like to save every message inside an Azure Blob Storage Container. Inside the azure portal, in the Message routing section I created a new route like this one.
New route in Azure IoT Hub
Next, I created a custom endpoint like this one:
Custom endpoint
By doing so I am able to save the messages inside the blob storage.
What I would like to do, is to create a dynamic route. I would like every sent message to be saved within this path: {iothub}/{deviceId}/{messageParameter}/{partition}/{YYYY}/{MM}/{DD}/{HH}/{mm} where deviceId is the name of the device registered in Azure IoT Hub and customValue is a property value from the json message that the IoT device send to the Azure IoT Hub.
This is the code that I use to send messages:

public class Sender : ISender
  {
    private static DeviceClient _deviceClient;

    public void SendDeviceToCloudMessage(string deviceId, string iotHubUri, string deviceKey, string message)
    {
      _deviceClient = DeviceClient.Create(iotHubUri,
        new DeviceAuthenticationWithRegistrySymmetricKey(deviceId, deviceKey), TransportType.Mqtt);

      var twin = _deviceClient.GetTwinAsync().ConfigureAwait(false).GetAwaiter().GetResult();
      var desiredProperties = twin.Properties.Desired;
      var messageObj = JObject.Parse(message);
      if (desiredProperties.Contains("TelemetryData"))
      {
        var telemetryData = (TwinCollection)desiredProperties["TelemetryData"];
        telemetryData["Temperature"] = messageObj["Temperature"];
        telemetryData["Humidity"] = messageObj["Humidity"];
        telemetryData["TimeStamp"] = messageObj["TimeStamp"];
      }
      else
      {
        var telemetryData = new TwinCollection();
        telemetryData["Temperature"] = messageObj["Temperature"];
        telemetryData["Humidity"] = messageObj["Humidity"];
        telemetryData["TimeStamp"] = messageObj["TimeStamp"];
        desiredProperties["TelemetryData"] = telemetryData;
      }

      // Update the reported properties with the updated desired properties
      var reportedProperties = new TwinCollection();
      reportedProperties["TelemetryData"] = desiredProperties["TelemetryData"];
      _deviceClient.UpdateReportedPropertiesAsync(reportedProperties).ConfigureAwait(false).GetAwaiter().GetResult();

      using var iotMessage = new Message(Encoding.UTF8.GetBytes(message))
      {
        ContentEncoding = "utf-8",
        ContentType = "application/json",
      };

      // Submit the message to the hub.
      _deviceClient.SendEventAsync(iotMessage).ConfigureAwait(false).GetAwaiter().GetResult();
    }
  }

The message input is a json string like this one:

{
  "Temperature": 20,
  "Humidity": 50,
  "TimeStamp": "2023-02-26 14:02:59.7715110 +00:00",
  "MessageId": "MessageIdentifier"
}

Is this possible or I need to manually save the message in Azure Blob Storage Container?

Note: My goal is to save the messages sent by the device and subsequently be able to read the messages sent by a specific device (for this reason I put the deviceId in the path) relating to a specific parameter found within the message sent (messageParameter)

Tags:

3

Answers


  1. 0

    This is not possible by leveraging the storage endpoint functionality in IoT Hub. When creating a storage endpoint, you can only use the following tokens (and have to use all of them):

    • {iothub}
    • {partition}
    • {YYYY}
    • {MM}
    • {DD}
    • {HH}
    • {mm}

    To include deviceId and customValue in the path, you must implement that functionality yourself. One option would be to write an Azure Function to store the message in the correct path.

    Login or Signup to reply.
  2. 0

    You could consider using routing queries and multiple endpoints to achieve this (https://learn.microsoft.com/azure/iot-hub/iot-hub-devguide-routing-query-syntax). However, it looks like you’d need a separate blob storage endpoint for each device, and you’re limited to 10 custom endpoints per hub.

    I notice that your proposed path includes both {deviceId} and {partition} – devices are "sticky" to partitions, so there’s some redundancy in the path segments.

    Overall, if you want that level of control of the path where the values are stored, you might be better off routing your messages to an Azure Function and then having some logic in the function that saves messages to blob storage in the structure you require.

    What’s your reason for wanting to store the messages in blob storage using that particular structure?

    Login or Signup to reply.
  3. 0

    As answers mentioned, there is no built-in IoT Hub feature for your needs in the custom endpoint for blob storage. The following example shows a workaround where the Device Telemetry Data are pushed to the Blob Storage using the Azure Event Grid Pub/Sub model:

    enter image description here

    The subscriber is an Azure EventGridTrigger Function with handling a blob name, etc.

    #r "Newtonsoft.Json"
#r "System.Text.Json"
#r "Azure.Storage.Blobs"
#r "System.Memory.Data"
#r "Azure.Core"

using System;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using Microsoft.Extensions.Logging;
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Specialized;
using Azure.Storage.Blobs.Models;
using System.Collections.Generic;

public static async Task Run(JObject eventGridEvent, BlobContainerClient blobContainer, ILogger log)
{
    log.LogInformation(eventGridEvent.ToString());

    string iothubname = eventGridEvent["data"]?["properties"]?["iothubname"]?.Value<string>() ?? "unknown";
    string deviceId = eventGridEvent["data"]?["systemProperties"]?["iothub-connection-device-id"]?.Value<string>() ?? "unknown";
    DateTime dt = eventGridEvent["data"]?["systemProperties"]?["iothub-enqueuedtime"]?.Value<DateTime>() ?? default(DateTime);
      
    string blobName = $"{iothubname}/{deviceId}/{dt.ToString("yyyy/MM/dd/HH/mm/ss")}.json";
    log.LogInformation($"blobname = {blobName}"); 

    var tags = new Dictionary<string, string>();
    tags.Add("iothubname", iothubname);
    tags.Add("deviceId", deviceId);
    tags.Add("value", "CustomValue");

    var blobClient = blobContainer.GetBlobClient(blobName);
    await blobClient.UploadAsync(BinaryData.FromString(eventGridEvent["data"]?["body"]?.ToString() ?? eventGridEvent.ToString() ), overwrite: true);
    //blobClient.SetTags(tags);   // option for 'blob index tags'
}

    Note, that the Azure IoT Hub routed message can be enriched by useful values from device twin (for example, $twin.tags.field or $twin.properties.desired.value) and used them in the subscriber as a part of the blob name or in the blob index tags.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search