Configure data flow profile

Article
2025-04-29

Important

This page includes instructions for managing Azure IoT Operations components using Kubernetes deployment manifests, which is in preview. This feature is provided with several limitations, and shouldn't be used for production workloads.

See the Supplemental Terms of Use for Microsoft Azure Previews for legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

Data flow profiles can be used to group data flows together so that they share the same configuration. You can create multiple data flow profiles to manage sets of different data flow configurations.

The most important setting is the instance count. For a given data flow, the instance count determines the number of copies that run on your cluster. For example, you might have a data flow profile with a single instance for development and testing, and another profile with multiple instances for production. Or, you might use a data flow profile with low instance count for low-throughput data flows and a profile with high instance count for high-throughput data flows. Similarly, you can create a data flow profile with different diagnostic settings for debugging purposes.

Default data flow profile

A data flow profile named default is created when Azure IoT Operations is deployed. You can use this data flow profile to get started with Azure IoT Operations.

To view the default data flow profile, go to your IoT Operations instance in the Azure portal.
In the left pane under Components, select Data flow profiles.
Select the default data flow profile.

param aioInstanceName string = '<AIO_INSTANCE_NAME>'
param customLocationName string = '<CUSTOM_LOCATION_NAME>'

// Pointer to the Azure IoT Operations instance
resource aioInstance 'Microsoft.IoTOperations/instances@2024-11-01' existing = {
  name: aioInstanceName
}

// Pointer to your custom location where AIO is deployed
resource customLocation 'Microsoft.ExtendedLocation/customLocations@2021-08-31-preview' existing = {
  name: customLocationName
}

// Pointer to the default data flow profile
resource defaultDataflowProfile 'Microsoft.IoTOperations/instances/dataflowProfiles@2024-11-01' = {
  parent: aioInstance
  name: 'default'
  extendedLocation: {
    name: customLocation.id
    type: 'CustomLocation'
  }
  properties: {
    instanceCount: 1
  }
}

apiVersion: connectivity.iotoperations.azure.com/v1
kind: DataflowProfile
metadata:
  name: default
  namespace: azure-iot-operations
spec:
  instanceCount: 1

Unless you need additional throughput or redundancy, you can use the default data flow profile for your data flows. If you need to adjust the instance count or other settings, you can create a new data flow profile.

Create a new data flow profile

To create a new data flow profile, specify the name of the profile and the instance count.

Important

Data profile name length must be between 1 and 39 characters.

In the Azure portal, go to your IoT Operations instance.
In the left pane under Components, select Data flow profiles.
Select + Data flow profile to create a new data flow profile.
In the Create data flow profile pane, enter a name for the data flow profile and set the instance count. You can also set other settings such as diagnostics settings.

resource dataflowProfile 'Microsoft.IoTOperations/instances/dataflowProfiles@2024-11-01' = {
  parent: aioInstance
  name: '<NAME>'
  properties: {
    instanceCount: <COUNT>
  }
}

apiVersion: connectivity.iotoperations.azure.com/v1
kind: DataflowProfile
metadata:
  name: '<NAME>'
  namespace: azure-iot-operations
spec:
  instanceCount: <COUNT>

Scaling

You can scale the data flow profile to adjust the number of instances that run the data flows. For a given data flow, instance count is the number of copies that run on your cluster. Increasing the instance count can improve the throughput of the data flows by creating multiple clients to process the data. When using data flows with cloud services that have rate limits per client, increasing the instance count can help you stay within the rate limits.

Scaling can also improve the resiliency of the data flows by providing redundancy in case of failures.

To manually scale the data flow profile, specify the number of instances you want to run. For example, to set the instance count to 3:

In the Azure portal, go to your IoT Operations instance.
In the left pane under Components, select Data flow profiles.
Select the data flow profile you want to configure.
Use the slider to set the instance count.

properties: {
  instanceCount: 3
}

spec:
  instanceCount: 3

Diagnostic settings

You can configure other diagnostics settings for a data flow profile such as log level.

In most cases, the default settings are sufficient. However, you can override the log level or other settings for debugging.

To learn how to configure these diagnostic settings, see ProfileDiagnostics.

In the Azure portal, go to your IoT Operations instance.
Under Components, select Data flow profiles.
Select the data flow profile you want to configure.

resource dataflowProfile 'Microsoft.IoTOperations/instances/dataflowProfiles@2024-11-01' = {
  parent: aioInstance
  name: '<NAME>'
  properties: {
    instanceCount: 1
    diagnostics: {
      {
        logs: {
          level: 'debug'
        }
      }
    }
  }
}

apiVersion: connectivity.iotoperations.azure.com/v1
kind: DataflowProfile
metadata:
  name: '<NAME>'
  namespace: azure-iot-operations
spec:
  instanceCount: 1
  diagnostics:
    logs:
      level: debug

Next steps

To learn more about data flows, see Create a data flow.

Share via