Production deployment guidelines

Security and scalability are a priority for deploying Azure IoT Operations. This article outlines guidelines that you should take into consideration when setting up Azure IoT Operations for production.

Decide whether you're deploying Azure IoT Operations to a single-node or multi-node cluster before considering the appropriate configuration. Many of the guidelines in this article apply regardless of the cluster type, but when there is a difference it's called out specifically.

Platform

Currently, K3s on Ubuntu 24.04 is the only generally available platform for deploying Azure IoT Operations in production.

Cluster setup

Ensure that your hardware setup is sufficient for your scenario and that you begin with a secure environment.

System configuration

Create an Arc-enabled K3s cluster that meets the system requirements.

• Use a supported environment for Azure IoT Operations.
• Configure the cluster according to documentation.
• If you expect intermittent connectivity for your cluster, ensure that you've allocated enough disk space to the cluster cache data and messages while the cluster is offline.
• If possible, have a second cluster as a staging area for testing new changes before deploying to the primary production cluster.
• Turn off auto-upgrade for Azure Arc to have complete control over when new updates are applied to your cluster. Instead, manually upgrade agents as needed.
• For multi-node clusters: Configure clusters with Edge Volumes to prepare for enabling fault tolerance during deployment.

Security

Consider the following measures to ensure your cluster setup is secure before deployment.

• Validate images to ensure they're signed by Microsoft.
• When doing TLS encryption, bring your own issuer and integrate with an enterprise PKI.
• Use secrets for on-premises authentication.
• Use user-assigned managed identities for cloud connections.
• Keep your cluster and Azure IoT Operations deployment up to date with the latest patches and minor releases to get all available security and bug fixes.

Networking

If you use enterprise firewalls or proxies, add the Azure IoT Operations endpoints to your allowlist.

Observability

For production deployments, deploy observability resources on your cluster before deploying Azure IoT Operations. We also recommend setting up Prometheus alerts in Azure Monitor.

Deployment

For a production-ready deployment, include the following configurations during the Azure IoT Operations deployment.

MQTT broker

In the Azure portal deployment wizard, the broker resource is set up in the Configuration tab.

• Configure cardinality settings based on memory profile and needs for handling connections and messages. For example, the following settings could support a single-node or multi-node cluster:

  Setting	Single node	Multi node
  frontendReplicas	1	5
  frontendWorkers	4	8
  backendRedundancyFactor	2	2
  backendWorkers	1	4
  backendPartitions	1	5
  Memory profile	Low	High

• Encrypt internal traffic.

• Set disk-backed message buffer with a max size that prevents RAM overflow.

Schema registry and storage

In the Azure portal deployment wizard, the schema registry and its required storage account are set up in the Dependency management tab.

• The storage account must have hierarchical namespace enabled.
• The schema registry's managed identity must have contributor permissions for the storage account.
• The storage account is only supported with public network access enabled.

For production deployments, scope the storage account's public network access to allow traffic only from trusted Azure services. For example:

1. In the Azure portal, navigate to the storage account that your schema registry uses.
2. Select Security + networking > Networking from the navigation menu.
3. For the public network access setting, select Enabled from selected virtual networks and IP addresses.
4. In the Exceptions section of the networking page, ensure that the Allow trusted Microsoft services to access this resource option is selected.
5. Select Save to apply the changes.

For more information, see Configure Azure Storage firewalls and virtual networks > Grant access to trusted Azure services.

Fault tolerance

Multi-node clusters: Fault tolerance can be enabled in the Dependency management tab of the Azure portal deployment wizard. It's only supported on multi-node clusters, and is recommended for production deployment.

Secure settings

During deployment, you have the option to use test settings or secure settings. For production deployments, choose secure settings. If you're upgrading an existing test settings deployment for production, follow the steps in Enable secure settings.

Post-deployment

After deploying Azure IoT Operations, have the following configurations in place for a production scenario.

MQTT broker

After deployment, you can edit BrokerListener resources:

• Configure TLS with automatic certificate management for listeners.

You can also edit BrokerAuthentication resources.

• Use X.509 certificates or Kubernetes service account tokens for authentication.
• Don't use no-auth.

When you create a new resource, manage its authorization:

• Create a BrokerAuthorization resource and provide the least privilege needed for the topic asset.

OPC UA broker

For connecting to assets at production, configure OPC UA authentication:

• Don't use no-auth. Connectivity to OPC UA servers isn't supported without authentication.
• Set up a secure connection to OPC UA server. Use a production PKI and configure application certificates and trust list.

Data flows

When using data flows in production:

• Use service account token (SAT) authentication with the MQTT broker (default).
• Always used managed identity authentication. When possible, use user-assigned managed identity in data flow endpoints for flexibility and auditability.
• Scale data flow profiles to improve throughput and have high availability.
• Group multiple data flows into data flow profiles and customize scaling for each profile accordingly.