Azure AI Language Retry policy

Hello Umer Rashid !

Thank you for posting on Microsoft Learn.

Azure AI Language services implement a default retry policy that utilizes exponential backoff with jitter to handle transient failures, such as network issues or throttling. This approach helps distribute retry attempts over time, reducing the likelihood of overwhelming the service.

To define your own retry policy with exponential backoff, you can configure the retry settings in the SDK you're using.

When using the Azure SDK for .NET, you can customize the retry policy by configuring the RetryOptions in the client options.

var clientOptions = new Azure.AI.TextAnalytics.TextAnalyticsClientOptions
{
    Retry =
    {
        Mode = RetryMode.Exponential,
        Delay = TimeSpan.FromSeconds(2),
        MaxDelay = TimeSpan.FromSeconds(16),
        MaxRetries = 5
    }
};

var client = new TextAnalyticsClient(new Uri(endpoint), new AzureKeyCredential(apiKey), clientOptions);

In this example, the retry policy is set to exponential mode with an initial delay of 2 seconds, a maximum delay of 16 seconds, and up to 5 retry attempts.

For Java applications, the Azure SDK provides the ExponentialBackoff class to implement exponential backoff strategies.

ExponentialBackoff retryStrategy = new ExponentialBackoff(5, Duration.ofSeconds(1), Duration.ofSeconds(16));
RetryPolicy retryPolicy = new RetryPolicy(retryStrategy);

HttpPipeline pipeline = new HttpPipelineBuilder()
    .policies(retryPolicy)
    .build();

TextAnalyticsClient client = new TextAnalyticsClientBuilder()
    .pipeline(pipeline)
    .endpoint(endpoint)
    .credential(new AzureKeyCredential(apiKey))
    .buildClient();

Here, the ExponentialBackoff is configured with a maximum of 5 retries, a base delay of 1 second, and a maximum delay of 16 seconds.

In Python, you can customize the retry policy using the RetryPolicy class from the Azure SDK.

from azure.core.pipeline.policies import RetryPolicy
from azure.ai.textanalytics import TextAnalyticsClient
from azure.core.credentials import AzureKeyCredential

retry_policy = RetryPolicy(
    retry_total=5,
    retry_backoff_factor=0.8,
    retry_backoff_max=16,
    retry_mode='exponential'
)

client = TextAnalyticsClient(
    endpoint=endpoint,
    credential=AzureKeyCredential(api_key),
    retry_policy=retry_policy
)

In this configuration, the client will retry up to 5 times, with an exponential backoff starting at 0.8 seconds and capping at 16 seconds.

Some links to help you :

https://stackoverflow.com/questions/76251868/how-to-log-retry-counts-when-using-azure-sdk-retry-policy

https://learn.microsoft.com/en-us/azure/storage/blobs/storage-retry-policy