Azure OpenAI text-embedding-ada-002 deployment returning 503 status codes

Question

Azure OpenAI text-embedding-ada-002 deployment returning 503 status codes

Robin Hes 0

I have been using a text-embedding-ada-002 deployment on my Azure OpenAI endpoint to embed large amounts of texts. This worked perfectly until some time last week, when the service started to at sometimes return 503 status codes with message "The service is temporarily unable to process your request. Please try again later.".

I'm staying within request and token limits and my request data has been batched according to the specified context length of 8191 tokens per input and max number of inputs of 2048. I have tried the "West Europe" and "UK South" regions.

Did something change in the limits, or is there an issue with the service?

Ravada Shivaprasad 40 Reputation points Microsoft External Staff

2025-05-09T21:41:04.1+00:00

Hi Robin Hes

Your implementation of Azure OpenAI text embedding aligns with best practices, ensuring efficient processing and compliance with service constraints. Given the consistent 503 errors across multiple regions such as West Europe and UK South, it appears to be a broader service-level issue rather than isolated regional capacity constraints. To mitigate transient failures, implementing a robust retry mechanism with exponential backoff can help maintain request stability. Additionally, setting up detailed monitoring—including tracking success rates, logging response times, and analyzing batch processing patterns—provides valuable insights for troubleshooting.
Alternative approaches such as distributing requests across multiple deployments, leveraging a circuit breaker pattern, and caching embeddings where feasible may further optimize performance. It's important to note that while your batching strategy adheres to documented limits, Azure may implement additional safeguards affecting request throughput. Since the issue spans multiple regions, Microsoft is likely addressing a broader mitigation strategy. For the most up-to-date documentation and best practices, you can refer to Microsoft Learn.

Thanks

Your answer

Ravada Shivaprasad 40 Reputation points Microsoft External Staff

2025-05-09T21:41:04.1+00:00

Hi Robin Hes

Your implementation of Azure OpenAI text embedding aligns with best practices, ensuring efficient processing and compliance with service constraints. Given the consistent 503 errors across multiple regions such as West Europe and UK South, it appears to be a broader service-level issue rather than isolated regional capacity constraints. To mitigate transient failures, implementing a robust retry mechanism with exponential backoff can help maintain request stability. Additionally, setting up detailed monitoring—including tracking success rates, logging response times, and analyzing batch processing patterns—provides valuable insights for troubleshooting.
Alternative approaches such as distributing requests across multiple deployments, leveraging a circuit breaker pattern, and caching embeddings where feasible may further optimize performance. It's important to note that while your batching strategy adheres to documented limits, Azure may implement additional safeguards affecting request throughput. Since the issue spans multiple regions, Microsoft is likely addressing a broader mitigation strategy. For the most up-to-date documentation and best practices, you can refer to Microsoft Learn.

Thanks

Share via

Azure OpenAI text-embedding-ada-002 deployment returning 503 status codes

Your answer